AlphaGo and AlphaZero

1. Purpose

• AlphaGo: Designed specifically to play the board game Go. It made history in 2016 by defeating world champion Lee Sedol in a series of matches, demonstrating AI's ability to tackle complex strategy games.
• AlphaZero: A general-purpose reinforcement learning system that can master multiple games like Go, Chess, and Shogi from scratch without prior human knowledge or input beyond the rules of the game.

2. Training Approach

• AlphaGo:
  • Used a combination of supervised learning (training on human professional games) and reinforcement learning (improving by self-play).
  • Heavily relied on human expert data as a starting point.
• AlphaZero:
  • Relied purely on reinforcement learning, learning by playing games against itself.
  • No input from human games, making it more generalized and autonomous.

3. Architecture

• AlphaGo:
  • Featured a more complex architecture combining:
    • Policy networks (to suggest moves).
    • Value networks (to evaluate positions).
    • Monte Carlo Tree Search (MCTS) to plan and evaluate game states.
• AlphaZero:
  • Simplified the architecture into a single deep neural network that predicts both policy (best moves) and value (winning probability).
  • Still used Monte Carlo Tree Search but with a more efficient integration of the neural network.

4. Performance

• AlphaGo:
  • Had variants (e.g., AlphaGo Fan, AlphaGo Master), each fine-tuned to improve performance.
  • Demonstrated superhuman ability in Go but was game-specific.
• AlphaZero:
  • Outperformed AlphaGo in Go after a short training period.
  • Became a state-of-the-art player in Chess and Shogi, defeating top AI programs like Stockfish and Elmo.
  • Demonstrated superior adaptability to new games.

5. Generalization

• AlphaGo: Specialized for Go and unable to generalize to other games without extensive reengineering.
• AlphaZero: Generalized framework capable of mastering a wide range of games with little modification, showcasing the potential of reinforcement learning for broader applications.

Legacy

• AlphaGo: Pioneered the AI revolution in games and inspired advances in deep learning and reinforcement learning.
• AlphaZero: Took AlphaGo’s success to the next level, showing that AI could be generalized across multiple domains, thus influencing research in AI for other real-world applications like protein folding (AlphaFold).

AlphaZero is considered the more advanced of the two due to its simplicity, generality, and greater performance across different domains.