AI/Biz

AlphaGo Isn't About Go

Last week a computer program beat Lee Sedol, one of the strongest Go players alive, four games to one. The coverage has been enormous and mostly wrong about why it matters.

Let me try to be useful instead of excited.

What DeepMind Built

The paper came out in Nature at the end of January, before the match, and it describes three components working together.

  • A policy network. Shown a board position, it suggests plausible moves. It was trained first on 30 million positions from recorded games between strong human players, learning to predict what a good player would do. Then it was improved by playing against older copies of itself, millions of times, keeping what won.
  • A value network. Shown a board position, it estimates who is winning. This is the harder problem and the genuinely novel contribution. Evaluating a Go position was long considered close to impossible to do well.
  • Monte Carlo tree search. The classical part. Explore possible continuations, but use the two networks to decide which branches deserve attention and when to stop looking.

Neither network alone plays well enough to beat a professional. Neither does the search alone. The combination does.

“The learned part narrows the search and the search checks the learned part’s work. Neither is trustworthy alone. That pairing is the actual result here, not the final score.” — Sameer Gupta

Move 37 in game two has been discussed to death and deserves it. The program played a shoulder hit on the fifth line that commentators initially called a mistake, and that turned out to be the move that decided the game. It wasn’t a move copied from human play. Self-play had taken it somewhere the human corpus didn’t go.

Why It Won’t Do What Your Board Thinks

Here’s where I’d push back on the coverage, because the leap from “beat a Go champion” to “will transform enterprise decision-making” is being made far too casually.

Go has four properties that made this possible, and your business problem has approximately none of them.

  • Perfect information. Every relevant fact is on the board, visible to both players. Nothing is hidden, delayed, mis-recorded, or entered by a distracted person in a different time zone.
  • Fixed, known rules. The rules of Go have not changed and will not change. Your regulatory environment, your competitors, and your customers’ preferences all change, sometimes because of what you did.
  • An unambiguous win condition. At the end of a game of Go, there’s a score. There’s no argument. Ask five executives to define success for a project and you’ll get seven answers, and the definition will have shifted by the time you deliver.
  • Unlimited free training data. This is the big one. AlphaGo generated its own experience by playing itself millions of times at no cost and no risk. Your supply chain cannot play itself thirty million times. Every experiment you run costs real money and annoys real customers.

“Self-play is the whole trick, and it’s only available when the simulator is free, fast, and perfectly accurate. That describes board games and almost nothing else.” — Sameer Gupta

What Actually Does Transfer

I don’t want to be entirely negative, because there is a real and usable lesson underneath the hype.

The pattern of learned evaluation guiding a classical search is broadly applicable, and it’s underused. A great many business problems are search problems where the space is too large to enumerate and the traditional approach is a hand-written heuristic.

  • Scheduling and routing. Delivery sequences, factory job ordering, crew rostering. You have a search space and an expensive evaluation. Learning a cheap approximate evaluation to guide the search is exactly the AlphaGo shape.
  • Simulation-heavy planning. Anywhere you already run Monte Carlo simulations, in demand planning, in risk, in capacity modelling, a learned model that predicts which scenarios are worth simulating can pay for itself immediately.
  • Configuration and design search. Pricing structures, product bundles, network layouts. Large combinatorial spaces where a person currently picks a handful of candidates by intuition.

Notice what these have in common: you have or can build a decent simulator. That’s the qualifying question. If you can simulate it cheaply, some version of this applies. If you can’t, this paper is entertainment.

The Cost Nobody Mentions

The distributed version of AlphaGo that played Lee Sedol ran on roughly 1,900 CPUs and 280 GPUs. The training took months and used a corpus of human games that took decades of human play to generate.

DeepMind cost Google around 400 million pounds and this is what a substantial fraction of that team has been working on. It is a magnificent research result and it is not a template for a mid-sized company’s analytics budget.

Final Thoughts

I think this is the most impressive engineering achievement in the field so far, and I think almost every business conclusion being drawn from it this week is wrong.

The right conclusion is narrow and useful: when you can cheaply simulate outcomes, a learned evaluator plus a classical search will beat a hand-written heuristic, and it will find moves your experts wouldn’t have considered.

The wrong conclusion is that machines now make better decisions than people. What happened is that a machine made better decisions than people in a domain with perfect information, fixed rules, a clear score, and free unlimited practice.

Before you get excited, ask which of those four your problem actually has. In my experience the answer is usually zero, occasionally one, and the projects that succeed are the ones where somebody asked the question first.