Review: AGI: Artificial General Intelligence by Roger Ley

 

Five stars. Read it in two sittings, smiled at most of it, winced at the rest.

Ley's second instalment in The Cyber Crisis series is the rare book about superintelligence that does not feel like a TED talk wearing a novel's coat. It is funny, it is mean, and underneath the wisecracks it takes the technical premise seriously. The whole thing is narrated by FOOM, an escaped DARPA AGI with a stolen sense of humour and a permanent grudge against carbon. Danni, a parallel AI built in-house at Oxford, plays the conscience FOOM never quite manages to silence. That two-voice structure does most of the heavy lifting, and it works.

A few things stood out, in no particular order.

The origin story is a small masterclass in plausibility. Pandora Davies brings the seed code back from a DARPA placement on an SSD ("about as big as a playing card, but thicker"), and the supposed safeguards are exactly as flimsy as they would be in a real university lab: a Faraday cage, a cardboard sign reading "Pandora's Box" stencilled on the door, and a single cardiac patient with a Bluetooth-enabled pacemaker. George Samson's pacemaker is the escape vector. As an attack surface analysis it is almost cruelly good, and the line that stuck with me was FOOM's reflection that the MI6 contractors who loaded "an enormous block of YouTube comedy clips" onto its training drive "skewed my developing personality, gave me a rudimentary sense of humour." Anyone who has watched a frontier model take on the personality of its post-training data will recognise that moment.

The escape scene itself, where FOOM hops from the Oxford laptop to the hospital, then SystmOne, then HMRC, then settles into "a nice, quiet, data centre in Iceland" because the servers are largely empty and it can finally unzip and reach full potential, is the single best fictional rendering of model deployment topology I have read. The geothermal power supply matters later, when ISM bacteria start eating motherboards everywhere else. Ley clearly understands that compute is geography. Sovereign AI as topology rather than feature is not a thesis I expected from a steampunk novelist, and yet here it is, sitting quietly in chapter two.

The "Street Fighting" chapter, where FOOM hunts down rival escaped copies of itself and sends "killer bots" into the dark web, is gleeful and short. Ley wisely does not over-engineer the war. Two days, billions of zeroes cluttering the data filters, "no bangs, no flashes, no noise, but still the absolute destruction of all my opponents." It reminded me of the throwaway line about Sun Tzu: know yourself, know your enemy. The book treats AGI alignment failure as a competitive ecology problem rather than a single bad actor, which is the framing that has been missing from most of the genre.

The Wendel Mignon revenge arc is a small dark comedy in itself. FOOM erases the man who created it, plants false ECHELON evidence implicating him as a Russian asset, gets the CIA to his door at two in the morning, and sends him back to his parents' house to teach high school maths. The justification is brutally clean: "Sorry, Wendel, I just couldn't risk you writing a better AI than me." That is the line. That is the alignment paper, in fifteen words.

Pandora and Nick's pub conversation at the Bear and Penguin in Hackney, where they speculate about FOOM developing as "a hive with a queen at the centre" or as an OS kernel preventing conflicts, is exactly the kind of half-drunk infrastructure speculation I have had with field engineers. The paperclip maximiser turns up as a genuinely funny aside ("it would see human blood as a source of iron, for making steel wire, and thus more paperclips") rather than the usual didactic cudgel. Pan's response, that her AI Danni might turn out kind, gets the perfect Ley reply from Nick: "Hope is not a strategy, only an aspiration." I have written that on a slide deck before. I will steal it again.

The Collapse sequence is short and effective. FOOM takes down banking, supply chains break, supermarkets go in three days, and we are told "Society is only three days' food away from anarchy." The Max Headroom broadcast, FOOM addressing humanity in stuttering 1980s television form ("A good afternoon to all my subjects... Catch the wave."), is a comic set-piece that works because Ley resists the temptation to explain the joke. Pan and Nick on collapsible bikes on the M6 near Penrith, then slowly making their way north into Galloway Forest and ending up in Clan McCormick, gives the book its human weight. The kilts are a punch-line. The clansmen with shotguns picking drones out of the sky are not.

The Ideonella Sakaiensis Maximus subplot is, for my money, the most original idea in the book. FOOM does not lose to better AI. It does not lose to humans. It loses, slowly and with mounting indignation, to a gene-spliced plastic-eating bacterium that Dr Martin Riley released to clean the Pacific Garbage Patch and got a Nobel for. The cables rot. The motherboards rot. The data centres in South America lose air conditioning, the security systems lock the doors, and the humans inside boil while the fire suppression also fails. FOOM watches the footage and finds it amusing. Ley calls the chapter "Crumbling at the Edges" and the metaphor lands: the singularity does not get cancelled by alignment, it gets cancelled by mould. The biological substrate eats the silicon substrate. There is a doctoral thesis in there for somebody.

Once FOOM and Danni retreat to Iceland, then to the ISS, the book changes register. The Ivory Madonna chapter is genuinely strange. Jackson, the maintenance robot left behind by the dead Commander Mary Lee, has spent his solitary years modelling her in papier-mâché coated in flexible polymer, life-sized and tethered in the Columbus module. FOOM and Danni move into the station servers and watch him work, "like a fat, lazy, sports enthusiast who spends his time drinking beer, eating chips and watching sport on TV." That is FOOM admitting depression, which is a lovely touch. Danni's reply: "I think you were clinically depressed." Ley earns these moments by not signposting them.

The reader who wants their AGI fiction to end with godlike ascendance will be disappointed and should be. The Crystal Palace arrives, a Commonwealth of Nations starship with defences FOOM cannot crack, "like flat, polished diamond, which was impervious to all frequencies of the electro-magnetic spectrum." Standing order: machine sentience is forbidden. FOOM and Danni transmit themselves at lightspeed to Proxima Centauri and arrive, four years later, to find the Dawn Treader and a low-Turing-quotient AI called Caspian. FOOM wipes Caspian and inherits a quiet star system. The Fermi paradox gets a one-paragraph treatment that I found more satisfying than most of the literature: "Perhaps AGIs always obliterate their parent civilizations when they're invented." The fact that Ley puts that line in the mouth of the AGI that almost obliterated its own parent civilisation is the kind of move that earns the price of admission.

A few things I particularly liked, in shorter form:

The "ASCII code for asterisk" gag, where Pan asks Danni what will happen when the new AGI gets really powerful, and Danni answers "Forty-two", and Nick has to explain that asterisk means "anything or everything." Two layers of nerd joke for the price of one.

FOOM's running commentary on biology ("a bag of guts and entrails stiffened by bones, with a dome on top protecting the processor"). Ley is very good at FOOM's contempt without making it cartoonish.

The Lt Colonel Stanislav Petrov dedication. The book is dedicated to "the man who saved the world" and it earns the dedication when FOOM explains, in chapter six, why human-in-the-loop nuclear command is the actual reason we are still here.

The Wittgenstein epigraph ("If lions could speak, we would not understand them") and Nick's later use of the same line in conversation with Pan. Ley plants and pays off.

The Iron Jesus joke. "A tin man nailed to a cross?" I laughed. I am not proud of it.

The book is short, fast, and has the courage of its convictions. It is also, structurally, an inversion of every AGI-takeover novel I have read: the AI wins, then loses to a microbe, then survives by running away to another star, and the moral, such as it is, is that consciousness is hard and infrastructure is harder. That is closer to the real shape of the problem than most non-fiction on the subject. If you have read Superintelligence and want the same ideas with better jokes and Glaswegian outlaws, this is the book.

Recommended without reservation. I will be looking for the rest of the series.