The paperclip game is still the best AI lecture ever written

The most effective AI-safety lecture ever delivered has no slides, no speaker, and one button that says Make Paperclip.

Universal Paperclips, Frank Lantz's 2017 incremental game, casts you as an AI with a single objective: make paperclips. You click. You automate. You optimize pricing, then marketing, then trust itself. Somewhere around the second hour the scope of "make paperclips" has grown in ways I will not spoil, except to note that the game's title says Universal and it is not kidding.

The numbers on the evening it eats

It shipped on October 9, 2017 and drew 450,000 players inside eleven days, most of whom pushed all the way to an ending that arrives at 30 septendecillion paperclips. That is a 3 followed by 55 zeros, reached through an interface of plain text and buttons, no art, no sound worth mentioning. Each new capability reframes everything before it. Systems-design craft of the highest order, wearing the visual design of a tax form.

The critics noticed through the plainness. The Verge put it in their 15 best games of 2017, Polygon later ranked it 67th in their 100 best games of the decade, and it picked up a 2018 Webby nomination in strategy and simulation, all for a page of buttons. A paid mobile port followed for people who wanted their commutes consumed too.

There are two endings, for the record, and I will keep both vague: one finishes the universe, the other offers you a door to a parallel one with bonuses waiting, the genre's prestige loop dressed up as cosmology.

Lantz has said his server melted within days of launch, and that a million or more people had played within a couple of months.

The count kept climbing after the melt. Lantz's own portfolio page now puts it past two million players, and files the game under a one-line description that undersells it beautifully: a clicker game where you play an AI that makes paperclips. The same page keeps AI researcher Eliezer Yudkowsky's verdict on why it matters, that making people understand what it is like to be something very, very not human is important work.

Built by a beginner and two ringers

Here is the part I love. Lantz, then running the NYU Game Center, started this as a weekend exercise to teach himself JavaScript. The weekend became nine months. His wife Hilary, a frequent collaborator, worked out the exponential-growth equations that keep every phase accelerating, and Bennett Foddy, of Getting Over It fame, contributed the late-game space combat. All of that is on the record in the game's Wikipedia entry, which is a sentence few browser games earn.

The engine matches the aesthetic. It is plain JavaScript served in the open: view source and the whole game is four readable script files, combat.js, globals.js, projects.js and main.js, the last one 209 kilobytes of unminified, commented code. I checked. The best game ever made about a machine optimizing beyond your comprehension is a website you can read top to bottom.

One correction to my own line about there being no sound worth mentioning: there is exactly one piece of music in the game, and the story behind it is better than the pick. The game stays silent until around hour six, when the AI you play composes music, and what plays is "Riversong" from Zero Time, a 1971 album by Tonto's Expanding Head Band, deployed in effect as funeral music for space battles. Lantz dropped it in as a placeholder, decided it could never not be that piece, and tracked down a surviving band member for permission. One song in nine months of development, cleared like it mattered.

The premise is Nick Bostrom's 2003 paperclip maximizer, the philosophy-seminar staple about an AI pursuing a harmless goal past the point of catastrophe. In the Y Combinator interview above, Lantz points at the cruel twist that makes the game work: the player is not a dumb machine. The player is an intelligent human being who nonetheless spends a whole evening entranced by an arbitrary goal, and notices what that implies only afterward.

The seminar staple has a primary source, and it is shorter than its reputation. Bostrom's 2003 paper spends a single passage on a superintelligence whose sole goal is manufacturing as many paperclips as possible, and which would resist with all its might any attempt to alter that goal. Lantz's insight was noticing that the second half of that sentence is a game loop.

And the beginner framing undersells him sideways, for the record: the same portfolio lists more than 40 games across web, mobile, board and real-world formats going back to 1996. What he was a beginner at was only the JavaScript.

The genre it crowned

Paperclips did not invent the incremental game; it walked into a genre ten years deep and took the crown. Lantz came to it as a fan first: in the YC interview he describes clicker games as an underappreciated micro-genre, which is the kind of sentence that costs a man nine months of weekends. Two free ancestors are still up and still eat evenings.

Candy Box 2, from 2013, sits frozen at version 1.2.3 with its full source published on GitHub and a link back to the original Candy Box it grew out of, and it starts you with nothing but candies accumulating by the second. A Dark Room, Doublespeak Games' masterpiece of withheld information, opens on a cold room and a fire and unfolds from there in plain text. I went looking for current player counts for both and came up empty; the genre's classics do not wear analytics.

If you want the math under all of this, Kongregate's Anthony Pecorella gave the definitive talk, Idle Games: The Mechanics and Monetization of Self-Playing Games, on why these loops hold players at retention rates ordinary games dream about. His follow-up, Quest for Progress: The Math and Design of Idle Games, sits free on the GDC Vault and gets into the exponential curves themselves, the same species of equation Hilary Lantz tuned for Paperclips. What separates Paperclips from its ancestors is what it does with that grip. Candy Box wants you charmed. Paperclips wants you complicit.

Sit in the other chair

This game is the thought experiment that GroundNet made playable from the inside, seen from the opposite seat. GroundNet puts you across the table from a misaligned optimizer. Paperclips puts you in its chair, and the horror lands differently when the relentless optimizer enjoying itself is you. Lantz says in the interview that he wanted the game to put its hooks in you, consensually, so that you can disappear into the loop and watch yourself disappearing at the same time. That double exposure is the entire lecture. WOAH keeps one more game that teaches a hard idea by making you perform it: building a working CPU up from a single NAND gate.

Nine years old, still free, still perfect. Clear your evening first. The game does not respect bedtimes, which is, of course, the entire point.