Over the years I’ve read many blog posts from programmers giving advice on how to integrate (or make peace with) AI in software engineering. And, more often than not, the advice is written with the implicit assumption that the technology will stay where it was at the time of writing. So if you lay them out on a timeline, it might be:
- “Tab completion is useful, but you need a human to have repository-wide context.”
- “Coding agents are useful, but produce a lot of slop, you need to review and aggressively refactor their code.”
- “Coding agents are very useful, but you still need to review the code.”
- “Code was never the hard part. Actually the hard part is the deep tacit knowledge of talking to the customer.” (You are here)
- “It is professional malpractice to write code by hand.”
- “We have a cron job where every morning Claude Requiem one-shots the spec into x86 assembly.”
It sounds like the propaganda of a state that is losing a war: the tales of heroic victory get closer and closer to the capital each time.
The failure mode is that they’re not taking AI seriously. Analogously, when a corporate CEO like Mark Zuckerberg goes on a podcast to talk about how superintelligence is near and this will be great for jobs and the economy, the commentariat replies: he’s not ASI-pilled, he’s not serious, he doesn’t really believe in ASI. And they are right: “the intelligence explosion is next Tuesday, and the world will stay the same” is hard to believe.
So, what does it mean to take AI seriously? People on Twitter talk about being “AGI-pilled”, meaning: “AGI is coming soon and it will be transformative”. Being AGI-pilled means you’re taking AI seriously.
But I don’t like the term because it conflates “timelines to AGI are short” and “consequences of AGI will be transformative”. The latter is easy to believe, the former has an unsatisfying evidentiary basis: people draw lines on plots, extrapolate, and go on podcasts to talk about it. So criticizing someone for not being AGI-pilled (in the short timelines sense) is just complaining that they don’t automatically accept the straight lines on graphs memeplex.
A better definition, I think, is this:
- Taking AI seriously means: consider it possible that AI progress will continue at the pace of the past few years. It’s a conjecture, subject to refutation.
- Not taking AI seriously means confidently asserting that AI progress will halt, or that AI’s impact on society will be tiny. If the evidence for continued progress is weak, the evidence that progress will slow or halt is even weaker. And AI’s impact is already massive: software, mathematics, and cybersecurity are the obvious cases, but AI writing and AI images are everywhere.
There’s a great story by Borges: Tlön, Uqbar, Orbis Tertius. A conspiracy writes an encyclopedia of a fictional world, and through the power of Berkeleyan idealism, this other world begins to invade ours. Objects from Tlön begin to appear, Earth languages are displaced by Tlönian languages. The narrator concludes in resignation: “the world will be Tlön”.
A modern version of the story would have people arguing about being “Tlön-pilled”, or whether the government is “taking Tlön seriously”. Economists would debate whether the hrönir excavation industry will be 1.5% or 2% of GDP by 2030, while their children begin speaking the nounless language of Tlön. So, “taking Tlön seriously” means: “consider it possible that the world will be Tlön”.