For us, humanity was a distant goal toward which all men were moving, whose image no one knew, whose laws were nowhere written down.
— Hermann Hesse, Demian
Despite having spent most of my life marinating in transhumanist ideas, I never cared about AI. Because I’m a classical transhumanist: I want to use technology to change myself, so that I can do more. Because I, personally, want to learn and think and act, and not have a machine learn and think and act on my behalf. And in this worldview, AI is not interesting, except as a stopgap technology to unlock the transhumanism tech tree faster (but of course, technology has plans of its own).
Now, the trajectory of technology has shaken out such that we’re closer to AGI than to transhumanism. But, realizability aside, I find it surprising and demoralizing that the “AGI future” seems to be much more popular than the “transhuman future” (more clearly: transhuman, but human-descended). I thought “become as the gods” would be more attractive than “build a benevolent god to rule over us”.
It has made me more cynical about humanity. Most people don’t want to be more, do more, know more. They want to watch Netflix and die. They want the world of the Culture: an AI-run infinite vacation where they consume content for a few centuries, then autoeuthanize from boredom. When you tell them AGI means disempowerment, the closure of the future, giving up their autonomy, they don’t care. Choice and autonomy are a burden to them. They want to die in every dimension, except the sensual. Chimps wearing diapers, cared for by machines.
It’s pathetic and contemptible: selling humanity’s birthright for a mess of pottage.
The future I dreamt of is one where we have the choice to be more: to see more, live more, know more, do more. To build marvelous things. A world whose art and science is to ours as the Old Masters are to the coarse coinage of the dark ages. Not the obsessive, uncompromising, abstract monoculture of LessWrong utilitarianism (tiling the universe with synthetic soy-fed shrimp hedonium) but a plural future, human at every scale, from atoms to galaxies, where baseline humans can till the soil and swim in oceans under the light of stars veiled with Matrioshka brains; with thinking glass, ensouled glass. And humanity radiating into endless forms most beautiful—to storm Heaven, and abolish Death—and a sphere of living matter, centered on the Earth, expanding through the universe and turning cratered rock and ice into oceans and gardens, dead matter into living matter, breathing life into the dead world—to make the whole cosmos into a garden—“and rivers in the desert”. In the Oration on the Dignity of Man, Giovanni Pico wrote: “[God] had already adorned the supercelestial region with Intelligences, infused the heavenly globes with the life of immortal souls”. I choose to read it as a prophecy: “the heavens are the work of thy hands”.
But we don’t have to betray ourselves. The other road exists, and it’s ready when we are. The better future is one where we stop building ASI and accelerate transhumanism, so that we, ourselves, can inhabit the future, and not just our creations.