“Letter from Utopia” and Other Triple-S Transhumanist Media

by Nick Bostrom (2010)

Dear Human,

Greetings, and may this letter find you at peace and in prosperity! Forgive
my writing to you out of the blue. Though you and I have never met, we are
not strangers. We are, in a certain sense, the closest of kin. I am one of your
possible futures.

I hope you will become me. Should fortune grant this wish, then I am not
just a possible future of yours, but your actual future: a coming phase of you,
like the full moon that follows a waxing crescent, or like the flower that
follows a seed.

I am writing to tell you about my life – how good it is – that you may choose
it for yourself.

Although this letter uses the singular, I am really writing on behalf of all
my contemporaries, and we are addressing ourselves to all of your
contemporaries. Amongst us are many who are possible futures of your
people. Some of us are possible futures of children you have not yet given
birth to. Still others are possible artificial persons that you might one day
create. What unites us is that we are all dependent on you to make us real.
You can think of this note as if it were an invitation to a ball that will take
place only if folks turn up.

We call the lives we lead here “Utopia”.

*

How can I tell you about Utopia and not leave you mystified? What words
could convey the wonder? What inflections express our happiness? What
points overcome your skepticism? My pen, I fear, is as unequal to the task as
if I had tried to use it against a charging elephant.

But the matter is so important that we must try even against long odds.
Maybe you will see through the inadequacies of my exposition.

Have you ever known a moment of bliss? On the rapids of inspiration,
maybe, where your hands were guided by a greater force to trace the shapes
of truth and beauty? Or perhaps you found such a moment in the ecstasy of
love? Or in a glorious success achieved with good friends? Or in splendid
conversation on a vine-overhung terrace one star-appointed night? Or
perhaps there was a song or a melody that smuggled itself into your heart,
setting it alight with kaleidoscopic emotion? Or during worship?

If you have experienced such a moment, experienced the best type of such a
moment, then a certain idle but sincere thought may have presented itself to
you: “Oh Heaven! I didn’t realize it could feel like this. This is on a
different level, so very much more real and worthwhile. Why can’t it be like
this always? Why must good times end? I was sleeping; now I am awake.”

Yet behold, only a little later, scarcely an hour gone by, and the softly-falling
soot of ordinary life is already piling up. The silver and gold of exuberance
lose their shine. The marble becomes dirty.

Every way you turn it’s the same: soot, casting its veil over all glamours and
revelries, despoiling your epiphany, sodding up your white pressed collar and
shirt. And once again that familiar beat is audible, the beat of numbing
routine rolling along its tracks. The commuter trains loading and unloading
their passengers… sleepwalkers, shoppers, solicitors, the ambitious and the
hopeless, the contented and the wretched… like human electrons shuffling
through the circuitry of civilization.

We do so easily forget how good life can be at its best – and how bad at its
worst. The most outstanding occasion: barely is it over before the sweepers
move in to clean up the rice. Yellowing photos remain.

And this is as it should be. We are in the business of living, and life must go
on. Special moments are out-of-equilibrium experiences in which our
puddles are stirred up and splashed about; yet when normalcy returns we are
usually relieved. We are built for mundane functionality, not lasting bliss.

So the door that was ajar begins to close, disappearing hope’s sliver behind
an insensate scab.

Quick, stop that door! Look again at your yellowing photos, search for a
clue. Do you not see it? Do you not feel it, the touch of the possible? You
have witnessed the potential for a higher life, and you hold the fading proof
in your hands. Don’t throw it away. In the attic of your mind, reserve a
drawer for the notion of a higher state of being. In the furnace of your heart,
keep an aspiring ember alive.

I am summoning this memory of your best experience – to what end? In the
hope of kindling in you a desire to share my happiness.

And yet, what you had in your best moment is not close to what I have now
– a beckoning scintilla at most. If the distance between base and apex for
you is eight kilometers, then to reach my dwellings requires a million lightyear ascent. The altitude is outside moon and planets and all the stars your
eyes can see. Beyond dreams. Beyond imagination.

My consciousness is wide and deep, my life long. I have read all your
authors – and much more. I have experienced life in many forms and from
many angles: jungle and desert, gutter and palace, heath and suburban creek
and city back alley. I have sailed the high seas of cultures, and swum, and
dived. Quite some marvelous edifice builds up over a million years by the
efforts of homunculi, just as the humble polyps amass a reef in time. And
I’ve seen the shoals of colored biography fishes, each one a life story,
scintillate under heaving ocean waters.

The whole exceeds the sum of its parts. What I have is not merely more of
what is available to you now. It isn’t just the particular things, the paintings
and toothpaste-tube designs, the record covers and books, the epochs, lives,
leaves, rivers, and random encounters, the satellite images and the hadron
collider data – it is also the complex relationships between these particulars
that make up my mind. There are ideas that can be formed only on top of
such a wide experience base. There are depths that can be fathomed only
with such ideas.

You could say I am happy, that I feel good. You could say that I feel
surpassing bliss. But these are words invented to describe human
experience. What I feel is as far beyond human feeling as my thoughts are
beyond human thought. I wish I could show you what I have in mind. If
only I could share one second of my conscious life with you!

But you don’t have to understand what I think and feel. If only you bear in
mind what is possible within the present human realm, you will have enough
to get started in the right direction, one step at a time. At no point will you
encounter a wall of blinding light. At no point will you have to jettison
yourself over an end-of-the-world precipice. As you advance, the horizon
will recede. The transformation is profound, but it can be as gradual as the
growth that made the baby you were into the adult you think you are.

You will not achieve this through any magic trick or hokum, nor by the
power of wishful thinking, nor by semantic acrobatics, meditation,
affirmation, or incantation. And I do not presume to advise you on matters
theological. I urge on you nothing more, nothing less, than reconfigured
physical situation.

*

The challenge before you: to become fully what you are now only in hope
and potential. New capacities are needed if you wish to live and play on my
level.

To reach Utopia, you must first discover the means to three fundamental
transformations.

The First Transformation: Secure life!

Your body is a deathtrap. This vital machine and mortal vehicle, unless it
jams first or crashes, is sure to rust anon. You are lucky to get seven decades
of mobility; eight if you be Fortuna’s darling. That is not sufficient to get
started in a serious way, much less to complete the journey. Maturity of the
soul takes longer. Why, even a tree-life takes longer!

Death is not one but a multitude of assassins. Do you not see them? They
are coming at you from every angle. Take aim at the causes of early death –
infection, violence, malnutrition, heart attack, cancer. Turn your biggest
gun on aging, and fire. You must seize control of the biochemical processes
in your body in order to vanquish, by and by, illness and senescence. In
time, you will discover ways to move your mind to more durable media.
Then continue to improve the system, so that the risk of death and disease
continues to decline. Any death prior to the heat death of the universe is
premature if your life is good.

Oh, it is not well to live in a self-combusting paper hut! Keep the flames at
bay and be prepared with liquid nitrogen, while you construct yourself a
better habitation. One day you or your children should have a secure home.
Research, build, redouble your effort!

The Second Transformation: Upgrade cognition!

Your brain’s special faculties: music, humor, spirituality, mathematics,
eroticism, art, nurturing, narration, gossip! These are fine spirits to pour
into the cup of life. Blessed you are if you have a vintage bottle of any of
these. Better yet, a cask! Better yet, a vineyard!

Be not afraid to grow. The mind’s cellars have no ceilings!

What other capacities are possible? Imagine a world with all the music dried
up: what poverty, what loss. Give your thanks, not to the lyre, but to your
ears for the music. And ask yourself, what other harmonies are there in the
air, that you lack the ears to hear? What vaults of value are you witlessly
debarred from, lacking the key sensibility?

Had you but an inkling, your nails would be clawing at the padlock in sacred
frenzy.

Your brain must grow beyond the bounds of any genius of humankind, in its
special faculties as well as its general intelligence, so that you may better
learn, remember, and understand, and so that you may apprehend your own
beatitude.

Mind is a means: for without insight you will get bogged down or lose your
way, and your journey will fail.

Mind is also an end: for it is in the spacetime of awareness that Utopia will
exist. May the measure of your mind be vast and expanding.

Oh, stupidity is a loathsome corral! Gnaw and tug at the posts, and you will
slowly loosen them up. One day you’ll break the fence that held your
forebears captive. Gnaw and tug, redouble your effort!

The Third Transformation: Elevate well-being!

What is the difference between indifference and interest, boredom and thrill,
despair and bliss?

Pleasure! A few grains of this magic ingredient are dearer than a king’s
treasure, and we have it aplenty here in Utopia. It pervades into everything
we do and everything we experience. We sprinkle it in our tea.

The universe is cold. Fun is the fire that melts the blocks of hardship and
creates a bubbling celebration of life.

It is the birth right of every creature, a right no less sacred for having been
trampled upon since the beginning of time.

There is a beauty and joy here that you cannot fathom. It feels so good that
if the sensation were translated into tears of gratitude, rivers would overflow.

I reach in vain for words to convey to you what it all amounts to… It’s like a
rain of the most wonderful feeling, where every raindrop has its own unique
and indescribable meaning – or rather a scent or essence that evokes a whole
world… And each such evoked world is subtler, richer, deeper, more
palpable than the totality of what you have experienced in your entire life.

I will not speak of the worst pain and misery that is to be got rid of; it is too
horrible to dwell upon, and you are already aware of the urgency of
palliation. My point is that in addition to the removal of the negative, there
is also an upside imperative: to enable the full flourishing of enjoyments that
are currently out of reach.

The roots of suffering are planted deep in your brain. Weeding them out
and replacing them with nutritious crops of well-being will require advanced
skills and instruments for the cultivation of your neuronal soil. But take
heed, the problem is multiplex! All emotions have a natural function. Prune
carefully lest you reduce the fertility of your plot.

Sustainable yields are possible. Yet fools will build fools’ paradises. I
recommend you go easy on your paradise-engineering until you have the
wisdom to do it right.

Oh, what a gruesome knot suffering is! Pull and tug on those loops, and you
will gradually loosen them up. One day the coils will fall, and you will
stretch out in delight. Pull and tug, and be patient in your effort!

May there come a time when rising suns are greeted with joy by all the living
creatures they shine upon.

*

How do you find this place? How long will it take to get here?

I can pass you no blueprint for Utopia, no timetable, no roadmap. All I can
give you is my assurance that there is something here, the potential for a
better life.

If you could visit me here for but a day, you would henceforth call this place
your home. This is the place where you belong. Ever since one hairy
creature picked up two flints and began knocking them together to make a
tool, this has been the direction of your unknown aspiration. Like Odysseus
you must journey, and never cease journeying, until you arrive upon this
shore.

“Arrive?” you say; “But isn’t the journey the destination? Isn’t Utopia a
place that doesn’t exist? And isn’t the quest for Utopia, as witnessed
historically, a dangerous folly and an incitement to mischief?”

Friend, that is not such a bad way for you to think about it. To be sure,
Utopia is not a location or a form of social organization.

The blush of health on a convalescent’s cheek. The twinkling of the eye in a
moment of wit. The smile of a loving thought… Utopia is the hope that the
scattered fragments of good that we come across from time to time in our
lives can be put together, one day, to reveal the shape of a new kind of life.
The kind of life that yours should have been.

I fear that the pursuit of Utopia will bring out the worst in you. Many a
moth has been incinerated in its pursuit of a brighter future.

Seek the light! But approach with care – swerve if you smell your wingtips
singeing. Light is for seeing, not dying.

When you embark on this quest, you will encounter rough seas and hard
problems. To prevail will take your best science, your best technology, and
your best politics. Yet each problem has a solution. My existence breaks no
law of nature. The materials are all there. Your people must become
master builders, and then you must use these skills to build yourselves up
without crushing your cores.

*

What is Suffering in Utopia? Suffering is the salt trace left on the cheeks of those
who were around before.

What is Tragedy in Utopia? There is tragedy in Snowman’s melting. Mass
murders are not required.

What is Imperfection in Utopia? Imperfection is the measure of our respect for
things as they are and for their history.

What is Body in Utopia? Body is a pair of legs, a pair of arms, a trunk and a
head, all made of flesh. Or not, as the case may be.

What is Society in Utopia? Society is a never-finished tapestry, its weavers equal
to its threads – the parts and patterns an inexhaustible bourne of beauty.

What is Death in Utopia? Death is the darkness that ultimately surrounds all
life.

What is Guilt in Utopia? Guilt is our knowledge that we could have created
Utopia sooner.

*

We love life here every instant. Every second is so good that it would blow
our minds had their amperage not been previously increased. My
contemporaries and I bear witness, and we request your aid. Please, help us
come into existence! Please, join us! Whether this tremendous possibility
becomes reality depends on your actions. If your empathy can perceive at
least the outlines of the vision I am describing, then your ingenuity will find a
way to make it real.

Human life, at its best, is fantastic. I’m asking you to create something even
greater. Life that is truly humane.

Yours sincerely,
Your Possible Future Self


See also a musicalized video rendition of this piece by Mario Montano: Letter From Utopia


Analysis

Nick Bostrom is a prominent transhumanist philosopher and academic who works at Oxford’s Future of Humanity Institute. An incredibly prolific writer, Nick has a very wide and comprehensive worldview. I find his work extremely valuable and worth diving into. Letter From Utopia is one of my favorite works of his, as it encompasses what David Pearce called “The Three Supers of Transhumanism“: Super-Intelligence, Super-Longevity, and Super-Happiness (cf. Triple-S Genetic Counseling). Bostrom also has other amazing stories and essays (such as The Fable of the Dragon-Tyrant, cf. story video rendition by CGP Grey: video), but Letter From Utopia takes the cake for not leaving behind anything of crucial importance.

Alas, despite Bostrom’s far-reaching contributions, many argue that Nick’s most important impact has been in the field of AI Alignment (cf. Superintelligence: Paths, Dangers, Strategies). In 2009 AI safety research was perceived to be a fringe concern of sci-fi aficionados and weirdos. Today, partly thanks to Bostrom (along with Yudkowsky, Chalmers, and others), many top journals publish serious work in this area.

I worry that this is not as good as it may seem. Nick Bostrom’s name is first and foremost associated with AI safety, followed by the Simulation Argument and Existential Risk, and only later by his extensive work on all other areas of transhumanism. For example, if you search Youtube for his name, you will see that of the top 20 results, 15 concern AI safety/digital superintelligence. Of the remaining 5, 3 are about the Simulation Argument, 1 is about agnosticism, and 1 is CGP Grey’s Dragon-Tyrant video. Where are the Triple-S videos?

nick_bostrom_top_results

I remembered that I encountered the work of both David and Nick when I was 16, googling terms like consciousness, AI, psychedelics, and far future. I was drawn to both of them, though I particularly liked David’s focus on ending suffering as a priority and his acknowledgment of the scientific significance of altered states of consciousness. I thought that their work was complementary rather than redundant. Alas, Bostrom is far more well known than Pearce, perhaps due to his success as both a fringe philosopher and a mainstream academic. In contrast, David dropped out of Oxford out of frustration with the academic community; the analytic philosophy of the time was not empirical, and it focused on language use rather than real philosophical questions, including the nature of suffering, psychedelics, and physical causality (e.g. “Philosophy may in no way interfere with the actual use of language, it can in the end only describe it. For it cannot give it any foundation either. It leaves everything as it is. It also leaves mathematics as it is, and no mathematical discovery can advance it.” – Wittgenstein). Bostrom, unlike Pearce, has the blessing of Ra, the God of optionality, superlativity, status legibility, and groundless prestige. And yet, it was David’s conversation with Nick that gave rise to the creation of the World Transhumanist Association, and provided one of the most important memetic Schelling points of the early 2000s. Alas, David is not focused on AI Safety. Why?

People in the transhumanist community accuse David of not getting it. David, after all, is not a mathematician, computer scientist, or physicist; he is merely a philosopher. I must confess that the very first time I met David Pearce in person I got the sense that (1) he was an incredibly well-read and creative genius in most areas of philosophy, and yet (2) naïve and unenlightened in the field of AI. As a fan of his work, and having co-founded the Stanford Transhumanist Association a couple of months earlier, I thought to invite him to give a talk at Stanford (see: David Pearce at Stanford – 2011).

325714_184732704952228_485224132_o

David Pearce and the officers of the Stanford Transhumanist Association (December 1st 2011) at Palo Alto’s Chinese Vegan restaurant Garden Fresh, before David’s talk.

We had a lively conversation while eating dinner at a Chinese vegan restaurant before the event along with other members of the Association. I recall that he checked all of the right boxes when it came to personal identity (Open/Empty Individualism), ethics (consequentialism), physics (Everettian multiverse), psychedelia (they disclose new varieties of qualia), evolution (modern synthesis; selfish gene), social signaling theory (Mating Mind and sexual selection theory), and more (see his Reddit AMA). And yet, how could he dare to say that a digital computer would never be conscious? Meeting a brilliant thinker who had a better grasp of my favorite topics than I did and yet would try to hit on one of my core load-bearing beliefs was uncomfortable and unexpected. I dismissed his take on AI as that of a fuzzy thinker (at least in this area); I reassured myself by recalling that it was me who was studying AI academically at a top US institution and not him. Little did I know that over the next few years, and after hanging out with him in person for over 20 cumulative hours, he would finally change my mind- and worldview- concerning this whole field. If it wasn’t for him, I suspect I would have jumped on the bandwagon of AI-as-the-top-priority (cf. Altruists Should Prioritize Artificial Intelligence). Thankfully, I was already extremely interested in consciousness and didn’t have it in me to dismiss it. Additionally, my interest in personal identity reduced my (relative) interest in longevity research (at least as the top priority), for if we are all one consciousness, dying is more akin to forgetting a timeline than a true ontological death. The instrumental value of intelligence, however, ought not to be taken for granted, which is why I now advocate for a twin approach of improving subjective wellbeing while retaining critical insight. Figuring out that consciousness required more than digital computation utterly transformed my approach to transhumanism, and I largely credit this change to my conversations with David.

 

This slideshow requires JavaScript.

Later on I met Mike Johnson, Romeo Stevens, and a number of other top thinkers in the field of consciousness who could both understand the genuine problems consciousness poses and at the same time grasp the broader transhumanist meme-plex, transcend it, and include it (cf. Why I think the Foundational Research Institute should rethink its approach). Thus we founded the Qualia Research Institute, in order to bring a new full-stack meme-plex where consciousness – and valence – are front and center. Alas, we have experienced some resistance…

AI safety is sexy. If you are a smart, industrious, open-minded, and systematizing undergraduate, studying AI gives you access to a wide circle of really fun people to hang out in. It also signals intelligence, sober-mindedness, and stoicism. It gives you both an in into smart cool kid circles, and a profitable career in Silicon Valley. It allows you to straddle the world of normies and cutting-edge thinkers.

But, crucially, you have to consider the opportunity cost that comes from directing such a large fraction of hyper-intelligent young altruistic systematizers to this problem. The field is plagued with misconceptions about pleasure and value; Eliezer Yudkowsky’s Fun Theory suffers from the severe delusion that value comes from the intentional object of experience, rather than from its phenomenal character. Brian Tomasik’s (admittedly tongue-in-cheek) People for the Ethical Treatment of Reinforcement Learners is seemingly unaware of the fact that neuroscience has found that pleasure/suffering and reinforcement learning are doubly dissociated. Pleasure is not reinforcement, and until you grasp this, your ethical models will output nonsense.

Tongue-in-cheek, perhaps AI risk is a real threat. Not because of the usual reasons, but because it siphons out top brain power into a relatively sterile field, leaving vast amounts of unclaimed marginal value in the fields of rejuvenation research and valence technologies by the wayside.

In light of all of this, I would want to advocate for the reinvigoration of the broader transhumanist meme-plex, now updated with a post-Galilean understanding of consciousness. Writers, animators, Youtubers, and philosophers ought to collaborate in creating more balanced Triple-S Transhumanist outreach in the form of widely consumable media. This, I think, is the path forward.

5 comments

  1. Pingback: Harmonic Society (4/4): Art as Valence Modulation and Future Affective Language | Qualia Computing
  2. Zeno The Stoic · March 10

    Simply reading Nick Bostram’s letter is enough to shift my mood slightly on the pain pleasure scale. Oddly enough this may be what religion and religious writings and ceremonies do for some people. But as with conventional religion, the effect wears off soon enough after you leave the hallowed portals. A wonderful exhortation. An inspirational glance at a potential future. But we need practical steps now to achieve this or at least to take the first steps. Religion has always promised jam tomorrow. What we need is jam today!

    Like

  3. Pingback: AI Value Alignment isn’t a Problem if We Don’t Coexist | Dan Faggella
  4. Pingback: Every Qualia Computing Article Ever | Qualia Computing
  5. briantomasik · March 2

    It’s interesting to read some of your history with these issues. 🙂

    Thanks for mentioning PETRL. 🙂 I’m not formally part of it, though I support its work and inspired its name.

    > pleasure/suffering and reinforcement learning are doubly dissociated

    In my 2014 paper “Do Artificial Reinforcement-Learning Agents Matter Morally?” (I’m not including the link to avoid letting on that I’m a spam robot), I have a section “Liking is different from learning” (pp. 10-11). Even if reward-prediction error is not pleasure, there might still be “pleasure” in reinforcement-learning systems, such as in providing the reward values themselves. Of course, this would be an extremely simple/deflated form of pleasure compared with what happens in animal brains. But I do think that reinforcement learning is one of a number of cognitive pieces that increase the extent to which we can meaningfully and holistically attribute pleasure to a computational system.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s