Why I think the Foundational Research Institute should rethink its approach

by Mike Johnson

The following is my considered evaluation of the Foundational Research Institute, circa July 2017. I discuss its goal, where I foresee things going wrong with how it defines suffering, and what it could do to avoid these problems.

TL;DR version: functionalism (“consciousness is the sum-total of the functional properties of our brains”) sounds a lot better than it actually turns out to be in practice. In particular, functionalism makes it impossible to define ethics & suffering in a way that can mediate disagreements.

I. What is the Foundational Research Institute?

The Foundational Research Institute (FRI) is a Berlin-based group that “conducts research on how to best reduce the suffering of sentient beings in the near and far future.” Executive Director Max Daniel introduced them at EA Global Boston as “the only EA organization which at an organizational level has the mission of focusing on reducing s-risk.” S-risks are, according to Daniel, “risks where an adverse outcome would bring about suffering on an astronomical scale, vastly exceeding all suffering that has existed on Earth so far.”

Essentially, FRI wants to become the research arm of suffering-focused ethics, and help prevent artificial general intelligence (AGI) failure-modes which might produce suffering on a cosmic scale.

What I like about FRI:

While I have serious qualms about FRI’s research framework, I think the people behind FRI deserve a lot of credit- they seem to be serious people, working hard to build something good. In particular, I want to give them a shoutout for three things:

  • First, FRI takes suffering seriously, and I think that’s important. When times are good, we tend to forget how tongue-chewingly horrific suffering can be. S-risks seem particularly horrifying.
  • Second, FRI isn’t afraid of being weird. FRI has been working on s-risk research for a few years now, and if people are starting to come around to the idea that s-risks are worth thinking about, much of the credit goes to FRI.
  • Third, I have great personal respect for Brian Tomasik, one of FRI’s co-founders. I’ve found him highly thoughtful, generous in debates, and unfailingly principled. In particular, he’s always willing to bite the bullet and work ideas out to their logical end, even if it involves repugnant conclusions.

What is FRI’s research framework?

FRI believes in analytic functionalism, or what David Chalmers calls “Type-A materialism”. Essentially, what this means is there’s no ’theoretical essence’ to consciousness; rather, consciousness is the sum-total of the functional properties of our brains. Since ‘functional properties’ are rather vague, this means consciousness itself is rather vague, in the same way words like “life,” “justice,” and “virtue” are messy and vague.

Brian suggests that this vagueness means there’s an inherently subjective, perhaps arbitrary element to how we define consciousness:

Analytic functionalism looks for functional processes in the brain that roughly capture what we mean by words like “awareness”, “happy”, etc., in a similar way as a biologist may look for precise properties of replicators that roughly capture what we mean by “life”. Just as there can be room for fuzziness about where exactly to draw the boundaries around “life”, different analytic functionalists may have different opinions about where to define the boundaries of “consciousness” and other mental states. This is why consciousness is “up to us to define”. There’s no hard problem of consciousness for the same reason there’s no hard problem of life: consciousness is just a high-level word that we use to refer to lots of detailed processes, and it doesn’t mean anything in addition to those processes.

Finally, Brian argues that the phenomenology of consciousness is identical with the phenomenology of computation:

I know that I’m conscious. I also know, from neuroscience combined with Occam’s razor, that my consciousness consists only of material operations in my brain — probably mostly patterns of neuronal firing that help process inputs, compute intermediate ideas, and produce behavioral outputs. Thus, I can see that consciousness is just the first-person view of certain kinds of computations — as Eliezer Yudkowsky puts it, “How An Algorithm Feels From Inside“. Consciousness is not something separate from or epiphenomenal to these computations. It is these computations, just from their own perspective of trying to think about themselves.

In other words, consciousness is what minds compute. Consciousness is the collection of input operations, intermediate processing, and output behaviors that an entity performs.

And if consciousness is all these things, so too is suffering. Which means suffering is computational, yet also inherently fuzzy, and at least a bit arbitrary; a leaky high-level reification impossible to speak about accurately, since there’s no formal, objective “ground truth”.

II. Why do I worry about FRI’s research framework?

In short, I think FRI has a worthy goal and good people, but its metaphysics actively prevent making progress toward that goal. The following describes why I think that, drawing heavily on Brian’s writings (of FRI’s researchers, Brian seems the most focused on metaphysics):

Note: FRI is not the only EA organization which holds functionalist views on consciousness; much of the following critique would also apply to e.g. MIRI, FHI, and OpenPhil. I focus on FRI because (1) Brian’s writings on consciousness & functionalism have been hugely influential in the community, and are clear enough *to* criticize; (2) the fact that FRI is particularly clear about what it cares about- suffering- allows a particularly clear critique about what problems it will run into with functionalism; (3) I believe FRI is at the forefront of an important cause area which has not crystallized yet, and I think it’s critically important to get these objections bouncing around this subcommunity.

Objection 1: Motte-and-bailey

Brian: “Consciousness is not a thing which exists ‘out there’ or even a separate property of matter; it’s a definitional category into which we classify minds. ‘Is this digital mind really conscious?’ is analogous to ‘Is a rock that people use to eat on really a table?’ [However,] That consciousness is a cluster in thingspace rather than a concrete property of the world does not make reducing suffering less important.”

The FRI model seems to imply that suffering is ineffable enough such that we can’t have an objective definition, yet sufficiently effable that we can coherently talk and care about it. This attempt to have it both ways seems contradictory, or at least in deep tension.

Indeed, I’d argue that the degree to which you can care about something is proportional to the degree to which you can define it objectively. E.g., If I say that “gnireffus” is literally the most terrible thing in the cosmos, that we should spread gnireffus-focused ethics, and that minimizing g-risks (far-future scenarios which involve large amounts of gnireffus) is a moral imperative, but also that what is and what and isn’t gnireffus is rather subjective with no privileged definition, and it’s impossible to objectively tell if a physical system exhibits gnireffus, you might raise any number of objections. This is not an exact metaphor for FRI’s position, but I worry that FRI’s work leans on the intuition that suffering is real and we can speak coherently about it, to a degree greater than its metaphysics formally allow.

Max Daniel (personal communication) suggests that we’re comfortable with a degree of ineffability in other contexts; “Brian claims that the concept of suffering shares the allegedly problematic properties with the concept of a table. But it seems a stretch to say that the alleged tension is problematic when talking about tables. So why would it be problematic when talking about suffering?” However, if we take the anti-realist view that suffering is ‘merely’ a node in the network of language, we have to live with the consequences of this: that ‘suffering’ will lose meaning as we take it away from the network in which it’s embedded (Wittgenstein). But FRI wants to do exactly this, to speak about suffering in the context of AGIs, simulated brains, even video game characters.

We can be anti-realists about suffering (suffering-is-a-node-in-the-network-of-language), or we can argue that we can talk coherently about suffering in novel contexts (AGIs, mind crime, aliens, and so on), but it seems inherently troublesome to claim we can do both at the same time.

Objection 2: Intuition duels

Two people can agree on FRI’s position that there is no objective fact of the matter about what suffering is (no privileged definition), but this also means they have no way of coming to any consensus on the object-level question of whether something can suffer. This isn’t just an academic point: Brian has written extensively about how he believes non-human animals can and do suffer extensively, whereas Yudkowsky (who holds computationalist views, like Brian) has written about how he’s confident that animals are not conscious and cannot suffer, due to their lack of higher-order reasoning.

And if functionalism is having trouble adjudicating the easy cases of suffering–whether monkeys can suffer, or whether dogs can— it doesn’t have a sliver of a chance at dealing with the upcoming hard cases of suffering: whether a given AGI is suffering, or engaging in mind crime; whether a whole-brain emulation (WBE) or synthetic organism or emergent intelligence that doesn’t have the capacity to tell us how it feels (or that we don’t have the capacity to understand) is suffering; if any aliens that we meet in the future can suffer; whether changing the internal architecture of our qualia reports means we’re also changing our qualia; and so on.

In short, FRI’s theory of consciousness isn’t actually a theory of consciousness at all, since it doesn’t do the thing we need a theory of consciousness to do: adjudicate disagreements in a principled way. Instead, it gives up any claim on the sorts of objective facts which could in principle adjudicate disagreements.

This is a source of friction in EA today, but it’s mitigated by the sense that

(1) The EA pie is growing, so it’s better to ignore disagreements than pick fights;

(2) Disagreements over the definition of suffering don’t really matter yet, since we haven’t gotten into the business of making morally-relevant synthetic beings (that we know of) that might be unable to vocalize their suffering.

If the perception of one or both of these conditions change, the lack of some disagreement-adjudicating theory of suffering will matter quite a lot.

Objection 3: Convergence requires common truth

Mike: “[W]hat makes one definition of consciousness better than another? How should we evaluate them?”

Brian: “Consilience among our feelings of empathy, principles of non-discrimination, understandings of cognitive science, etc. It’s similar to the question of what makes one definition of justice or virtue better than another.”

Brian is hoping that affective neuroscience will slowly converge to accurate views on suffering as more and better data about sentience and pain accumulates. But convergence to truth implies something (objective) driving the convergence- in this way, Brian’s framework still seems to require an objective truth of the matter, even though he disclaims most of the benefits of assuming this.

Objection 4: Assuming that consciousness is a reification produces more confusion, not less

Brian: “Consciousness is not a reified thing; it’s not a physical property of the universe that just exists intrinsically. Rather, instances of consciousness are algorithms that are implemented in specific steps. … Consciousness involves specific things that brains do.”

Brian argues that we treat conscious/phenomenology as more ‘real’ than it is. Traditionally, whenever we’ve discovered something is a leaky reification and shouldn’t be treated as ‘too real’, we’ve been able to break it down into more coherent constituent pieces we can treat as real. Life, for instance, wasn’t due to élan vital but a bundle of self-organizing properties & dynamics which generally co-occur. But carrying out this “de-reification” process on consciousness– enumerating its coherent constituent pieces– has proven difficult, especially if we want to preserve some way to speak cogently about suffering.

Speaking for myself, the more I stared into the depths of functionalism, the less certain everything about moral value became– and arguably, I see the same trajectory in Brian’s work and Luke Muehlhauser’s report. Their model uncertainty has seemingly become larger as they’ve looked into techniques for how to “de-reify” consciousness while preserving some flavor of moral value, not smaller. Brian and Luke seem to interpret this as evidence that moral value is intractably complicated, but this is also consistent with consciousness not being a reification, and instead being a real thing. Trying to “de-reify” something that’s not a reification will produce deep confusion, just as surely trying to treat a reification as ‘more real’ than it actually is will.

Edsger W. Dijkstra famously noted that “The purpose of abstraction is not to be vague, but to create a new semantic level in which one can be absolutely precise.” And so if our ways of talking about moral value fail to ‘carve reality at the joints’- then by all means let’s build better ones, rather than giving up on precision.

Objection 5: The Hard Problem of Consciousness is a red herring

Brian spends a lot of time discussing Chalmers’ “Hard Problem of Consciousness”, i.e. the question of why we’re subjectively conscious, and seems to base at least part of his conclusion on not finding this question compelling— he suggests “There’s no hard problem of consciousness for the same reason there’s no hard problem of life: consciousness is just a high-level word that we use to refer to lots of detailed processes, and it doesn’t mean anything in addition to those processes.” I.e., no ‘why’ is necessary; when we take consciousness and subtract out the details of the brain, we’re left with an empty set.

But I think the “Hard Problem” isn’t helpful as a contrastive centerpiece, since it’s unclear what the problem is, and whether it’s analytic or empirical, a statement about cognition or about physics. At the Qualia Research Institute (QRI), we don’t talk much about the Hard Problem; instead, we talk about Qualia Formalism, or the idea that any phenomenological state can be crisply and precisely represented by some mathematical object. I suspect this would be a better foil for Brian’s work than the Hard Problem.

Objection 6: Mapping to reality

Brian argues that consciousness should be defined at the functional/computational level: given a Turing machine, or neural network, the right ‘code’ will produce consciousness. But the problem is that this doesn’t lead to a theory which can ‘compile’ to physics. Consider the following:

Imagine you have a bag of popcorn. Now shake it. There will exist a certain ad-hoc interpretation of bag-of-popcorn-as-computational-system where you just simulated someone getting tortured, and other interpretations that don’t imply that. Did you torture anyone? If you’re a computationalist, no clear answer exists- you both did, and did not, torture someone. This sounds like a ridiculous edge-case that would never come up in real life, but in reality it comes up all the time, since there is no principled way to *objectively derive* what computation(s) any physical system is performing.

I don’t think this is an outlandish view of functionalism; Brian suggests much the same in How to Interpret a Physical System as a Mind“Physicalist views that directly map from physics to moral value are relatively simple to understand. Functionalism is more complex, because it maps from physics to computations to moral value. Moreover, while physics is real and objective, computations are fictional and ‘observer-relative’ (to use John Searle’s terminology). There’s no objective meaning to ‘the computation that this physical system is implementing’ (unless you’re referring to the specific equations of physics that the system is playing out).”

Gordon McCabe (McCabe 2004) provides a more formal argument to this effect— that precisely mapping between physical processes and (Turing-level) computational processes is inherently impossible— in the context of simulations. First, McCabe notes that:

[T]here is a one-[to-]many correspondence between the logical states [of a computer] and the exact electronic states of computer memory. Although there are bijective mappings between numbers and the logical states of computer memory, there are no bijective mappings between numbers and the exact electronic states of memory.

This lack of an exact bijective mapping means that subjective interpretation necessarily creeps in, and so a computational simulation of a physical system can’t be ‘about’ that system in any rigorous way:

In a computer simulation, the values of the physical quantities possessed by the simulated system are represented by the combined states of multiple bits in computer memory. However, the combined states of multiple bits in computer memory only represent numbers because they are deemed to do so under a numeric interpretation. There are many different interpretations of the combined states of multiple bits in computer memory. If the numbers represented by a digital computer are interpretation-dependent, they cannot be objective physical properties. Hence, there can be no objective relationship between the changing pattern of multiple bit-states in computer memory, and the changing pattern of quantity-values of a simulated physical system.

McCabe concludes that, metaphysically speaking,

A digital computer simulation of a physical system cannot exist as, (does not possess the properties and relationships of), anything else other than a physical process occurring upon the components of a computer. In the contemporary case of an electronic digital computer, a simulation cannot exist as anything else other than an electronic physical process occurring upon the components and circuitry of a computer.

Where does this leave ethics? In Flavors of Computation Are Flavors of Consciousness, Brian notes that “In some sense all I’ve proposed here is to think of different flavors of computation as being various flavors of consciousness. But this still leaves the question: Which flavors of computation matter most? Clearly whatever computations happen when a person is in pain are vastly more important than what’s happening in a brain on a lazy afternoon. How can we capture that difference?”

But if Brian grants the former point- that “There’s no objective meaning to ‘the computation that this physical system is implementing’”– then this latter task of figuring out “which flavors of computation matter most” is provably impossible. There will always be multiple computational (and thus ethical) interpretations of a physical system, with no way to figure out what’s “really” happening. No way to figure out if something is suffering or not. No consilience; not now, not ever.

Note: despite apparently granting the point above, Brian also remarks that:

I should add a note on terminology: All computations occur within physics, so any computation is a physical process. Conversely, any physical process proceeds from input conditions to output conditions in a regular manner and so is a computation. Hence, the set of computations equals the set of physical processes, and where I say “computations” in this piece, one could just as well substitute “physical processes” instead.

This seems to be (1) incorrect, for the reasons I give above, or (2) taking substantial poetic license with these terms, or (3) referring to hypercomputation (which might be able to salvage the metaphor, but would invalidate many of FRI’s conclusions dealing with the computability of suffering on conventional hardware).

This objection may seem esoteric or pedantic, but I think it’s important, and that it ripples through FRI’s theoretical framework with disastrous effects.

 

Objection 7: FRI doesn’t fully bite the bullet on computationalism

Brian suggests that “flavors of computation are flavors of consciousness” and that some computations ‘code’ for suffering. But if we do in fact bite the bullet on this metaphor and place suffering within the realm of computational theory, we need to think in “near mode” and accept all the paradoxes that brings. Scott Aaronson, a noted expert on quantum computing, raises the following objections to functionalism:

I’m guessing that many people in this room side with Dennett, and (not coincidentally, I’d say) also with Everett. I certainly have sympathies in that direction too. In fact, I spent seven or eight years of my life as a Dennett/Everett hardcore believer. But, while I don’t want to talk anyone out of the Dennett/Everett view, I’d like to take you on a tour of what I see as some of the extremely interesting questions that that view leaves unanswered. I’m not talking about “deep questions of meaning,” but about something much more straightforward: what exactly does a computational process have to do to qualify as “conscious”?

There’s this old chestnut, what if each person on earth simulated one neuron of your brain, by passing pieces of paper around. It took them several years just to simulate a single second of your thought processes. Would that bring your subjectivity into being? Would you accept it as a replacement for your current body? If so, then what if your brain were simulated, not neuron-by-neuron, but by a gigantic lookup table? That is, what if there were a huge database, much larger than the observable universe (but let’s not worry about that), that hardwired what your brain’s response was to every sequence of stimuli that your sense-organs could possibly receive. Would that bring about your consciousness? Let’s keep pushing: if it would, would it make a difference if anyone actually consulted the lookup table? Why can’t it bring about your consciousness just by sitting there doing nothing?

To these standard thought experiments, we can add more. Let’s suppose that, purely for error-correction purposes, the computer that’s simulating your brain runs the code three times, and takes the majority vote of the outcomes. Would that bring three “copies” of your consciousness into being? Does it make a difference if the three copies are widely separated in space or time—say, on different planets, or in different centuries? Is it possible that the massive redundancy taking place in your brain right now is bringing multiple copies of you into being?

Maybe my favorite thought experiment along these lines was invented by my former student Andy Drucker.  In the past five years, there’s been a revolution in theoretical cryptography, around something called Fully Homomorphic Encryption (FHE), which was first discovered by Craig Gentry.  What FHE lets you do is to perform arbitrary computations on encrypted data, without ever decrypting the data at any point.  So, to someone with the decryption key, you could be proving theorems, simulating planetary motions, etc.  But to someone without the key, it looks for all the world like you’re just shuffling random strings and producing other random strings as output.

You can probably see where this is going.  What if we homomorphically encrypted a simulation of your brain?  And what if we hid the only copy of the decryption key, let’s say in another galaxy?  Would this computation—which looks to anyone in our galaxy like a reshuffling of gobbledygook—be silently producing your consciousness?

When we consider the possibility of a conscious quantum computer, in some sense we inherit all the previous puzzles about conscious classical computers, but then also add a few new ones.  So, let’s say I run a quantum subroutine that simulates your brain, by applying some unitary transformation U.  But then, of course, I want to “uncompute” to get rid of garbage (and thereby enable interference between different branches), so I apply U-1.  Question: when I apply U-1, does your simulated brain experience the same thoughts and feelings a second time?  Is the second experience “the same as” the first, or does it differ somehow, by virtue of being reversed in time? Or, since U-1U is just a convoluted implementation of the identity function, are there no experiences at all here?

Here’s a better one: many of you have heard of the Vaidman bomb.  This is a famous thought experiment in quantum mechanics where there’s a package, and we’d like to “query” it to find out whether it contains a bomb—but if we query it and there is a bomb, it will explode, killing everyone in the room.  What’s the solution?  Well, suppose we could go into a superposition of querying the bomb and not querying it, with only ε amplitude on querying the bomb, and √(1-ε2) amplitude on not querying it.  And suppose we repeat this over and over—each time, moving ε amplitude onto the “query the bomb” state if there’s no bomb there, but moving ε2 probability onto the “query the bomb” state if there is a bomb (since the explosion decoheres the superposition).  Then after 1/ε repetitions, we’ll have order 1 probability of being in the “query the bomb” state if there’s no bomb.  By contrast, if there is a bomb, then the total probability we’ve ever entered that state is (1/ε)×ε2 = ε.  So, either way, we learn whether there’s a bomb, and the probability that we set the bomb off can be made arbitrarily small.  (Incidentally, this is extremely closely related to how Grover’s algorithm works.)

OK, now how about the Vaidman brain?  We’ve got a quantum subroutine simulating your brain, and we want to ask it a yes-or-no question.  We do so by querying that subroutine with ε amplitude 1/ε times, in such a way that if your answer is “yes,” then we’ve only ever activated the subroutine with total probability ε.  Yet you still manage to communicate your “yes” answer to the outside world.  So, should we say that you were conscious only in the ε fraction of the wavefunction where the simulation happened, or that the entire system was conscious?  (The answer could matter a lot for anthropic purposes.)

To sum up: Brian’s notion that consciousness is the same as computation raises more issues than it solves; in particular, the possibility that if suffering is computable, it may also be uncomputable/reversible, would suggest s-risks aren’t as serious as FRI treats them.

Objection 8: Dangerous combination

Three themes which seem to permeate FRI’s research are:

(1) Suffering is the thing that is bad.

(2) It’s critically important to eliminate badness from the universe.

(3) Suffering is impossible to define objectively, and so we each must define what suffering means for ourselves.

Taken individually, each of these seems reasonable. Pick two, and you’re still okay. Pick all three, though, and you get A Fully General Justification For Anything, based on what is ultimately a subjective/aesthetic call.

Much can be said in FRI’s defense here, and it’s unfair to single them out as risky: in my experience they’ve always brought a very thoughtful, measured, cooperative approach to the table. I would just note that ideas are powerful, and I think theme (3) is especially pernicious if incorrect.

III. QRI’s alternative

Analytic functionalism is essentially a negative hypothesis about consciousness: it’s the argument that there’s no order to be found, no rigor to be had. It obscures this with talk of “function”, which is a red herring it not only doesn’t define, but admits is undefinable. It doesn’t make any positive assertion. Functionalism is skepticism- nothing more, nothing less.

But is it right?

Ultimately, I think these a priori arguments are much like people in the middle ages arguing whether one could ever formalize a Proper System of Alchemy. Such arguments may in many cases hold water, but it’s often difficult to tell good arguments apart from arguments where we’re just cleverly fooling ourselves. In retrospect, the best way to *prove* systematized alchemy was possible was to just go out and *do* it, and invent Chemistry. That’s how I see what we’re doing at QRI with Qualia Formalism: we’re assuming it’s possible to build stuff, and we’re working on building the object-level stuff.

What we’ve built with QRI’s framework

Note: this is a brief, surface-level tour of our research; it will probably be confusing for readers who haven’t dug into our stuff before. Consider this a down-payment on a more substantial introduction.

My most notable work is Principia Qualia, in which I lay out my meta-framework for consciousness (a flavor of dual-aspect monism, with a focus on Qualia Formalism) and put forth the Symmetry Theory of Valence (STV). Essentially, the STV is an argument that much of the apparent complexity of emotional valence is evolutionarily contingent, and if we consider a mathematical object isomorphic to a phenomenological experience, the mathematical property which corresponds to how pleasant it is to be that experience is the object’s symmetry. This implies a bunch of testable predictions and reinterpretations of things like what ‘pleasure centers’ do (Section XI; Section XII). Building on this, I offer the Symmetry Theory of Homeostatic Regulation, which suggests understanding the structure of qualia will translate into knowledge about the structure of human intelligence, and I briefly touch on the idea of Neuroacoustics.

Likewise, my colleague Andrés Gómez Emilsson has written about the likely mathematics of phenomenology, includingThe Hyperbolic Geometry of DMT Experiences, Tyranny of the Intentional Object, and Algorithmic Reduction of Psychedelic States. If I had to suggest one thing to read in all of these links, though, it would be the transcript of his recent talk on Quantifying Bliss, which lays out the world’s first method to objectively measure valence from first principles (via fMRI) using Selen Atasoy’s Connectome Harmonics framework, the Symmetry Theory of Valence, and Andrés’s CDNS model of experience.

These are risky predictions and we don’t yet know if they’re right, but we’re confident that if there is some elegant structure intrinsic to consciousness, as there is in many other parts of the natural world, these are the right kind of risks to take.

I mention all this because I think analytic functionalism- which is to say radical skepticism/eliminativism, the metaphysics of last resort- only looks as good as it does because nobody’s been building out any alternatives.

IV. Closing thoughts

FRI is pursuing a certain research agenda, and QRI is pursuing another, and there’s lots of value in independent explorations of the nature of suffering. I’m glad FRI exists, everybody I’ve interacted with at FRI has been great, I’m happy they’re focusing on s-risks, and I look forward to seeing what they produce in the future.

On the other hand, I worry that nobody’s pushing back on FRI’s metaphysics, which seem to unavoidably lead to the intractable problems I describe above. FRI seems to believe these problems are part of the territory, unavoidable messes that we just have to make philosophical peace with. But I think that functionalism is a bad map, that the metaphysical messes it leads to are much worse than most people realize (fatal to FRI’s mission), and there are other options that avoid these problems (which, to be fair, is not to say they have no problems).

Ultimately, FRI doesn’t owe me a defense of their position. But if they’re open to suggestions on what it would take to convince a skeptic like me that their brand of functionalism is viable, or at least rescuable, I’d offer the following:

Re: Objection 1 (motte-and-bailey), I suggest FRI should be as clear and complete as possible in their basic definition of suffering. In which particular ways is it ineffable/fuzzy, and in which particular ways is it precise? What can we definitely say about suffering, and what can we definitely never determine? Preregistering ontological commitments and methodological possibilities would help guard against FRI’s definition of suffering changing based on context.

Re: Objection 2 (intuition duels), FRI may want to internally “war game” various future scenarios involving AGI, WBE, etc, with one side arguing that a given synthetic (or even extraterrestrial) organism is suffering, and the other side arguing that it isn’t. I’d expect this would help diagnose what sorts of disagreements future theories of suffering will need to adjudicate, and perhaps illuminate implicit ethical intuitions. Sharing the results of these simulated disagreements would also be helpful in making FRI’s reasoning less opaque to outsiders, although making everything transparent could lead to certain strategic disadvantages.

Re: Objection 3 (convergence requires common truth), I’d like FRI to explore exactly what might drive consilience/convergence in theories of suffering, and what precisely makes one theory of suffering better than another, and ideally to evaluate a range of example theories of suffering under these criteria.

Re: Objection 4 (assuming that consciousness is a reification produces more confusion, not less), I would love to see a historical treatment of reification: lists of reifications which were later dissolved (e.g., élan vital), vs scattered phenomena that were later unified (e.g., electromagnetism). What patterns do the former have, vs the latter, and why might consciousness fit one of these buckets better than the other?

Re: Objection 5 (the Hard Problem of Consciousness is a red herring), I’d like to see a more detailed treatment of what kinds of problem people have interpreted the Hard Problem as, and also more analysis on the prospects of Qualia Formalism (which I think is the maximally-empirical, maximally-charitable interpretation of the Hard Problem). It would be helpful for us, in particular, if FRI preregistered their expectations about QRI’s predictions, and their view of the relative evidence strength of each of our predictions.

Re: Objection 6 (mapping to reality), this is perhaps the heart of most of our disagreement. From Brian’s quotes, he seems split on this issue; I’d like clarification about whether he believes we can ever precisely/objectively map specific computations to specific physical systems, and vice-versa. And if so— how? If not, this seems to propagate through FRI’s ethical framework in a disastrous way, since anyone can argue that any physical system does, or does not, ‘code’ for massive suffering, and there’s no principled way to derive any ‘ground truth’ or even pick between interpretations in a principled way (e.g. my popcorn example). If this isn’t the case— why not?

Brian has suggested that “certain high-level interpretations of physical systems are more ‘natural’ and useful than others” (personal communication); I agree, and would encourage FRI to explore systematizing this.

It would be non-trivial to port FRI’s theories and computational intuitions to the framework of “hypercomputation”– i.e., the understanding that there’s a formal hierarchy of computational systems, and that Turing machines are only one level of many– but it may have benefits too. Namely, it might be the only way they could avoid Objection 6 (which I think is a fatal objection) while still allowing them to speak about computation & consciousness in the same breath. I think FRI should look at this and see if it makes sense to them.

Re: Objection 7 (FRI doesn’t fully bite the bullet on computationalism), I’d like to see responses to Aaronson’s aforementioned thought experiments.

Re: Objection 8 (dangerous combination), I’d like to see a clarification about why my interpretation is unreasonable (as it very well may be!).

 


In conclusion- I think FRI has a critically important goal- reduction of suffering & s-risk. However, I also think FRI has painted itself into a corner by explicitly disallowing a clear, disagreement-mediating definition for what these things are. I look forward to further work in this field.

 

Mike Johnson

Qualia Research Institute


Acknowledgements: thanks to Andrés Gómez Emilsson, Brian Tomasik, and Max Daniel for reviewing earlier drafts of this.

Sources:

My sources for FRI’s views on consciousness:

Flavors of Computation are Flavors of Consciousness:

https://foundational-research.org/flavors-of-computation-are-flavors-of-consciousness/

 

Is There a Hard Problem of Consciousness?

http://reducing-suffering.org/hard-problem-consciousness/

Consciousness Is a Process, Not a Moment

http://reducing-suffering.org/consciousness-is-a-process-not-a-moment/

 

How to Interpret a Physical System as a Mind

http://reducing-suffering.org/interpret-physical-system-mind/

 

Dissolving Confusion about Consciousness

http://reducing-suffering.org/dissolving-confusion-about-consciousness/

 

Debate between Brian & Mike on consciousness:

https://www.facebook.com/groups/effective.altruists/permalink/1333798200009867/?comment_id=1333823816673972&comment_tracking=%7B%22tn%22%3A%22R9%22%7D

Max Daniel’s EA Global Boston 2017 talk on s-risks:
https://www.youtube.com/watch?v=jiZxEJcFExc
Multipolar debate between Eliezer Yudkowsky and various rationalists about animal suffering:
https://rationalconspiracy.com/2015/12/16/a-debate-on-animal-consciousness/
The Internet Encyclopedia of Philosophy on functionalism:
http://www.iep.utm.edu/functism/
Gordon McCabe on why computation doesn’t map to physics:
http://philsci-archive.pitt.edu/1891/1/UniverseCreationComputer.pdf
Toby Ord on hypercomputation, and how it differs from Turing’s work:
https://arxiv.org/abs/math/0209332
Luke Muehlhauser’s OpenPhil-funded report on consciousness and moral patienthood:
http://www.openphilanthropy.org/2017-report-consciousness-and-moral-patienthood
Scott Aaronson’s thought experiments on computationalism:
http://www.scottaaronson.com/blog/?p=1951
Selen Atasoy on Connectome Harmonics, a new way to understand brain activity:
https://www.nature.com/articles/ncomms10340
My work on formalizing phenomenology:

My meta-framework for consciousness, including the Symmetry Theory of Valence:

http://opentheory.net/PrincipiaQualia.pdf

 

My hypothesis of homeostatic regulation, which touches on why we seek out pleasure:

http://opentheory.net/2017/05/why-we-seek-out-pleasure-the-symmetry-theory-of-homeostatic-regulation/

 

My exploration & parametrization of the ‘neuroacoustics’ metaphor suggested by Atasoy’s work:

http://opentheory.net/2017/06/taking-brain-waves-seriously-neuroacoustics/

My colleague Andrés’s work on formalizing phenomenology:
A model of DMT-trip-as-hyperbolic-experience:
https://qualiacomputing.com/2017/05/28/eli5-the-hyperbolic-geometry-of-dmt-experiences/
June 2017 talk at Consciousness Hacking, describing a theory and experiment to predict people’s valence from fMRI data:
https://qualiacomputing.com/2017/06/18/quantifying-bliss-talk-summary/
A parametrization of various psychedelic states as operators in qualia space:
https://qualiacomputing.com/2016/06/20/algorithmic-reduction-of-psychedelic-states/
A brief post on valence and the fundamental attribution error:
https://qualiacomputing.com/2016/11/19/the-tyranny-of-the-intentional-object/
A summary of some of Selen Atasoy’s current work on Connectome Harmonics:
https://qualiacomputing.com/2017/06/18/connectome-specific-harmonic-waves-on-lsd/

Connectome-Specific Harmonic Waves on LSD

The harmonics-in-connectome approach to modeling brain activity is a fascinating paradigm. I am privileged to have been at this talk in the 2017 Psychedelic Science conference. I’m extremely happy find out that MAPS already uploaded the talks. Dive in!

Below is a partial transcript of the talk. I figured that I should get it in written form in order to be able to reference it in future articles. Enjoy!

[After a brief introduction about harmonic waves in many different kinds of systems… at 7:04, Selen Atasoy]:

We applied the [principle of harmonic decomposition] to the anatomy of the brain. We made them connectome-specific. So first of all, what do I mean by the human connectome? Today thanks to the recent developments in structural neuroimaging techniques such as diffusion tensor imaging, we can trace the long-distance white matter connections in the brain. These long-distance white matter fibers (as you see in the image) connect distant parts of the brain, distant parts of the cortex. And the set of all of the different connections is called the connectome.

selen_connectome_image

Now, because we know the equation governing these harmonic waves, we can extend this principle to the human brain by simply solving the same equation on the human connectome instead of a metal plate (Chladni plates) or the anatomy of the zebra. And if you do that, we get a set of harmonic patterns, this time emerging in the cortex. And we decided to call these harmonic patterns connectome harmincs. And each of these connectome harmonic patterns are associated with a different frequency. And because they correspond to different frequencies they are all independent, and together they give you a new language, so to speak, to describe neural activity. So in the same way the harmonic patterns are building blocks of these complex patterns we see on animal coats, these connectome harmonics are the building blocks of the complex spatio-temporal patterns of neural activity.

Describing and explaining neural activity by using these connectome harmonics as brain states is really not very different than decomposing a complex musical pieces into its musical notes. It’s simply a new way of representing your data, or a new language to express it.

What is the advantage of using this new language? So why not use the state-of-the-art conventional neurimaging analysis methods? Because these connectome harmonics, by definition are the vibration modes, but applied to the anatomy of the human brain, and if you use them as brain states to express neural activity we can compute certain fundamental principles very easily such as the energy or the power.

The power would be the strength of activation of each of these states in neural activity. So how strongly that particular state contributes to neural activity. And the energy would be a combination of this strength of activation with the intrinsic energy of that particular brain state, and the intrinsic energy comes from the frequency of its vibration (in the analogy of vibration).

So in this study we looked at the power and the energy of these connectome harmonic brain states in order to explore the neural correlates of the LSD experience.

We looked at 12 healthy participants who received either 75µg of LSD (IV) or a placebo, over two sessions. These two sessions were 14 days apart in counter-balanced order. And the fMRI scans consisted of 3 eyes-closed resting states scans, each lasting 7 minutes, in the first and the third scan the participants were simply resting, eyes closed, but in the second scan they were also listening to music. And after each scan, the participants rated the intensity of certain experiences.

activity_power_energy_scans.png

So if you look at, firstly, at the total power and the total energy of each of these scans under LSD and placebo, what we see is that under LSD both the power as well as the energy of brain activity increases significantly.

And if we compute the probability of observing a certain energy value on LSD or placebo, what we see is that the peak of this probability distribution clearly shoots towards high energy values under LSD.

energy_difference

And that peak is even slightly higher in terms of probability when the subjects were listening to music. So if we interpret that peak as, in a way, the characteristic energy of a state, you can see that it shifts towards higher energy under LSD, and that this effect is intensified when listening to music.

And then we asked, which of these brain states, which of these frequencies, were actually contributing to this energy increase. So we partitioned the spectrum of all of these harmonic brain states into different parts and computed the energy of each of these partitions individually. So in total we have around 20,000 brain states. And if you look at the energy differences in LSD and placebo, what we find is that for a very narrow range of low frequencies actually these brain states were decreasing their energy on LSD. But for a very broad range of high frequencies, LSD was inducing an energy increase. So this says that LSD alters brain dynamics in a very frequency-selective manner. And it was causing high frequencies to increase their energy.

So next we looked at whether these changes we are observing in brain activity are correlated with any of the experiences that the participants themselves were having in that moment. If you look at the energy changes within the narrow range of low frequencies, we found that the energy changes in that range significantly correlated with the intensity of the experience of ego dissolution. The loss of subjective self.

ego_dissolution

And very interestingly, the same range of energy change within the same frequency range also significantly correlated with the intensity of emotional arousal, whether the experience was positive or negative. This could be quite relevant for studies looking into potential therapeutic applications of LSD.

emotional_arousal

Next, when we look at a slightly higher range of frequencies, what we found was that the energy changes within that range significantly correlated with the positive mood.

higher_frequencies

In brief, this suggests that it’s rather the low frequency brain states which correlated with ego dissolution or with emotional arousal, and it’s the activity of higher frequencies that is correlated with the positive experiences.

Next, we wanted to check the size of the repertoire of active brain states. And if you look at the probability of activation for any brain state (so we are not distinguishing for any frequency brain states), what we observe is that the probability of a brain state being silent (zero contribution), actually decreased under LSD. And the probability of a brain state contributing very strongly, which corresponds to the tails of these distributions, were increased under LSD. So this suggests that LSD was activating more brain states simultaneously.

repertoaire

And if we go back to the music analogy that we used in the beginning, that would correspond to playing more musical notes at the same time. And it’s very interesting, because studies that have looked at improvising, those who have looked at jazz improvisation, show that improvising jazz musicians play significantly more musical notes compared to memorized play. And this is what we seem to be finding under the effect of LSD. That your brain is actually activating more of these brain states simultaneously.

cross-frequency

And it does so in a very non-random fashion. So if you look at the correlation across different frequencies. Like at the co-activation patterns, and their activation over time. You may interpret it as the “communication across various frequencies”. What we found is that for a very broad range of the spectrum, there was a higher correlation across different frequencies in their activation patterns under LSD compared to placebo.

So this really says that LSD is actually causing a reorganization, rather than a random activation of brain states. It’s expanding the repertoire of active brain states, while maintaining -or maybe better said- recreating a complex but spontaneous order. And in the musical analogy it’s really very similar to jazz improvisation, to think about it in an intuitive way.

Now, there is actually one particular situation when dynamical systems such as the brain, and systems that change their activity over time, show this type of emergence of complex order, or enhanced improvisation, enhanced repertoire of active states. And this is when they approach what is called criticality. Now, criticality is this special type of behavior, special type of dynamics, that emerges right at the transition between order and chaos. When these two (extreme) types of dynamics are in balance. And criticality is said to be “the constantly shifting battle zone between stagnation and anarchy. The one place where a complex system can be spontaneous, adaptive, and alive” (Waldrop 1992). So if a system is approaching criticality, there are very characteristic signatures that you would observed in the data, in the relationships that you plot in your data.

And one of them is -and probably the most characteristic of them- is the emergence of power laws. So what does that mean? If you plot one observable in our data, which for example, in our case would be the maximum power of a brain state, in relationship to another observable, for example, the wavenumber, or the frequency of that brain state, and you plot them in logarithmic coordinates, that would mean that if they follow power laws, they would approximate a line. And this is exactly what we observe in our data, and surprisingly for both LSD as well as for placebo, but with one very significant and remarkable difference: because the high frequencies increase their power on LSD, this distribution follows this power law, this line, way more accurately under LSD compared to placebo. And here you see the error of the fit, which is decreasing.

This suggests that LSD shoots brain dynamics further towards criticality.  The signature of criticality that we find in LSD and in placebo is way more enhanced, way more pronounced, under the effect of LSD. And we found the same effect, not only for the maximum power, but also for the mean power, as well as for the power of fluctuations.

criticality_signature

So this suggests that the criticality actually may be the principle that is underlying this emergence of complex order, and this reorganization of brain dynamics, and which leads to enhanced improvisation in brain activity.

So, to summarize briefly, what we found was that LSD increases the total power as well as total energy of brain activity. It selectively activates high frequency brain states, and it expands the repertoire or active brain states in a very non-random fashion. And the principle underlying all of these changes seems to be a reorganization of brain dynamics, right at criticality, right at the edge of chaos, or just as the balance between order and chaos. And very interestingly, the “edge of chaos”, or the edge of criticality, is said to be where “life has enough stability to sustain itself, and enough creativity to deserve the name of life” (Waldrop 1992). So I leave you with that, and thank you for your attention.

[Applauses; ends at 22:00, followed by Q&A]

Qualia Computing at Consciousness Hacking (June 7th 2017)

I am delighted to announce that I will be presenting at Consciousness Hacking in San Francisco on 2017/6/7 (YMD notation).

Consciousness Hacking (CoHack) is an extremely awesome community that blends a genuine interest in benevolence, scientific rationality, experiential spirituality, self-experimentation, and holistic wellbeing together with an unceasing focus on consciousness. Truth be told, CohHack is one of the reasons why I love living in the Bay Area.

Here are the relevant event links: Eventbrite, FacebookMeetup.

And the event description:


What would happen if a bliss technology capable of inducing a constant MDMA-like state of consciousness with no negative side effects were available? What makes an experience good or bad? Is happiness a spiritual trick, or is spirituality a happiness trick?

At this month’s speaker presentation, Consciousness Hacking invites Data Science Engineer, Andrés Gómez Emilsson to discuss current research, including his own, concerning the measurement of bliss, how blissful brain states can be induced, and what implications this may have on quality of life and our relationship with the world around us.

Emilsson’s research aims to create a mathematical theory of the pleasure-pain axis that can take information about a person’s brain at a given point in time and return the approximate (or even true) level of happiness and suffering for that person. Emilsson will explore two dimensions that have been studied in affective neuroscience for decades:

  • Arousal: how much energy and activation a given emotion has
  • Valence: the “feel good or feel bad” dimension of emotion

If the purpose of life is to feel happy and to make others happy, then figuring out how valence is implemented in the brain may take us a long way in that direction. Current approaches to valence, while helpful, usually don’t address the core of the problem (ie. usually just measuring the symptoms of pleasure such as the neurotransmitters that trigger it, brain regions, positive reinforcement, etc. rather than getting at the experience of pleasure itself).

A real science of valence would not only be able to integrate and explain why the things people report as pleasurable are pleasant, it would also make a precise, empirically falsifiable hypothesis about whether arbitrary brain states will feel good or bad. This is what Emilsson aims to do.

You will take away:

  • An understanding about the current scientific consensus on the nature of happiness in the brain, and why it is incomplete
  • A philosophical case for both the feasibility and desirability of a world devoid of intense suffering
  • A new candidate mathematical formula that can be used to predict the psychological wellbeing of a brain at a given point in time
  • An argument for why bliss technology that puts us in a constant MDMA-like state of consciousness with no negative side effects is likely to become available within the next two to five decades
  • The opportunity to network with other people who are serious about figuring out the meaning of life through introspection and neuroscience

About our speaker:

Andrés Gómez Emilsson was born in México City in 1990. From an early age, he developed an interest in philosophy, mathematics, and science, leading him to compete nationally and internationally in Math and Science Olympiads. At 16, his main interest was mathematics, but after an unexpected “mystical experience”, he turned his attention to consciousness and the philosophical problems that it poses. He studied Symbolic Systems with an Artificial Intelligence concentration at Stanford, and later finished a masters in Computational Psychology at the same university. During his time at Stanford he co-founded the Stanford Transhumanist Association and became good friends with transhumanist philosopher David Pearce, taking on the flag of the Hedonistic Imperative (HI). In order to pursue the long-term goals of HI, his current primary intellectual interest is to reverse-engineer the functional, biochemical and/or quantum signatures of pure bliss.

He is currently working at a Natural Language Processing company in San Francisco, creating quantitative measures of employee happiness, productivity, and ethics at companies, with the long-term intent of creating a consciousness research institute that’s also a great place to work for (i.e. one in which employees are happy, productive, and ethical). In his free time he develops psychophysical tools to study the computational properties of consciousness.

Schedule:

6:30: Check in, snacks

6:45: Structured schmoozing

6:55: Event intro and meditation

7:00: Andrés Gómez Emilsson

7:50: Break

8:00: Break-out Sessions (small group discussion)

9:00: Break-out Recap

9:15: Closing meditation

About our venue:

ECO-SYSTM is a dynamic community of creative professionals, startups, and freelancers, founded on the idea that entertainment, creativity and business can come together to offer a truly unique work experience for Bay Area professionals. Check out membership plans here: http://eco-systm.com/


 

The Most Important Philosophical Question

Albert Camus famously claimed that the most important philosophical question in existence was whether to commit suicide. I would disagree.

For one, if Open Individualism is true (i.e. that deep down we are all one and the same consciousness) then ending one’s life will not accomplish much. The vast majority of “who you are” will remain intact, and if there are further problems to be solved, and questions to be answered, doing this will simply delay your own progress. So at least from a certain point of view one could argue that the most important question is, instead, the question of personal identity. I.e. Are you, deep down, an individual being who starts existing when you are born and stops existing when you die (Closed Individualism), something that exists only for a single time-slice (Empty Individualism), or maybe something that is one and the same with the rest of the universe (Open Individualism)?

I think that is a very important question. But probably not the most important one. Instead, I’d posit that the most important question is: “What is good, and is there a ground truth about it?”

In the case that we are all one consciousness maybe what’s truly good is whatever one actually truly values from a first-person point of view (being mindful, of course, of the deceptive potential that comes from the Tyranny of the Intentional Object). And in so far as this has been asked, I think that there are two remaining possibilities: Does ultimate value come down to the pleasure-pain axis, or does it come down to spiritual wisdom?

Thus, in this day and age, I’d argue that the most important philosophical (and hence most important, period) question is: “Is happiness a spiritual trick, or is spirituality a happiness trick?”

What would it mean for happiness to be a spiritual trick? Think, for example, of the possibility that the reason why we exist is because we are all God, and God would be awfully bored if It knew that It was all that ever existed. In such a case, maybe bliss and happiness comes down to something akin to “Does this particular set of life experiences make God feel less lonely”? Alternatively, maybe God is “divinely self-sufficient”, as some mystics claim, and all of creation is “merely a plus on top of God”. In this case one could think that God is the ultimate source of all that is good, and thus bliss may be synonymous with “being closer to God”. In turn, as mystics have claimed over the ages, the whole point of life is to “get closer to God”.

Spirituality, though, goes beyond God: Within (atheistic) Buddhism the view that “bliss is a spiritual trick” might take another form: Bliss is either “dirty and a sign of ignorance” (as in the case of karma-generating pleasure) or it is “the results of virtuous merit conducive to true unconditioned enlightenment“. Thus, the whole point of life would be to become free from ignorance and reap the benefits of knowing the ultimate truth.

And what would it mean for spirituality to be a happiness trick? In this case one could imagine that our valence (i.e. our pleasure-pain axis) is a sort of qualia variety that evolution recruited in order to infuse the phenomenal representation of situations that predict either higher or lower chances of making copies of oneself (or spreading one’s genes, in the more general case of “inclusive fitness”). If this is so, it might be tempting to think that bliss is, ultimately, not something that “truly matters”. But this would be to think that bliss is “nothing other than the function that bliss plays in animal behavior”, which couldn’t be further from the truth. After all, the same behavior could be enacted by many methods. Instead, the raw phenomenal character of bliss reveals that “something matters in this universe”. Only people who are anhedonic (or are depressed) will miss the fact that “bliss matters”. This is self-evident and self-intimating to anyone currently experiencing ecstatic rapture. In light of these experiences we can conclude that if anything at all does matter, it has to do with the qualia varieties involved in the experiences that feel like the world has meaning. The pleasure-pain axis makes our existence significant.

Now, why do I think this is the most important question? IF we discover that happiness is a spiritual trick and that God is its source then we really ought to follow “the spiritual path” and figure out with science “what is it that God truly wants”. And under an atheistic brand of spirituality, what we ought to figure out is the laws of valence-charged spiritual energy. For example, if reincarnation and karma are involved in the expected amount of future bliss and suffering, so be it. Let’s all become Bodhisattvas and help as many sentient beings as possible throughout the eons to come.

On the other hand, IF we discover (and can prove with a good empirical argument) that spirituality is just the result of changes in valence/happiness, then settling on this with a high certainty would change the world. For starters, any compassionate (and at least mildly rational) Buddhist would then come along and help us out in the pursuit of creating a pan-species welfare state free of suffering with the use of biotechnology. I.e. The 500 odd million Buddhists world-wide would be key allies for the Hedonistic Imperative (a movement that aims to eliminate suffering with biotechnology).

Recall Dalai Lama’s quote: “If it was possible to become free of negative emotions by a riskless implementation of an electrode – without impairing intelligence and the critical mind – I would be the first patient.” [Dalai Lama (Society for Neuroscience Congress, Nov. 2005)].

If Buddhist doctrine concerning the very nature of suffering and its causes is wrong from a scientific point of view and we can prove it with an empirically verified physicalist paradigm, then the very Buddhist ethic of “focusing on minimizing suffering” ought to compel Buddhists throughout the world to join us in the battle against suffering by any means necessary. And most likely, given the physicalist premise, this would take the form of creating a technology that puts us all in a perpetual pro-social clear-headed non-addictive MDMA-like state of consciousness (or, in a more sophisticated vein, a well-balanced version of rational wire-heading).