## Estimated Cost of the DMT Machine Elves Prime Factorization Experiment

“Okay,” I said. “Fine. Let me tell you where I’m coming from. I was reading Scott McGreal’s blog, which has some good articles about so-called DMT entities, and mentions how they seem so real that users of the drug insist they’ve made contact with actual superhuman beings and not just psychedelic hallucinations. You know, the usual Terence McKenna stuff. But in one of them he mentions a paper by Marko Rodriguez called A Methodology For Studying Various Interpretations of the N,N-dimethyltryptamine-Induced Alternate Reality, which suggested among other things that you could prove DMT entities were real by taking the drug and then asking the entities you meet to factor large numbers which you were sure you couldn’t factor yourself. So to that end, could you do me a big favor and tell me the factors of 1,522,605,027, 922,533,360, 535,618,378, 132,637,429, 718,068,114, 961,380,688, 657,908,494, 580,122,963, 258,952,897, 654,000,350, 692,006,139?

gwern says:
I was a little curious about how such a prime experiment would go and how much it would cost. It looks like one could probably run an experiment with a somewhat OK chance at success for under \$1k.
We need to estimate the costs and probabilities of memorizing a suitable composite number, buying DMT, using DMT and getting the requisite machine-elf experience (far from guaranteed), being able to execute a preplanned action like asking about a prime, and remembering the answer.

1. The smallest RSA number not yet factored is 220 digits. The RSA numbers themselves are useless for this experiment because if one did get the right factors, because it’s so extraordinarily unlikely for machine-elves to really be an independent reality, a positive result would only prove that someone had stolen the RSA answers or hacked a computer or something along the lines. RSA-768 was factored in 2009 using ~2000 CPU-years, so we need a number much larger; since Google has several million CPUs we might want something substantially larger, at least 800 digits. We know from mnemonists that numbers that large can be routinely memorized, and an 800 digit decimal can be memorized in an hour. Chao Lu memorized 67k digits of Pi in 1 year. So the actual memorization time is not significant. How much training does it take to memorize 800 digits? I remember a famous example in WM research of how WM training does not necessarily transfer to anything, of a student taught to memorize digits, Ericsson & Chase’s whose digit span went from ~7 to ~80 after 230 hours of training; digit span is much more demanding than a one-off memorization. This does something similar using more like 80 hours of training. Foer’s _Moonwalking With Einstein: The Art and Science of Remembering Everything_ doesn’t cover much more than a year or two and fairly undemanding training regimen, and he performed well. So I’m going to guess that to memorize a number which would be truly impressive evidence (and not simply evidence for a prank or misdeeds by a hobbyist, RSA employee, Google, or the NSA) would require ~30h of practice.
2. some browsing of the DMT category on the current leading black-market suggests that 1g of DMT from a reputable seller costs ฿0.56 or ~\$130. The linked paper says smoking DMT for a full trip requires 50mg/0.05g so our \$130 buys ~19 doses.
3. The linked paper says that 20% of Strassman’s injected-DMT trips give a machine-elf experience; hence the 1g will give an average of ~3-4 machine-elfs and 19 trips almost guarantees at least 1 machine-elf assuming 20% success-rate (1-(1-0.2)^19 = 98%). Since the 20% figure comes from injected DMT and DMT of a controlled high quality, probably this is optimistic for anyone trying out smoking DMT at home, but let’s roll with it.
4. in a machine-elf experience, how often could we be lucid enough to wake up and ask the factoring question? No one’s mentioned trying so there’s no hard data, but we can borrow from a similar set of experiments in verifying altered states of consciousness, Laberge’s lucid dreaming experiments in which subjects had to exert control to wiggle their eyes in a fixed pattern. This study gives several flows from # of nights to # of verifications, which all are roughly 1/3 – 1/4; so given our estimated 3-4 machine-elfs, we might be able to ask 1 time. If the machine-elves are guaranteed to reply correctly, then that’s all we need.
5. at 30 hours of mnemonic labor valued at minimum wage of \$8 and \$130 for 19 doses, that gives us an estimate of \$370 in costs to ask an average of once; if we amortize the memorization costs some more by buying 2g, then we instead spend \$250 per factoring request for 2 tries; and so on down to a minimum cost of (130/19)*5 = \$34 per factoring request. To get n=10 requests, we’d need to spend a cool ((30*8) + 10*130)=\$1540.
6. power analysis for a question like this is tricky, since we only need one response with the *right* factors; probably what will happen is that the machine-elfs will not answer or any answer will be ‘forgotten’. You can estimate other stuff like how likely the elves are to respond given 10 questions and 0 responses (flat prior’s 95% CI: 0-28%), or apply decision-theory to decide when to stop trying (tricky, since any reasonable estimate of the probability of machine-elves will tell you that at \$35 a shot, you shouldn’t be trying at all).

Hence, you could get a few attempts at somewhere under \$1k, but exactly how much depends sensitively on what fraction of trips you get elves and how often you manage to ask them; the DMT itself doesn’t cost *that* much per dose (like ~\$7) but it’s the all the trips where you don’t get elves or you get elves but are too ecstatic to ask them anything which really kill you and drive up the price to \$34-\$250 per factoring request. Also, there’s a lot of uncertainty in all these estimates (who knows how much any of the quoted rates differ from person to person?).

I thought this might be a fun self-experiment to do, but looking at the numbers and the cost, it seems pretty discouraging.

Related Empirical Paradigms for Psychedelic Research:

1. LSD and Quantum Measurement (an experiment that was designed, coded up, and conducted to evaluate whether one can experience multiple Everett branches at once while on LSD).
2. How to Secretly Communicate with People on LSD (a method called Psychedelic Cryptography which uses the slower qualia decay factor induced by psychedelics, aka. “tracers”, in order to encode information in gifs that you can only decode if you are sufficiently high on a psychedelic).
3. Psychophysics for Psychedelic Research: Textures (an experimental method developed by Benjamin Bala based on the textural mongrel paradigm proposed by Eero Simoncelli and extended to provide insights into psychedelic visual perception. See: analysis).

## Algorithmic Reduction of Psychedelic States

Only when sexual choice favored the reportability of our subjective experiences- with the emergence of the mental clearing-house we call consciousness- did our strangely promiscuous introspection abilities emerge, such that we seem to have instant conscious access to such a range of impressions, ideas, and feelings. This may explain why philosophical writing about consciousness so often sounds like love poetry- philosophers of mind, like lovesick teenagers, dwell upon the redness of the rose, the emotional urgency of music, the soft warmth of skin, and the existential loneliness of the self. The philosophers wonder why such subjective experiences exist, given that they seem irrelevant to our survival prospects, while the lovesick teenagers know perfectly well that their romantic success depends, in part, on making a credible show of aesthetic sensitivity to their own conscious pleasures.

The Mating Mind: How Sexual Choice Shaped the Evolution of Human Nature (pg. 365) by Geoffrey F. Miller

## A Darwinian Set and Setting

According to The Mating Mind, human sexual selection favors particular fitness-indicating traits, both physical and mental. In the context of mental traits, we have verbal and introspective abilities, agreeableness, conscientiousness, openness to experience, low neuroticism and extroversion. No matter how verbally capable and introspective a given person is, unless that is balanced with some degree of agreeableness, conscientiousness, etc. the person will not be all that attractive. But, when all else is being held equal, stronger verbal and introspective abilities are favored. Teenagers, arguably, know this best of all: courtship is intensely verbal.

Our minds evolved in a Darwinian environment. If people like Miller are right in thinking that language evolved as a fitness indicator, we are right to expect that the way we think and verbalize is biased to be impressive to the members of the opposite sex during courtship. Powerful introspective abilities, as it were, can make one’s language seem deeper, more romantic, and even at an entirely different level than that of one’s peers. In this backdrop of sexual choices and judgements, it is not surprising that humans would develop ever-increasing verbal and introspective capacities. At some point everyday life could not present sufficient opportunities for people, especially males, to show off their own abilities. And as these abilities increased over time, culture was forced to invent handicaps so that people could display their top capabilities. Over time, elaborate and competitive handicaps were integrated into the culture. Even verbal and introspective abilities at the top of the scale can still be compared side by side by using carefully selected handicaps: for example, poetry is exactly that; rhyme, rhythm and meter make it easier for the best poets to show off their excellent abilities. The handicaps adjust to the maximum level of competence in the population.

The space of handicaps that are used to show off traits that are reliable indicators of fitness is very large. From Greek Symposiums to modern day Frat Parties, Western civilization has embraced a niche subculture that uses chemical handicaps as a means to display verbal, social and creative skills. If you can philosophize after drinking a gallon of wine, or stay capable of managing the playlist after 16 cheap cans of beer, you are showing off your biological robustness. Clearly, many of our ancestors were capable of impressing potential sexual mates with a mixture of booze, loud music and stunning philosophical conversations.

One could argue that psychedelics have come to disrupt our traditional games of handicaps. “Sure you can drink a bottle of tequila and sing in a band, but can you take three hits of acid and tell me what your experience reveals about the intrinsic nature of consciousness?” Psychedelics are, in a way, a cultural hyper-stimulus that presents the most difficult and interesting handicap currently in existence for verbal and introspective abilities.

Cultures can have an allergic reaction to the states of consciousness that these agents can disclose; people are afraid that psychedelic users will discover something that they themselves don’t know. Notably, psychedelicists have been both demonized and deified since the 60s. Sure, these researchers became extremely open minded, and in many ways weird. But, above all, they became extremely interesting people. And interesting people who challenge the current games of status can cause cultural allergic reactions.

Every acid head and psychedelic researcher has a pet theory of what these compounds are really doing in one’s mind. Many of these folk theories about the effects of psychedelics involve ontologies that currently have little scientific support (such as souls, thought fields, spirit worlds, archetypes, alien conspiracies, and so on). Although we cannot rule out explanations of this sort out of hand, the ontologies themselves are so abstract and poorly defined that we cannot accept them as useful forms of reductions. That said, their future versions will be more interesting. It is likely that committed, rational, spiritual psychedelic users will formalize models of this sort at some point. Rather than talking about a “spirit world,” they will talk about “mind-independent extra-dimensional space that consciousness can access in altered states” and then go on to define the differential equations that govern consciousness’s interactions with this space. When this happens, we will be in a much better position to assess the validity of these models, test the reality of those spaces, and perhaps even recruit the extra-dimensional inhabitants of these worlds for computational tasks.

Psychedelic experiences drastically increase people’s introspection, capacity for deep aesthetic appreciation, while at the same time increasing their ability to entertain unusual ideas. Insofar as the selection pressures of our introspective abilities have been heavily biased towards courtship ability, it is not surprising that people tend to immediately cast self-enhancing, life-affirming and magical narratives into their interpretations of their personal psychedelic experiences. After all, having a very interesting story to tell is highly praised during courtship. Are people’s psychedelic narratives a modern day form of the peacock’s tail? While psychedelic talk does not yet form part of any mainstream game of courtship, I envision this changing in the next decades. Undoubtedly, the most insightful, sound, and scientifically rigorous members of the Super-Shulgin Academy will attract attention, status, resources and… desirable mates.

## What is the deep structure of psychedelic experiences?

Psychedelics seem to have a generalized effect on one’s consciousness. At minimum, we could talk of experience amplification. Without delving into specifics, psychedelics introduce spontaneous activity into our consciousness that our mind is compelled to integrate somehow. Our state of consciousness changes dynamically as our mind adjusts itself to the incoming stimulation. The result is tightly dependent on the interplay between our brain anatomy, motivational system and the actual changes to the micro-structure of consciousness induced by LSD.

As John Lilly noted in light of his psychedelic experiences: “in the province of the mind, what one believes to be true is true or becomes true, within certain limits to be found experientially and experimentally. These limits are further beliefs to be transcended. In the mind, there are no limits…”.* While there are reasons not to take this literally, we have grounds for claiming that a large number of limits on our experience are placed there by our deeply held beliefs and attitudes. The space of possible LSD experiences that a single individual can experience is much larger than what said individual will typically be able to explore in practice. Many limits are imposed by his or her beliefs and background assumptions, rather than by physiology per se. Social cognition is a profound attractor in psychedelic experiences. “What will I say about this? What would this person think about this experience? etc.” are captivating thoughts. However, they occupy valuable mental space. And the thick mental judgements that people naturally focus on come with large conceptual and emotional baggage that taints the experience. Meditators, philosophers and scientists are more likely to set aside some time during their explorations to delve more deeply into what the energy introduced by LSD can produce in one’s consciousness.

After extreme training and tens (or hundreds) of trips, dedicated psychonauts will discover qualities that all of the trips share. Most people will likely experience a variant of Lilly’s realization that whatever you believe can be perceived as true during psychedelic experiences. Lilly emphasized the limitless quality of the mind, but one must wonder: If one can experience as true anything conceivable, are we not, then, limited by what we can conceive? No matter how much time one spends with an open mind waiting for new and interesting ideas to take shape, one cannot know the nature of what one has not yet even conceived of.

It may be true that we will always find fundamental limits that cannot be overcome. There are fundamental physiological constraints to the possible configurations of our consciousness, and arguably, chemical agents, while capable of expanding the space of possibilities, will not automatically give access to all possible states of consciousness. As future research is likely to show, 2C-B and LSD probably facilitate slightly different kinds of thoughts and experiences. Thus the limits of our mind are at least to a large extent the result of our physiology. Memes and meditation can only go so far.

In addition to physiological limits, the structure of the state-space of qualia is itself a constraint on what can and cannot be experienced. To the extent that psychedelic states enable the exploration of a larger space of possible experiences, we are more likely while on psychedelics to find states of consciousness that demonstrate fundamental limits imposed by the structure of the state-space of qualia. In normal everyday experience we can see that yellow and blue cannot be mixed (phenomenologically), while yellow and red can (and thus deliver orange). This being a constraint of the state-space of qualia itself is not at all evident, but it is a good candidate and many introspective individuals agree. On psychedelic states one can detect many other rules like that, except that they operate on much higher-dimensional and synesthetic spaces (E.g. “Some feelings of roughness and tinges of triangle orange can mix well, while some spiky mongrels and blue halos simply won’t touch no matter how much I try.” – 150 micrograms of LSD).

One of the objectives of Qualia Computing is to define the state space of possible experiences and the interdependencies between them. While normal everyday states of consciousness are important datapoints, I predict that the bulk of the most useful information will come from studying the behavior and mechanics of consciousness in radically altered states. To this end, however, we should focus on simple explanations that can be generalized to all psychedelic experiences.

## Starting Background Assumptions

For the purpose of this article I will assume that direct realism, in all of its guises, is wrong. That is, I will assume that any mind-independent object can only be experienced indirectly. What we experience is not the object (or beings) themselves, but a qualia-furnished representation entirely contained within one’s mind (this is often called the simulationist account of perception). Furthermore, I will also assume that the behavior of  the universe can be fully described with the Standard Model of physics (or a future version of it).

In what is to follow I will propose, as a first approximation, an algorithmic reduction of psychedelic states; I will propose a set of changes in our consciousness that (1) is as simple and assumption-free as possible, and (2) can be used to reconstruct as many psychedelic effects as possible.

## Two Kinds of Reduction

The word reduction in the context of philosophy of science has a lot of historical and conceptual baggage. In the context of this article, I will use the word in the following sense: We say that a property of a given phenomenon X reduces to Y if we can fully explain X’s property by referencing Y’s properties. X can be a physical phenomenon, a mathematical construct or even an experience. Y is an ontology with interaction rules, which allow the pieces of said ontology to interact with one another. We do not commit to the idea that Y itself needs to be the fundamental (or true) ontology of X. But we do want to make sure that Y is at least more fundamental than X in some appropriate sense. So what kind of ontologies can Y have? In the context of philosophy of mind, reductions usually attempt to account for not only the behavior of consciousness but also for its underlying nature. Thus, functionalism is both a reduction program as well as a philosophical take on what the mind fundamentally is.

Thankfully, we do not need to commit to any ontology in order to advance a particular style of reduction. Reductions are useful regardless: they reduce the amount of information needed to describe a phenomenon, and if accurate, they can also make useful predictions. Finally, these reductions can provide hints for how to bridge different areas of science; by identifying isomorphisms or even further reductions, entire fields can cross-pollinate once their respective reductions are compatible (such as biology and chemistry or chemistry and physics).

### Atomistic Reduction

For most intents and purposes, science relies on a particular kind of reduction that we can call atomistic reduction. This style of reduction focuses on explaining macroscopic phenomena by modeling it as the emergent structure of many particles interacting with one another at a much finer level of resolution. Even though this style of reduction is usually fruitful (e.g. thermodynamics), it can be counter-productive to assume in some situations. An extreme case would be the quantum computer. If states of superposition help a computer find an answer, it will be hard to explain the behavior of said superposition by postulating that it actually reduces to little particles interacting using simple rules. The model could in principle be worked out, but at the cost of very high complexity. It would be much easier to start with a quantum-mechanical ontology that allows the superposition of wavefunctions! Then what is left is to reduce the rest of the computer to quantum mechanics (which is possible, given that particle models and quantum mechanical models usually converge at the macroscopic limit).

It is tempting to try to reduce the properties of the mind (including psychedelic states) using an atomistic reduction. Unfortunately, the phenomenal binding problem adds a complication to this reduction. Rather than discussing (right now) whether an atomistic (and thus classical) account will ultimately be capable of modeling conscious experience, we will side-step this problem by using a different style of reduction. We will focus only on the algorithmic level of analysis.

### Algorithmic Reduction

Without assuming a fundamental ontology (atoms, fields, wavefunctions, etc.) we can still make a lot of progress. We can restrict ourselves to identifying what we call an algorithmic reduction: find a set of procedures, state-spaces, shapes and overall main effects out of which you can reconstruct as much of the observed behavior as possible.

In reality, every reduction is, at least in part, an algorithmic reduction. By specifying a particular ontology such as “particles”, we restrict the shape of our possible reductions. By keeping the reduction at the algorithmic level, we allow arbitrary ontologies to be the final explanations (then depending on actual empirical measurements). The main criteria for success still includes (1) the overall complexity of the model, and (2) the explanatory power of the model. In other words, how easily and precisely does the model reconstruct the behavior of our experiences?

## A Zoo of Psychedelic Effects

PsychonautWiki has a detailed and fascinating taxonomy of reported psychedelic visual effects. One could argue that all of these countless effects are completely unique. As a philosopher might put it, these effects may ultimately be qualitatively irreducible to one another. But what are the chances that a simple molecule would happen to trigger a whole zoo of unrelated effects? As a form of reduction, nothing is achieved by stating that every effect is its own unique phenomenon.

## Four Principal Operators: A Simple Algorithmic Model of Psychedelic States

In trying to account for the strange effects of psychedelics, we will aim to propose as few main effects as possible and then use these effects, and their interactions, to derive all of the remaining effects. By doing this, we will be algorithmically reducing the complex phenomena found in psychedelic states. In turn, this will allow us to increase our understanding of the source of information processing benefits provided by psychedelic states, and to derive new and exciting applications of such states. Additionally, by identifying a good algorithmic reduction, we might be able to refine the states themselves, to amplify their benefits while minimizing the drawbacks.

The model we will treat for now has four main effects, and with those four effects we will attempt to reconstruct the rest. These effects are:

1. control interruption
2. drifting
3. eidetic hallucinations/enhanced pattern recognition/apophenia
4. symmetry detection/symmetry propagation

Control interruption is the simplest and most universal psychedelic effect. It enables the buildup of qualia in one’s consciousness. People say that psychedelics are intense, deep, bright, etc. Every experience, whether a thought, a smell or an emotion, seems to be both stronger and longer-lasting on psychedelics.

Things seem more lively, and this is not because a switch is suddenly turned on and your experience of the current input is amplified. Rather, one seems to be experiencing a gentle overlap of many previous frames (and feature bundles) of one’s experience. In medium to high doses, this can give rise to solid frame stacking. In turn, the buildup of sensation creates complex patterns of interference:

In order for a perceptual system to transition from a linear to a nonlinear state, negative feedback control must be subverted. If control is entirely removed then perception becomes totally unconstrained, leaving a system that is quickly overloaded with too much information. If control is placed in a state where it is partially removed or in a toggled superposition where it is alternately in control and not in control over the period of a rapid oscillation, then the constraints of linear sensory throughput will bifurcate into a nonlinear spectrum of multi-stable output with signal complexity correlating to the functional interruption of control. Common entheogenic wisdom states that you must relinquish control and submit to the experience to get the most out of psychedelics. Holding onto control causes negative experiences and amplifies anxiety; letting go of control and embracing unconstrained perception is a central psychedelic tenet. This demonstrates that psychedelics directly subvert feedback control over linear perception to promote states of unconstrained consciousness.

– Control Interrupt Model of Psychedelic Action, PIT

Control interruption explains a large variety of effects, including the increase in the raw intensity (and amount) of experience, as well as the longer lasting positive afterimages (and thus tracers). Here we show a simple example of this effect. Consider the “original stimuli” to be what one experiences under a sober state. Likewise, consider the 9 squares to be different states of consciousness brought up by various psychotropic combinations.

Original

The 9 gifs you see above are simulations of control interruption using a simple feedback model (which we will describe in detail in a later article). The x-axis has different “echo strengths” while the y-axis has varying feedback strengths. These are two of the model parameters. Notice that the lower right corner is a credible rendition of something that people describe as moments of eternity. These are experiences where time seems to stop due to an over-saturation of regular and ordered qualia.

When considering the following effects, don’t forget that control interruption is also going on all the time. The stranger the psychedelic effect, the more intense it is.

Drifting is responsible for breathing walls, animated plants, feelings of boundary dissolution, merging and melting, and so on. Small amounts of drifting usually involve individual feature detachments from perceptual objects (such as the color and shape of a chair becoming dissociated). Medium amounts of drifting make textures flow constantly. If one’s experience was made of tiny magnetic gears that are usually aligned in a coherent way, drifting would result from increasing the overall energy of the system. Thus, the visual system is constantly descending to “more aligned local states” while incoming energy is constantly adding noise and destroying all of the alignment progress made.

Source: PsychonautWiki, Anonymous

A particularly salient aspect of drifting is that features and locally-bound fragments of experience can drift in any direction in 3D. Pieces of the wall don’t only drift left and right, but also forwards and backwards.

On high doses of psychedelics or synergistic combinations of dissociatives and psychedelics (e.g. LSD + nitrous, 2C-B + ketamine, etc.), drifting can become all-encompassing. A critical point is crossed when one loses the capacity to define a mainframe of experience (the dominating orientation-giving island of locally bound experience that we use as a reference point). When this happens, one feels like one cannot tell left from right, or up from down. One simply experiences a constant chaotic flow of experience. In some cases one can even spot interesting instabilities that resemble actual physical instabilities found in fluid mechanics (such as the Kelvin–Helmholtz instability).

Drifting does not occur in isolation, and its mechanics are dependent on the particular set and setting in which the psychedelic experience is developing. From a computational point of view, drifting can be useful because it allows a quick exploration of the state-space of possible local binding configurations between the phenomenal objects present in one’s experience. Indeed, not only does red fail to mix with green, but many of the synesthetic qualia varieties present in a scene with constant drifting will refuse to touch each other. Drifting feels like there is some sort of psychedelic energy (somewhat reminiscent of anxiety, but not restricted to body feelings) that overheats certain parts of one’s conscious experience, and in turn disassembles the local connections there.

Enhanced Pattern Recognition: This effect refers to the transient (but often powerful) lowering of the detection threshold for previously experienced patterns and known ontologies (e.g. animals, plants, people, etc.). Psychedelics, in other words, temporarily increase one’s degree of apophenia. Another name given to this effect is eidetic hallucinations. From a Bayesian point of view, the effect could be described thus: psychedelics intensify the effect of our priors. As explained in Getting Closer to Digital LSD, Google’s deep belief neural network inceptionist technique works by finding bundles of features that trigger high-level neurons (such as face-detectors, object-detectors, etc) at sub-threshold levels (e.g. “this almost looks like a frog”) and then modifying the picture so that the network more strongly detects those same high level features. This particular algorithm can be understood in terms of the pharmacological action of psychedelics: one can have breakthroughs of eidetic hallucinations by impairing the inhibitory control coming from the cortex.

In a sense we could say that while tracers are the result of “simple cell control interruption”, eidetic hallucinations are the result of “complex cell control interruption.” The former allows the build-up of colors, edges and simple shapes, while the latter amplifies the features that trigger high-level percepts such as faces and objects.

Enhanced Pattern Recognition / Eidetic Hallucinations / Visial Apophenia

The way one directs attention during a psychedelic trip influences the way eidetic hallucinations evolve over time. For this reason any psychedelic replication movie will probably require human input (in the form of eye-tracking) in order to incorporate human saliency preferences and interests into an evolving virtual psychedelic trip simulated with the Inceptionist Method.

Lower Symmetry Detection and Propagation Thresholds: Finally, this is perhaps the most interesting and scientifically salient effect of psychedelics. The first three effects are not particularly difficult to square with standard neuroscience. This fourth effect, while not incompatible with connectionist accounts, does suggest a series of research questions that may hint at an entirely new paradigm for understanding consciousness.

I have not seen anyone in the literature specifically identify this effect in all of its generality. The lowering of the symmetry detection threshold really has to be experienced to be believed. I claim that this effect manifests in all psychedelic experiences to a greater or lesser extent, and that many effects can in fact be explained by simply applying this effect iteratively.

Psychedelics make it easier to find similarities between any two given phenomenal objects. When applied to perception, this effect can be described as a lowering of the symmetry detection threshold. This effect is extremely general and symmetry should not be taken to exclusively refer to geometric symmetry.

How symmetries manifest depends on the set and setting. Researchers interested in verifying and exploring the quantitative and subjective properties of this effect will probably have to focus first on a narrow domain; the effect happens in all experiential modalities.

For now, let us focus on the case of visual experience. In this domain, the effect is what PsychonautWiki calls Symmetrical Texture Repetition:

Credit: Chelsea Morgan from PsychonautWiki and r/replications

Symmetry detection can be (and typically is) recursively applied to previously detected symmetry bundles. A given symmetry bundle is a set of n-dimensional symmetry planes (lines, hyperplanes, etc.) for which the qualities of the experience surrounding this bundle obey the symmetry constraints imposed by these planes. The planes can create mirror, rotational or oblique symmetry. Each symmetry bundle is capable of establishing a merging relationship with another symmetry bundle. These relationships are fleeting, but they influence the evolution of the relative position of each plane of symmetry. When x symmetry planes are in a merging relationship, one’s mind tries to re-arrange them (often using drifting) to create a symmetrical arrangement of these x symmetry planes. To do so, the mind detects one (or several) more symmetry planes, along which the previously-existing symmetry planes are made to conform, to organize in a symmetrical way (mirror, rotational, translational or otherwise). There is an irresistible subjective pull towards those higher levels of symmetry. The direction of highest symmetry and meta-symmetry feels blissful, interesting, mind-expanding, and awe-producing.

If one meditates in a sensorially-minimized room during a psychedelic experience while being aware that one’s symmetry detection threshold has been lowered by the substance, one can recursively re-apply this effect to produce all kinds of complex mathematical structures in one’s mind.

In the future, perhaps at a Super-Shulgin Academy, people will explore and compare the various states of consciousness that exhibit peak symmetry. These states would be the result of iteratively applying symmetry detection, amplification and re-arrangement. We would see fractals, tessellations, graphs and higher dimensional projections. Which one of these experiences contains the highest degree of inter-connectivity? And if psychedelic symmetry is somehow related to conscious bliss, which experience of symmetry is human peak bliss?

The pictures above all illustrate possible peak symmetry states one can achieve by combining psychedelics and meditation. The pictures illustrate only the core structure of symmetries that are present in these states of consciousness. What is being reflected is the very raw “feels” of each patch of your experiential field. Thus these pictures really miss the actual raw feelings of the whole experience. They do show, however, a rough outline of symmetrical relationships possible in one of these experiences.

Since control interruption is also co-occurrent with the psychedelic symmetry effect, previously-detected symmetries tend to linger for long periods of time. For this reason, the kinds of symmetries one can detect at a given point in time is a function of the symmetries that are currently being highlighted. And thanks to drifting and pattern recognition enhancement, there is some wiggle room for your mind to re-arrange the location of the symmetries experienced. The four effects together enable, at times, a smooth iterative integration of so many symmetries that one’s consciousness becomes symmetrically interconnected to an unbelievable degree.

What may innocently start as a simple two-sided mirror symmetry can end up producing complex arrangements of self-reflecting mirrors showing glimpses of higher and higher dimensional symmetries. Studying the mathematical properties of the allowed symmetries is a research project that has only just begun. I hope one day dedicated mathematicians describe in full the class of possible high-order symmetries that humans can experience in these states.

Anecdotally, each of the 17 possible wallpaper symmetry groups can be instantiated with this effect. In other words, psychedelic states lower the symmetry detection threshold for all of the mathematically available symmetrical tessellations.

All of the 17 2-dimensional wallpaper groups can be experienced with symmetry planes detected, amplified and re-arranged during a psychedelic experience.

Revising the symmetrical texture repetition of grass shown above, we can now discover that the picture displays the wallpaper symmetry found in the lower left circle above:

In very high doses, the symmetry completion is so strong that at any point one risks confusing left and right, and thus losing grasp of one’s orientation in space and time. Depersonalization is, at times, the result of the information that is lost when there is intense symmetry completion going on. One’s self-models become symmetrical too quickly, and one finds it hard to articulate a grounded point of view.

## The Micro-Structure of Consciousness

At Qualia Computing we explore models of consciousness that acknowledge the micro-structure of consciousness. Experiences are not just higher-order mental operations applied to propositional content. Rather, an instant of experience contains numerous low-level textural properties. This is true for every sensory modality, and I would argue, even for the what-its-likeness of thought itself. Even just thinking about a mathematical idea (ex. “the intersection of two arbitrary sets”) is done by interacting with a background of raw feels, and these raw feels determine our attitudes and interactions with the ideas we are trying to abstract (some people, for example, experience emotional distress when trying out mathematical problems, and this is not because certain mathematical spaces are inherently unpleasant or anxiety-inducing).

In the case of vision, the micro-structure of consciousness is capable of supporting at least the following low-level features: color, color gradients, points, edges, oriented movement, and acceleration. A full conversation about the range of visual features that we are capable of experiencing is a discussion for another time. But for the time being, it will suffice to point out that (static) models of peripheral vision only need 5 summary statistics. With only five summary stats you can create textures that a human will find impossible to distinguish in peripheral vision.

These so-called mongrels are textural metamers (equivalence classes of subjectively indistinguishable input patterns). The state-space of perceivable visual textures is the space of possible mongrels, and that is an example of the sort of micro-structure we are looking for. Unlike the cozy high-definition space inscribed in the fovea, most of the information found in our sensory modalities comes in the form of textures that are mappable to state-spaces of summary statistics.

Psychedelic symmetry detection and amplification operates on the inner structure of mongrels. The fact that the mongrels are the objects becoming symmetric is something that can elude introspection until someone points it out. It happens right in front of any tripper’s eyes and yet people don’t seem to report it very often (if at all). This may be a result of the fact that the fine-grained structure of consciousness is rarely a topic of conversation, and that we usually describe what we see in the fovea (unless we have no other option). Our words usually refer to whole percepts or, at best, the simplest raw values of experience (such as the hue of colors or the presence of edges). And yet, the structure of our mongrels is quite obvious once symmetry propagation has conformed a large patch of your experience to have a tessellated identical mongrel repeating across it.

## How Are these Components Related to Each Other?

The Kaleidoscopic technique to induce qualia annealing relies on a combination of drifting and symmetry detection in order to resolve implicit inconsistencies within one’s own memory gestalts. As we live and grow our experienced evidence base, we accumulate memories and impressions of many worldviews. Each worldview is, in a way, a response to all of the previous ones (or at least the memorable ones) and the current situation and the problems one is facing. Thanks to the four effects here described, a person can utilize a psychedelic state to increase the probability of the systematic co-occurrence of (usually) mutually-exclusive gestalts (worldviews) and thus enable their mutual awareness. And with mutual awareness, the symmetry detection and amplification effect creates (somehow forcefully) a unified phenomenal object that incorporates the inconsistent views into an unbiased (or less biased) point of view. One can achieve a higher order of memetic and affective integration.

## Psychedelics as Introspectoscopes**

Given the symmetry detection and amplification property of psychedelics, one can reasonably argue that psychedelic states may be able to reveal the properties of the micro-structure of consciousness. Timothy Leary, among others, described LSD as a sort of microscope for one’s psyche. The very word psychedelic means mind-manifest (the manifestation of one’s mind). Given the four components of these experiences, the fact that psychedelics work as some sort of microscope should not be surprising. Symmetry detection and control interruption multiply the amount of raw experience, while pattern recognition shows you what you are expecting (your priors become evident) and drifting makes the fleeting synesthetic effects malleable and easier to move around. People generally agree that psychedelics can show you subtle aspects of your own mind with stark clarity. But can they reveal the intrinsic properties of the nature of qualia at the most fundamental level?

The way to achieve this may be to create a fractal structure of symmetries in such a way that any tiny part of one’s experience can get reflected throughout the entirety of the phenomenal structure. One can then use eidetic hallucinations (or further symmetry detection) to focus and stabilize the fractal structure. Thus one would multiply the surface area of all of one’s attention into countless replicas of the micro-structure of a given part of one’s experience. A fractal kaleidoscopic mirror amplifier chamber is exactly what I imagine when I think about how to analyze the fine-grained structure of consciousness. And it so happens that meditation plus psychedelics can allow you to (fleetingly) build just that.

Psychedelic Introspectoscope (fractal kaleidoscope of generalized symmetries) to amplify arbitrary qualia values (such as particular emotions, phenomenal colors, synesthetic inter-junctions, etc.)

Any subtle qualia space can be multiplied countless times in such a way that all of one’s experience becomes a coherent interlocking structure that can be perceived all at once. If one wants to study, for example, the possible interactions between two hues of color, one can amplify the boundary between two regions that make the desired contrast of hues and make the entire fractal structure amplify this boundary hundreds of times.

Arguably, if one discovers that certain qualia values cannot be mixed in the introspectoscope (such as blue and yellow), one may still not know if these are fundamental constraints, or if they are the result of our connectome structure. If, on the other hand, two qualia values can mix in the introspectoscope, then we would know that they are not fundamentally mutually exclusive. Thus we would find out relational properties of the very state-space of qualia.

## Reducing All Effects

Can we derive all psychedelic effects using the four components discussed above? While this is not yet possible, I trust that further work will show how most of the weird (and weirder) effects of psychedelics may be reduced to relatively simple (but not always atomistic) algorithms applied to the micro-structure of consciousness. I anticipate that we will discover that high doses actually produce entirely new effects (for example, what happens on 400 micrograms of LSD often include qualitative jumps from what happens at 150 micrograms). To note, ontological qualia and other subtle aspects of consciousness may resist reduction for still many more decades to come.

**An Introspectoscope is a hypothetical apparatus that enables a person to study the deep structure of his or her own consciousness. The concept comes from a paper in the making by Andrew Y. Lee. Obviously this comes with significant challenges. Some challenges come from the fact that we are trying to analyze something very small, and other challenges come from the fact we are trying to analyze qualia. Additionally, there are unique challenges that come from analyzing microscopic qualia qua microscopic qualia. I suggest that we use methods that amplify the micro-structure by taking advantage of fractal states: recursive and scale-free symmetry planes can amplify anything minute to a prominent place in the entire consciousness.

## How to secretly communicate with people on LSD

About 18 months ago I had a really cool idea: What if we could communicate with people who are high on LSD in such a way that sober people can’t understand?* I call this idea psychedelic cryptography (PsyCrypto for short).

The GIFs above do just that: The left one is the “original” and it shows how you perceive it while sober. The GIF on the right shows what it’s like to see the GIF on the left after taking 100 micrograms of LSD. Notice anything different?

The first thing to note is that it is easier to see what letter is hidden here (C). On a closer inspection, you can also notice another amazing fact: It turns out that there are gaps between the vertical columns! This feature pops-up with clarity and is self-evident on the right GIF, and yet one needs to carefully observe the left GIF to notice that this is happening. That piece of information is not obvious when you are sober. Hence, while a sober person may infer what the hidden letter is, only a person on a psychedelic will see right away that there are gaps between the columns. Can you think of how to use this as a communication tool?

The approach shown above is only one of a plethora of ways of communicating with people on psychedelics. Here I will mention just a couple low-hanging fruits, give a few ideas for how to extend psychophysical research to build animations in a principled way, and discuss an awesome speculative application of this research.

Hopefully this article will spark interest and motivate both the psychedelic replication and the psychophysics community to come up with more innovative communication methods.

### Do psychedelics enhance performance?

Drug “education” emphasizes the functional, perceptual, cognitive and affective impairments caused by the acute and chronic use of psychedelic substances. Psychedelics impair reaction time, linear thought, verbal expression, and a large range of everyday activities. This much is clear. It is undeniable that not all tasks are suitable for psychedelic experiences: Filing taxes, giving lectures to large audiences, and passing the polygraph test may all be rather poor choices for psychedelic activities.

But impairment is not the whole story. It is obvious to anyone who has researched the matter that psychedelics have some peculiar mind-enhancing properties. Any decent scientific account of psychedelic states has to provide information about the ways in which this particular state of consciousness confers genuine advantages.

And a great scientific account will explain why these particular trade-offs exist, and how we can best use them to (1) understand the mind, (2) achieve our human potential, and (3) address mental illness in a meaningful way.

Harman & Fadiman found a very large performance enhancement in the Witkins Embedded Figures test upon the administration of 100µg of LSD or 200mg of Mescaline. That is one of the most remarkable results of their study, which of course is not to diminish the relevance of their results concerning the rate of outstanding scientific discoveries. Unfortunately, the absence of drug-free controls in that study makes it less useful for convincing skeptics. When the study is replicated, it would be ideal to make it double-blind and not only include drug-free placebo controls, but also use an active performance-enhancing placebo, such as amphetamine.

Likewise, it is now clear that the self-insight concerning difficult emotional subjects can be radically amplified during therapeutic psychedelic sessions.  How and why this happens is still a rather difficult mystery.

Finally, we are currently experiencing a memetic explosion with regards to the use of micro-doses. Although we don’t yet have formal double-blind placebo-controlled research on the benefits of micro-dosing LSD, the wealth of anecdotal evidence is too large to ignore. For LSD, a micro-dose is defined as a dose in the 10-20 microgram range. The awesome Gwern is, to my knowledge, one of the few biohackers to have run a placebo-controlled experiment on himself. Although he found no positive effects, I suspect that is largely due to the sort of activities that he cares about. A more noticeable enhancement would be observed on artists, writers and possibly mathematicians. It is genuinely exciting that there is a new wave of attention to this particular application of psychedelics: General, all-purpose life-enhancement.

### A Fantastic Speculative Application

If psychedelic states of consciousness provide some sort of information-processing advantage over sober states, this advantage may be possible to exploit for secret communication. Conversely, if there is any information-delivery method that only people on psychedelics can understand, it follows that psychedelic states have distinct information-processing advantages over sober states. From a purely PR point of view, obtaining a portfolio of methods to secretly transmit information to high-people will do a lot of beneficial work in showing the potential benefits of psychedelics. This is partly what motivates my research.

Even more awesome is the idea that this technology can lead to the creation of a video-game that only people on psychedelics can understand and play. For a sober person the game would look like an incomprehensible bundle of dots, edges, colors, sounds, etc. But a person sufficiently zonked would perceive crystal-clear images and easy-to-infer objectives. Only a sufficient amount of LSD would allow you to score a single point in this game.

### Low Hanging Fruit

The simplest method is to take advantage of the longer-lasting after-images experienced under the influence. This happens to be one of the most robust effects that psychedelics have, and there seems to be a very clear dose-dependent curve in the intensity of these lingering phosphenes. Neurologically, this is explained by the Control Interrupt Model of Psychedelic Action, which can be summarized as follows: Our cortex’s main role is to provide inhibitory control on thalamic activity. The serotonergic activity of psychedelics blocks this control signal, and thus prevents the swift extinction of qualia once the triggering stimuli (whether internal or external) is removed.**

Credit: Chelsea Morgan. Source.

The basic idea for using tracers to communicate information is to provide, little-by-little, pieces of information that can be assembled into a coherent whole only if you use lingering after-images as building blocks.

### Psychophysics for Psychedelic Research: Afterimages/Tracers

In order to find the right parameters to make awesome visualizations that can only be interpreted during psychedelic experiences, we will need to do a lot of trial-and-error, and ideally, build quality psychophysical tools. The following are some of the most important questions that we need to answer before we can go wild and build the psychedelic video-game:

1. What is the dose-dependent decay function of tracers’ brightness?
2. What is the additive function? Do similar colors average out? Do opposite colors cancel out?
3. What is the range of features that remain in one’s experiential field? Is this dose-dependent?
4. Do lingering features interact with one another? Do they achieve after-the-fact local phenomenal binding?
5. What is the role of synesthesia in tracers?

To elaborate a little: The first question is about the rate of decay of phosphenes as a function of the dose and the time since the presentation of the stimuli. The GIF at the top of this page assumes an exponential gamma-corrected decay function.

The second question goes a little deeper, and it inquires about the way in which successive after-images of simple features (such as color and brightness) interact. If you first show a red square followed by a yellow one, do you then experience two overlapping but unblended colors? Or do you experience the average of the two (a hue of orange)? (If you know the answer from first-hand experience, please comment below!)

According to abundant anecdotal evidence (erowid, PsychonautWiki, r/psychonaut, etc.) the kind of perceptual objects that linger in one’s experiential field is dose-dependent. On small doses only colors and edges linger, while on higher doses you may experience emotions, faces, abstract concepts and even ontological qualia for many more seconds than normal. But what is the precise equation that describes this?

The fourth question is getting into more serious and difficult-to-research territory. Namely, we would want to know how different features interact with one another once they are lingering in one’s experiential field. If you first look at the blue sky and then look at a white cube, do you perceive a blue cube? More stunningly: If you think about the concept of recursion and then look at a tree, do you see recursion in the tree? (anecdotally, this definitely happens). The amazing thing about this particular question is that it may get at the very reason why consciousness was recruited by natural selection for information-processing purposes: There are some qualia that can be locally bound and some that can’t. This determines the range of constraints with which our mind implements constraint satisfaction solvers. But that’s a story for another post.

Finally, studying synesthesia during psychedelic experiences will almost certainly require the combination of neuro-imaging (such as fMRI) and quality psychophysics. I will explore this question further at a later time.

In order to answer most of the above questions, we can use the following paradigm: In order to test a theory you will want to (1) create interesting animations that produce particular effects, (2) create simulations of how these animations should look like under psychedelic vision, and (3) ask participants to rate the degree of similarity between the actual and predicted experiences.

For example, the GIFs below illustrate how an image might be seen if after-images are additive in nature. In other words, if you do experience orange when you flash red and yellow in quick succession, we can predict that the image on the left would be seen as the image on the right while on LSD. Is this so? I don’t know! Let me know if you happen to try it out.

Answering these questions using this and other paradigms will be very valuable to neuroscience.***

### Textures (Once Again)

In Psychophysics for Psychedelic Research: Textures we discussed how we can use psychophysical tools and computational models in order to measure deficits and enhancements in our visual pattern recognition ability while under the influence of LSD. This is done by measuring the size of the Just Noticeable Differences (JND) for each of the summary statistics our visual system can recognize in peripheral vision. I have yet to collect real data from people under the influence, but thankfully the paradigm is already fleshed out. (Dear psychophysical researcher reading this blog, please feel free to use this approach!). Presumably both textures and after-images can be used to encode information that only high people can read.

A proof of concept for how to do this would be to encode information in binary code: Take a set of summary statistics that high people are good at distinguishing (and sober people confuse). Then show pairs of textures, one on the left and one on the right, so that the texture on the right has either the same or different summary statistics as the one on the left. If the textures are different then that encodes a 0. And if they are the same, that encodes a 1. Make sure that this particular summary statistic difference is only noticeable by people on psychedelics and you will have a state-dependent visual binary encryption!

Since you can communicate anything using a binary sequence, you can use this to provide any information you may want. But will your zonked friends be able to string together 1024 1s and 0s in order to decode a verbal message? Unlikely.

As the sole way of communicating information, textures are an unlikely candidate. But they may fit well as a component of a complex array of stimuli. If we can answer questions (3) and (4) we may be able to flash textures in sequence in such a way that their summary statistics are combined. While a pair of textures may not provide a lot of information, a sequence of them may overlap in such a way that high-level features begin to appear.

Hence, maybe we can build a sequence of textures that will make a person on LSD experience a particular face, or a dog. The sober person will remain clueless, though, since the consecutive textures fail to become integrated into a coherent percept.

### Using Text

According to Shulgin there was a study conducted in the 60s that showed that people on psilocybin can read a text with fewer letters. What does this mean? Take a random text like a children’s story. Then remove X% of letters from it at random (substituting them by an underscore to show that a letter is missing).

Every person has a comprehension threshold: A 55-percenter would only be able to read texts that have 55% or more of their letters remaining. If that person takes psilocybin, then the comprehension threshold may drop to, say, 44%. This test should be particularly easy to replicate since it does not require any sort of image processing. Would you be interested in building an online test that determines your comprehension threshold? If you do, make sure to ask “are you on a psychedelic currently?” and collect the data!

Perhaps this generalizes to other areas of verbal comprehension. For instance, can you understand spoken words with more syllables taken out? What about sign language?

### Inspired Work

These are just a few promising approaches. I am confident that by opening this idea up to the broader academic and psychedelic community a lot more ideas will blossom. If you were inspired by this article to build your own psychophysical toolkit, make sure to let me know in the comments below. And remember: I’m always looking for collaborators. 🙂

* LSD here is a shorthand for psychedelics in general.

** Control Interrupt Model of Psychedelic Action: In his awesome book called Psychedelic Information Theory, James Kent argues that the visual and cognitive components of psychedelic experiences can be explained as the effect of subtle disruptions to the inhibitory control cycle of perception. He calls this theory the Control Interrupt Model of Psychedelic Action. The basic idea is that in order for our experience of the world to be linear and stable there must be mechanisms in place that regulate the overall loop of consciousness. In other words, when we open our eyes, the image in out visual field does not become arbitrarily brighter over time. Nor is it the case that our visual field gets as bright as it can if you give it enough time. Rather, we have in place a negative feedback mechanism involving lateral inhibition and inhibitory projections from the cortex to the thalamus that regulates the brightness of our experience.

This inhibitory control mechanism occurs a discrete number of times per second. Therefore “control interruption” caused by psychedelics, in this model, is conceived as a periodical failure of inhibitory control that allows aspects of one’s experience to be sustained for longer than usual. The frequency of control interruption is specific to the psychedelic used. As the article conjures, salvia and nitrous oxide produce control interruption at a frequency of 8-11 and 12-15 Hz, respectively. On the other hand, DMT disrupts control at a much higher frequency (24-30+ Hz). This control interrupt creates “a standing hallucinogenic interference pattern in the consciousness of the subject”.

*** As argued by Julien Dubois  and Rufin VanRullen in “Visual Trails: Do the Doors of Perception Open Periodically?” tracers may be very significant when it comes to reverse-engineering the human visual system. How many frames per second do we experience? How long do the images last in the visual field? Does this effect generalize to high-level features, or is it specific to colors and edges? Thus, building psychedelic communication tools would be of great value to neuroscience.

## Psychophysics for Psychedelic Research: Textures

In this post I will provide an account of my personal research project to understand the algorithms that underly human visual pattern-recognition. This project is multidisciplinary in nature, combining paradigms from three fields: (1) the analysis and synthesis of textures, (2) psychophysics and (3) psychedelic research. I will explain in detail how these areas can synergistically help us understand the computational properties of consciousness. In the process of doing so I will describe some of the work I have done in this direction.

tl;dr: With texture synthesis algorithms we can control the statistical features present in textures. By using an odd-one-out paradigm where participants have to find the “different texture” we can identify the statistical signatures of the visual patterns people can perceive. Collecting these signatures under various states of consciousness will reveal the information processing limitations of visual experience. It may turn out that some patterns can only be seen on LSD (psilocybin, mescaline, etc.), and this information will inform a general theory of vision’s algorithms, expanding the scope we have studied so far and suggesting relevant applications of psychedelic consciousness.

Introduction to Spatial Patterns

The world is patterned. In fact, it is so patterned, that it is difficult to identify natural surfaces that have no perceptible regularities. The grass, the trunk of trees, the surface of rocky mountains, the dancing and dissolving of the clouds. All of these natural scenes are full of regularities. For hundreds of millions of years animals on this planet have existed in an environment where regularities are not inconsequential: being able to use or detect camouflage is a matter of life or death to some species. The insects who hide between the leaves pretending to be part of the scene are in an evolutionary arms race against predators and their sensory apparatus. (Here is a neat collection of insect camouflage). Arguably, there are strong evolutionary selection pressures that push predator’s visual system into adapting to recognize the differences between the scene’s visual statistics and the prey’s body appearance.

Other widespread examples for the evolutionary relevance of pattern recognition can be found: Birds may take advantage of the look of cloud formations to determine if they should fly or find refuge, herbivores may seek only plants with specific visual properties to avoid poisonous lookalikes and parasites, the health of potential mates can be assessed by the uniformity of their fur patterns. You get the idea.

Not surprisingly, you today can look at a plain rock and see a lot of visual properties pointed out by patterns you can perceive. Unfortunately, something has kept us quiet about this aspect of our perception: most of these properties are hard to verbalize. Often, you will be able to tell apart two kinds of rocks by grasping the subtle visual differences between them, but at the same time still be unable to explain what makes them different.

What exactly is going on in your mind/brain when you are recognizing characteristic features in textures? We don’t know how the information is processed, why we perceive the features we perceive, or even how the various features are put together in a unified (or semi-unified) conscious representation. The hints we do have, however, are precious.

Receptive Fields

A big hint we can build on is that many neurons in the primary visual cortex (of cats, monkeys, and probably all mammals) respond to visual stimuli in specific areas of the visual field. For a given neuron, the shape of this region is an instance of a well-studied canonical function, as shown in the images below. The area of the visual field a neuron best responds to (by becoming excited or inhibited) is called its receptive field, and the canonical function is the Gabor filter.

As far back as the early 60s, research has been conducted to map the receptive fields of neurons by inserting electrodes into the brain of animals and presenting them with visual input of lines and shapes. In 1961 D. H. Hubel and T. N. Wiesel showed for the first time that neurons in a cat’s cortex and lateral geniculate have receptive fields that look like this:

The crosses indicate regions in which stimuli excites the neuron’s activity while the triangles represent regions in which stimuli inhibits the neuron’s activity. Due to the arrangement of these excitatory and inhibitory regions, these neurons functionally work as edge-detectors. A computer rendering of these receptive fields looks like this:

Since then a tremendous amount of research has repeatedly confirmed the existence of such neurons, and also uncovered a large number of more complex receptive fields. Some neurons even respond specifically to abstract concepts and high-level constructs. More recently, the simple receptive fields shown above have been modeled as Gabor filters in quantitative simulations. This, in turn, has been successfully used to build brain activity decoders that reconstruct the image that a person is seeing by assuming that the activity of fMRI’s voxels approximately matches the added activity of neurons with Gabor receptive fields (see: Identifying natural images from human brain activity by Kay, Naselaris, Prenger and Gallant).

Thus we can say that we know that a large number of neurons in our visual cortex have Gabor receptive fields and that the collective activity of these neurons contains enough information to at least approximately reconstruct the image a person is seeing (well enough to identify it from a pool of candidates).

We can’t jump from these findings alone to a global theory of visual processing. That is, without also considering what people actually experience. It may turn out, for example, that the activity of the visual cortex contains a lot of information that can be decoded using machine learning techniques applied to fMRI’s voxel brightness. And yet, simultaneously, it could be that people do not consciously represent all of the {decodable} information.

Likewise, a priori we cannot rule out the possibility that some of the information we consciously experience is not actually decodable using brain activity alone. (A quick remark: this may be the case even if one assumes physicalism. Why this is the case will be explained in a future article).

To illustrate this example we can consider the information available in the retina and before it. The light that reaches the outer surface of our eyes contains all of the information available to our mind/brain to instantiate our visual experience. Yet, there is a lot of information there that is ultimately irrelevant for our conscious experience. For instance, there is infrared and ultraviolet light, as well as light that does not make it to our retina, light that fails to elicit an action potential, and so on. If you can discriminate between a really hot and a cold metal using the infrared signature of the light that reaches the eye, you have certainly not shown that we perceive infrared light or that we use it to make distinctions. It merely means that such information is sufficient. We wouldn’t yet know that we actually use it or that it shapes our experience.

But how exactly do we develop an experiment to infer the statistical properties represented by our experience? Here is where the analysis and synthesis of textures becomes relevant.

Analysis and Synthesis of Textures

The idea of using oriented Gabor-like filters (also known as “steerable pyramids”) to analyze and synthesize visual textures was, as far as I know, first proposed by David J. Heeger and James R. Bergen in “Pyramid-based texture analysis/synthesis.” Texture analysis in this context means the use of algorithms to characterize the properties of textures to capture what makes them unique. In turn, texture synthesis, refers to the application of texture analysis to produce an arbitrarily large patch of a synthesized (synthetic) texture so that the synthetic and original texture are as indistinguishable as possible. Of course whether something is “indistinguishable” or not is to a certain degree subjective. Here the criteria for indistinguishability between the original and the synthetic textures is whether a person comparing them side by side could confuse them. In the following section I address how to operationalize and formalize the indistinguishability between patterns using psychophysics.

This particular texture synthesis algorithm works by forcing a white noise image (of any size) to conform to the statistics obtained in the texture analysis step. This is done iteratively, matching the histograms of the synthesized canvas to the various statistics computed from the original texture. I recommend reading the paper to gain a better grasp of what the algorithm does, and to see some stunning examples of the output of this algorithm.

The use of steerable pyramids was later refined by Portilla & Simoncelli‘s texture synthesis algorithm, which currently plays an important role in my research. This algorithm extends Heeger et al. by including additional statistics to enforce, which are (roughly) computed by measuring the autocorrelation between the various components of the steerable pyramid texture representation. Below you can see two pairs of pictures (two originals and their corresponding same-sized synthetics versions), that I recreated using Potilla & Simoncelli’s matlab code:

As you can see, the original and synthetic images are fairly similar to each other. Close inspection is sufficient to notice deadly differences. If you only use your peripheral vision it is very challenging to see major differences.

Psychophysics

Psychophysics is the study of the relationship between physical stimuli and experience (and often behavior). Thanks to psychophysics we now have:

1. A good approximate map of the phenomenal space of color (CIELAB)
2. A strong grasp of the nature of color metamers, which in turn underlies all of our color display technologies.
3. The ability to predict the subjective intensity of the experience elicited by stimuli as a function of the energy of the stimulus (see Weber–Fechner law).

A very powerful idea in psychophysics is the use of just-noticeable differences (JND): Carry a bucket of water. How much water should I add to it so that you are capable of perceiving a difference in the weight of the bucket? Look at a pair of identical light sources. How much blue (in this case, the specific and pure frequency of light that elicits the blue qualia) can I add to one of the lights before you perceive them as being differently colored? Pinch your skin with two needles. How far can their pointy part be before you perceive two needles rather than one?

In these cases, though, there is a natural (and sometimes unique) dimension along which the stimuli can be varied in order to compute the JND. What about visual textures? Here the problem becomes non-trivial. In what way should we change a texture? And how should we accomplish such? There is no clear and obvious scale for describing textures. So what can we do?

Psychophysics of Textures (Take 1):

Before learning about steerable pyramid representations of textures I developed a variety of psychophysical tests to identify the JND between textures. I did this by changing the value of parameters of images with ground-truth statistical properties. If you are curious, feel free to try the first iteration of my experimental paradigm. (Note: I am not collecting data from that experiment, so you should not expect to contribute to science by finishing the task. That said, you can have fun, and a pop-up window with your raw score will appear when you finish both tasks). The average accuracy of the texture discrimination part is about 13/21 (with a chance performance of 3/21), while the performance in the numerical pattern completion task is about 3/5 (with a near 0/5 expected performance when answering randomly).

One of the statistical properties I was changing in the stimuli was the variance of a 2D Gaussian process I implemented with a python script and the magic of Gibbs sampling. Thus, a subset of the images I tested were complete noise except for the first order statistic of local autocorrelation created by the differently parametrized Gaussian process. Here are a couple of examples of such patterns:

These pictures, it turns out, are relatively easy to tell apart when the parameters are sufficiently different. There is a threshold of similarity in the variance parameter after which people cannot distinguish between them. A nice property of this particular method is that you can be sure that if people do recognize the differences, it is because they are somehow extracting features that are a direct consequence of a different value for the variance parameter.

The textures above instantiate the simplest statistical differences that can be created, after the mean and standard deviation of the pixel values. To explore more broadly a wider variety of patterns, I also created textures with fairly complex parametrizable Turing machines (TMs). This, unfortunately, does not lend itself to a clear analysis. In brief, this is because changing a single parameter of such TMs can produce profoundly different results with unclear ground-truth properties:

What information could I gain from the fact that changing x, y or z parameter in a Turing machine by a, b, or c amounts enables accurate discrimination between textures? In principle one could try to explain the performance obtained by making an a posteriori analysis of the correlation between a variety of statistics measured in the textures. But since the textures were not created to have specific statistical properties, the distribution of such properties will not be ideal to find JNDs or discriminability thresholds.

Even if you do this, you will still have more problems. The images are more different (and different in more ways) than what your texture analysis algorithm is capable of detecting. Thus the reason why people can tell these textures apart is not possible to extract from the statistical differences you measure between them. At least not without being extremely lucky and hitting the visual features that matter.

And here, I was stuck several months.

Psychophysics of Textures (Take 2):

After learning about the steerable pyramid model, the work of Eero Simoncelli and the state of the art in fMRI decoding of visual experience, I decided to shift gears and approach the problem with some of the best of the tools created so far.

It turns out that the concept of texture metamers had been developed to describe the perceptual indistinguishability between textures in peripheral vision. Taken from here, the following two images are texture metamers. The images look the same when you center your vision in either of the central red dots. Close foveal inspection of the image, however, will reveal that these are very different pictures!

A specific study caught my attention, and I decided to replicate it as a final class project. Specifically, Texture synthesis and perception: Using computational models to study texture representations in the human visual system by Benjamin J. Balas. I attempted to replicate some of the main results of that study using Mechanical Turk. A full account of the experimental procedure, results and discussion can be found in this wiki here, (written for Stanford’s Psych 221 – Applied Vision Systems). To see the actual experiment performed, try it out here: Replication (and the github repository).

The main idea goes as follows: If textural metamerism can be verified using a given texture synthesis algorithm, then we can be reasonably certain that the algorithm is capturing a large component of what makes textures different from a human point of view. In particular, Benjamin Balas noted that Prtilla & Simoncelli’s algorithm could be “lesioned” by failing to enforce specific statistical features obtained in the texture analysis step. In this way, one can purposefully fail to match specific statistical features between the original and synthesized texture, and then measure how this partial texture synthesis algorithm affects the performance of texture discrimination in humans.

For illustration, here is a set of possible texture synthesis “lesions:”

The operationalization of the experiment used an “odd one out” paradigm: in each trial, three images are presented for 250 milliseconds at 3.5 degrees of vision from the fovea (each image being 2 degree wide in diameter). Two of the three images will come from the same group (ex. two original textures, two marginals removed, etc.). The remaining image is the odd one out. The study measures how often participants detect the odd one out depending on the statistical feature that is not enforced in the synthesized textures (for a more in-depth description: wiki).

Overall the performance of the participants in my replication was much closer to chance than Balas’ results. That siad, qualitatively the replication was successful. Removing the marginals is the lesion that increases performance the most. Magnitude correlation comes in second place. All of the other conditions are at chance level (as far as my sample n=60 with 60 trials each can discern).

As you can see, this specific paradigm is now much more robust than before. And the paradigm is not hard to translate for application in psychedelic research. Unfortunately, Portilla & Simoncelli’s algorithm only creates texture metamers for the peripheral vision in pre-attentive conditions. Upon central inspection, nearly every synthesized texture is at least somewhat distinguishable from the original texture.

The more interesting problem, as I see it, is found in our capacity to see differences between textures upon close and careful examination. This, I think, addresses more directly the subject of this blog. Namely, what is the information that we can represent and distinguish at the (resolution) peak of human consciousness? I imagine that there must be statistical features that we simply do not perceive even when we are looking at them directly and using all of our attention. What is the set of properties that our everyday consciousness can represent centrally?

Psychedelic Research

To understand how a machine works, it helps to know what happens when you break it. You can’t ignore the extra settings and claim you’ve got a theory of everything. Would we be satisfied with the work of neuroscientists and psychologists if they only studied people who are colorblind? They may claim that they have “the essentials” of vision. That color is just a perturbation in the “optimal basics.” And yet, today we know that color plays a relevant computational role in visual discrimination. Suffices to mention that people with grapheme-color synesthesia have an improved performance in “odd one out” identification tests like this (find all the 2’s as fast as possible):

This is because fast low-level association between graphemes and colors help them see clearly and quickly the graphemes that are different. Likewise, every variety of conscious experience may potentially play a computationally relevant role in specific situations. A priori, no variety of consciousness can be dismissed as irrelevant.

Thus, to understand, model and engineer consciousness, we should not prematurely close varieties of consciousness to study. Not only would that prevent us from finding out about consciousness more generally. It may also conspire that we will never understand even what we do decide to study, simply because key pieces of the puzzle are found elsewhere.

Another point is that even though sober human vision is a special case of general vision (and general vision-like qualia), general vision may still be relatively small. The general principles and conditions for vision to work in the first place may be somewhat restricted. Unless the elements are placed just right, the visual system breaks down, at least when it comes to fulfilling a computationally meaningful purpose. Thus, psychedelic research of visual experience may help us quickly distinguish what is essential from what is incidental in vision.

By introducing a psychedelic substance into the nervous system of a person, a remarkable set of visual effects are produced. To anyone interested in seeing a good representation of the way in which various psychedelics affect a person’s conscious experience I highly recommend seeing Disregard Everything I Say’s entry on the visual components of a psychedelic experience (while you are at it, I recommend also checking out his/her post on the corresponding cognitive components of a psychedelic experience). These images will provide a great intuition pump about the kind of beast we are facing to anyone who is psychedelic-drug-naïve (and hopefully inspire a sense of “WOW, you can represent some of what you experience after all!” in those who are less psychedelic-drug-naïve).

Given the same visual input presented to a particular person, a psychedelic substance will in many cases drastically shift the interpretation of such stimuli. Both the interpretation (specifically, what an image is about) and how the image looks and feels tend to vary in synchrony. Likewise, people feel more able to see personal issues in a new light by interpreting them with new schemas and from a different level of awareness. Personally, I suspect that there is a strong and measurable connection between the fluidity of interpretation of personal issues, and the fluidity of interpretation of visual experience. Perhaps all phenomenal constructs are affected in a similar way: By breaking down the previously enforced patterns of thought and perception and opening the way to seeing things differently.

All of the above, while probably true, is still far too vague for a scientific theory. Given our set of tools, and experimental paradigms, to me it makes a lot of sense to start studying the effects of psychedelics in terms of texture perception. As we develop more and better texture analysis/synthesis algorithms, we will acquire a larger repertoire of mathematical properties that describe what is seen during a psychedelic experience. My hope is that we will someday know exactly how to simulate a visual trip.

Evolutionary selection pressures on perceptual systems do not guarantee that information processing tasks will be solved optimally. In fact, “optimal” only really makes sense in relation to some metric you choose. In some way, everything created by the evolutionary process will be optimal in the sense that is produces the local maximization of inclusive fitness (admittedly it’s more complicated than that). But this is just a tautological notion: Sure evolution is optimal at doing what evolution does. Likewise, rocks have found the optimal way to be themselves. Our visual system works optimally, if you define optimal in terms of 20/20 every-day visual experience.

Instead, it makes more sense if we focus on the specific computational trade-offs that various resource allocation methods and designs provide. We can certainly predict that the particular set of algorithms that our visual system employs to detect visual patterns will satisfy some properties. For instance, they will be good as survival tools in the African Savanna. But just as for our hardwired tastes in food (and our default emotional palette), survival value in the ancestral environment is not necessarily what we currently need or want. And just as it would make sense to modify what we enjoy eating the most (moving away from sugar) to adapt to the current post-industrial environment, it may also turn out to be the case that our visual system is miscalibrated for the tasks we want to solve, and the joys and meaning we would like to experience today.

Psychedelics change our visual system in many ways, some of them more predictable than others. Some people report that small doses of psychedelics increase one’s overall visual acuity (this has yet to be verified empirically). Although counter-intuitive at first, this cannot be ruled out, again, simply because of evolution does not rule out things of this nature. After all, one of the main constraints placed on animals in natural environments is caloric consumption. If a higher visual acuity is possible with your current brain, the marginal benefit such acuity adds may still not surpass the marginal loss of calories that result from excessive brain activity.

Additionally, while higher doses of psychedelics tend to impair many aspects of visual perception (with people on Erowid explaining how extreme tracers can make it hard to walk), low and moderate doses do not have simple one-sided effects even when it comes to accurately representing the world around us. Rather than simply breaking up the pattern recognition capabilities of the visual system, small and moderate doses seem to also open up additional kinds of patterns up for inspection. Pareidolia, for example, is greatly enhanced. Thus, “connecting dots and edges” to match the outline of higher-level phenomenologies (like faces in the mud) happens with ease. Whether this is good or bad depends on the context, and the specific task one is trying to solve.

Perceptual Enhancement via Psychedelics

We have theoretical and anecdotal reasons to suspect psychedelics may turn out to be performance-enhancing in certain visual tasks. We also already have quantitative evidence that this is the case. In the 60s Harman and Fadiman conducted a study about the creativity and problem-solving properties of psychedelics. The study included a paper and pencil component in which the following tests were administered: Purdue Creativity, the Miller Object Visualization, and the Witkin Embedded Figures. The authors conclude that “[m]ost apparent were enhanced abilities to recognize patterns, to minimize and isolate visual distractions, and to maintain visual memory in spite of confusing changes of form and color.” Specifically, the Witkin test is singled as a test in which the ingestion of mescaline produced a consistent performance improvement (p<0.01).

Example of a Witkin Embedded Figure. You have to determine if the left shape is in the right figure within 30 seconds.

In personal communication Fadiman has said that these tests were mostly a waste of time: They used one valuable hour of the problem-solving session. I disagree. After 50 years, those results are incredibly valuable to me and the next wave of computational researchers of the mind. I think that the recorded performance enhancement is a key piece of information. We certainly do not expect an enhancement on all areas of cognition and perception (we know, for example, that reaction time and verbal fluency are impaired). Identifying the kinds of tasks that do receive a boost can inform future research. In future posts I will explain my theory for why Witkin Embedded Figures test was particularly benefited from a psychedelic, and how we can create a new test that takes this into account.

New Possible Protocols for Psychedelic Research

Nowadays it is very hard to obtain the required permits and affiliations to conduct academic research on psychedelic consciousness. The key rate-limiting factor is the restrictions that apply to research with controlled substances. Thankfully, we do happen to be living at the beginning of a psychedelic renaissance. It is not hard to imagine that as psychedelics start to (re-)enter the mainstream in psychiatry, a large number of clinical trials will be conducted. In principle, a collaboration can be accorded between a psychophysics lab and a clinical research group to conduct psychophysical assessments during the course of treatment.

Clinical trials for medical applications of psychedelics, though, are still limited in scope and focus. Eventually the medical applications of psychedelics will run out and it will no longer be possible to enroll patients into psychophysical studies. Thus, either more permissive rules and regulations will emerge and society’s rules will be more lax, or we will potentially endure decades or centuries of unnecessary barriers to scientific discoveries about consciousness.

As it turns out, tens (if not hundreds) of millions of persons have tried psychedelic substances. Interest is not slowing down, and no propaganda effort save for literal brain-washing will prevent people from developing a genuine interest in the field. There will be millions of volunteers and hundreds of thousands of willing researchers. As I envision in The Psychedelic Future of Consciousness, the future of consciousness studies will be nothing like what it is today. Once those who are interested are given the opportunity to study psychedelics seriously, the research panorama will look very differently.

But What to Do in the Meantime?

Investigating the psychophysics of texture perception more thoroughly may require at least mild to moderate doses for noticeable effects, and a period of examination at a time when acute effects are present. How to overcome the hurdles ahead of any current academic investigation of this nature? This ultimately depends on the specific constraints required for the study. It is true that people would have to be high on LSD (or Mescaline, mushrooms, 2C-B, etc.) while they perform the experiment. But who says that they have to conduct the experiment in your presence? That you have to give them the substance yourself? Without the typical background assumptions that are assumed in psychophysics research labs, can we do anything else?

I suspect that we can do much better. We can come up with protocols that side-step current obstacles. We need to be creative. And I do have some ideas, with varying levels of plausibility for how to implement psychedelic research in legal, viable and immediate ways. Consider, for example, how James Fadiman has been collecting hundreds of micro-dose reports via email without any difficulty for many years. He is taking advantage of the fact that the optimal format of micro-dose reports tends to be “summary and retrospective” narrative. A single dose on a single day on a single person is not likely to be life-changing. So his particular line of research is well suited for the means available to him. Likewise, as I requested in the psychedelic experience of visual textures post, people’s subjective judgements of visual textures while high on LSD can be recorded online. These are just two examples of how non-mainstream research approaches can be taken to study psychedelics. Unfortunately, both of these protocols lack proper controls and standardized settings. But this is all I will say for now.

Stay tuned: In a future post I will propose a set of protocols for independent studies on psychedelics, including additional methods for studying psychedelic visuals.

If you would like to be a collaborator with me, please email me by finding my contact info in the contact section of this blog. We’ll take it from there.

## Psychedelic Perception of Visual Textures

On March 24th 2015 Team Qualia Reverse Engineering (TQRE) went for a long walk within the Stanford campus and around Palo Alto. The purpose of this walk- the Pattern Walk -was to snap a picture of every interesting pattern (or texture) out there that got on our way. The following gallery contains 74 of these patterns. These display a wide range of texture properties: Natural/synthetic, regular/irregular, 2D/2.5D/3D, symmetric/asymmetric, structured/unstructured, etc.

Here are a few observations:

1. Human languages do not have the necessary vocabulary (and conceptual primitives) to talk about visual textures adequately. When two images belong to the same category (say, “plants” vs. “rock tilings”), and have roughly similar first order statistics (mean, standard deviation, kurtosis, etc. of the RGB values) there is relatively little else to say about a texture in a way that a person would understand.
2. Our visual system can recognize extraordinarily subtle properties that distinguish textures from one another. For instance, I bet you can recognize at an immediate experiential level the differences between picture 61 and 62. But can you verbalize such difference?
3. Mathematics, and statistics in particular, may provide helpful semantic seeds for describing patterns. Indeed, having a basic handle on a few mathematical concepts can leverage one’s ability to talk about the differences between textures. For example, compare images 43 and 44. They are perceptually very different. But how long would it take you to convey the difference to a random person? If there was a person who could only hear you, how would you signal that you are not talking about 43 but 44? If both of you know of the concept of concavity you might only need a few words! Without it, you’d be fairly lost.

Fancifully, we may someday produce a good vocabulary that can effectively allow us to talk about visual textures without having to be currently sharing the same (similar) visual experience.

In practice, we already have some vocabulary that accomplishes this, but it is very obscure and sufficiently technical that its widespread adoption is unrealistic. In particular, I encourage anyone interested in the topic to read “A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients” by Javier Portilla and Eero P. Simoncelli. They analyze (and synthesize) visual textures by computing a set of highly descriptive statistical properties characteristic of the pattern in question.

As we will see in future posts, their model can be used to point out perceptible statistical features that are perceived as regularities by the human visual system. It may not be sexy to say “Hey Ma’m I really dig the Cross-scale phase statistics of the pattern in your dress.” For now, that’s what we have.

If you want to help me figure out how psychedelics affect your visual experience:

Please browse through these images by clicking on the first one and exploring the slideshow. See which images you like, which produce “odd or interesting visual effects” and which “stand out” in however way you want to define that. Feel free to comment right below any of the images (there is a comment section beneath each image when you click through them as a slideshow) to point out the peculiarities that you notice.

Critically, also include your state of consciousness in the comment. If you took LSD (or any visually-affecting substance) two hours ago (or you are still high), it would be great if you could point that out. Please explain how you think that your visuals are affecting your experience of the various patterns. Everyone loves to talk about their LSD visuals. Now you can do it all you want! And your efforts may actually enable us to understand the way psychedelics affect the algorithms of human vision 🙂

The best case scenario:

You would make comments on these images while sober, and then add comments while high on a psychedelic (doesn’t have to by psychedelic – could be dissociative, though typing might be particularly hard in that condition). Point out the main differences between the textures as perceived on each of the states of consciousness you happen to be in. If you do decide to follow the above protocol, please provide information about the specific substance(s) you consumed and how long ago you did so.

That is, do this if you were planning on taking a hallucinogen to begin with. Independently of that, baseline data is still very valuable, so do add comments about these patterns even if you are sober and plan on staying sober 🙂

In the following post I will explain how this Pattern Walk, the statistical analysis of visual textures, psychophysics and psychedelics can ultimately fit into the larger project of reverse-engineering the computational properties of consciousness.