Information-Sensitive Gradients of Bliss

“Normal” or so-called “euthymic” people are inclined to judge that hyperthymics/”optimists” view the world through rose-tinted spectacles. Their central information-processing system is systematically biased. Conversely, hyperthymics see the rest of us as unreasonably pessimistic. Chronic depressives, on the other hand, may view euthymic and hyperthymic people alike as deluded. Indeed victims of melancholic depression may feel the world itself is hateful and meaningless. For evolutionary reasons (cf. rank theory), a genetic predisposition to hyperthymia and euphoric unipolar mania are rarer than dysthymia or unipolar depression. Most of us fall somewhere in between these temperamental extremes, though the distribution is skewed to the southern end of the axis. Genetics plays a key role in determining our hedonic set-point, as does the ceaseless interplay between our genes and environmental stressors. Inadequate diet, imprudent drug use, and severe, chronic, uncontrolled stress can all reset an emotional thermostat at a lower level than its previous norm – though that norm may be surprisingly robust. Unlike recreational euphoriants, delayed-onset antidepressants may restore a lowered set-point to its former norm, or even elevate it. Antidepressants may act to reverse stress-induced hypertrophy of the basolateral amygdala and contrasting stress-induced dendritic atrophy in the hippocampus. Yet no mood-brightener currently licensed for depression reliably induces permanent bliss, whether information-signalling or constant, serene or manic. A genetically-determined ceiling stops our quality of life as a whole getting better.


Is the future of mood and motivation in the universe destined to be an endless replay of life’s evolutionary past? Are the same affective filters that were genetically adaptive for our hominid ancestors likely to be retained by our transhuman successors? Will superintelligent life-forms really opt to preserve the architecture of the primordial hedonic treadmill indefinitely? In each case, probably not, though it’s controversial whether designer drugs, neuroelectrodes or gene therapies will make the biggest impact on recalibrating the pleasure-pain axis. In the long-run, perhaps germline genetic engineering will deliver the greatest global enhancement of emotional well-being. For a reproductive revolution of designer babies is imminent. Thanks to genomic medicine, tomorrow’s parents will be able to choose the genetic make-up and personality of their offspring. Critically, parents-to-be will be able to select the emotional dial-settings of their progeny rather than play genetic roulette. In deciding what kind of children to create, tomorrow’s parents will (presumably) rarely opt for dysfunctional, depressive and malaise-ridden kids. Quite aside from the ethical implications of using old corrupt code, children who are temperamentally happy, loving and affectionate are far more enjoyable to bring up.


The collective outcome of these individual parental genetic choices will be far-reaching. In the new era of advanced biotechnology and reproductive medicine, a combination of designer drugs, autosomal gene therapies and germline interventions may give rise to a civilisation inhabiting a state-space located further “north” emotionally than present-day humans can imagine or coherently describe. Gradients of heritable, lifelong bliss may eventually become ubiquitous. The worst post-human lows may be far richer than the most sublime of today’s peak experiences. Less intuitively, our superwell descendants may be constitutionally smarter as well as happier than unenriched humans. Aided by synthetic enhancement technologies, fine-textured gradients of intense emotional well-being can play an information-signalling role at least as versatile and sophisticated as gradients of emotional ill-being or pain-sensations today. Simplistically, it may be said that posterity will be “permanently happy”. However, this expression can be a bit misleading. Post-humans are unlikely to be either “blissed out” wireheads or soma-addled junkies. Instead, we may navigate by the gradients of a multi-dimensional compass that’s designed – unlike its bug-ridden Darwinian predecessor – by intelligent agents for their own ends.


– Life in the Far North: An Information-Theoretic Perspective on Heaven by David Pearce


36 Textures of Confusion

Formal Logic

When I was in 10th grade I took a course in formal logic. I had been a big fan of logic (and math in general) for several years, so I was looking forward to seeing how the class would approach the subject. I personally liked the teacher and I knew he thought very deeply about a range of topics (including aesthetics and philosophy). I was sure I was going to have a great time.

Unfortunately, the overall learning strategy of the class consisted of studying the textbook in extreme detail. The way I remember the textbook was that it featured a mixture of very casual and naïve paragraphs interspersed with blocks of rigid definitions and formulaic procedures for solving logic problems. My overall perception of the textbook was that anyone with a genuine interest in the beauty of math would experience the exercise of reading this book as particularly unpleasant.

I was used to math classes that didn’t actually require you to study anything; usually, problem solving skills and pragmatic inference of the meaning of words during the exam was good enough. In contrast, most questions on the exams for this class had a very particular style. The answers had to be verbatim repeats of the specific idiosyncratic responses found in the textbook. If you knew the contents of the textbook by heart, then the exam would be trivial. If you didn’t, then no amount of problem solving would get you anywhere.

These exams were open-note but closed-textbook, which meant that if you simply copied the entire textbook into your notebook you would easily be able to respond accurately to the vast majority of the questions. And if you didn’t, then you were almost guaranteed to fail. This meant that the largest fraction of the variance of grades in the class was determined by whether or not students took the time to do the grueling task of transcribing an entire textbook into their notebooks.

Needless to say, I intensely disliked this approach.

Thankfully, in every bad situation you can always find something good that redeems it a little bit [citation needed]. And in this case, what could be rescued from the situation was the man from Figure 5.9:


Figure 5.9: This man is confused

This must have been around page 150, which dealt with the need for logic. The textbook said, in a very informal way, something along the lines of: “Imagine a man without any logic. This person would have disjointed thoughts with no objectives, and he would be incapable of making sense of anything. The man in question would be confused. See Figure 5.9”

The teacher joked that the man in the figure could be experiencing one of many possible states of mind. His expression is somewhat ambiguous and it is unclear what exactly it adds to the conversation. Likewise, the facial features are not even the most salient component of the picture; his hair looks completely bizarre.

The Value of Confusion

This picture made me reflect on the difficulty of expressing mental states using drawings and pictures. A facial expression is perhaps a good start. Words, of course, and dialogue can help you trigger an emotion or state of being. But that only takes you so far, and it restricts you to what are largely social emotions.

Confusion, on the other hand, is an umbrella term for many states that are hard to communicate and describe. There is perceptual confusion, emotional confusion, cognitive confusion and even ontological confusion. Each of these varieties contains many flavors; there is a combinatorial explosion of possible reasons for the confusion.

Subjectively, confusion is an extremely interesting state of consciousness, since it spawns a lot of novelty. Even though it can and often is unpleasant (especially when what’s at stake is something one values), confusion comes in all shades of hedonic tone. Pleasant confusion is possible, and indeed it may play an important role in philosophical and spiritual euphoria. Likewise, one can achieve fantastic levels of neutral-valence confusion during meditation (alternating, at times, with states of very high clarity). Epiphanic, wondrous and mystical states are also often proceeded by profound confusion of the ontological kind (where you doubt the deepest background assumptions that provide the stilts upon which your worldview is suspended).

The fact that the texture of one’s experience has information processing properties (aka qualia computing) is itself more evident during states of confusion. For example, when you are confused about the meaning of something, this will have implications for the way you experience language and encode gestalts of experience (ex: “This synesthetic sensation here is usually paired up with meaning, but what is the meaning of it now? Without experiencing the meaning I usually ascribe to the sensation, I can’t compare it to other sensations I’ve had before.”)

Since language and facial expressions have their limitations, one might prefer to communicate confusion and other states using non-symbolic expressions. Visual textural gestalts, it seems to me, may take us the farthest in this direction, at least with the current level of technology (that is, unless we also include music, which itself has textural qualities).

In order to visualize new kinds of confusion, we can project the textural gestalt that the man from Figure 5.9 is experiencing into the picture itself, and imagine that we were given private access to his state of consciousness. We can then experience what it would be like to be him in these different experiential worlds, and introspect on the subjective nature of his confusion.

Doing this is now easier than ever thanks to the recent and fantastic developments in deep neural networks. In order to try out this technology, I decided to texturize the confusion of the man from Figure 5.9 in a myriad of ways. The textures themselves are a mixture of pictures I’ve taken or synthesized in the last couple of years and textures I’ve found online. I used the cool online service developed by the Bethge lab at the University of Tübingen to make these pictures. Feel free to try it yourself, it’s really fun.

So here you have it folks. The man of Figure 5.9, experiencing 36 different kinds of confusion:

Getting closer to digital LSD

I am very pleased with the recent work on psychedelic replications by communities such as the wonderful Psychonaut Wiki and r/replications. There is a lot of great work in the area, a little too much to discuss at length in one post. Keep up the good work!

A recent source of marvelous psychedelic replication techniques has just come into the scene, and from an unlikely source. Of course, we are talking about inceptionism applied to deep belief networks.

Someone said DMT?

Someone said DMT?

First of all, who says these pictures are actually trippy? Is there evidence of that? I intend to fully operationalize the concept of trippiness for the classification of pictures; I believe the question is empirically approachable. In the meantime I will simply point out that a lot of people are talking about the peculiar trippiness of these pictures. To give an example, look at some of the comments on the Google blogpost:

Help! We’ve created AIs more powerful than us, and now we need to feed them hallucinogenic drugs to subdue them…. – Urs

Either somebody has been feeding hallucinogens to Google’s image-recognition neural networks, or computer comprehension is alien! Well, actually, I wonder how this compares to visualizations of how the human brain stores images for pattern-matching purposes. – Stephen

Computers are all on drugs. – Matt

And from the Vice article:

“Its incredible how close it looks to an LSD trip, that is normally so hard to describe.” – corners

There are ongoing discussions in a lot of forums about this right now. Somehow, it seems that these new pictures are hitting a particular component of the psychedelic experience that previous replications have missed or at least not fully captured. What is that?

For the purpose of this post I will use a particular classification of phenomenal effects caused by psychedelics. Specifically, the one proposed by Psychedelic Information Theory. In order to fully grasp the motivation for this classification I highly recommend reading the control interrupt model of psychedelic action. In summary, it seems that there are natural inhibitory processes that prevent features of our current experience to build up over time. Psychedelics are thought to chemically interrupt inhibitory control signals from the cortex, which in turn results in a non-linear interaction between the unmitigated characteristics of your conscious experience. I will explain in a bit how this model provides a good framework for explaining the way recent Google Inceptionist (GI) pictures fit into the broader world of visual psychedelic replication.

But now let’s start with the three classes of hallucinations discussed:

  1. Entropic hallucinations describe the visual effects of gently pushing one’s eyes as well as the amazing interaction between LSD and strobes
  2. Eidetic hallucinations are the result of interpreting ambiguous stimuli using high-level concepts
  3. Erratic hallucinations result from the chaotic binding and over-saturation of sensory modalities, which affect the stability of the global perceptual frame (and probably disrupts the continuity field too)

Zooming into the phenomenology of eidetic hallucinations:

The most commonly reported eidetic hallucinations seen on psychedelics are of people, faces, animals, plants, flowers, spirits, aliens, insects, and other similar archetypes. Eidetic hallucinations can sometimes take the form of entire virtual worlds, spirit dimensions, invisible landscapes, and so on. Eidetics often emerge within a pre-existing entoptic interference pattern that grows in intensity over time to produce more photographic or 3D rendered objects. Eidetics under the influence of psychedelics are most often reported with eyes closed or while sitting motionless in meditative trance. On high doses of psychedelics eidetic hallucinations may materialize with eyes open on any surface, pattern, or texture that’s gazed at for more than a few seconds.*

If you surf the internet looking for replications of psychedelic experiences, you will notice that there are great examples of a wide range of effects, but compelling software-generated images of eidetic hallucinations are rare. The challenge here is the complexity of creating actionable tools that highlight high-level features in pre-existing pictures. Amazingly, people can make successful and stunning pictures with eidetic tones, but this requires a lot of dedication and artistic experience. The mighty human artistic effort is unstoppable, though:

Thanks to this 3-fold classification of psychedelic effects we can isolate the quality of experience that both Dali and the recent GI pics specifically enhance. Of course, the phenomenology of most psychedelic experiences incorporate elements of each of these classes, and the interaction between them is certainly non-trivial. In addition, specific substances may have a larger loading of each type, and signature proportions with peculiar results.

It is also worth mentioning the existence of other classification systems, within and beyond visual phenomenology. For example the subjective effect index of Psychonaut Wiki and even the various circuits proposed by ancient Leary and Dass writings have very worthwhile observations that may come useful in one context or another. For the level of resolution here discussed giving eidetic hallucinations their own class is particularly useful.

How the Inceptionist method and psychedelic experiences work similarly

Here is the core of the explanation for how the Google trippy pictures were made:

In this case we simply feed the network an arbitrary image or photo and let the network analyze the picture. We then pick a layer and ask the network to enhance whatever it detected. Each layer of the network deals with features at a different level of abstraction, so the complexity of features we generate depends on which layer we choose to enhance. For example, lower layers tend to produce strokes or simple ornament-like patterns, because those layers are sensitive to basic features such as edges and their orientations.

In some sense this is basically the same eidetic effect we find in psychedelic experiences. For one reason or another, there are moments during a psychedelic experience in which strong eidetic effects manifest. As if a specific layer (or hierarchy level) of one’s model of reality is chosen for being enhanced and fractally iterated in a scale-free manner. Referencing back the control interrupt model of psychedelic action, we can reason that what is going on involves a reduction in the amount of inhibition that highlighted high-level features receive. Again, this resembles the Inceptionist algorithm:

If we choose higher-level layers, which identify more sophisticated features in images, complex features or even whole objects tend to emerge. Again, we just start with an existing image and give it to our neural net. We ask the network: “Whatever you see there, I want more of it!” This creates a feedback loop: if a cloud looks a little bit like a bird, the network will make it look more like a bird. This in turn will make the network recognize the bird even more strongly on the next pass and so forth, until a highly detailed bird appears, seemingly out of nowhere.

Now, this only really shows a snapshot of a psychedelic experience with a heavy eidetic bent. In actual psychedelic experiences there are other common factors that come into play that influence the experience. First, not only are specific features highlighted, but, on the whole, we could say that there is an increase in the overall amount of sensations experienced together. The overall amplitude of your experience goes up, if that makes sense. In other words, although this is hard to imagine, the overall amount of experience increases relative to baseline. That is not evoked using external stimuli, of course, since the actual change in the intensity of your experience requires direct control interruption. The overall information content globally available in the field of awareness of a person tripping increases in a dose-dependent way.

The second hallmark characteristic of psychedelic experiences, which gives them a powerful edge over current digital techniques, is that the state highlights already salient stimuli. High-level psychedelic pattern recognition seems to be based on attention-modulated saliency enhancement. Let me explain:

Our visual system automatically recognizes salient features in our experience. This is not an exclusive property of visual consciousness, by the way. Here we must notice that awareness and attention are distinct but related aspects of our mind. Awareness happens effortlessly, and its visual variety arises as soon as we open our eyes (within 200 milliseconds therefrom). Even at the level of awareness we see a fast sorting of perceived features by their overall saliency, which is a function both of their intrinsic properties and those relative to every other feature in the awareness field. Attention, which is slower and builds on top of the awareness field, enables a variety of high level cognitive activities to interplay with the features highlighted by awareness. In turn, the overall state of consciousness of a person changes as attention moves the reference point for awareness to bring forth new salient features. Iteratively, these processes allow a mind to surf through states of consciousness.

In summary, awareness creates the marketplace of salient features that compete for attention. As attention is recentered on a new cluster of features, the field of awareness is modified and the new salient features again have a chance to change the focus of attention.

With psychedelic-induced control interruption, the intensity by which saliency of features in the field of awareness is highlighted goes up significantly. In turn, the attention-modulated perception of the intensely salient features highlights specific high-level features suggested by the field of awareness. And finally, this conceptual mental state highlighted via attention, results in an even higher saliency for conceptually-related features. And hence come the eye reality, fish realty, tree reality, abstract concept reality, divine reality, fractal reality, etc. people discover on LSD.

Although a difficult challenge, I predict that a well-trained, dedicated and mentally healthy psychonaut would be able to paint psychedelic experiences of her own that highlight similar high-level features as those highlighted on specific Inceptionist works of art. Probably a long meditative practice would help in the process, since the specific saliency of various features is attention-modulated, and thus requires inhibiting unrelated salient directions (e.g. deep philosophical questions, personal issues, etc.) and focus exclusively on, say, dogs.

Who chooses what is salient?

If you already know what class of features you want to highlight, then the inceptionist method will help you. But what about choosing what to highlight to begin with? This, I believe, is the crux of what makes psychedelic experiences (and minds in general) still unbeatable by neural networks. Once you know what to look for, your cortex and inceptionist methods (and their future incarnations) might be on the same playing field. But what enables you to decide what is worth looking for?

The key unresolved problem standing for a fully-digital psychedelic experience replication algorithm is what I call the saliency-attention mapping. This is: Given a particular conscious experience that is highlighting a set of features, how does attention ultimately find what to focus on? How are the subsequent relevant features to be highlighted? In many cases we choose to ignore all of the immediately salient features in a scene precisely to see more subtle patterns. And during a psychedelic experience, directing your attention to entirely unsuspecting places has the effect of switching off previously salient features and activating a new class of them (for example, choosing to focus on the music rather than the visual scene).

Is there any way of modeling the saliency-attention mapping without taking into account all of the information present in the field of awareness at the time? Indeed, an ongoing hypothesis here in Qualia Computing is that consciousness itself is required for this step. The very computational advantage of being conscious seems to be related to the unitary nature of experiences: Your choices are not only the result of parallel processing or implicit information integration. They stem from what you choose to pay attention to considering the entirety of your field of awareness. You do this at every point in time. Thus, a sort of instantaneous and ontological unity is required to account for a significant step of the information processing pipeline of the mind. And this may lead to a saliency-attention function whose runtime complexity is impossible to match with digital computers.

The conceivability horizon

Now, this unitary field of awareness step also has large down-stream effects. In particular, subsets of the phenomenology of experiences can be reinterpreted in very novel ways. Psychedelics are likewise famous for unlocking entirely new conceptual ontologies and points of view that remain with the person long after acute effects subside. We could call this, an extension of the horizon of conceivability. This comes about from considering many of the features of the particular conscious experience at once and identifying a new private referent (such as a concept) whose meaning is derived from the unique combination of those elements.

Without a unitary conscious experience this step would be impossible, and it remains to be seen for an artificial neural network to accomplish this on its own. For completeness, it is worth mentioning that phenomenal binding also has strong implications for memory. Every time we experience a new situation a new ‘situational snapshot’ is added to the collection of — and network of relationships between — memories that can be triggered with temporal lobe stimulation. Thus, incorporating a human (or whatever implements phenomenal binding) into the loop may be unavoidable.

The future of psychedelic replications and consciousness engineering

Eidetic art is marvelous, and for a long time we didn’t have any idea about how to systematize it in software. Now we have some wonderful examples of a fully scalable approach. Inevitably, we will soon have visual editing software that incorporates neural networks.

Deep belief networks applied to replications will allow us to drastically increase the level of realism of simulated trips. This will thus draw a lot more attention to this fascinating field, and bring engineers, artists and mathematicians onboard. They will have a wonderful synergy in this sphere.

But how practical are these techniques? If you want to find fundamentally new patterns in an image, what should you use… neural networks or LSD? The answer is: why do you have to choose only one? Here is where I casually mention that if you were planning on taking a psychedelic sometime in the future, why not tell us how the trippy images of Google look like during a trippy experience? I bet a lot of people would appreciate your input.

Presumably, incorporating a human in the loop could actually empower these networks to recreate remarkably psychedelic progressions of scenes and features (and high level ideas!). To do so you need to somehow identify what the human finds salient in the picture/video being explored, and how her attention is directed as a consequence of that. Obvious candidates here are eye tracking devices and the general class of bio/neurometrics. More speculatively, endocrine measurements of the chemical markers of saliency and attention may be of tremendous value too. What would this look like? A person hooked to a series of tubes that provide fast feedback using a lab-on-a-chip, and a deep belief network with flexible Inceptionist dynamics guided by the person’s measured center of attention. In case you haven’t noticed, I think that this area of exploration is extremely promising. Go ahead and do it!

Now, if you want to figure out a hard technical problem, currently mild psychedelic experiences are more promising than deep belief networks. This, again, is because the attention-modulated saliency enhancement of psychedelics can allow you to discover, explore and reinterpret the features that matter for a particular problem. Assisted digital exploration, however, may someday surpass the effectiveness of psychedelics, or better yet: A smart combination of techniques –chemical, biological and digital– will incite in the field of consciousness research what the Galilean revolution was to physics. The hands-on collective exploration science needs in order to fully thrive is about to arrive for consciousness. Finally!

Psychedelic Perception of Visual Textures 2: Going Meta

Some time has passed since we did the pattern walk. I was happy to see some psychedelic participation on that first wave of textures. Since then I have been gathering more and more textures from all over the place, so many that the ones below are just a tiny fraction of the total. The idea of this second wave is to go meta: Now a few of the Inceptionist pictures recently unveiled by Deep Belief Networks are included, as well as several other cool psychedelic replications. The question is… how does a psychedelic replication look like through an actual psychedelic lens? Let’s find out!

You know what to do: If you were planning on taking a psychedelic (dissociative, or God forbid, delirant) hallucinogen, feel free to browse through these pictures and add comments on the salient features you experience from them. To do so click on the pictures that interest you and leave a comment below. Please provide information about the subtance(s) you took, their dosages and how long ago you took them.

What patterns do you see? What stands out? What amazes you?

Special thanks to Mark Gomer, the family of graduates at the 2015 Stanford Psychology Commencement (where I took pictures of cool dress and shirt patterns), and the very diverse and beautiful carpet store right next to Jawbone in San Francisco. Without them, the second wave would have been less diverse and novelty rich.

Enjoy! 🙂

Psychophysics for Psychedelic Research: Textures

In this post I will provide an account of my personal research project to understand the algorithms that underly human visual pattern-recognition. This project is multidisciplinary in nature, combining paradigms from three fields: (1) the analysis and synthesis of textures, (2) psychophysics and (3) psychedelic research. I will explain in detail how these areas can synergistically help us understand the computational properties of consciousness. In the process of doing so I will describe some of the work I have done in this direction.

tl;dr: With texture synthesis algorithms we can control the statistical features present in textures. By using an odd-one-out paradigm where participants have to find the “different texture” we can identify the statistical signatures of the visual patterns people can perceive. Collecting these signatures under various states of consciousness will reveal the information processing limitations of visual experience. It may turn out that some patterns can only be seen on LSD (psilocybin, mescaline, etc.), and this information will inform a general theory of vision’s algorithms, expanding the scope we have studied so far and suggesting relevant applications of psychedelic consciousness.

Introduction to Spatial Patterns

The world is patterned. In fact, it is so patterned, that it is difficult to identify natural surfaces that have no perceptible regularities. The grass, the trunk of trees, the surface of rocky mountains, the dancing and dissolving of the clouds. All of these natural scenes are full of regularities. For hundreds of millions of years animals on this planet have existed in an environment where regularities are not inconsequential: being able to use or detect camouflage is a matter of life or death to some species. The insects who hide between the leaves pretending to be part of the scene are in an evolutionary arms race against predators and their sensory apparatus. (Here is a neat collection of insect camouflage). Arguably, there are strong evolutionary selection pressures that push predator’s visual system into adapting to recognize the differences between the scene’s visual statistics and the prey’s body appearance.

Other widespread examples for the evolutionary relevance of pattern recognition can be found: Birds may take advantage of the look of cloud formations to determine if they should fly or find refuge, herbivores may seek only plants with specific visual properties to avoid poisonous lookalikes and parasites, the health of potential mates can be assessed by the uniformity of their fur patterns. You get the idea.

Not surprisingly, you today can look at a plain rock and see a lot of visual properties pointed out by patterns you can perceive. Unfortunately, something has kept us quiet about this aspect of our perception: most of these properties are hard to verbalize. Often, you will be able to tell apart two kinds of rocks by grasping the subtle visual differences between them, but at the same time still be unable to explain what makes them different.

What exactly is going on in your mind/brain when you are recognizing characteristic features in textures? We don’t know how the information is processed, why we perceive the features we perceive, or even how the various features are put together in a unified (or semi-unified) conscious representation. The hints we do have, however, are precious.

Receptive Fields

A big hint we can build on is that many neurons in the primary visual cortex (of cats, monkeys, and probably all mammals) respond to visual stimuli in specific areas of the visual field. For a given neuron, the shape of this region is an instance of a well-studied canonical function, as shown in the images below. The area of the visual field a neuron best responds to (by becoming excited or inhibited) is called its receptive field, and the canonical function is the Gabor filter.

As far back as the early 60s, research has been conducted to map the receptive fields of neurons by inserting electrodes into the brain of animals and presenting them with visual input of lines and shapes. In 1961 D. H. Hubel and T. N. Wiesel showed for the first time that neurons in a cat’s cortex and lateral geniculate have receptive fields that look like this:


The crosses indicate regions in which stimuli excites the neuron’s activity while the triangles represent regions in which stimuli inhibits the neuron’s activity. Due to the arrangement of these excitatory and inhibitory regions, these neurons functionally work as edge-detectors. A computer rendering of these receptive fields looks like this:


Since then a tremendous amount of research has repeatedly confirmed the existence of such neurons, and also uncovered a large number of more complex receptive fields. Some neurons even respond specifically to abstract concepts and high-level constructs. More recently, the simple receptive fields shown above have been modeled as Gabor filters in quantitative simulations. This, in turn, has been successfully used to build brain activity decoders that reconstruct the image that a person is seeing by assuming that the activity of fMRI’s voxels approximately matches the added activity of neurons with Gabor receptive fields (see: Identifying natural images from human brain activity by Kay, Naselaris, Prenger and Gallant).

Thus we can say that we know that a large number of neurons in our visual cortex have Gabor receptive fields and that the collective activity of these neurons contains enough information to at least approximately reconstruct the image a person is seeing (well enough to identify it from a pool of candidates).

We can’t jump from these findings alone to a global theory of visual processing. That is, without also considering what people actually experience. It may turn out, for example, that the activity of the visual cortex contains a lot of information that can be decoded using machine learning techniques applied to fMRI’s voxel brightness. And yet, simultaneously, it could be that people do not consciously represent all of the {decodable} information.

Likewise, a priori we cannot rule out the possibility that some of the information we consciously experience is not actually decodable using brain activity alone. (A quick remark: this may be the case even if one assumes physicalism. Why this is the case will be explained in a future article).

To illustrate this example we can consider the information available in the retina and before it. The light that reaches the outer surface of our eyes contains all of the information available to our mind/brain to instantiate our visual experience. Yet, there is a lot of information there that is ultimately irrelevant for our conscious experience. For instance, there is infrared and ultraviolet light, as well as light that does not make it to our retina, light that fails to elicit an action potential, and so on. If you can discriminate between a really hot and a cold metal using the infrared signature of the light that reaches the eye, you have certainly not shown that we perceive infrared light or that we use it to make distinctions. It merely means that such information is sufficient. We wouldn’t yet know that we actually use it or that it shapes our experience.

But how exactly do we develop an experiment to infer the statistical properties represented by our experience? Here is where the analysis and synthesis of textures becomes relevant.

Analysis and Synthesis of Textures

The idea of using oriented Gabor-like filters (also known as “steerable pyramids”) to analyze and synthesize visual textures was, as far as I know, first proposed by David J. Heeger and James R. Bergen in “Pyramid-based texture analysis/synthesis.” Texture analysis in this context means the use of algorithms to characterize the properties of textures to capture what makes them unique. In turn, texture synthesis, refers to the application of texture analysis to produce an arbitrarily large patch of a synthesized (synthetic) texture so that the synthetic and original texture are as indistinguishable as possible. Of course whether something is “indistinguishable” or not is to a certain degree subjective. Here the criteria for indistinguishability between the original and the synthetic textures is whether a person comparing them side by side could confuse them. In the following section I address how to operationalize and formalize the indistinguishability between patterns using psychophysics.

This particular texture synthesis algorithm works by forcing a white noise image (of any size) to conform to the statistics obtained in the texture analysis step. This is done iteratively, matching the histograms of the synthesized canvas to the various statistics computed from the original texture. I recommend reading the paper to gain a better grasp of what the algorithm does, and to see some stunning examples of the output of this algorithm.

The use of steerable pyramids was later refined by Portilla & Simoncelli‘s texture synthesis algorithm, which currently plays an important role in my research. This algorithm extends Heeger et al. by including additional statistics to enforce, which are (roughly) computed by measuring the autocorrelation between the various components of the steerable pyramid texture representation. Below you can see two pairs of pictures (two originals and their corresponding same-sized synthetics versions), that I recreated using Potilla & Simoncelli’s matlab code:

As you can see, the original and synthetic images are fairly similar to each other. Close inspection is sufficient to notice deadly differences. If you only use your peripheral vision it is very challenging to see major differences.


Psychophysics is the study of the relationship between physical stimuli and experience (and often behavior). Thanks to psychophysics we now have:

  1. A good approximate map of the phenomenal space of color (CIELAB)
  2. A strong grasp of the nature of color metamers, which in turn underlies all of our color display technologies.
  3. The ability to predict the subjective intensity of the experience elicited by stimuli as a function of the energy of the stimulus (see Weber–Fechner law).

A very powerful idea in psychophysics is the use of just-noticeable differences (JND): Carry a bucket of water. How much water should I add to it so that you are capable of perceiving a difference in the weight of the bucket? Look at a pair of identical light sources. How much blue (in this case, the specific and pure frequency of light that elicits the blue qualia) can I add to one of the lights before you perceive them as being differently colored? Pinch your skin with two needles. How far can their pointy part be before you perceive two needles rather than one?

In these cases, though, there is a natural (and sometimes unique) dimension along which the stimuli can be varied in order to compute the JND. What about visual textures? Here the problem becomes non-trivial. In what way should we change a texture? And how should we accomplish such? There is no clear and obvious scale for describing textures. So what can we do?

Psychophysics of Textures (Take 1):

Before learning about steerable pyramid representations of textures I developed a variety of psychophysical tests to identify the JND between textures. I did this by changing the value of parameters of images with ground-truth statistical properties. If you are curious, feel free to try the first iteration of my experimental paradigm. (Note: I am not collecting data from that experiment, so you should not expect to contribute to science by finishing the task. That said, you can have fun, and a pop-up window with your raw score will appear when you finish both tasks). The average accuracy of the texture discrimination part is about 13/21 (with a chance performance of 3/21), while the performance in the numerical pattern completion task is about 3/5 (with a near 0/5 expected performance when answering randomly).

One of the statistical properties I was changing in the stimuli was the variance of a 2D Gaussian process I implemented with a python script and the magic of Gibbs sampling. Thus, a subset of the images I tested were complete noise except for the first order statistic of local autocorrelation created by the differently parametrized Gaussian process. Here are a couple of examples of such patterns:

These pictures, it turns out, are relatively easy to tell apart when the parameters are sufficiently different. There is a threshold of similarity in the variance parameter after which people cannot distinguish between them. A nice property of this particular method is that you can be sure that if people do recognize the differences, it is because they are somehow extracting features that are a direct consequence of a different value for the variance parameter.

The textures above instantiate the simplest statistical differences that can be created, after the mean and standard deviation of the pixel values. To explore more broadly a wider variety of patterns, I also created textures with fairly complex parametrizable Turing machines (TMs). This, unfortunately, does not lend itself to a clear analysis. In brief, this is because changing a single parameter of such TMs can produce profoundly different results with unclear ground-truth properties:

What information could I gain from the fact that changing x, y or z parameter in a Turing machine by a, b, or c amounts enables accurate discrimination between textures? In principle one could try to explain the performance obtained by making an a posteriori analysis of the correlation between a variety of statistics measured in the textures. But since the textures were not created to have specific statistical properties, the distribution of such properties will not be ideal to find JNDs or discriminability thresholds.

Even if you do this, you will still have more problems. The images are more different (and different in more ways) than what your texture analysis algorithm is capable of detecting. Thus the reason why people can tell these textures apart is not possible to extract from the statistical differences you measure between them. At least not without being extremely lucky and hitting the visual features that matter.

And here, I was stuck several months.

Psychophysics of Textures (Take 2):

After learning about the steerable pyramid model, the work of Eero Simoncelli and the state of the art in fMRI decoding of visual experience, I decided to shift gears and approach the problem with some of the best of the tools created so far.

It turns out that the concept of texture metamers had been developed to describe the perceptual indistinguishability between textures in peripheral vision. Taken from here, the following two images are texture metamers. The images look the same when you center your vision in either of the central red dots. Close foveal inspection of the image, however, will reveal that these are very different pictures!

A specific study caught my attention, and I decided to replicate it as a final class project. Specifically, Texture synthesis and perception: Using computational models to study texture representations in the human visual system by Benjamin J. Balas. I attempted to replicate some of the main results of that study using Mechanical Turk. A full account of the experimental procedure, results and discussion can be found in this wiki here, (written for Stanford’s Psych 221 – Applied Vision Systems). To see the actual experiment performed, try it out here: Replication (and the github repository).

The main idea goes as follows: If textural metamerism can be verified using a given texture synthesis algorithm, then we can be reasonably certain that the algorithm is capturing a large component of what makes textures different from a human point of view. In particular, Benjamin Balas noted that Prtilla & Simoncelli’s algorithm could be “lesioned” by failing to enforce specific statistical features obtained in the texture analysis step. In this way, one can purposefully fail to match specific statistical features between the original and synthesized texture, and then measure how this partial texture synthesis algorithm affects the performance of texture discrimination in humans.

For illustration, here is a set of possible texture synthesis “lesions:”

The operationalization of the experiment used an “odd one out” paradigm: in each trial, three images are presented for 250 milliseconds at 3.5 degrees of vision from the fovea (each image being 2 degree wide in diameter). Two of the three images will come from the same group (ex. two original textures, two marginals removed, etc.). The remaining image is the odd one out. The study measures how often participants detect the odd one out depending on the statistical feature that is not enforced in the synthesized textures (for a more in-depth description: wiki).

Overall the performance of the participants in my replication was much closer to chance than Balas’ results. That siad, qualitatively the replication was successful. Removing the marginals is the lesion that increases performance the most. Magnitude correlation comes in second place. All of the other conditions are at chance level (as far as my sample n=60 with 60 trials each can discern).

As you can see, this specific paradigm is now much more robust than before. And the paradigm is not hard to translate for application in psychedelic research. Unfortunately, Portilla & Simoncelli’s algorithm only creates texture metamers for the peripheral vision in pre-attentive conditions. Upon central inspection, nearly every synthesized texture is at least somewhat distinguishable from the original texture.

The more interesting problem, as I see it, is found in our capacity to see differences between textures upon close and careful examination. This, I think, addresses more directly the subject of this blog. Namely, what is the information that we can represent and distinguish at the (resolution) peak of human consciousness? I imagine that there must be statistical features that we simply do not perceive even when we are looking at them directly and using all of our attention. What is the set of properties that our everyday consciousness can represent centrally?

Psychedelic Research

To understand how a machine works, it helps to know what happens when you break it. You can’t ignore the extra settings and claim you’ve got a theory of everything. Would we be satisfied with the work of neuroscientists and psychologists if they only studied people who are colorblind? They may claim that they have “the essentials” of vision. That color is just a perturbation in the “optimal basics.” And yet, today we know that color plays a relevant computational role in visual discrimination. Suffices to mention that people with grapheme-color synesthesia have an improved performance in “odd one out” identification tests like this (find all the 2’s as fast as possible):


This is because fast low-level association between graphemes and colors help them see clearly and quickly the graphemes that are different. Likewise, every variety of conscious experience may potentially play a computationally relevant role in specific situations. A priori, no variety of consciousness can be dismissed as irrelevant.

Thus, to understand, model and engineer consciousness, we should not prematurely close varieties of consciousness to study. Not only would that prevent us from finding out about consciousness more generally. It may also conspire that we will never understand even what we do decide to study, simply because key pieces of the puzzle are found elsewhere.

Another point is that even though sober human vision is a special case of general vision (and general vision-like qualia), general vision may still be relatively small. The general principles and conditions for vision to work in the first place may be somewhat restricted. Unless the elements are placed just right, the visual system breaks down, at least when it comes to fulfilling a computationally meaningful purpose. Thus, psychedelic research of visual experience may help us quickly distinguish what is essential from what is incidental in vision.

By introducing a psychedelic substance into the nervous system of a person, a remarkable set of visual effects are produced. To anyone interested in seeing a good representation of the way in which various psychedelics affect a person’s conscious experience I highly recommend seeing Disregard Everything I Say’s entry on the visual components of a psychedelic experience (while you are at it, I recommend also checking out his/her post on the corresponding cognitive components of a psychedelic experience). These images will provide a great intuition pump about the kind of beast we are facing to anyone who is psychedelic-drug-naïve (and hopefully inspire a sense of “WOW, you can represent some of what you experience after all!” in those who are less psychedelic-drug-naïve).

Given the same visual input presented to a particular person, a psychedelic substance will in many cases drastically shift the interpretation of such stimuli. Both the interpretation (specifically, what an image is about) and how the image looks and feels tend to vary in synchrony. Likewise, people feel more able to see personal issues in a new light by interpreting them with new schemas and from a different level of awareness. Personally, I suspect that there is a strong and measurable connection between the fluidity of interpretation of personal issues, and the fluidity of interpretation of visual experience. Perhaps all phenomenal constructs are affected in a similar way: By breaking down the previously enforced patterns of thought and perception and opening the way to seeing things differently.

All of the above, while probably true, is still far too vague for a scientific theory. Given our set of tools, and experimental paradigms, to me it makes a lot of sense to start studying the effects of psychedelics in terms of texture perception. As we develop more and better texture analysis/synthesis algorithms, we will acquire a larger repertoire of mathematical properties that describe what is seen during a psychedelic experience. My hope is that we will someday know exactly how to simulate a visual trip.

Why care about psychedelics? Evolution already created the optimal vision system!

Evolutionary selection pressures on perceptual systems do not guarantee that information processing tasks will be solved optimally. In fact, “optimal” only really makes sense in relation to some metric you choose. In some way, everything created by the evolutionary process will be optimal in the sense that is produces the local maximization of inclusive fitness (admittedly it’s more complicated than that). But this is just a tautological notion: Sure evolution is optimal at doing what evolution does. Likewise, rocks have found the optimal way to be themselves. Our visual system works optimally, if you define optimal in terms of 20/20 every-day visual experience.

Instead, it makes more sense if we focus on the specific computational trade-offs that various resource allocation methods and designs provide. We can certainly predict that the particular set of algorithms that our visual system employs to detect visual patterns will satisfy some properties. For instance, they will be good as survival tools in the African Savanna. But just as for our hardwired tastes in food (and our default emotional palette), survival value in the ancestral environment is not necessarily what we currently need or want. And just as it would make sense to modify what we enjoy eating the most (moving away from sugar) to adapt to the current post-industrial environment, it may also turn out to be the case that our visual system is miscalibrated for the tasks we want to solve, and the joys and meaning we would like to experience today.

Psychedelics change our visual system in many ways, some of them more predictable than others. Some people report that small doses of psychedelics increase one’s overall visual acuity (this has yet to be verified empirically). Although counter-intuitive at first, this cannot be ruled out, again, simply because of evolution does not rule out things of this nature. After all, one of the main constraints placed on animals in natural environments is caloric consumption. If a higher visual acuity is possible with your current brain, the marginal benefit such acuity adds may still not surpass the marginal loss of calories that result from excessive brain activity.

Additionally, while higher doses of psychedelics tend to impair many aspects of visual perception (with people on Erowid explaining how extreme tracers can make it hard to walk), low and moderate doses do not have simple one-sided effects even when it comes to accurately representing the world around us. Rather than simply breaking up the pattern recognition capabilities of the visual system, small and moderate doses seem to also open up additional kinds of patterns up for inspection. Pareidolia, for example, is greatly enhanced. Thus, “connecting dots and edges” to match the outline of higher-level phenomenologies (like faces in the mud) happens with ease. Whether this is good or bad depends on the context, and the specific task one is trying to solve.

Perceptual Enhancement via Psychedelics

We have theoretical and anecdotal reasons to suspect psychedelics may turn out to be performance-enhancing in certain visual tasks. We also already have quantitative evidence that this is the case. In the 60s Harman and Fadiman conducted a study about the creativity and problem-solving properties of psychedelics. The study included a paper and pencil component in which the following tests were administered: Purdue Creativity, the Miller Object Visualization, and the Witkin Embedded Figures. The authors conclude that “[m]ost apparent were enhanced abilities to recognize patterns, to minimize and isolate visual distractions, and to maintain visual memory in spite of confusing changes of form and color.” Specifically, the Witkin test is singled as a test in which the ingestion of mescaline produced a consistent performance improvement (p<0.01).

Example of a Witkin Embedded Figure. You have to determine if the left shape is in the right figure within 30 seconds.

Example of a Witkin Embedded Figure. You have to determine if the left shape is in the right figure within 30 seconds.

In personal communication Fadiman has said that these tests were mostly a waste of time: They used one valuable hour of the problem-solving session. I disagree. After 50 years, those results are incredibly valuable to me and the next wave of computational researchers of the mind. I think that the recorded performance enhancement is a key piece of information. We certainly do not expect an enhancement on all areas of cognition and perception (we know, for example, that reaction time and verbal fluency are impaired). Identifying the kinds of tasks that do receive a boost can inform future research. In future posts I will explain my theory for why Witkin Embedded Figures test was particularly benefited from a psychedelic, and how we can create a new test that takes this into account.

New Possible Protocols for Psychedelic Research

Nowadays it is very hard to obtain the required permits and affiliations to conduct academic research on psychedelic consciousness. The key rate-limiting factor is the restrictions that apply to research with controlled substances. Thankfully, we do happen to be living at the beginning of a psychedelic renaissance. It is not hard to imagine that as psychedelics start to (re-)enter the mainstream in psychiatry, a large number of clinical trials will be conducted. In principle, a collaboration can be accorded between a psychophysics lab and a clinical research group to conduct psychophysical assessments during the course of treatment.

Clinical trials for medical applications of psychedelics, though, are still limited in scope and focus. Eventually the medical applications of psychedelics will run out and it will no longer be possible to enroll patients into psychophysical studies. Thus, either more permissive rules and regulations will emerge and society’s rules will be more lax, or we will potentially endure decades or centuries of unnecessary barriers to scientific discoveries about consciousness.


As it turns out, tens (if not hundreds) of millions of persons have tried psychedelic substances. Interest is not slowing down, and no propaganda effort save for literal brain-washing will prevent people from developing a genuine interest in the field. There will be millions of volunteers and hundreds of thousands of willing researchers. As I envision in The Psychedelic Future of Consciousness, the future of consciousness studies will be nothing like what it is today. Once those who are interested are given the opportunity to study psychedelics seriously, the research panorama will look very differently.

But What to Do in the Meantime?

Investigating the psychophysics of texture perception more thoroughly may require at least mild to moderate doses for noticeable effects, and a period of examination at a time when acute effects are present. How to overcome the hurdles ahead of any current academic investigation of this nature? This ultimately depends on the specific constraints required for the study. It is true that people would have to be high on LSD (or Mescaline, mushrooms, 2C-B, etc.) while they perform the experiment. But who says that they have to conduct the experiment in your presence? That you have to give them the substance yourself? Without the typical background assumptions that are assumed in psychophysics research labs, can we do anything else?

I suspect that we can do much better. We can come up with protocols that side-step current obstacles. We need to be creative. And I do have some ideas, with varying levels of plausibility for how to implement psychedelic research in legal, viable and immediate ways. Consider, for example, how James Fadiman has been collecting hundreds of micro-dose reports via email without any difficulty for many years. He is taking advantage of the fact that the optimal format of micro-dose reports tends to be “summary and retrospective” narrative. A single dose on a single day on a single person is not likely to be life-changing. So his particular line of research is well suited for the means available to him. Likewise, as I requested in the psychedelic experience of visual textures post, people’s subjective judgements of visual textures while high on LSD can be recorded online. These are just two examples of how non-mainstream research approaches can be taken to study psychedelics. Unfortunately, both of these protocols lack proper controls and standardized settings. But this is all I will say for now.

Stay tuned: In a future post I will propose a set of protocols for independent studies on psychedelics, including additional methods for studying psychedelic visuals.

Thanks for reading! 

If you would like to be a collaborator with me, please email me by finding my contact info in the contact section of this blog. We’ll take it from there.

Note: If any link is broken, please leave a comment and I’ll provide an updated version. Thanks!

Psychedelic Perception of Visual Textures

On March 24th 2015 Team Qualia Reverse Engineering (TQRE) went for a long walk within the Stanford campus and around Palo Alto. The purpose of this walk- the Pattern Walk -was to snap a picture of every interesting pattern (or texture) out there that got on our way. The following gallery contains 74 of these patterns. These display a wide range of texture properties: Natural/synthetic, regular/irregular, 2D/2.5D/3D, symmetric/asymmetric, structured/unstructured, etc.

Here are a few observations: 

  1. Human languages do not have the necessary vocabulary (and conceptual primitives) to talk about visual textures adequately. When two images belong to the same category (say, “plants” vs. “rock tilings”), and have roughly similar first order statistics (mean, standard deviation, kurtosis, etc. of the RGB values) there is relatively little else to say about a texture in a way that a person would understand.
  2. Our visual system can recognize extraordinarily subtle properties that distinguish textures from one another. For instance, I bet you can recognize at an immediate experiential level the differences between picture 61 and 62. But can you verbalize such difference?
  3. Mathematics, and statistics in particular, may provide helpful semantic seeds for describing patterns. Indeed, having a basic handle on a few mathematical concepts can leverage one’s ability to talk about the differences between textures. For example, compare images 43 and 44. They are perceptually very different. But how long would it take you to convey the difference to a random person? If there was a person who could only hear you, how would you signal that you are not talking about 43 but 44? If both of you know of the concept of concavity you might only need a few words! Without it, you’d be fairly lost.

Fancifully, we may someday produce a good vocabulary that can effectively allow us to talk about visual textures without having to be currently sharing the same (similar) visual experience.

In practice, we already have some vocabulary that accomplishes this, but it is very obscure and sufficiently technical that its widespread adoption is unrealistic. In particular, I encourage anyone interested in the topic to read “A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients” by Javier Portilla and Eero P. Simoncelli. They analyze (and synthesize) visual textures by computing a set of highly descriptive statistical properties characteristic of the pattern in question.

As we will see in future posts, their model can be used to point out perceptible statistical features that are perceived as regularities by the human visual system. It may not be sexy to say “Hey Ma’m I really dig the Cross-scale phase statistics of the pattern in your dress.” For now, that’s what we have.

If you want to help me figure out how psychedelics affect your visual experience:

Please browse through these images by clicking on the first one and exploring the slideshow. See which images you like, which produce “odd or interesting visual effects” and which “stand out” in however way you want to define that. Feel free to comment right below any of the images (there is a comment section beneath each image when you click through them as a slideshow) to point out the peculiarities that you notice.

Critically, also include your state of consciousness in the comment. If you took LSD (or any visually-affecting substance) two hours ago (or you are still high), it would be great if you could point that out. Please explain how you think that your visuals are affecting your experience of the various patterns. Everyone loves to talk about their LSD visuals. Now you can do it all you want! And your efforts may actually enable us to understand the way psychedelics affect the algorithms of human vision 🙂

The best case scenario:

You would make comments on these images while sober, and then add comments while high on a psychedelic (doesn’t have to by psychedelic – could be dissociative, though typing might be particularly hard in that condition). Point out the main differences between the textures as perceived on each of the states of consciousness you happen to be in. If you do decide to follow the above protocol, please provide information about the specific substance(s) you consumed and how long ago you did so.

That is, do this if you were planning on taking a hallucinogen to begin with. Independently of that, baseline data is still very valuable, so do add comments about these patterns even if you are sober and plan on staying sober 🙂

In the following post I will explain how this Pattern Walk, the statistical analysis of visual textures, psychophysics and psychedelics can ultimately fit into the larger project of reverse-engineering the computational properties of consciousness.