Harmonic Society (2/4): Art as Schelling Point Creation and the Pursuit of Sacred Experiences

The following essay was recently published in the Berlin-based art magazine Art Against Art (issue). Below you will find models 3 and 4 (out of 8). I will be sharing 2 new models each week until I’ve shared all of them (see part 1/4).


3. Schelling Point Creation

[Psychoanalysis teaches us:] When somebody complains, always be careful and try to find, identify, what type of additional pleasure, satisfaction, does the act of complaining itself bring to you. We all, when we complain, almost always, find a perverse satisfaction in the act of complaining itself.
– Slavoj Zizek (2019)

I certainly feel compelled to complain about the tyranny of genetic fitness signaling in art. That said, people who excel at games who are not played by many people will have an incentive to undermine the popular games and frame their favorite game as somehow superior. Why are Hipsters and Nerds allied against Cool Kids? Because the Cool Kids can decide on a whim that the games the Hipsters and Nerds play are uncool and not worthy of public fitness displays. Even if they happen to be of superb quality!

In many cases, the exploration of uncommon games can give rise to major innovations, so there is a utilitarian reason to promote some degree of exploration outside of the aesthetics that most people can enjoy.

This line of reasoning gives rise to a new interpretation for what a Hipster is. To be a Hipster is not, as popularly believed, to merely desire the uncommonly desired. The whole thrust of hipsterism is a promise of superior quality in at least some actually relevant area, even at the cost of severely reduced quality across the board. (Using an analogy from the field of statistics: Cool Kids favor L2 normalization[1] as it signal-boosts people who are well-rounded, whereas Hipsters and Nerds favor L1 normalization which improves the outlook for imbalanced minimax strategies).

Many people believe that all Hipsters are Cool Kids. Many believe something slightly weaker, which is that to be a Cool Kid you also need to be a Hipster. But in fact this is absolutely not the case, and it is a category error to think otherwise. Cool Kids and Hipsters were correlated when being Hipster had mainstream appeal. That is, Hipsters were cool when Cool Kids used to challenge people to show how Hipster they could be. But this should not be in any way an indication that Hipster aesthetics are intrinsically related to Cool Kids, for the same reason that e.g. Country Music, Normcore, or Bolshevik aesthetics are not intrinsically invented by Cool Kids. Hipsters are individual contributors to the frontier of culture. Indeed, it is rare to find a place that produces genuinely innovative content while also being saturated with Cool Kids.

Cool Kids, in large quantities, eventually form cliques that become voting blocs. These frustrate innovation by fully orthogonalizing what is socially cool from what is socially valuable. A Hipster under those circumstances tends to feel stifled. Cool Kids tend to be above-average in openness to experience, but they are rarely in the top 2% of openness to experience – more like one standard deviation above the mean. This is because they need to be open enough to look at new trends but also sufficiently closed to be able to relate to the bulk of the consumers of new trends. Genuine Hipsters are usually above the 98th percentile of openness to experience. In turn, the sexual attraction of some people is focused on this particular trait, and Hipsters compete at signaling it to the highest extent possible. In the process, they discover interesting things. But this does not mean they can sustainably stay cool in the eyes of the average person.

High openness to experience allows you to appreciate minimax players. It allows you to accept artists who are ridiculously good at making a specific point but lack talent in every other respect. Ultimately, the innovations produced by these extreme artistic explorations sometimes radically transform social reality.

In “Ads Don’t Work That Way”, Kevin Simler discusses how advertisement’s power is not through direct persuasion, but through shaping the landscape of cultural meaning. You don’t bring a 6-pack of Coronas to a party because the ads have subconsciously conditioned you to think that this beer in particular is more likely to make you and your friends feel like you are a chill group. Rather, you buy it in order to signal the fact that you see yourself as a chill person, and to bring that mindset to those who see you bring the product. It is by virtue of common knowledge that ads can do this; if every single person received a different custom-made AI-powered neural net ad, ads would stop having the function of shaping the landscape of cultural meaning, and perhaps lose a significant portion of their power.

Art, likewise, can also change the landscape of cultural meaning. In contrast to ads, art might perhaps be described as high-bandwidth low-distribution as opposed to high-distribution low-bandwidth. And to the extent that Hipsters discover new aesthetics, they are a big source of novel cultural Schelling points for subcultures to form around.

4. Creating Sacred Experiences

Art could be the next religion – Alex Grey

Below you will find an example of a piece that aims to create a sacred experience, which I recently encountered at the Santa Cruz Regional Burn. It is called Mementomorium, and it is a mixture of a sensory-deprivation-chamber and a symbolic self-burial experience crafted in order to simulate your own death and to attempt to see your life in its finitude. This art piece plays with one’s experience of time and sense of mortality, and helps you cut through delusion in order to re-interpret one’s time on earth as finite and priceless.

Why is the above art? Cool Kids might find this too morbid, and Hipsters are likely to see
it as too real. So what is the thrust behind artistic visions like the above?

Sacred experiences are an aspect of social and phenomenological reality. Art, it turns out, is deeply entwined with such sacredness. Now, much has been said about the sublime in relation to art. What else is there to say?

Life isn’t about finding yourself. Life is about creating yourself. – George Bernard Shaw

Contrary to the three previous models, here the culminating emotion that is sought is not the vindication of self, but rather, the elicitation of a sense of self-transcendence. This 4th model would say that art creates some of the most valuable experiences there are, because it makes us experience a sense of transcendence. And relative to the previous three models, this model is the first to consider art as involved in the quest of finding the ultimate answer, as opposed to merely providing incremental benefits to humanity.

Cutting to the chase, let us jump right into a list of possible intentional sources for phenomenal sacredness (i.e. the possible targets of art according to this model). From John Lilly’s “Simulations of God”, below you find the most common types of self-transcendence catalogued:

  1. God As the Beginning
  2. I Am God
  3. God Out There
  4. God As Him/Her/It
  5. God As The Group
  6. God As Orgasm and Sex
  7. God As Death
  8. God As Drugs
  9. God As the Body
  10. God As Money
  11. God As Righteous Wrath
  12. God As Compassion
  13. God As War
  14. God As Science
  15. God As Mystery
  16. God As the Belief, the Simulation, the Model
  17. God As the Computer
  18. God Simulating Himself
  19. God As Consciousness-without-an-Object
  20. God As Humor
  21. God As Superspace, the Ultimate Collapse into the Black Hole, the End.
  22. The Ultimate Simulation
  23. God As the Diad

According to John Lilly’s view, each of us lives in a world simulation (whether this is generated by our brains or by a higher power is something Lilly himself went back and forth on for decades). He makes the case that our world simulation is run by a hierarchical chain of programs and meta-programs. One’s locus of control[2] is what he calls the Self Meta-Programmer, which is roughly equivalent to the ego (or at least a healthy one with high levels of self-control). Implicitly, however, the Self Meta-Programmer is subordinated to something higher, something he calls the Supra-Self Meta-Programmer (SSMP for short; see: “Programming and Metaprogramming in the Human Biocomputer”).[3] Our SSMPs are responsible for our notions of a higher power, higher values, and higher purpose. One’s religion is determined by the SSMPs to which one is subordinated. In Lilly’s view, it is one’s SSMPs that give rise to one’s understand- ing of God. And as the list above shows, there are many possible versions of God. That is, there are many possible meta-programmings for what the highest power, value, and purpose might be. In light of this, art as the pursuit of sacred experiences would not be restricted to a particular view of God. Rather, it encapsulates every possible notion of God – where the art that hits hardest is the art that resonates the most with one’s implicit conception of God.

A parallel here could be made with adult developmental models (such as those of Wilber’s Integral Theory, Kegan’s Evolving Self, Common’s and Richard’s Model of Hierarchical Complexity, etc.). At each level of development, one’s conception of the highest value transcends and includes those of the developmental stages below. Let’s take for example Integral Theory’s levels 4, 5, and 6. Level 4, aka. “Amber” (ethno- or nation-centric, values rules, discipline, faith in transcendent God or preordained high- er order, socially conservative, etc.) would derive a sense of sacredness from religious imagery, a nationalist spirit, and art that fosters traditional values. Level 5, aka. “Orange” (values science and rationality, democracy, individualism, materialism, entrepreneurship, etc.) gets off on experiences that bring about a reductionist scientific world picture compatible with self-reliance (“the world is made of atoms, and this, rather than being tragic, is an opportunity to have fine-grained control over the elements”). And Level 6, aka. “Green” (values pluralism and equality, multiple points of view, no true reality, embraces paradox, considers civil rights and environmentalism to be the frontier of culture, etc.) would find art projects that highlight the multiplicity of perspectives to be key to a sense of the sacred.[4] In this framework we can explain people’s negative reaction to art as a misfit between the developmental level of the target audience and the developmental level of the person who gets to experience it. Art targeted to people in a higher level of development than oneself will be perceived as heretical (e.g. postmodern art from the point of view of a traditionalist monotheist), while art targeted to people on a lower level of development than oneself will be perceived as childish or naïve (e.g. traditional religious iconography from the point of view of a scientific rationalist humanist). We could thus predict that if there are even higher developmental levels above ours, we will most likely think of the art targeted to them as deeply troubling.

The core quality of the experience is the feeling and recognition that oneness is truth. – Martin Ball on 5-MeO-DMT

At the upper levels of development, one could argue, we find sacredness based on concepts like pure consciousness, emptiness, and the clear white light of the void, etc. Famously, psychedelics, and in particular 5-MeO-DMT, seem to trigger direct experiences of this type of sacredness, which, according to its proponents, encapsulates all other kinds of transcendence within. If this is so, then we could anticipate that agents like 5-MeO-DMT will play an important role in the future of art as more people climb the ladder of adult psychological development.

On a social level, art as the pursuit of the sacred can be interpreted as an adaptive behavior aimed at taming envy. “Keeping up with the Joneses” is (artistically or other- wise) capable of diverting a group’s energy away from tasks that need to be done for individual and collective survival. When done in excess, wasteful displays of fitness make communities suffer. Runaway signaling has serious drawbacks, and sacred experiences seem to calm people down a bit, especially if the sense of sacredness comes along with social reassurance in the form of being able to hang out together without having to compete all the time, for Christ’s sake! Ahem. To be chill with one another.

As we saw with the previous models, this one, too, has its own aesthetic. The aesthetic of the model would perhaps manifest in the form of a museum that caters to every possible sense of sacredness. From aboriginal shamanism to monotheistic conservativism to punk rock concerts to transhumanism, this aesthetic recognizes the fact that sacredness is catalyzed by many different inputs depending on the psychological traits of the people who consume it.



[1] L1 and L2 normalization are ways of talking about how to describe the distance between points in a given space. L2 takes into account the mean squared difference along each dimension, whereas L1 simply uses the average difference in each dimension. If one is thinking about an ideal art piece within a given aesthetic, then using L2 would penalize very heavily exemplars that deviate from the archetype and generally favor well-roundedness, whereas an L1 normalization would accept large differences from the ideal along several dimensions as long as at least a fraction of the dimensions are very good.

[2] One’s locus of control is the part of our experience that comes with a felt sense of agency. That is, what feels like is in charge of determining the direction of one’s attention, intention, and behavior. Typically, a person’s locus of control is tied to their sense of self – or ego – but this is not true in the general case (as demonstrated by the shattered locus of control present in schizophrenia, and absent locus of control during states of depersonalization and derealization).

[3] According to John Lilly, a Supraself-Metaprogramer is an agent outside our locus of control that runs below our threshold of awareness and which ‘codes’ Supraself-Metaprograms. In turn, Supraself-Metaprograms are the mental “programs” that determine our sense of the highest values, which we typically inherit from our culture, influence from others, implicit historical beliefs, and so on.

[4] The colors of Integral Theory: Ken Wilber’s Integral theory was developed by identifying the commonalities among many different types of adult developmental models, spiritual stage maps, and meditation progression systems. The progression could broadly be described as a generalized expansion of the circle of compassion and increased acceptance of complexity. The color associated with each level is arranged from low-frequency to high-frequency parts of the spectrum. Specifically, infrared – archaic, magenta – tribal, red – warrior, amber – traditional, orange – modern, green – postmodern, teal/turquoise – integral, ultraviolet – post-integral.


Featured image credit: Mementomorium by Oleg Muir Lou Goff

* The full essay’s title is: Harmonic Society: 8 Models of Art for a Scientific Paradigm of Aesthetic Qualia

More Dakka in Medicine

By Sarah Constantin (blog – 1, 2)

The More Dakka story is common in medicine. You do an intervention; the disease doesn’t get better, or gets only marginally better; the research literature concludes it doesn’t work; nobody tries doing MORE of that intervention, but when somebody just raises the dose high enough, it does work.

Examples:

a.) Chemotherapy didn’t work on cancer until doctors made cocktails of drugs, raised the dose so high it would kill you, and then mitigated the side effects with prednisone and intermittent dosing schedules. If they just used a safe daily dose of a single chemotherapeutic agent, they’d have concluded chemo didn’t work.

Prednisone-2D-skeletal

Prednisone

b.) Light therapy barely works for SAD; two internet-famous people have independently found that REALLY BRIGHT light therapy completely fixes SAD.

c.) The example in the post is about allopurinol. Allopurinol prevents gout attacks by lowering uric acid. “In studies, [allopurinol] improved [uric acid] linearly with dosage. Studies observed that sick patients whose [uric acid] reached healthy levels experienced full remission. The treatment was fully safe. No one tried increasing the dose enough to reduce [uric acid] to healthy levels.

d.) The standard treatment for hypothyroidism is thyroid hormone. People with “subclinical hypothyroidism”– people whose thyroid hormone levels are lower than average, but still above the cutoff for hypothyroid, and still suffer from exactly the same symptoms as hypothyroid–, ALSO benefit from thyroid hormone therapy. It’s not standard of care yet, though.

e.) I believe some vitamin deficiencies, don’t remember which exactly, are the same way; there’s an official cutoff for “deficient” but people slightly above that cutoff still have symptoms and still experience symptom relief from supplementation.

f.) Same deal with HIV. Virus has a replication rate & a clearance rate; its replication rate is also its mutation rate; an antiviral drug can raise the clearance rate above the replication rate, which will make the population drop exponentially, but if there’s only one drug the virus will have a chance to evolve to be resistant before the population drops low enough to be undetectable. And this is a simple differential equation that you can calculate years before you know what the drugs even are. One drug: death. Two drugs: death. Three or more drugs: survival.

Luckily David Ho was a physicist and thought about it this way, so when the antiviral drugs came out he was ready to test them in cocktails.

So “single antibiotics don’t work for chronic Lyme but cocktails do and this wasn’t realized for decades” isn’t an unprecedented story. It could turn out that way.

I bet this is something that has a more formal and accurate phrasing, but: if there’s an exponential-growth dynamic (like in a malignant cancer or an infection) where you’re trying to kill the exponentially-growing population, and if there’s a dose-response relationship where higher dose = more killing, then you have a bifurcation point in the outcome as t -> infinity, where a dose below that point means the enemy takes over and the patient dies and a dose above that point means “the enemy is killed faster than it can reproduce and so dies out in the long run.” And in principle you can calculate this cutoff if you know the dose-response relationship, as Ho did.

And separately, there’s a safety threshold; is the minimum effective dose safe or unsafe? With chemotherapy, the minimum effective dose is UNSAFE, which is why they have to get clever with ways to give you doses high enough to kill you while keeping you alive anyway. (Or “find a better drug”, but nobody has found a cytotoxic drug with strictly better tolerability/effectiveness tradeoffs since the 1960’s.)

This is kinda how you get a continuous/analog system to give you discrete outcomes: bifurcation points! Works in gene regulation too. “This regulatory gene turns on that gene’s transcription” – well, what’s actually happening is a continuous scalar, a rate of transcription and a rate of clearance, but because exponential functions are involved you get bifurcations in “steady-state” outcomes over the several-hour timescales needed to get to “this cell has tons of mRNAs for that gene or it’s literally empty of them”.

Systems biology is cool, it explains the math that gets you from a statistical-chemistry model of the cell (as a bag of molecules that bump into each other and have a probability of interaction) to a tinkertoy model that you can treat like a graph. (Gene regulatory networks, protein-protein interaction networks, neuron networks, etc.)

Why Care About Meme Hazards and Thoughts on How to Handle Them

By Justin Shovelain and Andrés Gómez Emilsson

Definition

Nick Bostrom defines an “Information Hazard” as: “A risk that arises from the dissemination or the potential dissemination of (true) information that may cause harm or enable some agent to cause harm.” A more general category is that of “Memetic Hazard”, which is not restricted to the potential harms of true information. False claims and mistaken beliefs can also produce harm, and should thus also be considered in any ethically-motivated policy for information dissemination. 

Introduction

Perhaps one of the best known analysis of meme hazards is the work of Nick Bostrom concerning: Information Hazards, the Unilateralist’s Curse, and Singletons. His focus could roughly be described as one of classifying the types of situations that can give rise to information hazards. A parallel set of problems to that of categorizing memetic hazards is the problem of coming up with policies for dealing with them, and the problem of convincing people that they should care. In this post we suggest some basic heuristics for dealing with meme hazards, and explain why you should care about them even when your work seems unambiguously positive.

Motivation

Why You Should Care

A big problem with getting people to engage with any kind of memetic hazard policy is that it may be perceived as a voluntary constraint on one’s behavior with little to no personal benefit. Nobody (well, at least nobody we know*) gets excited about compliance training at a new job, or inspection day at a manufacturing facility. Subjectively, most people perceive compliance and oversight as something that gets in the way of doing one’s work and as a hassle for one’s organization. That said, there is reason to believe that as the world’s technologies become both more powerful and more widely accessible, that there will be increasingly more dangerous information around. Considering the possible downsides of sharing information will thus become increasingly more important. So at least on a global scale, it will be increasingly more important for people to consider the impact of the information they choose to share. But at an individual level, why would they care about meme hazards policies and not think of them as a bothersome constraint?

Just like there are actions that can help or harm there are ideas that can help or harm. Furthermore, some ideas produce their primary good or bad effect through social transmission, which we can call memes. There are several ways to prevent the harm from memes: not producing them in the first place, not sharing them, or fixing the situation so that when dispersed they do not do damage (before or after dispersal). Let’s call policies to prevent harm from meme hazards, meme hazard policies. Because in a world with increasingly accessible technological power a lot of our largest effects are likely to be produced by memetic hazards, a good way to improve the chances of achieving one’s goals is to tilt things as much as possible towards our goals with good meme hazard policies. It thus makes sense to read works about meme hazard policy and to think about how it bears on one’s work. This way you can improve your implementation and design of meme hazard policies to avoid hampering your own goals. In particular, assuming that you are a rational agent (who both attempts to be epistemically and instrumentally rational) you will generally find that spreading dangerous information that causes large negative effects (even if by accident!) will interfere with your ability to carry out your own goals.

Why Good Work May Have Bad Net Effects

When one engages in very novel research one should be careful to consider the ratio with which one’s work advances desired outcomes relative to undesired outcomes. This may yield surprising results for the net effect of one’s work, sometimes flipping the net effect of research that at first may have seemed unambiguously good. For example, Artificial Intelligence Alignment research may in principle increase the chances of unaligned AI by virtue of providing insights into how to build powerful AIs in general. If it is 100 times harder to build an aligned AI than an unaligned AI, and researching AI alignment advances the goal of building unaligned AIs by more than 1/100 relative to how it advances building aligned AIs, then such research would (counter-intuitively) increase the chances of building unaligned AIs relative to aligned AIs.

As another example of how seemingly good work may have bad net effects let’s consider how information mutates in a social network. As discussed in previous articles such as consciousness vs. replicators there is no universal reason why causing large effects and causing good effects have to be correlated (see also: Basic AI Drives and Spreading happiness more difficult than just spreading). With an evolutionary view, it becomes clear that memes that are good and beneficial to everyone can eventually evolve to become bad and harmful to everyone if by doing so they gain a reproductive edge. As a rule of thumb, you can expect ideas to mutate towards:

    1. Noise due to generation loss
      1. Unless your copying method is perfect or has error correction methods, every time you make a copy of something the information will degrade to some extent. This is called generation loss and it leads to more noisy copies over time.
    2. Simplicity
      1. Since information transmission incurs a cost, simpler mutations of the meme have a reproductive edge.
    3. Ease of memorization and communication
      1. Mutations to the memes that are easier to memorize and communicate are more likely to spread.
    4. Inciting arms races
      1. If the meme provides a competitive edge in a zero-sum game, it may give rise to an arms race between agents who engage in such zero-sum game. For example, a new marketing method discovered by a given agency would force other marketing agencies to invest in researching how to achieve the same results. Since the rate of evolution of a meme is partly determined by the rate at which iterations over it are performed, a lot of memetic evolution takes place in arms races.
    5. Saliency (cognitive, emotional, perceptual, etc.)
      1. Saliency refers to the probability of noticing a given stimuli. Memes that mutate in a way that makes them more noticeable have a reproductive edge. Thus, many memes may over time acquire salient features, such as causing strong emotions.
    6. Uses for social signaling (such as used for signaling intelligence, knowledge, social network, local usefulness, etc.)
      1. Consider the difference between manufacturing a car that focuses exclusively on basic functionality and a car that in addition also signals wealth. Perhaps it would be better if everyone bought the first kind of car because the second kind incites the urge in others to get a new car more often than necessary. Namely, people might want to buy a new car whenever the neighbors have upgraded to a more luxurious car (see: Avoid Runaway Signaling in Effective Altruism and Keeping up with the Joneses).
    7. Overselling
      1. As a general heuristic, memes will spread faster when they are presented as better than they really are. Unless there is a feedback mechanism that allows people to know the true value of a meme, those that can oversell themselves will tend to be more common relative to those that are honest about the value they provide.
    8. Usefulness
      1. The usefulness of a meme increases the chances that it will be passed on.

Given considerations like the above, it’s clear that in order to achieve what we want we need to  think carefully about the possible impacts of our research and efforts, even when they seem unambiguously positive. Now, when should one give special thought to memetic hazard policies?

When Should You Care the Most?

meme_hazard_action_space

Meme Hazard Action Space – Worry when the ideas are both novel and have the potential to have large effects

There are two key features of potential memetic hazards that should be taken into account when thinking about whether to pursue the research that is bringing them to life. 

The first one is how large their effects may be, and the second is how novel they are. How large an effect is depends on factors such as how many people it may affect, how intense the effects would be on each person affected, how long the effects would last, and so on. How novel a meme is depends on factors like how many people know about it, how much specialized knowledge you require to arrive at it, how counter-intuitive it is, and so on.

No matter how novel a piece of information may be, if it does not have the potential to cause large effects we can disregard it in the context of a meme hazard policy. When the potential to cause large effects is there but the idea is not very novel, then one should focus on actions to mitigate risks. For instance, if everyone knows how to build nuclear bombs, then the real bottleneck to focus as a matter of policy would be on things like the accessibility to rare or expensive materials needed to build such bombs.

But when the information is both novel and can cause large effects, then the appropriate focus is that of a meme hazard policy based on strategies to handle information dissemination.

Examples

Ignore:

  • What you had for breakfast, yet another number sorting algorithm, how to get the hair of a cat to be more fluffy

Focus on ideas:

  • A more efficient deep learning technique, a chemical to improve exercise response efficiency, a new rationality technique, information on where the world’s biggest tree is

Focus on actions:

  • The idea of guns, the idea of washing hands for sanitary purposes, running an Ayahuasca retreat in the amazon

Suggested Heuristics

yes_no_diagram_3

Suggested Responses

To wrap up, here we provide a very high-level set of suggested heuristics to consider if one is indeed discovering ideas that are both very novel and capable of producing large effects:

  • Develop
    • Develop if you conclude that there is no risk
  • Share
    • Share if you conclude that there is no risk
  • Log your analysis and proceed
    • Store the results of your analysis for future use by others who may overlook the risks and then continue developing or sharing it
  • Think more about it
    • Conclude that it would be valuable to analyze the risks of the meme (e.g. a new technology) further
  • Develop cure
    • Develop a cure of the meme hazard’s downsides
    • This approach may entail selectively sharing the information with people who are highly benevolent, good at keeping secrets, and capable in the relevant domains of expertise
  • Improve the groups that receive it so that it is safe
    • Some information is only risky if certain types of groups get it, so if you change the nature of the groups then there is no risk
  • Framing it so it goes to the right people or only yields good effects
    • The way an idea is posed or framed determines a fair amount of who will read it and how they will act on it
  • Selecting a safe subset to share
    • When you have information it could be that some parts are good or safe to share and you can selectively share those parts
    • Make sure those parts are not sufficient to reconstruct the original (unsafe) information
  • Selecting a safe subset to develop
    • When developing some information it can be that some parts are good or safe to develop and you can selectively develop those parts
  • Selectively share to a subset of people
    • Some information is only risky if certain types of groups get it; if you can aim where the information goes you can avoid the risk
    • Report the information to proper authorities
  • Don’t develop
    • Some information is too risky to develop
  • Don’t share
    • Some information is too risky to share
  • Monitor to see if others move towards developing or sharing it
    • If you’ve identified something risky it may make sense to see if others are developing it or likely to share it so that you can warn them, focus on building a cure, contact authorities, or start changing your actions knowing that a disaster is likely. 
  • Try to decrease the likelihood of rediscovery
    • If it’s really risky you may want to see what you can do about decreasing the likelihood that it is rediscovered

Conclusion

In this post we discussed why you should consider following heuristics to deal with meme hazards as an important part of achieving your goals rather than as a chore or hassle. We also discussed how work that may seem unambiguously good may turn out to have negative effects. In particular, we mentioned the “ratio argument” and also brought up some evolutionary considerations (where memes may mutate in unhelpful ways to have a reproductive edge). We then considered when one should be especially cautious about meme hazards: when the information is both highly novel and capable of producing large effects. And finally, we provided a list of heuristics to consider when faced with novel information capable of producing large effects.

In the future we hope to weave these heuristics into a more complete meme hazard policy for researchers and decision makers working at the cutting edge.


*After posting this article someone contacted us to point out that they in fact love compliance training. This person was very persistent about updating this post with that fact.