Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio.
This is: Moloch Hasn’t Won, published by Zvi on the LessWrong.
This post begins the Immoral Mazes sequence. See introduction for an overview of the plan. Before we get to the mazes, we need some background first.
Meditations on Moloch
Consider Scott Alexander’s Meditations on Moloch. I will summarize here.
Therein lie fourteen scenarios where participants can be caught in bad equilibria.
In an iterated prisoner’s dilemma, two players keep playing defect.
In a dollar auction, participants massively overpay.
A group of fisherman fail to coordinate on using filters that efficiently benefit the group, because they can’t punish those who don’t profi by not using the filters.
Rats are caught in a permanent Malthusian trap where only those who do nothing but compete and consume survive. All others are outcompeted.
Capitalists serve a perfectly competitive market, and cannot pay a living wage.
The tying of all good schools to ownership of land causes families to work two jobs whose incomes are then captured by the owners of land.
Farmers outcompeted foragers despite this perhaps making everyone’s life worse for the first few thousand years.
Si Vis Pacem, Para Bellum: If you want peace, prepare for war. So we do.
Cancer cells focus on replication, multiply and kill off the host.
Local governments compete to become more competitive and offer bigger bribes of money and easy regulation in order to lure businesses.
Our education system is a giant signaling competition for prestige.
Science doesn’t follow proper statistical and other research procedures, resulting in findings that mostly aren’t real.
Governments hand out massive corporate welfare.
Have you seen Congress?
Scott differentiates the first ten scenarios, where he says that perfect competition wipes out all value, to the later four, where imperfect competition only wipes out most of the potential value.
He offers four potential ways out, which I believe to be an incomplete list:
Excess resources allow a temporary respite. We live in the dream time.
Physical limitations where the horrible thing isn’t actually efficient. He gives the example of slavery, where treating your slaves relatively well is the best way to get them to produce, and treating them horribly as in the antebellum South is so much worse that it needs to be enforced via government coordination or it will die out.
The things being maximized for in competitions are often nice things we care about, so at least we get the nice things.
We can coordinate. This may or may not involve government or coercion.
Scott differentiates this fourth, ‘good’ reason from the previous three ‘bad’ reasons, claiming coordination might be a long term solution, but we can’t expect the ‘bad’ reasons to work if optimization power and technology get sufficiently advanced.
The forces of the stronger competitors, who sacrifice more of what they value to become powerful and to be fruitful and multiply, eventually win out. We might be in the dream time now, but with time we’ll reach a steady state with static technology, where we’ve consumed all the surplus resources. All differentiation standing in the way of perfect competition will fade away. Horrible things will be the most efficient.
The optimizing things will keep getting better at optimizing, thus wiping out all value. When we optimize for X but are indifferent to Y, we by default actively optimize against Y, for all Y that would make any claims to resources. Any Y we value is making a claim to resources. See The Hidden Complexity of Wishes. We only don’t optimize against Y if either we compensate by intentionally also optimizing for Y, or if X and Y have a relationship (causal, correlational or otherwise) where we happen to not want to optimize against Y, and we figure this out rather than fall victim to Goodhart’s Law.
The greater the ...
view more