Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio.
This is: Specializing in Problems We Don't Understand , published by johnswentworth on the AI Alignment Forum.
Most problems can be separated pretty cleanly into two categories: things we basically understand, and things we basically don’t understand. Some things we basically understand: building bridges and skyscrapers, treating and preventing infections, satellites and GPS, cars and ships, oil wells and gas pipelines and power plants, cell networks and databases and websites. Some things we basically don’t understand: building fusion power plants, treating and preventing cancer, high-temperature superconductors, programmable contracts, genetic engineering, fluctuations in the value of money, biological and artificial neural networks. Problems we basically understand may have lots of moving parts, require many people with many specialties, but they’re generally problems which can be reliably solved by throwing resources at it. There usually isn’t much uncertainty about whether the problem will be solved at all, or a high risk of unknown unknowns, or a need for foundational research in order to move forward. Problems we basically don’t understand are the opposite: they are research problems, problems which likely require a whole new paradigm.
In agency terms: problems we basically understand are typically solved via adaptation-execution rather than goal-optimization. Problems we basically don’t understand are exactly those for which existing adaptations fail.
Main claim underlying this post: it is possible to specialize in problems-we-basically-don’t-understand, as a category in its own right, in a way which generalizes across fields. Problems we do understand mainly require relatively-specialized knowledge and techniques adapted to solving particular problems. But problems we don’t understand mainly require general-purpose skills of empiricism, noticing patterns and bottlenecks, model-building, and design principles. Existing specialized knowledge and techniques don’t suffice - after all, if the existing specialized knowledge and techniques were sufficient to reliably solve the problem, then it wouldn’t be a problem-we-basically-don’t-understand in the first place.
So. how would one go about specializing in problems we basically don’t understand? This post will mostly talk about how to choose what to formally study, and how to study it, in order to specialize in problems we don’t understand.
Specialize in Things Which Generalize
Suppose existing models and techniques for hot plasmas don’t suffice for fusion power. A paradigm shift is likely necessary. So, insofar as we want to learn skills which will give us an advantage (relative to existing hot plasma specialists) in finding the new paradigm, those skills need to come from some other area - they need to generalize from their original context to the field of hot plasmas. We want skills which generalize well.
Unfortunately, a lot of topics which are advertised as “very general” don’t actually add much value on most problems in practice. A lot of pure math is like this - think abstract algebra or topology. Yes, they can be applied all over the place, but in practice the things they say are usually either irrelevant or easily noticed by some other path. (Though of course there are exceptions.) Telling us things we would have figured out anyway doesn’t add much value.
There are skills and knowledge which do generalize well. Within technical subjects, think probability and information theory, programming and algorithms, dynamical systems and control theory, optimization and microeconomics, linear algebra and numerical analysis. Systems and synthetic biology generalize well within biology, mechanics and electrodynamics are necessary for fermi estimates in most physical sciences, continuum mechanics and PDEs are useful for a wide ...
view more