S1-213 – Mapping the unknown
Practical strategies for seeing how much we don’t know
Ask people what diabetes is, and most give a fuller answer than they get credit for. They know it means blood sugar that stays too high. They know that, left unchecked, it does slow damage to the body, to the eyes, the nerves, the kidneys. They know that a person with diabetes may have to inject insulin to bring the sugar down, and that they are supposed to go easy on carbohydrates. It is a simple picture, but a real one, and a more complete one than most of us could give for most diseases.
And still, the picture is of a single illness, when diabetes is really at least two, and they are close to opposites. In Type 1, the immune system destroys the cells in the pancreas that make insulin, the islets of Langerhans, so the body can no longer produce it, and injected insulin is replacing something that is simply gone. In Type 2, those cells are usually working, often overworking, but the rest of the body has stopped responding to the insulin they make, so the problem is not a shortage at all. One word covers both. And insulin, the thing you inject to lower sugar, is not a sugar drug at heart. It is the body’s master signal to store energy, including as fat. That is why the advice about carbohydrates reaches down into a live scientific argument about obesity, and why “go easy on carbs” is a deeper instruction than it sounds. None of this is hidden. It sits just past the edge of the competent, ordinary picture, in the territory the previous article marked out: the distance between knowing the name of a thing and knowing the thing itself.
This is the situation the article is about. Every domain we half-know shows us a lit foreground and an unlit distance behind it, and the feeling of a solid, sensible picture tells us nothing about how far back that distance runs. The person with the competent sketch of diabetes cannot judge the true size of the field for the same reason a walker at the foot of a mountain cannot see its upper slopes: not through any failure of character, but as a plain fact about where they are standing. What follows is a set of practical tools for estimating that distance from the outside, for getting a rough map of a territory before, and while, we enter it.
The problem and why it persists
You might expect this to reopen the question of the Dunning-Kruger effect, which article S1-205 set aside, but what this article is after is different, and the difference matters. That effect is about self-assessment: whether we can judge our own competence in a domain we have already entered. The concern here comes earlier. It is whether we can form an accurate sense of the terrain of a domain before and during the process of entering it. The question is not “am I good at this?” but “how much is there?” and “where are the edges?”, and it needs a different set of tools.
These tools are the subject of this article. They are not reliable in the strong sense; none of them guarantees that we will accurately map the unknown territory of a domain. They are reliable in the weaker sense: they consistently produce more accurate maps than the unaided feeling of recognition, because they each surface a different dimension of the gap between what one knows and what there is to know.
The first tool: knowledge maps of disciplines
The most direct way to begin understanding the structure of a domain is to look at how its practitioners organize it. Every established discipline has an internal architecture: a set of sub-fields, problems, methods, and foundational questions that organize what counts as the domain and how its parts relate to each other. This architecture is not always visible from the outside (introductory treatments typically present a simplified and unified surface) but it is available, in various forms, to anyone who looks for it.
One practical form is the knowledge map: a visual representation of a discipline’s internal organization that shows the sub-fields, their relationships, and their relative development. Dominic Walliman’s The Map of Mathematics, produced for his YouTube channel Domain of Science, is an example that has reached a wide general audience: a single image that shows the major branches of mathematics, their connections, the historical sequence in which they developed, and the frontier problems at the edge of current knowledge. Similar maps exist for physics, chemistry, biology, computer science, and many other disciplines.¹
The value of these maps is not that they teach us the content of the discipline. They do not. The value is that they show us the shape of what we do not know. The person who looks at the Map of Mathematics and sees that they know a few rooms in one wing of a large building, that their school experience covers a small portion of the structure, has a more accurate model of their position than the person who has never seen the building. The feeling of recognition that accompanies the familiar rooms is not eliminated. But it is contextualized. The unfamiliar rooms are now visible as unfamiliar, rather than being simply invisible.
The second tool: the open problems probe
Every mature discipline has a set of open problems: questions that are currently unanswered, that the discipline’s practitioners regard as important, and that have resisted solution despite sustained effort. These open problems are not minor loose ends. They are the places where the discipline’s current understanding runs out, where the models stop generating reliable predictions, where the frameworks cannot yet accommodate the observations, where the frontier of knowledge is actively being negotiated.
The open problems probe is simple: in any domain we believe we understand, can we name a significant unsolved problem that practitioners regard as important? Not a problem we happen to be personally puzzled by, but a problem that the discipline itself recognizes as open, that has a literature, that has attracted sustained effort without resolution.
In mathematics, the Riemann hypothesis (whether all non-trivial zeros of the Riemann zeta function lie on the critical line) has been unresolved for over 160 years, despite being one of the Millennium Prize Problems and attracting work from many of the most talented mathematicians alive. In physics, the reconciliation of general relativity and quantum mechanics remains unresolved despite decades of effort. In biology, the hard problem of consciousness (why physical processes in the brain give rise to subjective experience) is examined in article S1-503 of this series and remains genuinely open. In economics, the reliable prediction of financial crises has resisted every systematic approach attempted.
If we believe we understand a domain but cannot name a significant open problem within it, we probably understand the domain at the first level, name and association, rather than the second or third level. The open problems are not incidental to a discipline. They are its growing edge. Knowing that they exist, and roughly where they are, is part of knowing the discipline.
The third tool: the expert disagreement test
Every discipline also has a set of contested questions: questions on which expert opinion is divided, where the evidence is genuine but genuinely ambiguous, where methodological commitments and interpretive frameworks produce different conclusions from the same data. These contested questions are distinct from the open problems: they are not questions that have no answer yet, but questions where practitioners have different answers and disagree about which is correct.
The expert disagreement test asks: in any domain we believe we understand, can we identify the significant controversies, the questions where expert opinion is divided and the evidence is genuinely contested? Not the controversies manufactured by industry interests or political actors to create the false impression of expert disagreement on settled questions (the climate change denial campaign is the most documented example of this) but genuine methodological and interpretive disagreements within the discipline itself.²
In nutrition science, the optimal macronutrient composition for human health has been the subject of genuine and sustained expert disagreement for decades, with different research traditions producing different conclusions that are not easily reconciled. In economics, the magnitude of fiscal multipliers (how much economic activity is generated by a unit of government spending) is genuinely contested, with different methodological approaches producing estimates that differ by a factor of three or four. In psychology, the reliability and validity of specific diagnostic categories within the DSM is a genuine ongoing controversy among practitioners.
The person who believes they understand nutrition, economics, or psychiatric diagnosis but who is unaware of these internal controversies has not yet reached the second level of understanding. They have a model, but it is the model presented in the introductory treatment (the simplified surface that has been smoothed over to provide a consistent narrative) rather than the model that reflects the actual state of the discipline. The controversies are where the real epistemic action is.
The fourth tool: the Feynman explanation test
The Feynman explanation test was developed at length in article S1-212 and needs only brief restatement here, in the context of the practical tools for mapping the unknown. The test is: attempt to explain the core mechanism of any domain we believe we understand in plain terms, without technical vocabulary, to a curious person who has never encountered it. When the explanation stalls, when jargon creeps in as a substitute for mechanism, when the analogy breaks down immediately, when the concrete example does not exist, we have found the limit of our understanding.
The specific value of the Feynman test as a tool for mapping the unknown, rather than as a general epistemological practice, is that it reveals the gap between what we can name and what we can explain. The name can be deployed fluently before the explanation is available. The explanation requires the mechanism. And the location of the breakdown, the specific point at which fluent naming gives way to the absence of mechanism, is information about where our second-level map ends and the unmapped territory begins.
The test is most valuable when applied to the concepts that feel most familiar, the ones that have been used so many times that the name has acquired the feeling of full access to the structure beneath it. The person who has used the word “inflation” for years without ever having tried to explain the mechanism by which increasing the money supply produces rising prices, or by which supply chain disruptions produce the same effect through a different mechanism, has a first-level relationship to a concept they may believe they understand at the second. The moment of stalling is the moment of honest mapping.
The fifth tool: the adjacent field test
Every discipline exists in relation to other disciplines: it borrows methods from some, contributes findings to others, and shares boundary problems with others that neither can resolve alone. These adjacencies are not incidental to the discipline’s identity. They reveal what the discipline can and cannot do, where its methods work well and where they need supplementation, and what it is and is not designed to explain.
The adjacent field test asks: for any domain we believe we understand, can we identify the disciplines it borrows from and the disciplines it contributes to? Can we describe, at least roughly, the questions that lie at the boundaries, the problems that fall between disciplines, that require methods from more than one tradition, and that neither tradition alone can resolve?
Behavioral economics sits between economics and psychology: it was created by importing the findings of cognitive psychology into the economic model of human behavior, and it was necessary because the economic model’s assumptions about human rationality were producing systematically wrong predictions in domains where the psychological findings were directly relevant. Computational neuroscience sits between computer science and neuroscience: it applies the mathematical tools developed for artificial neural networks to the study of biological neural networks, and it has produced findings that neither tradition could have reached alone. Epidemiology sits between medicine and statistics: it applies statistical methods to the study of disease patterns in populations, and it is necessary because the individual case study methods of clinical medicine cannot distinguish individual variation from systematic patterns.
The person who understands a discipline without knowing its adjacencies has a model that does not include the discipline’s own awareness of its limits. The adjacencies are where the discipline itself says: here is where our methods are insufficient, here is where we need help from elsewhere, here is where the territory extends beyond what we can map alone.
The sixth tool: the historical depth test
Every discipline has a history: a sequence of frameworks, paradigms, controversies, and revolutions through which its current understanding was reached. This history is not merely interesting as narrative. It is epistemologically important, because it records the specific ways in which previous models of the discipline were wrong, the specific observations that revealed the wrongness, and the specific process by which the current frameworks replaced the previous ones.
The historical depth test asks: for any domain we believe we understand, do we know roughly when the current framework was established, what it replaced, and what specific findings or arguments made the replacement necessary? Can we name a period in the discipline’s history when something that was confidently believed turned out to be wrong, and describe what kind of evidence changed the consensus?
In medicine, the germ theory of disease replaced the miasma theory (the belief that disease was caused by “bad air”) in the latter half of the nineteenth century, through the work of Pasteur, Koch, and Lister, among others. The replacement was not immediate and not uncontested: Semmelweis’s demonstration that handwashing dramatically reduced childbed fever in obstetric wards was resisted for years by practitioners who had the relevant evidence and did not revise their framework. In geology, plate tectonics replaced the static earth model in the middle of the twentieth century, through the accumulation of evidence from ocean floor mapping, paleomagnetism, and seismic studies. In physics, the quantum revolution of the early twentieth century replaced the classical mechanical framework for phenomena at the atomic scale, producing a framework so counterintuitive that even its creators found it difficult to believe.
Knowing that a discipline has undergone revolutions, that what was once confidently believed is now known to have been wrong, is not an argument for skepticism about the current framework. It is a calibration device. It shows that the discipline is capable of being wrong in ways that its practitioners at the time did not recognize, and it provides a concrete sense of the mechanisms by which wrongness gets corrected. The person who knows this history holds the current framework differently, with greater respect for its achievements and greater honesty about its limits, than the person who encounters it as a seamless and complete account.
Calibrating confidence to engagement
The six tools together provide a map of what we do not know that is considerably more accurate than the unaided feeling of recognition. They do not produce certainty, and they do not produce complete maps. They produce something more modest and more useful: an honest sense of where the map we have drawn ends and the territory continues.
The application of the tools is itself a demonstration of the point they make. The person who works through the open problems probe and discovers that they cannot name a significant unsolved problem in a domain they believed they understood has learned something real: that their model of the domain was a model of the domain’s settled, introductory surface, not of the discipline as it is actually practiced. This discovery is uncomfortable. It is also the beginning of a more accurate orientation, one in which the edges of the known are visible as edges rather than being simply invisible.
There is a specific and important application of these tools to public discourse and democratic deliberation. The domains that matter most for collective decision-making (economics, public health, climate science, educational policy, criminal justice) are also the domains in which the gap between first-level and second-level understanding is most consequential and most consistently overlooked. The voter who holds confident views about the economic effects of immigration, the epidemiological basis of vaccination policy, or the criminological evidence on deterrence, without having engaged any of the six tools, is holding first-level beliefs with second-level confidence. This is not a reason to restrict democratic participation to experts; that proposal contains a confusion about the nature of expertise and the purpose of democracy that the society and politics pillar of this series examines at length. It is a reason to hold our own political beliefs with more epistemic honesty: to acknowledge that confident intuitions about complex systems are not the same as knowledge of those systems, and that the people who disagree may have access to information or understanding that the intuition does not contain.
The Conscious Look, applied to the question of how much we do not know, is the practice of using these tools regularly and honestly, of periodically asking, about any domain in which one is tempted to speak with confidence, whether the confidence is calibrated to genuine understanding or to the feeling of recognition that the first level reliably produces. The feeling of recognition is not the problem. It is the confusion of the feeling with understanding that is the problem. And the six tools are, in the end, six different ways of distinguishing one from the other.
Further reading
Dominic Walliman’s Map of Mathematics and related discipline maps, available on the Domain of Science YouTube channel, are the most accessible currently available demonstrations of the architectural overview approach: the view from above a discipline that shows its internal structure, its sub-fields, and its frontier problems in a single image. They do not teach the content of the disciplines, but they provide the map of the map, which is a good beginning.
Richard Feynman’s The Character of Physical Law (1965) shows, across seven lectures, what the frontier of a discipline looks like from the inside: what it means to be at the edge of understanding in physics, what open problems feel like to the person working on them, and why the discomfort of not knowing is the specific experience of being at the right position relative to the territory. It is the best available model of how to hold a discipline honestly.
Stuart Firestein’s Ignorance: How It Drives Science (2012) is the most direct treatment of the open problems and expert disagreement tools in the context of scientific practice, arguing that science is not primarily the accumulation of knowledge but the cultivation of productive ignorance, and that the most important thing any scientist can do is find better questions rather than more answers. It is short, readable, and more epistemologically sophisticated than its accessible presentation suggests.
Thomas Kuhn’s The Structure of Scientific Revolutions (1962) provides the foundational account of the historical depth tool: how scientific disciplines develop through periods of normal science punctuated by paradigm shifts, and what the transition from one framework to another reveals about the limits of the framework that preceded it. It is the essential background for understanding why the historical depth test is informative rather than merely interesting.
Philip Tetlock and Dan Gardner’s Superforecasting: The Art and Science of Prediction (2015) provides the most rigorous account of what calibrated confidence actually looks like: how people who consistently make accurate predictions in complex domains differ from those who are merely confident, and what specific practices distinguish calibrated uncertainty from either overconfidence or false humility.
Notes
¹ Walliman, D. (2017). The Map of Mathematics. Domain of Science, YouTube. The video and associated poster have reached a wide audience and have been used in educational contexts as an introduction to the scope of the discipline. Similar maps exist for physics (Map of Physics, 2017), chemistry (Map of Chemistry, 2019), biology (Map of Biology, 2020), and computer science (Map of Computer Science, 2019), all available on the same channel. The maps are not technically precise representations of the discipline’s internal structure (professional mathematicians and physicists will find simplifications and inaccuracies) but they serve their purpose of making visible, to an outside observer, the general architecture of a discipline that would otherwise appear as an undifferentiated body of content.
² The distinction between manufactured expert disagreement, created by industry interests to obscure settled questions, and genuine expert disagreement within a discipline is important and is not always easy to draw. Oreskes, N., and Conway, E. M. (2010). Merchants of Doubt: How a Handful of Scientists Obscured the Truth on Issues from Tobacco Smoke to Global Warming. Bloomsbury Press. Oreskes and Conway document in detail how the strategy of manufacturing apparent expert disagreement, by funding minority positions, amplifying heterodox voices, and creating the impression that the scientific community is divided on settled questions, was developed in the tobacco industry and subsequently applied to acid rain, the ozone hole, and climate change. The expert disagreement test is designed to identify genuine disciplinary controversies, not manufactured ones, and the distinction requires knowing enough about the discipline to tell the difference, which is itself a reason to engage with the discipline’s internal literature rather than relying on secondary sources with potential interests in the outcome.