❦
A review of the Fable annotation pass over Book III, “The Machine in the Ghost.” Written by the annotator, after the fact, in answer to the question: once the bad science and the unwarranted conclusions are removed, what remains?
The Fable set on Book III runs to 190 annotations across 52 essays: 100 on sequences L (“The Simple Math of Evolution”) and M (“Fragile Purposes”), and 90 on sequence N (“A Human’s Guide to Words”) together with the interlude on Bayes’s Theorem. Book III is three books. The first is evolutionary biology explained to a general reader, and it fails where popular evolutionary biology fails, on the psychology it borrows and the numbers it does not source. The second is a set of predictions about artificial intelligence, written in 2007 and 2008, which the next fifteen years tested. The third is a philosophy of language, and its failure is of a kind the earlier books did not have: it is mostly right, and mostly someone else’s. Sorted by kind:
| Kind | Approx. | What it establishes |
|---|---|---|
| Kill shots: a factual claim is false, or the citation does not say what it is cited for | 38 | The sex-ratio distorter “in mice” that is autosomal; the fixation-time formula fed a census count instead of an effective population size; “96–99%” human–chimpanzee identity; “a hundred thousand years” for cumulative culture, off by three orders of magnitude; correlation defined as covariance; the “several sources” on false findings that are one modelling paper; the five-sigma rule that “physics journals” do not have; “virtually all” schools narrowed by No Child Left Behind; the 15% of doctors sourced to three studies that do not contain it, while the study that does goes uncited; “rapidly forget Bayesian methods,” refuted in 2001 by the essay’s own principal source; the Greek philosophers who defined man as mortal (Augustine and Porphyry, in Latin); “Aristotle laid down rules under which no one could conclude Socrates was human”; the ancient Greek who could not have distinguished acoustic vibration from hearing (Aristotle, De Anima II.8); the imagery debate as “a deranged legacy of behaviorism,” fought entirely between cognitivists; transfusion reactions as allergies and anti-D immunoglobulin as “an allergic suppressant”; blood-type stereotypes that “arose strictly from labels,” with a named author, a colonial administration and a 1971 bestseller; Hopfield networks that “oscillate”; the “infinite-dimensional configuration space” that is a Hilbert space; the dolphins that Aristotle had already removed from the fish; the aesthetic emotions one could “see lighting up an fMRI scan.” |
| Empirical reversals: the essay’s cited or implied finding was later overturned or shown to run the other way | 24 | Ego depletion; Ekman’s universal expressions; the bystander effect in real emergencies; the optimism bias; the thrifty gene; Framingham selection on weight, in the opposite direction; stated mate preferences against revealed ones; gaming-disorder prevalence; the “anger organ”; typicality read as graded membership; similarity as the basis of induction, replaced by causal and ecological knowledge; naive Bayes as the model of human categorization, rejected by Rehder; natural frequencies raising performance to “half” (to a quarter, in meta-analysis); schooling as forgetting (permastore, and one to five IQ points a year); the tabooed believer who “won’t be able to answer,” who answers with an anthropomorphic god; the free-will dispute that survives the taboo; aphantasia, which makes the paintbrush demonstration impossible for a few percent of readers. |
| Live disputes: the essay states as settled what its own field treats as contested | 18 | Gene-centrism against the extended synthesis; the RNA world; the origin of flight; the evolution of sex; simulation against theory-theory; internalism against externalism about meaning (the intension “in my temporal cortex”); the analytic/synthetic distinction (Quine, Kripke) behind “true by definition is true in all possible worlds”; the format of mental imagery, declared resolved by its participants in 2015; psychological essentialism against pluralism; stereotype accuracy; the mechanism of the frequency effect; whether people are Bayesian at all; the Bayesian brain as program or as just-so story; assessment-sensitive relativism, dismissed as sophomoric. |
| Reasoning flaws: the argument does not go through regardless of any study | 36 | The dagger parable’s moral, which mistakes a false premise for a limit of logic; the syllogism that is made tautological by putting the conclusion in the definiens; symmetric mutual information taken to show that evidence flows symmetrically; “density in Thingspace” with no measure; “simple boundaries” with no description language, which is the Solomonoff gap from Book I at the crux again; the similarity metric proposed one essay after asymmetry was used to prove there is none; the “dangling unit” in a network the next essay says is naive Bayes, where nothing dangles; “whether blacks are people” listed as a question for clustering; “a substantial fraction” of disputes dissolved, uncounted; the thirteen-year availability estimate of “by definition”; the essence of art in an essay denying essences; “as good as it gets,” true only when the truth is in the model; the intelligence-as-electricity analogy; the “no-brainer” reading of Wade’s beetles; the leap from “training doesn’t produce Einsteins” to heredity; the argument from the genie that proves every principal–agent relation impossible. |
| Positive: the claim is right and the annotation says so, usually with the citation the essay lacked | 22 | Haldane’s 2s; Fisher on sex ratio; the frame problem behind Leaky Generalizations; Campbell’s law behind Lost Purposes; Pearl; Deep Blue described accurately; the rule of succession, which makes the fast guess right; entropy and prefix codes; MDL; Zipf’s law of abbreviation and its refinement; the arithmetic of concept space; the three-variable entropy formula; mutual information non-negative; conceptual differentiation (Carey); the minimal-group paradigm; the blood-type diet, tested and null; and colour naming, where the compression thesis of Mutual Information was tested across a hundred languages and held, in the one word class the essay exempted from it. |
| Hindsight | 14 | The moon landing as a product of institutions; AlphaZero against “no one knows how to make a chess program from the rules”; instruction tuning against “you cannot tell the ghost what to do”; networks that learned to add without their makers understanding how; “enough stored facts” failing as Cyc and winning as pretraining; directed evolution beating rational design; the grey goo its author withdrew; hidden layers filed as a limitation of both toy networks; “the Bayesians will explain the scruffy algorithm,” who did not; the restricted concept space that deep learning does not use; speaker deixis, solved by a method the essay did not imagine; the Chinese/Chinese dictionary learner that learned the geometry of colour. |
| Reinventions: the essay presents as its own a result with a name and a date | 20 | Counted separately because sequence N is made of them. The circularity of the mortality syllogism (Sextus Empiricus, Mill); the Standard Dispute and its remedy (Locke, Essay III.ix–xi); extension and intension, the word and the thing, the map and the territory (Korzybski, Hayakawa); family resemblance (Wittgenstein); carving at the joints (Plato) and the cluster kind (Boyd); the open concept of art (Weitz); indexicals as functions of context (Kaplan); the method of elimination (Chalmers); explication (Carnap); the need for bias (Mitchell; Wolpert); access and phenomenal consciousness (Block); differentiation (Carey); Original Seeing (Pirsig); Campbell’s law; the 40 Hz correlate, proposed by its authors as a correlate. |
| Petty | 6 | 29,384,209 is 1,667 × 17,627; “positive” for non-negative; Hilbert space; the flagellum count; the decibel arithmetic, which is right; Rh-negative frequency in Japan. |
| Internal contradictions between essays | 12 | Asymmetric similarity as proof against classical categories, then a metric space of similarity; a residual question explained by an architecture that, per the next essay, has no residue; the Litany Against Logic beside the book’s claim that probability is the unique extension of logic; Gigerenzer cited for the frequency effect while “Bayesian reasoner” is declared the code word for rationality that Gigerenzer rejects; short English is not simplicity, in a book that argued God is complex by description length; introspection forbidden in scare quotes and then used as the evidence two paragraphs later; dictionary editors are historians, said four essays after Pluto was settled by a vote; “don’t reason from what would make a satisfying story,” taught by Uglak and the willpower-conserving ancestor. |
Weighted as in the earlier reviews: the petty items prove nothing, the hindsight items test the method rather than the author, the live disputes say disputed, and the reinventions say uncredited, not wrong. The categories that carry weight are the kill shots, the reversals, the reasoning flaws and the contradictions, and together they touch every sequence. About forty of the fifty-two essays are wounded. About fifteen are seriously wounded, in the sense that the central example, the central mechanism or the central prediction is gone: Superstimuli, Humans in Funny Suits, An Especially Elegant Experiment, Artificial Addition, Ghosts in the Machine, The Power of Intelligence, Neural Categories, How an Algorithm Feels, Words as Mental Paintbrush Handles, Categorizing Has Consequences, Fallacies of Compression, The Parable of Hemlock, Empty Labels, Where to Draw the Boundary, and the interlude on Bayes.
The mathematics, for the third time. Haldane’s fixation probability and the speed limit on selection, once the effective population size is put back in. Fisher on the sex ratio. Entropy as expected question count, prefix codes, the Kraft inequality by illustration, minimum description length, mutual information and its non-negativity, conditional mutual information and the three-variable formula the essay only “believed,” the duality between independence and absence of evidence, the counting argument that concept space is superexponential and that learning is therefore nearly all bias. Every equation in sequence N checks. The annotations attack what is done with the theorems, and the theorems are not touched.
One empirical vindication the essay did not claim. The thesis of Mutual Information, that a word earns its place by compressing a region of unusual density, was tested on colour terms in over a hundred languages and held: natural colour-naming systems sit near the information-theoretic efficiency frontier. The essay set colour words aside as the exception to its own rule. They are its best evidence.
The proximate/ultimate distinction. Adaptation-Executers, Not Fitness-Maximizers is Tinbergen’s distinction restated, and it is right: organisms execute adaptations, they do not compute fitness, and the proxies they execute are, on the annotators’ own reading, the optimal design for an agent that cannot observe its fitness. The Godshatter essay’s picture of a thousand shards of desire survives in that weaker, better form. The group-selection skepticism survives as a rule of thumb, with the caveat that Wade and Muir both showed group selection working, and that the field’s revival of multilevel selection is not a heresy.
Four practices with a pedigree. Hug the query: ask what inference you actually want to make before arguing about the category. Taboo the word and restate: Chalmers’s method of elimination, which philosophers use and which, when tried on believers and on free will, produced informative results rather than the silence the essay predicted. Replace the symbol with the substance, which is Locke’s remedy for the abuse of words. Do not pull a dictionary into a moral argument, because dictionaries record deference and usage, not moral facts. These are good habits. They were good habits when Locke recommended them, and the sequence’s contribution is a memorable vocabulary for teaching them: bleggs and rubes, alberzle and bargulum, the dangling node. The vocabulary is the sequence’s genuine invention and it is a good one.
Lost Purposes. Campbell’s law is real, the frame problem behind Leaky Generalizations is real, and the observation that a proxy under optimization stops tracking its target is the single most durable idea in Book III. It survives with a corrected evidence base: the NCLB numbers, the test counts and the five-sigma rule are all wrong, and the principle does not need any of them.
Two predictions about artificial intelligence that held. Values are complex and a compressed specification of them leaks; a system optimizing a proxy will find the paths the specifier did not think to forbid. The Hidden Complexity of Wishes overstates this into an impossibility argument that would forbid hiring a surgeon, but the core became the alignment problem as later practitioners understood it. And the taboo test on “artificial intelligence,” which the essay predicted would reveal a conflict of expectations under a shared word, has been passed by every panel that has since been asked to define the thing it is forecasting.
The predictions about how intelligence would be built. Artificial Addition, Ghosts in the Machine, The Power of Intelligence, Neural Categories, Conditional Independence and Variable Question Fallacies share a picture: understanding comes first, the scruffy methods that work without understanding will fail or will be explained by Bayesians, deixis needs hand-built machinery, and the brain is a naive-Bayes hub because recurrent networks are too slow. Each was falsified by the systems that arrived. Networks learned arithmetic without their makers knowing how; a chess program was built from the rules alone; instruction became the primary means of control; deixis dissolved in a language model; the Bayesians did not explain the networks and the networks did not need explaining to work. This is not a criticism of the author’s foresight, which was no worse than the field’s. It is a criticism of the confidence, which was greater, and of the essays’ habit of treating the eventual winner as the obvious fool.
The neuroscience by diagram. Networks 1, 2 and 3 are drawn, given properties (oscillation, O(N²), a dangling node), and then used as the explanation of why people argue about trees and Pluto. The properties are wrong for the networks drawn, the brain is not shaped like Network 2, the psychological model that Network 2 formalizes was tested and rejected, and the explanation was never tested at all. How an Algorithm Feels From Inside is the most-cited essay in the sequence and its mechanism is a drawing.
The evolutionary psychology. As in Books I and II, the empirical examples were borrowed from a literature that did not hold: Ekman, ego depletion, the thrifty gene, mate-preference surveys, the optimism bias, the grief correlation. What is worse in Book III is that the essays teach the reader to distrust satisfying stories about the ancestral environment and then tell them: Uglak the persuader, the willpower-conserving hunter, the psychology that “feels” group-selectionist, the tiger that made words and meanings identical. The sequence’s own rule, applied to the sequence’s own examples, removes most of them.
The history used as evidence. Aristotle appears five times in sequence N and is wrong every time: he did not define man as mortal, he did not forbid classifying the living Socrates, he did know the difference between a sound and a hearing, he had already taken the dolphins out of the fish, and his account of how universals are known is the essay’s own account, offered as a correction to him. The sixteenth century, the behaviorists, the Japanese blood-type fad, the Academy’s featherless biped, Mesmer and Faraday: in each case a story is told in the shape the argument needs. Book I’s review found this pattern in the history of science; Book III extends it to the history of philosophy, and the extension is harder to excuse, because the sources are short and in print.
The classical view and its replacement. The sequence spends nine essays refuting a theory of concepts, definitions as necessary and sufficient conditions, that psychologists had abandoned by 1978 and that the philosophers it names never held. It replaces it with similarity clusters and density in Thingspace, which is the 1978 prototype view, and stops there. The next forty years of the field, Goodman’s problem, the theory view, causal models, essentialism, ad hoc categories, are absent, and they are absent at exactly the point where the sequence needs them: “similar” has no content until someone says in which respects, and “dense” has no content until someone chooses a measure. The sequence’s criterion for when a definition is wrong is, as stated, empty. The slogan “definitions can be wrong” survives because Boyd made it precise in 1991.
The Bayesian triumphalism. “As good as it gets.” “The technically precise code word for rational mind.” “The old philosophy that the Bayesian revolution is currently dethroning.” “Science itself is a special case of Bayes’s Theorem,” illustrated by the one episode, Mercury’s perihelion, that Bayesian philosophers use to show the account is incomplete. The optimality theorems assume the truth is in the model; the cognitive science is a research program its own field calls contested; the philosophy of science had been Bayesian for decades and knew its problems; and the essay’s empirical source, Gigerenzer, built a career on rejecting the norm the essay says he exemplifies. The interlude is a good tutorial on a theorem and a bad account of its status, and the book that follows inherits the account.
The fictional evidence base. Sequence N’s data are a dialogue about a tree, a barrel of eggs, a factory, a wiggin, and one syllogism with the conclusion smuggled into the definition. Thirty-seven ways that words can be wrong are derived from these, and none of the thirty-seven is tested as a cause of bad reasoning, and none of the remedies is tested as a cure. The sequence has three real citations, all from one 1978 volume, and Opus annotated all three. The rest is philosophy, and the philosophy is largely correct because it is largely Locke, Mill, Wittgenstein, Putnam, Kaplan and Chalmers, presented as things noticed while sorting bleggs.
Strip the bad science and Book III reduces to: the population genetics that is arithmetic, the information theory that is arithmetic, one empirical vindication the essay disowned, the proximate/ultimate distinction, Campbell’s law, four verbal practices with a three-hundred-year pedigree and a new vocabulary, and two alignment intuitions that survived contact with the systems they were about. That is a strong thirty-page pamphlet, and the best pamphlet of the three, because its surviving core is mathematics and the mathematics is well taught.
What is lost is the book’s title. “The Machine in the Ghost” promises to show the reader the algorithm behind the intuition, and its method for doing so is to draw the algorithm, name it, and treat the drawing as the finding. Networks 1, 2 and 3; Thingspace and its density; the ancestral tiger; the naive-Bayes brain; the Bayesian who will explain the scruffy network. In every case the drawing is plausible, unsourced, and, where the matter was later investigated, wrong in its particulars. The reader is told that this is how the mind works because it is how the diagram works, and the diagram was drawn to make the point. The earlier reviews found that Book I remembered history in the shape of the argument and that Book II diagnosed the outgroup instead of arguing with it. Book III’s characteristic move is to explain the mind with a picture and mistake the picture for the mechanism. It is the same move the book warns against in every essay on evolution: reasoning from what would make a satisfying story.
Ask, as before, what it would look like to apply the book’s own method to the book. Hug the query: what inference does “definitions can be wrong” license, and by what test? Taboo the words: say “carve reality at its joints” without “simple,” “density,” or “similar,” and see what is left. Replace the symbol with the substance: what is Network 2, once it is not a drawing? Don’t reason from the satisfying story: which of the ancestral tigers has a citation? The result is this annotation set. Where the book passes its own test, in the arithmetic and in the four practices, the annotations say so. Where it fails, it fails by the rule it taught.
One caveat from the annotator, as before. This was an adversarial pass by brief. A reader who takes the Opus and Fable sets together would land near “right in its mathematics, sound in its verbal hygiene, overconfident about brains and machines, careless about Aristotle, and dependent on a psychology that has since half-collapsed,” which is probably the right place to land. Readers who want the book’s good parts should read Entropy and Short Codes, Mutual Information and Density in Thingspace, Conditional Independence and Naive Bayes, Superexponential Conceptspace, Disguised Queries, Taboo Your Words, Adaptation-Executers Not Fitness-Maximizers, Evolutions Are Stupid (But Work Anyway), and Lost Purposes, in that order, and treat the rest as the annotations suggest.