Basic HTML Version

Chapter 9
Complexity, Ockham’s Razor, and Truth
Kevin T. Kelly and Konstantin Genin
Introduction: The Simplicity Puzzle
Ockham’s razor says: “Choose the simplest theory compatible with the data.” Without Ockham’s
razor, theoretical science cannot get very far, since there are always ever more complicated
explanations compatible with current evidence. Scientific lore pretends that reality is simple – but
gravitation works by a quadratic, rather than a linear law; and what about the shocking failure of
parity conservation in particle physics? Ockham speaks so strongly in its favor that demonstrating
its falsity resulted in a Nobel Prize in physics (Lee and Yang 1957). So why trust Ockham?
It is tempting, but philosophically disastrous, to conceive of scientific method as a kind of
of the truth. That wishful view would require that Ockham’s razor works
something like a
guaranteed to point at the truth, whatever the truth might be.
But Ockham’s razor cannot really work like a compass, because its needle is forever
at the
same reading―simplicity. Imagine the surprise of the captain of a sailing ship, secure in the
completion of a long, prosperous voyage, who suddenly discovers that the compass needle was
frozen the entire time. How did
work? It is very hard to explain without some strange,
providential story―for instance, that God ensures that the truth is simple, as Gottfried Leibniz and
other early scientific luminaries proposed. Yet who can miss the irony of invoking an undetectable
Providence in defense of the principle that science should get along with the bare minimum of
ontological assumptions?
It would be better to give up on unverifiable, metaphysical connections between simplicity
and truth, and to seek, instead, an
a priori
justification. One familiar
methodological approach is to substitute some
goal for the putative goal of finding the truth.
For example, it is a familiar story in machine learning that simple theories can yield more accurate
predictions than complex theories. The idea is that empirical estimates based on more complex
models can have a greater expected distance from the truth because you are fitting many
parameters with a small sample, so the statistical shake in the resulting predictions is higher
because each parameter is estimated with a tiny sample. The added statistical shake impairs the
predictive accuracy of a complex model―even if the complex model is
. So the over-fitting
argument for Ockham’s razor is not about finding
theories; it is, at best, an instrumental
account of how to minimize statistical spread in empirical estimates by
a theory that you
may even know a priori to be false. In the philosophy of science, one substitutes
for truth. Simpler theories are more testable (Glymour 1980), more unified
and explanatory (Kitcher 1993; Friedman 1986), more symmetrical (Malament 1977), and more
bold (Popper 1959). In a similar spirit, information theorists (Rissanen 1978) say that simple
theories compress the data better. We are not opposed to any of that, so far as it goes, but it does