Part philosophical, part scientific, Leibniz believed that our world - "the best of all possible worlds" - must be governed by what is known as the Principle of Optimality. This seemingly outlandish idea proved surprisingly powerful and led to one of the most profound ideas in theoretical physics. Jeffrey K. McDonough tells the story.

The great German polymath Gottfried Wilhelm Leibniz famously insisted that ours is the best of all possible worlds. The claim that our world couldn’t possibly be better has never been very plausible. It was hard to believe when Leibniz made it in the seventeenth century on the heels of the horrific Thirty Years War. It didn’t seem any more likely when Voltaire heaped ridicule upon it following the Lisbon Earthquake of 1755. And, of course, it will probably not find many adherents today as we trudge along under the weight of a global pandemic, political uncertainty, and an environment on the verge of collapse. Leibniz’s thought that ours is the best of all possible worlds is, in short, incredible. Incredible or not, however, Leibniz’s implausible idea lies at the heart of one of the most profound, most successful, most tantalizing developments in the theoretical physics. Call it the story of Leibniz’s Principle of Optimality.

The roots of Leibniz’s principle reach back to at least Heron of Alexandria’s discovery of the optical law of reflection and to ancient thinking about optimizing territories and storage containers. The story of Leibniz’s principle begins in earnest, however, with a controversy that erupted between two of the finest mathematicians of the early modern era, René Descartes and Pierre de Fermat. Descartes was the first to publish the optical law of refraction in essentially the form we accept today. Many at the time, however, doubted his mechanistic demonstration of the law, which involved drawing clever analogies to the behavior of tennis balls and rackets. Seeking a more rigorous derivation, Fermat showed how both the optical law of reflection and the optical law of refraction could be derived from a quickest path principle: in a standard set of cases, a ray of light will take the quickest path from, say, a lamp to an eye regardless of how it is reflected or refracted.

As was typical of the era, Fermat and the followers of Descartes managed to snatch bitter controversy from clear progress. Fermat claimed that Descartes had never proved the law of refraction and insinuated that he had stolen his results from the Dutch astronomer Willebrord Snell. Cartesians insisted that Fermat’s derivation was technically flawed and was at any rate a regression from mechanistic ideals. Leibniz stepped into this controversy with a remarkable paper published in 1682. The paper aimed to show that Descartes’s mechanistic approach to the laws of optics and Fermat’s optimization approach could be reconciled. Leibniz sided with Descartes on some technical points and agreed that a mechanistic explanation of the laws of optics could be given. Nonetheless, he also embraced the spirit of Fermat’s proposal, showing how the laws of optics could be derived from optimal an easiest path principle and applied to an even greater variety of cases than Fermat had considered. The paper was a multifaceted breakthrough that showed how optimization methods could be reconciled with mechanistic explanations, how such methods could be married to Leibniz’s powerful new infinitesimal calculus, and – perhaps most profoundly – that optimization principles needn’t be restricted to kinematic notions such as distance and time but could be extended to dynamic notions such as “ease,” work, and energy.

___

In the notion of an optimal form, Leibniz found a rigorous model for his thesis that this is the best of all possible worlds.

___

In a series of papers written over the next decade and half, Leibniz and his cohort extended his optimization approach to other cases of natural phenomena by showing how they too could be viewed as instances of optimal form. One such case concerns the shape of a freely hanging chain suspended at two ends:

Such a chain can be thought of as an optimal form that, in contemporary terms, minimizes potential energy, that is, the energy a system has in virtue of its position. As an optimal form, it has two remarkable properties. First, while the chain as a whole minimizes overall potential energy, it does not minimize the potential energy of every part. We can, for example, lower the potential energy of the middle link by pulling down on it. Doing so, however, must come the expense of raising the other links in such a way that the overall potential energy of the string is increased.

Second, since the hanging chain is an optimal form, it must be the case that every subsection of the chain is also an optimal form. In fact, we can see this by reasoning alone. Suppose that figure ACDB represents a chain that minimizes potential energy, and that CD is a segment of ACDB.

If the segment CD did not minimize its potential energy – if it were not itself an optimal form – it could be replaced by a different segment with less potential energy so that the chain as a whole would have less than its minimal potential energy – an absurdity! On pain of contradiction, any subsection of an optimal form must itself be an optimal form.

In the notion of an optimal form, Leibniz found a rigorous model for his thesis that this is the best of all possible worlds. The world as a whole is analogous to the chain as a whole. Just as the chain as a whole minimizes overall potential energy, the world as a whole maximizes overall goodness (or minimizes overall badness). That doesn’t mean that individual aspects of the world couldn’t be better. Judas would have been better if he hadn’t betrayed Christ. But any such local “improvement” – according to Leibniz – would have to be more than counterbalanced by negative consequences. If Judas hadn’t sinned, Judas would have been better, but the world as a whole would have been worse, just as pulling down on the middle link of the chain would decrease the potential energy of that middle link but only at the cost of increasing the potential energy of the other links.

___

The development of physics since Leibniz’s time has largely vindicated his audacious conjecture.

___

It’s surprising that Leibniz was able to draw deep connections between his seemingly fantastic view that this is the best of all possible worlds and his concrete scientific discoveries. The real twist in the story of Leibniz’s Principle of Optimality, however, played out only after his death. On the basis of some philosophical assumptions and a handful of technical cases, Leibniz had audaciously conjectured that it should be possible to explain all natural phenomena in terms of optimization principles. Even more surprising than the connections between Leibniz’s philosophical theology and his scientific studies is the fact that the development of physics since Leibniz’s time has largely vindicated his audacious conjecture concerning the scope of optimality principles.

Eighteen years old when Leibniz died, Pierre Louis Maupertuis cut his teeth as a scientist by applying Leibniz’s calculus to Newtonian mechanics. He rose to international prominence, however, with a swashbuckling account of his scientific expedition to Lapland, an account that mixed exact science with tales of bitter cold, reindeer, and local women. He was poised to assume the presidency of the Royal Prussian Academy of Science, when he published a paper that echoed the results and methods of Leibniz’s early 1682 optics paper. Two years later, he published a second paper in which he formally announced his Principle of Least Action, according to which “in Nature, the action necessary for change is the smallest possible.” As the leader of one of the great scientific societies of the era, Maupertuis had just thrown the prestige of his presidency and the weight of the Royal Prussian Academy of Science behind a version of Leibniz’s bold hypothesis that all natural phenomena could be explained in terms of optimization principles.

Once again, however, the march of science got bogged down in the mire of petty controversy. Samuel König, a mathematician and student of Leibniz’s philosophy, publicly accused Maupertuis of plagiarizing Leibniz. Leonard Euler rose to his president’s defense. Voltaire –poison pen in hand – countered on behalf of König. The debate that ensued did little to resolve the issue and no doubt contributed to the deterioration of Maupertuis’s health. It did have one good result, however. The controversy catalyzed Euler – the greatest mathematician and physicist of his time – to formulate a rigorous version of the Principle of Least Action and to apply it to cases that were beyond Maupertuis’s impressive but merely mortal abilities. Remarkably, Euler came to hold essentially the same opinion as Leibniz. He concluded that “all natural effects follow some law of maximum or minimum” – that is some principle of optimization – so that “nothing whatsoever takes place in the universe in which some relation of maximum and minimum does not appear.”

In the years that followed Euler’s pioneering efforts, other luminaries of the age of rational mechanics continued to develope optimization principles and confirm their essentially universal applicability. The French-Italian mathematician and astronomer, Joseph-Louis Lagrange generalized Euler’s results, showing how optimization principles could be derived from principles of virtual work as well as from Newton’s laws. With Lagrange we finally get the general principle that – for each particle in a conservative system – the particle’s action taken from its initial position to its final position is optimal. A few decades later, the great Irish mathematician, William Rowan Hamilton – who had already made important contributions to the study of optics – further generalized Lagrange’s pioneering work. Hamilton’s generalized version of the Principle of Least Action is applicable not only to the cases considered by Lagrange but to non-conservative systems as well.

___

It was applied to specific problems in the 17th century, generalized in the 18th and 19th centuries, confirmed in the 20th century, and remains at the foundations of our best physical theories today.

___

Today, least action principles are expressed in what is known as “Lagrangian formulation,” and it is accepted that for any physical system one can uniquely specify a function, called the “Lagrangian,” that is determined by the nature of the system as a whole. Given a Lagrangian, one can (in principle) determine the actual sequence of a system’s states by considering all its possible states and identifying that sequence of states that “optimizes” its action. The Lagrangian of a system applies, in one form or another, to all current physical theories including general and special relativity, quantum mechanics, and even string theory. Reflecting in the mid-twentieth century on “the more or less general laws which mark the achievements of physical science during the course of the last centuries,” the founder of quantum mechanics, Max Plank, concluded that “the principle of least action is perhaps that which, as regards form and content, may claim to come nearest to that ideal final aim of theoretical research.”

In an article written five years before his death, Albert Einstein proposed that every theoretical physicist is “a kind of tamed metaphysicist.” The philosopher and physicist alike must believe that “the totality of all sensory experience can be ‘comprehended’ on the basis of a conceptual system built on premises of great simplicity.” They must have faith that the world is governed by a hidden, simple order. “The skeptic,” Einstein suggested “will say that this is a ‘miracle creed.’” And, Einstein acknowledged, she’ll be right. Nonetheless, while the miracle creeds of philosophers and physicists must, by definition, outstrip all empirical evidence – while they must be audacious – many have, as Einstein put it, been “borne out to an amazing extent by the development of science.”

Leibniz’s principle of optimality is perhaps the most miraculous of all miracle creeds. Rooted in an implausible conviction that this is the best of all possible worlds, it was applied to specific problems in the seventeenth century, generalized in the eighteenth and nineteenth centuries, confirmed in the twentieth century, and remains at the foundations of our best physical theories today. Leibniz’s principle of optimality is no doubt a miracle creed worthy of the sceptic’s incredulous stare. It implies that for all the world’s faults, there is a sense in which it and its parts are indeed optimal. And it shows that even an implausible idea, born of faith and hope, might bear long term, concrete results. In a world currently beaten down by disease, uncertainty, and conflict, Leibniz’s Principle of Optimality has somehow triumphed, a small victory for optimism in a pessimistic time.