CEO and co-founder of Google DeepMind, Demis Hassabis, has recently speculated that information might be as fundamental as mass and energy. As AI hype rolls on, many like him are increasingly tempted to apply computing principles to reality as a whole. Those who support this view rely on the idea that information has a thermodynamic cost, as proposed by physicist Leo Szilard. Here, philosopher John D. Norton argues that information is not fundamental. Considerations of information and computing complicate thermodynamic analysis and contribute precisely nothing.
An intriguing idea
Is there a profound connection between the abstract realms of information and computation and the physical realm of heat and thermodynamics? Do learning and computing necessarily involve creation of the physical quantity, thermodynamic entropy, that governs which thermal processes can occur? Might it be that learning one bit of information comes at a cost of the creation of at least k log 2 of thermodynamic entropy, as Leo Szilard suggested? Or might this thermodynamic cost really come when we erase this one bit of information, as Rolf Landauer proposed? Are information and computation connected at a fundamental level to the laws that govern steam engines? Are information and computation connected at a fundamental level to the laws that govern steam engines?
This intriguing idea is suggested by a formal coincidence in two expressions. If a thermal system at equilibrium adopts a state i with probability pi, then we compute its thermodynamic entropy by summing the expression –k pi log pi over all states i, where k is Boltzmann’s constant. This same “p log p” formula, without the constant k, is the measure of information in Shannon’s celebrated information theory. The enticing supposition is that this coincidence of formulae can underwrite a deep connection between information and steam engines.
___
There is no foundational connection between information, computation and thermodynamics.
___
This coincidence of formulae is just a first, slender indication of an intriguing possibility. Whether the connection suggested is real requires considerable further investigation. We should not be hasty. A closer look at the coincidence of formulae suggests that there is no deeper connection. The p log p formula is just a parameter that measures how spread out a probability distribution is. In this regard, this entropy parameter is akin to statistical parameters like the mean and variance of a probability distribution. They appear everywhere without signifying a deeper connection. That this same p log p parameter appears in two investigations is no basis for inferring that their probability distributions have the same meaning. Prima facie, they do not in the case of thermodynamics and information.
Dynamic probabilities of thermal equilibrium
10th July 2026
Join the conversation
Brian Balke 10 July 2026
Thank you. I hope that you turn your attention towards disentangling the confusion in quantum mechanics between stochastic (entropic) randomness and fundamental quantum uncertainty. This confusion has stimulated a huge body of literature that misleads the public regarding, for example, the prospects for quantum computing. To illustrate, you can build a large number factoring machine in a swimming pool, just so long as your wave generator has sufficient bandwidth. This is the breakthrough that needs to be elevated: the generation of extremely broad-bandwidth light sources. The "quantum" aspects of the system are unimportant.
That said, I will note that Bateson, in "Mind and Nature," discusses how entropy is essential to our processing of information. Perception itself depends upon entropic gradients, and creative intellectual leaps can be linked to the instability of neuronal activation potentials. What is typical of much of modern physics, however, is to take a derivative concept (such as information) and treat it as axiomatic. The theorists then amuse themselves with "proofs" that they can "derive" the axioms of the original theory.