Nick Bostrom reasoned that if any civilisation might feasibly produce an 'ancestor simulation', we're almost certainly living in a simulated world. Rizwan Virk explains why reaching this simulation point is no longer a fantasy, and why video games might offer the best description of reality.
While most people first considered the idea that we may be living inside a computer simulation after seeing The Matrix - the blockbuster movie which came out in 1999 - it is really the modern version of an old argument that the world around us isn’t the “real world”. Versions of this argument have ranged from Plato’s allegory of the cave to Descartes dreams about an “evil demon” that was deceiving him, and different variations have played a part in almost every major religious tradition.
The “new” element that has brought simulation theory to the forefront, is the development of computation. Perhaps one of the first modern incarnations of this idea came from science fiction writer Philip K. Dick (whose stories, incidentally, served as inspiration for The Matrix). He declared at the Metz sci-fi conference in 1977 that “we are living in a computer programmed reality”. Dick’s speech was looked on with eyebrows raised, almost as if he’d gone mad.
Cutting edge science at HowTheLightGetsIn Global Read more Fast forward a quarter century or more, and suddenly this idea is being taken seriously by everyone from technology moguls like Elon Musk, physicists like Neil DeGrasse Tyson, and philosophers like Oxford’s Nick Bostrom, whose landmark 2003 paper, Are You Living in A Computer Simulation, helped ignite the current wave of interest.
What happened in the intervening years? In short, video games happened.
In 1977, when Dick made his proclamation, video games were in their infancy, and coincidentally this was the year that Atari released the 2600 VCS. It wasn’t until the early 1980s when video games went from a novelty to something that “everyone had” in their home. In fact, the Atari VCS, complete with games like Space Invaders and Pacman, were my introduction to video games. If you were to ask anyone at the time if we could render all of the pixels in a three-dimensional depiction of the world, the answer would have been that it wasn’t possible: there are just too many pixels and not enough computing power.
Today, reaching the Simulation Point by creating a virtual world that is indistinguishable from the physical world doesn’t seem that far off.
In the intervening years, not only has computing power grown significantly (thanks to Moore’s law of processor speed doubling every 18 months), but the history of computing has been intertwined with the history of video games, which always required optimizations to get the best performance out of the machine. GPU’s, or Graphics Processing Units, were created to make rendering faster, and 3D Modeling, texturing and most importantly, conditional rendering, have all played important roles in moving us closer to the “Simulation Point”, a theoretical point at which we can build a video game that is indistinguishable from physical reality, like that depicted in The Matrix.
Today, video games have advanced apace to the point where rendering a three dimensional world with millions of online players concurrently is now possible, as shown by games such as World of Warcraft and Fortnite. In fact, the world’s largest music concert was held on Fortnite, with over 1 million players attending concurrently.
Today, reaching the Simulation Point by creating a virtual world that is indistinguishable from the physical world doesn’t seem that far off. A few years ago, I was playing a virtual reality ping pong game. The movements and reactions were so realistic that I forgot I was inside a virtual world. The experience was so authentic that I put down my virtual paddle on the virtual table at the end of the game and even leaned against the table, just like I might do after a “real” game of table tennis. Of course, there was no table – the controller fell to the ground and I almost fell over.
So, what would it take to build a simulation as realistic as the Matrix? It turns out that pixel resolution isn’t the issue. If you watch a movie like the new Star Wars movies or Blade Runner 2049, the special effects, which are built with the same tools that are used by video game developers, merge seamlessly with the live actors amidst a mixed reality landscape. Most movies today are transmitted at 2K pixel resolution; we already have screens that are at 4k and 8k pixels, and there are VR headsets which are looking to duplicate the pixel resolution of our eyes. Augmented reality (or AR), which has made leaps and bounds over the past years, attempts to put digital objects into the physical world around us. Today, we need AR glasses to see these objects, but new light field displays are being created that will allow us to place digital objects into the physical world without the need for glasses.
Simulation theory has now moved beyond philosophy into the sciences.
These are all among the stages on the road to the Simulation Point. In The Matrix, Neo’s character wakes up outside of the simulation and it turns out that he is living in a pod with a wire connected into his cerebral cortex, beaming signals in and out.
Would this be possible? Today we are in the infancy of Brain Computer Interfaces (BCIs), with companies like Elon Musk’s Neuralink having created implantable chips that will enhance the power of the human brain, and Neurable, which use mind-controlled VR headsets to understand how a user wants to navigate in the virtual world. While we aren’t at the point where we can send in an electrical signal and suddenly “know Kung Fu” as Keanu Reave’s character did in The Matrix, this may not be that far off either as BCIs develop.
Estimates for getting to the Simulation Point range from a decade to 100 years. Certainly, almost everyone agrees we’ll be able to achieve full immersion within 200 years, which in the overall scheme of the history of the Earth or the Universe, isn’t very much time at all.
So how does this relate to the idea that we live inside a video game?
In what Bostrom called the Simulation Argument in his 2003 paper, he stated that if any civilization ever gets to the point of creating realistic “ancestor simulations”, then we are almost certainly inside one of these. His argument was that there would be many more simulated beings than organic beings, because you can simply turn on anther server and create another million or trillion beings in no time at all. Simple statistics then shows that if you are a being, then you are more likely to be a simulated being than one in “base reality”.
But simulation theory has now moved beyond philosophy into the sciences. As I started to investigate the technology needed to build more realistic simulations, I found striking parallels to how the physical world works.
In a computer generated reality, there are no “real objects” - they are just being rendered as pixels by the rendering engine of the video game. In our “physical reality”, the more that physicists try to find this thing called matter, the more elusive it becomes. Physicist John Wheeler said that physics went through several phases in his lifetime– in the first phase, everything was a physical particle; then physicists decided that everything was a field; finally, he says physics came to the conclusion that everything was information. When you get down to it, there is no matter, just properties of so-called “particles”. Wheeler coined the term “It from bit”, emphasizing that everything we think of as physical is actually just information.
Perhaps the biggest mystery in quantum physics has been the “observer effect”, or “quantum indeterminacy”, which says that unless someone or something measures or observes a choice, then all possibilities exist until the observation is made. While there is some debate about whether the observer has to be a conscious person or a measurement device, there is no opposition to the idea that quantum indeterminacy exists - we just can’t figure out why. This is what led Niels Bohr to make the statement that if you aren’t shocked by what quantum mechanics tells us, then you haven’t understood it.
Quantum indeterminacy could be an optimization technique for a simulated reality.
It turns out that the key technique that has allowed video games to move from flat 2D games like Pac Man and Space Invaders to three dimensional immersive worlds is what we call “conditional rendering”. Rather than rendering all the pixels in a 3D scene, we only render the pixels that your avatar, a representation of you, can see. This is a key technique that we use to give the appearance that your avatar and mine are in the same room – when in fact, we are rendering on different computers. In fact, in video games, there is no “shared rendering”. The golden rule in video games is to only use computing power to render that which can be seen. It seems strikingly similar to how the world of quantum indeterminacy works – starting with a wave of probabilities and then only rendering those probabilities when they are observed. Quantum indeterminacy could be an optimization technique for a simulated reality – and just like that something which was completely baffling makes sense.
But computer scientists and physicists are really the newcomers to the debate about the simulation hypothesis. Many of the world’s religions have been claiming that the physical world around us isn’t the “real world” for thousands of years. In the Hindu and Buddhist traditions, there exists the idea that the world we think of as real is actually part of “maya” or “illusion”. In the Judeo-Christian traditions, including Islam, there is the idea of “recording angels” who are watching our every move and recording them in the Book of Life and the Scroll of Deeds. Even Shakespeare seemed to agree in one of his most recognisable lines: “All the world’s a stage, and all the men and women merely players.”
Of course, these religions were formed long ago, and the metaphors they used made sense at the time. But as poetic as these metaphors are, they are incomplete. If Shakespeare were alive today, I like to think he might say: “All the world’s a video game, and the men and women are merely avatars in this simulated virtual world”.