Hidden Variables

I've been having a few conversations about hidden variables lately, so I thought I would post about it. First a little background:

Quantum mechanics (QM), as we know it, is weird. It is a classic example of science as a selection process, as I talked about in the previous post. It set about to solve the problem of predicting the locations, energies, and other microscopic attributes of fundamental particles. When the dust settled, we had one of the most accurate theories ever made (it predicts the mass of the electron to 14 decimal places), but the model used to make these predictions countermanded a lot of things we'd thought were true but never actually got around to testing--intuitive things like "a particle can only be one place at a time", well-established things like "no particle may carry information faster than the speed of light" (which is true, but can be violated over short distances), and most importantly for this post "if you had enough information, you could determine the outcome of any experiment." Nature's rejection of this last idea comes dangerously close to undermining the pillar of scientific philosophy that the universe is predictable insofar as it can be modeled and tested, and it offended a lot of scientists (including Einstein).

What QM says is that there are certain pieces of information that are not simultaneously knowable. If you know a particle's position perfectly accurately, then it is impossible to know its momentum. If you know a particle's orientation along one axis, you can't know it along any other axis. For a long time, many people (including Einstein) thought that this was a shortcoming of QM--that these particles have "actual, hidden values" that QM just didn't know how to predict, and eventually there would be a better theory that could tell us what these values are. Those hopes were shattered in 1964 when John S. Bell proved that there can exist no hidden variables in a way that is compatible with QM. His predictions have been validated by experiment, showing that the reason we don't know the state of a particle is because the universe hasn't made up its mind (excuse the anthropomorphification) until you measure the particle. Crazy.

I like to think of QM like a black curtain at a magic show that allows the universe (the magician) to perform all sorts of sleight-of-hand shielded from the eyes of the audience. On our side of the curtain, there are rules you have to follow--conservation of energy, speed of light, definity of location and state, etc. On the other side of the curtain, anything can happen, so long as when you pull it back out (when we make a measurement), the rules have been obeyed. The "curtain" idea isn't so far-fetched; it's an analogy to Feynman's well-tested theory that the outcome of an interaction is the sum over all possible interaction pathways. The universe takes advantage of this curtain to do things that we think should be impossible, like transmitting information about a measurement instantaneously between two particles that share a quantum state. Furthermore, the universe relies on the fact that we can never see behind the curtain (this is an interpretation of Bell's theorem) because if we could, we could use that machinery to transmit our own information faster than the speed of light, and that violates causality. Causality, by the way, is another principle that we cling to because it seems self-evident, but may in fact be wrong. It will take a unification of the theories of QM and general relativity to sort that out.

Shifting gears into philosophy, I was talking with my uncle, who came to visit last week, about how people look for meaning in their lives. His argument was that back in human history, when the universe seemed a jumble of arbitrary events, a physical, all-powerful deity with direct control over all that happened was a powerful metaphor for finding meaning in the events of ones life. However, as scientific knowledge has gradually encroached on the idea that a god can take direct physical action in the universe, religion has had to respond to the sense that science is pushing away meaning in life. My uncle's thoughts were that a physical deity is becoming an outdated way of looking for meaning in life, and that we need to think about a more spiritual, humanistic God. I agree with this philosophy, but I wonder if the rejection of the hidden variable hypothesis (the magician's curtain) provides a home for religions that require a physical deity.