## Tuesday, 6 November 2012

### Negative Probabilities & the Bhagvad Gita.

Here is a story about negative probabilities called Albert's Socks

## Spooky Socks

Once upon a time in a small university town, a natural philosopher called Albert filled his days contemplating life, the universe and everything. Like many of his colleagues, Albert struggled each morning to equip his feet with a matching pair of socks. Would you spot Albert on a number of days, chances are you would occasionally observe his shoes filled with a red left foot and a green right foot or any other combination of colors.

Although his absent-mindedness was strong enough to serve as explanation for any ill-fitting garments, Albert did have a valid excuse for his poor choice of outfit. His Danish housekeeper, Niela Bohr, kept his socks in a chest of drawers. Three rows, each consisting of three drawers, made up this piece of furniture. Whenever Albert pulled open a drawer in search for socks to wear, he would be presented either a pair of matching socks or a single sock. Every subsequent drawer he opened, would reveal socks of a color different from those in the drawers already opened. To make things worse, each drawer opened would block from opening all drawers not in the same row and all drawers not in the same column. This effectively limited Albert each morning to the opening of three drawers configured in a horizontal row or in a vertical column.

Each night Niela prepared the chest of drawers for the next morning. To Albert's frustration, he couldn't figure out what procedure Niela followed. Each morning when opening a line of three drawers, the outcome came to him as a compete surprise. Albert labeled the drawer rows A, B and C, and the columns X, Y and Z, and started recording his observations. Each morning he wrote down a line like  B121, indicating the opening of the drawers in row B containing 1, 2, and 1 socks respectively.

Following a few weeks of observations, Albert has recorded the following set of data:

C112     B222     X212     Y111
A211     Z111     Y221     B121
Y221     X122     A112     A222
B112     C211     Z212     X221
Z122     Y212     Y122     Z221
A121     C112     C121     B211

When questioning Niela about the way she filled the chest of drawers each day, she responded that she didn't fill the drawers, rather she prepared them according to the laws of quantum physics. "What do you mean you don't fill the chest of drawers?" Albert asked, "surely you fill it as I have never encountered an empty drawer." Niela hesitated. "Sir, this is a quantum chest. There is no reality associated with the contents for each drawer." Albert looked puzzled. "You mean the unopened drawers don't contain any socks?" Albert focused at her face. Was she making a joke? She seemed perfectly serious. "Sir, an observation not made is a non-existent observation. Now if sir would please excuse me, I need to wash sir's socks for tomorrow and prepare sir's chest of drawers." And off she went.

Albert thought about Niela's puzzling remarks. It all didn't make sense. He knew about this weird quantum theory. A statistical theory that he was sure, could not represent the deepest truth of nature. He knew for a fact that each time all drawers are filled. If that was not the case, surely he would on occasions have hit an empty triplet of drawers. There must be some explanation. Probably she was playing a game with him, and filling the drawers according to some secret allocation algorithm.

Months go by, the list of drawer observations kept growing, but Albert didn't manage to work out the algorithm. One day, he explains the issue to his colleague, Jim Bell. Jim was a practical guy and an expert on quantum theory. "Can I have a look at the data?", he asked. Albert handed over a sheet of paper. It took Jim only a few seconds to remark "This is interesting, a horizontal line of drawers always contains an even number of socks, while a vertical line always contains an odd number of socks". He handed back the paper to Albert, who once more  inspected the data. His mouth opened. With his eyes wide open and still fixed on the paper, he uttered "But this is impossible". Jim smiled, "Well, the results are puzzling indeed. But those are your own observations. If you doubt them, you have to redo them."

Albert was still staring at the paper, and didn't look up. "This really is impossible. If at any given morning I would open three rows, I would end up with an even number of socks. But would I open three columns I would end up with an odd number of socks. Yet in both cases I would have opened the same nine drawers. This is absolutely impossible."

"Right. Albert, can I remind you that you started by telling me that chest of yours contains quantum drawers and that on each given day you can open only one row or one column of drawers at a time?"
"Yes, but let's assume, just for sake of argument, that we can open all drawers."
"Albert, you have to make up your mind. Can you, or can you not open all drawers? If not, then you should realize it is not that you don't know the facts about the contents of the drawers that can not be opened, there simply aren't any such facts."

Hours later, back at home Albert was staring at his spooky drawers. He had checked the data many times. There was no doubt, Jim's observation on even and odd sock counts was correct. Jim had tried to convince him it is meaningless to discuss the contents of drawers that can not be opened. But still, a-priori there is no drawer that can not be opened. Each morning he can decide to open any of the nine drawers, it is just that already opened drawers limit the opening of subsequent drawers. So each drawer must contain either one or two socks. Or not? This quantum stuff was really driving him crazy.

Could it be that the chest contained a hidden mechanism that played tricks on him? Maybe the socks could move from one drawer into the other based on the drawers that he opened. The next few mornings Albert checked the drawers that he pulled open and inspected them for any hidden mechanics or other tricks. Nothing of that. There was no way for the socks to move from one drawer to the other.

Could it be that Niela knows in advance if he was going to select a row or a column of drawers? No, this is a crazy thought. Precognition is pseudoscientific nonsense. But physical reality not allowing him to talk about the contents of unopened drawers seemed even crazier. So what the heck. Albert took a die and marked it with the symbols A, B, C, X, Y and Z. Henceforth, each morning he threw the die and opened the row or column of the cupboard corresponding to the symbol on the die.

Basing the choice of the drawers to be opened on the throw of a die didn't change anything to the outcomes. Rows continued to come up with even numbers of socks, and columns with odd numbers of socks. Albert looked again at the chest of drawers. What a spooky device! A spooky and revealing cupboard that was telling him something deep about the nature of physical reality. His observations on drawer contents did not leave room for any other explanation than what Jim was telling him all along: we are living in an utterly strange quantum universe. A universe in which what could have happened but didn't has no meaning.

I suppose from an Anthropic, Many World, point of view, we're talking about a constraint on the outcome that our own 'quantum immortality' imposes.  What can't be denied is that experimental evidence exists for negative probabilities, for e.g. in Bell's theorem, because our 'common sense' says logical possibilities exist which tests proved to be wholly imaginary.

An interesting paper by Burgin (available here)  uses the following analogy to elucidate negative probability-
'Let us consider the situation when an attentive person A with the high knowledge of English writes some text  T. We may ask what the probability is for the word “texxt” or “wrod” to appear in his text T. Conventional probability theory gives 0 as the answer. However, we all know that there are usually misprints. So, due to such a misprint this word may appear but then it would be corrected. In terms of extended probability, a negative value (say, -0.1) of the probability for the word “texxt” to appear in his text T means that this word may appear due to a misprint but then it’ll be corrected and will not be present in the text T.   Negative probability becomes even less (say, -0.3)  when people use word processors because misprints become more probable.  For instance, it is possible to push a wrong key on the keyboard or pushing one key also to push its neighbor.

In physics, negative probability may reflect the situation when instead of a particle its anti-particle appears. For instance, probability -0.3 that in a given interaction an electron appears means that there is probability 0.3 that in this interaction a positron appears.

The wikipedia article on negative probability states-
In Convolution quotients of nonnegative definite functions[6] andAlgebraic Probability Theory [7] Imre Z. Ruzsa and Gábor J. Székely proved that if a random variable X has a signed or quasi distribution where some of the probabilities are negative then one can always find two other independent random variables, Y, Z, with ordinary (not signed / not quasi) distributions such that X + Y = Z in distribution thus X can always be interpreted as the `difference' of two ordinary random variables, Z and Y.

I guess one way of looking at the 'difference' of Z & Y is to think of X as the site of a constraint or concurrency problem- i.e. there is some buffering or lagging or information bottleneck 'off screen'- which is picked up by the negative values in its probability distribution.

This is a link to a fascinating video lecture on why negative probabilities are essential in Machine Learning. The same points may be made for any Social or Cognitive Science.

Suppose an Occasionalist writer wanted to set up a plot in which every episode is a balanced game. One way to do so would be by creating characters in pairs so they either balance or  'cancel each other'- i.e. act as either commuting or non-commuting 'ghosts' such that either you get partial-avatars, on an analogy with bosons, or mimetic rivals, on an analogy with fermions

Another method of constraining the plot to conserve symmetries would be to introduce negative probabilities. But the sort of considerations discussed above show a deep connection between negative probabilities and anti-particles, with the former appearing more fundamental as representing the way our consciousness constrains Reality such that novelty, freedom, apoorvata, is the way we experience the lag between cause and the apurva effect.

This gives a- to my mind bitter- twist to the Gita as the dual of the episode where the Just King's Depression is dispelled by gaining instruction in Probability theory.