33.2 Calculating randomness by Benjamin Crowell, Light and Matter  licensed under the Creative Commons Attribution-ShareAlike license.

33.2 Calculating randomness

You should also realize that even if something is random, we can still understand it, and we can still calculate probabilities numerically. In other words, physicists are good bookmakers. A good bookie can calculate the odds that a horse will win a race much more accurately that an inexperienced one, but nevertheless cannot predict what will happen in any particular race.

Statistical independence

As an illustration of a general technique for calculating odds, suppose you are playing a `25`-cent slot machine. Each of the three wheels has one chance in ten of coming up with a cherry. If all three wheels come up cherries, you win `$100`. Even though the results of any particular trial are random, you can make certain quantitative predictions. First, you can calculate that your odds of winning on any given trial are `1"/"10×1"/"10×1"/"10 =1"/"1000=0.001`. Here, I am representing the probabilities as numbers from `0` to `1`, which is clearer than statements like “The odds are `999` to `1`,” and makes the calculations easier. A probability of 0 represents something impossible, and a probability of `1` represents something that will definitely happen.

Also, you can say that any given trial is equally likely to result in a win, and it doesn't matter whether you have won or lost in prior games. Mathematically, we say that each trial is statistically independent, or that separate games are uncorrelated. Most gamblers are mistakenly convinced that, to the contrary, games of chance are correlated. If they have been playing a slot machine all day, they are convinced that it is “getting ready to pay,” and they do not want anyone else playing the machine and “using up” the jackpot that they “have coming.” In other words, they are claiming that a series of trials at the slot machine is negatively correlated, that losing now makes you more likely to win later. Craps players claim that you should go to a table where the person rolling the dice is “hot,” because she is likely to keep on rolling good numbers. Craps players, then, believe that rolls of the dice are positively correlated, that winning now makes you more likely to win later.

My method of calculating the probability of winning on the slot machine was an example of the following important rule for calculations based on independent probabilities:

the law of independent probabilities

If the probability of one event happening is `P_A`, and the probability of a second statistically independent event happening is `P_B`, then the probability that they will both occur is the product of the probabilities, `P_AP_B`. If there are more than two events involved, you simply keep on multiplying.

This can be taken as the definition of statistical independence.

Note that this only applies to independent probabilities. For instance, if you have a nickel and a dime in your pocket, and you randomly pull one out, there is a probability of `0.5` that it will be the nickel. If you then replace the coin and again pull one out randomly, there is again a probability of `0.5` of coming up with the nickel, because the probabilities are independent. Thus, there is a probability of `0.25` that you will get the nickel both times.

Suppose instead that you do not replace the first coin before pulling out the second one. Then you are bound to pull out the other coin the second time, and there is no way you could pull the nickel out twice. In this situation, the two trials are not independent, because the result of the first trial has an effect on the second trial. The law of independent probabilities does not apply, and the probability of getting the nickel twice is zero, not `0.25`.

Experiments have shown that in the case of radioactive decay, the probability that any nucleus will decay during a given time interval is unaffected by what is happening to the other nuclei, and is also unrelated to how long it has gone without decaying. The first observation makes sense, because nuclei are isolated from each other at the centers of their respective atoms, and therefore have no physical way of influencing each other. The second fact is also reasonable, since all atoms are identical. Suppose we wanted to believe that certain atoms were “extra tough,” as demonstrated by their history of going an unusually long time without decaying. Those atoms would have to be different in some physical way, but nobody has ever succeeded in detecting differences among atoms. There is no way for an atom to be changed by the experiences it has in its lifetime.

Addition of probabilities

The law of independent probabilities tells us to use multiplication to calculate the probability that both A and B will happen, assuming the probabilities are independent. What about the probability of an “or” rather than an “and?” If two events A and B are mutually exclusive, then the probability of one or the other occurring is the sum `P_A+P_B`. For instance, a bowler might have a `30%` chance of getting a strike (knocking down all ten pins) and a `20%` chance of knocking down nine of them. The bowler's chance of knocking down either nine pins or ten pins is therefore `50%`.

It does not make sense to add probabilities of things that are not mutually exclusive, i.e., that could both happen. Say I have a `90%` chance of eating lunch on any given day, and a `90%` chance of eating dinner. The probability that I will eat either lunch or dinner is not `180%`.

Normalization

If I spin a globe and randomly pick a point on it, I have about a `70%` chance of picking a point that's in an ocean and a `30%` chance of picking a point on land. The probability of picking either water or land is `70%+30%=100%`. Water and land are mutually exclusive, and there are no other possibilities, so the probabilities had to add up to `100%`. It works the same if there are more than two possibilities `-` if you can classify all possible outcomes into a list of mutually exclusive results, then all the probabilities have to add up to `1`, or `100%`. This property of probabilities is known as normalization.

Averages

Another way of dealing with randomness is to take averages. The casino knows that in the long run, the number of times you win will approximately equal the number of times you play multiplied by the probability of winning. In the game mentioned above, where the probability of winning is `0.001`, if you spend a week playing, and pay `$2500` to play `10,000` times, you are likely to win about `10` times`(10,000×0.001=10)`, and collect `$1000`. On the average, the casino will make a profit of `$1500` from you. This is an example of the following rule.

rule for calculating averages

If you conduct `N` identical, statistically independent trials, and the probability of success in each trial is `P`, then on the average, the total number of successful trials will be `NP`. If `N` is large enough, the relative error in this estimate will become small.

The statement that the rule for calculating averages gets more and more accurate for larger and larger `N` (known popularly as the “law of averages”) often provides a correspondence principle that connects classical and quantum physics. For instance, the amount of power produced by a nuclear power plant is not random at any detectable level, because the number of atoms in the reactor is so large. In general, random behavior at the atomic level tends to average out when we consider large numbers of atoms, which is why physics seemed deterministic before physicists learned techniques for studying atoms individually.

We can achieve great precision with averages in quantum physics because we can use identical atoms to reproduce exactly the same situation many times. If we were betting on horses or dice, we would be much more limited in our precision. After a thousand races, the horse would be ready to retire. After a million rolls, the dice would be worn out.

self-check:

Which of the following things must be independent, which could be independent, and which definitely are not independent?

(1) the probability of successfully making two free-throws in a row in basketball

(2) the probability that it will rain in London tomorrow and the probability that it will rain on the same day in a certain city in a distant galaxy

(3) your probability of dying today and of dying tomorrow

(answer in the back of the PDF version of the book)

Discussion Questions

A   Newtonian physics is an essentially perfect approximation for describing the motion of a pair of dice. If Newtonian physics is deterministic, why do we consider the result of rolling dice to be random?

B   Why isn't it valid to define randomness by saying that randomness is when all the outcomes are equally likely?

C   The sequence of digits `121212121212121212` seems clearly nonrandom, and `41592653589793` seems random. The latter sequence, however, is the decimal form of pi, starting with the third digit. There is a story about the Indian mathematician Ramanujan, a self-taught prodigy, that a friend came to visit him in a cab, and remarked that the number of the cab, `1729`, seemed relatively uninteresting. Ramanujan replied that on the contrary, it was very interesting because it was the smallest number that could be represented in two different ways as the sum of two cubes. The Argentine author Jorge Luis Borges wrote a short story called “The Library of Babel,” in which he imagined a library containing every book that could possibly be written using the letters of the alphabet. It would include a book containing only the repeated letter “a;” all the ancient Greek tragedies known today, all the lost Greek tragedies, and millions of Greek tragedies that were never actually written; your own life story, and various incorrect versions of your own life story; and countless anthologies containing a short story called “The Library of Babel.” Of course, if you picked a book from the shelves of the library, it would almost certainly look like a nonsensical sequence of letters and punctuation, but it's always possible that the seemingly meaningless book would be a science-fiction screenplay written in the language of a Neanderthal tribe, or the lyrics to a set of incomparably beautiful love songs written in a language that never existed. In view of these examples, what does it really mean to say that something is random?

33.2 Calculating randomness by Benjamin Crowell, Light and Matter  licensed under the Creative Commons Attribution-ShareAlike license.