LOOK up at the night sky and you will see just one of a vast number of possible universes that might have existed. That’s a consequence of quantum mechanics, which was responsible for generating the initial ripples from which galaxies and stars later formed.
Quantum mechanics describes reality in terms of probabilities rather than specifics. This makes uncertainty an intrinsic ingredient in the standard model of cosmology – our best explanation of the origin and evolution of the universe.
For that reason, statistics is hugely important in cosmology. The standard model predicts the average properties of all possible universes, not the specific properties of our universe. If we see a discrepancy between our model and the real universe, this may be nothing more than a probabilistic fluke.
Advertisement
Over the past decade, cosmologists have measured in increasing detail the cosmic microwave background radiation (CMB) – the afterglow of the big bang. Its properties not only tell us about the history of the universe, but also give clues about physics at immensely high energies which experiments on Earth cannot conceivably test. So it is crucial that we interpret it correctly.
Vast numbers of scientists have pored over every detail of the CMB measurements. Some have found unexpected anomalies – the cold spot for instance, which has a much lower temperature than average, or the so-called axis of evil, which is an alignment of large-scale hot and cold patches.
According to the standard model, the chance of seeing such patterns is tiny. For example, the “correlation function anomaly”, which is related to the axis of evil, has odds of just 0.05 per cent. With tiny probabilities like this floating about, some cosmologists have concluded that the standard model must be wrong and that all its predictions about the universe should be called into question.
Accepting this dramatic assessment would mean giving up the vital assumption of statistical isotropy – that the universe should, on large scales, appear the same in every direction.
In the face of such a radical revision of cherished principles, we need to be sure we are asking the right question of the data. All we’ve calculated so far is the chance of seeing the anomalies under our standard assumptions. The question that needs to be asked is the opposite one – what is the chance of the standard assumptions being correct, given that these anomalies exist?
Asking the first question – what is the probability of seeing what we see? – is called the frequentist approach, and it can bamboozle. Imagine, for instance, tossing a coin five times and getting five heads. The frequentist would ask: what is the probability of this happening with a normal, unweighted coin? The answer is about 3 per cent, which makes it seem as if the coin must be biased.
But now ask the revised question: how likely is it that the coin is unweighted? Imagine yourself doing the experiment with a coin from your pocket and getting five heads. What would be your reaction? Most likely you would reject the idea that the coin is weighted. Intuitively, you know that most coins are not. Chances are what you saw was just a fluke.
This kind of feeling is not unjustified. You are using your prior knowledge of the world to inform your rational inferences. And this highlights our unease with frequentist statistics: they just don’t seem flexible enough to tell us about the real world.
Ruling out hypotheses on the basis of a frequentist interpretation of results can lead to catastrophically wrong conclusions. Suppose, for instance, that aliens are collecting samples of Earth life for study. Remarkably, the first organism they beam up happens to be an airline pilot. The aliens know that only about 1 in 2000 of the human population are pilots. Adopting frequentist reasoning, they would conclude that their subject cannot be human.
Luckily there is an alternative approach which eradicates the problem: Bayesian statistics. This takes into account both information from the experiment and, crucially, any relevant real-world information. If the aliens had applied Bayes’s theorem, they would have reached the correct answer: there is a 100 per cent probability that the organism is human, because no other Earth creatures are capable of piloting planes. That piece of information is vital, but is totally ignored by the frequentist analysis.
In a recent paper, we have argued that ruling out the entire cosmological model on the basis of a 0.05 per cent probability is similarly ill-advised (). In cosmology and elsewhere, Bayes tells us it is justifiable to be conservative in the face of statistical anomalies.
That is not to say we are desperate to preserve the status quo. But, after decades of patient data-gathering, the standard model has an enormous base of support. Bayesian statistics shows us that the anomalies in the data are insufficient on their own to motivate drastic revision. In the absence of a plausible new theory which explains all the data better, we simply can’t tell whether an anomaly is just a fluke.
Getting cosmology wrong has few real-world consequences. It is perhaps more worrying that statistical blunders of the kind made by some cosmologists seem ingrained in many other sciences. We are willing to bet that erroneous leaps of statistical faith are being made in other fields. Biology, medicine, economics and environmental science all rely on statistics to make sense of their models. It’s time to make Bayesian reasoning part of the standard model of science.
“Blunders of the kind made by some cosmologists seem ingrained in many other sciences”
