THERE IS a quote from Nate Silver about statistics which I think is very nice: “The numbers have no way of speaking for themselves. We speak for them, we imbue them with meaning.” You can’t just collect some data and it’ll tell you the answer. There is an art to trying to extract information, knowledge and understanding from data, and even in choosing what data to collect. It’s something we’ve all been dealing with over the past few months with covid-19: can we believe these numbers? What do they mean?
Now if this were a live audience, I’d be asking how many people have done stats courses. If people put their hand up, I’d ask how many people actually enjoyed them, and most of the hands would go down. That makes me upset. I love statistics, I think it’s great. But it has tended to be taught in the past as a series of formulae and tests and regression and things like that.
Advertisement
My book The Art of Statistics takes a very different approach. It spends a lot of time on problem solving, on things like: what are you trying to do? Is this data suitable for what you’re trying to answer? What can we conclude from it? It’s amazing how far you can get without ever doing any fancy statistical methods or using probability theory or the sample distribution of the sample mean and all this sort of stuff we all had to endure – and which I’ve always taught, of course.
The key is what is called the “data cycle”. You don’t start off with data, you start off with a problem. You plan how are you going to try to answer it. Is there any data, and what might we collect? Then you collect data and wrangle it and manage it and clean it up. Only then do you come to the analysis. That’s normally the only thing that is taught in stats courses, but it’s only a small part of the whole cycle.
It’s followed by the communication, drawing the appropriate conclusions, putting the message out. And there always you have to start again. Because as we’re going to see again and again, how you do an analysis just leads to more questions.
I find the data cycle an immensely powerful way to structure the use of data to solve problems. How many sexual partners have people had? Is it worth me taking statins? Who was the luckiest person on the Titanic? What’s the probability that the skeleton found in that Leicester car park really was Richard the Third? Why do old men have big ears? I mean, these really important questions for the future of humanity.
Many of these issues I’ve actually been working on. An example I use in the book is that of the doctor Harold Shipman. He murdered at least 215 of his patients, and probably considerably more, over 24 years working in the Hyde suburb of Manchester. I worked on the Shipman inquiry as one of the statisticians who were asked to look at the data and answer various questions. The first question really was, what was happening?
So here’s a graphic that just shows the pattern of his murders: the red dots are women, the blue are men. The histograms at the top and on the left show the distribution of the years of the killings and the age of the victims. So just from this data alone, the ages and sex of the victims and the dates of their deaths, what can we conclude?
Is he mainly killing old people? Yes, but it looks like more near the end, some of his victims were much younger. And do you notice that gap of a year – did he go on holiday? No, up to then he had been working in a joint practice, and tt’s thought that he suspected that he was being suspected. After that, he set up his own single-handed practice, and after that there’s no sort of control over him. By the time he was arrested, he was murdering people at a huge rate, quite extraordinary. We don’t know why: he never spoke, and then committed suicide in prison.
So this data just generates more questions. What was Shipman doing? What system was he using? How was he killing people? The general practitioner Richard Baker went and looked at all the death certificates that Shipman had signed, and those his colleagues had as well. It was a massive data collection exercise, but the analysis was completely trivial. It hits you between the eyes. The times of day when his colleagues’ patients had died were spread pretty uniformly over 24 hours, but Shipman’s victims were dying around two or three in the afternoon. We think that is when he did home visits, visiting elderly people on their own and giving them a huge dose of morphine. They would die in front of him peacefully. It’s chilling.
Again, it generates more questions. All the families wanted to know, could he have been caught earlier? Now this is tricky. Those of you who have suffered statistics courses will know that to test a hypothesis, you have to set up a null hypothesis. A null hypothesis is boring: in this case, that there’s nothing wrong with Shipman at all. How soon could we have rejected that hypothesis – how soon could we have detected something strange was going on?
So you work out how many deaths you expected to occur over time with different sex and age groups if things were normal. You can compare that with the observed number, subtract the one from the other and you get excess mortality. The reality is a bit more complex and you actually use a slightly different statistical measure. But in Shipman’s case you can say that, based on female deaths alone, by 1985, after only about 40 deaths, we could have been very confident something was strange. Of course nobody did do that at the time; it was no one’s job to do that.
The other interesting thing is that when they tried this method out on 1000 other doctors around the country, they found a few who were even worse than Shipman. It turned out that these were enormously generous and responsible, kind, caring doctors, who were just living in places with a lot of elderly people and allowing them to die peacefully at home. That’s the difference between correlation and causation: we could conclude that these doctors are statistically odd, but not why. We have to be enormously cautious. That data does not speak for itself.
The covid mortality spike
So let’s look at coronavirus. We can look at the data for deaths in England and Wales, as in the diagram. The blue is the deaths happening with covid not mentioned on the death certificate, and the red is deaths with covid mentioned. The base line is the five-year average of deaths for the same period. We can see a massive spike starting at the end of March, but we can see that now, even allowing for continuing covid deaths, there are fewer deaths than normal. Is it just continuing the pattern of lower deaths that we observed early in the year, when we had a milder flu season than normal? Or is this the first sign of what’s called mortality displacement, in which deaths of frail elderly people that would have occurred later in the year have been brought forward?
And there are other fascinating patterns going on when you look closer. For example, we know that young men between 20 and 24 had lower mortality than usual over the lockdown period, at one point 30 per cent down, even including a few covid deaths. We don’t know the details, but we can probably guess it might have to do with fewer car accidents and maybe less going out getting drunk.
Displaced deaths
But let’s go deeper: let’s look at the place in which people are dying. Let’s look at care homes. There’s been a big spike in deaths in care homes, but with a lot of non-covid deaths. It’s thought many of these could be covid deaths, especially of elderly people with dementia, just not labelled as such on the death certificate. It’s very difficult to make a definite diagnosis in these cases, and doctors have been unwilling to put it on the certificate.
When we look at hospitals, we can see this big rise starting in the middle of March. This is when they emptied the hospitals and sent people back home without testing whether they had covid, or sent people back to care homes. It’s been strongly suggested that that was one of the causes of the rise in deaths in care homes, along with the staff moving between different institutions.
But there is also a dip in non-covid deaths in hospitals. There’s been a huge reduction in attendance in hospitals of people with heart symptoms, with strokes and so on. Where have these people been dying? Well, they’ve been dying at home. It’s extraordinary. There has been an enormous number of extra deaths at home, some 15,000 over the period we’re looking at – and that’s still happening. As we speak, there’s still a deficit of non-covid deaths in hospital and an excess in the home.
Again, that just raises more questions. We don’t know the quality of these deaths. Most people would prefer to die at home; dying in hospital in this current crisis has not been a good thing. But maybe some of these people might have lived longer, had they gone to hospital. So this just raises more questions, as we go round and round that data cycle.
Coronavirus and age
Let’s look now not just at the death rates, but the chance of someone in the population catching covid-19 and then dying. I think this is almost the most staggering bit of analysis I’ve ever done. In England and Wales, there is a completely exponentially increasing risk of catching it and dying from it for the different age groups. It increases by about 12 to 13 per cent for each year older you are, and so doubles for every five to six years. That means that someone who is 20 years older has got 10 times the risk: compared to a 25-year-old, a 45-year-old has ten times the risk, a 65-year-old 100 times, an 85-year- old 1000 times. Draw a straight line on the logarithmic scale representing exponentially increased risk and it just carries on across the age range all the way from essentially five to 95. I’ve never seen anything like that, it is quite extraordinary.
For school kids, five- to 14-year-olds, of 7 million in England and Wales, just three have died – a 1 in 2.5 million risk. Meanwhile, 138 have died of other causes over the period of the epidemic. So this is both a staggeringly low risk, and very low compared to the normal risk. But for people over 90, more than 2 per cent, one in 50, have died with covid. That represents about a one-third increase over the normal risk they would have had over this period.
The media have not been great always in covering this. The “risk of dying from coronavirus” is a very misused phrase. I’ve been talking about what’s called the population fatality rate, the chance of getting it and then dying. There’s another risk, which is if you get it, you die – the infection fatality rate. It’s very easy to mix those up.
That happened when the UK Office for National Statistics did a very good report on risks for ethnic minorities. That’s an incredibly important issue. All the risks that were discussed are the risks of catching it and dying, and that includes the increased risk of catching it that many ethnic minority communities clearly have, because of their jobs, or perhaps because of deprivation, overcrowding and poor working conditions. However, some reports had it that ethnic minorities were 90 per cent more likely to die if they became seriously ill with covid-19. That’s not true.
We’ve done lots of experiments at the Winton Centre, and we find that people have got a complete misapprehension about covid risk. Many people say, if I catch it, it’s 50/50 whether I die or not. People are very anxious. They really think if I get it, this is a very high-risk situation. It is for some people, but for most people, it isn’t, even if you catch it.
You can look at the actuarial risk of dying in a given year. I produced a graph back in March, before we’d had almost any deaths in the UK at all, saying that the risks if you get the virus are very similar to the risks that are there anyway for dying in the subsequent year. That means that if you get the virus it roughly doubles your chance of dying this year. I’d still argue that that’s roughly true. But I originally expressed this by saying the risks from covid were about the same as dying this year, which was very bad communication, as some people interpreted it meaning the death rate from the virus didn’t add to normal risk.
For a statistician, this has been quite a stressful time. It’s been great, but it’s been hard work, and I’ve got things wrong. But a lot of what we’ve seen is what I call number theatre, a lot of numbers put out there to impress people. This is not trustworthy use of statistics. People delivering the statistics should be proper professionals who know what they’re talking about, and they should treat the public with respect.
Data literacy is a vital skill in modern life. This covid crisis that made this even clearer: not just in the ability to manipulate data, but also to critique the numbers we are told.
Sniffing out the dodgy stats
In response to an audience question, David Spiegelhalter gave his tips for not being blinded by numbers
There are tricks, but it’s not a simple thing. A lot of it is feeling, what I call “sniffing the number”. My first question is always “why am I hearing this number?”: to be sceptical about the motivations of the people telling you the number. Are they trying to make it big or small? Are they trying to persuade me, rather than inform me? Almost always they’re trying to persuade.
That leads to subsidiary questions. What am I not being told about? Can I believe this number? Where does it come from? Does it actually represent what I think it represents? It’s a bit like judging fake news. You often can’t tell from the claim itself; you have to look outside and see what other people are saying about it, do what’s called horizontal searching. That’s a very basic skill that you can teach people. It’s being taught in US schools now to show people how not be taken in by fake websites.
Want to see David Spiegelhalter’s full talk?
Sign up for the event on-demand, including his answers to more audience stats questions, at
