What’s all the fuss about?
At Saturday’s press conference, Sir Patrick Vallance and Professor Chris Whitty presented a graph suggesting that England could see 4,000 coronavirus deaths a day by mid-December.
The figure, produced by the University of Cambridge in conjunction with Public Health England, was presented as a reasonable worst-case scenario, assuming no changes in policy or behaviour.
The figure formed part of the government’s case for a second nationwide lockdown, but its validity has been questioned by the scientific community.
Why has the figure been criticised?
It’s been revealed that the scenario described was drawn up on 9 October, meaning it is three weeks out of date. According to the outdated modelling, deaths in England should now be around 1,000 a day. The actual figure is just over 200.
Professor Carl Heneghan and Dr Daniel Howdon have warned in a joint paper for the Centre for Evidence-Based Medicine that the projection used by the government could be between four and five times too high.
Cambridge University has twice updated the model used in the press conference as new data has come in. While the updated modelling does not look ahead as far as December, the trend it shows is far less alarming than the one presented on Saturday, showing significantly fewer deaths.
The latest version, updated on 28 October, estimates around 500 daily deaths by the middle of November rather than the 3,000 estimated in the outdated model. Sir Patrick acknowledged that the modelling was “from two weeks ago” but it is not clear why the updated version was not used.
The study’s methodology has also been called into question. The Cambridge model defines a Covid death as a death that occurs within 60 days of a positive diagnosis, as opposed to Public Health England’s 28 days. Doubling the cut-off time will inevitably lead to a higher daily death projection.
Prof Heneghan and Dr Howdon have also cast doubt on the ability of models to peer so far ahead: “Estimating beyond [two weeks in the future] gives rise to highly inaccurate estimates,” they write.
Sir Patrick and Professor Whitty have been summoned before MPs this afternoon to explain why they used the graph.
Do we know how the 4,000-death figure was reached?
The study has not yet been published, so the full range of assumptions underpinning it is not known.
The Cambridge team’s findings were not shared online, as previous work on the subject has been, but were instead sent directly to SPI-M, Sage’s Scientific Pandemic Influenza Group on Modelling.
Along with the ‘do-nothing’ assumption, Sir Patrick said that the scenario also assumed “that R stays above 1 – and goes between 1.3 and 1.5 – and possibly up – over the course of the winter.”
This is too vague to be of much use. With no upper limit given to the R number – the average number of people someone with Covid will go on to infect – plugged into these models, it’s difficult to tell how plausible the resulting estimates will be.
The previous slide showed a declining R number that had fallen from an estimated peak of 1.3-1.6 at the start of October to 1.1-1.3 in the last week of October. It was not made clear why this downward trend was set to change.
What do other models tell us?
The Cambridge/PHE model was one of several presented at the press conference. The other models – produced by Imperial College London, the London School of Hygiene and Tropical Medicine and Warwick University – estimated peaks of 2,700, 2,000 and 1,900 daily deaths respectively between mid-December and mid-January.
The figures, while lower than the Cambridge model, are surprisingly high, especially given that Sage’s own reasonable worst case scenario estimate over winter is 800 deaths a day.
The way the models were presented suggested a rough consensus. But recent studies have pointed in different directions. Last week, Imperial College released interim data from its React-1 study showing that there are almost 100,000 new coronavirus cases a day in England, making the R number around 1.6.
Researchers from King’s College London, however, have said that data from their symptom tracker app – that has been monitoring the symptoms and test results of millions of users – points to around 43,500 daily cases currently, which would put the R number at around 1.1.
It is not clear whether KCL was asked by SAGE to contribute to its modelling or whether all the models produced by institutions approached by SAGE were included in the presentation.
Why has the estimate been called a “scenario”?
“These are scenarios – not predictions or forecasts”, reads the small print on the bottom of the death modelling slide, only legible when viewing the presentation online.
The caveat leaves a lot of wiggle room to plug numbers into models and use whatever comes out to buttress the argument being made. Sir Patrick said: “Different groups come up with different answers depending on their models, but what is clear from all of them, in terms of deaths over the winter, [is that] there is the potential for this to be twice as bad or more compared to the first wave.
“The models are clearly showing that this could be the case in these scenarios presented here.”
This last tautology illustrates the problem; models can be made to show anything if the right numbers and assumptions are fed in. What matters is the accuracy of the data and the plausibility of the assumptions. This has not been revealed.
Where does this leave us?
Sir Patrick and Professor Whitty are under pressure to defend their use of the alarming graph. But even if the Cambridge/PHE study is published, the impression that the government is playing fast and loose with the data will be difficult to shift. If the justification for a return to lockdown is based on sound science, there should be nothing to fear from greater transparency.
