Mass testing: is it worth it?

A recent article in the Observer has addressed the latest controversy regarding Lateral Flow vs Polymerase Chain Reaction testing controversy: LFT vs PCR. 

In short, the testing frenzy that has engulfed the UK has resulted — predictably — in a blizzard of false positives and negatives. What does it all mean, apart from a stupendous amount of money being funnelled away from primary care and in the direction of diagnostic test providers? 

PCR has been treated as the “gold standard” ever since the so-called Corman-Drosten paper that characterised the SARS-CoV-2 PCR assay was peer-reviewed and published in double-quick time in January 2020. PCR — run on a lab-based diagnostic machine — is known to be quite sensitive (does not miss many infections, i.e. has a low false negative rate) but is also claimed to have a relatively high specificity (low false positive rate). Despite numerous challenges to these claims, its use has been the bedrock of the world’s response to the pandemic, perhaps most importantly to confirm the very few (nine) cases of Covid-19 in the active arm of the Pfizer Phase III trials. 

LFTs, on the other hand, are simple to use, and are very specific (unlikely to give a false positive), yet with relatively low sensitivity (so may often incorrectly give the all clear). LFT has had a very bad rep. While two new providers have recently been introduced in the UK, for many months the only supplier has been Innova, despite the US FDA issuing a Class I Recall Notice, the most serious type of recall for such devices.  Users are instructed to “Destroy the tests by placing them in the trash or return the tests to Innova using the FedEx return label that was included with the recall letter that Innova sent to customers”. Yet these LFT devices are still a core part of the UK’s eye-wateringly expensive “Operation Moonshot” screening programme.

The pros and cons of both types are a stark reminder that there is usually no such thing as an accurate test — performance depends on background prevalence and what one is trying to rule out or rule in.  Ideally, a test is both highly sensitive and highly specific.  LFT and PCR have different weaknesses: even if virus is present, (an incorrect) negative LFT will be relatively common.  Similarly, if you are not ill, then a PCR positive test should be treated with caution.  Drosten himself — speaking back in 2014 during the MERS outbreak — was unequivocal: “asymptomatic people should not be tested with PCR”. But that is another story.

Back to the Observer article. Co-authors Professor Sir David Spiegelhalter and Anthony Masters discuss the two diagnostics in a scenario where both testing methods are on firm ground: an individual has a positive LFT followed by a negative PCR.  Which to believe — the quick “at home” LFT or the “gold standard” PCR?  The question was triggered by the suspension of a test centre for issuing numerous negative PCR tests after positive LFTs.  It seems this testing centre was a bad egg, but it has crystallised a question that many have been asking — what is the point of all this screening? The ethics of mass screening decree that great care should be taken: the costs can very quickly outpace the benefits.  Not a trivial matter when waiting lists for life-saving treatments are getting longer.

Using a Bayesian likelihood ratio (LR), Spiegelhalter and Masters calculate that out of a cohort of 100 people who have all tested negative PCR after a positive LFT, around 40 of these 100 — i.e. a minority — are likely to have been incorrectly “exonerated” by the negative PCR, while the negative PCR would have correctly “exonerated” the rest, i.e. a majority of the cohort. This is for an assumed background viral prevalence of 1 in 60 cases in the initial sample of tests — at last week’s reported case rate for the UK, 1 in 224, the numbers shift considerably and the negative PCR would have incorrectly “exonerated” around 15 of every 100 of those that had tested LFT+ and then PCR-.

It is probably also worth mentioning at this stage that the very idea of “harbouring virus” being linked to the legal concept of “guilt” is a path that — given historical precedent — should be trodden with great care.  While the authors correctly explain how their LR calculation might be used in a (hypothetical) legal case to infer “moderate evidence in favour of you having an infection…”, they might also have emphasised that this is a relative measure that a court of law would apply to modify its prior views on “guilt” or “innocence”. The legal system, of course, starts with the presumption of innocence or, in this case, might take into account background prevalence and reports of any symptoms at the time the tests were taken, as well as any uncertainties associated with the quoted test sensitivities and specificities.   

Professor Norman Fenton, Professor of Risk Information Management at Queen Mary (University of London), explains this in an article published in Law, Probability and Risk: “One of the greatest challenges to the use of probabilistic reasoning in the assessment of criminal evidence is the ‘problem of the prior’, i.e. the difficulty in establishing an acceptable prior probability of guilt. Even strong supporters of a Bayesian approach have often preferred to ignore priors and focus on the likelihood ratio (LR) of the evidence. But to calculate if the probability of guilt, given the evidence, reaches the probability required for conviction (the standard of proof), the LR has to be combined with a prior”.

It is therefore curious that the authors’ overall conclusion is that “the negative PCR does not outweigh the positive LFD”.  This may be semantics.  But it would be wrong to interpret their phrase to mean that it is “moderately likely” that a person after LFT+ and then PCR- would — in absolute terms — be more likely than not to be “with virus”.  A criminal court requires proof “beyond reasonable doubt”; a civil court settles matters “on the balance of probability”, i.e. over 50 per cent. Only were background viral prevalence to rise to 1 in 40 (i.e. 2.5 per cent of every person is “with virus”) would one then expect half of our 100-strong cohort to actually be “with virus” following their sequential LFT+ and PCR- test results.

Perhaps the bigger question, therefore, is whether we should be spending all this money on testing.  If the NHS is under such pressure, would it not be better to direct this resource away from mass testing in the community and instead shore up critical care facilities? 

Given the numerous downsides (mostly cost but also, as outlined above, data that are anything but clear-cut), it is not abundantly clear why we do it — even after two tests per head, our 100-strong cohort is ultimately none the wiser regarding whether they are ill or not.  In the olden days (i.e. pre 2020), one would have just stayed at home if one was ill – a far cheaper method of stopping the spread.

Perhaps it is time for a David to slay the Operation Moonshot goliath… or is mass community testing now an untouchable shibboleth? 

Let us sincerely hope not — it is a frivolous luxury we can ill-afford.

Dr Alex Starling (@alexstarling77) is an advisor to and non-executive director of various early-stage technology companies.