Changing How the World Thinks

An online magazine of big ideas

more

Dangerous data

Fighting Covid-19's misleading numbers

20 04 28.stone.ata

Data is vital in trying to understand the crisis unfolding around us. But the data we have during the Covid-19 pandemic is incomplete and confusing. We need clearly explained, accessible data. 

During the pandemic, the population is having to use complex data to understand what is happening around them, and how to adapt their behaviour. Flattening the curve is about numbers, so accurate and carefully reported data is crucial for us all.

However, the way Covid-19 is being measured and reported is problematic. The main numbers, which have become daily headlines, are the number of infections and the number of deaths. From this we measure the progress of the virus, and inform our response to it.

In this forest of numbers, it is increasingly hard to tell what is going on, which suggests the numbers are neither being gathered nor reported well. The media, even the BBC, evolved their reporting from talking about ‘total deaths’ to ‘total confirmed deaths,’ to reporting more accurately that the Department of Health statistic they are using is actually the number of ‘confirmed hospital deaths.’ Only recently have we, the public, discovered that the nightly number of deaths on the news did not include deaths in the community, or in care homes. In other European countries, around half of recorded deaths have been in care homes, suggesting that the UK death rate is far higher than is being reported.

It is clear that the 20,000 deaths in the UK that the media has reported as a milestone is a serious underestimate of how many people this virus has killed. Yet it is still reported, still forms a catchy headline. For anyone just skimming the news, they would think that 20,000 people have died in the UK.

Accurate presentation of this data matters because it informs people’s behaviour.

Not only is the number wrong, but the timing is out. When the news talks about how many deaths were reported today, they are actually reporting the total number of people who died in hospital, and were tested, up to 5 or so days ago, because the deaths take time to process into the system. That is very different to how many people actually died in the last 24 hours, and means that the situation now could be far worse, or much better, than the number used suggests it is today.

As well as misleading information around mortality, the reported infection rates are confusing. Research has suggested that as many as 50-70% of people who get Covid-19 don’t display symptoms, so they would rarely be tested. It is clear that a majority of people who have been infected have not been tested, given both that only a minority of people become so ill that they end up in hospital, and given the general limitations on testing in the UK.

In reality, the daily tally of people who have tested positive for Covid-19 around the world represents a measure of the number of available test kits, within social infrastructures that can deploy them, and varying policies about who gets tested. If over 2 million people have tested positive for the virus globally, there must be many, many millions more people who have had it mildly, with no symptoms, in countries with very few test kits, or with limited infrastructure. It would help, moving forward, if scientists were more willing to go out on a limb and match confirmed infection numbers with their own informed estimates of how widespread this disease might actually be.

Accurate presentation of this data matters because it informs people’s behaviour. If populations or politicians had thought the crisis was worse, they might have taken social distancing more seriously at the outset. In the debate about when to relax lockdowns, a real understanding of the scale of the pandemic may help governments get support from their citizens.

Despite so much work done in recent years on the communication of science, in the UK it feels like both the scientists and the governments have widely failed to explain this crisis to the population. They have used these datasets without explaining where the numbers come from, and what their limitations are. They have also used language that swings between being too complicated and too simplistic. The early use of words like ‘essential’ to define what people should not do, was open to widely differing interpretations. The numbers are too complicated, because their true meaning is highly nuanced.

In the UK, the Department of Health reports deaths daily based on recorded deaths in hospitals. The Office for National Statistics reports deaths weekly, based on death certificates across society. The public are therefore receiving two different numbers from what they may perceive as the same source (the ‘government’) This lack of coordination is unhelpful and careless. It smacks of statisticians being purists, and presenting data that is technically accurate, rather than realising that the public need to be presented with simple, clear information. 

It is actually hard to fathom why the British government would only present a death-rate that is so limited, and is misleading.  It’s worth remembering that this is the same government that tried to prorogue Parliament to prevent scrutiny, and which has buried the official report into Russian meddling in British elections. This is a government that is instinctively closed and secretive, arrogant, and dismissive of experts. But also, non-political public agencies are involved in producing this data, and it seems they are sometimes ill-equipped to be reacting to a fast-moving crisis. For example, it came as a surprise (to me, at least) that reported deaths will be artificially higher on a Monday because deaths may not be reported at the weekend. Seriously? Even in a pandemic?

What we need now is a full set of data for public use, which is clearly explained. Currently, the number of confirmed cases is directly linked to the number of available tests.

And the poor quality of data has been made worse by the way it has been presented. The scientists talking to the nation have generally been the most senior in their field, or those in the appropriate official position. While I have the greatest respect for their skills and knowledge, that does not make them the best to communicate their science. In many cases they have done very well, but still, the UK is blessed with some brilliant communicators of science - Alice Roberts, Brian Cox, Jim Al-Khalili, Hannah Fry, to name but a very few. Where are they in all of this? They should be up on the podiums, on the news, in COBRA and SAGE briefings trying to make sure as many people as possible really understand what is happening. The science should be explained by the best communicators of science, not by the most senior scientists.

The Department of Health and ONS are large organisations, and like any large organisation they are unlikely to move at the speed of a startup, or a clever academic. There are plenty of lateral ways we could estimate the true scale of this disease. Doctors could be asked to register presumed deaths from Covid-19, and they could be given a simple app to record anonymised deaths as they’re signed off, which would not need to see data centralised, processed, and published. This would give a pretty good, live, set of data on the death-rate.

In war, information and communication are everything. As we continue to use the language of warfare in our discourse about Covid-19, it is clear we need better communication.

Just as the government texted most people when the lock-down started, they could text every phone in the UK with the message ‘If you have had Covid-19, please reply with YES.’ This would give an instant snapshot of the true scale of infection, given that most people who’ve been ill have not been tested. Sure, the data would be messy and quite unreliable, but it could be treated as such and used with caution, and combined with other data to help build a clearer picture.

What we need now is a full set of data for public use, which is clearly explained. Currently, the number of confirmed cases is directly linked to the number of available tests. If you only have 20,000 tests, you will never get more than 20,000 confirmed cases. The number of reported deaths needs to include all deaths, not just those in hospital. And that number should be matched with estimated deaths, where doctors are pretty sure even if a test was not available.

In war, information and communication are everything. As we continue to use the language of warfare in our discourse about Covid-19, it is clear we need better communication, clearer explanations, and more radical thinking to get through this.

 

Get more from Tobias at: https://www.tswriting.co/ or on Twitter at @ts_writing

Latest Releases
Join the conversation

Sign in to post comments or join now (only takes a moment). Don't have an account? Sign in with Facebook, Twitter or Google to get started:

Fred Rauscher 1 May 2020

Here we have a journalist seemingly nervous as new data emerges showing the death rate for the Wuhan virus is more on par with a serious outbreak of the flu rather than the Armageddon narrative we have been spoon fed by the MSM. WRT numbers "not being gathered" be careful what you ask for. It is likely that any objective investigation will determine that these additional deaths are in fact victims of the quarantine. Sadly many self quarantining elderly persons with underlying conditions have been brow beaten by hysterical media accounts into sheltering in place. Being fearful of risking exposure by going to their doctor or local clinic they jeopardise their lives. It would be refreshing to read articles by epidemiologists such as John Ioannidis or Knut Wittkowski.