The accuracy of COVID-19 death statistics

In summary:

    • Three sets of figures regarding numbers of deaths attributed to COVID-19 are collated which crucially vary in their timeliness. 
    • We now know that COVID-19 was not the cause of death for one in 10 people who had symptoms of the virus mentioned on their death certificate.
    • Conversely, 90% of people whose death certificate mentions COVID-19 were killed by it.

The number of deaths recorded due to COVID-19 in Northern Ireland is a figure that informs much reporting and fuels much debate about the current pandemic.

 Several distinct sets of figures are produced, each with different timeliness, regularity, and levels of scrutiny. They are intended to be used in different ways. In general, the more accurate the information, the longer it takes to be produced. So, while operational data may be less accurate, it can be validated over time to verify its usefulness.

Waiting weeks or months until all deaths are registered and their primary cause have been codified and checked is not an appropriate measure, given the clear urgency of needing to track the spread and impact of the virus. More immediate analysis is required to support those making daily decisions about how to configure the health service, make legislative changes, and inform public health messaging. 

Back in April, FactCheckNI investigated some of the different ways COVID-deaths were being reported.

This explainer article looks at the three main sets of figures, and compares their consistency.

Three sets of figures

Period

DoH: cumulative daily figures of deaths occurring based on positive COVID-19 tests

NISRA: weekly figures based on mentions on registered death certificates

General Registrar Quarterly Report: registered deaths with a primary cause of COVID-19

1 January – 30 June 2020*

551

830

755

Lag time

1 day

6 days

between 11–24 weeks

* Note that the daily figures are for deaths occurring in the period, whereas the weekly and quarterly figures are for deaths registered in the period. For example, figures for week ending 3 July show that 839 deaths had occurred (which mentioned COVID-19 on the death certificate), while only 835 deaths had been registered by that date. This minor discrepancy does not alter the 10% difference between weekly and quarterly figures.

Daily

Northern Ireland’s Department of Health (DoH) publishes a daily figure for deaths reported in the 24 hours up to 10am the previous day. (Responsibility for these figures switched from the Public Health Authority on 19 April.) This is based on an accelerated process of reporting deaths and will include deaths which have occurred during that period, as well as any that occurred earlier but whose reporting was delayed.

Deaths are attributed as “COVID-19 deaths” in the DoH’s figures if the person had a positive test for COVID-19 within the previous 28 days. This measure is now broadly consistent across the UK and allows regional comparisons to be made.

Someone with a positive test could die after being struck by lightning, or die from injuries sustained in a car crash. However, medium- and long-term analysis of COVID-19 deaths suggests that the over-reporting of COVID-19 deaths in daily figures is minimal, given the number of COVID-19 deaths already not included in these daily figures.  

The Death Details pane of the DoH dashboard shows that 551 COVID-19 deaths occurred by 30 June. 

Weekly

The Northern Ireland Statistics and Research Agency (NISRA) publishes weekly figures based on COVID-19 being mentioned on death certificates (with or without a positive test). Unlike the DoH figures, NISRA’s weekly statistics include deaths of people who did not have a positive test at the time of death. Statistics are published at 9.30am every Friday morning for the seven days up to the previous Friday, a reporting lag of six days. 

NISRA’s figures will include people whose certifying doctor notes COVID-19 symptoms, but died of a different primary cause once further analysis is conducted. 

During the first six months of 2020, NISRA’s weekly figures show that there were 830 deaths registered mentioning COVID-19 on the death certificate (835 registered by week ending 3 July). 

Quarterly

The Registrar General Quarterly Report is published by NISRA in line with other UK statistical agencies and categorises deaths by cause. The primary cause of death is precisely coded against the ICD-10 international model. These are the most accurate, but also the slowest to produce, figures for COVID-19 deaths. 

During the first six months of 2020, NISRA’s quarterly figures explain that there were 755 deaths where COVID-19 was the primary cause.

Comparing figures  

The daily figures from DoH do not include deaths of people who had not been tested or had not received a positive test result for COVID-19. NISRA explains that these operational data “are important because they are available earlier, and therefore give a quicker indication of what is happening day by day”. 

The weekly figures from NISRA based on mentions of COVID-19 are therefore higher than the daily DoH figures: 50.6% higher across the first six months of 2020. But it takes nearly a week for them to be compiled. (Comparing the latest figures available, the DoH reported 587 deaths associated with COVID-19 by 9 October, while NISRA figures show 913 COVID-19-related deaths, 55.5% higher than DoH.) 

The quarterly figures are the most accurate, but the slowest to produce. For the first half of 2020, the cumulative weekly figures are 10% higher than the quarterly statistics. However, the second quarter analysis (ending 30 June) was only published on 17 September.

In summary:

    • COVID-19 was not the cause of death for one in 10 people who had symptoms of the virus mentioned on their death certificate.
    • Conversely, 90% of people whose death certificate mentions COVID-19 were killed by it.

Other statistics

During the pandemic, FactCheckNI has also looked into excess deaths and COVID-19-related health inequalities.

One thought on “The accuracy of COVID-19 death statistics

Comments are closed.