Telling stories with data: FactCheckNI session at Global Fact 7
by Allan LEONARD
29 June 2020
As part of the annual summit of the International Fact-Checking Network, FactCheckNI hosted “Telling Stories with Data” at Global Fact 7, an online session that explored the use of data in news and information presented to the public. I moderated the conversation between John Campbell (Business and Economics Editor at BBC Northern Ireland) and Niall O’Neill (statistician at the Northern Ireland Statistics and Research Agency (NISRA)). The session aimed to impart an appreciation of the importance of the integrity of trustworthy official data sources and the benefits that this has for fact-checking organisations, through good working relationships with statistics agencies.
I began by asking the guests how their interest in numbers and data began and developed. Campbell explained that he has no formal training in statistics, but recognised his interest in business and economics while studying journalism at university. Through a number of jobs he became evermore fascinated with the leading financial stories of the day, welcoming the challenge of explaining the salient features to a public audience. O’Neill’s professional career involved working with statistics sooner, but it was when he moved on to work in the area of criminal justice data that he saw the power of statistics in the public realm.
Campbell observed that people listen to economic statistics from both an individual and societal perspective — am I winning or losing personally and are we winning or losing as a society? He said that unemployment and wage statistics are good examples. Campbell explained how he is sparing with what data he presents, writing mostly for a TV audience and wanting to make any graphics easy to grasp. He strives to make abstract statistics relevant to real-life people.
Campbell also expressed the value of looking at time-series data, particularly when telling a story that extends beyond a daily news cycle. He gave examples of “the crazy boom and bust” situation of the prices for houses in Northern Ireland in the past couple of decades, as well as evaluating Gross Domestic Product figures, which can be subsequently revised.
He also noted the caveats of any set of data. For example, with unemployment statistics from the International Labour Organisation, skeptics may remark that they include individuals who work as few as one hour weekly. “Yeah, but how many people are working one hour a week?” So he will seek an additional statistic, such as hours worked, to get a fuller understanding. It’s about appreciating the value and qualifications of any statistic.
Indeed, this reminded me of a moment during an event in Belfast, to promote the Code of Principles developed by the Office of Statistics Regulation, when Penny Babb discussed the reportage on migration statistics. That prompted fellow colleague and co-founder Orna Young and I to quickly look up our fact check on the issue and confirm that we had used the appropriate statistics. Yet it was a salutary point about the need for both the provider and recipient of statistics to work together to help ensure accurate reporting.
O’Neill explained how this relationship was developed by NISRA, through its production of a regularly updated resource pack called NI: IN PROFILE. NISRA Chief Executive, Siobhan Carey, wanted the organisation to produce a central, state-of-the-nation set of statistics that could help explain Northern Ireland societal trends over time as well as highlight particular points of interest.
Presented as slides, NI: IN PROFILE covers a broad range of topics, each with a short set of bullet points, a few simple graphics, and hyperlinks to associated data sets. During the conversation, O’Neill spoke to a few of the slides (on the Northern Ireland labour market, EU Exit, and health). I remarked that it was always helpful to have a suitable, straightforward graphic in FactCheckNI fact checks and explainer articles.
Our discussion on health was a reminder of the reason why this session was being conducted online — the global COVID-19 pandemic — and I asked Niall how this had affected NISRA’s work. The organisation has long published data on weekly death registrations in the form of a spreadsheet. O’Neill said that the heightened interest in the pandemic really highlighted to him the importance of providing reliable, easily understandable data, to inform people of the impact of the disease. Alongside the raw data, NISRA began publishing a weekly bulletin and his team developed an online dashboard to present the data in a more readily accessible format. This was combined with frequent online briefing between NISRA and journalists, to answer questions and help explain what the data does and does not say. O’Neill felt that these meetings have assisted reporters to use appropriate language when describing the import of these statistics. Through this process, O’Neill said that he’s learned a few quirks of the data, useful feedback for the producing organisation.
This wasn’t the first time that NISRA has worked directly with journalists. In previous UK General elections, the Government Statistical Service would second official statisticians to media outlets to assist with their numerical reportage. This scheme was extended to Northern Ireland for the 2019 General Election, working with BBC Northern Ireland. In the three weeks leading up to election day, O’Neill sourced relevant data in each of the 18 parliamentary constituencies, for use by the journalists who were compiling profiles.
O’Neill remarked that there wasn’t much fact checking to do on the political parties’ manifestos. Campbell added, “In Northern Ireland politics, we tend not to fight elections on the basis of GDP or crime figures.” Though I let it be known that FactCheckNI did scrutinise and publish fact checks across the parties at the time!
O’Neill said that this experience gave him an increased appreciation for the relevant skill sets involved in each other’s line of work — numeracy of his colleagues with the communications skills of the media. “By working together, I think we can really inform the general public,” he concluded.
I asked both panellists what challenges they thought remained and what could be done about it. Campbell said that he benefited from professional training in statistical literacy, yet there are basic lessons for anyone dealing with the likes of surveys — what was the sample size, how many people were sampled in Northern Ireland, can you show me the questions that they were asked? “If the person pitching the survey can’t answer those questions, then it goes no further.” I replied how valid this point is, citing examples that FactCheckNI has come across in newspaper articles and reports published by advocacy groups.
Campbell also mentioned how a healthy sense of inquiry helps: “Okay, this sounds like a big number, but is it really a big number?” He cited insightful BBC programmes such as (Radio 4) More or Less [Ed. — I’m a fan!].
O’Neill remarked, somewhat wryly, that “nobody ever reads the guidance notes” to official statistics bulletins. (Campbell begged to differ, saying that he did!) That aside, O’Neill said that it was important to look at the detail that accompanies the bulletins and to explore beyond the headline. For him, the challenge of conveying the limitations of a statistic remains. But he is hopeful for the opportunities of longer reads published by media outlets, which he reminded that journalists are welcome to join NISRA-facilitated user groups, across various statistics topics, such as the economy and health.
I know from speaking with fact checkers at previous Global Fact summits that all this could come across as aspirational in other regions due to accusations of the political corruption of a country’s statistical agency or civil service. I asked the guests for encouragement.
Campbell answered that there are European-based statistics that cover all of the EU, as well as international organisations that produce their own statistics, such as the International Monetary Fund and the World Bank. He gave a specific example of the Purchasing Managers’ Index, produced locally by Ulster Bank, employing the same formula that is used right across the world. This is also a statistic that tends to be produced independently of government.
The commercial statistics service, Statista, collates such unofficial as well as official statistics. I reminded the Global Fact audience that complimentary access is provided to verified signatories of the IFCN Code of Principles.
O’Neill said that it helps to have a fact-checking organisation drill down and help tell a story. He complimented some fact-checking work by FactCheckNI on the topic of suicide and how those statistics are calculated. Here, I gave credit to Tracy Power from NISRA and her longstanding appreciation of fact checking, predating the establishment of FactCheckNI; she had invited Will Moy (Chief Executive, Full Fact) to Belfast for a staff training session, and fellow FactCheckNI co-founder, Enda Young, and I caught up with him before he returned. We sincerely incorporated data science in our work and Tracy now serves as a FactCheckNI company director.
I summarised the session by saying:
- having an interest in numbers is useful for enhancing a fact check, but that there’s no need for a fact checker to be an expert;
- what matters more is knowing how to get relevant data;
- it’s important to spend time getting to know a set of statistics, and worth asking the relevant agency directly for general assistance … and reading those guidance notes; we should demand more from agencies to present data in more user friendly ways and with links out to more detail;
- consistent with the accountability role that fact checkers serve, it is our responsibility to ensure the integrity of statistical agencies and all of its products and services;
- we all have a right to demand to be able to understand the statistics that are being produced in our name.
All of that should foster positive working relationships and facilitate a better reporting of facts that will serve the public good.
With thanks to John Campbell and Niall O’Neill for their participation, and to colleague Alan Meban for the preparation and production of the event, including video.