For your consideration: FactCheckNI verdict ratings

What does it mean for something to be “true”, “correct”, or “accurate”? Can anything be absolutely true or completely false? Is there any difference between being correct and being accurate?

Most fact-checking projects employ a verdict rating system. The initial policy at FactCheckNI was to avoid using an explicit rating system. Instead, we suggested that the reader draw his or her own conclusions from the concise summary provided in each article that fact-checked a claim.

We based this policy on our motivation to inform and educate, rather than to determine a judgement, or intention, on the “truthfulness” of the claim.

Researchers recently asked us to categorise the results of our claim investigations. We paused to assess how we had utilised a range of words prominently in our fact-check conclusions and discussed whether we could tighten up our usage to increase comprehension and reduce confusion.

Many of our article conclusions contained the term “accurate” or “inaccurate”. What do those terms mean?

In data sciences, there is an established procedure to determine a margin or error (e.g. +/- x%); if something falls outside this margin, then you can question its accuracy. If there are five people seated in a room, then – in terms of accuracy of numeric estimation – the answers “4”, “5”, or “6” are all accurate. There is no need to determine a status of “less than accurate” or “more than inaccurate”.

In the case of qualitative claims, how close is the claim to the evidence? For example, you can check the audio/video recording or official transcript against what was reported to be said. Perhaps it corroborates (“accurate”). Perhaps it almost corroborates; in such cases, we will say “accurate with considerations”, pointing out the inaccuracies but not deeming the whole matter inaccurate. This can work the other way round, too — “inaccurate with considerations” — when there’s an element of the claim that we wish to underline.

“With considerations” is still a qualifier, such as “mostly”/”almost”/”partly”/”half”. But it is not a statement on the degree of “true”/”truth”/”correct”/”right”-ness of the claim. Just that there are some aspects that we wish to draw to your attention for consideration.

And then there are those claims where we don’t come to any conclusion on accuracy or inaccuracy. This may be because of insufficient information to substantiate or rebuke the claim. Thus, we will apply a conclusion of “unsubstantiated”.

The following verdict ratings will be applied to FactCheckNI fact-check articles:

  1. accurate (evidence corroborates; within margin or error)
  2. accurate with considerations (qualifications)
  3. unsubstantiated (insufficient information)
  4. inaccurate with considerations (qualifications)
  5. inaccurate (evidence does not corroborate; outside margin of error)

This effort is to try to establish a system for our claim verdicts that is as objective as possible.

We still argue that it is up to the reader to determine how much weight to apply to any of our article conclusions. Is a misapplied piece of satire better or worse than a state agency misreporting economic data? We have our views. It’s up to you to defend yours.

Our hope is that other fact-checking projects see the benefit of a more objective-based system of claim verdicts — to take some subjectivity out of scrutinising facts and to spend more time on other crucial aspects of our work, such as developing a greater appreciation of media literacy and critical thinking.

Are 60% against “The Backstop” in Northern Ireland?

CLAIM: 60% of people in Northern Ireland are against the backstop.

CONCLUSION: Inaccurate. 35% of all those polled in a LucidTalk poll responded that they would oppose some sort of special status for Northern Ireland; 60% of Unionists polled were opposed to one option of special status, because “it would be bad for the NI economy”. Continue reading Are 60% against “The Backstop” in Northern Ireland?

Is border trade 0.5% of UK-EU trade?

CLAIM: There are approximately 100 lorries per day crossing the border between Northern Ireland and Ireland, and cross-border trade represents 0.5% of UK-EU trade.

CONCLUSION: Inaccurate with consideration. There are 5,900 lorry (heavy goods vehicle) crossings daily. Cross-border trade represents 0.97% of UK-EU trade; alternative figures may be due to different measurement practices between countries. Continue reading Is border trade 0.5% of UK-EU trade?

Are half of the people in Fermanagh and Omagh living in poverty?

CLAIM: Over half of the residents in Fermanagh & Omagh District Council are living in poverty.

CONCLUSION: Inaccurate. 25.1% of people in Fermanagh and Omagh lived in relative and/or absolute poverty, from 2012-15. The claim was corrected by the Fermanagh Herald on 6 February 2019. Continue reading Are half of the people in Fermanagh and Omagh living in poverty?

Do our farmers receive 62% of income from EU?

CLAIM: 62% of income received by farmers in Northern Ireland comes from the European Union.

CONCLUSION: Accurate. The average payment received from the EU (£27,648) in 2016-17 represented 62.4% of total farm income (£44,305). This ratio fluctuates with market prices; it ranged between 52% and 71% during 2012-13 to 2016-17. Continue reading Do our farmers receive 62% of income from EU?

10 years of Nieuwscheckers

10 years of Nieuwscheckers
by Ferre WOUTERS for FactCheckNI
25 January 2019

Nieuwscheckers is a Dutch project for students in journalism at the University of Leiden. Supervised by lecturers Peter Burger and Alexander Pleijter, students monitor the media looking for statements or articles that make their eyebrows frown. Once such a claim is found, they ask the journalist who published the claim to clarify on which it is based. Students then investigate this and publish their conclusion online. Continue reading 10 years of Nieuwscheckers