For your consideration: FactCheckNI verdict ratings

What does it mean for something to be “true”, “correct”, or “accurate”? Can anything be absolutely true or completely false? Is there any difference between being correct and being accurate?

Most fact-checking projects employ a verdict rating system. The initial policy at FactCheckNI was to avoid using an explicit rating system. Instead, we suggested that the reader draw his or her own conclusions from the concise summary provided in each article that fact-checked a claim.

We based this policy on our motivation to inform and educate, rather than to determine a judgement, or intention, on the “truthfulness” of the claim.

Researchers recently asked us to categorise the results of our claim investigations. We paused to assess how we had utilised a range of words prominently in our fact-check conclusions and discussed whether we could tighten up our usage to increase comprehension and reduce confusion.

Many of our article conclusions contained the term “accurate” or “inaccurate”. What do those terms mean?

In data sciences, there is an established procedure to determine a margin or error (e.g. +/- x%); if something falls outside this margin, then you can question its accuracy. If there are five people seated in a room, then – in terms of accuracy of numeric estimation – the answers “4”, “5”, or “6” are all accurate. There is no need to determine a status of “less than accurate” or “more than inaccurate”.

In the case of qualitative claims, how close is the claim to the evidence? For example, you can check the audio/video recording or official transcript against what was reported to be said. Perhaps it corroborates (“accurate”). Perhaps it almost corroborates; in such cases, we will say “accurate with considerations”, pointing out the inaccuracies but not deeming the whole matter inaccurate. This can work the other way round, too — “inaccurate with considerations” — when there’s an element of the claim that we wish to underline.

“With considerations” is still a qualifier, such as “mostly”/”almost”/”partly”/”half”. But it is not a statement on the degree of “true”/”truth”/”correct”/”right”-ness of the claim. Just that there are some aspects that we wish to draw to your attention for consideration.

And then there are those claims where we don’t come to any conclusion on accuracy or inaccuracy. This may be because of insufficient information to substantiate or rebuke the claim. Thus, we will apply a conclusion of “unsubstantiated”.

The following verdict ratings will be applied to FactCheckNI fact-check articles:

  1. accurate (evidence corroborates; within margin or error)
  2. accurate with considerations (qualifications)
  3. unsubstantiated (insufficient information)
  4. inaccurate with considerations (qualifications)
  5. inaccurate (evidence does not corroborate; outside margin of error)

This effort is to try to establish a system for our claim verdicts that is as objective as possible.

We still argue that it is up to the reader to determine how much weight to apply to any of our article conclusions. Is a misapplied piece of satire better or worse than a state agency misreporting economic data? We have our views. It’s up to you to defend yours.

Our hope is that other fact-checking projects see the benefit of a more objective-based system of claim verdicts — to take some subjectivity out of scrutinising facts and to spend more time on other crucial aspects of our work, such as developing a greater appreciation of media literacy and critical thinking.

10 years of Nieuwscheckers

10 years of Nieuwscheckers
by Ferre WOUTERS for FactCheckNI
25 January 2019

Nieuwscheckers is a Dutch project for students in journalism at the University of Leiden. Supervised by lecturers Peter Burger and Alexander Pleijter, students monitor the media looking for statements or articles that make their eyebrows frown. Once such a claim is found, they ask the journalist who published the claim to clarify on which it is based. Students then investigate this and publish their conclusion online. Continue reading 10 years of Nieuwscheckers

Semtex and powdered milk: A history of Ireland-Northern Ireland border checks

Semtex and powdered milk: A history of Ireland-Northern Ireland border checks
by Ferre WOUTERS for FactCheckNI
21 January 2019

At an event in London on 15 January 2019, Arlene Foster (Leader, DUP), stated that during the Troubles there has not been a hard border between Ireland and Northern Ireland. When asked what she meant by this, she explained: “The borders of the past were there for a completely different reason. They were there to stop terrorists, they were there to stop the flow of Semtex (explosives), as opposed to the flow of powdered milk.” Continue reading Semtex and powdered milk: A history of Ireland-Northern Ireland border checks

The Santa myth: A joyful act of confirmation bias

The Santa myth: A joyful act of confirmation bias
by Ferre WOUTERS for FactCheckNI
21 December 2018

At FactCheckNI we are interested in facts. Through the training we deliver, we also come face-to-face with many of the beliefs people hold dear. Here we unpack one of the most cherished beliefs, and show how even fact checkers can engage with a little bit of magic … Continue reading The Santa myth: A joyful act of confirmation bias

What is fact checking and why is it important?

What is fact checking and why is it important?
by FactCheckNI
16 November 2018

WHAT IS FACT CHECKING?

The Oxford Dictionary describes fact checking as a process seeking to “investigate (an issue) in order to verify the facts”. However, while instructive, this necessarily concise definition is limited in its understanding of the practical outworkings of what constitutes fact checking, the variation and scope of its practices, as well as the factors and social, political and cultural contexts in which fact checking has become an established practice. Continue reading What is fact checking and why is it important?

No university in Derry-Londonderry?

Labour politician Lord Adonis recently tweeted that he finds it “scandalous that Ulster’s second city is still denied its own university”. He later acknowledged that Derry-Londonderry had its own campus as part of Ulster University, however its headquarters are in Coleraine. The university used to be called the New University of Ulster (NUU), which opened 50 years ago, and could have been based in Derry-Londonderry. Continue reading No university in Derry-Londonderry?