How is this calculated?
An AI picks out what an article states as fact, then looks for other newsrooms reporting the same thing. The score says how well the article’s facts are backed by independent sources. It is not a verdict on the publisher, and AI can make mistakes, so every source is listed.
1. Picking out the statements
The AI reads the article and lists up to 15 statements, each marked as a fact (something checkable: who, what, when, how many), an assumption (a prediction or inference) or an opinion. Only facts are checked.
2. Checking each fact
For the first 8 facts, we search the web and give the AI only the search results. It answers confirmed, contradicted or could not verify, and must cite the results it relied on. Results from the article’s own publisher and from social media never count. A cited link that wasn’t in the results is thrown away.
3. The score
A fact backed by more independent publishers weighs more: 1 publisher = 0.6, 2 = 0.8, 3 or more = 1.0.
- Accuracy = weight of confirmed facts ÷ weight of confirmed and contradicted facts.
- Coverage = facts confirmed or contradicted ÷ facts checked.
- Score = 10 × accuracy × (0.6 + 0.4 × coverage), rounded to one decimal.
So a fully confirmed article scores 10 only if every checked fact was settled; facts nobody else reported lower the score a little, but never as much as a contradiction.
4. Confidence
- Not enough data (grey bar): fewer than 2 facts confirmed or contradicted, or less than 40% of checked facts settled.
- High: score at least 8.0 and at least 50% of facts settled.
- Medium: score at least 5.0.
- Low: anything below.
Bar colours: green from 9.0 (well supported), orange from 7.0 (partly supported), red below (poorly supported).
5. What the score can’t tell you
- Breaking or local news often has few other reports yet, so many facts stay “could not verify”.
- Several outlets can repeat the same mistake; independent sources make that less likely, not impossible.
- The AI can misread a source. That’s why every verdict links to the sources it used.
- Opinions and assumptions are shown, never scored.
Spotted a mistake? Every report has a Report an error button.
Limits
To keep HeadlineCheck free, we run about 12 checks a day, and each person can start 3. Opening a check someone already ran is always free.