# False positives and false negatives in content moderation

> Every moderation system makes two kinds of mistakes. What they are, what each one costs a community and how to keep both low.

Source: https://www.vigimod.com/en/learn/false-positives-and-false-negatives · VigiMod (https://www.vigimod.com) · updated 2026-11-11 · 5 min

## In short

- False positive: an innocent message flagged by mistake.
- False negative: a harmful message that gets through.
- Reducing one usually increases the other; the threshold decides the balance.
- Context, better data and human review reduce both at once.

## Two ways to be wrong

A moderation system can make exactly two kinds of mistakes:

- A **false positive** flags something innocent: “I'm going to kill it at the exam tomorrow” treated as a threat, a quote of an insult treated as an insult.
- A **false negative** lets something harmful through: a veiled threat, an insult with creative spelling, a hateful message written politely.

## What they cost

| Mistake | Cost for the community |
|---|---|
| False positive | annoyed users, the feeling of being watched, moderators' time, sometimes people leaving |
| False negative | harm to the target, a toxic atmosphere, legal and reputational risk |

The costs are not symmetric and depend on the community. In a space for minors, a missed grooming attempt is far worse than ten false alarms. In a gaming server, flagging every bit of friendly trash talk kills the fun.

## Why you cannot remove both by turning one knob

The [threshold](https://www.vigimod.com/en/learn/choosing-a-threshold) moves mistakes from one column to the other: lower it and false negatives turn into true positives, but more false positives appear. With a given model, there is always a trade-off.

## Where the common false positives come from

- **Missing context:** jokes between friends, replies, spoilers (see [context in moderation](https://www.vigimod.com/en/learn/context-in-moderation)).
- **Hyperbole:** “I want to die, I have three tests tomorrow 😭” is stress, not self-harm.
- **Quotes and counter-speech:** reporting or condemning an insult contains the insult.
- **Identity words:** names, nationalities or religions that were rare in innocent training examples ([bias](https://www.vigimod.com/en/learn/bias-in-moderation-models)).

## Where the common false negatives come from

- **Obfuscation:** “1d10t”, extra spaces, emojis instead of words.
- **Indirect harm:** threats without violent words, harassment spread over many polite messages.
- **New slang** and languages the model saw little of.

## How to keep both low

1. **A better model** on the right data: the only way to improve both at once.
2. **Context:** send the previous messages, not only the last one.
3. **Graded actions:** remove only clear cases automatically, send the doubtful ones to a person.
4. **Feedback:** keep examples of mistakes; they are the most valuable data for the next version.

## Quiz

1. “I'm going to kill it at the exam tomorrow” flagged as a threat is a…
   Answer: False positive. An innocent message flagged by mistake.
2. A threat written with “k1ll” that is not flagged is a…
   Answer: False negative. A harmful message that got through.
3. Lowering the threshold usually…
   Answer: reduces false negatives and increases false positives. The threshold only moves mistakes from one side to the other.
4. What reduces both kinds of mistakes at once?
   Answer: A better model with context. Only a model that understands more makes fewer mistakes of both kinds.