How to choose a moderation threshold

What a threshold is

A classifier gives a score from 0 to 1 (see what is a text classifier). The threshold is the line you draw: score ≥ threshold → flagged. With a threshold of 0.8, a message scored 0.83 for “insult” is flagged, one scored 0.79 is not.

The trade-off

Moving the threshold moves you along a curve:

Threshold	What happens
Low (e.g. 0.3)	catches almost everything (high recall), but many false alarms (low precision)
Middle (e.g. 0.6)	a balance
High (e.g. 0.9)	flags only clear cases (high precision), lets subtle ones through (low recall)

There is no threshold that is right for everybody. Precision and recall explain the two sides.

Why the default is only a start

API providers tune default thresholds on validation data, usually at the point with the best F1. That data mixes many communities. Yours has its own slang, its own jokes and its own tolerance: a competitive gaming server and a support forum for teenagers should not use the same numbers.

A simple method to tune them

1. Collect 300–500 real messages from your community (with the consent your rules require), including some you would remove.
2. Label them yourself, by category: is it an insult? A threat?
3. Score them with the API and save the scores.
4. For each category, try a few thresholds (0.5, 0.6, 0.7, 0.8, 0.9) and count false alarms and misses.
5. Pick the threshold whose mistakes you can live with, and write down why.

With VigiMod you can then send your thresholds in every request ("thresholds": {"toxic": 0.8, "insult": 0.7}) without changing anything else. See the documentation.

Use more than one line

You do not have to choose between “block” and “allow”. A common pattern:

Score	Action
≥ 0.95	remove automatically
0.75 – 0.95	hide and send to a moderator
0.5 – 0.75	allow, but log it
< 0.5	allow

This keeps automatic actions for clear cases and puts human judgement where the model is unsure.

Re-check from time to time

Language changes, new slang appears, a model gets updated. Repeat the check every few months, and whenever you switch model version.