Why Word Filters Aren’t Enough for Game Moderation

The most dangerous conversations in a game can start with friendly words. Detecting grooming and manipulation takes more than a word filter: it takes technology that recognises patterns early and people who can make informed decisions about what happens next.

Most studios approach moderation the same way:

Block the bad words and deal with the harm when someone has reported it, very often after it has happened. A filter, a word list, an auto-flag when someone types something ugly. Job done, community safe. Except the most dangerous conversations in your game contain no bad words at all. Grooming starts as friendly attention. Manipulation starts with isolation, a kid being told nobody else cares about them. By the time the language turns explicit, the harm is already moving off your platform.

And when a real flag finally lands on a moderator’s desk, the decision is difficult. Ban this user. Report this to the police. Or sometimes: this is a child showing signs of self-harm, often the result of bullying, and someone needs to respond like a human being.

Underneath all of this sits an uncomfortable trade-off.

You can tune a detection model to produce fewer false positives. Less noise, less manual review, a cleaner dashboard. But every conversation you filter out of the queue is one nobody will ever look at. And some of them are real. At Aiba, we made that trade-off the customer’s choice. Every studio decides how many false positives they’re willing to review. Every single one has chosen the more sensitive setting. More flags, more manual review, on purpose. They’d rather check a false alarm than miss one real case of harm.

On discovery calls I often hear the opposite instinct: “the AI should just take care of it, it should be automated.” I understand where it comes from. But the flag is not the decision. We believe in a human on top of the system, not just in the loop. Our customers get it. Their community is the heart of their game, and they take care of their players accordingly.

Grooming is one of the hardest problems on the internet. It hides in plain text, and it’s easy to miss. It’s happening right now, and it can be solved. Not with a word list. Not with full automation. With technology that finds the pattern early, and people who decide what happens next. Bad actors find new ways in every day. So we improve every day.

That’s the job. Aiba is the firewall for humans.