Roblox Rolls Out System to Spot Child-Endangerment Chat Messages

From CNET: Roblox is a popular online gaming platform for children. But it's also a place where people who want to exploit children know they can find an audience. On Thursday, Roblox announced Roblox Sentinel, an artificial intelligence system that's designed to flag inappropriate messages in its chat feature. Roblox already prohibits sharing real-world images and personal information, like phone numbers and usernames. The company hopes Sentinel will flag more messages sooner for investigation. It's been running on the platform since late 2024, but was just announced this week.

A representative for Roblox did not immediately respond to a request for comment.

Roblox created Roblox Sentinel, an AI system to help detect signs of child endangerment. Once Roblox is aware of the problems, representatives can investigate and report to law enforcement.

Sentinel runs an analysis in real time across over six billion chat messages every day. It continuously takes one-minute snapshots, which are automatically analyzed by AI to identify messages that could be harmful to children. These messages are also compiled over time to show patterns that can be further investigated and reported.

View: Full Article