Improving trust and safety policy-enforcement accuracy with chain-of-thought reasoning data

The challenge

A global video-sharing platform with billions of monthly users relied on an automated model to detect policy violations in user-generated video. Although the model could flag explicit violations and was reliable on overt content, it fell short wherever enforcement depends on judgment rather than keyword or symbol matching. It missed:

• Nuanced violation signals: The model under-detected violations carried by framing, timing or audience cues rather than an explicit term.
• Evasive formats: Weaponized memes, coded language and fair-use exploitation were built to read as harmless to a system scanning for known signals.
• Reasoning over missing context: Many verdicts hinge on what a video does not show or say. Historical footage with no critical framing reads very differently from the same footage presented as documentary.

The model also defaulted to generic violation labels and could not consistently weigh policy exceptions. The platform needed expert analysts to apply structured chain-of-thought (COT), identify the performance gaps and produce training data that strengthens policy-aligned reasoning in model behavior.

The TELUS Digital solution

We designed a multi-level framework that included a three-tier reasoning approach (literal, contextual, inferential) with standardized sections for evaluating video against platform policy. We then trained analysts to apply it consistently. This framework reduced rater variance and made every verdict parseable by both policy reviewers and the model being trained. When reviewing each content piece, the expert annotator documents the full content package, maps evidence to the most specific policy, explains how framing and intent create the violation and states why no exception applies.

Each review moves through five sections:

• Content description: A chronological narrative of what a viewer sees and hears, including explicit notes where critical context is absent.
• Metadata and context: The signals that frame intent such as title, upload date, video length, channel name, description and pinned comments.
• Policy analysis and reasoning: The analyst names the most specific violation, quotes the direct evidence, explains how the content is framed and connects that framing to the policy it breaches.
• Contextual factors and exceptions: The analyst identifies any exception that could apply, such as educational, documentary, scientific or artistic use, then states explicitly why it does or does not hold.
• Conclusion: A policy-grounded verdict in one or two sentences that ties the core evidence to the violation.

Analysts evaluate visual, audio and metadata signals together. For borderline cases, they weigh the signal across the content itself, the surrounding context and, where relevant, audience engagement to ensure harmless-looking media paired with hateful text gets flagged.

The results

TELUS Digital turned a detection gap into a measurable quality layer. Across the work, the human-on-the-loop layer — where expert analysts supervised and corrected the model's reasoning rather than just reviewing its outputs — closed the gap between policy text and policy intent. Four consistent improvements were delivered:

1. Contextual analysis replaced surface signal detection, so verdicts reflect intent and framing;
2. Every violation maps to the most specific applicable policy category;
3. Methodical, step-by-step reasoning removes ambiguity from borderline calls and
4. Each verdict stands as an evidence-based argument that addresses exceptions head-on.

By producing structured, human-enriched chain-of-thought annotations, our analysts gave the platform high-quality training data that teaches the next generation of models to weigh context, intent and policy nuance rather than match terms. The COT annotations made enforcement more defensible: When a creator appeals, each verdict carries a clear, policy-grounded explanation that traces evidence to the specific policy it breaches, which reduces ambiguity. And, because our team consistently annotated gray-area content, the work built a repository of expert decisions on the hardest cases, giving the platform a reference point for clarifying its policy definitions over time. The result is a smarter model, enforcement the platform can stand behind and a feedback loop that keeps improving both.