Data for Search & Ads

Frequently asked questions

• Why is human evaluation still needed for AI search?

  Automated metrics can measure speed and format, but they cannot reliably assess factual grounding, reasoning quality, cultural appropriateness or nuanced policy compliance — all of which require human judgment. Human evaluation remains the gold standard for identifying where AI-generated outputs fail users and for generating the feedback data needed to correct those failures at scale.
• How do you measure the quality of training data for search relevance and LLM evaluation programs?

  Quality in training data is measured across several dimensions: label accuracy (do raters apply guidelines correctly?), inter-rater agreement (do independent raters reach the same conclusion?), coverage (does the dataset represent the full distribution of real-world queries?) and temporal stability (do quality signals hold up as content and user behavior evolve?). Programs that lack robust measurement across all four dimensions typically produce training data that improves model benchmarks without translating to real-world search quality gains.
• How do you maintain rating consistency at scale?

  Inconsistent rater judgments are a primary driver of model drift, and addressing it requires more than basic training. TELUS Digital tackles this through rigorous qualification, continuous performance monitoring, AI-augmented calibration and structured inter-rater agreement frameworks, ensuring relevance signals remain stable and comparable across thousands of raters, time zones and languages to produce the reliable human judgments search algorithms depend on.
• How is search evaluation changing with AI-generated answers?

  As search shifts from returning links to generating answers, evaluation requirements move from assessing relevance and ranking to judging accuracy, reasoning quality and factual grounding. TELUS Digital's domain experts use AI-native tooling to assess multiturn dialogue, reasoning quality, factual grounding and response coherence, providing the structured human signal that AI search platforms need to continuously improve answer quality, reduce hallucinations and maintain user trust at scale.