Why doesn't your contact center AI know what your best agents know?

Episode topics

• 00:00 — Why doesn't your contact center AI know what your best agents know?
• 02:06 — What is tacit knowledge in the contact center?
• 03:34 — How does agent behavior evolve away from standard training materials?
• 05:50 — What is the difference between quality analysis and outcome-tied annotation?
• 08:05 — How do you build an enterprise data annotation taxonomy?
• 10:38 — How do you feed top agent insights back into AI assist tools?
• 14:06 — How do you protect institutional knowledge when top agents leave?

Transcript

[00:00:01] Robert Zirk: Somewhere in your contact center right now, an agent is handling a call better than your training manual says they should, using a workaround, a phrase, a sales strategy nobody taught them. That knowledge is valuable, and it's not anywhere your systems can find it.

[00:00:19] On an earlier episode, when we asked how brands can design and deliver seamless customer experiences, Jim Mitchell, TELUS Digital's vice president of customer experience and digital innovation, shared an anecdote from a time when he was onboarding a new client and toured the client's in-house contact center. While sitting side by side with an agent during a call, he was surprised when the agent didn't open a knowledge base or a copilot. Instead...

[00:00:47] Jim Mitchell: He pulls his drawer open (sound of drawer opening) and he's got these spreadsheets laid out in his drawer. And on these spreadsheets were the most commonly asked questions.

[00:00:55] So after the call, I said, "Hey," I said, "you opened your drawer during the call. What was that all about?"

[00:01:00] He said, "The knowledge base is just so vast. You get in there, you can't find information. I have to put callers on hold. So we have this, like, under the waterline cheat sheet when new employees come on. Really not supposed to have paper on the floor, but we still hand them out for survival. We have to have them."

[00:01:20] Robert Zirk: The knowledge that would make an agent's job easier — and a customer's call better — is usually already sitting in call transcripts and in the instincts of top agents. It just never reaches the systems built to help everyone else.

[00:01:35] So today on Questions for now, I'm joined by Derek Brameyer, vice president of engineering for CX AI at TELUS Digital, as we ask: Why doesn't your contact center AI know what your best agents know?

[00:01:55] Welcome to Questions for now, a podcast from TELUS Digital where we ask today's big questions in digital customer experience. I'm Robert Zirk.

[00:02:04]

[00:02:07] Robert Zirk: In the contact center, agent-facing AI is supposed to make every agent better at the job. When it doesn't work, it could be because the tacit knowledge top agents build on the floor, things like workarounds and phrasing that lands, never makes it into the knowledge base or AI assist tool in the first place.

[00:02:27] Forrester has called tacit knowledge the defining challenge facing AI-led contact centers. A report from Panopto and YouGov found that 42% of what an employee knows about doing their job well never gets documented anywhere else.

[00:02:43] The best agents know what to say when a call gets complicated, how to deescalate toward a resolution and how to approach an upsell opportunity. In a contact center, that knowledge disappears from the floor the day they leave.

[00:03:02] That's the scenario I put in front of Derek Brameyer, vice president of engineering for CX AI at TELUS Digital. He's spent his career scaling global engineering teams across digital agency consulting and product engineering, and he's currently focused on building the data annotation systems that turn contact center interactions into working feedback loops.

[00:03:24] Derek traces the missed opportunities — lost customer satisfaction and lost revenue — back to one place: how knowledge moves once it's on the floor.

[00:03:34] Derek Brameyer: As knowledge enters the call center and then as it actually gets implemented on call over call, and this is tens of thousands of hours of interactions with customers, it evolves into a behavior set of its own. So the agent group that works through training materials, then goes and deploys it on the floor, those training materials were conceived with a certain set of outcomes in mind that don't always materialize.

[00:03:56] Robert Zirk: Derek says that that's when agents begin to adapt their scripts or what they've learned in training to try and drive better outcomes, whether that's closing more sales or raising CSAT by speaking to a customer's need before they even bring it up.

[00:04:11] This evolution tends to happen organically, so it's difficult to capture in a way that can inform new agents.

[00:04:18] Derek Brameyer: The intelligence that I think everyone is seeking to make their AI tooling better, to make their guidance better, it already exists.

[00:04:24] It's happening on the floor every day. Every call, every interaction, there is knowledge there that can be gleaned.

[00:04:30]

[00:04:32] Robert Zirk: Derek's team has found that what works is already sitting in the data, recorded in the calls of agents who've already figured it out. The challenge is finding that signal across tens of thousands of hours of calls, far more than any person could review manually. That's where data annotation comes in — tagging what's happening on every call so AI can surface the patterns at scale. For example, they found how the best agents frame a sales opportunity.

[00:05:02] Derek Brameyer: We've talked a lot to clients about the best ways to frame specific sales opportunities, whether it's adding a new SKU to an existing product space. And there's prevailing notions that like, "Hey, people care a lot about talking about value and kind of wanna avoid price."

[00:05:18] Well, in a lot of our research, and if you kind of go and annotate thousands of hours of transcripts, the quickest path to a sale turns out to, in a lot of cases, be very price-focused and price-based. It is what resonates directly with a customer. It's directly what they wanna hear.

[00:05:32] We're not seeing that you get to a better outcome by focusing on a value conversation or things like that. So that's kind of a specific component and converting that sale faster also is leading to a better customer satisfaction. You're not leading them through this longer script to drive to the same conclusion, which is a place where you see loss in the pipeline.

[00:05:50] Robert Zirk: That's the kind of insight that a standard quality review usually misses. Most transcript analysis is built to catch what went wrong on a call, not to explain what's driving the wins.

[00:06:01] Derek Brameyer: There is a lot of pressure and kind of opportunity in leveraging AI to do quality analysis, to do evaluations of what's happening on calls. And I think there's actually a much cheaper and also much more effective way of doing this, and it's to actually tie it into the business outcomes that are happening on these calls as well. And this is a very hard space for clients and BPOs to operate because that kind of data is generally very hard to integrate and hard to work with. But if you're using inference and AI to actually glean, "hey, did a sale happen on this call?", for example, that is in some CRM somewhere. That's in a system of record, and looking that up is deterministic, it's cheaper, it's faster.

[00:06:42] I think there's a major missing component here in tying the quality analysis into actual business outcomes and doing that actually in a deterministic way. None of this stuff needs to have tons of inference piled on top of it to actually make it happen. So I think that's a major shift in the approach, and it's something that we've been doing with some of our clients to really positive outcomes.

[00:07:00] Robert Zirk: Derek calls this outcome-tied annotation, and he's careful to draw a line between that and simply letting an AI infer whether a call went well. He observed that the industry's excitement around AI is causing companies to overlook a simpler solution that's already sitting in their own systems.

[00:07:17] Derek Brameyer: I'm very firmly of the opinion that we have not maximized just the amount of deterministic work we can do in the call center. We should certainly not overshadow that by just jumping into what an AI solution can do.

[00:07:29] We're in a space where the cost of AI is gonna continue to see a lot of scrutiny. There's a lot of unknowns around it, and making sure that we anchor our solutions and leverage AI where it certainly makes sense but not just throw it at every single problem.

[00:07:42] So we talk about the outcomes because that's real structured data that exists somewhere. We don't have to apply inference. We don't have to think generatively about that. As long as we build the right connectors and integrations, it's very accessible.

[00:07:55] Robert Zirk: Derek's quick to note that approach still isn't automatic, and it isn't infallible either. So what does it take to build the annotation system that underpins this approach?

[00:08:05] Derek Brameyer: There's a heavy amount of data analysis and AI analysis that we are applying here. So you've got data engineers, AI engineers working right alongside our operator team. We're constructing a taxonomy of calls at the end of the day. Through a call, how are we able to evaluate and structure each turn? What's happening there? Who's speaking? The subject, the topics, things like that.

[00:08:25] Robert Zirk: Applying that taxonomy across tens of thousands of hours of calls isn't something a person could do manually. And that's where AI comes in to handle the volume.

[00:08:35] But nothing ships to production without a human validation layer checking the work first.

[00:08:40] Derek Brameyer: We look at the call level, so we're looking at outcomes of calls, FCR quality, transfers, escalations, what's offered, things like that.

[00:08:48] Then we're looking at conversation segments, so where are we? Are we in the greeting stage? Are we in the solutioning stage? Are we handling an objection? And capturing sentiment also in those stages as well. And then the turn level, so individual segments. What is the agent intending to do? What's our level of quality on empathy? How are we framing a specific offer? Bridging, things like that.

[00:09:07] So there is definitely a level and kind of tiered structure to how we look at annotation. Then AI is doing that labeling at scale, at this point, probably north of 80,000 hours of calls that we've transcribed across our client base.

[00:09:18] But that taxonomy that I was mentioning, it's built with partnership through our operations leaders and, actually, top agents as well. We're able to see, from a performance perspective, our top-tier agents. They are the ones who know what good looks like on the floor. They're the folks where, if we can pull out some of the golden conversations that they're having, it has major positive influence across our entire call center team.

[00:09:41] So those experts are helping to both handle the taxonomy at the start, the inputs there or the kind of structuring of the taxonomy, as well as the validation before we take those golden conversations and place those into systems to scale them up to our entire workforce.

[00:09:56] Robert Zirk: Derek describes quality review as an inspection process, where the call is scored for quality and empathy after it's happened. He says that, in the industry, contact centers often don't capture the parts of the call that are generative — the ones that need to feed back into the overall process.

[00:10:13] Derek Brameyer: The way we look at it, it is really an ecosystem. The call is one component. We apply this to chat interactions as well. Any customer interaction is a component here.

[00:10:20] But, I mentioned previously, the actual data-driven outcomes, the data that exists about customers, about sales success, all of those, like, hard data that exists inside customer ecosystems. And then the other piece that's, I think, very interesting is how this ties into the tooling that's used on the call as well.

[00:10:38] Robert Zirk: Derek has seen a clear change in the guidance TELUS Digital agents have received since these annotation practices were implemented. Copilots and AI assist tools used to lean on best practice guidance that didn't go much further than what was already in the knowledge base. Automating that lookup might have reduced a few seconds of latency, but it didn't go as far as it can today.

[00:11:01] Derek Brameyer: So then we annotated the calls and we tied all of these patterns to the outcomes that are happening on the calls and those statements and positioning that are driving sales and saves, bridging from resolving a billing issue into an offer and framing the offer around, "Hey, this is what a customer actually said that they needed," and the language that an agent would use to actually pitch that effectively, and then looking at where in conversations those save attempts break down, where objection handling and the specific technique actually is correlated with resolution. So we bring that annotation and that kind of taxonomy and the insights that we gain from that tooling, and then we layer that back into the assist tooling.

[00:11:38] So what we saw is all of our lower performing cohorts, now they're running these plays, these playbooks that come from our top performing teams. So there's tremendous, like, very positive directional improvement in FCR, in our sales conversion, in save metrics and retention and things like that.

[00:11:55] And then the decision making around all these calls actually changes as well. So we correlate our copilot logs, like the assist tooling, with our transcripts and outcomes, so we can actually see the suggestions that are coming from our assist tooling that agents adopt, and then also the suggestions that they ignore. So this is kind of this new phase of, "Hey, we've determined what happens from our agents on the call, what works. Now, how do we further tune the guidance that we get from our assist tooling to reinforce that?"

[00:12:22] Robert Zirk: For CX leaders, the challenge in getting buy-in for an approach like this can be not knowing how it performs in a real contact center environment.

[00:12:31] I asked Derek about the advantage of working with a partner with decades of operator experience like TELUS Digital.

[00:12:37] Derek Brameyer: If you look at a kind of pure software vendor, they're seeing your operations team, which is hundreds if not thousands of agents across the globe, seeing it through, like, an API lens, a data lens.

[00:12:48] Our team, we are living this out day to day, so this is the attrition that we see in our call center. This is our exposure to compliance and things like that. And then, ultimately, our change management components about agents who, they're not gonna adopt a tool if it doesn't actually work well for them, right?

[00:13:01] People are very eager to adopt tools that make their lives easier and that make their jobs better, so when we look at how we build our taxonomy I think it's critical to understand that it is designed by the folks who have taken calls. It's built by engineers who sit on the call center floor and work directly with our agent teams.

[00:13:19] That infrastructure gets actually deployed inside our group because the data is there for us to use and it can't leave our walls, so the development teams here that we have working in this forward-deployed motion right alongside our frontline teams, that improvement cycle, I think it cannot really be understated. It's accelerating the outcomes that we're seeing. We're building the solutions directly with the people that are just living it in real time. It's not this situation where we're building a product and throwing it over the wall for folks to use.

[00:13:45] So that advantage of really having the in-house expertise, the top performing agents that then drive these outcomes in the annotation space, that's tremendous for us.

[00:13:55] Robert Zirk: As we wrapped up our conversation, I asked Derek what one question he'd want every CX leader to ask themselves about how their organization currently handles institutional knowledge.

[00:14:06] Derek Brameyer: It ultimately comes down to the performance on the floor. So when your best agent leaves, which no one wants to have happen and we work really hard to drive our attrition numbers to be best in class. But when your best agent leaves, how much of what made them your best agent is still left the next day?

[00:14:23] So I think the organizations that win here, it's not that they have the best model or they're paying for the best tooling. It's making that institutional knowledge, the stuff that sits on the floor, make it legible to your AI solutions before your competitors do.

[00:14:45] Robert Zirk: Thank you so much to Derek Brameyer for joining me and sharing his insights today. And thank you for listening to Questions for now, a TELUS Digital podcast.

[00:14:54] If you want to learn more, you'll find a link to Derek's article on institutional knowledge and contact center AI data strategy in the show notes.

[00:15:02] And if you found this episode insightful, be sure to follow Questions for now wherever you listen to podcasts.

[00:15:09] I'm Robert Zirk, and until next time, that's all... for now.

[00:15:13]