1. Insights
  2. Trust & Safety
  3. Article
  • Share on Facebook
  • Share via email

Protect the protectors: Wellness strategies for content moderators

Posted November 9, 2022 - Updated January 12, 2024
Illustration of a person speaking to a therapist or counsellor

As digital first responders, content moderators are the protectors of positive user experience on the internet. Despite many individuals stepping into the role because of their desire to help make online spaces safer for the public, the content they may come across can be challenging. Creating safe workplaces is critical to support content moderators’ mental health and overall well-being.

By having a robust wellness strategy in place that proactively addresses and supports content moderators day-in and day-out, companies can help safeguard the health and well-being of digital first responders.

Leveraging technology

These days, many conversations about content moderation center on the use of artificial intelligence (AI) to support human moderators. With the help of AI, a content moderation operation can review far more content and in a shorter time frame — advantages that are critical in the face of the exponentially increasing volume of user generated content (UGC). The downside is that it’s not yet advanced enough to accurately understand all context and tone, which means content moderation still cannot be done without a human-in-the-loop approach.

AI can also be leveraged to identify some of the more violent and graphic content online and remove it more quickly, protecting not only the general public but also saving the moderators from ever having to see it. Using AI to help blur images and pre-tag content so that moderators aren’t surprised by the images they are about to see, can all work hand-in-hand to mitigate exposure to the most toxic content and prevent psychological and emotional distress.

Kavya Pearlman, founder and CEO of XR Safety Initiative (XRSI), has had her finger on the pulse of content moderation since her time working as a third-party security risk advisor for a well-known social media platform in 2016. She says that when it comes to delivering digital content, companies shouldn’t simply think about the general public as the end-user, but also recognize its impact on the content moderators — the people behind the scenes — who take on the responsibility of ensuring that content is safe.

Pearlman explains that “Content moderators see what the (rest of the) internet doesn’t,” adding that this content is “the worst of the internet.” It’s for this very reason that employers must ensure they provide a solid support system and a holistic and proactive approach to wellness for the individuals who do this work, right from the start.

Recruitment

Just like not everyone is suited to be an emergency room physician, not everyone is cut out for a job in content moderation. These are difficult roles to undertake given the inherent risks of exposure to difficult and challenging scenarios.

For this reason, employers should have strategies in place to help them recruit individuals who have specific personality traits for content moderator roles, including resilience, self-regulation and the ability to ask for support. At the outset, this could involve companies being transparent about the role in job postings to enable candidates to properly assess job desirability and conducting psychological profiling that’s specific to content moderation. Moreover, recruiters for these specialized roles should be trained to recognize specific resilient attributes like strong analytical skills and heightened attention to detail for maximum suitability when hiring for a content moderator role.

Additionally, as each individual has different boundaries, interviews should ask specific questions to give candidates an opportunity to share if certain specific types of content could be especially upsetting. This type of information can help hiring managers assign people to the tasks and responsibilities that are most appropriate for them. While these added layers may stretch out the screening and interview process, spending this extra time to ensure an appropriate fit from the start will benefit both your organization and candidates down the line, and mitigate risk for all involved.

Onboarding and training

Part of the process of developing a healthy and resilient content moderation team is making sure employees are properly trained and equipped to do their jobs. Once hired, it’s important to share learning materials and resources in a variety of formats including virtual, in-person, gamified or self-directed so that it’s easily accessible and digestible. Employers should also help their employees set both short-term and long-term goals for success that include career, health and wellness, and learning and development goals to support long-term engagement and retention.

On-the-job

Employers need to take a strategic and holistic approach to employee well-being programs for content moderators that support the entire spectrum of wellness, including physical, social, emotional and financial. This should involve proactive and preventative wellness practices and resources that are available through onsite, virtual and self-directed options. Some examples include developing personal resilience plans, sharing newsletters or webinars about mental health and one-on-one sessions with experts trained to recognize signs of distress.

Physical health is important too, so consider perks like gym memberships, yoga, meditation and other fitness activities designed with health in mind. Establishing an alliance with clinics, doctors, nutritionists, meditation therapists, and of course mental health experts like clinical counselors and psychologists is key.

Companies can also support and improve employee well-being and engagement with regular social activities, team celebrations, community giving days and sports leagues. Even activities that take place outside of work, like book clubs or personal finance workshops, can keep your team feeling healthy and contribute to their overall well-being.

Protect the protectors

Taking proactive and preventative measures to protect the health and well-being of your digital first responders must be a top priority when designing your overall content moderation strategy. Part of this approach includes making wellness a key pillar of your overall company culture.

To convey to your team that well-being and mental health matters and is a true priority in your workplace culture, brands should integrate and promote well-being initiatives throughout the employee journey, starting at the recruitment phase through to onboarding, training and beyond. It’s crucial that your team members know that support and resources are available to them whenever and wherever needed, and that well-being is fully integrated into daily activities and conversations.

Content moderation is a critical component of customer experience in today’s digital world, but it can’t be done accurately and effectively without humans. It’s up to companies and employers to provide the end-to-end health and wellness support that’s needed to protect their teams and keep them engaged.

This article is part of a four-part series on content moderation. Check out our other articles on the evolving nature of digital content, the increasing sophistication of AI and content moderation regulations.


Check out our solutions

Protect the safety and well-being of your user communities to maintain customer trust.

Learn more

Related insights