The challenge is urgent, and therefore the Trust & Safety profession must be built up as quickly as possible. One way to accelerate this process is to learn from other disciplines, such as law, which have already had to confront similar challenges. Consider the case of Mark Lin Youcheng, a volunteer at a dog shelter who witnessed a hit-and-run incident involving one of the shelter’s dogs.
- The first decision an organization typically makes is whether their trust and safety function should be centralized or dispersed.
- The Trust and Safety India Festival (TASI) is India’s first national convening focused on trust and safety, led by civil society.
- As a merchant, whether they like it or not, they need to manage payment fraud rate, not just to reduce the financial losses, but to protect brand and reputation, to build trust among their customers.
Platforms must also continually update their moderation policies to reflect company values, social norms, novel abuse tactics, and evolving legal frameworks so that the content hosted on their sites promotes a safe and respectful environment. Successful businesses invest in trust and safety teams to protect their users, their brand, and their bottom line from malicious users. While trust and safety, risk, fraud, and application security have some things in common, there’s a key difference.
While all industries benefit from Trust and Safety, certain sectors rely more heavily on robust safety measures due to the nature of their services and the sensitive information they handle. The next key to creating and filling out a Trust and Safety team is to make a case for its value to senior marketplace personnel. Point out where the marketplace’s current Trust and Safety operations could use improvement, and – when possible – quantify how much current solutions are saving the company in resources (especially money). Building a successful Trust and Safety division at a marketplace starts with establishing why the platform needs it, and then selling that need to other marketplace stakeholders.
A rapid, coordinated response helps mitigate harm and reinforces platform safety. While much of the conversation around T&S focuses on social media, the reality is that any service with a social or interactive element is also responsible for user safety. Online marketplaces have safeguards to prevent scams and counterfeit goods. While there is not a clear agreed upon date and time that the work of Trust & Safety started, many believe a few key teams at Yahoo and eBay were the earliest movers in this space.
These are not the only trust and safety metrics that exist, but these are some of the most common ones businesses measure in order to evaluate their team’s performance. Knowing why is trust and safety important is not only to list the benefits you’ll get by hiring these services but also knowing how to react to the challenges you are going to be facing. Said regulations need to be industry-related, and generally accepted like national or international standards. So, platforms need to comply with these regulations that ensure a safe space for users. It’s important to implement training courses for all your employees, so they can stay up-to-date with the latest fraud detection and scam prevention techniques. Following a continuous training culture will ensure that your employees will not easily fall for fraud and will be able to detect potential risks for your company and customer support operations outsourcing customers.
The Digital Services Act (DSA), introduced by the European Union (EU), was launched to establish a unified set of rules for digital service providers operating in the EUʼs 27 member states. It builds on existing e-commerce rules and adds new responsibilities for platforms with users in the EU, especially for Very Large Online Platforms (VLOPs). Key requirements of the DSA include detecting and swiftly removing illegal content, enhancing user control over content visibility, and providing transparency reports. It also requires VLOPs to conduct mandatory risk assessments to combat disinformation and other harmful content.
This is necessary to ensure quality work while establishing an organization centered around shared interests, open communication, honesty, positivity, good intentions, and strong relationships. Such judgment cannot be provided by technology alone, which relies on rigid metrics to make decisions and lacks moral agency. Important and complex decisions require a human mind that is capable of subtlety, moral agency, and, perhaps most importantly, compassion. Similarly, reviewing cases, making decisions, and handling appeals can draw on lessons from the legal system, while professionals engaged in research could benefit from skills such as futures thinking and horizon scanning.
Scale trust & safety operations at hyper speed by resolving complex violations, adapting to evolving regulations, and delivering measurable business outcomes that drive growth, resilience, and operational excellence. Deliver unique, positive experience for all participants on your platform, including users, creators, advertisers and merchants, by safeguarding brand reputation and enhancing online safety through pre-emptive detection of abuse. Anywhere user-generated content (including comments, reviews and product listings) touches your business processes, is a place that needs the application of trust and safety principles. It must also be a consideration when concepting and designing new processes, capabilities or applications. We’ve created this blog post to give you a crash course in trust and safety and share best practices for building an effective team.
Each industry has its own set of regulatory compliance standards to which they must adhere to ensure a safe customer and employee journey. The first decision an organization typically makes is whether their trust and safety function should be centralized or dispersed. As a social entrepreneur, impact investor, and engineer, her experience spans over two decades in the MENA region, South East Asia, and Japan. Alexander Joseph Woon Wei-Ming is the Provost’s Chair and Lecturer for the Law Programmes at Singapore University of Social Sciences (SUSS) School of Law. He is also a key voice in legal innovation and policy discussions, notably serving as part of the State Counsel team for the Committee of Inquiry into the SingHealth cyberattack. Technology safeguards alone cannot be trusted to provide an adequate level of protection.
Engineers develop machine learning models to automate and scale enforcement against policy violations, creating tools for user reporting, internal reviews, and content moderation. Product managers partner with engineering, policy, and other teams to drive the strategy, vision, and execution for reducing harmful content. Both groups ensure these systems are scalable, efficient, and aligned with safety guidelines, continuously improving them to meet changing needs. Content moderators are on the front lines, reviewing and managing user-generated content to see if it meets community guidelines. They handle high volumes of UGC that require quick, accurate decisions and work to filter out harmful material like extremism, CSAM, hate speech, and misinformation. Involving frequent exposure to distressing content, this stressful line of work often causes emotional trauma, which leads to a high turnover rate.
This often involved using a mix of messaging tools, spreadsheets, backend management systems, and modified case management software – an approach that quickly becomes complicated and inefficient. Dedicated tools now provide a comprehensive solution, offering seamless integration with multiple data sources and APIs to deliver a holistic view of potential risks. Features like queue management help Trust and Safety teams prioritize content based on risk level and type of abuse, ensuring efficient use of resources. Additionally, automated actions allow platforms to take immediate action based on predefined rules, minimizing the burden on human moderators. Given the complexities of building such tools in-house, many platforms turn to specialized Trust & Safety vendors. TSPA partners with those trust and safety teams to equip them with the expertise and skills they need to successfully navigate this incredibly high-stakes environment.
A general sentiment is that the merchant shouldn’t have allowed the transaction, or, some might even blame the merchant for the fact the card is stolen. Or, more interestingly, people will blame the merchant more than the fraudsters themselves – after all, fraudsters are virtual, merchant’s name is on the statement. Only when people feel confident and comfortable about the safety of their presence and activities, in other words, there is no negative implication or loss to themselves, then they trust the platform and other people on the platform. Trust and Safety is a term commonly used on platforms where people interact. It is the foundation to enable unfamiliar or total strangers to treat each other peacefully and fairly. When such platforms involve transactions or trading, they are called marketplaces, such as eBay; but they don’t have to involve money, for example, a dating service needs Trust and Safety as much as a marketplace.
It targets 13 categories of illegal content, including child sexual abuse material (CSAM), drugs, and terrorism-related content. The bill also addresses issues like disinformation and cyberbullying, with significant fines for non-compliance. They’re responsible for developing automated content moderation algorithms as well as embedding mechanisms into the products they develop that meet trust and safety guidelines and protect the users of those products. Trust and Safety (T&S) refers to the framework, processes, and technologies that organizations use to create a secure, reliable, and positive online environment for their users. It encompasses everything from content moderation to fraud prevention, compliance, user privacy, and community protection. Trust and safety encompasses activities designed to secure digital platforms and protect users from harmful content, such as implementing policies that safeguard personal data and manage user interactions on digital platforms.
Staying ahead of emerging threats, understanding new abuse trends, and preemptively countering potential risks before they escalate is essential. For instance, comprehensive systems that combine technology and human intelligence, such as Alice’s Threat Intelligence Solutions, enable platforms to detect and analyze threats from various sources, including the clear, deep, and dark web. These systems support agile threat intelligence operations across multiple languages, helping platforms manage inauthentic activity and moderate harmful content more effectively.
Everyone working in Trust & Safety would also benefit from a solid grounding in ethics, as the field fundamentally revolves around treating people right. On the research and training front, in October 2025, Google announced it would award $5.6 million in funding to 84 researchers as part of a push to advance AI safety. The grants, awarded through the Google Academic Research Awards (GARA) program, support researchers working on AI safety, addressing technical challenges, and ensuring responsible AI development. Explore the real-world insights we’ve gained from helping companies build safer platforms.
Trust And Safety Services
The Federal Trade Commission (FTC) enforces COPPA , and violations can lead to major penalties. COPPA is especially important for platforms and apps in the education and entertainment industries. Each platform faces unique risks based on its industry, platform type, and target audience. For example, social networks, gaming platforms, GenAI chatbots, dating apps, and e-commerce sites all encounter distinct safety concerns.
Another way to minimize moderator harm is reducing workloads and exposure by using automated tools that moderate clearly harmful content without moderator review, also helping improve productivity and efficiency. Moderators’ insights are essential for improving moderation tools and shaping policies, ensuring platforms remain safe and welcoming for all users. The rapid adoption of Generative AI tools has introduced unprecedented risks for the Trust and Safety community. Offenders like child predators, terrorists, and racists now have easier access to create highly realistic, harmful content. Unlike established UGC platforms, new GenAI platforms face a complex threat landscape.
Trust And Safety Solutions That Help Companies Stay Ahead Of Evolving Threats And Regulations
All major platforms like Google, Facebook / Meta, and hundreds of other online platforms now have T&S teams and functions. They each have their own approach, to be sure, so it can be challenging to get started in the industry or even to move laterally for existing professionals. That’s why we at the Trust & Safety Network (TSN) put together this Starter Kit to hopefully help you figure out how to start or grow your T&S career. Your marketing function is, of course, responsible for managing public perception of the brand—and maintaining trust and safety for your platform users is essential for keeping that perception positive. Customer/user experience teams can sometimes view the mechanisms required by trust and safety with suspicion, suspecting them of adding needless friction to customer journeys.
While individuals and teams have diligently worked to protect user safety and ensure trust in digital products and services since the advent of the World Wide Web, the term “trust & safety” (often shortened to T&S) emerged from 1999 to 2010. It’s now commonly used as an umbrella term to describe the teams at internet companies and service providers that work to ensure users are protected from harmful and unwanted experiences. Long before the phrase became popularized, eBay was an early adopter of the term “trust and safety,” where “trust” referred to trust among eBay users and of the company itself, and “safety” referred to keeping platform users safe (Boyd, 2022). Over the short course of its existence, the Trust and Safety industry has made significant strides in ensuring digital safety and protecting users online. Initially, its focus was on basic content moderation, removing illegal or unsafe content from social media platforms.
Challenges In Trust And Safety
Your trust and safety policies may change from country to country, depending on what’s seen as acceptable or appropriate in a given culture. Ultimately, your online platform lives or dies on the trust its users have in it. Trust and safety therefore potentially touches every part of your business.
Trust is likened to perception, but it is firmly rooted in the basis of safety. Inadequate moderation can result in cyberbullying, the illicit sale of weapons, the spread of misinformation, and radicalized extremist behaviors that put both users and non-users at risk. The Introduction to Online Trust and Safety course helps learners understand the essential principles of staying safe online. Through nine short, self-paced videos, participants gain practical knowledge on how to protect personal data, secure accounts and devices, recognize scams and misinformation, and browse safely. The course emphasizes simple, actionable steps that individuals can apply immediately to strengthen their online habits and reduce everyday digital risks. Trust and Safety can’t always catch every instance of abusive behavior on a marketplace by itself.
Our content moderators, all TSPA members, provide moderation services to more than 250 clients across the globe in over 20 languages, making their online platforms trusted and safe for billions of users. A trust and safety team will build and preserve consumer trust in your brand by ensuring that your online platform is a trustworthy and safe place for them to visit and interact with. They do this by drawing up and enforcing policies that regulate the behavior of your platform’s users and by investigating the root causes of major policy violations. So, to face the task of building safety measures companies must establish robust policies that address data protection and user privacy, they must also prevent fraudulent activities.
The company references its quarterly Transparency Reports to emphasize the importance of establishing trust with users and remind them of its promise to keep the Discord Trust and Safety team and policies accountable. The Trust and Safety Curriculum is written by members of the TSPA’s Curriculum Development Working Group, who are trust and safety professionals. For author and additional acknowledgements, visit the acknowledgement page.
