Why trust and safety is risky business

Know the risks you take on while conducting a trust and safety investigation, how adversaries could retaliate and how to counteract these risks.

Protecting online communities and services from harmful content and inappropriate use is essential for maintaining user confidence and brand integrity. That’s the focus of a trust and safety team. In our earlier blog in this series, What is trust and safety, we highlighted the increasing importance of these teams, and key challenges they face in getting the job done.

Here we’ll take a look at the real risks involved when investigators need to collect evidence, and how they can leverage innovative tools for more secure, safe and efficient tradecraft.

Risks to trust and safety analysts and their organizations

Automated content moderation and fraud detection is no longer enough to effectively manage online environments. Trust and safety teams often need to dive deeper into issues, and that’s where the risks start piling up. Analysts don’t know where an investigation may lead. Digging below the surface could require interaction with bad actors and malicious sites, which can introduce significant risk — to the analysts, the organization and the integrity of investigations.

Let’s look at some common scenarios…

• Acceptable use enforcement on social media: A trust and safety team needs to reduce and prevent content abuse and criminal activity, but they also want to avoid incorrectly removing “flagged" users from the platform. It requires investigation to understand, for example, is the content really from a hate group or just someone expressing opinions? To uncover the whole story, analysts might have to research in potentially dangerous forums used by racist groups or terrorist organizations.
• Marketplace surveillance of counterfeit/stolen goods: Beyond verifying legitimate merchants, the trust and safety team for a marketplace platform needs to identify and remove counterfeit and stolen physical or digital goods. Investigations may require research on the dark web or other illegal marketplaces to gather evidence for reporting to law enforcement.
• Compromised accounts and fraud prevention: Credit card fraud and account takeovers are a constant threat. In addition to fraudulent transactions, online communities are plagued with fake accounts used to spread misinformation and legitimize appearances for illegal activity. Analysts may need to go beyond individual accounts and into the broader source of criminal activity.

Dealing with untrusted content and environments is risky. Investigators need to gain a complete picture for analysis, and establish a chain of evidence — but do it safely and securely. If an analyst’s presence or identity is exposed, targets may get “tipped off” and disappear. Or worse, they might retaliate with anything from phishing, malware and DDoS attacks on enterprise networks, to threatening investigators personally.

Learn more in our white paper, Why online investigators need managed attribution >

Overcoming risk in trust and safety investigations

Successful investigations require the ability to eliminate as much risk as possible — and that starts with securely isolating and anonymizing browsing. VPNs, private browsing and parallel networks cannot fully safeguard online researchers.

For the most stringent protection, investigations need cloud isolation and managed attribution, which enable analysts to separate research from the network and customize their online presence for hyper-secure anonymity. Combining these capabilities gives trust and safety teams the power to shield identities, devices and enterprise resources from risk.

Cloud-isolated browsing

Isolation enables investigators to work on the same computer they use every day, yet conduct online research via a secure cloud-based service. Cloud browsing and storage is isolated from the workstation and network, solving common problems such as:

• Alleviate need for difficult IT maintenance: Analysts are often blocked from sites in certain categories, and need to get special access privileges for investigations. Or IT may allow view-only access, making it hard to collect evidence. Cloud isolation eliminates the need for IT to hassle with these exceptions. And it equips analysts to safely research across the surface, deep and dark web, based on their organization’s policies.
• Reduce risk of exposure: Cloud browsing safeguards the analyst’s devices and network from malware, phishing, hacking and other cyberattacks.
• Ensure a complete audit trail: Each cloud session is unique and logged for an accurate audit trail. Trust and safety teams can more easily meet regulations and compliance, and maintain the chain of custody essential for submitting cases to law enforcement.

Anonymity with managed attribution

Investigators are much more at risk than they realize. Browsers and websites track a user’s digital fingerprint in numerous hidden ways. Online presence can be identified through browser and device attributes such as device types, OS, software/plugins installed, time zone and language settings. Browsing behavior also helps identify a user, with fingerprinting based on search terms used, websites visited, time of use, social media connections and account activity.

Even VPNs and private browsing only conceal a fraction of the digital fingerprint. Many details — enough for an adversary to understand your identity and intent — are still conveyed to the sites you visit.

Learn more: What VPNs and Incognito Mode still give away in your online identity >

It’s nearly impossible to browse anonymously (and lack of any identifying details can also arouse suspicion). Instead, researchers need to holistically alter their online identity. That’s why managed attribution is vitally important for trust and safety investigators. It minimizes the risk of being tracked, identified and targeted. And it helps analysts avoid appearing to users as if they’re “snooping,” when maintaining due diligence over online communities and services.

With this powerful capability, trust and safety teams can increase the efficiency and effectiveness of casework, including:

• Customize and cloak appearance: Analysts can create a custom digital identity that appears as if they’re on any chosen device and browser, from a specific region and time zone, and using the local language. For example, the investigator may be in the U.S. on a Mac with Safari browser, but access a Middle Eastern marketplace and appear as if they are local and using a mobile device.
• Eliminate geoblocking and misinformation: Having a digital disguise can make a considerable difference in the content users see. For example, accessing foreign sites using a local IP address and browser, analysts can often avoid propaganda and instead see content relevant to local audiences perceived as ‘trusted’ visitors.

Intensifying security to optimize investigations

The volume and complexity of trust and safety issues is constantly increasing, amplifying the need for more secure and efficient ways to conduct investigations. The key is to minimize risk, while making it easier for analysts to resolve cases faster. That’s where cloud isolation and managed attribution make a measurable difference.

Want to learn more about empowering trust and safety teams? Download our white paper. 

See for yourself how analysts can conduct hyper-secure, anonymous investigations – request a Silo for Research demo.

And keep an eye out for the next blog is this series: Go-to tools for trust and safety analysts