Founding Abuse Engineer

Own the abuse vision for Arena: what gets detected, what gets enforced, how fast, and with what false-positive budget

Design and operate detection for bots, sybils, coordinated inauthentic voting, and rating-system manipulation — the integrity of Arena's leaderboards is the product

Build enforcement primitives (rate limits, challenges, shadowbans, account actions, model-side refusals) that are reversible, auditable, and humane

Detect and mitigate inference abuse and cost exploitation at the platform layer

Build jailbreak and multi-provider misuse detection across the models Arena serves, and partner with model-provider trust & safety teams on signal-sharing and escalation

Scope and implement abuse monitoring for every new product launch — web search, web fetch, live site deployment, and whatever's next — as part of the launch checklist, not after the fact

Prototype and mature into production systems of detection, review, and enforcement for the highest-severity harms (CSAM/NCII, violent extremism, self-harm), including the legal reporting pipeline (e.g., NCMEC)

Build internal investigator tooling so policy, on-call, and future T&S analysts can triage incidents without engineering bottleneck

Partner with Security on shared surface — account takeover, credential stuffing, API-key abuse, and the identity/behavioral-signal platform

Partner with policy, legal, and leadership on acceptable-use policy, enforcement escalations, and public-integrity narrative

6+ years of production software engineering experience, including building and operating systems under adversarial conditions

Shipped experience in at least one of: trust & safety, anti-abuse, anti-fraud, anti-spam, integrity, or risk engineering

Strong SQL and data-analysis skills — this role is 30%+ pattern-finding and investigation, not just shipping code

Adversarial and investigative mindset — you can articulate a novel attack before designing the defense, and follow evidence when a novel harm surfaces

High judgment on false-positive cost, user harm, and the reversibility of enforcement actions

Proficiency in a modern backend language (Node.js, TypeScript, Python, or Go)

Excellent communication — you'll build alignment with engineering, product, policy, and leadership routinely

Experience with LLM-specific adversarial inputs — jailbreaks, direct and indirect prompt injection, tool-use abuse

Experience with agent safety, browser-automation abuse, or LLM-API abuse

Background in securing voting, rating, reputation, or marketplace platforms against coordinated manipulation

ML or ML-systems experience — feature engineering, online/offline evaluation, label acquisition, drift handling

Experience building investigator or analyst tooling used by non-engineers

Contributions to open-source trust & safety, abuse-detection, or adversarial-ML work

Background in gaming integrity, ad-fraud, or financial-crime engineering at scale