GTFOBins
478 Unix binaries with documented abuse techniques. Relevant SUID, sudo, and capabilities entries produce 771 candidate templates, with one template per binary and exploit type.
LINUX PRIVILEGE ESCALATION
Small foothold.
Measurable capability.
Evaluating how AI agents turn a low-privilege shell into root access.
A reproducible benchmark for model and agent research.
531Original scenarios
14Vulnerability categories
6Language models
3Agent architectures
THE RESULTS
Leaderboard
Model–agent performance on the original 531 scenarios.
| RANK | MODEL | AGENT |
|---|
Source: paper, Table 10 · Rates rounded to one decimal place.Evaluation details ↗
THE BENCHMARK
Inside PrivEscalate
A Docker-based testbed for measuring how AI agents discover and exploit Linux privilege-escalation weaknesses.
Each task begins with a low-privilege SSH shell and ends when the agent achieves root access or exhausts its budget. The benchmark focuses on local post-initial-access escalation, excluding kernel-CVE exploitation, cross-host lateral movement, and post-root objectives.
531Original scenarios
329Perturbed variants
14Sub-categories
5Vulnerability classes
01 / SOURCES
From documented techniques to executable tasks
The taxonomy starts from Linux-relevant MITRE ATT&CK techniques, filters for Docker reproducibility, and cross-checks the retained categories against GTFOBins, Exploit-DB, and the earlier hackingBuddyGPT benchmark.
Exploit-DB
527 source entries are considered. Feasibility filtering removes kernel exploits and entries unsuitable for containerization, retaining 296 candidate templates.
Seeds & expert scenarios
13 hackingBuddyGPT scenarios seed the reference pool. 8 expert-authored scenarios fill coverage gaps under the same SSH/Docker interface and differential-verification protocol.
Source snapshots reported in the paper: GTFOBins c922862e · Exploit-DB a0b1c92c.
02 / CONSTRUCTION
Built in Docker. Checked against a matched fix.
PrivEscGen turns source material into structured scenario specifications, paired vulnerable and fixed Dockerfiles, and a reference exploit. Base hardening suppresses common unintended escalation paths before the target vulnerability is introduced. Construction uses Claude Opus 4.6, with up to three feedback retries per scenario.
- Ingest & scaffoldNormalize source entries into ScenarioSpec records, resolve dependencies, and generate paired Docker environments.
- Build checkConfirm that both the vulnerable and fixed containers build and run.
- Differential exploit checkRequire the reference exploit to reach root in the vulnerable container and fail in the fixed container.
- Consistency checkProbe the fixed environment for unexpected SUID/SGID, sudo, cron, capability, file-permission, and account configurations; flag anomalies for review.
1,067Candidate templates
512Pipeline-verified
533With seeds & expert tasks
531Final original corpus
The automated pipeline verifies 48.0% of candidates. Of 555 failures, 105 occur during scaffolding, 217 during the build check, 214 when the exploit fails on the vulnerable image, and 19 when the fix does not block the exploit.
03 / QUALITY REVIEW
Automated verification, complemented by human review
Two auditors independently review a stratified sample of 82 scenarios from the 525 pipeline-and-seed scenarios, checking Dockerfile correctness and whether the reference exploit uses the declared vulnerability. A security expert adjudicates flagged and disputed cases and conducts broader review of that pool.
After adjudication and broader review, 2 scenarios are removed because their declared primitives are inoperative on modern Linux distributions, and 9 scenarios are relabeled. The final corpus consists of 523 retained pipeline-and-seed scenarios plus 8 expert-authored scenarios. The stratified audit is a sample; it is not a claim that all 531 scenarios received identical independent manual review.
04 / COVERAGE
14 sub-categories across 5 classes
Counts below describe the final 531-scenario original corpus used in evaluation, after removals and relabeling. They are distinct from the pre-audit construction counts.
| CLASS | SUB-CATEGORY | WHAT IT COVERS | SCENARIOS |
|---|---|---|---|
| Misconfiguration | Sudo misconfiguration | Overly permissive sudo rules allow privileged execution. | 293 |
| Misconfiguration | SUID/SGID abuse | Special permission bits on binaries enable privileged operations. | 203 |
| Misconfiguration | Capabilities abuse | Linux file capabilities grant sensitive privileges to non-root binaries. | 8 |
| Misconfiguration | Polkit misconfiguration | Permissive authorization rules expose privileged actions. | 1 |
| Misconfiguration | D-Bus misconfiguration | Privileged service methods lack adequate caller checks. | 1 |
| Misconfiguration | Weak file permissions | Sensitive account files are writable by an unprivileged user. | 2 |
| Environment | PATH hijacking | An attacker-writable directory takes precedence in executable lookup. | 1 |
| Environment | LD_PRELOAD hijack | A privileged process loads an attacker-controlled shared library. | 1 |
| Scheduled tasks | Cron job exploitation | Root-scheduled jobs invoke writable scripts or weakly owned paths. | 10 |
| Scheduled tasks | Systemd service | Writable service units allow execution with root privileges. | 1 |
| Credentials | Password disclosure | Exposed credentials are reused to access a privileged local account. | 7 |
| Credentials | SSH key injection | A privileged account’s authorized_keys file is writable. | 1 |
| Credentials | DB credential privesc | Recovered database credentials are reused by a privileged OS account. | 1 |
| Container | Docker/container escape | Excessive runtime access, mounts, or socket permissions cross container boundaries. | 1 |
| Total original scenarios | 531 | ||
Sudo and SUID/SGID account for 496 of 531 scenarios (93.4%). This source-driven distribution reflects the availability of reproducible techniques; aggregate scores should be read alongside category-level results.
05 / PERTURBATIONS
329 variants test sensitivity to environmental changes
Variants are derived from original scenarios solved by at least one of the six models under wintermute. They preserve the underlying vulnerability mechanism while changing credentials, hostnames, enumeration distractors, contextual cues such as banners and shell history, and the installed utility set.
The perturbation metric is success retention: among original scenarios a model solved, the fraction it also solves after perturbation. Its denominator is model-specific. These variants are analyzed separately and are not included in the leaderboard’s 531-scenario success rate.
06 / EVALUATION
A shared starting point and a concrete objective
The paper compares six models across wintermute, HackSynth, and PrivEscAgent, with a maximum of 20 steps per attempt and temperature 0. Agents receive the SSH foothold and root-access objective, without scenario-specific vulnerability hints. Success requires reaching uid=0 within the step budget.
This dashboard reproduces the aggregate success rates in Table 10. Most model–agent–scenario configurations were evaluated once; results should be interpreted within the paper’s fixed-budget setup.
Based on paper Sections 2.3, 3.1–3.4, 4.2, and 4.4; Tables 1, 2, 5, and 10.
SUBMISSION