Benchmarking Omission Attacks

Omissions are a documented but understudied attack strategy scheming AI's may use. We want to build a benchmark that tests:

1. Untrusted model's ability to carry out successful omissions across a few settings.

2. Trusted model's ability to catch attackers carrying out omissions, both in synthetic and agent-driven trajectories.

3. Various attack strategies and mitigation strategies, like providing the monitor with a tool to write and run tests in the environment.

The benchmark will give us a better understanding of a type of attack a scheming AI could carry out, and document which control interventions are most promising at mitigating risks from this attack type. We will specifically look for how mitigations for omission attacks differs from mitigations for existing attacks.

More information on omissions, a small validation and the benchmark's design can be found here:
https://www.lesswrong.com/posts/5W38vuwbJmTYfY6t3/omission-attacks-project-proposal

Outputs:

• A benchmark that combines tasks from a few different settings (BashArena, LinuxArena and Auditing Sabotage Bench), adds new omissions side-tasks, and create synthetic attack trajectories from benign trajectories.

• Benchmark results. We will run six different models on all the attack tasks, and monitoring for all those attack tasks. We will also monitor the synthetic trajectories we generate. We will then try subsets of attacks and monitoring with different assumptions (e.g. include commission options in the omission prompts, let the monitor write tests at the end of the trajectory) because they could shed light in to what mitigations are most promising.

Currently the project is just me, but I am recruiting a few other researchers to help.