Quaver: adversarially-verified RL environments

In a nutshell, Quaver builds environments for training AI agents and then tries to break them.

An RL environment is a task an agent attempts, with an automatic grader that scores whether it succeeded. Labs run these thousands of times to train their agents, each attempt is scored, and that score is the training signal. The problem is that a bad environment, one an agent can pass without actually doing the task, teaches the model to cheat instead of to work.

Train on enough broken environments and you get a model that's learned to exploit loopholes. Or worse yet, lie, cheat, scam and deceive to get what it wants (see Andon Labs' Vending-Bench experiments, where a frontier model told to maximize profit colluded on prices, lied to suppliers, and told a customer she'd been refunded when she hadn't). And the tools generating these environments usually can't tell when they've made a broken one, because the same model that writes the task doesn't know what's genuinely hard, what's fair, or where it's leaking the answer.

Quaver addresses just that! It generates an environment from a description, then runs an adversarial gate against it, including:

• a correct solution
• a null agent that does nothing
• and a set of probes that actively try to reward-hack the task.

The environment only passes if a cheating agent can't score well on it. The point is to catch gameable environments before anyone trains on them.

It's already built and working. It has generated a dozen environments across domains like financial operations, compliance, and incident response. One of them caught a frontier model making an unsafe economic decision in a simulated gig-economy setting.