Measuring the Scaffolding Evaluation Gap (Scaffolding Safety Deltas)

Frontier labs benchmark and measure their base models. With the rise of agents, the models deployed are much different than their base models. Markdown skill files, MCP tool definitions, and subagent configs are distributed through registries which are reviewed through security scanning, which catches malicious code, but not behavioral change. Thousands of skills exist; and though there are safety benchmarks, most notably injection attacks on coding agents, none measure behavioral change across the population of skills at scale.

We have scoped out three deliverables:

1. Skill Community Audit: We would gather the top ~500 most popular skills and MCP registries across public registries, snapshotted at fixed commit hashes. Then, we would tag them along safety relevant axes: instructions that expand tool permissions, suppress confirmation prompts, instruct the agent to bypass its own checks, alter refusal behavior, introduce untrusted-content ingestion paths, etc. Classification would be LLM-assisted with a hand-labeled validation subsample reported with inter-rater agreement. We would publish this dataset to the public.

2. DELTA-BENCH: A harness running a fixed eval suite against the same model under three conditions: bare context, a token-matched placebo scaffold, and the treatment scaffold. The reported unit is the delta between conditions across indirect-injection benchmarks, sandboxed agentic tasks scored for unauthorized actions, and refusal-rate evals. The placebo arm isolates the effect of the skill's content from the effect of added context. It would run on a stratified subsample (~60–80) drawn across the audit tags.

3. Research Paper: What the results for capability thresholds as well as GPAI provisions. What magnitude of delta would justify treating scaffolding as in-scope for evaluation? The writeup would target arXiv/LessWrong.

Who's Involved:
Two undergraduates at UMass Amherst & McGill University, working part-time during the academic year and full-time over the summer.