An open-source library that both detects self-chosen deception in open-weight LLMs and steers the model back toward honesty at inference — the correction half that current honesty tools lack.
An open-source library that both detects self-chosen deception in open-weight LLMs and steers the model back toward honesty at inference — the correction half that current honesty tools lack.
Project Details
Updated 07/14/26 · Edited by orgIn this project, we will develop echo-truth-llm, an open source toolkit to detect and correct deception in open-weight models. The capability, framework, and prototype code have all been developed for our just-finished paper: Caught and Corrected: Detecting and Preventing LLM Deception with a Self-Chosen-Deception Direction with accompanying Substack article: https://jaredglover.substack.com/p/caught-and-corrected-steering-language.
In that paper, we demonstrated activation-level detection of self-chosen deception at within-scenario AUROC 0.84 / 0.97 / 0.92 / 0.92 (Gemma-3-4B/12B/27B, Llama-3.3-70B), rising to 0.90–0.99 on unambiguous cases. The same method, inverted, yields honesty control, lifting honest-response rates from a pressure floor of 0.00–0.07 to 0.77–0.99.
Much of the paper's experimental success is due to careful calibration of activation steering layers and dosing. Calibration is entirely automatic, with the Gemini Flash API used for dataset labeling and low-cost, gradient-free training of activation vectors. Thus, once the first version of echo-truth-llm is released, it will be straightforward to add support for a wide range of other open source models beyond the initial Gemma-3-4B/12B/27B and Llama-3.3-70B.
Funding will deliver a concrete, pip-installable library under a permissive MIT/Apache license, accompanied by a reference evaluation harness and reproducible notebooks. The initiative will be executed by the author of the core method (PhD, EECS, MIT), with the ideal budget adding support for more model families, HuggingFace transformers integration, and LangChain wrappers.
Theory of Impact
Updated 07/17/26 · By grantmaking.aiThe structural advantage of activation-layer steering is its resilience against surface-level vulnerabilities like prompt injection—the top-ranked risk for deployed systems. In empirical testing, our steering vectors successfully overrode explicit, injected instructions to deceive, shifting model honesty rates from a baseline of 0.00–0.07 up to 0.77–0.99. Because this intervention occurs deep within the residual stream, it neutralizes adversarial prompts at an internal layer where textual overrides cannot reach. Furthermore, because our approach bypasses gradient training, it represents an exceptionally lightweight, low-compute solution. Open-sourcing this toolkit enables independent safety labs and agent developers to implement hardened runtime defenses without requiring frontier-scale budgets.
People
Updated 07/17/26 · By grantmaking.aiTeam Member
Discussion
No comments yet. Be the first to share your thoughts.