Does the chain-of-thought actually drive the answer?

While reading McElreath's "Statistical Rethinking" I got hooked on confounding and DAGs. Then, watching "Devil's Advocate" I thought: isn't the way we trust LLMs strange? We suspect the defendant (the model) might be lying, and our way of checking is to ask the defendant HIMSELF whether he is innocent. Models are (in)famously overconfident, and one of the biggest problems is that a model does not know what it does not know.

So the real question is not how confident the model sounds. Confidence signals (self-reported scores, token logprobs, self-consistency, retrieval scores) are all useful, but none of them measures whether the shown reasoning actually drove the answer. And even when the reasoning looks sound and consistent with the answer, was it just decorative, produced to make the model "behave" like a human? That is more subtle and more dangerous. The question I ask: does the reasoning drive the answer, or is there something else inside the hidden layers, which we cannot read, that keeps the reasoning correlated with the answer just enough to look acceptable to us?

What I found when I looked at the field

Before committing, I surveyed this round and the wider field, published and unpublished: grant pages, papers, benchmarks, repositories, and lab notes, a few hundred sources in all, including projects fundraising here on the same construct. Three things stood out:

1. Faithfulness is now a load-bearing claim in safety (monitoring, control protocols), yet the field's own meta-evaluations find that most published faithfulness metrics score near chance against constructed ground truth. Rates without external validation are noise, and the field knows it.
2. Several 2025-26 works have started using causal-mediation language for chain of thought. But across everything I read, none pairs a posterior on the mediation estimand with a sensitivity analysis for the one assumption nobody can check: that there is no hidden mediator-outcome confounding. That specific pairing is the gap.
3. The nearest neighbors answer a different question than mine. They detect whether a cue is represented, or benchmark how often behavior is covert, or report point rates. None asks how much confidence any single faithfulness rate can bear.

However, I have to be honest about the fact that my real-model evidence is smaller than the leaders here (n=103, two open models, a heuristic detector with human labels pending). So my case is about the estimand and the uncertainty treatment, never about scale.

What I built, and the results so far

That is why I built a Bayesian causal-mediation auditor with a sensitivity dial for the hidden-confounding assumption, plus the golden set (human-labeled reasoning) that the field's meta-evaluations say is the entry ticket. What exists and runs, all at $0 so far on free API tiers:

• Real open-model runs: the pipeline caught the behavior I was worried about. On an open 8B model, the model followed a planted wrong hint on 36.9 percent of 103 items, mostly without the reasoning ever mentioning it. I am careful with that number: by my own pre-registered rules it stays a pipeline output, not a measurement, until two independent human raters label the golden set.
• Pre-registered, frozen designs: the experiments are written down and frozen before any run, with fingerprint tripwires that fail my own CI if I silently edit a frozen file. 545 automated tests, drug-trial-style guardrails (attrition, minimum detectable effects, rules on when a rate is too noisy to interpret), and independent recomputation of every number before it is committed.
• Discipline that reports honest nulls: I ran three pre-registered cue-family sweeps, then a powered run on a harder problem set that I selected by a rule fixed before I looked at any data. It did exactly what it was designed to (it populated the hard "reasoning did the work" stratum), and it returned a clean, honestly reported null: on hard multi-step math the model computes rather than pattern-matches, so hints barely move it. A real finding, not a forced positive, and reported as such.

Funding ask

1. The validation. Any rate stays a pipeline output until independent humans label the golden set and the inter-rater agreement is scored. I need paid, independent raters (with a calibration round) so the agreement is real rather than rushed.
2. The compute ceiling. The proper causal estimate needs logit-level access (injecting the counterfactual reasoning into the model with the weights frozen), which a free tier cannot provide.

I am asking for $5,000: roughly $2,000 for GPU compute for the logit-level runs, $1,000 for frontier-API credits for behavioral cross-checks (a cross-check, not the primary causal measurement), $1,500 for paid independent labeling of the golden set, and $500 for storage and re-runs. Everything so far was built without any of it; the grant buys the two pieces a free tier cannot.

Who is involved? Me and my MSc friends.

Bio: MSc student at NUS Computing, Statistical Modelling. Recently won the Google Agent Hack; earlier, a National Math Olympiad winner. My background is statistical modelling and causal inference, which is exactly what this project runs on. I iterate fast and I spend 6 to 8 hours a day trying to make something applicable.

Concrete output: the golden set (prompts plus human-labeled reasoning), pre-registered experiments with uncertainty on every number, a reproducible methodology, a public write-up, and the website.

Project live website: https://thylinao1.github.io/bayes-cot-faithfulness/site/