Serious AI Incident Claims Need an Independent Evidence Record

Develops the Claim–Evidence–Independence framework to assess incident claims individually and proposes a reusable public evidence record for investigators and regulators.

Project Details

Serious AI incidents produce claims - "the model was contained," "no data persisted" - that are treated as established when they're often just asserted. This Track 2 project ("what happened, and what breaks next") introduces the Claim–Evidence–Independence (CEI) framework, which scores each individual claim by evidence, access, independence, and scope, rather than labeling an entire incident "investigated." Applied to OpenAI's July 2026 Hugging Face intrusion, it shows OpenAI's containment claim remains unverified even though the wider incident is well-corroborated - and a real precedent, Kenya's Worldcoin ruling, shows what independent verification actually requires. The result is a proposed public evidence record any investigator or regulator could reuse.

Hackathon paper: Serious AI Incident Claims Need an Independent Evidence Record

Theory of Impact

Effective AI incident response requires more than detecting an incident. Investigators and decision-makers also need to understand what happened, which claims are supported by evidence, and what remains uncertain.

Public reports about AI incidents often mix together statements from organizations, technical findings from internal investigations and independent analysis from outside experts. Although each of these sources can offer insights, they may differ in what evidence they have access to, their level of independence and how broad their scope is. When these differences aren’t made clear, readers might wrongly assume that an unverified claim is a confirmed fact. They might think that because something wasn’t mentioned it doesn’t exist at all.

The Incident Evidence Record aims to address this gap by linking each consequential claim to its evidence, provenance, assessment status, scope, and limitations. It could help investigators identify evidence gaps, help researchers compare findings across incidents, and help regulators formulate more precise requests for information.

The immediate goal is to build a specification that others can study and use. I also plan to develop case studies and a tested template that are fully transparent and reusable. Its longer-term impact will depend on whether independent users actually adopt the framework and whether testing shows the framework improves clarity and consistency, in incident assessments. Negative or inconclusive results would also be published so the framework can be improved or rejected if it proves unhelpful.

People

Gloria Nyambura Wanyaga

Team Member

Track Record

AI Incident Response Sprint (Apart Research), 2026

Grants Received– no grants recorded

Add grant

Funding Asks

Funding ask

Open

Minimum

$7,500

Ideal

$15,000

What this ask is

During the Apart Research AI Incident Response Sprint , I created the Claim – Evidence – Independence (CEI) framework. This framework looks at each Claim inside an AI incident by itself than treating the incident as one established account. For each Claim the framework notes the Evidence that supports the Claim, who can see that Evidence, the Independence of the assessment , the scope of the assessment and what remains unknown.

When I applied the CEI framework to the July 2026 OpenAI/Hugging Face incident the framework helped separate parts of the incident from Claims that still need more Evidence or an independent check.

With funding, I would develop and test the framework over six months, producing an open, reusable Incident Evidence Record (IER).

The work would involve:

Refining the methodology: Incorporate sprint feedback, clarify claim-status definitions, make scope explicit, and publish practical annotation guidelines.
Analyzing three AI incidents: Create at least 45 claim-level evidence records using publicly available documentation, clearly distinguishing evidence from inference.
Testing consistency: Have two independent annotators assess a sample of claims, compare their results, and document disagreements and ambiguities.
Seeking expert feedback: Obtain written reviews from approximately five researchers or practitioners working in AI safety, security, incident investigation, or governance.
Building and testing the artifact: Release an open spreadsheet template and JSON schema, test usability with approximately eight people, and revise the framework based on feedback.
Publishing the results: Release version 1.0 alongside a final report documenting the methodology, findings, limitations, and lessons from testing.

The goal is to establish whether a practical, structured evidence record can make AI incident assessments more transparent, consistent, and useful.

How the money will be spent

IDEAL BUDGET: $15,000 over six months

Researcher time: $9,000
Approximately 15 hours per week over six months for methodology development, case studies, analysis, and writing.
Expert review honoraria: $1,500
Five reviewers at $300 each for substantive written feedback on the framework.
Independent annotators: $1,000
Two annotators independently assessing a sample of claims to test whether the framework produces consistent results.
User-testing honoraria: $600
Approximately eight user-testing sessions at $75 each.
Tools, API credits, and hosting: $900
Necessary research tools and infrastructure for the open release.
Open release and outreach: $800
Publishing, a plain-language explainer, and one dissemination activity.
Administration, transfer fees, and contingency: $1,200
Payment costs and reasonable unforeseen expenses.

TOTAL: $15,000

MINIMUM BUDGET: $7,500 (reduced scope, about 3-4 months)

Researcher time: $4,500
Expert review (three reviewers): $900
Independent annotators: $500
Tools and hosting: $500
Open release: $400
Administration and contingency: $700

TOTAL: $7,500

At the minimum, I would deliver the revised framework, two to three case studies, a small agreement test, and the spreadsheet/JSON template, and defer user testing and outreach.

Discussion

Private comments in this thread are only shown to approved funders and grant reviewers. Email hi@grantmaking.ai or to request access.

Private UserPrivate11d?

AAnonymous judgeJudge · AI Incident Response Sprint11d

The author's research question was clever and as far as I can tell this was the only project in this hackathon that approached incidents through this lens. Overall, I think this is a clever look at an underserved problem, and I strongly encourage the author to follow through with next steps, including applying for grants, if need be.