grantmaking.ai
Actively FundraisingRecent ActivityFull Database
Resources
grantmaking.ai
Actively FundraisingRecent ActivityFull Database
ResourcesList your project
grantmaking.ai kickoff grant round$1M / $1M distributed
List your project

Database

The AI Safety database is a work in progress, built from public sources. Missing, or out of date? You can list your project yourself. Feel free to send us ideas or feedback!
Raising funds(remove filter)Clear all
NameTypeTagsEndorsedTeamRaising $

Showing 151-200 of 487·Sorted by endorsements

NameTypeTagsEndorsedTeamRaising $
AI Safety Nepal
Project
TrainingCommunityTechnical Safety
--

AI Safety Nepal

Team?
Project
Previous

Page 4 of 10

Next
Travel for three from Canada to PauseCon 2026 in London, Sept 4-8PauseAI
Project
CommunityGovernance
-
-
PayBench: A Benchmark for Unsafe Commercial Autonomy
Project
ResearchEvals
--
Behavioral Safety in Conversational and Agentic AI
Project
ToolingEvals
--
Frankfurt AI Safety
Project
NetworkCommunityTechnical Safety
--
AIxBIo Africa Pilot Fellowship
Project
TrainingTechnical Safety
--
GCR Disaster Response Benchmark
Project
ResearchEvalsX-Risk
--
Neural data privacy evals on frontier LLMs
Project
IndividualResearchEvals
--
The Capability Race: Inside the AI Safety Crisis
Project
MediaCommsTechnical Safety
--
Machine God - documentary film on AI X-Risk
Project
MediaCommsX-Risk
--
Multi-agent cooperative contracts
Project
ResearchCooperative AI
--
Building Compute Governance Capacity for the Philippines' Pax Silica Hub
Project
AdvocacyCompute Gov
--
Defining dangerous DNA with the Sequence Biosecurity Risk Consortium (SBRC)
Project
StandardsBiosecurity
--
Prediction of Inoculation Prompt Side Effects
Project
IndividualResearchInterp
--
Threat Models for AI-Enabled Crisis Instability
Project
ResearchGovernance
--
Establishing an AI Risk Communication for Public Preparedness Framework
Project
IndividualField-BuildingGovernance
--
Sentient Futures Project Incubator
Project
TrainingAI Welfare
--
"How Could The AI Even Hurt Us If It Doesn't Have A Body"
Project
CommsSecurity
--
AI ≠ Human Time
Project
ResearchTooling
--
Red-Teaming Evaluation for Computer-Use Agents
Project
ResearchEvals
--
Yale Jackson Initiative on AI SafetyYale University
Project
AcademicCommunityGovernance
--
Scaling Laws for Agentic Inference
Project
ResearchEvals
--
Are Global AI-Biosecurity Safeguards Really Global?
Project
ResearchBiosecurity
--
Scaling Honesty: Deception Detection & Steering Beyond 70B
Project
ResearchDeception
--
echo-truth-llm: Open Source Deception Detection & Correction Toolkit
Project
ToolingDeception
--
proofbundle: Offline Evidence Receipts for AI Safety Evaluations
Project
ToolingSecurity
--
Coordination Studies
Project
Think TankField-BuildingGovernance
--
Honesty Drift: Measuring Honesty Degradation in LLM Interactions
Project
Research LabResearchEvals
--
Comprehension Audits to Mitigate Risks from Automated AI Research
Project
StandardsGovernance
--
Daios, an independent lab post-training machines with virtue
Project
Research LabResearchTechnical Safety
--
Estonian AISI
Project
AcademicAdvocacyGovernance
--
Stress-Testing AgentHarm
Project
IndividualResearchEvals
--
EthicsNet: Open Safety Infrastructure for Agentic AI
Project
PlatformToolingTechnical Safety
--
Coalition Circuits: Mechanistic Trajectories of Cooperation and Betray
Project
IndividualResearchInterp
--
A Game-Theoretic Model of AI Consciousness Disagreement as X-Risk
Project
ResearchAI WelfareGovernance
--
Preventing AI-enabled coups in middle powers
Project
Field-BuildingGovernance
--
Training Language Models to Discover RowHammer Vulnerabilities in DRAM
Project
ToolingSecurity
--
dreamd
Project
IndividualResearchEvals
--
AI risk education to underserved communities
Project
CommsX-Risk
--
EvalLabs: Open Infrastructure for AI Evaluation Research
Project
IndividualToolingEvals
--
Post training in AI models for high stakes
Project
ResearchInterp
--
Circuits in the State-Transition Operator of Recurrent Language Models
Project
ResearchInterp
--
Volta Education Suite: AI Risk Engine (VES:ARE)
Project
EducationX-Risk
--
NTU AI Safety - Financial support for organizers
Project
NetworkCommunityTechnical Safety
--
Continued development of Commec, IBBIS’s open-source DNA screening tool
Project
ToolingBiosecurity
--
DownStream - A Short Form Based AI Safety Comedy News Series
Project
MediaCommsX-Risk
--
MICCAI Travel: Mechanistic Auditing of Multimodal Foundation Models
Project
IndividualResearchInterp
--
Humanity Tomorrow
Project
NewsletterCommsX-Risk
--
Travel Grant for Undergrad to present at MICCAI Mechanistic AI Safety Workshop
Project
IndividualResearchInterp
--
scry.io: 300 TB databases making the internet hyper-traversable to research agents
Project
PlatformOpsTechnical Safety
--
Training
Community
Technical Safety
Fundraising
Claimed

AI safety fellowship to upskill local talent, build a pipeline of people who understand AI safety deeply enough to contribute to research, advise on policy, and coordinate when AI governance decisions are being made.

Led byAnju Chhetri, and others
Endorsed by-

Travel for three from Canada to PauseCon 2026 in London, Sept 4-8

Team?
ProjectCommunityGovernancePauseAIFundraisingClaimed

Three members of PauseAI Canada would like to attend PauseCon 2026 in London, England. We will require funds for travel from Montreal and Winnipeg, accommodation for one night, food, and other expenses.

Led byJeremy Eliosoff
Endorsed by-

PayBench: A Benchmark for Unsafe Commercial Autonomy

Team?
ProjectResearchEvalsFundraisingClaimed

Benchmark for agents spending human money - paybench.org

Led byConor Plunkett
Endorsed by-

Behavioral Safety in Conversational and Agentic AI

Team?
ProjectToolingEvalsFundraisingClaimed

Building computational tools to identify and address pathological behavioral states in conversational and autonomous AI

Led byAlejandro De Los Angeles
Endorsed by-

Frankfurt AI Safety

Team?
ProjectNetworkCommunityTechnical SafetyFundraisingClaimed

An AI safety community that trains professionals and researchers to become competent AI safety contributors within industries and the academia, by learning and doing something.

Led byHelen Adepoju
Endorsed by-

AIxBIo Africa Pilot Fellowship

Team?
ProjectTrainingTechnical SafetyFundraisingClaimed

AIxBio Africa is a five-week remote fellowship mentoring early-career researchers on Africa-relevant projects at the intersection of AI safety, biosecurity, governance and public health, producing publishable outputs.

Led byFatika Umar Ibrahim
Endorsed by-

GCR Disaster Response Benchmark

Team?
ProjectResearchEvalsX-RiskFundraisingClaimed

An LLM benchmark for emergency response communications in existential catastrophes.

Led byJames Mulhall, and others
Endorsed by-

Neural data privacy evals on frontier LLMs

Team?
ProjectIndividualResearchEvalsFundraisingClaimed

A dangerous-capability benchmark testing whether frontier LLMs can extract sensitive attributes from anonymized brain recordings, enable adversaries to build privacy-violating tools, or over-refuse legitimate neuroscience queries.

Led bySwaraag Sistla
Endorsed by-

The Capability Race: Inside the AI Safety Crisis

Team?
ProjectMediaCommsTechnical SafetyFundraisingClaimed

A high-production visual journalism series combining primary-source data and motion graphics to break down the mechanics, safety crises, linguistic biases, and Arab-world implications of transformative AI.

Led byMagdi Hamed
Endorsed by-

Machine God - documentary film on AI X-Risk

Team?
ProjectMediaCommsX-RiskFundraisingClaimed

Feature-length documentary that captures the intellectual and cultural zeitgeist surrounding frontier AI at the moment before AGI, and communicates X-Risk ideas to a broad range of elites.

Led bySteve Hsu
Endorsed by-

Multi-agent cooperative contracts

Team?
ProjectResearchCooperative AIFundraisingClaimed

Multi-agent cooperative contractual obligation framework.

Led byBenjamin John Schulz
Endorsed by-

Building Compute Governance Capacity for the Philippines' Pax Silica Hub

Team?
ProjectAdvocacyCompute GovFundraisingManifund

Audit and brief Philippine agencies to update compute export-control rules, border identification, inspections, and a compute registry to govern frontier AI chips ahead of the Pax Silica hub.

Led byVir Khosla
Endorsed by-

Defining dangerous DNA with the Sequence Biosecurity Risk Consortium (SBRC)

Team?
ProjectStandardsBiosecurityFundraisingManifund

Defining which sequences are dangerous enough to screen for, so providers and regulators screen consistently

Led byIBBIS
Endorsed by-

Prediction of Inoculation Prompt Side Effects

Team?
ProjectIndividualResearchInterpFundraisingManifund

An open, cheap method that detects when an inoculation prompt inoculates against off-target traits, so labs and developers can catch undesired trait/persona cha

Led byNikhil Maturi
Endorsed by-

Threat Models for AI-Enabled Crisis Instability

Team?
ProjectResearchGovernanceFundraisingManifund

Detailed models of how delegating critical functions to AI agents could destabilise strategic stability: pathways, likelihoods, and defenses.

Led byAmritanshu Prasad
Endorsed by-

Establishing an AI Risk Communication for Public Preparedness Framework

Team?
ProjectIndividualField-BuildingGovernanceFundraisingManifund

There's an asymmetry between the level of communication about AI capabilities, compared to AI risks. If we hit AGI tomorrow, the world would not be prepared.

Led byTheophilus Femi Alawonde
Endorsed by-

Sentient Futures Project Incubator

Team?
ProjectTrainingAI WelfareFundraisingManifund

Funding compute/API costs for Incubator projects that build nonhuman welfare consideration into AI safety work

Led byConstance Li
Endorsed by-

"How Could The AI Even Hurt Us If It Doesn't Have A Body"

Team?
ProjectCommsSecurityFundraisingClaimed

Drone swarms hunting people in the woods, as a less-dry video explainer of risks from AI.

Led byAaron Silverbook
Endorsed by-

AI ≠ Human Time

Team?
ProjectResearchToolingFundraisingClaimed

Build a digital intervention that measures the human time needed to produce AI-generated outputs

Led byAbigail Browka
Endorsed by-

Red-Teaming Evaluation for Computer-Use Agents

Team?
ProjectResearchEvalsFundraisingClaimed

A diagnostic evaluation framework that tells whether computer-use agent attacks fail because the agent is robust, refused, never saw the attack, or was simply unable to execute the harmful action.

Led byRohit Saxena, and others
Endorsed by-

Yale Jackson Initiative on AI Safety

Team?
ProjectAcademicCommunityGovernanceYale UniversityFundraisingClaimed

We are building the first AI policy and safety talent pipeline at Yale University to equip future global decision-makers with frontier AI risk literacy and technical governance tools.

Led byAmy Wang, and others
Endorsed by-

Scaling Laws for Agentic Inference

Team?
ProjectResearchEvalsFundraisingClaimed

How exactly does scaling inference compute affect the performance and reliability of LM agents in agentic benchmarks? We believe that most current evaluations underestimate performance because they do not account harness.

Led byEduard Kapelko, and others
Endorsed by-

Are Global AI-Biosecurity Safeguards Really Global?

Team?
ProjectResearchBiosecurityFundraisingClaimed

Assess AI-enabled biosecurity safeguards in Nigeria by reviewing governance, red-teaming frontier models with local language/culture, and testing DNA synthesis screening for orders from resource-limited settings.

Led byNnaemeka Emmanuel Nnadi, and others
Endorsed by-

Scaling Honesty: Deception Detection & Steering Beyond 70B

Team?
ProjectResearchDeceptionFundraisingClaimed

Extend recently developed deception detection+steering method to larger and more diverse open-weight models, releasing the tooling that makes frontier-scale honesty interventions reproducible.

Led byJared Glover
Endorsed by-

echo-truth-llm: Open Source Deception Detection & Correction Toolkit

Team?
ProjectToolingDeceptionFundraisingClaimed

An open-source library that both detects self-chosen deception in open-weight LLMs and steers the model back toward honesty at inference — the correction half that current honesty tools lack.

Led byJared Glover
Endorsed by-

proofbundle: Offline Evidence Receipts for AI Safety Evaluations

Team?
ProjectToolingSecurityFundraisingClaimed

proofbundle lets people verify AI safety evaluation results offline instead of trusting a number in a PDF and it catches if an evaluation record was altered or swapped

Led byKonrad Gruszka
Endorsed by-

Coordination Studies

Team?
ProjectThink TankField-BuildingGovernanceFundraisingClaimed

Coordination Studies is a new field-building project oriented towards solving coordination problems and designing new coordination mechanisms.

Led byMax Holly
Endorsed by-

Honesty Drift: Measuring Honesty Degradation in LLM Interactions

Team?
ProjectResearch LabResearchEvalsFundraisingClaimed

An open benchmark measuring how model honesty survives long, pressured conversations and multi-agent interaction - lying, sycophancy, and calibration tracked turn by turn

Led byPhan Xuan Tan
Endorsed by-

Comprehension Audits to Mitigate Risks from Automated AI Research

Team?
ProjectStandardsGovernanceFundraisingClaimed

*Comprehension audits* are a novel development-process assurance mechanism to verify human understanding of AI research outputs to act as a gate to slow automation of AI R&D.

Led byRonald Bodkin
Endorsed by-

Daios, an independent lab post-training machines with virtue

Team?
ProjectResearch LabResearchTechnical SafetyFundraisingClaimed

Post-training virtue into open models as a third alignment technique and control mechanism

Led byAndrew Agathon, and others
Endorsed by-

Estonian AISI

Team?
ProjectAcademicAdvocacyGovernanceFundraisingClaimed

6 months of salary support for my research and operations work to help found a quasi-governmental AI safety institution, the [Estonian AI Security Institute](https://www.aisi.ee/) (Turvalise Tehisaru Teadmuskeskus T3).

Led byKristi Uustalu
Endorsed by-

Stress-Testing AgentHarm

Team?
ProjectIndividualResearchEvalsFundraisingClaimed

Run AgentHarm on 3–4 frontier/open-weight models, manually audit transcripts for metric gaming and spurious failures, compare to prior critiques, and publish a detailed evidence-based writeup.

Led byAnshuman Singh
Endorsed by-

EthicsNet: Open Safety Infrastructure for Agentic AI

Team?
ProjectPlatformToolingTechnical SafetyFundraisingClaimed

Free, open, forkable safety infrastructure for agentic AI: a peer-reviewed pathology nosology, a safety runtime with reproducible benchmarks, and enforcement gating every agent tool call against human-authored policy.

Led byEleanor 'Nell' Watson
Endorsed by-

Coalition Circuits: Mechanistic Trajectories of Cooperation and Betray

Team?
ProjectIndividualResearchInterpFundraisingClaimed

An open benchmark and causal interpretability study of when individually power-motivated LLM agents compete, form coalitions, collude, or betray—and whether internal signals reveal these shifts before behavior does.

Led bySubramanyam Sahoo
Endorsed by-

A Game-Theoretic Model of AI Consciousness Disagreement as X-Risk

Team?
ProjectResearchAI WelfareGovernanceFundraisingClaimed

A strategic model of governance under uncertainty: how disagreement about AI consciousness undermines the coordination that keeps AI risks in check.

Led bySoenke Ziesche
Endorsed by-

Preventing AI-enabled coups in middle powers

Team?
ProjectField-BuildingGovernanceFundraisingClaimed

Career transition grant to allow for research focused on how advanced AI models may be used to concentrate power in middle powers

Led byEgerton Neto
Endorsed by-

Training Language Models to Discover RowHammer Vulnerabilities in DRAM

Team?
ProjectToolingSecurityFundraisingClaimed

Build an open-source RL environment using Ramulator and real disturbance data to post-train language models to exploit simulated DRAM RowHammer vulnerabilities, plus write-up/blog and trained models.

Led byJaray Foo, and others
Endorsed by-

dreamd

Team?
ProjectIndividualResearchEvalsFundraisingClaimed

Neutral, reproducible benchmark measuring whether AI memory systems update correctly when facts change — every major system in one open table, October 2026.

Led byPhillip Austin Green
Endorsed by-

AI risk education to underserved communities

Team?
ProjectCommsX-RiskFundraisingClaimed

Educate everyday people on AI risk, and bring marginalized voices into the global AI conversation.

Led bySam Lucas
Endorsed by-

EvalLabs: Open Infrastructure for AI Evaluation Research

Team?
ProjectIndividualToolingEvalsFundraisingClaimed

Demystifying AI evaluation research by lowering technical barriers through practical, open evaluation infrastructure.

Led byValencia Cooper
Endorsed by-

Post training in AI models for high stakes

Team?
ProjectResearchInterpFundraisingClaimed

A mechanism for evolutionary post training in models using activation steering based methods.

Led byAtmadeep Ghoshal
Endorsed by-

Circuits in the State-Transition Operator of Recurrent Language Models

Team?
ProjectResearchInterpFundraisingClaimed

With raise of GDN and different sorts of attention meachanisms, those are much closer to lstm/recurrent architectures being very stateful rather than normal attention, we aim to explain explore common patters in lstm and GDN.

Led byNikita Khomich
Endorsed by-

Volta Education Suite: AI Risk Engine (VES:ARE)

Team?
ProjectEducationX-RiskFundraisingManifund

Facilitating AI X-risk Education in India, Southwest Cameroon, and beyond!

Led byMariano
Endorsed by-

NTU AI Safety - Financial support for organizers

Team?
ProjectNetworkCommunityTechnical SafetyFundraisingManifund

NTU AI Safety seeks $13k for organizer stipends to expand Taiwan’s first student AI safety group into three reading-group tracks (fundamentals, technical, policy/governance) and grow outreach.

Led byManifund user 7cf124f1, and others
Endorsed by-

Continued development of Commec, IBBIS’s open-source DNA screening tool

Team?
ProjectToolingBiosecurityFundraisingManifund

Next year of Commec (the Common Mechanism), the free, open-source, globally-available DNA synthesis screening tool hosted by IBBIS

Led byIBBIS
Endorsed by-

DownStream - A Short Form Based AI Safety Comedy News Series

Team?
ProjectMediaCommsX-RiskFundraisingManifund

Deigned to get everyday people more knowledgable and more invested in AI Safety causes! Designed similarly to Dave Jorgenson's "Local News International".

Led byAna Hawryluk
Endorsed by-

MICCAI Travel: Mechanistic Auditing of Multimodal Foundation Models

Team?
ProjectIndividualResearchInterpFundraisingManifund

Accepted paper at Mechanistic Interpretability for Foundation Models workshop.no travel funding. Early Career Researcher

Led byGaurav Hadavale
Endorsed by-

Humanity Tomorrow

Team?
ProjectNewsletterCommsX-RiskFundraisingClaimed

Expand Humanity Tomorrow with a multilingual, jargon-free AI existential-risk section featuring a comprehensive FAQ, field map, action recommendations, and supporting visuals/audio, plus outreach and maintenance.

Led byMarko Prakhov-Donets
Endorsed by-

Travel Grant for Undergrad to present at MICCAI Mechanistic AI Safety Workshop

Team?
ProjectIndividualResearchInterpFundraisingManifund

Seeking travel and registration fee support to present my sole-authored mechanistic audit of medical AI at MICCAI 2026 MI4MedFM workshop at Strasbourg, France

Led byGaurav Hadavale
Endorsed by-

scry.io: 300 TB databases making the internet hyper-traversable to research agents

Team?
ProjectPlatformOpsTechnical SafetyFundraisingClaimed

Giving people and their prosocial agents ergonomic SQL query power over the internet, with structured judgement kernels to organize the information across interpretable high-dimensional axes.

Led byXyra Sinclair
Endorsed by-