AI safety projects actively seeking funding: what they’re working on and how much they need.
Showing 51-85 of 85 Top rated
An open-source benchmark that measures whether tool-using AI agents faithfully report their actions, failures, and policy violations, using deterministic execution traces as ground truth. Read more
An evaluation harness testing whether AI judgment holds up under contradicting evidence Read more
OASIS will build and test a reproducible multi-agent AI safety sandbox for coordination, memory, oversight, and accountability, releasing an open-source prototype, eval protocols, experiments, and a report. Read more
A graduate student in mathematical ML and a postdoc in technology law, trained together at the Fields Institute and uOttawa's CLTS, on research aimed squarely at reducing existential risk from AI. Read more
A controlled digital ecology for studying how strategies, causal models, errors, and safety-relevant behaviors are selected and transmitted across generations of LLM agents. Read more
A pilot curriculum of six Socratic seminars for educators and other nontechnical learners. Read more
Developing evaluation methods to determine when CV models can be trusted by addressing the gap between benchmark metrics and real-world performance. Read more
Focusing more on the intersection of red teaming and interp, while creating a strong rooted community. Read more
A scalable fellowship training researchers to develop interventions for achieving high-value long-term futures Read more
An open-source, model-agnostic governance layer that uses constitutional state, trajectory memory, and mathematical control mechanisms to govern LLM and AI-agent behavior at runtime. Read more
An additional year funding for the co-working hub in Sydney, Australia, which offers free office space for people working in the field of AI safety. Read more
OSMSI : Open Source Model Safety Index | https://osmsi.disobedientgeometry.com/ Read more
Help Sue Private Intelligence Firms Hitting AI Whistleblowers Read more
Audit and brief Philippine agencies to update compute export-control rules, border identification, inspections, and a compute registry to govern frontier AI chips ahead of the Pax Silica hub. Read more
Defining which sequences are dangerous enough to screen for, so providers and regulators screen consistently Read more
An open, cheap method that detects when an inoculation prompt inoculates against off-target traits, so labs and developers can catch undesired trait/persona cha Read more
Detailed models of how delegating critical functions to AI agents could destabilise strategic stability: pathways, likelihoods, and defenses. Read more
There's an asymmetry between the level of communication about AI capabilities, compared to AI risks. If we hit AGI tomorrow, the world would not be prepared. Read more
Facilitating AI X-risk Education in India, Southwest Cameroon, and beyond! Read more
NTU AI Safety seeks $13k for organizer stipends to expand Taiwan’s first student AI safety group into three reading-group tracks (fundamentals, technical, policy/governance) and grow outreach. Read more
Next year of Commec (the Common Mechanism), the free, open-source, globally-available DNA synthesis screening tool hosted by IBBIS Read more
Deigned to get everyday people more knowledgable and more invested in AI Safety causes! Designed similarly to Dave Jorgenson's "Local News International". Read more
Accepted paper at Mechanistic Interpretability for Foundation Models workshop.no travel funding. Early Career Researcher Read more
Creating a reference model for mechanistic interpretability without assuming that at auditing time we have a safe model to compare the suspicious model against. Read more
Civil-society infrastructure against AI-enabled power concentration, built around autonomous weapons Read more
An administrable protocol distinguishing emergent self-modeling from roleplay — 90-day validation, open publication, audit-ready scoring. Read more
Testing whether explicit source structure changes how AI systems reason from consequential documents. Read more
Testing whether dynamic boundaries, history, feedback, and uncertainty-aware decisions can make AI behavior more interpretable and auditable. Read more
Fly two volunteer leaders of PauseAI Australia to PauseCon London, to bring UK's learnings home and train our all-volunteer chapter Read more
Catching scoring flakiness and prompt sensitivity in AI benchmark scripts. Read more
A cryptographic security auditor that uses Merkle hash trees to catch tool poisoning in Model Context Protocol. Read more
A benchmark measuring whether autonomous AI agents can escape sandboxes, exfiltrate data, or hack reward scorers. Read more
Building Hispanic participation in AI safety through bilingual education, policy research, and public communication. Read more
Context-Aware Defenses Against Indirect Prompt Injection in Agentic AI System Read more
