Mitigating emerging AI safety risks via new universal order parameter
The finding of neural optimization exhibiting phase transitions with a universal order parameter has direct implications for AI safety, and control. Standard monitoring is blind to dangerous regime changes, we are not.
The finding of neural optimization exhibiting phase transitions with a universal order parameter has direct implications for AI safety, and control. Standard monitoring is blind to dangerous regime changes, we are not.
Project Details
Updated 07/17/26 · Edited by orgWhen training or inferencing LLMs, there are many signals that can be measured (loss, confidence, coherence, forecast error), and there are various "pressure" metrics from the model's internal latent control systems which we can establish and access. The conventional assumption in machine learning is that these signals are sufficiently informative and conclusive. E.g., that lower loss means better learning, that higher confidence means better performance, that coherence indicates good health.
Our research revealed that this assumption is systematically wrong. These intuitive signals are either uncorrelated with actual training progress or, in some cases, inversely correlated. A model can look "healthy" by every standard metric while actually being in a state where learning has stopped or the architecture is faulty. Surprisingly many current models which have been accessible for research show significant instabilities.
When looking into how different components of the models internal governance system exerted "pressure" on the optimization dynamics. We distinguished three types of pressure Π (Pi for "Pressure"):
-
Alpha pressure (Π_α): The pressure exerted by the adaptive attention regulation mechanism. This is the component that actively modifies the model's internal representations during learning.
-
FFN pressure (Π_FFN): The pressure exerted by the Feed-Forward Network controller. This component acts as a stabilizer, clamping the model's activations to prevent runaway dynamics.
-
MPC pressure (Π_MPC): The pressure exerted by the Model Predictive Control system. This is a safety mechanism that intervenes when the model's internal state deviates from expected trajectories through state space.
There is a ratio between those three pressures: R = Π_α / (Π_FFN + Π_MPC)
It is the genuinely meaningful ratio of constructive pressure (alpha) to compensatory pressure (FFN + MPC). It is a single number that summarizes the distribution of optimization effort across the model's internal governance mechanisms.
Over and above that, R appears to be a macroscopic state variable that distinguishes qualitatively different optimization regimes. The relationship between R and loss is not fixed; it changes sign across the phase boundary. This is the reliable indication of a phase transition.
If the Constructive-Compensatory Ratio R (CCR) is indeed a universal observable of neural optimization, and we believe it is, then several important consequences follow:
-
Intuitive metrics are misleading. Confidence, coherence, and loss do not reliably indicate (training or inference) health. While R-number does.
-
Phase transitions exist. Neural optimizers can undergo qualitative changes in behavior without being noticed, the correlation between R and entropy loss vanishes at the critical point, so loss gives no signal of impending regime change.
-
Perturbations can be dangerous. It has been observed, that an architectural perturbation can push a model into the compensatory regime. We do not know yet, what other perturbations have the same effect like removing the Positional Encoding.
-
Safety monitoring can be universal. As R seems to be architecture-independent, we can build safety monitors that work across different model architectures. This must be further corroborated, nonetheless.
Perhaps it does not sound like much - in terms of saving the humanity from extinction. It is not like abolishing nuclear weapons. But it makes a remarkable difference in AI safety monitoring of the hidden space that is mostly considered as a black-box.
As to the question "who is involved" - although you see me referring to "us". "our" and "we" as is customary, in fact i am just one researcher involved with this project. Although tribute goes to my wife Hend, who supports my research with all her love. My name is Daniel Solis, I am a Czech national, born in Prague, currently residing at Hend's family place in the Nile Delta in Egypt. This project description is 100% human-written, although most terminology is ML and NN related (and machine spell-checked).
Theory of Impact
Updated 07/17/26 · By grantmaking.aiThe potential impact of the above presented discovery for the reduction of x-risk from AI is significant.
The core safety problem of current approaches to AI safety assume that models can be monitored by looking at standard metrics (loss, confidence, coherence, etc.). Our research shows that these metrics are systematically misleading. A model can appear stable and confident while actually being in the compensatory regime, coasting, broken, unstable and potentially unsafe.
R provides a fundamentally different kind of signal. Instead of asking "Is the model confident?" (which is ambiguous), R asks "Is the model in a constructive or compensatory regime?" This is a categorical distinction.
In production systems, this ration R could be monitored in real time:
-
If R > 1, the model is in the constructive regime (safe)
-
If R < 1, the model is in the compensatory regime (unsafe)
-
If R ≈ 1, the model is at the critical point (early warning)
People
Updated 07/17/26 · SourceTeam Member
Funding Details
Only visible to verified funders, reviewers, and admins.
Email hi@grantmaking.ai to get verifiedTrack Record
For the track record, see our publications
Emergent Phase-Locked Control and Optimization Pressure as a Macroscopic State Variable in Self-Governed LLM Training https://doi.org/10.5281/zenodo.21402446
Mycelia: Recursive Instability Forecasting and the Pressure Tensor - A Self-Aware Training Paradigm for Language Models https://doi.org/10.5281/zenodo.21261148
Dynamical Taxonomy of Recursive Collapse in Autoregressive Language Models https://doi.org/10.5281/zenodo.20576209
Recursive Entrainment and Latent Synchronization Regimes in Autoregressive Language Models
https://doi.org/10.5281/zenodo.20575616
Dynamical Classification System for Recursive Collapse in Autoregressive Language Models
https://doi.org/10.5281/zenodo.20542872
Discussion
No comments yet. Be the first to share your thoughts.