Jiayi Li

Geometry · Learning dynamics · AI alignment

From the geometry of learning to the mathematics of AI alignment.

My research asks how the geometry of a learning system shapes what it can represent, which solutions training selects, and which features of the resulting model are identifiable. I use algebraic geometry, singular learning theory, and statistics to develop rigorous foundations for interpretability, reliability, and AI alignment.

I am a postdoctoral researcher in the Section of Mathematics and Artificial Intelligence at MPI-CBG in Dresden and an ILIAD Fellow in applied mathematics for AI alignment. I received my PhD in Statistics from UCLA's Mathematical Machine Learning Group.

Explore my research Curriculum vitae
Google Scholar GitHub Email Short bio

Research agenda

From structure to dynamics to guarantees

Questions of interpretability and alignment depend on three prior questions: what a model can represent, how training selects its behavior, and which properties remain meaningful under reparameterization. My work follows this chain from structure to dynamics to guarantees.

01

Structure

What can a network represent?

Neuromanifolds, neurovarieties, parameter symmetries, and identifiability reveal the geometry behind a model's function space.

Neuroalgebraic geometry

02

Dynamics

Why does learning change phase?

Singular invariants such as the local learning coefficient offer a new language for loss landscapes, generalization, and grokking.

Singular learning theory

03

Guarantees

Which claims survive?

Identifiability and invariant structure separate stable facts about a model from artifacts of parameterization. This distinction is the basis for rigorous interpretability, reliability, and alignment.

Mathematical guarantees

Selected work

Three views of learning through geometry

These projects move from the geometry of representable functions, through the dynamics that select solutions, to the identifiability needed for reliable interpretation.

Parameters map to a neuromanifold in function space Network parameters map to the neuromanifold of representable functions. Its dimension measures expressivity, while its learning degree measures algebraic training complexity. PARAMETER SPACE FUNCTION SPACE θ ∈ Θ φ 𝓜 = im(φ) dim 𝓜 measures expressivity learning degree measures training complexity

01 · Structure · Neuroalgebraic geometry

Geometry of Polynomial Neural Networks

What geometry is traced out by the functions a network can represent?

We describe the function spaces of polynomial networks as semialgebraic sets. Their dimension measures expressivity, while the learning degree captures an algebraic aspect of training complexity.

With Kaie Kubjas and Maximilian Wiesmann

Paper Research context
Training moves between competing solution basins The memorizing and generalizing basins both have training loss close to zero. A late optimization transition moves to the generalizing basin, whose lower local learning coefficient indicates greater statistical preference. TWO SOLUTIONS WITH LOSS NEAR ZERO late optimization transition train loss ≈ 0 train loss ≈ 0 MEMORIZATION λmem GENERALIZATION λgen < λmem lower LLC → more posterior mass → lower expected test error

02 · Dynamics · Singular learning theory

A Basin-Selection Perspective on Grokking via Singular Learning Theory

Why does a network generalize long after it has already fit the training data?

We interpret grokking as a transition between competing basins whose losses are nearly zero. The local learning coefficient ranks their statistical preference; optimization dynamics determine when training moves between them.

With Ben Cullen, Sergio Estan-Ruiz, and Riya Danait

Paper Research context
Third and fourth cumulants identify a directed dynamical system For a vector autoregressive model at steady state, third and fourth cumulants supplement covariance information and allow the directed parameter matrix to be recovered when the noise is not Gaussian. VECTOR AUTOREGRESSION AT STEADY STATE xt+1 = A xt + εt x₁x₂x₃ A ENCODES THE GRAPH κ₂covariance κ₃non-Gaussian κ₄information Arecovered DAG + self-loops → generic identifiability parameters are rational functions of the cumulants

03 · Guarantees · Identifiability

Identifiability in Graphical Discrete Lyapunov Models

When can a directed mechanism be recovered from observations at steady state?

Third and fourth cumulants reveal parameters that covariance alone cannot. This establishes identifiability beyond the Gaussian setting and provides a foundation for structure learning in directed systems.

With Cecilie Olesen Recke, Sarah Lumpp, Nataliia Kushnerchuk, Janike Oldekop, Jane Ivy Coons, and Elina Robeva

Paper Research context

Now & next

Research in motion

Recent papers, talks, programs, and community building around the mathematics of learning.

I will participate in IPAM's Foundations of Interpretability workshop at UCLA, connecting mathematical perspectives on learning with questions in interpretability and AI safety.

Through the ILIAD Fellowship in applied mathematics for AI alignment, I am developing a project that connects learning geometry with reliable AI.

Beyond the papers

Mathematics is also a community.

I care deeply about mentoring, teaching across mathematical backgrounds, and creating spaces where new collaborations can take shape. Outside work, you will usually find me with my husband and cat, playing badminton, or trying to keep everyone near a computer properly stretched.

A little more about me
A capybara, Jiayi's lighthearted spiritual portrait

Spiritually, a slightly more faithful portrait.