Research

In this paper, we study the expressivity and learning process for polynomial neural networks (PNNs) with monomial activation functions. The weights of the network parametrize the neuromanifold. In this paper, we study certain neuromanifolds using tools from algebraic geometry: we give explicit descriptions as semialgebraic sets and characterize their Zariski closures, called neurovarieties. We study their dimension and associate an algebraic degree, the learning degree, to the neurovariety. The dimension serves as a geometric measure for the expressivity of the network, the learning degree is a measure for the complexity of training the network and provides upper bounds on the number of learnable functions. These theoretical results are accompanied with experiments.

Joint work with Kaie Kubjas and Maximilian Wiesmann. [Paper] [Code]

In this work, we evaluate the betting method proposed by Waudby-Smith and Ramdas in generating confidence intervals and time-uniform confidence sequences for mean estimation with bounded observations. The methodology utilises composite non-negative martingales and establishes a connection to game-theoretic probability. We perform numerical comparisons with alternative methods and propose extension to vector settings.

Joint work with Yuantong Li and Xianwu Dai. [Paper] [Code]

This paper investigates the spectrum of the Jacobian of PH data encodings and the pull-back geometry that they induce on the data manifold. Then, by measuring different perturbations and features on the data manifold with respect to this geometry, we can identify which of them are recognized or ignored by the PH encodings. This also allows us to compare different encodings in terms of their induced geometry. Importantly, the approach does not require training and testing on a particular task and permits a direct exploration of PH. We experimentally demonstrate that the pull-back norm can be used as a predictor of performance on downstream tasks and to select suitable PH encodings accordingly. All experiments were conducted by the first author; I worked with formulating the pull-back norm and the differential geometry background.

Joint work with Shuang Liang, Renata Turkeš, Nina Otter and Guido Montúfar. [Paper]

In this paper, we establish a mathematical framework to analyze the resilience of artificial neural networks through the lens of differential geometry. Our geometric language provides natural algorithms that identify local vulnerabilities in trained networks as well as recovery algorithms that dynamically adjust networks to compensate for damage. We reveal striking vulnerabilities in commonly used image analysis networks, like MLP's and CNN's trained on MNIST and CIFAR10 respectively. We also uncover high-performance recovery paths that enable the same networks to dynamically re-adjust their parameters to compensate for damage. Broadly, our work provides procedures that endow artificial systems with resilience and rapid-recovery routines to enhance their integration with IoT devices as well as enable their deployment for critical applications.

Joint work with Guruprasad Raghavan and Matt Thomson. [paper]

Tropical geometry is a variant of algebraic geometry where people study polynomials and their geometric properties with addition replaced by minimization and multiplication replaced by ordinary addition. Under this formulation, the polynomial graphs would resemble piecewise linear meshes where numbers belong to the tropical semiring instead of a field. The maximum operation and the piecewise linear property of the mesh leads us to think of neural networks with a particular family of activations and the linear regions cut out by the activation functions. In 2018, Zhang et al. established the first connection between tropical geometry and feedforward neural networks with ReLU activation by showing that the family of such neural networks is equivalent to the family of tropical rational maps. We generalized the results from Zhang's paper and applied other techniques such as patchworking to study the expressive power of neural networks with piecewise linear activation functions.

We utilize voting records in conjunction with clustering and community detection algorithms to classify legislators into communities by political stance. The underlying assumption is that legislators with more similar voting records have more similar political stances. We consider legislatures from multiple countries: the United States House of Representatives, German Bundestag, Legislative Council of Hong Kong, and South Korean National Assembly. For each legislature, we collect roll call voting data and apply five different similarity functions to construct similarity matrices of the legislators. We then apply spectral clustering, Louvain with and without k-nearest neighbors preprocessing, and MBO modularity maximization methods to the similarity matrices.

Joint work with Kyung Ha, Grace Li, Blaine Talbut and Thomas Tu from Department of Mathematics, UCLA.