Probabilistic symmetry and invariant neural networks (IS Colloquium)
In an effort to improve the performance of deep neural networks in data-scarce, non-i.i.d., or unsupervised settings, much recent research has been devoted to encoding invariance under symmetry transformations into neural network architectures. We treat the neural network input and output as random variables, and consider group invariance from the perspective of probabilistic symmetry. Drawing on tools from probability and statistics, we establish a link between functional and probabilistic symmetry, and obtain functional representations of probability distributions that are invariant or equivariant under the action of a compact group. Those representations characterize the structure of neural networks that can be used to represent such distributions and yield a general program for constructing invariant stochastic or deterministic neural networks. We develop the details of the general program for exchangeable sequences and arrays, recovering a number of recent examples as special cases. This is work in collaboration with Yee Whye Teh. https://arxiv.org/abs/1901.06082
Biography: My research focuses on probabilistic and statistical analysis of discrete data. In particular, I have worked on probabilistic models and inference for objects like graphs, partitions, and permutations. Natural applications of these ideas arise in, for example, modeling networks or text, and in matrix factorization. Recently, I have also worked on incorporating probabilistic symmetry into neural networks, and on probabilistic programming, particularly in the context of Bayesian nonparametric models. I am generally interested in all aspects of machine learning, both theoretical and applied. Previously, I completed my Ph.D. in Statistics at Columbia University, where I was advised by Peter Orbanz. I completed my B.S. in Physics at Stanford University, and my M.S. in Physics at Northwestern University, where I worked in the lab of William P. Halperin. Prior to studying at Columbia, I worked for three years as a research analyst at The Brattle Group in Washington, D.C.
