| Contents | In this talk, we will explore the geometry of the function spaces represented by fixed neural-network architectures. We will focus on algebraic feedforward neural networks with polynomial, rational, and ReLU activation functions. For a fixed architecture, we will associate a geometric object called a neuromanifold: the image of its parameter space in an ambient space of functions. Taking the Zariski closure of the neuromanifold allows us to associate an algebraic variety with the architecture. I will explain how geometric invariants of this variety, such as dimension, degree, defining equations, and singularities, can reveal structural properties of neural networks and shed light on their black-box behavior. In particular, I will present recent results showing how the study of shallow rational neural networks connects to several classical objects and questions in algebraic geometry, including Chow varieties, secant varieties of Veronese and Grassmann varieties, and Prony varieties.
|
