Covers: theory of Universal Approximation with Deep Narrow Networks

- Does there exist a universal approximation theorem for neural networks on bounded width and arbitrary depth that acts on Euclidean spaces?

Read the background and main discussion

Patrick Kidger, Terry Lyons

0 comment

Contributors

- Objectives
- Learn about the developments of Non-Euclidean Universal Approximation, and how it allows estimation of approximation bounds given a density of your neural network.
- Potential Use Cases
- Test how estimation changes with different NN densities.
- Who is This For ?
- ADVANCEDAdvanced audience looking to mathematically deduce estimation capabilities of NN given specified densities

Click on each of the following **annotated items** to see details.

Resources4/4

VIDEO 1. Intro to Universal Approximation Theorem

- What even is the universal approximation theorem and why should I care about it?

10 minutes

PAPER 2. Approximation by Superpositions of a Sigmoidal Function

- Can we estimate the approximation capabilities of a feed forward network with sigmoidal activation functions and one single hidden internal layer?

30 minutes

PAPER 3. Universal Approximation with Deep Narrow Networks

- Does there exist a universal approximation theorem for neural networks on bounded width and arbitrary depth that acts on Euclidean spaces?

30 minutes

PAPER 4. Quantitative Rates and Fundamental Obstructions to Non-Euclidean Universal Approximation with Deep Narrow Feed-Forward Networks

- Can we specify estimates of a deep and narrow network with general activation functions, irregardless if the spaces it acts on are Euclidean?

30 minutes

0 comment