Covers: theory of Non-Eucliden Universal Approximation of Deep Feed-Forward Neural Network

- Can we specify estimates of a deep and narrow network with general activation functions, irregardless if the spaces it acts on are Euclidean?

Read the background and main discussion

Anastasis Kratsios, Leonie Papon

Nour Fahmy**Total time needed: **~2 hours

- Learning Objectives
- Learn about the developments of Non-Euclidean Universal Approximation, and how it allows estimation of approximation bounds given a density of your neural network.
- Potential Use Cases
- Test how estimation changes with different NN densities.
- Target Audience
- ADVANCEDAdvanced audience looking to mathematically deduce estimation capabilities of NN given specified densities

Go through the following **annotated items** *in order*:

VIDEO 1. Intro to Universal Approximation Theorem

- What even is the universal approximation theorem and why should I care about it?

10 minutes

PAPER 2. Approximation by Superpositions of a Sigmoidal Function

- Can we estimate the approximation capabilities of a feed forward network with sigmoidal activation functions and one single hidden internal layer?

30 minutes

PAPER 3. Universal Approximation with Deep Narrow Networks

- Does there exist a universal approximation theorem for neural networks on bounded width and arbitrary depth that acts on Euclidean spaces?

30 minutes

PAPER 4. Quantitative Rates and Fundamental Obstructions to Non-Euclidean Universal Approximation with Deep Narrow Feed-Forward Networks

- Can we specify estimates of a deep and narrow network with general activation functions, irregardless if the spaces it acts on are Euclidean?

30 minutes

Previewing stream ** Math and Foundations**

Upcoming Live Sessions

Videos

Learning Packages

Past Capstones

People

Search for a tag: