ARTICLEOverview of Neurons and Activation Functions

Covers: theory of Neuron
Estimated time needed: 5 minutes
Questions this item adddesses:
  • What is an Artificial Neural Network?
  • What is a neuron?
  • What is an Activation Function?
  • What are the most common activation functions?
How to use this item?

This online article provides the main concepts related with neural networks, including the definition of neuron and activation function, as well as the mathematical representation of the most common activation functions

URL: https://srnghn.medium.com/deep-learning-overview-of-neurons-and-activation-functions-1d98286cf1e4
Author(s) / creator(s) / reference(s)
Stacey Ronaghan
Shortlist
publicShare

Understanding Hidden layers

Sandra Lopez-ZamoraTotal time needed: ~52 minutes
Learning Objectives
The user of this shortlist will be able to understand the concept of hidden layers and the most common activation functions
Potential Use Cases
You are looking to understand the mathematical foundations for Deep Learning
Target Audience
BEGINNERDeep Learning practitioners interested in Mathematical foundations
Go through the following annotated items in order:
ARTICLE 1. An Introduction to Neural Networks
  • What are neurons?
  • How to code a neuron?
  • How to combine neurons in a neural network?
  • How to train a neural network?
15 minutes
ARTICLE 2. Overview of Neurons and Activation Functions
  • What is an Artificial Neural Network?
  • What is a neuron?
  • What is an Activation Function?
  • What are the most common activation functions?
5 minutes
VIDEO 3. Activation Function
  • What is an activation function?
  • Which activation function could work better?
  • What are the most common activation functions?
  • Which activation function should I use for the output layer?
  • When to use the most common activation functions?
11 minutes
ARTICLE 4. Activation Functions in Neural Networks
  • What is Activation Function?
  • Why we use Activation functions with Neural Networks?
  • Sigmoid or Logistic Activation Function
  • Tanh or hyperbolic tangent Activation Function
  • ReLU (Rectified Linear Unit) Activation Function
  • Leaky ReLU
  • Derivatives of activation functions
10 minutes
VIDEO 5. Layers in a Neural Network explained
  • What are the type of layers in ANN?
  • What kind of calculations occur within a hidden layer?
  • How to build a hidden layer in Keras?
6 minutes
ARTICLE 6. Simplified Guide to Hidden Layer Neural Networks by DL Practitioner
  • What is a multilayer perceptron (MLP)?
  • What's the role of Feedforward and back propagation algorithms?
  • Chain rule
  • Gradient Descend Algorithm
15 minutes

Concepts Convered