In this video you will learn about:
Besides other properties, Deep learning is also the powerful technique because it to accounts for non-linearities in data. A fully-conmnected NN is only a bunch of regression equations. One neuron would be a sum of all neurons in previous layer wired via this equation: , where is the weight learnd in training . By introducing a non-linearity, you make the neuron either to FIRE or NOT in a small interval (e.g 0,1 or -1,1) !
Non-linearity introduced by the Activation function. Source: towards.ai