PAPERIs it Time to Swish? Comparing Deep Learning Activation Functions Across NLP tasks
Star
Covers: theory of Activation Functions
Estimated time needed to finish: 20 minutes
Questions this item addresses:
  • When should you use alternative types of activation functions?
How to use this item?

Read sections 1, 2, 4, 5.

URL: https://www.aclweb.org/anthology/D18-1472.pdf
Author(s) / creator(s) / reference(s)
Steffen Eger, Paul Youssef, Iryna Gurevych
0 comment
Recipe
publicShare
Star0

Activation Functions for Neural Networks in Natural Language Processing

Contributors
Total time needed: ~2 hours
Objectives
You will learn about activation functions in general, and about how they are used in neural networks for NLP.
Potential Use Cases
Creating an NLP neural network, or experimenting with different activation functions for an NLP neural network.
Who is This For ?
BEGINNERThis list would be useful for someone new to NLP or neural networks.
Click on each of the following annotated items to see details.
ARTICLE 1. Fundamentals of Deep Learning – Activation Functions and When to Use Them?
  • What are the different types of activation functions?
15 minutes
BOOK_CHAPTER 2. Neural Network Methods for Natural Language Processing
  • What activation functions are used in NLP neural networks?
10 minutes
PAPER 3. A Primer on Neural Network Models for Natural Language Processing
  • What are the activation functions used within a neural network for NLP, and for the output layer?
10 minutes
ARTICLE 4. 7 types of Artificial Neural Networks for Natural Language Processing
  • What types of neural networks are used for NLP?
15 minutes
PAPER 5. Is it Time to Swish? Comparing Deep Learning Activation Functions Across NLP tasks
  • When should you use alternative types of activation functions?
20 minutes

Concepts Covered

0 comment