Covers: theory of Transformer-based LMs
Estimated time needed to finish: 50 minutes
Questions this item addresses:
  • What is self-attention?
  • What is the Transformer?
How to use this item?

Watch all 4 parts of the Transformer mechanism

Author(s) / creator(s) / reference(s)
Rasa Youtube content
0 comment

Language Modelling From N-grams To Transformer

Total time needed: ~4 hours
With this list, you will learn about Language Modelling and the three families of methodology to model language and use it for text understanding or text generation.
Potential Use Cases
Chatbot, Speech recognition, NLU and NLG for Downstream tasks
Who is This For ?
INTERMEDIATEPython Developers who have basic understanding of NLP (experienced with text processing, n-grams, word-vectors)
Click on each of the following annotated items to see details.
ARTICLE 1. Introduction to Language Modelling and N-gram LM
  • What is a language model?
  • How does an N-gram language model work?
10 minutes
ARTICLE 2. In-depth take into N-gram modelling
  • How was language modelling traditionally done?
  • What was the drawbacks of traditional LM techniques?
  • What methods were proposed to mitigate the drawbacks?
40 minutes
ARTICLE 3. An Intuitive understanding of the use of NN in LM
  • How does a NN achieve LM?
  • What are the typical NN architecture that can do LM?
15 minutes
VIDEO 4. A mathematical understanding of how RNNs model language
  • How exactly does a Neural Network model sequences?
12 minutes
PAPER 5. A Survey on Neural Network Language Models
  • What are NN architecture used for Language Modelling?
  • How did they contribute to improve SOTA performance on Language Modelling?
15 minutes
VIDEO 6. Rasa's series for Explanation of the Transformer Architecture and its components
  • What is self-attention?
  • What is the Transformer?
50 minutes
VIDEO 7. The Narrated Transformer Language Model
  • How does the Transformer achieve Language Modelling?
  • What are the reasons why this modern architecture performs well?
30 minutes
USE_CASE 8. Implementation of Language Modelling with three common algorithms
  • How do we implement language models?
20 minutes
USE_CASE 9. 8 Leading Language Models For NLP In 2020
  • What are the pre-trained model you can use in the market?
  • What are the details about each pre-trained model (architecture, achievement, use cases)?
10 minutes

Concepts Covered

0 comment