AI-Accelerated Product Development
Language Modelling from N-grams to Transformer
Total time needed:
With this list, you will learn about Language Modelling and the three families of methodology to model language and use it for text understanding or text generation.
Potential Use Cases
Chatbot, Speech recognition, NLU and NLG for Downstream tasks
Who is This For ?
Python Developers who have basic understanding of NLP (experienced with text processing, n-grams, word-vectors)
Click on each of the following
to see details.
1. Introduction to Language Modelling and N-gram LM
What is a language model?
How does an N-gram language model work?
2. In-depth take into N-gram modelling
How was language modelling traditionally done?
What was the drawbacks of traditional LM techniques?
What methods were proposed to mitigate the drawbacks?
3. An Intuitive understanding of the use of NN in LM
How does a NN achieve LM?
What are the typical NN architecture that can do LM?
4. A mathematical understanding of how RNNs model language
How exactly does a Neural Network model sequences?
5. A Survey on Neural Network Language Models
What are NN architecture used for Language Modelling?
How did they contribute to improve SOTA performance on Language Modelling?
6. Rasa's series for Explanation of the Transformer Architecture and its components
What is self-attention?
What is the Transformer?
7. The Narrated Transformer Language Model
How does the Transformer achieve Language Modelling?
What are the reasons why this modern architecture performs well?
8. Implementation of Language Modelling with three common algorithms
How do we implement language models?
9. 8 Leading Language Models For NLP In 2020
What are the pre-trained model you can use in the market?
What are the details about each pre-trained model (architecture, achievement, use cases)?