Covers: theory of Differences in BERT and RoBERTa
Estimated time needed to finish: 20 minutes
Questions this item addresses:
  • What are the differences between BERT and RoBERTa?
How to use this item?

Entire video

Fail to play? Open the link directly: https://www.youtube.com/watch?v=-MCYbmU9kfg
Author(s) / creator(s) / reference(s)
Yannic Kilcher
0 comment
Recipe
publicShareStar

Roberta

Collaborators
Total time needed: ~4 hours
Objectives
Here, you will be learn about the RoBERTa, the way it differs from BERT, and how to use in your notebook.
Potential Use Cases
To use RoBERTa in your NLP models
Who is this for ?
INTERMEDIATENLP users trying to implement RoBERTa
Click on each of the following annotated items to see details.
ARTICLE 1. A Summary of RoBERTa
  • What is RoBERTa about?
  • Why RoBERTa matters?
10 minutes
PAPER 2. The RoBERTa Paper
  • What are the problems associated with language models?
  • How does RoBERTa compare with BERT in terms of structure, training, and performance?
60 minutes
OTHER 3. From BERT to RoBERTa - 1
  • What are the differences between BERT and RoBERTa?
30 minutes
VIDEO 4. From BERT to RoBERTa - 2
  • What are the differences between BERT and RoBERTa?
20 minutes
ARTICLE 5. Fine-tuning RoBERTa
  • How to use the RoBERTa model from the transformers library by Hugging Face?
  • How to fine-tune the model to get better results?
40 minutes
ARTICLE 6. A Comparison of various post-BERT
  • How to use the RoBERTa model from the simpletransformers library?
  • How do different post-BERT models compare in terms of performance and training time?
40 minutes

Concepts Covered

0 comment