OTHERRoBERTa: A Robustly Optimized BERT Pretraining Approach
Star
Covers: theory of Differences between BERT and RoBERTa
Estimated time needed to finish: 30 minutes
Questions this item addresses:
  • What are the differences between BERT and RoBERTa?
How to use this item?

Watch the entire presentation.

URL: https://cs.uwaterloo.ca/~mli/Bin.pptx
Author(s) / creator(s) / reference(s)
Bin Zhan
0 comment
Recipe
publicShare
Star0

Roberta

Contributors
Total time needed: ~4 hours
Objectives
Here, you will be learn about the RoBERTa, the way it differs from BERT, and how to use in your notebook.
Potential Use Cases
To use RoBERTa in your NLP models
Who is This For ?
INTERMEDIATENLP users trying to implement RoBERTa
Click on each of the following annotated items to see details.
ARTICLE 1. A Summary of RoBERTa
  • What is RoBERTa about?
  • Why does RoBERTa matter?
5 minutes
PAPER 2. The RoBERTa Paper
  • What are the problems associated with language models?
  • How does RoBERTa compare with BERT in terms of structure, training, and performance?
60 minutes
OTHER 3. RoBERTa: A Robustly Optimized BERT Pretraining Approach
  • What are the differences between BERT and RoBERTa?
30 minutes
VIDEO 4. RoBERTa: A Robustly Optimized BERT Pretraining Approach
  • What are the differences between BERT and RoBERTa?
20 minutes
ARTICLE 5. Detailed notebook to fine tune RoBERTa (for beginners)
  • How to use the RoBERTa model from the transformers library by Hugging Face?
  • How to fine-tune the model to get better results?
40 minutes
ARTICLE 6. Battle of the Transformers: ELECTRA, BERT, RoBERTa, or XLNet
  • How to use the RoBERTa model from the simple transformers library?
  • How do different post-BERT models compare in terms of performance and training time?
40 minutes

Concepts Covered

0 comment