Covers: theory of Transformer
0
Questions this item addresses:
  • How longformer is implemented?
How to use this item?

Main longformer code repo.

Author(s) / creator(s) / reference(s)
refer to the link
Programming Languages: Python
0 comment
Recipe
publicShare
Star(0)

Understand The Paper : Learning Longformer:the Long-document Transformer

Contributors
Total time needed: ~2 hours
Objectives
With this list you can learn about Longformer and how to implement it.
Potential Use Cases
Long text summarization, Long text question answering.
Who is This For ?
ADVANCEDNLP Data scientist from all audience levels
Click on each of the following annotated items to see details.
Resources5/7
ARTICLE 1. Introduction to Transformer Encoder Decoder Model
  • What is transformer and how it works?
10 minutes
ARTICLE 2. Attentions Mechanism In Neural Machine Translation
  • What is attention and type of attention mechanisms?
20 minutes
PAPER 3. Longformer : The Long-Document Transformer (Original Paper)
  • What is long former and attention mechanism to process long sequence?
20 minutes
ARTICLE 4. Understanding Transformer-Based Self-Supervised Architectures - LongFormer
  • How can we reduce the computational cost of the attention calculations, which grow quadratically with sequence length?
7 minutes
REPO 5. Longformer : The Long-Document Transformer Github Repo
  • How longformer is implemented?
10 minutes
ARTICLE 6. Train a Longformer for Detecting Hyper-partisan News
  • How to train a longformer?
7 minutes
ARTICLE 7. Train a Longformer for the Question Answering
  • How to implement longformer for question answering task?
7 minutes

Concepts Covered

0 comment