Covers: theory of Attention Mechanism
Estimated time needed to finish: 30 minutes
Questions this item addresses:
  • How do Transformers differ from CNNs and RNNs?
How to use this item?

The dominant sequence transduction models are based on complex recurrent or convolutional neural networks that include an encoder and a decoder. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.

Author(s) / creator(s) / reference(s)
Vaswani et al.
0 comment
Recipe
publicShare
Star(0)

Understanding the Paper: Progressive Transformer-Based Generation of Radiology Reports

Contributors
Total time needed: ~3 hours
Objectives
Learn what goes into building a multi-stage transformer model by exploring three fundamental topics: Feature Extraction, Attention Mechanism and Visual Language Models.
Potential Use Cases
Clinical Data Mining, Automated & Reproducible Medical Diagnosis, Coherent Report Generation from Images
Who is This For ?
INTERMEDIATENLP or Computer Vision Specialists or Enthusiasts
Click on each of the following annotated items to see details.
Resource Asset7/10
WRITEUP 1. Introduction: Supporting Concepts to Understand Transformer-Based Report Generation
  • Why does the recipe for "Understanding the Paper: Progressive Transformer-Based Generation of Radiology Reports" have the three supporting concept assets sorted in this order?
3 minutes
PAPER 2. An Overview of Image Caption Generation Methods
  • What are the main types of feature extraction methods for images?
  • How may transformers be leveraged to extract features from images?
30 minutes
PAPER 3. When Radiology Report Generation Meets Knowledge Graph
  • How did Noorallahzadeh et al., the authors of our paper of interest "Progressive Transformer-Based Generation of Radiology Reports", construct the training datasets through the usage of MIRQI?
15 minutes
PAPER 4. Attention Is All You Need
  • How do Transformers differ from CNNs and RNNs?
30 minutes
ARTICLE 5. Transformers in Computer Vision: Farewell Convolutions!
  • How can transformers with attention mechanism overcome limitations in convolutional models?
14 minutes
ARTICLE 6. An Overview of ResNet and its Variants
  • What is needed to construct a visual backbone?
15 minutes
PAPER 7. Visual Language Model Content: Generating Radiology Reports via Memory-driven Transformer
  • How was the original idea for the visual language modeling described?
10 minutes
ARTICLE 8. Revealing BART : A denoising objective for pretraining
  • How does BART contribute to the fine-tuning of the Language Model within Noorallahzadeh et al.'s framework?
6 minutes
PAPER 9. Natural Language Generation Content: Generating Radiology Reports via Memory-driven Transformer
  • How were recent radiology report generation conducted using memory-driven transformers?
10 minutes
PAPER 10. Progressive Transformer-Based Generation of Radiology Reports
  • How may image-to-text-to-text be leveraged to generate radiology reports?
25 minutes

Concepts Covered

0 comment