1. Predictive uncertainty in the model is responsible for hallucination. Intuitively, the higher the predictive uncertainty is, the more probable some of the probability mass gets assigned to unsuitable tokens. An important decision in using uncertainty-aware beam search is the choice of uncertainty term u(y|x). The authors could use either the aleatoric or epistemic part of the predictive uncertainty or both.
  2. Penalising epistemic uncertainty is better as the authors found a higher correlation between that and hallucination and worked better when empirically tested compared with aleatoric uncertainty or both. Read the introduction and section 5.0—Reducing Hallucination—link here.
  3. Results: UABS works but can be more complex to allow for higher generation quality. Read section 7.0, here.
  4. The proposed UABS reduces hallucination by limiting the total uncertainty of the generated text. As a result, it might lead to shorter generations and lower generation quality. Devising more sophisticated uncertainty aware training and decoding methods with less adverse effects on the generation quality is a future direction to explore. The authors saw that a relatively small penalty weight leads to a reduced hallucination chance (hence more faithful) with a cost on the BLEU score and fluency.
Covers: application of Uncertainty-Aware Beam Search
Estimated time needed to finish: 8 minutes
Questions this item addresses:
  • What is Uncertainty-Aware Beam Search (UABS)?
  • How does UABS work?
  • How well does UABS work compared to regular model?
0 comment
Recipe
publicShare
Star(0)

Understanding the paper: On Hallucination and Predictive Uncertainty in Conditional Language Generation

Contributors
Total time needed: ~2 hours
Objectives
Understanding the On Hallucination and Predictive Uncertainty in Conditional Language Generation paper.
Potential Use Cases
Neural natural language generation: image captioning, data-to-text generation, abstractive summarization, and neural machine translation.
Who is This For ?
INTERMEDIATENatural Language Processing (NLP) developers looking to better understand and correct for hallucination in a variety of Natural Language Generation tasks.
Click on each of the following annotated items to see details.
Resource Asset5/10
ARTICLE 1. Hallucination in Neural NLG
  • What is hallucination in Neural NLG?
  • What are some examples of hallucination in Neural NLG?
  • Why is hallucination unacceptable in many NLG applications?
5 minutes
PAPER 2. Hallucination in Image Captioning - Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4035–4045
  • How do hallucinations occur during image captioning?
  • Why do hallucinations occur during image captioning?
12 minutes
PAPER 3. Challenges in Data-to-Document Generation
  • Why do hallucinations occur during data-to-text generation?
  • How do hallucinations occur during data-to-text generation?
4 minutes
PAPER 4. Hallucination in Abstractive Summarization - FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization
  • Why do hallucinations occur during abstractive summarization?
  • How do hallucinations occur during abstractive summarization?
15 minutes
PAPER 5. Domain Robustness in Neural Machine Translation
  • Why do hallucinations occur during Neural machine translation (NMT)?
  • Why do hallucinations occur during Neural machine translation (NMT)?
6 minutes
PAPER 6. Aleatoric and epistemic uncertainty in machine learning: an introduction to concepts and methods
  • What is aleatoric uncertainty?
  • What is epistemic uncertainty?
  • How are aleatoric uncertainty and epistemic uncertainty different?
10 minutes
WRITEUP 7. Therefore, How Hallucination Occurs
  • So, overall, how does hallucination in NLG occur?
2 minutes
WRITEUP 8. Uncertainty-Aware Beam Search (UABS) from the Paper "On Hallucination and Predictive Uncertainty in Conditional Language Generation
  • What is Uncertainty-Aware Beam Search (UABS)?
  • How does UABS work?
  • How well does UABS work compared to regular model?
8 minutes
PAPER 9. Further Research: Deep and Confident Prediction for Time Series at Uber
  • How does uncertainty apply to other domains?
6 minutes
PAPER 10. Quantifying Uncertainties in Natural Language Processing Tasks
  • How does uncertainty apply to other domains within NLP?
5 minutes

Concepts Covered

0 comment