Predictive uncertainty in the model is responsible for hallucination. Intuitively, the higher the predictive uncertainty is, the more probable some of the probability mass gets assigned to unsuitable tokens. An important decision in using uncertainty-aware beam search is the choice of uncertainty term u(y|x). The authors could use either the aleatoric or epistemic part of the predictive uncertainty or both.
Penalising epistemic uncertainty is better as the authors found a higher correlation between that and hallucination and worked better when empirically tested compared with aleatoric uncertainty or both. Read the introduction and section 5.0—Reducing Hallucination—link here.
Results: UABS works but can be more complex to allow for higher generation quality. Read section 7.0, here.
The proposed UABS reduces hallucination by limiting the total uncertainty of the generated text. As a result, it might lead to shorter generations and lower generation quality. Devising more sophisticated uncertainty aware training and decoding methods with less adverse effects on the generation quality is a future direction to explore. The authors saw that a relatively small penalty weight leads to a reduced hallucination chance (hence more faithful) with a cost on the BLEU score and fluency.
Covers: application of Uncertainty-Aware Beam Search
Estimated time needed to finish: 8 minutes
Questions this item addresses:
What is Uncertainty-Aware Beam Search (UABS)?
How does UABS work?
How well does UABS work compared to regular model?