Improving Word Embeddings with Retrofit

Total time needed: ~2 hours
Learning Objectives
Learn about how word embeddings can be improved by using knowledge base, a technique called retrofitting
Potential Use Cases
Improve the domain-specific word vectors you use by incorporating semantics.
Target Audience
INTERMEDIATENLP Developers who understand Word Embeddings and want to improve them in their own use cases.
Go through the following annotated items in order:
REPO 1. Fun exercises to recap Word Vectors
  • How does Word Vectors work?
  • How to build your own Word Vectors?
  • How to use pre-trained Word Vectors?
40 minutes
BOOK_CHAPTER 2. Embeddings in Natural Language Processing
  • What are knowledge base?
10 minutes
PAPER 3. The Retrofitting Paper
  • What is retrofitting?
  • How does retrofitting work?
30 minutes
ARTICLE 4. A short guide on how to implement Retrofitting with Pre-trained Word Vectors and Paraphrase database
  • How retrofitting is done?
15 minutes

Concepts Covered