PAPERDataset Distillation

Covers: theory of Dataset Distillation
Estimated time needed: 18 minutes
Questions this item adddesses:
  • Is it possible to train a model on synthetic data out of the manifolds of the original data?
  • How much data is encoded in a given training set and how compressible it is?
How to use this item?

Read the introduction and approach

URL: https://arxiv.org/abs/1811.10959
Author(s) / creator(s) / reference(s)
Tongzhou Wang, Jun-Yan Zhu, Antonio Torralba, Alexei A. Efros
Shortlist
publicShare

Learning N Classes From M<N Samples

Nour FahmyTotal time needed: ~28 minutes
Learning Objectives
Learn about developments in deep learning research for few-shot learning
Potential Use Cases
In the scenario you too have less than optimal sample sizes!
Target Audience
ADVANCEDIf you're already familiar and have worked with NNs in the past, and looking to navigate suboptimal sample sizes
Go through the following annotated items in order:
PAPER 1. Dataset Distillation
  • Is it possible to train a model on synthetic data out of the manifolds of the original data?
  • How much data is encoded in a given training set and how compressible it is?
18 minutes
PAPER 2. Soft-Label Dataset Distillation and Text Dataset Distillation
  • How can we train a model using data distillation and relaxing the hard labels to a probability distribution?
10 minutes
PAPER 3. ‘Less Than One’-Shot Learning: Learning N Classes From M<N Samples
  • How can we learn from a dataset that has less classes than what we're hoping to classify?
10 minutes

Concepts Convered