๐ก๐๐๐๐๐
publicShareStarEvaluating Recommender Systems
Collaborators
Reviewers
Total time needed: ~3 hours- Learning Objectives
- This list will cover error-based methods, rank-based methods, and other metrics for evaluating recommender systems.
- Potential Use Cases
- Assess performance of product recommendation, friend recommendation, movie/video recommendation, etc.
- Target Audience
- BEGINNER
Go through the following annotated items in order:
ARTICLE 1. List of Methods to evaluate RecSys
- What is Mean Absolute Error?
- What are MSE and RMSE?
- What is Precision and Recall?
- What is ROC curve?
- What is nDCG?
- What are Coverage, Popularity, and Novelty?
30 minutes
ARTICLE 2. List of Evaluation Methods
- What is MAE?
- What is HIT rate?
- What is Coverage, Diversity, and Novelty?
15 minutes
ARTICLE 3. Root Means Squared Error and Mean Absolute Eerror
- What is RMSE?
- What is MAE?
- Which method (RMSE or MAE) is better to use in what context?
5 minutes
ARTICLE 4. Mean Average Precision
- What is Precision and Recall at Cutoff-K?
- What is Average Precision?
- What are some variants of Average Precision formula?
20 minutes
ARTICLE 5. Normalized Discounted Cumulative Gain (nDCG)
- What is cumulative gain?
- What is normalized discounted CG?
- What are some limitations of nDCG?
10 minutes
ARTICLE 6. Comparison of Rank-based Evaluation Methods
- What is MRR?
- What is MAP?
- What is nDCG?
- How do these different methods compare?
30 minutes
PAPER 7. Beyond Accuracy: Coverage and Serendipity in RecSys
- What are different ways to calculate coverage (e.g. precision coverage)?
- What is serendipity in recommender systems?
30 minutes
Concepts Covered