Recommendation Systems: Collaborative & Content-Based Filtering

Recommendation systems are among the most financially lucrative applications of machine learning in production, directly driving over 35% of Amazon purchases, 70% of YouTube watch-time, and 80% of Netflix streams. Modern recommenders combine Collaborative Filtering and Content-Based Filtering.

Figure 37.1: Matrix Factorization Recommender Systems vs Time-Series Decomposition Pipelines.

1. Collaborative Filtering: The Wisdom of Shared Behavior

Collaborative filtering relies purely on historical user interaction data (ratings, clicks, purchases) without needing to understand what items actually contain:

• User-Based Collaborative Filtering: "Find users similar to Alice based on past movie ratings, and recommend movies they loved that Alice hasn't seen yet."
• Item-Based Collaborative Filtering: "Find items that are frequently rated similarly by the same users. If Bob liked Inception, recommend Interstellar because fans of Inception consistently rate Interstellar highly."

2. Matrix Factorization via SVD (Singular Value Decomposition)

In production, the User-Item interaction matrix R of size m × n is over 99% sparse (most users have only rated a tiny fraction of products). Matrix Factorization decomposes R into two low-rank dense matrices representing latent feature preferences:

R ≈ P · Qᵖ

• P (User Matrix): Size m × k. Each row represents a user's affinity for k latent dimensions (e.g., preference for Sci-Fi, Action, Romance).
• Q (Item Matrix): Size n × k. Each row represents how strongly an item exhibits those same k latent dimensions.
• Predicted rating for User u on Item i: r̂_{u,i} = p_u · q_iᵖ.

3. Content-Based Filtering & The Cold-Start Problem

The Achilles' heel of collaborative filtering is the Cold-Start Problem: when a brand-new user registers or a new product is added, there are zero interaction records, rendering collaborative matrix factorization useless!

Content-Based Filtering solves cold-start by building feature profiles from metadata (product categories, brand, actors, text embeddings). Hybrid recommendation systems use content metadata for new items, and transition to collaborative filtering as behavioral interactions accumulate.