07 · Make Models Efficient

Distillation

Training small student models to mimic large teacher models

🟡 GOOD

🧠 Understand👁️ Visualize⚖️ Compare🗣️ Explain

7 subtopics — click to study

Why Distillation? 🔴 Teacher Model 🔴 Student Model 🔴 Knowledge Transfer 🔴 Soft Targets 🟡 Training Student Model 🟡 Student vs Teacher Behavior 🟡

← Back to Roadmap

This page 0:00

This session 0:00

Total (all pages) 0:00

Today 0:00

▶ Running

📊 View all study stats →

0:00 studying