What is DPO (Direct Preference Optimization)? Why does it exist? Where does it appear in Reinforcement Learning (RLHF)?
Draw or describe a visual model of DPO (Direct Preference Optimization). What does it look like conceptually?
Work through a concrete manual example of DPO (Direct Preference Optimization) with real numbers or pseudocode.
How is DPO (Direct Preference Optimization) different from related concepts in Reinforcement Learning (RLHF)? What are the trade-offs?
Explain DPO (Direct Preference Optimization) in plain language using an analogy. Could a non-expert follow it?