What is MoE vs Dense Models? Why does it exist? Where does it appear in Mixture of Experts (MoE)?
Draw or describe a visual model of MoE vs Dense Models. What does it look like conceptually?
How is MoE vs Dense Models different from related concepts in Mixture of Experts (MoE)? What are the trade-offs?
Explain MoE vs Dense Models in plain language using an analogy. Could a non-expert follow it?