EMO: Pretraining mixture of experts for emergent modularity
- Allen Institute for AI: 2 events in the last 90 days
What happened
EMO is a new mixture-of-experts model trained so modular expert groups emerge from data, enabling users to select small task-specific expert subsets while preserving near full-model performance.
Summary assembled by rule from the sources below
Why it's spreading
Sources
Official
- EMO: Pretraining mixture of experts for emergent modularityAllen Institute for AI