Article

Scalable and Interpretable Mixture of Experts Models in Machine Learning: Foundations, Applications, and Challenges

2025-07-03

Abstract excerpt

Mixture of Experts (MoE) models have emerged as a powerful framework in machine learning, combining multiple specialized expert networks through a gating mechanism to enable scalable, efficient, and adaptive computation. This survey provides a comprehensive and mathematically rigorous overview of efficient and explainable MoE architectures, encompassing their theoretical foundations, optimization properties, and g...

Topics

Open a Topic to create a Post that cites this publication.

Identifiers and source

Literature Corpus work: af2b0002-7625-5b0d-a5c8-3a8532cd6547
DOI: 10.20944/preprints202507.0283.v1

Open publication

Related research

Semantic proximity does not establish scientific evidence.

Click a neighbor to travelStep 1 · 12 closest

Scalable and Interpretable Mixture of Experts Models in Machine Learning: Foundations, Applications, and ChallengesDOI 10.20944/preprints202507.0283.v1Read

Achieving Explainable, Scalable, and Robust Machine Learning for Real-World ApplicationsSemantic proximity Read

Mechanistic Interpretability for Large Language Model Alignment: Progress, Challenges, and Future DirectionsSemantic proximity Read

Integrating Multimodal Data with Large Foundation Models in HealthcareSemantic proximity Read

Learning Representations by Humans, for HumansSemantic proximity Read

Exploring Explainability in Large Language ModelsSemantic proximity Read

A Comprehensive Review of DeepSeek: Performance, Architecture and CapabilitiesSemantic proximity Read

BOPA-Fast: Bipolar Outer Product Attention for Efficient and Interpretable Multimodal FusionSemantic proximity Read

Self-Supervised Learning Principles Challenges and Emerging DirectionsSemantic proximity Read

Sparse Autoencoders as an Interpretable Interface for LLM Development and ControlSemantic proximity Read

Domain-adapted language model using reinforcement learning for various dementiasSemantic proximity Read

Why Compression Creates Intelligence: The Architecture of Experience in Large ModelsSemantic proximity Read

Mixture of Experts Enable Efficient and Effective Protein Understanding and DesignSemantic proximity Read