首页 > AI前沿 > Do Mixture-of-Experts Routers Benefit from Ensembling?

Do Mixture-of-Experts Routers Benefit from Ensembling?

arXiv机器学习 2026-11-13 03:58 10 阅读 查看原文

Ensembles of neural networks have repeatedly been shown to outperform single networks. Ensemble theory attributes this gain to the accuracy of the individual members and the diversity among them. Mixture-of-experts models already combine many experts, but a single router decides how the experts are weighted for each input. Motivated by ensemble theory, we apply ensembling to the routing step: an ensemble of routers, trained jointly with the experts, proposes the expert weights. We systematically evaluate this approach using underparameterized networks on two image-classification and two tabular-regression tasks. We analyse each experiment with a performance-diversity decomposition