Multilingual / Foundations
verifiedMultilingual Model
One model for many languages, sharing parameters across all of them. Languages help each other where they share structure and compete where they do not, so adding a language improves the low-resource ones and can cost the high-resource ones — a trade that has a name and a measurable shape.
The curse of multilinguality is that per-language quality rises with more languages up to a point and then falls, because a fixed parameter budget is being divided. Raising capacity moves the peak rather than removing it. The practical decision is which languages share a model at all, and it is usually made by data availability rather than by linguistic relatedness, which is why the groupings in released models rarely match anything a linguist would draw.
With a fixed budget and L languages, per-language capacity falls roughly as 1/L while cross-lingual sharing supplies a gain that saturates. The sum peaks at some L and declines, so the optimum is interior — which is why the answer is neither one model per language nor one model for all.
languages-added holds 50% of the budget; rest holds the remaining 50%.
Capacity divided among added languages against the capacity left to any one of them, in equal units. Drag the language count up to watch each share thin — the gain from sharing saturates while this does not.
Reviewed by opendroid · 2026-08-18
- arXiv:1911.02116 — Unsupervised Cross-lingual Representation Learning at Scale
- arXiv:2003.11080 — XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization