Multilingual / Methods
verifiedCross-Lingual Transfer
Fine-tune a multilingual model on a task in English and it often performs that task in languages it never saw labelled examples for. The representation has aligned the languages well enough that the task head does not care which one the input is in. Nobody engineered that alignment; it emerges from joint training on unaligned text.
Transfer is strongest between languages sharing script and word order and weakest across both, which makes the usual English-to-everything evaluation flattering to related languages and misleading about the rest. Reporting an average over a benchmark's languages hides that entirely — the distribution over languages is the result, and the mean is not.
Zero-shot transfer is measured as task performance in a target language after fine-tuning only in a source. The gap to in-language fine-tuning is the transfer cost, and it tracks typological distance more closely than it tracks the target language's pretraining share — which is why adding pretraining data narrows it by less than the volume suggests it should.
transferred holds 50% of the budget; rest holds the remaining 50%.
Task ability that arrives without in-language labels against the ability that still needs them, in equal units. Drag the transfer up to watch the labelled requirement shrink — it shrinks furthest for languages closest to the source.
Reviewed by opendroid · 2026-08-18
- arXiv:1906.01502 — How multilingual is Multilingual BERT?
- arXiv:2003.11080 — XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization