← Sunday, October 4, 2026Models & trainingRelease

Aleph Alpha releases Kolibri, a 78B-parameter MoE with 1M-token context

aleph-alpha.com 24 min read Editor score 7/10

Kolibri is an English-German mixture-of-experts model with 78B total parameters, 3B active, a 1M-token context window, and Apache 2.0 weights. Its training pipeline automated recovery from hardware failures and dropped data connections, a useful production lesson for model training.

What

Kolibri is an English-German mixture-of-experts language model with 78 billion total parameters, 3 billion active, and context lengths up to 1 million tokens.

How

Its development pipeline spans data ingestion and curation, ablations, pre-training, post-training, and final evaluations; it also handled hardware or connection failures without human intervention.

Why it matters

The pipeline supports repeatable model iteration, while the open weights and on-premise focus target regulated and mission-critical use.

Read on aleph-alpha.com → Hacker News · 311 comments

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.