← Saturday, October 3, 2026Models & trainingPaper

Omni-Embed-Mini adds modalities without changing text embedding weights

huggingface.co by Mohammed Irfan Kurpath et al. Editor score 8/10

The 0.9B-parameter model aligns text, speech, audio, images, video and documents while keeping text weights bit-identical; it reports 49.57 nDCG@10 on MTEB-v2 BEIR-8. The distillation recipe offers a practical route to multimodal retrieval without regressing the text encoder through training.

What: Omni-Embed-Mini maps text, speech, audio, images, video, and visually rich documents into one shared cosine space.
Method: It freezes text-side weights and aligns modality encoders with lightweight projectors and phased LoRA adapters, using frozen-backbone caption embeddings as teacher targets.
Why it matters: Bit-identical text weights prevent training from regressing text retrieval while extending the model to five additional modalities.

Caption embeddings provide teacher targets as modality encoders are aligned into a shared space. Sketched from the article's own text.

Read on huggingface.co → Code · 3 stars

Discussion

Discussion is not switched on yet. Use Note above to keep your own thoughts on this story.

Headline, summary, brief and diagram are written by a model from the source's own text; the link, byline and numbers come from the source. Read the original before relying on a claim.