Soofi Consortium Releases Soofi S 30B-A3B: An Open Hybrid Mamba-Transformer MoE Foundation Model For German And English
A German research consortium has published the pretraining report for Soofi S 30B-A3B. It is an open base model for German and English. Training ran end to end on Deutsche Telekom’s Industrial AI Cloud in Munich. Preview weights are on Hugging Face. It is worth noting that among some of the fully open base models tested, Soofi S records the highest English and German aggregate scores. What is Soofi S 30B-A3B? Soofi S is a Mixture-of-Experts (MoE) hybrid Mamba Transformer foundation
Get smart on it
A German research consortium has released Soofi S 30B-A3B, an open foundation model designed to work with both German and English languages. The model uses a hybrid architecture combining different sequence-processing techniques and activates only a portion of its total parameters per token, making it more efficient than traditional dense models. Among fully open base models tested using the same evaluation methods, Soofi S achieved the highest combined scores for both English and German, suggesting it performs particularly well on German-language tasks and bilingual applications. The model was trained on a large cluster of GPUs and is available as preview weights on Hugging Face for researchers and developers to download and use.
Related news
NVIDIA AI Releases Nemotron 3 Embed: An Open Embedding Collection Whose 8B Checkpoint Ranks #1 on RTEB
Embedding models decide which passages an agent ever sees. NVIDIA released Nemotron 3 Embed model to work on that layer. It targets production-scale RAG, agentic retrieval, code retrieval, and agent memory. What is Nemotron 3 Embed? The model collection includes three open checkpoints. Nemotron-3-Embed-8B-BF16 is the accuracy-first option. Nemotron-3-Embed-1B-BF16 carries the same design into a smaller footprint. Nemotron-3-Embed-1B-NVFP4 is the Blackwell-optimized 4-bit path. All thre
Moonshot AI Releases Kimi K3: A 2.8 Trillion Parameter Open MoE Model With Kimi Delta Attention and 1M Context
Moonshot AI just released Kimi K3. It is a 2.8-trillion-parameter model with native vision and a 1-million-token context window. Moonshot calls it the world’s first open 3T-class model. What is Kimi K3? Kimi K3 is a sparse Mixture-of-Experts (MoE) model built on two architectural updates. Those are Kimi Delta Attention (KDA) and Attention Residuals (AttnRes). Both change how information flows across sequence length and model depth. K3 targets long-horizon coding, knowledge work, an
