Moonshot AI Releases Kimi K3: A 2.8 Trillion Parameter Open MoE Model With Kimi Delta Attention and 1M Context
Moonshot AI just released Kimi K3. It is a 2.8-trillion-parameter model with native vision and a 1-million-token context window. Moonshot calls it the world’s first open 3T-class model. What is Kimi K3? Kimi K3 is a sparse Mixture-of-Experts (MoE) model built on two architectural updates. Those are Kimi Delta Attention (KDA) and Attention Residuals (AttnRes). Both change how information flows across sequence length and model depth. K3 targets long-horizon coding, knowledge work, an
Get smart on it
Moonshot AI released Kimi K3, an open-source artificial intelligence model with 2.8 trillion parameters designed for tasks like coding, knowledge work, and reasoning. The model uses a sparse architecture that activates only a subset of its total components at any time, along with new attention mechanisms that enable it to process very long documents (up to 1 million tokens) while decoding faster than previous approaches. While K3 leads on several coding and document-handling benchmarks, it still trails the most powerful proprietary models in overall performance. The model is available through Moonshot's API and priced at a flat rate based on token usage, with pricing benefits for cached inputs.
Related news
NVIDIA AI Releases Nemotron 3 Embed: An Open Embedding Collection Whose 8B Checkpoint Ranks #1 on RTEB
Embedding models decide which passages an agent ever sees. NVIDIA released Nemotron 3 Embed model to work on that layer. It targets production-scale RAG, agentic retrieval, code retrieval, and agent memory. What is Nemotron 3 Embed? The model collection includes three open checkpoints. Nemotron-3-Embed-8B-BF16 is the accuracy-first option. Nemotron-3-Embed-1B-BF16 carries the same design into a smaller footprint. Nemotron-3-Embed-1B-NVFP4 is the Blackwell-optimized 4-bit path. All thre
Our approach to bioresilience
Google DeepMind and Isomorphic Labs are sharing our joint approach to bioresilience and AI models.
