TokenGate speeds up AI language model processing by up to 11.3x
MarkTechPost: Researchers from Stanford and UC Berkeley Introduce TokenGate: Token-Gated Attention Makes Long-Context Prefill Up to 11.3x Faster
Stanford and UC Berkeley researchers released TokenGate, a module for long-context prefill. TokenGate learned token pruning lets an existing LLM skim its input. Only the tokens that matter get full processing. The research team reports 1.4x to 11.3x faster time-to-first-token (TTFT) with no accuracy loss. Peak prefill memory drops 1.5x to 3.4x. Code and checkpoints are on GitHub under Apache-2.0. TL;DR MLP gate: A small MLP gate and LoRA (r=16) added to Qwen3-1.7B or Qwen3.5-2B. Tested
Get smart on it
TokenGate is a module that speeds up how AI language models process long documents by learning which tokens, or word pieces, are important and need full processing versus which can be skipped. Normally, transformers process every token through every layer, which becomes increasingly slow as documents get longer. TokenGate adds a small trainable gate that scores tokens and keeps only the most relevant ones for deeper processing, achieving speed improvements of 1.4x to 11.3x faster while using 1.5x to 3.4x less memory without losing accuracy. The approach uses a technique borrowed from mixture-of-experts models to make the token selection trainable end-to-end, and it works on both standard attention models and hybrid models that mix different attention types.
Relevant courses
Want to go deeper than the news? Explore live, cohort-based AI courses taught by practitioners.
Browse AI courses on MavenRelated news
Moonshot AI Releases Kimi K3.1 Pro: A 2.8T Total/104B Active Params (MoE) Model with 1M Context
Moonshot AI has just released Kimi K3.1 Pro, the first model in its K3.1 series. Kimi K3.1 Pro is a 3T-class open-weight* model with a 1-million-token context window. Moonshot team positions it against the strongest proprietary models on coding, agentic and knowledge benchmarks. For developers, it is cheaper API pricing than K3 and open weights yet not available. *open weights are expected to release in November 2026. TL;DR Size: 2.8T total, 104B active parameters (MoE), 1M-token context
Also reported by SiliconANGLE and The Decoder
Jack Dorsey’s Block made an AI model that talks to you like an awkward new friend
Jack Dorsey's Block announced an AI model called Dessa that's trained "from scratch on nothing but human conversation," and to show off the model, Block shared chats it has had with humans that can occasionally be a little awkward and uncanny. Take this one, for example, shared by Dessa lead Ragavan Thurairatnam. The conversation between the model and the speaker does have a mostly natural back and forth (though for some reason, the model initially mistakes the speaker for some
Also reported by TechCrunch and The Decoder
Canva releases its first image generation model
Canva is making its new image model available to all users, promising better, more customizable performance at lower costs.
Also reported by The Verge and The Decoder
Recursive launches Flint, a laptop coding model it says was largely built by AI
Startup Recursive has released Flint, a free, laptop-sized coding model built largely by its own AI research system, as an early test of AI that improves itself.
Also reported by Wired AI and MarkTechPost
Ai2 releases OlmoEarth v1.3 state-of-the-art AI-ready satellite embeddings for the whole planet
Ai2, the Allen Institute for Artificial Intelligence, today released a satellite map embedding model called OlmoEarth v1.3 and precomputed annual embeddings for the whole planet at various resolutions with a browser-based map-making tool. The model compresses a year of satellite observations into a numerical fingerprint for each 10-by-10-meter patch of ground at the smallest resolution, […] The post Ai2 releases OlmoEarth v1.3 state-of-the-art AI-ready satellite embeddings for the whole pl
Also reported by Hugging Face
NVIDIA Releases Nemotron 3.5 Bolt E4B 4GB: An Open Small Language Model for Local Voice Agents, Game Characters and Robots
NVIDIA has just released Nemotron 3.5 Bolt E4B 4GB, an open small language model for fast conversational agents. It targets voice assistants, game NPCs and robot interfaces running locally on RTX, Jetson and Spark hardware. The model is post-trained from Google’s Gemma 4 E4B using reinforcement learning only. For developers, this is agent behavior in about 4GB of VRAM. That leaves room for speech and vision models. TL;DR Size: 8B total parameters, 4B effective, 64K context. Runs