Meet Needle 2: An Open 45M-Parameter Tool-Calling Model That Ships as a 14MB Binary and Runs a Full Session in 28MB of RAM
Cactus Compute has released Needle 2, an open 45M-parameter model for tool calling, device use, and structured extraction. The entire model ships as a single 14MB binary that runs a full session in about 28MB of RAM. Weights are trained and deployed at CQ2-bit using Cactus Quants, and the model is sealed inside the company’s own C++ engine, so there is no runtime to install and no download at inference time. Reported decode throughput is 500 tokens/sec on a Raspberry Pi 5, 400–1,500 token
Get smart on it
Needle 2 is a small artificial intelligence model with 45 million parameters that runs on devices with limited computing power, such as phones and smartwatches, without requiring internet connection. The model specializes in interpreting spoken or written commands and converting them into structured function calls, which is useful for offline voice control and device automation. It matters because it can operate entirely on local hardware in just 28 megabytes of RAM while shipped as a 14-megabyte binary, making it deployable across smartphones, wearables, smart home devices, and other constrained hardware without cloud dependency. The model uses a specialized neural network architecture with a 256-token sliding window and includes built-in confidence scoring to determine when to escalate tasks to cloud systems, making it suitable for industries like smart home, automotive control, and retail applications where processing must happen locally.
Related news
Fine-Tuning Tool-Calling LLMs: A Complete Guide Using XYZ-Aquila-SFT and Qwen3
In this tutorial, we implement an end-to-end supervised fine-tuning pipeline for the XYZ-Aquila-SFT dataset, Hugging Face Transformers, PyTorch, and PEFT. We stream and inspect the dataset, parse multi-turn tool-use trajectories, extract structured tool calls, analyze corpus characteristics, and preserve embedded reasoning and observation patterns. We then convert tool schemas between message-embedded and structured formats, render Qwen-compatible ChatML with assistant-only loss masking, prepar
Meta’s ‘open’ AI, and a $250M deal gone very wrong
Meta released Glimmer this week, an open-weight AI model anyone can download and run on their own hardware — a contrast to Muse Spark, the company’s more powerful model that stays locked behind its own APIs. The release landed alongside a letter from Mark Zuckerberg arguing AI should be “for everyone” rather than controlled by a handful of labs, but as Equity’s […]
