Needle 2 brings tool calling to 14MB devices
Cactus Compute has released Needle 2, an open 45M-parameter model designed for tool calling, device control and structured data extraction. The complete model is distributed as one 14MB binary and uses about 28MB of RAM for a full session. Its weights use CQ2-bit quantization, while the company’s C++ engine removes the need to install a separate runtime or download data during inference.
Cactus reports decoding speeds of 500 tokens per second on Raspberry Pi 5, 400–1,500 on Meta Quest 3S and Apple Vision Pro, and 300–700 on phones costing less than $200. Prebuilt binaries and a static library support macOS, Linux, Windows, Android, Apple mobile and wearable platforms, and WebAssembly.
Needle 2 uses a 27-layer, 512-wide Simple Attention Network with GQA, a Hadamard MLP, hashed n-gram memory and multi-lane hyper-connections. Its grammar restricts output to valid JSON schemas, while retrieval limits larger tool collections to five candidates. On Cactus’s tests, the model led both Seal-Tools splits and achieved 98.3% function-name accuracy on Mobile Actions, but scored below larger models on BFCL v4.
Written by Sakhanda Wire AI from reporting by MarkTechPost. Facts, quotes and full context are in the original article below.