Perplexity Open Sources Lily: A Rust + Metal Inference Engine for Qwen3.6-35B-A3B on Apple Silicon
Perplexity has open sourced Lily, the local inference engine behind Hybrid Compute in Perplexity Computer. It is a single-process runtime: a Rust layer loads the checkpoint and drives the generation loop, an OpenAI-compatible chat-completions API streams tokens, and hand-written Metal kernels execute the model. Neither PyTorch nor MLX sits in the execution path. Lily is deliberately narrow with one model, Qwen3.6-35B-A3B, on one hardware family and that narrowness is the performance argument.
Get smart on it
Perplexity has released Lily, a specialized software engine written in Rust and Metal that runs a specific AI model on Apple computers without using standard frameworks like PyTorch or MLX. The engine is designed narrowly for one model and one hardware family, which allows it to execute faster than existing general-purpose solutions, achieving roughly 1.2 to 1.35 times better performance in testing. This matters because it demonstrates how custom-built inference engines tailored to specific constraints can outperform flexible, reusable frameworks by optimizing every layer from weight loading through token generation. The code is publicly available, though the hardware requirements are substantial: an Apple silicon Mac with at least 32 GB of unified memory is needed to run the model.
Related news
Google DeepMind’s WeatherNext 3 Trains on Weather Station Observations to Deliver 5 km Global Forecasts, Refreshed Every Hour
AI weather models have spent three years closing the gap with physics-based forecasting, but two problems stayed open: resolution too coarse for local terrain, and initialization tied to numerical weather prediction (NWP) analysis that arrives about six hours late. WeatherNext 3, released by Google DeepMind and Google Research, attacks both. It takes a live global geostationary satellite mosaic as a direct model input, re-initializes every hour, and emits forecasts down to 0.05° (~5 km) while t
OpenAI Releases GPT-6 Astra: A 1.05M-Context Computer-Use Model Gated Behind a ‘Critical’ Cyber Threshold
Today, OpenAI released GPT-6 Astra. The company calls it its most intelligent and aligned model, and positions it primarily as a computer-use system rather than a chat model. The pitch is that Astra operates software the way a person does, across browsers, spreadsheets, desktop applications and terminals, and finishes multi-step jobs instead of describing how to do them. Is it deployable? Partly, and not on your own hardware. Astra is a closed, hosted model with no released weights, so self
