Buoyant Releases Kinference: An Open Source Inference Control Plane for Kubernetes and vLLM
Buoyant, the company behind Linkerd, has released Kinference, an open source project for building inference platforms on Kubernetes. Kinference is an inference control plane for Kubernetes and vLLM. It sits in front of a fleet of vLLM workers and decides where each request goes. It applies routing policies, token rate limits, KV-cache-aware load balancing and per-tenant metrics in one layer. For dev teams serving many open-weight models to many internal clients, it targets the layer above the m
Get smart on it
Kinference is an open source control plane that manages how requests are routed to language model servers running on Kubernetes clusters with vLLM. It sits between incoming requests and a fleet of model servers, applying routing policies, rate limits, and load balancing based on factors like cache overlap and queue length. The tool is significant because benchmarks show it can reduce response time by 95 percent on requests with shared prefixes compared to standard Kubernetes routing, though these results are self-reported and the project is still in alpha. It targets development teams that need to serve multiple open-weight models to many internal clients while controlling costs and managing different tenant requirements through declarative policy rules.
Relevant courses
Want to go deeper than the news? Explore live, cohort-based AI courses taught by practitioners.
Browse AI courses on MavenRelated news
Perplexity launches its own take on AI agents for work
Perplexity is taking on OpenAI and Anthropic with its own work-focused AI agents. The startup's new always-on "Teammates" can take on multiple roles within a workspace, while pulling existing projects, skills, and memory from the Perplexity tools you already use. Teammates integrate with Perplexity Computer, an AI system that can deploy agents on your device to complete tasks. But Perplexity says Teammates "build on top" of Computer by offering the ability to collaborate and co
Also reported by The Decoder
Airbnb says it’s launching an AI travel agent in the first half of 2027
Airbnb says its upcoming AI agent will help travelers plan trips, compare options, manage bookings, and get assistance throughout their stay.
Also reported by The Verge
Ramp data shows AI spending stalling just as Anthropic nears its IPO
AI usage keeps climbing, but spending has stalled. Price cuts and smarter cost controls among the biggest spenders could challenge investor assumptions about AI hypergrowth.
Uber only fulfilled 4 rides from AI agents last quarter
AI agents won't be stealing Uber's lunch any time soon. Only four rides were arranged by third-party AI agents during the third quarter of 2026, Uber executives disclosed during an earnings call this morning. The volume of Uber Eats orders fulfilled by AI agents was a little higher, with the company recording "around 500" during the week of Halloween, out of a total 320 million Uber rides and Uber Eats deliveries. This is the first time that Uber has disclosed the number of t
Also reported by TechCrunch
Muse has a voice mode now
Meta now lets you have voice conversations with your Muse AI agent in real time, CEO Mark Zuckerberg announced today. The feature has been rolled out to me, and I gave it a spin on the iOS app while writing this article. As you chat with your Muse agent, the avatar at the top of your conversation animates in time with the words that it's saying. You can also change the tone of the voice by just asking your agent about what you want and letting it decide or by browsing through a
Meta says US users can now video chat with their Muse AI avatar
Meta's AI agent can now take calls as a customizable animated avatar, ahead of the launch of a Tamagotchi-like wearable designed to bring Muse to life.