HOPE preserves expert partnerships in language model pruning
Hugging Face: Your Experts Have Partners. There's HOPE for Keeping Them Together
As AI systems become more autonomous, organizations face pressure to ensure human experts remain meaningfully involved in critical decisions rather than sidelined by automation.
Get smart on it
Mixture-of-Experts language models use hundreds of experts per layer but only activate a small fraction for any given input, creating significant memory waste during inference. Existing pruning methods remove underperforming experts one at a time, but they fail to account for the fact that experts work in stable cooperative groups and removing one expert can dismantle partnerships with others. HOPE is a new pruning method that considers pairwise interactions between experts when deciding which ones to delete, achieving better performance than existing approaches especially when removing large percentages of experts. The method works by building an interaction matrix from a calibration dataset and solving an optimization problem that minimizes errors while preserving important expert partnerships.
Relevant courses
Want to go deeper than the news? Explore live, cohort-based AI courses taught by practitioners.
Browse AI courses on MavenRelated news
CodeRabbit scales agentic software development and ships 20x more
CodeRabbit uses OpenAI models to govern and trust AI-generated code and Codex to ship 20 times more software with the same engineering team.
Introducing Control Room: Human control over AI workloads
Learn how OpenAI’s Control Room tracks model workloads across our research infrastructure and can rapidly stop them when safety or security concerns arise.
Disney and Adobe are building generative AI tools together
Who wants a hand-designed and creatively curated monorail anyway. | Image: Disney / Adobe The Walt Disney Company has teamed up with Adobe to develop proprietary "artist-led" generative AI models to support Disney's production pipeline. The multiyear partnership was announced today at Adobe's MAX conference, and it will see Adobe providing Disney with early access to its latest technologies for everything from Disney's theme parks and cruise line to ESPN content. Disney and Adob
Apple is racing to make Siri more agentic
Bloomberg's Mark Gurman says CEO John Ternus is pressuring AI czar Craig Federighi to accelerate the Siri roadmap and deliver a powerful AI agent that can compete with OpenAI's Dots or Meta's Muse. While Federighi has delivered a functional version of Siri AI on an incredibly short timeline, the company has no illusions that it's keeping pace with its rivals. Two significant improvements are expected to be made to Siri in the coming months. One will give the assistant fine-grai
World Models Will Soon Be Inescapable. Here’s What You Need to Know
Moving AI from text to 3D space, world models simulate real-world physics to teach robots and autonomous cars how to navigate reality.
Perplexity launches its own take on AI agents for work
Perplexity is taking on OpenAI and Anthropic with its own work-focused AI agents. The startup's new always-on "Teammates" can take on multiple roles within a workspace, while pulling existing projects, skills, and memory from the Perplexity tools you already use. Teammates integrate with Perplexity Computer, an AI system that can deploy agents on your device to complete tasks. But Perplexity says Teammates "build on top" of Computer by offering the ability to collaborate and co
Also reported by The Decoder