Meet Nemotron Labs 3 Puzzle 75B A9B: A Compressed Hybrid MoE LLM Delivering 2.03x Server Throughput

A compressed hybrid MoE language model has been released that reduces the total and active parameters of its parent model while preserving the same 88-block architecture. The compression matters because it enables significantly higher server throughput, with gains ranging from 1.60x to 2.14x on multi-GPU nodes and allowing a single GPU to handle 8 concurrent large-context requests instead of 1. The model was created using an iterative neural architecture search framework that selectively pruned different layer types at varying depths rather than uniformly scaling down the original, then recovered quality through knowledge distillation across multiple stages.