Bringing Nunchaku 4-bit Diffusion Inference to Diffusers
Making image generation models faster and cheaper to run could expand access to AI art tools beyond well-funded organizations.
Get smart on it
Large diffusion models that generate images require significant computer memory, typically 20-30 GB of VRAM, which limits access to most consumer GPUs. SVDQuant is a quantization method that reduces both memory usage and inference time by running transformer layers with 4-bit weights and activations, compared to standard quantization approaches that only reduce memory. Nunchaku Lite integrates this quantization method directly into Diffusers, allowing users to load quantized models with a simple command and no separate inference engine or local compilation required. This integration provides approximately 30 percent speedup while maintaining the same memory reduction as the original Nunchaku engine.
Related news
FAIRChem v2 UMA for Multidomain Atomistic Simulation across Molecules, Catalysts, Materials, Vibrations, and Molecular Dynamics
In this tutorial, we explore FAIRChem v2 and the UMA universal machine-learning interatomic potential as a unified framework for atomistic simulation across molecular chemistry, catalysis, and inorganic materials. We configure an environment, authenticate with Hugging Face to access the gated UMA model weights, and initialize task-specific calculators for the omol, oc20, and omat domains. We then apply the same pretrained potential to a broad set of computational chemistry workflows, including
Induction Labs Photon-1 Simulates Desktops, Plays Checkers, and Models Billiard Physics From One Pretraining Run
Most agents that learn from video need to know what action produced each frame. Induction Labs is arguing that this requirement is the bottleneck. Last week, they released imagination models, a foundation model architecture that pretrains on raw video with no action labels at all. Their test system is Photon-1, a sparse 106B-A5B mixture-of-experts (MoE) transformer trained on 18 years of computer demonstration video. On an internal computer use benchmark, Induction Labs reports that Photon-1
