Quantization-Aware Healing: a compressed, 4-bit model that outperforms its full-precision original

Making AI models smaller usually reduces their performance. The standard approach compresses the model architecture first, then converts its weights to lower precision (4-bit), then applies a recovery step called healing to restore lost capabilities. A new method called Quantization-Aware Healing (QAH) improves this process by distilling the compressed, quantized model directly from the original full-size model rather than from the already-degraded recovered version. This approach produced a 4-bit model that outperformed its full-precision equivalent on most tested benchmarks, making it smaller and more accurate than the version it came from.