Unsloth vs Axolotl vs TRL vs LLaMA-Factory: A Fine-Tuning Framework Comparison on Speed, VRAM, and Multi-GPU

Four open source projects dominate the fine-tuning of large language models: Unsloth, Axolotl, TRL, and LLaMA-Factory. These frameworks wrap the same underlying PyTorch and Hugging Face stack but differ in their engineering approaches, with Unsloth rewriting kernels for speed, Axolotl composing parallelism strategies, TRL defining trainer APIs, and LLaMA-Factory optimizing for model coverage and ease of use. A comparison of these frameworks across training speed, peak memory usage, and multi-GPU scaling shows that Unsloth leads on single-GPU throughput through hand-written kernel optimization, while Axolotl offers deeper parallelism options for multi-GPU setups. The tradeoffs matter because engineers choosing a fine-tuning framework must balance competing demands: raw training speed, memory efficiency across different sequence lengths, and the ability to scale across multiple processors.