Beyond LoRA: Can you beat the most popular fine-tuning technique?

Parameter-efficient fine-tuning, or PEFT, is a set of techniques that reduce the memory needed to adapt existing AI models to specific tasks or datasets. One technique called LoRA has become overwhelmingly dominant, used in over 98 percent of cases on one platform, though this popularity may stem partly from its early emergence and self-reinforcing visibility rather than proven superiority. Researchers frequently publish papers claiming their alternative techniques outperform LoRA, but these comparisons are often problematic due to inconsistent testing conditions, different benchmarks across studies, and inherent pressure to show improvements over existing methods. To help users make better-informed choices, developers have created standardized benchmarks that evaluate multiple PEFT techniques under identical conditions on the same models, datasets, and hardware, tracking not just performance but also memory usage, runtime, and other practical factors.