Liquid AI Releases LFM2.5-VL-3B-DSpark: Speculative Decoding for Vision-Language Models With Up to 3.13x Faster Decoding

Speculative decoding is a technique that speeds up vision-language models by using a smaller "drafter" model to propose multiple tokens ahead, which a larger model then verifies in one pass instead of generating tokens one at a time. A new drafter model has been released that adds about 280 million parameters to an existing vision-language model and achieves up to 3.13 times faster decoding on Apple silicon and up to 2.66 times faster on an NVIDIA H100. The drafter works by reading hidden states from the target model's layers and predicting the next several tokens, with the key insight being that the technique works regardless of whether the input is text or image data since both are represented as tensors in the hidden layers. The speedup is most noticeable during the decoding phase, while image encoding and initial processing remain at the same speed, which limits the total end-to-end acceleration especially on edge devices where these other steps take up a larger share of the total time.