Multi-Vector (Late Interaction) Embedding Models with Sentence Transformers

Sentence Transformers version 6.0 introduces a new model type called MultiVectorEncoder for multi-vector embedding models, which keep one vector per token instead of compressing entire texts into a single vector. Multi-vector models defer the interaction between query and document until scoring time using the MaxSim operator, which compares every query token against every document token to find the best matches. This approach preserves token-level matching information and performs better on queries requiring specific pieces of documents or multiple requirements at once, particularly on out-of-domain data, but the tradeoff is a substantially larger index size since one vector per token generates many more vectors than traditional single-vector models.