NeoMME: an efficient Multimodal-native and Multilingual Encoder

NeoMME is a multilingual multimodal encoder that processes both text and images through a single bidirectional Transformer, rather than using separate pretrained vision and language components like many existing systems. It comes in two sizes and was pretrained using a masked discrete-diffusion objective that forces the model to learn image-grounded descriptions by masking text while keeping images visible. When fine-tuned for visual document retrieval, NeoMME-Retriever can return both dense and late-interaction embeddings in a single pass and achieves competitive performance while being significantly more efficient than comparable models. This approach eliminates unnecessary architectural overhead by using one unified pathway for both image patches and text tokens, enabling easier support for pretraining, fine-tuning, and deployment across modalities.