Kyutai Releases MuScriptor: An Open-Weight Decoder-Only Transformer for Multi-Instrument Music Transcription to MIDI

Automatic music transcription converts audio recordings into symbolic musical notation, usually in MIDI format. While transcribing single instruments works reasonably well, transcribing full multi-instrument mixes has remained difficult. MuScriptor is an open-weight decoder-only transformer model designed to close this gap by converting audio into MIDI-like tokens for pitch, timing, and instrument through a three-stage training process using synthetic data, real recordings, and reinforcement learning. The model achieves significant improvements over existing baselines, with a Multi F1 score of 48.2 compared to 21.9 for the previous best system, and is released in three size variants with inference code and a web interface for practical use.