StateBench measures how models track information

StateBench is a benchmark that measures how well language models track changes to information about entities across long documents. The benchmark matters because state tracking underlies important tasks like reading code where variables are reassigned and following narratives where objects and their locations change. Different model architectures have theoretical limits on state tracking, with transformers predicted to struggle on the most complex types while hybrid and recurrent models are designed to handle them better. Testing shows that frontier models perform well on simpler state-tracking problems but accuracy drops sharply on harder problems as documents grow longer, though models trained directly on state-tracking tasks show architectural differences in their ability to generalize beyond their training length.