Empty shelves or lost keys? Recall is the bottleneck for parametric factuality

Researchers introduced a framework called knowledge profiling that distinguishes between two different reasons why language models get facts wrong: either they never learned the fact in the first place, or they learned it but cannot retrieve it when needed. The distinction matters because encoding failures and recall failures require different solutions, with encoding problems addressed through scaling and data expansion, while recall problems might benefit from post-training and inference-time methods. Testing frontier language models revealed that nearly all facts are encoded, yet these models frequently fail to recall encoded facts, suggesting that the main bottleneck for factual accuracy is shifting from learning facts to accessing them. The researchers created WikiProfile, a benchmark of 2,150 Wikipedia-derived facts with multiple question variations, to measure both encoding and recall across different language models.