"Do Language Models Track Entities Across State Changes?" was accepted at ICML 2026.

Led by Zilu Tang, this work asks whether language models really keep track of the state of the world as it changes. It turns out they don’t track states incrementally; instead, they aggregate the relevant information in parallel at the last token, once the query becomes clear.

• Paper
• Code

Here is a short thread about the work:

View the thread on X