"Pinpointing Attention-Causal Communication in Language Models" was accepted at NeurIPS 2025.

When an attention head attends from one token to another, what actually caused it? This paper isolates that as a small, low-dimensional signal with a provable causal link to attention, and uses it to trace circuits from a single forward pass.

• Paper
• Code

Here is a short thread about the work:

View the thread on X