Handling Bias and RoPE in QK Attention with a Unified Geometric View

A practical geometric view for analyzing QK circuits with bias and RoPE using a single bilinear form

7 min read   ·   February 23, 2026

2026   ·   mechanistic interpretability   transformers   attention   linear algebra