Attention is all you need (in life too)

The transformer architecture was a breakthrough because it asked: what if we could learn which parts of the input matter for the output?

Before transformers, we were stuck with RNNs and their limited context windows. We couldn't attend to what was far away. This mirrors Notes on attention — what you attend to shapes reality.

But human cognition is like this too. Attention is the scarce resource. You can't process everything. You have to decide what matters. Writing is a search procedure is partly about directing attention through language.

The things I've learned best are the things I paid attention to. Not just surface-level attention—deep attention. Obsessive attention. This is why Why I write matters for understanding myself.

Entropy of mind is low when attention is high. The shape of a thought explores how attention crystallizes formless ideas.

Maybe the breakthrough isn't in the architecture of models. Maybe it's in knowing how to direct your attention. And that's the real work of being human.