Inference
Attention internals
Attention as it runs during generation: causal masks, rotary position embeddings, grouped-query and multi-head latent attention, and stretching a model to longer contexts. Each choice changes how much memory every generated token costs.
Questions
Question Difficulty Success