Attention Mechanism

An attention mechanism allows a model to focus on the most relevant parts of its input when generating each token of output. Understanding attention helps explain why models sometimes miss context or over-focus on certain input segments.