Glossary
Self-attention
A step where each token looks at the visible tokens, including itself, and takes a weighted mix of their information. It lets the model link distant words and runs in parallel across positions.
Where it is explained
See also
Terms explained in the same lessons.