Multi-head attention

Several attention computations run in parallel (heads), each free to learn a different relationship such as syntax or coreference. Their outputs are concatenated.

All terms, A to Z