Transformer learning notes: from the overall architecture to multi-head attention
Transformer study notes ordered whole-first and then part-by-part: the data flow through the encoder and the decoder; the three questions to answer before learning large models (neural networks, attention, what PyTorch is made of); what Token, the word embedding matrix, and positional encoding each solve; the meaning of Q, K, and V, why the Q·K dot product stands for relevance, and why the dot product is divided by the square root of the dimension before softmax; the difference between self-attention and cross-attention, and why cross-attention needs no mask; training predicts several tokens in parallel while inference generates one token at a time, which is why a causal mask is needed to prevent peeking, implemented by setting later positions to negative infinity so that softmax turns them into 0; the definitions of the feed-forward network, multi-head attention, and layer normalization, including how layer norm and batch norm differ in which dimension they normalize over.