Provided by the Swiss Non Profit Humanais.org

Attention Is All You Need

Transformer Visual Explainer

Click any box in the middle diagram, or use Next/Back, to jump to that step.

Our example sentence: "The dog is too tired to cross the street" → translating into French. Each step also shows the real numbers the model actually uses (from the original paper's settings)[cite: 1].

Original Architecture

Transformer Architecture transformer.png

Attention Is All You Need

ENCODER DECODER Input Output (shifted right) 1 · Input embedding 2 · Multi-head attention 3 · Add & norm 4 · Feed forward 5 · Add & norm Context ↻ repeated ×6 6 · Output embedding 7 · Masked attention 8 · Add & norm 9 · Cross-attention 10 · Add & norm 11 · Feed forward 12 · Add & norm repeated x6 13 · Linear 14 · Softmax

👆 Tap any box above to jump straight to that step

Authors of the original 2017 paper[cite: 1]