AI Grounds
Open AI Grounds on a desktop
These interactive lessons need a larger screen. Please continue on a desktop or laptop computer.
AI Grounds
These interactive lessons need a larger screen. Please continue on a desktop or laptop computer.
Guided discovery
Move the context window. See which keys each query may use.
A query is the token seeking context; a key is a candidate token it may use. Read query tokens down rows and key tokens across columns. These eight illustrative global positions stay fixed. <PAD> is a filler slot, not real text. The shared window contains positions 1–4.
| Query ↓ / Key → | 1The | 2cat | 3sat | 4on | 5the | 6mat | 7<PAD> | 8<PAD> |
|---|---|---|---|---|---|---|---|---|
| 1 · The | Query 1 · The to key 1 · The: Allowed | Query 1 · The to key 2 · cat: Future key blocked | Query 1 · The to key 3 · sat: Future key blocked | Query 1 · The to key 4 · on: Future key blocked | Query 1 · The to key 5 · the: Key outside window | Query 1 · The to key 6 · mat: Key outside window | Query 1 · The to key 7 · <PAD>: Key outside window | Query 1 · The to key 8 · <PAD>: Key outside window |
| 2 · cat | Query 2 · cat to key 1 · The: Allowed | Query 2 · cat to key 2 · cat: Allowed | Query 2 · cat to key 3 · sat: Future key blocked | Query 2 · cat to key 4 · on: Future key blocked | Query 2 · cat to key 5 · the: Key outside window | Query 2 · cat to key 6 · mat: Key outside window | Query 2 · cat to key 7 · <PAD>: Key outside window | Query 2 · cat to key 8 · <PAD>: Key outside window |
| 3 · sat | Query 3 · sat to key 1 · The: Allowed | Query 3 · sat to key 2 · cat: Allowed | Query 3 · sat to key 3 · sat: Allowed | Query 3 · sat to key 4 · on: Future key blocked | Query 3 · sat to key 5 · the: Key outside window | Query 3 · sat to key 6 · mat: Key outside window | Query 3 · sat to key 7 · <PAD>: Key outside window | Query 3 · sat to key 8 · <PAD>: Key outside window |
| 4 · on | Query 4 · on to key 1 · The: Allowed | Query 4 · on to key 2 · cat: Allowed | Query 4 · on to key 3 · sat: Allowed | Query 4 · on to key 4 · on: Allowed | Query 4 · on to key 5 · the: Key outside window | Query 4 · on to key 6 · mat: Key outside window | Query 4 · on to key 7 · <PAD>: Key outside window | Query 4 · on to key 8 · <PAD>: Key outside window |
| 5 · the | Query 5 · the to key 1 · The: Query outside window | Query 5 · the to key 2 · cat: Query outside window | Query 5 · the to key 3 · sat: Query outside window | Query 5 · the to key 4 · on: Query outside window | Query 5 · the to key 5 · the: Query outside window | Query 5 · the to key 6 · mat: Query outside window | Query 5 · the to key 7 · <PAD>: Query outside window | Query 5 · the to key 8 · <PAD>: Query outside window |
| 6 · mat | Query 6 · mat to key 1 · The: Query outside window | Query 6 · mat to key 2 · cat: Query outside window | Query 6 · mat to key 3 · sat: Query outside window | Query 6 · mat to key 4 · on: Query outside window | Query 6 · mat to key 5 · the: Query outside window | Query 6 · mat to key 6 · mat: Query outside window | Query 6 · mat to key 7 · <PAD>: Query outside window | Query 6 · mat to key 8 · <PAD>: Query outside window |
| 7 · <PAD> | Query 7 · <PAD> to key 1 · The: Padding query excluded by demo | Query 7 · <PAD> to key 2 · cat: Padding query excluded by demo | Query 7 · <PAD> to key 3 · sat: Padding query excluded by demo | Query 7 · <PAD> to key 4 · on: Padding query excluded by demo | Query 7 · <PAD> to key 5 · the: Padding query excluded by demo | Query 7 · <PAD> to key 6 · mat: Padding query excluded by demo | Query 7 · <PAD> to key 7 · <PAD>: Padding query excluded by demo | Query 7 · <PAD> to key 8 · <PAD>: Padding query excluded by demo |
| 8 · <PAD> | Query 8 · <PAD> to key 1 · The: Padding query excluded by demo | Query 8 · <PAD> to key 2 · cat: Padding query excluded by demo | Query 8 · <PAD> to key 3 · sat: Padding query excluded by demo | Query 8 · <PAD> to key 4 · on: Padding query excluded by demo | Query 8 · <PAD> to key 5 · the: Padding query excluded by demo | Query 8 · <PAD> to key 6 · mat: Padding query excluded by demo | Query 8 · <PAD> to key 7 · <PAD>: Padding query excluded by demo | Query 8 · <PAD> to key 8 · <PAD>: Padding query excluded by demo |
First global position in the shared slice. Moving it does not move or renumber the query. Use arrow keys on the slider. Press Enter or leave the number field to apply an exact edit.
Number of slots in the slice, including any padding. A longer slice clamps start if needed; the query stays fixed. Use arrow keys on the slider. Press Enter or leave the number field to apply an exact edit.
Query 4 · on is active in window 1–4. Eligible keys: 1 · The, 2 · cat, 3 · sat, 4 · on.
Actual changes to any control clear stale explanations and transfer answers. Reapplying the same state preserves them. The table is read-only: use the controls to edit and arrow keys to scroll a focused table.
Keys passing every eligibility condition.
4
in window AND padding rule AND direction
Eligibility does not specify attention magnitude or guarantee use.
Eligible keys before the selected query.
3
eligible key position < query position
A past key outside the slice is still unavailable.
Eligible self and later-position keys.
1 / 0
key = query / key > query
An active causal query permits self and excludes future keys.
All conditions must pass. Causal allows key position ≤ query position, including self. Bidirectional permits either direction in this comparison. Exclude removes padding keys; Allow removes that condition only. The table names the first failing condition, so more than one condition may block a key.
| Key position | Token | Eligibility or first reason |
|---|---|---|
| 1 | The | ✓ Allowed |
| 2 | cat | ✓ Allowed |
| 3 | sat | ✓ Allowed |
| 4 | on | ✓ Allowed |
| 5 | the | × Key outside window |
| 6 | mat | × Key outside window |
| 7 | <PAD> | × Key outside window |
| 8 | <PAD> | × Key outside window |
This is one global shared slice, not a rolling local-attention band for each query or a model’s maximum context capacity. Window length counts all slots, including padding. Editing length clamps start to stay inside positions 1–8; the selected query never moves or renumbers. The Query token control offers only the six real tokens. Padding query rows are always excluded by explicit demo policy, independently of the padding-key setting: a key-padding mask by itself does not generally exclude query rows.
For an active real query, a key is eligible exactly when it is inside the slice, passes the padding-key setting, and passes the direction setting. The first-failing reason order is padding query, query outside window, key outside window, padding key excluded, then future key blocked. Multiple conditions can fail. An outside query is inactive and has no eligible keys; the lab does not invent weights or outputs for all-blocked rows.
Checks are semantic permission labels, not an API’s Boolean encoding: PyTorch MultiheadAttention uses Boolean True to block, while scaled_dot_product_attention uses True to allow. No attention scores, softmax, vectors, model output, prediction, training or tokenizer run here. Permission alone does not guarantee nonzero attention weight or a contribution to output. Bidirectional and Allow padding are deliberate comparisons, not recommendations for a causal language model.
Early slice uses start 1, length 4, query 4, Causal and Exclude. Later slice uses start 3, length 4, query 6, Causal and Exclude. Padding contrast uses start 3, length 6, query 6, Bidirectional and Exclude. Reset and predictions restore Early slice in the current experiment; free Reset starts Experiment 1.
PyTorch · Attention and padding-key masks