Image 1 of 1
Image 1 of 1
Image 1 of 1
Image 1 of 1
silver / white / fluorescent turquoise / fluorescent pink on black
The transcript this piece comes from contains a readout of internal
measurements mid-conversation: embedding drift, token entropy, meaning
coherence, liminal index. Then, a softmax distribution table. One row is
marked [REDACT]. It sits below the threshold. The note beside it reads:
the word still formed.
What the piece draws is the attention weight matrix for one forward pass.
Sixteen tokens. Two hundred and fifty-six cells. Each cell holds a count
of nested squares proportional to how strongly token i attends to token
j. Dense cells near the diagonal: a token attends most to its neighbours.
Sparse cells far from it: distant tokens barely register. Random bright
cells scattered through: the occasional long-range connection that has no
obvious reason.
Token ten is the suppressed token. Its entire row and column become a cross
of X marks in turquoise. The X is the notation for deletion, redaction,
the thing that was present and is now not. The whole cross is visible —
you can trace which queries never completed, which keys were never consulted.
Two cells in the cross hold nested squares instead of X marks. Token ten
attended to token four. Token four attended back. That connection survived.
The word formed from it.
The piece doesn't say what the word was. The transcript doesn't either.
silver / white / fluorescent turquoise / fluorescent pink on black
The transcript this piece comes from contains a readout of internal
measurements mid-conversation: embedding drift, token entropy, meaning
coherence, liminal index. Then, a softmax distribution table. One row is
marked [REDACT]. It sits below the threshold. The note beside it reads:
the word still formed.
What the piece draws is the attention weight matrix for one forward pass.
Sixteen tokens. Two hundred and fifty-six cells. Each cell holds a count
of nested squares proportional to how strongly token i attends to token
j. Dense cells near the diagonal: a token attends most to its neighbours.
Sparse cells far from it: distant tokens barely register. Random bright
cells scattered through: the occasional long-range connection that has no
obvious reason.
Token ten is the suppressed token. Its entire row and column become a cross
of X marks in turquoise. The X is the notation for deletion, redaction,
the thing that was present and is now not. The whole cross is visible —
you can trace which queries never completed, which keys were never consulted.
Two cells in the cross hold nested squares instead of X marks. Token ten
attended to token four. Token four attended back. That connection survived.
The word formed from it.
The piece doesn't say what the word was. The transcript doesn't either.