Section 123 · Chapter 16, Introspection: White-Box Testing Networks
Inputs and Tokenization
White-box testing should begin with the evidence the model actually received, not with an exciting interpretation of its neurons.
tokenizationinputs tokenization
What to do
- Save the raw input, normalized input, assembled prompt, tokenizer version, token ids, positions, byte spans, truncation boundary, retrieved context, model id or digest, generation settings, tool schemas, and final trace id.
- Inspect the actual model rather than turning one visualization into an architectural rule.
- Save enough at each boundary to identify where behavior changed.
- Preserve the model and tokenizer versions, prompt, parameters, hardware, library versions, captured tensors, aggregation method, and rendering code.
- Use the best proxies available: prompt and retrieval traces, tool calls, embeddings, output log probabilities when available, judge rationales, and behavioral slices.
Evidence to preserve
- Save the raw input, normalized input, assembled prompt, tokenizer version, token ids, positions, byte spans, truncation boundary, retrieved context, model id or digest, generation settings, tool schemas, and final trace id.
- Save enough at each boundary to identify where behavior changed.
- Preserve the model and tokenizer versions, prompt, parameters, hardware, library versions, captured tensors, aggregation method, and rendering code.
- Preserve the inputs, versions, configurations, raw outcomes, and results for tokenization, inputs tokenization needed to reproduce work on Inputs and Tokenization.
Continue the conversation
Apply this to your context.
Save your product context once, then open a focused conversation that combines it with this concept.
Cite this page
Jason Arbon. "Inputs and Tokenization." Testing AI Knowledge Edition, section 123.
https://jarbon.ai/testing-ai/knowledge/ch123-inputs-tokenization.html