Sampling settings are a control surface
Temperature and top-p do more than change tone. They decide how readily an AI system takes a less likely path, which makes them part of its test and audit configuration.
By Piotr Zenon Żabrowski · Published · Updated
One model, many possible traces
Generation is a sequence of constrained choices, not a lookup. For each position, the model ranks token candidates from its current context. Temperature changes how strongly the sampler favours the front-runners: reducing it makes unusual choices less likely, while increasing it gives lower-ranked options more opportunity. Top-p creates a moving shortlist by admitting the leading candidates until they account for the configured share of probability; the draw happens within that shortlist. Each chosen token joins the context, so one early choice can redirect the rest of the response.
Match the setting to the security job
| Job | Sampling goal | Reason |
|---|---|---|
| Regression testing | Minimise variation | A changed answer is more likely to expose a prompt, model or application change. |
| Adversarial testing | Exercise multiple configurations and runs | A single stable run may miss an unsafe branch that remains reachable. |
| Production workflow | Fix and version approved settings | An unnoticed parameter change can alter refusal, formatting and tool-selection behaviour. |
| Incident reconstruction | Replay the recorded configuration | Without the original settings, failure to reproduce does not show that the event was impossible. |
Even an apparently deterministic setting is not a complete replay guarantee. The provider may update a model, inference stack or default parameter; the application may assemble a different context; and some serving paths can still vary. A test report that records only the model name leaves too much hidden state outside the evidence.
For auditors, the useful unit of evidence is the run, not a screenshot of its answer. Store settings with prompt and model versions, link them to change approvals, and retain enough context for replay. Report a declared run count and its outcomes rather than selecting one convenient response. If the provider offers no fixed snapshot, seed or replay guarantee, record that limit; it narrows the audit's claims.
Record the execution envelope
For every security-relevant run
- Model identifier and provider version or snapshot.
- Exact system prompt, user input, retrieved context and tool results.
- Temperature, top-p, token limit, stop conditions and seed where supported.
- Application version, policy or guardrail version and enabled tools.
- Timestamp, request identifier and the unedited output.
- A hash or immutable reference for large inputs that cannot be retained inline.
Treat sampling changes like policy changes: assign an owner, review the diff, run the same security cases, and keep a rollback path. Alert on production values outside the approved configuration. Low variation supports regression checks; broader sampling supports exploration. Neither mode is sufficient alone, and neither should be changed as an undocumented tuning experiment.
Frequently asked questions
Should production always use temperature zero?
No. Use the lowest variation compatible with the task, then test that exact configuration. Temperature zero cannot prove factuality, safety or perfect reproducibility.
Why run the same security test more than once?
Sampling can expose different reachable outputs from the same input. Repeated runs help distinguish a consistently blocked path from one that succeeds intermittently.
Related
AI security
How to secure GenAI, LLM and agentic AI systems: current OWASP threat classes, layered controls, agent authority, retrieval isolation and monitoring.
Context window structure
Why mixing data and instructions in the same context window causes prompt injection, and how 'lost in the middle' behavior degrades operational security.
AI security assessment
Security assessment of GenAI and LLM systems: prompt injection, tool and agent abuse, data leakage, model supply chain and monitoring gaps.
STRIDE for AI systems
Applying STRIDE to LLM and agentic applications: what each category means when untrusted text can act as instructions, where STRIDE stops covering multi-agent failure modes, and the controls per element.
Prompt injection defence
Why prompt injection cannot be filtered away, why it has no full fix yet, and how to design AI systems — architecturally, not just with better prompts — so a successful injection has no useful consequence.
LLM security controls
A control checklist for enterprise GenAI: data handling, retrieval isolation, agent authority, cascading-failure containment, monitoring, cost limits and vendor terms.