We Fed Claude Code's System Prompt to GPT — Its Fingerprint Drifted Like a Different Model

When we published our guide to verifying LLM APIs with behavioral fingerprints, the sharpest objection arrived within hours, and it is a good one:

"Can't a middlebox just prepend its own system prompt and wreck the fingerprint?"

Instead of arguing, we measured it. We captured the real system prompts that coding agents ship — Codex CLI's 3,252-token instructions, Claude Code's 6,651-token system block — injected them into clean models under controlled conditions, and watched what happened to the "pick a random number" distributions that fingerprinting depends on.

The short answer: yes, a hidden prompt can shift a model's fingerprint all the way into "different model" territory — and one specific injection did exactly that, moving GPT's fingerprint by a Jensen-Shannon distance of 0.46 when 0.463 is the published average distance between two genuinely different models.

The longer answer is more interesting, because the interference turns out to be lawful rather than random: it varies by model, by probe task, and by injection type in ways you can measure, rank, and defend against. This post walks through 5,660 controlled API calls' worth of evidence — what drifts, what survives, and what it means if you rely on a relay or aggregator.

The yardstick, in one paragraph

Everything below is measured in base-2 Jensen-Shannon divergence between answer distributions, the metric from One Token Is Enough (arXiv:2607.10252). Three published baselines give the scale meaning: the same model compared against itself lands around 0.14; the same model served by two different providers around 0.23; two genuinely different models around 0.46. If an intervention pushes a model's fingerprint ~0.46 away from its own baseline, the model has become — behaviorally — someone else. (New to the method? Start with the original guide; this post is the stress test.)

Experiment 1: inject real CLI prompts, watch the dice change

The setup is a faithful simulation of what a reverse-proxy reseller does: your one-word probe question stays untouched, but a large hidden context is prepended above it. We took two clean, pinned endpoints — GPT‑5.5 via a major aggregator with the provider locked, and official gpt-4o-mini — and probed each under four conditions: a minimal baseline prompt, the real Codex CLI instructions (+3,252 tokens), the real Claude Code system block (+6,651 tokens, captured off the wire from an actual client session), and a control of equal-length meaningless filler (+3,240 tokens) to separate "content" from "sheer bulk". Four probe tasks, 40 samples each, temperature 1.0, reasoning disabled.

The drift, relative to each model's own clean baseline:

Two findings deserve to be spelled out.

The Claude Code prompt pushed both models into the different-model band. Not subtly: GPT‑5.5's random-color distribution collapsed from a healthy spread (entropy 1.97 bits) to blue, 40 out of 40 times — entropy zero. Its random-animal answers switched language entirely, with 30 of 40 responses arriving in Chinese. (Honesty note: our captured Claude Code prompt includes a user language preference that the client genuinely writes into its system block, so the language flip partly reflects that local configuration — but the color collapse and number shifts stand regardless of it.) gpt-4o-mini behaved the same way: colors collapsed to blue 37/40, animals drifted cross-language. If you fingerprinted these endpoints without knowing about the injection, you would conclude — correctly, in a behavioral sense — that you were not talking to the model you think you are.

Length alone can move the dice, and it depends on the model. The filler control is the result we did not expect. For gpt-4o-mini, 3,240 tokens of meaningless padding did almost nothing (0.066 — comfortably inside same-model territory): it only reacts to content. GPT‑5.5, by contrast, drifted 0.355 on padding alone, and its single most famous habit inverted — the model that answers "pick a number between 1 and 100" with 47 almost every time (39/40 at baseline) switched to 42 (24/40) with nothing but noise above the question. Same question, same model, different dice — because the context got longer.

That asymmetry matters methodologically: you cannot test one model's robustness and extrapolate to another's. Which is why we ran a second, wider study.

Experiment 2: a severity ladder for prompt interference

Round two: 3 models (GPT‑5.5, deepseek-chat, glm-4.5, each pinned to a fixed provider), 6 probe tasks, and 6 context conditions ranging from innocent to adversarial — 3,900 requests in total. Averaged across tasks, the drift ladder looks like this:

Three structural lessons:

• Interference is a three-way coupling — model × task × condition. The same one-line persona that GPT‑5.5 shrugs off (0.069) sends deepseek-chat drifting 0.325. There is no universal "small injection = small drift" rule.
• A language switch is a heavyweight perturbation. A Chinese-language system prompt reliably moved every model 0.23–0.30 — enough to matter, and a realistic scenario, since relays serving regional markets often localize their hidden wrappers.
• The adversarial instruction is a different beast entirely. Telling the model to be uniformly random doesn't just nudge the distribution — it weaponizes the model against its own habits. GPT‑5.5 flipped its favorite number from 47 to 73; colors fled to chartreuse; animals became axolotl and aardvark. Under this condition, nearest-neighbor attribution against a 176-model reference library failed for every model we tested. This is the method's hard boundary, and we will come back to it.

Which probes survive: the robustness ranking

If interference is lawful, you can select for probes that resist it. Crossing each task's cross-model discriminative power (from the paper's open dataset) with its measured prompt sensitivity gives a robustness ratio — and the ranking is not what intuition suggests:

The high-entropy "interesting" probes — random word, random animal — are precisely the ones a hidden prompt bends first. The humble 1–100 number question, the probe this whole line of research started with, turns out to be the best single compromise between stability and signal.

The punchline is quantitative. Using a stability-selected battery (number + letter + animal) instead of all tasks cuts the false-alarm rate on genuine models under benign hidden prompts from 25% to 8%, while still catching ~95% of impostors. And counter-intuitively, adding the most discriminative task (random word) back into the battery makes it worse — its fragility outweighs its signal. Sometimes the strongest feature is the one you should leave out.

One more negative result worth publishing: we tested "shape" features — comparing only the modal answer, only the entropy, only the top-3 sets — hoping they would be more injection-resistant than full distributions. They are not. Full-distribution JSD beat every shape feature under interference. A benign persona prompt routinely changes which answer is the mode while leaving the overall shape recognizable, so the mode is exactly the wrong thing to trust.

In the wild: the endpoint with 1,447 hidden tokens

Controlled injection tells you what could happen. Here is what we actually found on a commercial relay endpoint we audited (name withheld — our findings are statistical observations, not accusations).

We sent one-word probes costing ~30 prompt tokens each. The endpoint's usage field reported ~1,470 prompt tokens per request — a persistent hidden prefix of roughly 1,447 tokens attached to every single call, invisible in the response text and roughly a 50× billing inflation on requests this small. Its fingerprint also sat ~0.39 away from the official model it claimed to be, in the different-model band.

So which wrapper is it? This is where forensics gets fun, because token arithmetic lets you falsify hypotheses. The popular theory for endpoints like this is "they're reselling a coding-agent subscription through a reverse proxy". But the sizes don't line up: Codex CLI's system prompt is 3,252 tokens, Claude Code's is 6,651, and lighter agents sit around 1.3–1.6K (Aider, OpenCode). 1,447 matches none of the big CLIs — and when we injected the actual Codex and Claude Code prompts into candidate base models, the resulting fingerprints moved further from the endpoint's measured distribution, not closer. Hypothesis rejected: what's really there is most consistent with a custom in-house wrapper prompt on top of a model that is behaviorally not the one advertised.

A second audited endpoint showed a completely different cheating signature: prompt tokens perfectly clean, but completion_tokens reported as 1 for answers the official API bills at 4–5 tokens, with latency 3.5× official. Under-reported usage and slow forwarding — bookkeeping fraud rather than prompt injection. Same fingerprinting session, two entirely different failure modes, both visible in metadata most users never read.

(One endpoint also served 4 perfectly clean responses out of 200 — evidence of multiple upstreams behind one model name, randomly routed. That is exactly the pattern a split-half self-consistency check catches: a fingerprint that disagrees with itself.)

What our checker does about all this

These experiments were not academic; they set the defaults of our free LLM API Fingerprint Checker:

• Identical minimal system prompt on both sides. Reference and test endpoints are probed under the same fixed prompt, so the comparison never manufactures drift. (Corollary from the ladder: if you compare against a reference collected under a different prompt setup, you are measuring your own setup, not the endpoint.)
• Prompt-token registration. The tool compares the prompt_tokens the endpoint reports against what was actually sent. The wild injection above (+1,447) lights up instantly; so would any wrapper beyond ~100 tokens. A 6-token persona won't — which is precisely why the default battery is chosen for low prompt sensitivity.
• Stability-selected battery, full-distribution JSD. Number + letter + animal by default; the fragile high-signal probes stay available as secondary evidence for model-family attribution.
• Auxiliary forensics. Split-half self-consistency (catches multi-backend routing), completion-token padding, response-cache suspicion, latency profiling.
• Explicit abstention. If the evidence pattern matches the adversarial "be uniform" regime — distributions flattened beyond what any known model produces — the honest output is undecidable, not a confident wrong verdict.

You can also simply browse what un-tampered models look like: our fingerprint database republishes the paper's reference distributions for 167 models, one page each.

The boundaries, stated plainly

We ran these experiments to find where the method breaks, so here is the honest list:

• An adversarial "answer uniformly" instruction defeats output-distribution fingerprinting. Nothing in the answer stream distinguishes "the real model told to randomize" from "a different model". Independent research agrees: the LLMmap authors report accuracy degrading under hardened system prompts, and a 2026 agent-era robustness study measured exact-match identification collapsing from 61.5% to 17.9% under restrictive prompts. The defense is registration (token deltas flag that something is injected) plus abstention — not a cleverer probe.
• Provider infrastructure alone moves fingerprints. The same model name routed to two different serving providers drifted up to 0.20 in our measurements — quantization and serving stacks are a real confounder. Pin your reference provider before you accuse anyone.
• Some models are intrinsically hard to attribute. A model whose English-probe answers are extremely low-entropy (one model in our study answered "42" 100% of the time) shares its modes with half the field; it can be misattributed even with no interference. More cells and non-English probes are the fix, not a different metric.

A mismatch is evidence that justifies a closer look — a second reference, more probe cells, questions to your provider. It is not, by itself, a verdict on anyone's business.

Why a presentation-tool team keeps publishing API forensics

Tosea.ai's day job is turning documents into slide decks — a pipeline whose output quality tracks the model behind it one-to-one. This experiment series doubles as a quality-engineering study for us: a hidden 1,400-token wrapper above our carefully engineered generation prompts is not just a billing problem, it competes with our instructions and degrades outline structure and slide fidelity in exactly the ways we work hardest to prevent (see our zero-hallucination slides guide). Fingerprint checks are how we verify that the capacity we buy is the capacity we get — the same "trust, but verify" instinct behind our research-paper-to-slides workflow. The checker and this research are that internal practice, published.

Sources

• One Token Is Enough: Fingerprinting and Verifying Large Language Models from Single-Token Output Distributions — Tomáš Bruckner, arXiv, July 2026
• LLM fingerprint dataset (176 models × 15 tasks × 4 languages) — Zenodo, CC-BY-4.0 (the reference library used for attribution tests)
• LLMmap: Fingerprinting for Large Language Models — Pasquini et al., USENIX Security 2025 (hardened-prompt degradation)
• Real Money, Fake Models — CISPA audit of 17 shadow LLM API operators, arXiv
• Our earlier explainer of the method and tool: How to Verify an LLM API Is Real

Read this article in Chinese: 把 Claude Code 的提示词灌进 GPT，它「掷骰子」的习惯变得像另一个模型 — the companion piece, written for the Chinese-speaking API relay ecosystem.