How LLM API Fingerprint Checking Works
What is an LLM behavioral fingerprint?
Large language models cannot produce true randomness. Ask one for "a random number between 1 and 100" at temperature 1.0 and it will overwhelmingly return a handful of favorites — 42, 47, 57, 73 — with proportions that are stable for a given model but different between models. The paper "One Token Is Enough" (Tomáš Bruckner, arXiv:2607.10252) showed that the answer distributions of a battery of such one-word probe questions (random numbers, colors, letters, coin flips, and more, asked in multiple languages) form a reliable behavioral fingerprint that can identify which model actually sits behind a black-box API endpoint. No magic strings are involved: every probe is an ordinary semantic question drawn randomly from a pool of paraphrases, which makes keyword-based filtering by a dishonest proxy impractical.
How to verify an endpoint step by step
In Endpoint Audit mode, enter the base URL, API key, and model name of the endpoint you want to test. Then choose your reference: pick the claimed model from the built-in fingerprint library (each entry shows its collection date and covered probe cells), or point the tool at a second endpoint you trust — typically the official API — to collect a fresh reference in the same run. Choose an intensity (quick 4 cells × 15 samples, standard 8 × 25, or strict 16 × 25), review the request estimate, and confirm. You watch answers stream in live, cell by cell, and can cancel at any time. The report shows a three-tier verdict, the mean Jensen-Shannon divergence with published baselines on a scale, a per-cell JSD table with side-by-side distribution charts, plus purely factual technical details such as token usage, latency, caching, and routing stability.
The math: Jensen-Shannon divergence and baselines
Each probe cell (task × language) yields an empirical distribution over normalized one-word answers. Two fingerprints are compared cell by cell with the Jensen-Shannon divergence (base 2, bounded between 0 and 1), and the endpoint distance is the average JSD over cells where both sides have at least 10 valid samples. The paper's baselines: the same model compared with itself measures about 0.140, the same model across different providers about 0.227, and different models about 0.463. This tool therefore reports ≤0.25 as consistent, 0.25–0.35 as suspicious, and >0.35 as a likely mismatch — thresholds kept in one constants file and documented for recalibration. All built-in reference fingerprints come from the paper's official open dataset (Zenodo DOI 10.5281/zenodo.21278557, CC-BY-4.0; 167 models ship built in), collected under the same fixed minimal one-word system prompt this tool uses for live sampling; our own cross-check measured gpt-4o-mini official vs OpenRouter at JSD 0.117 and cross-model pairs at 0.63–0.77.
Accuracy, limitations, and responsible use
This is a statistical method with a known error rate — roughly 10.6% equal-error rate at 8 cells and 9.5% at 16 in the paper's evaluation — so a verdict is evidence, not proof, and must never be read as an accusation against any provider. Distributions can legitimately drift when a provider deploys a quantized build, silently updates a model version, or when a reference fingerprint has aged. Reasoning models add another caveat: if the endpoint accepts none of the standard reasoning-disable parameters, the tool falls back to a post-reasoning channel with a larger token budget and marks the whole result as lower confidence. When in doubt, re-run at a higher intensity, try a second reference, and weigh other signals before making decisions.
Why this tool, and why in the browser
Verifying whether someone else is trustworthy should not require you to hand your own API key to a third party. This checker runs entirely in your browser: the key stays in page memory (localStorage only if you explicitly opt in), probe requests go directly from your browser to the endpoint you configure, and Tosea.ai operates no proxy, collects no telemetry, and never sees your key, your endpoints, or your results — all of which you can verify in the DevTools Network panel. The method, the thresholds, and the reference data sources are all documented on this page, the probes work in both English and Chinese, and the whole thing is free with no run limits. If an endpoint blocks browser requests (CORS), the tool degrades gracefully by generating an equivalent curl command you can run from your own terminal.