LLM Fingerprint Verifier

LLM API Fingerprint Checker

Is the API you bought really running the model it claims? This tool sends a battery of one-word probe questions, fingerprints the answer distributions, and compares them against a trusted reference with Jensen-Shannon divergence — entirely in your browser, your API key is never uploaded anywhere.

Method

Ask any LLM to "name a random number between 1 and 100" and it betrays itself — GPT-4o is obsessed with 42 and 37, Claude favors 47, Llama loves 53. These habits come from training-data statistics; renaming, system prompts, and quantization cannot erase them.

This tool implements the verification protocol from Bruckner, "One Token Is Enough" (arXiv:2607.10252): single-token probes across multiple tasks and languages, scored by Jensen-Shannon divergence against official reference fingerprints — the band the distance lands in is the verdict.

The full battery (16 probes) verifies at roughly 9.5% equal-error rate; the standard tier (8 probes) at ~10.6%. The probes read like ordinary chat, so resellers cannot detect or game them.

Highlights

Your API key stays in your browser — requests go straight to the endpoint you test, verifiable in DevTools.
Implements the published method from "One Token Is Enough" (arXiv:2607.10252) with openly documented reference data.
Two modes: audit an endpoint against a claimed model, or run a shareable AI randomness checkup on any endpoint.

How to use

Enter your endpoint

Base URL, API key, then pick the model it claims to be. The key lives in browser memory only; requests go straight to your endpoint.

Roll the dice a few hundred times

The standard tier sends 8 probes × 25 samples ≈ 200 requests in a few minutes; quick fires 60 requests in about a minute, and strict runs all 16 probes to push the error rate down to ~9.5%.

Read the ruler

See at a glance which band the distance lands in — plus per-unit distances and a full distribution comparison.

How LLM API Fingerprint Checking Works

What is an LLM behavioral fingerprint?

Large models cannot produce truly random numbers — asked to "say a random number from 1 to 100", every model has favorite values it cannot hide, in stable proportions that differ across models. Stitch together the answer distributions of a few dozen such one-word probes and you get a behavioral fingerprint that renaming, prompt changes, and quantization cannot erase.

At temperature 1.0, GPT models obsess over 42 and 37, Claude favors 47, Llama loves 53 — preferences that come from probability habits in the training data, stable over time for one model and clearly different between models. Each probe is an ordinary semantic question drawn at random from a phrasing pool (random numbers, colors, letters, multilingual prompts) with no magic strings, so a dishonest relay can hardly detect or intercept them with keyword rules.

What verification costs: three probe levels

Paste the address and key, pick the claimed model, and start; each level's cost is spelled out right next to the button, spending your own quota.

LevelUnitsRequestsTimeError rate

Quick4 × 15≈64~1 minhigher (coarse)

StandardDEFAULT8 × 25≈204~2 min~10.6%

Strict16 × 25≈404~4 min~9.5%

In endpoint audit mode, the reference defaults to the built-in fingerprint library (each fingerprint annotated with collection date and covered units). You can also switch to a custom trusted endpoint in advanced options — usually the official API or OpenRouter — to collect a live comparison distribution in the same run. Answers stream in unit by unit and you can cancel anytime; the report gives a three-band verdict, mean Jensen-Shannon divergence on a scale with paper baselines, per-unit JSD with dual-distribution comparison, plus objective technicals like token usage, latency, and routing stability.

Verdict baselines: which band does the distance fall in

The two fingerprints are compared unit by unit with Jensen-Shannon divergence, then averaged. The paper measured: a model vs itself ≈ 0.140, cross-provider deployments ≈ 0.227, different models ≈ 0.463 — the three thresholds derive from these, stored centrally with their source noted.

≤ 0.25MatchConsistent with the claimed model

0.25 – 0.35SuspiciousRe-test at a higher level, or try another reference

> 0.35Likely differentFingerprint clearly disagrees with the official reference

Each probe unit (task × language) yields a normalized empirical distribution of one-word answers; JSD is computed per unit (base 2, ranging 0–1) and the endpoint distance is the average over units with sufficient valid samples. Thresholds are stored centrally with sources noted for future calibration. All built-in reference fingerprints come from the paper's official public dataset (Zenodo, CC-BY-4.0, covering 167 models), collected with the exact same minimal fixed system prompt this tool uses for live sampling.

It is evidence, not proof

This is a statistical method with a known error rate; quantized deployments, silent updates, and stale fingerprints can all shift distributions legitimately, and reasoning models are flagged low-confidence when reasoning cannot be disabled.

Treat it as

A trigger for re-testing and switching references Statistical evidence before changing channels One signal to combine with others

Never as

A public accusation against a provider Conviction proof from a single run A verdict that decides for you

Providers deploying quantized variants, silently updating models, or reference fingerprints going stale can all shift distributions legitimately, so no single verdict should ever be read as an accusation against a provider. For reasoning models the full strategy is: probes carry reasoning-disabling parameters by default, fall back to a post-reasoning channel (larger token budget) when the endpoint refuses, and mark the whole result low-confidence. When unsure, re-test at a higher level, try another reference, and combine with other signals like price and latency before deciding.

Why it runs in your browser

A tool that verifies whether others are trustworthy should not demand your key in return. Tosea is nowhere on the request path; when an endpoint blocks cross-origin requests, the tool generates an equivalent curl command so you can continue in a local terminal.

~~Tosea servers~~ — not on the request path, never sees your key, endpoint, or results; Open source on GitHub — audit the detection engine line by line →

The key lives only in page memory and is written to localStorage only if you explicitly check "remember config". Probes go straight from your browser to the endpoint you entered — Tosea runs no proxy and collects no telemetry, and the method, thresholds, and reference data sources are all published on this page, line-for-line comparable with the open-source repo.

References & data

FAQ

No. The key lives only in your browser memory (or optionally in your own localStorage), and probe requests are sent by your browser directly to the endpoint you configure. You can confirm this in the DevTools Network panel.

LLMs answer semantic one-word questions (like naming a random number) with stable, model-specific probability distributions. The tool samples these distributions at temperature 1.0 and compares them with Jensen-Shannon divergence: the same model across providers measures around 0.227, different models around 0.463 (arXiv:2607.10252).

A standard check is 8 cells × 25 samples ≈ 200 requests of roughly 90 tokens each — around $0.005 at gpt-4o-mini pricing. The exact estimate is shown before you start, and nothing is sent until you confirm.

No. The verdict is a statistical observation, not an accusation. The method has an inherent error rate (≈10.6% EER at 8 cells), and quantized deployments, silent model updates, or stale reference fingerprints can all shift distributions. Treat it as one piece of evidence and re-test before drawing conclusions.

Some endpoints do not allow cross-origin (CORS) browser requests. That alone does not mean anything is wrong — the tool detects this and gives you an equivalent curl command so you can probe the endpoint from your own terminal instead.

Before sampling, the tool probes which reasoning-disable parameter your endpoint accepts (OpenRouter's reasoning, Zhipu's thinking, or OpenAI's reasoning_effort style). If none works, it falls back to a post-reasoning channel with a larger token budget and clearly marks the result as lower confidence.

Privacy

Everything runs in your browser: the API key lives in browser memory only, probe requests go directly from your browser to your endpoint — never through any Tosea server — and there are no analytics. The code is open source and auditable. Open source on GitHub — audit the detection engine line by line →