lm-detector: reading the integer fingerprint of a hidden model

A task with no correct answer, only a stable wrong one

The problem lm-detector attacks is substitution: a relay or third party API advertises one model and quietly routes requests to another. The trick it uses is deliberately unrelated to language ability. The detector asks the model to write roughly three hundred integers between 1 and 355 on its first reaction, with no tool and no reasoning scaffold. Each answer must contain at least 80 valid integers to count.

The premise is that a language model asked for random numbers does not produce a uniform distribution. Every model family has preferred values, favoured ranges and a characteristic preference for last digits, and those habits stay stable across separate calls. The written integers become a fingerprint, and the detector compares that fingerprint against a reference bank to name the closest candidate. Nothing about the task tests writing quality, and nothing in the output can be graded against a right answer, because the whole point is the shape of the mistakes.

Three wire protocols and a proxy you may or may not need

Because the target is an endpoint rather than a chat partner, lm-detector speaks three API shapes: OpenAI Responses, Chat Completions and Anthropic Messages, with Server-Sent Events streaming as the default response mode. The web UI at lm.ikale.io offers two entry points, manual paste and a direct API connection, and the fpd command line tool adds a live terminal interface, multi round detection and JSON output for scripting.

The transport is also a choice. By default requests are relayed through the site's own proxy. An automatic mode lets the browser call the provider directly when the provider permits cross-origin requests, falling back to the site proxy when CORS blocks it. You can also force a fixed direct connection, or deploy your own Cloudflare Worker proxy, which the documentation links as a one click deployment for people who would rather not send traffic through a third party.

The repository is a Bun monorepo with web, cli, shared and docs workspaces, plus api, data, worker, functions and offline directories at the top level, and the project itself is written in TypeScript.

Running a first detection from the terminal

No installation step is needed for a quick look, because the CLI ships on npm and runs through the runtime you already have. Point it at a base URL, an API key and the model name you believe the endpoint is serving, and it will drive the integer task itself:

# Node.js ≥ 22
npx lmfpd@latest -b https://api.example.com/v1 -k sk-xxx -m gpt-6-astra

# Bun ≥ 1.4.2
bunx --bun lmfpd@latest -b https://api.example.com/v1 -k sk-xxx -m gpt-6-astra

The base URL, key and model name are supplied as flags, so the same command works against a provider endpoint, a relay you suspect, or your own gateway. The same three values feed the web UI, which is the better place to start if you want to watch the per candidate confidence rather than read JSON.

Working on the project itself is a Bun workflow, and the package scripts expose the full chain from data sync through type checking to the CLI bundle:

The detect, sample, enroll and retrain scripts all invoke cli/fpd.ts, so one entry point covers detection, sampling, enrollment and retraining.

Installing dependencies is a single command:

bun install

The reference bank is closed, so every model gets a winner

The documentation is explicit about the central limitation: a detection result is a ranking inside a closed set drawn from the reference bank. The bank covers common families including GPT, Claude, Gemini, Grok, Qwen and DeepSeek, and the web UI lets you browse and export it read only. A model that is not in the bank still receives a most similar candidate, so a confident answer can still be wrong when the true model sits outside the covered set. The ranking score and the confidence figure are not proof of identity.

That constraint changes how you should read the output. The useful signal is the shape of the ranking, and the way it moves when you change one variable. If the top candidate stays stable across several rounds and across two different request shapes, that is a data point about your channel. If the leaderboard reshuffles between rounds, the endpoint is more likely to be routing across a pool of backends than serving one fixed model.

The project also states the operational advice plainly: keep request parameters fixed, repeat the detection over multiple rounds, and combine the result with other evidence rather than treating the top row as a verdict.

Tokenizer probing answers a different question

An optional probe looks at the tokenization rules of the upstream endpoint instead of the integers. Turn it on in the web API configuration, or add the --tokenizer flag on the command line, and the detector sends roughly twelve additional short requests. It reads the usage figures returned by the interface, identifies the tokenizer that produced them, and compares that tokenizer against the model name you supplied.

The ceiling here is stated in the same documentation: the result is advisory only. Ranking and confidence come from the integer fingerprint alone, and the probe does not block or delay the detection. The reason is straightforward once you look at the tokenizers themselves. Many models share one tokenizer, so a matching tokenizer cannot establish which model answered. It can, however, disprove a claim, since a channel that reports a tokenizer family that the named model does not use has told you something useful.

The distinction matters when you write up findings. Tokenizer agreement and fingerprint agreement are two separate signals with two separate failure modes, and combining them without noting which one fired tends to overstate confidence.

Sampling, enrolling and retraining leave an audit trail

Keeping the reference bank honest is a three command pipeline. fpd sample collects new fingerprint samples from a model, fpd enroll takes those samples into the bank, and fpd retrain performs offline retraining. Failed records and earlier attempts are retained rather than overwritten, so the history of how a candidate profile was built stays inspectable.

This is the part of the design that makes the tool usable for a team. A fingerprint check is only meaningful if the reference data behind it can be explained, and a bank that silently drops failed samples cannot be audited when a result disagrees with what the vendor claims. The repository separates the data directory at the top level from the web and CLI workspaces, and the documentation includes a reference section covering the data files and their change records alongside the proxy API.

Cost of running your own copy comes down to a Bun toolchain, a static web build and, if you want it, a Cloudflare Worker for the proxy. The deployment pages cover Cloudflare Pages, GitHub Pages, Vercel and a self hosted Worker proxy, so the whole stack can sit under your own control.

What the reference bank does not tell you

Several things a reader might hope for are simply not addressed. The documentation does not describe a way to prove identity, and the project positions its own output as a reference ranking rather than a certificate. It does not document what happens when the same endpoint serves several models behind a load balancer beyond the advice to repeat rounds. It does not describe a sampling interval or a schedule for refreshing bank entries, so how fresh a given model profile is has to be read off the reference data change records.

There are no GitHub releases for the project, so version history for the published CLI is not visible through the release feed, and the package version in the root package.json is 0.1.0. The last push landed on 2026-10-19, which places the code close to current work rather than a dormant archive, but with no release tags there is no way to map a published artifact to a specific commit.

The licence is MIT, so redistribution and commercial use of the code and the reference data are permitted under that licence, and the project credits xqy2006/ModelTrace for the algorithmic idea and some original data. Any legal reading of how that applies to a specific deployment is a question for your own counsel rather than for this review.

Editorial conclusion

Adopt lm-detector if you buy model access through a reseller or relay and need a repeatable sanity check on what you are actually paying for; skip it if you need cryptographic proof of model identity, since every answer is a closed-set ranking and an unregistered model still receives a winner. Before trusting a verdict, fix the request parameters, repeat the detection over several rounds, and confirm that the candidate appears in the reference bank at all.

Frequently asked questions

Official sources

1. Ikaleio/lm-detector on GitHub
2. Issues
3. License: MIT
4. Project website
5. README