Osegi
AI·2 min read

Shadow APIs: cheap GPT access that may not be GPT at all

Researchers tested unofficial resellers of GPT and Gemini access. Nearly half of the endpoints they checked did not behave like the model they claimed to sell.

A server rack with a cheap price tag attached, hinting at a counterfeit AI model hidden behind an API.
Image: ITmedia NEWS

Pay for GPT-5 through a cheap reseller and you may get something else. A new study of so-called shadow APIs found that 11 of 24 tested endpoints failed a check on whether the advertised model was really running behind them.

The paper, "Real Money, Fake Models: Deceptive Model Claims in Shadow APIs," comes from researchers at CISPA in Germany and the University of Waterloo in Canada. Shadow APIs are unofficial services that resell access to the official APIs of large language models. They are cheaper, easier to pay for, and work in regions such as China and Russia where some official access is restricted.

Who uses shadow APIs?

The team searched conference papers and GitHub and found 17 shadow APIs used in 187 academic papers. About 62 percent of those papers (116) were accepted at peer-reviewed venues such as ACL, CVPR and ICLR. Roughly 80 percent of the researchers involved are at Chinese institutions. Fifteen of the 17 services are run by individuals whose identity can't be verified, and four have already shut down.

Do shadow APIs perform like the real thing?

Often not. The team tested three popular services on eight models from OpenAI, Google and DeepSeek. Tasks that need no reasoning scored close to official levels. Reasoning models did worse. On the AIME 2025 math benchmark, one service lost 40.00 points for Gemini-2.5-pro and 38.89 points for DeepSeek-Reasoner.

On the MedQA medical benchmark, Gemini-2.5-flash scored 83.82 percent through the official API. All three shadow APIs dropped it to about 37 percent. It also fell sharply on the LegalBench law benchmark. Safety tests were erratic: in one case harmfulness scores were five times the official figure, in others far lower.

How do you know a model was swapped?

The researchers used fingerprinting, which identifies a model from its response patterns, plus statistical tests on outputs. Some endpoints sold as GPT-5 or GPT-4o-mini matched the fingerprints of cheap open-source models GLM-4-9B and Qwen2.5-7B. Requests to DeepSeek-Reasoner, which has a thinking mode, behaved like DeepSeek-Chat, which doesn't.

Fingerprints aren't the whole story. A Gemini-2.5-flash endpoint matched its fingerprint yet lost about 47 points on the medical test and 40 to 43 on the legal one. For anyone building on these services, or citing results from them, the model name on the invoice is not proof of what answered.

Apps worth a look

SUMO: the number puzzle

Tap the right tiles, add them up, hit the target. Easy to learn, hard to master. Free, for all ages.

Tags:shadow apigpt-5geminillmcispadeepseek

More from this day

See all →

More in: AI

See all →