2 April 2026
Token counts and the bill are not the same story
A product team can watch tokens rise slowly while the invoice jumps. The gap is usually context length, retries, or a second model that never appeared on the dashboard the team actually looks at.
Token counters inside an application are a useful pulse. They are not a substitute for the vendor’s billed tokens. We keep seeing Kuala Lumpur teams bring a Grafana screenshot that does not match the invoice by a wide margin, and the first hour of a usage window briefing is spent finding which tokens were never counted.
Retries are the usual missing piece. A timeout that the user never notices still bills the first attempt. If the client library retries twice, the invoice holds three completions and the in-app counter may hold one. The briefing note has to say so, or finance will think the application “wasted” tokens that were in fact a patience setting.
Context length is the second gap. A retrieval feature that began with short chunks and later started stuffing whole PDFs into the prompt will move the bill far more than the call count. Call count stays flat; billed tokens do not. The product owner who only watches calls will be surprised, and that surprise is avoidable if someone reads the prompt size distribution for the month.
A third model sometimes sits on the invoice under a different product code: a moderation pass, an embedding rebuild, a translation step. If the application team does not own that code, they will not look for it. We ask for the full vendor statement, not the filtered view the engineering lead bookmarked in January.
When we write the memo, we put billed tokens, counted tokens, and the named gap in one table. We do not average them. If the gap cannot be explained from the logs we were given, the memo says the logs are incomplete rather than inventing a percentage.