NNMQWeb Systems

AI cost control · measured, not estimated

Your AI bill is bigger than your token bill — and nobody is measuring the difference.

I run a production LLM router on my own hardware: local models first, cloud only when the task needs it, a hard daily cap, and a cache-aware cost model. The four numbers below are a dated sample (05–08 Nov 2026), not a promise that your account will match them. Each one is reproducible with the commands — on that log set, with the provider's own usage fields.

99.7%
Prompt cache hit rate (sample)
31.8×
Estimate vs real bill (sample)
−97.5%
Context after pre-digest (one run)
$0.0030
Real cost per agent turn (median, sample window)

99.7% — Measured per request on that sample. On DeepSeek a cached input token costs $0.003/1M, an uncached one $0.15/1M — ~50× more. Cache hit ratio often dominates the bill; your mix will differ.

31.8× — On that traffic, a list-price estimate overstated the actual charge 31.8×. Caps built only on estimates can stop you far too early — or never fire. Re-measure on your logs.

−97.5% — One measured compaction: 36,599 → 904 input tokens before send. Long agent sessions keep recent turns and replace older body with a digest; results vary by session shape.

$0.0030 — Median over a multi-day window on my agent loop (large tool schema). Same turn estimated ~$0.044 by a naive calculator. Peak hours and fat sessions cost more — I do not claim this is every day.

Why estimates lie

Agent traffic is mostly the same prompt again: a large tool schema plus a long conversation, resent every turn. Providers charge the repeated part at a cache-hit rate — on DeepSeek $0.003 per million tokens instead of $0.15. A calculator that multiplies all input by the list price is not conservative, it is wrong by an order of magnitude.

The consequences are practical, not academic. A $1/day cap set on estimates stops a team at about three cents of real spend. A router that ranks models by list price picks a short-context model for a 600k-token prompt, gets a 400, escalates to an expensive one, and pays for both. A product team concludes "AI is expensive" when the bill was mostly avoidable.

Three commands, three proofs

Set ROUTER_URL to your gateway and LOG_DIR to its log directory. These are the exact commands that produced the numbers above — requests are logged with the provider's own usage object, so the cache split is not my arithmetic.

1. Estimate vs real, from the gateway itself

curl -s "$ROUTER_URL/health" | jq '{
  est:  .stats.spentEstUsd,
  real: .stats.spentUsd,
  ratio: (.stats.spentEstUsd / .stats.spentUsd)
}'

→ est 0.2652 · real 0.0133 · ratio 19.9 (this box, 08.11.2026)

2. Cache split of a single call

tail -n 3 "$LOG_DIR/llm-router-debug.jsonl" | jq -c '{
  model, tools: .hasTools,
  in:  .usage.prompt_tokens,
  hit: .usage.prompt_cache_hit_tokens,
  out: .usage.completion_tokens
}'

→ {"model":"deepseek-v4-flash","tools":236,"in":631529,"hit":629760,"out":1422} → 99.7% cached

3. Daily budget state (example snapshot)

curl -s "$ROUTER_URL/api/state" | jq '.budzet'

→ {"limitUsd":1,"danasUsd":0.5638,"ostaloUsd":0.4362,"prekoracen":false} ← example from 08.11.2026 when the cap was set to $1; the limit is configurable

The 3-day audit

Fixed scope, fixed price, written output. You give read access to billing and to the request path (or a log export). I return the six items below.

Where the money actually goes

Per provider, per model and per task kind: input, cached input, output, retries, failed calls that were still billed. Cache hit ratio per route. No dashboards — one table you can act on.

Routing rules that match your traffic

Which requests must go to a cheap model, which need a long context, which must never touch an expensive one. The rule set is written as config, not as a promise.

A hard daily cap

Spend stops at the limit you set, with two behaviours: agent turns fail loudly (no silent no-op answers), plain chat degrades to a local model. The cap reads real usage, not estimates.

Context discipline

Pre-digest of long sessions, tool-schema bloat, cache-friendly prompt prefixes, output-token budgets. Usually the largest single lever after caching.

Secrets control

Keys out of repositories, logs and screenshots; what to rotate and in which order; a minimal policy your team can follow without slowing down.

Written report + patch-ready config

A document with numbers and reproductions, plus configuration diffs you can apply yourself. No lock-in, no subscription.

Terms

Format
3 working days, fully async — no meetings required
Price
agreed in writing per engagement, invoiced through a registered EU-time-zone company
Prerequisite
You already spend ≥ $300/month on LLM APIs, or run agents in production
Output
Written report, config patches, 30-minute recorded walkthrough
Not included
Rewriting your product, vendor negotiation, model fine-tuning

Book the 3-day audit

First reply is a one-page plan: what I will measure, what I need from you, and the exact date the report lands. If your spend is under $300/month, I will tell you not to buy it.

Numbers above were measured on a private router between 05.11.2026 and 08.11.2026, with DeepSeek V4.1 Flash (off-peak, $0.15 / $0.60 per 1M, cache hit $0.003) and a local Ollama fallback. They describe the method, not your account — your ratio will differ. The router itself is open for inspection: reference implementation and manifests on request.