The token bill arrives, and nobody can say which conversations cost what — or why a two-line summary ran on your most expensive model. Adoption is finally up. So is the invoice, and it's harder to forecast every month.
The real question
Frontier models are priced per token, and the meter runs at the premium rate whether the task needed that power or not. Summarising a memo costs the same rate as deep analysis. Multiply that by every employee, every day, and the invoice turns unpredictable. Worse, when it lands, nobody can say which conversations drove it — so you can't cut cost without guessing. Standardising everyone on one premium model is expensive; capping usage per person is blunt and resented. Neither answers the question finance is actually asking: what does this cost, and why.
How to do it
Stop routing everything to the most expensive model. Set routing so routine work — drafting, summarising, tidying notes — runs on cheaper models, and only genuinely hard problems reach a frontier model. Choose which models are enabled per team, and let people switch models mid-conversation when a task grows in difficulty, without losing the thread.
Then take some workloads off the token meter entirely. Point a team at a model endpoint you run yourself — an OpenAI-compatible server such as Ollama. Local models carry no per-token charge, so you only pay frontier rates when a task truly needs frontier capability. High-volume internal Q&A is a good first candidate.
What this looks like
In Pebble, this looks like: an AI lead at a financial-services firm sets a routing profile so everyday drafting and summarising run on a lightweight model, while a frontier model is enabled only for the analyst workspace that needs deep reasoning. For high-volume internal Q&A, they point Pebble at an OpenAI-compatible endpoint the firm hosts itself — those calls carry no per-token charge at all. A month later, the Usage and Logs dashboards show cost broken down per model, key and endpoint, and every employee has a self-service usage page. Premium-model spend has dropped to the workloads that actually earned it.
Why this holds up in a regulated business
- Organisations run the models they choose — API keys, AWS Bedrock, or centrally managed ChatGPT/Claude subscriptions — and switch models mid-conversation without losing the thread.
- Routing profiles and model routing controls are admin-set and apply service-wide; enabled models are managed per organisation and workspace.
- You can connect OpenAI-compatible local providers such as Ollama, with no stored credentials for providers that don't need them.
- PebbleObserve breaks down usage, cost and logs per model, key, MCP server and endpoint, with a self-service usage page for every user.
Two honest limits: Pebble connects to a local model endpoint you run — it doesn't ship or host the models themselves — and the cost figures in the dashboard derive from the pricing you configure, not directly from vendor invoices.
Where to start
Pick your highest-volume, lowest-stakes workload and move it off your premium model this week — either to a cheaper hosted model or a local endpoint. Then open the usage dashboard and watch that line fall. Decide which model does which job, and only pay frontier rates when you mean to.
Pebble Powered AI.
