An ordinary API call either worked or returned an error, and that is almost enough for debugging. An LLM call is trickier: the response can be valid JSON and still be completely wrong in meaning, and the bill keeps growing even on calls that succeeded. None of that shows up without a purpose-built log.
Why a separate logging layer
Regular application logs record that a call happened and what status code came back. That is not enough for LLM calls, because where a normal API either failed or worked, a model can quietly get it wrong inside a perfectly successful response.
A dedicated log around every model call does two jobs at once: it lets you reproduce a specific case for debugging, and it lets you work out where the money went without reconciling spreadsheets by hand at month end.
A request id on every call, no exceptions
The first thing to record is a unique request identifier. Without it there is no way to connect a user's complaint, a line in the application log, and the actual model call, once thousands of requests are flowing through per day.
A good practice is generating the id at the point the request enters the system and threading it through the entire path, from the user's action to the model call and back. One support ticket then turns into a full chain of events instead of "something went wrong somewhere."
Tokens and latency, per call
Record input and output token counts every time, separately from each other. This is not only about the bill. It is diagnostics. A sudden jump in input tokens often means extra context leaked into the prompt by accident. A jump in output tokens usually means the model started generating longer than usual, sometimes because a prompt or system instruction changed underneath it.
Log latency separately from the application's total response time too. A slow reply to the user can have several causes: the model itself, the network, a queue ahead of the call. A separate measurement shows immediately which one is actually at fault.
Attribution to a key and a user
The most useful part, and the one most often skipped, is tying every call back to whoever made it: which API key, which team, which user or feature inside the product. A total monthly bill answers "how much was spent." It does not answer "by whom, and on what."
Without that attribution, investigating a cost spike turns into guessing hypotheses one by one. With it, a breakdown is enough: you spot the specific key or feature that suddenly started making far more calls, or generating noticeably longer answers.
Claudexia already tracks usage per key, so issuing a separate key per feature or per team gets you this attribution for free, with no extra code on the product side. Sub-organizations with their own budgets and thresholds exist for exactly this: you see which team or product is approaching its limit before the spend becomes a problem.
What to log from the content, and what not to
The full prompt and response text is useful for debugging but not always safe to store as-is, especially if a conversation might contain a user's personal information. A reasonable compromise: write the full content to a separate store with restricted access and a short retention window, and put only a hash or a short excerpt in the main logs and metrics.
Call metadata, model, prompt version, tool name if it is an agentic flow, is safe and worth logging without restriction. It carries no personal data and still gives you almost everything needed to debug without touching the conversation content itself.
Turning logs into budgets and alerts
Raw logs on their own do not solve anything, what you do with them does. Three numbers worth computing regularly from them: total spend per period broken down by key, the share of calls with an abnormally high token count, and the share of failed calls that had to be retried.
From there it turns into thresholds: a notification approaching the monthly limit, an alert on a sudden latency spike, a flag on a key whose tokens-per-call ratio suddenly jumped. Claudexia already surfaces part of this picture at the key and sub-organization level, and the public status page settles the "is this us or the provider" question without an internal investigation.
A real-world example: a key issued to an email-sending feature suddenly starts generating messages three times longer than usual after someone changes the system prompt. Without a per-key breakdown, that only shows up at month end on the total bill. With one, an alert on the abnormal token jump fires within the day.
In short
The minimum log around an LLM call: request id, input and output tokens counted separately, latency, attribution to a key and a user. Store conversation content separately and with restrictions, log metadata freely. On top of that, money is easy to track and bugs are easy to find, instead of guessing from a total bill at the end of the month.