Model infrastructure needs enough evidence to answer operational questions: Did the request authenticate? Which public model contract applied? Was balance reserved? How long did delivery take? What was charged? None of those questions requires storing the prompt or generated answer.
01
Store the envelope, not the letter
OurToken records bounded request identity, customer and key references, public model ID, request surface, timestamps, lifecycle state, normalized meters, immutable price snapshots, and charge. Provider credentials, prompts, uploaded content, raw responses, and private routing details stay out of customer usage records.
This is more than hiding fields in the dashboard. Content-blindness is enforced at persistence boundaries so an accidental object spread cannot quietly turn a diagnostic table into a conversation archive.
02
Operational questions still have answers
Errors are normalized into small codes rather than storing raw upstream bodies. That makes alerting and aggregation useful while preventing an upstream error message from smuggling customer content into a durable log.
- Authentication failures can be traced to a key and timestamp.
- Billing disputes can be reconciled from meters and price snapshots.
- Stale requests can be repaired from lifecycle and reservation state.
- Route health can be monitored with bounded internal reason codes.
- Customers can export useful usage history without exporting content.
03
Attachments need an explicit boundary
Private Playground attachments belong in authenticated object storage, not analytics tables. Text projection is bounded and used only for the request that needs it. File metadata can support lifecycle and deletion without making the file publicly addressable.
Retention is a capability question too. A gateway should disclose the upstream retention properties of a model route rather than implying that its own content-blind database controls every copy beyond the boundary.
04
Debugging changes when content is unavailable
Engineers lose the convenience of reading a failed prompt from production. In return, the system is forced to invest in better state machines, deterministic identifiers, capability validation, bounded error taxonomy, and reproducible client-side test cases.
That trade is usually healthy. Observability becomes evidence about system behavior instead of a quiet collection of user data.