Financial observability for an AI stack: a single monitoring circuit for external-provider balances

Context

A production circuit that seriously relies on external AI services quickly accumulates paid APIs: LLM request routing, image and video generation, speech synthesis, telephony, avatars. Each provider has its own dashboard, its own pricing model, and its own notion of what a "balance" is: for some it's credits, for others a quota, for others postpaid with no remaining balance as such. A single view of spending simply doesn't exist: to understand how much money is left for production, the operator has to walk through a dozen dashboards by hand.

This routine has two expensive failure modes. The first is a sudden zero: a generative pipeline hits an exhausted provider balance mid-render, and the production chain stalls until a manual top-up. The second is unnoticed overspend: without per-day deltas it's invisible which service started spending abnormally, and the question "where did the money go" arises after the fact, on seeing the bill.

An additional problem was infrastructural: provider keys had historically scattered across the env files of different studio projects. Each pipeline kept its own copy, there was no single source of truth — for monitoring, for rotation, or for auditing which keys are even alive versus which long ago turned into placeholders.

The task

Build a single point of financial observability for the AI stack: a tool that polls all of the company's paid providers itself, brings the balances into one screen, and warns about a critical drop before it stops production.

The requirements were framed strictly. First: the tool must live where the operator lives — not a separate tab you have to remember to open, but a permanent macOS menu-bar element that auto-launches at system start. Second: fidelity — it may show only the balances a provider actually returns via API; for services with no public balance endpoint, honestly show a link to billing rather than a made-up number. Third: secret centralization — all keys collected into one secured store outside the repository, so monitoring becomes a side audit of their validity.

A separate requirement was extensibility. The studio's provider roster changes constantly, so adding a new service must not require reworking the app: a new provider is a key in the store, one poller function, and an entry in the registry.

Approach

The first stage was not development but review: each provider API was checked live for whether it returns a balance and in what form. The result of this review defined the system's honesty architecture. Three providers return a live balance: OpenRouter via /api/v1/credits, KIE via /api/v1/chat/credit, HeyGen via /v2/user/remaining_quota. ElevenLabs formally has a subscription endpoint but is geo-blocked — the request goes to a 302 redirect and requires a VPN. A telephony provider's balance endpoint is not confirmed by documentation and is polled best-effort. Anthropic, Groq, and Gemini have no public balance API at all — they're shown in the app as informational rows with a link to billing. Three more services turned out on inspection to have empty keys or not return a balance — and that in itself became a result of the audit.

This separation is a matter of principle: a monitoring system that shows unreliable numbers is worse than none. Each provider in the interface has an explicit data-source status: live API balance, best-effort, or informational link. The operator always knows which figures to trust.

In parallel, secrets were consolidated: keys scattered across the env files of several projects were collected into a single keys.env with chmod 600 permissions and excluded from version control. The store became the shared source of truth for monitoring and for the studio's other pipelines. The form factor was chosen deliberately native: a Python app on rumps lives in the macOS menu bar, packaged into a full .app bundle with LSUIElement=true — no dock icon, no window, just the menu row — and installed to auto-launch via a LaunchAgent with a single install-autostart.sh script.

Architecture

The system is decomposed into four modules with clear separation of responsibility. The app.py core handles the app's menu-bar lifecycle: auto-refresh every 10 minutes, a menu with balances and the time of the last update. The provider layer providers.py holds the poller functions and the PROVIDERS registry; it also runs as a standalone script for terminal checks, independent of the GUI. The history.py module keeps the accounting: current state in state.json, a change log in history.jsonl, from which spend deltas are computed. The dashboard.py module generates a standalone HTML page with an expanded spending picture.

The key performance decision is parallel polling. The fetch_all function collects all providers' balances via ThreadPoolExecutor, and the full cycle takes about 1.5 seconds instead of sequentially waiting on each network response. Just as important is decoupling threads inside the GUI: rumps, like any wrapper over a native UI, requires UI updates strictly from the main thread, while network requests on the main thread would freeze the menu for the duration of the poll. The scheme is built on two rumps.Timer instances — one launches the network cycle on a background thread, the other picks up the ready result and renders it on the main one. The interface never blocks, no matter how long the slowest provider takes to answer.

The history layer turns a snapshot of balances into a trend. Each poll cycle is compared with the saved state, the difference is written to an append-only history.jsonl log, and a spend delta appears next to each balance — you see not just "how much is left" but "how fast it's draining." The dashboard is built on the same data: dashboard.py generates a page with clean white cards and SVG trend sparklines per provider, opened straight from the menu. When a balance enters the low or empty zone, the app raises a macOS system notification — the signal arrives before the production pipeline stalls, not after.

Product packaging completes the picture: the app is built as "API Balance.app" with an Info.plist, a launcher, and its own icon drawn programmatically by a separate gen_icon.py script via NSImage.lockFocus — with no design source files or binary artifacts in the repository. The tool looks and behaves like a native product, not a script launched in a terminal.

Provider-layer engineering

The main difficulty of aggregators like this isn't the GUI — it's the zoo of vendor APIs. None of the providers follows a common standard: authorization schemes, response formats, units, and the very semantics of a balance all differ. The provider layer isolates this chaos: each service is described by a separate poller function that knows its endpoint and format and returns a unified balance representation outward. The app core knows nothing about the providers' differences.

From this isolation follows cheap extensibility — proven in practice, not merely claimed: the full cycle of adding a new provider is three steps — a key in keys.env, a poller function in providers.py, an entry in the PROVIDERS list. Neither the core, nor the history, nor the dashboard changes.

A separate piece of engineering hygiene is degradation without crashes. The ElevenLabs geo-block, the unconfirmed telephony endpoint, the absence of an API at Anthropic, Groq, and Gemini — all of these are framed as normal states of a provider's row, not as app errors: an unavailable service shows its status and a billing link, the rest keep refreshing. Monitoring cannot afford to crash because one of ten sources is down.

Result

The tool is in daily use: balances across every paid AI provider the company uses are brought into one menu-bar spot, refreshed every 10 minutes and polled in ~1.5 seconds. Manually walking the provider dashboards is fully removed from the operational routine — the registry covers 8 providers, and every one of them is either polled live or shown with an honest billing link, so the single menu-bar row replaces the entire round of separate dashboards.

The "pipeline stalled on a zero balance" failure mode is closed architecturally: a macOS low-balance notification arrives before exhaustion, and spend deltas and history sparklines make abnormal spending visible the day it happens rather than at the end of the billing period. The append-only history.jsonl log preserves the full spending trail, so any anomaly can be traced back to the exact poll cycle where it started.

The side results turned out to be just as valuable. Consolidating keys into a single secured keys.env brought order to secrets across the whole studio circuit, and the review of provider APIs gave an honest map of which services are even suitable for automated financial monitoring — knowledge directly reusable in client projects where controlling AI-infrastructure costs comes up. The case itself is a demonstration of the studio's approach: even an internal tool is taken to the level of a native product with auto-launch, data history, and meaningful degradation, rather than left as a script in a terminal.

What we built

• Native menu-bar app

  Python + rumps, built into an "API Balance.app" bundle with an Info.plist (LSUIElement=true — no dock or windows), its own icon.icns and a launcher; auto-launch via a LaunchAgent, installed with an install-autostart.sh script.

• Parallel provider layer

  providers.py: poller functions for each service and a PROVIDERS registry; fetch_all polls all of them via ThreadPoolExecutor in ~1.5 seconds and runs standalone from the terminal for a GUI-free check.

• Network/UI thread decoupling

  Two rumps.Timer instances: the first runs the network cycle on a background thread, the second renders the ready result on the main one — the menu never blocks, even on a slow provider; auto-refresh every 10 minutes.

• History and spend deltas

  history.py keeps state.json and an append-only history.jsonl log; comparing cycles computes spend deltas shown next to each balance.

• HTML dashboard with sparklines

  dashboard.py generates a standalone page with white cards and SVG trend sparklines per provider; opened straight from the app menu.

• Critical-balance notifications

  When a balance enters the low or empty zone, the app raises a macOS system notification — the signal arrives before the production pipeline stalls.

• Single secret store

  Keys scattered across different projects' env files are consolidated into keys.env with chmod 600 and outside version control — a single source of truth for monitoring and the other pipelines.

• Programmatic icon generation

  gen_icon.py draws the app icon in code via NSImage.lockFocus — with no design source files or binary artifacts in the repository.

Engineering challenges