Technology
Decision API: Why Production Systems Need Typed Answers, Not Generated Text
Every production system eventually hits the same wall. On one side sits the rule engine that started as twelve readable conditionals and is now four thousand unreadable ones that nobody dares refactor. On the other side sits a chat model asked politely to “respond only in JSON,” wrapped in retry loops, regex fallbacks, and a pager rotation. Neither approach ages well. A decision API occupies the space between them: an endpoint that accepts state you already have and returns an answer with a fixed type, a calibrated confidence score, and a deterministic result.
This is written for the people who own that boundary — engineers wiring model output into services, product teams defining the thresholds that gate behavior, and ops teams who get paged when a score starts drifting. Below: what a decision API actually is, which output primitives matter, and where the pattern pays for itself.
What a Decision API Is, and Why Prompting a Chat Model for JSON Is Not One
A decision API is an HTTP endpoint that accepts state and a question, then returns an answer constrained to a schema you defined in code. State can be text, structured JSON, or up to sixteen images and video frames. The question is not a prompt in the conversational sense; it is a declaration — a choice among labels you supply, a score on an ordered scale, a numeric value on a grid, a location in an image. DecisionNode implements this as a single POST /v1/decide call that returns the typed answer in one pass. No multi-turn conversation, no tool-call choreography, no parsing ceremony.
The distinction matters more than it sounds. A general-purpose model is optimized to generate. When you ask it for structured output, you are negotiating with a generative process and hoping it complies. In practice that means output drift between model versions, occasional prose where a boolean should be, silent truncation, and a retry path that costs you latency and money every time the model decides to be helpful instead of terse. Every one of those failure modes surfaces as an incident, not as a unit test failure.
A decision API inverts the contract. Output type is a parameter of the request, not an instruction in the prompt. The response is schema-valid by construction, so validation happens once at the boundary instead of defensively at every call site. A service handling ticket routing can branch on a label directly; a scoring pipeline can compare a number to a threshold without coercion logic.
Two properties are worth calling out because they change how you test and deploy. First, determinism: the same request returns the same answer, which means you can snapshot decisions in CI, cache safely, and reproduce an incident from a logged payload. Second, latency and cost profile. DecisionNode Flash runs at roughly 5 ms server-side and 19 ms end to end at $0.035 per 1M input tokens; the MoE variant runs at about 9 ms at $0.042 per 1M input tokens. Output tokens are free on both, which matters when you are making millions of calls a day. For teams evaluating a decision api, the practical test is simple: can your code act on the response without an intermediate interpretation layer? If the answer is yes, you are looking at the right category of tool.
The Output Primitives: Choice, Score, Truth, Number, Point, and Box
The value of a decision API lives in how precisely it lets you express the question. Six primitives cover most production needs, and each maps cleanly onto code you already write.
Choice returns one of the labels you defined — billing, shipping, account access, other. This is a classification, and it plugs directly into a switch statement or a routing table. Score returns a level on an ordered scale, which is what you want when labels collapse the nuance you need: not “risky” but a position on a graded spectrum that your own thresholds interpret. Truth returns a calibrated probability between 0.00 and 1.00 — a number you can threshold, sort, aggregate, and tune economically. Number returns a value on a grid, for quantities where an interval estimate beats a guess. Point and box return locations inside an image, so a camera frame can answer “where” as readily as “what.”
Every answer carries a calibrated confidence, and calibration is the part most teams underestimate. A confidence of 0.90 should mean the answer is right roughly nine times in ten, not that the model felt good about it. Once that holds, confidence stops being decoration and becomes control flow. Code can act instantly above a threshold, and below it route to a second check, a stronger model, a human reviewer, or a safe default. You stop choosing between “block everything uncertain” and “let everything through” and start tuning a single dial against a labeled sample.
The primitives compose. A moderation flow might take sixteen video frames, ask for a choice on policy category, a truth on whether the content violates it, and a box around the offending region in the frame that triggered the decision. A quality-control station on a production line asks for a score plus a point on the defect. Boosters — human age estimation, AI-generated image detection, face detection and recognition, OCR, depth, and segmentation — extend the same endpoint to questions that would otherwise pull in a second vendor and a second SDK.
Throughput gets its own escape hatch. Above roughly ten decisions per second, DecisionNode exposes sessions over WebSocket rather than one-shot requests, which is the shape drones, robots, simulators, and machine-monitoring loops actually need. SDKs for TypeScript, Python, Go, Ruby, curl, MCP tools, and webhooks mean the integration is usually a few lines, and agents can call the same endpoint to decide before they act rather than after.
Where Decision APIs Earn Their Keep: Triage, Fraud, Moderation, and Agents
The pattern is easiest to justify wherever hand-written rules have become the bottleneck. Support ticket triage is the canonical case: routing depends on intent, customer tier, sentiment, and sometimes a screenshot attached to the ticket. A rules engine handles the first two and collapses on the rest. A decision API returns a priority choice, a routing choice, and a truth on whether escalation is warranted, with confidence attached to each so low-confidence tickets land in a review queue instead of bouncing between teams.
Fraud and promo-abuse checks follow the same shape but care more about calibration. A score on an ordered risk scale plus a truth probability on “is this abusive” lets a checkout flow make three distinct moves: approve, block, or step up verification. Because confidence is calibrated, the thresholds can be set against real loss data rather than intuition, and because responses are deterministic, an appeal or audit can replay the exact decision from the stored payload. Order and shipping decisions — hold, split, expedite, reroute — behave identically, with the added benefit that a failed decision returns a typed answer instead of throwing a parse error mid-fulfillment.
Visual workloads are where a single endpoint saves the most integration work. Content moderation consumes images and video frames and needs both a verdict and a location. Warehouse and traffic monitoring need per-frame judgments at rates that a request-and-wait pattern cannot sustain, which is what the WebSocket session model addresses. Production-line quality control needs a defect score with sub-second turnaround at the inspection station, and the ~5 ms server-side figure on Flash is the difference between sampling every hundredth unit and inspecting every unit.
AI agent builders get a different kind of value. An agent that reasons in natural language and then acts through tools has an unreliable middle: the step where it decides whether to proceed. Routing that step through a decision API turns an implicit judgment into an explicit, logged, threshold-governed call — proceed, ask for clarification, or stop. The agent keeps its flexibility; the guardrail becomes code that a platform team can audit and tune without rewriting prompts. For ops and automation teams replacing brittle scripts, the migration path is incremental: wrap one rule cluster at a time, compare the API’s typed answer against the existing conditional on live traffic, and retire the rule once the confidence distribution looks healthy.
Mexico City urban planner residing in Tallinn for the e-governance scene. Helio writes on smart-city sensors, Baltic folklore, and salsa vinyl archaeology. He hosts rooftop DJ sets powered entirely by solar panels.