Suppose you replace an analysis call with a classifier. It returns “feature request.” The next step still needs the requested feature and the customer’s goal. A correct label has left part of the original job unfinished.
To find a useful place for OpenAI’s Decisions API, we traced what LiteLedger’s analysis calls produce and inspected 217 saved message annotations. We were looking for removable work: which parts of the existing job could a decision replace?
A decision has a particular shape
The OpenAI Decisions API answers bounded questions: the probability that a condition is true, a choice from a predefined set, or a score against a rubric. Its guide points to Responses with Structured Outputs when an application needs its own extracted fields or written explanations.
Those answer shapes make classification, routing, and prioritization plausible uses. Whether they save work depends on what the rest of the application requires.
The classification call was also extracting evidence
LiteLedger analyzes conversations from AI products. In the pipeline we inspected, an early model call labels individual messages, records goals and mentioned features, and extracts quotations supporting signals such as frustration or success. Later calls use those annotations to summarize each session and write a report.
A label fits the Decisions output shape well. The evidence alongside it needs closer attention.
Consider this invented message to an AI website builder:
Could you add a contact form to this page?
A classifier might call this a feature request with neutral sentiment. An analysis report may still need the goal, “add a contact form,” and the feature, “contact form.” An emotionally quiet message can contain useful product information.
A predefined choice can select a feature if the application already supplies the right candidates. Our existing annotation format accepts open-ended feature names and goal descriptions. Replacing that format with a fixed vocabulary would be a separate design change, with its own evaluation.
We also checked the steps around the model calls. The initial human-versus-machine message split already uses rules. Later report writing needs generated explanations. Neither offered an obvious classification call to swap out.
What 217 saved annotations showed
We inspected 217 eligible saved message annotations across 20 customers. Of those annotations, 180 contained no quotation in any of the four sentiment-signal fields. Counting only those fields would make most of the sample look skippable.
The goal and feature fields changed that picture.
| Saved annotation contained | Count | Share |
|---|---|---|
| At least one sentiment-signal quotation | 37 / 217 | 17.1% |
| A stated goal | 212 / 217 | 97.7% |
| At least one product feature | 210 / 217 | 96.8% |
| Any quotation, goal, or feature | 217 / 217 | 100% |
Every annotation in this sample contained something beyond the typed labels. A gate that skipped extraction only when all those fields could remain empty would have skipped none of them while preserving the saved fields.
How we selected the sample. On October 22, 2026, we used the latest annotation version available in a local historical snapshot. We required a recognized message type and complexity value, plus a sentiment score within the schema’s range. We selected the 20 most recently annotated customers with at least five eligible annotations, taking up to the latest 100 eligible annotations per customer. This yielded 217. The source conversations dated from July 23 to August 20, 2026; the annotations were written from August 25 to September 1.
This convenience sample contains model-generated annotations from an older pipeline version. It lets us check which saved fields a replacement would need to preserve. The percentages describe field presence in these records.
The cost of keeping both calls
A simple cost model makes the dependency visible. Let D be the cost of screening a message, G the cost of its existing extraction, and f the fraction that still needs extraction after screening. For a gate that leaves the extraction call unchanged, the new average cost is:
D + f × G
It saves money when that total is below G. If every message still needs extraction, the screen adds cost. In a sequential workflow, it also adds a dependency before the original work can start.
Batching, caching, concurrency, and billing determine the actual cost. The architectural requirement remains: screening has to remove enough downstream work to cover its own cost.
A different output contract could create that room. Shorter annotations, supplied candidate text spans, or extraction inside a later call would each change the work required downstream. Report quality would be part of evaluating any of those designs.
Choosing what runs next
Background prioritization gives a decision a smaller job. When a fixed analysis budget cannot refresh every report, a small decision could help choose where the next full analysis is most useful. This is the design we would test first.
For the website-builder example, imagine new messages describing repeated failed attempts to publish a page. A screen could estimate whether those newly arrived human messages warrant earlier attention. The full analysis would still establish what happened, preserve the evidence, and update the report.
In that experiment, we would keep explicit analysis requests ahead of background work, retain a path for older reports to get refreshed, and fall back to the current ordering when screening fails or is uncertain. The full analysis would remain responsible for updating the report’s freshness.
The test would compare prioritization against the current ordering under the same total budget, including screening costs. Human-labeled examples would help measure missed urgent cases and false alarms. We would also track time to a useful completed analysis and how long low-ranked reports wait. A faster screen would count for little if it repeatedly postponed important findings.
Our annotation format makes the boundary concrete: labels sit alongside goals, features, and quotations. A replacement has to account for all of them. Background priority gives a decision one bounded choice: which full analysis should get the next slice of a limited budget. Finding removable work starts with the next step: what can the pipeline stop doing, defer, or bring forward once the decision arrives?