Astra turns reasoning effort from a mostly hidden model behavior into an explicit API control. Developers can choose among five levels: low, medium, high, xhigh, and max.
That choice is a product trade-off, not a free quality setting. More permitted reasoning may help on some tasks, but it can also affect response latency and usage cost. The source defines the five levels; it does not publish comparative performance results showing how answer quality, speed, or cost changes between them. Teams therefore need to treat the levels as configurable operating points and measure their effects on their own workloads.
What the reasoning-effort control changes
A reasoning-effort parameter changes the amount of internal deliberation a request permits or requests before the model returns an answer. At a lower setting, the system may spend less effort working through alternatives, intermediate steps, or consistency checks. At a higher setting, it may allocate more effort to those steps. The API exposes this as a request-level operating choice rather than leaving the behavior entirely implicit.
That makes reasoning effort different from model selection. Choosing a different model can change capabilities, training, context handling, and output behavior. Choosing a different effort level typically keeps the model choice fixed while changing how much work it is asked to apply to a request. It is also different from prompt quality: a clearer prompt can reduce ambiguity and improve the task specification at any effort level, while a higher setting cannot repair missing requirements or a poorly defined objective.
Nor is the setting a deterministic guarantee. It does not guarantee a particular chain of reasoning, answer quality, latency, token count, or failure rate. Two requests at the same level can still differ substantially because their inputs and difficulty differ, and repeated requests may not produce identical results. A higher setting should therefore be treated as an available operating point, not as a universal upgrade. It may help on tasks that benefit from additional deliberation, while adding cost or delay without meaningful benefit on simpler work. Only workload-specific evaluation can establish where that trade-off is worthwhile.
Treat the Levels as Operating Choices
The five levels are best treated as operating points, not as a universal ranking of answers.
- Low fits simple, latency-sensitive work: straightforward transformations, short answers, routine routing, or tasks where a quick response matters more than extended deliberation.
- Medium is a reasonable general-purpose default when workloads vary and the application needs a balance between response time and potential reasoning depth.
- High is appropriate to test for more involved analysis, multi-step instructions, or code and content tasks where a shallow response is more likely to miss important constraints.
- Xhigh is for especially demanding reasoning. It may be worth evaluating when the task has substantial dependencies, ambiguity, or a higher cost of mistakes, provided the added latency and resource use fit the workflow.
- Max should be reserved for cases where additional effort is worth testing against the application's requirements. It is not automatically the best choice, and its value should be established on representative tasks rather than assumed from the label.
These assignments are starting points, not guarantees. A simple-looking task may still benefit from a higher setting, while a complex task may not improve enough to justify the additional expense or delay. Teams should validate the choice against their own quality criteria, latency budget, and cost constraints.
Put the Control at the Product Layer
A single reasoning setting is rarely appropriate for every request in a product. Routing workloads to different levels lets teams spend deliberation where it can affect the outcome, while keeping routine paths responsive and economical.
- Interactive assistants: Low may fit short factual answers, simple transformations, and conversational turns where latency is visible to the user. Medium can be a better default when requests require modest synthesis. More effort may be justified for complex planning, but only if the added wait improves task success in testing.
- Batch classification and extraction: Low or medium may be sufficient for well-defined schemas and stable inputs. Higher settings could help with ambiguous documents or borderline cases, but the added processing cost should be compared with alternatives such as confidence thresholds, retries, or human review.
- Code generation: Medium or high can be evaluated for tasks involving multiple files, constraints, or unfamiliar APIs. The relevant measures are not just whether code is produced, but whether it builds, passes tests, meets requirements, and requires fewer repair cycles.
- Research workflows: High, xhigh, or max may be worth testing when requests require decomposing a question, reconciling sources, or producing a detailed analysis. Higher effort does not replace source verification or domain review; it only changes how much deliberation the model is allowed or asked to use.
- High-stakes review: Teams may reserve higher levels for outputs that receive expert validation or support consequential decisions. The setting should be evaluated alongside error severity, escalation rules, and review coverage rather than treated as a safety guarantee.
The implementation pattern is a routing policy, not a universal ranking. Classify the workload, assign an effort level based on its quality requirements and latency budget, and measure the result. Different products—and even different paths within one product—may arrive at different choices. Each choice needs validation on representative inputs, because the useful trade-off depends on the task, failure modes, and cost of being wrong.
Make reasoning effort measurable
Start with a documented default. State which level new workloads use, why it was chosen, and what would justify changing it. The default should be an operational policy, not an unexplained value buried in application code.
Expose reasoning effort as configuration. Keep it in request settings, routing rules, or deployment configuration rather than hard-coding a level into prompts. That makes experiments reversible and lets different workloads use different operating points without rewriting prompt templates.
Log the selected level with every request or batch. At minimum, capture latency, token usage, and failures alongside the effort setting. Add human ratings or task-specific quality signals where possible: extraction accuracy, code-test pass rates, resolution rates, review outcomes, or another metric that reflects the actual job. Aggregate results by workload, not only across the entire application; a setting that helps code generation may be wasteful for classification.
Test representative workloads before changing production defaults. Include easy and difficult cases, normal traffic, long inputs, malformed requests, and known failure modes. Compare levels under the same evaluation conditions and track quality, latency, resource use, and failure behavior. If the evidence is mixed, keep the current default and route only the workloads that benefit from more effort. Production changes should follow measured results, not the assumption that a higher level is automatically better.
What the Five Levels Do Not Tell You
A five-level API does not imply that answer quality improves monotonically from low to max. More permitted deliberation may help on some tasks, have little effect on others, or introduce different failure modes. The setting is an operating point, not a quality ranking.
The API also does not state a latency or cost multiplier for each level. A higher setting may consume more time or resources, but developers should not assume a fixed relationship—or assume that every request at the same level has the same runtime or token usage. Those effects need to be measured in the application’s actual serving environment.
Nor is there a universal mapping between level and task difficulty. A task that appears simple may still benefit from additional reasoning, while a complex-looking request may be constrained more by missing data, ambiguous requirements, or tool failures than by deliberation. The level cannot replace model selection, prompt design, input validation, or domain-specific safeguards.
There is no substitute for testing. Since comparative performance results for the five levels are not published, teams should measure representative workloads themselves: task success or human-rated quality, latency, usage and cost, failure modes, and any downstream impact. Those results—not the names of the levels or the assumption that max is best—should determine production routing and defaults.
Treat Reasoning Effort as a Configurable Operating Point
Reasoning effort is moving from an implicit model behavior to an explicit product control. Astra’s levels—low, medium, high, xhigh, and max—give developers distinct operating points for deciding how much deliberation to request for a workload.
The practical choice is not “which level is best?” It is which level fits the task’s quality requirements, latency budget, and cost constraints. A higher setting may be appropriate when additional reasoning is valuable, but max is not automatically the right production default. The source defines the available levels; it does not publish comparative performance results that establish a universal ranking.
Treat the levels as configuration that can vary by workload, tenant, request type, or service tier. Match them to requirements, document the rationale, and validate the choice with application-specific tests. Over time, measurements from representative traffic—not assumptions about the label “max”—should determine whether a change improves the product.