iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
A reasoning model that appears to return a bad or empty answer may not have produced a bad answer at all. Your application may be treating an incomplete response, a refusal, or output that is valid JSON but does not meet your needs as if it were a complete, usable result. Inspect the response state before parsing, handle refusals separately, and choose an output mode that matches the guarantee your code needs.
Why the parser can make a good response look bad
A JSON parser answers a narrow question: can this text be read as JSON? It does not tell you whether generation finished, whether the model refused, or whether the result satisfies your application’s business rules.
These are separate checks. A response can be interrupted before it forms a complete object; a refusal is a distinct outcome rather than the expected business object; and syntactically valid JSON can still be unsuitable for the task. If the application collapses those cases into a single parse error or empty-result state, the symptom points at the parser even when the underlying issue is completion, refusal handling, or schema fit.
JSON mode and Structured Outputs make different promises
OpenAI distinguishes basic JSON formatting from Structured Outputs. JSON mode is intended to produce valid JSON, while Structured Outputs is designed to match a supplied schema. OpenAI recommends Structured Outputs when it is supported for the use case and the application needs schema conformity. Neither option removes the need to inspect response state or check application-level meaning.
#1 Best Overall
| Approach | What it is intended to guarantee | What your application still needs to handle |
|---|---|---|
| JSON mode | Basic JSON-format output | Incomplete output and other documented edge cases; valid JSON alone does not enforce your supplied schema. |
| Structured Outputs | Output matching a supplied schema under the documented conditions | Refusals and interrupted output; schema conformity does not establish that the values satisfy every application requirement. |
OpenAI’s Structured Outputs announcement describes reliable schema-matching JSON under specific conditions: the response does not include a refusal, and generation has not been prematurely interrupted, as indicated by finish_reason. That qualification matters. It is not a promise that every response, every interruption, or every provider’s structured-output feature will produce a usable business object.
Diagnose the response in this order
- Inspect the raw response envelope and completion state. Before parsing content, determine whether the provider reports a normal completion or an incomplete response. Do not pass content marked incomplete to a parser as though it were a finished object. In OpenAI examples, JSON-mode handling calls out length truncation and content filtering as cases the application should detect.
- Check for a refusal outcome. Structured Outputs exposes a refusal signal that an application can detect and handle conditionally. Route that outcome through refusal handling instead of trying to coerce its content into the expected business object.
- Parse only completed, non-refusal content. Use the SDK parsing helper or JSON parser appropriate to the API. SDK helpers can turn structured output into an object, but they do not replace checks of the response state or guarantee that application-specific conditions are met.
- Validate requirements beyond the schema. A schema can constrain shape and types, but your application may have additional rules about acceptable values or whether the result is usable. Apply those checks after parsing, and treat failure as validation failure rather than silently substituting an empty result.
- Review the schema and output budget. Make important keys intuitive and provide clear descriptions for them. Ensure the output limit can accommodate the whole generation: for reasoning models, OpenAI notes that generated-token totals can include reasoning, visible output, and non-visible formatting tokens.
Use the failure pattern to find the likely cause
- The JSON parser reports malformed or unfinished content: inspect completion and interruption details first. A response cut off at its output limit may be incomplete, even if its beginning looks like the expected object.
- The SDK cannot parse the response into the target schema: check for a refusal or interruption before changing the schema. Both are documented cases in which content may not match the expected structure.
- The response parses but the application gets an empty or unusable result: inspect what the application does after parsing. Confirm that it reads the right fields and validates the values it actually needs; syntactic validity is not semantic correctness.
- The model repeatedly misses a field or produces an awkward structure: clarify key names and descriptions, reconsider whether the schema fits the task, and evaluate the structure against representative inputs. OpenAI recommends clear, intuitive keys, descriptions for important keys, and evaluations to determine whether the structure works for the use case.
Budget for the whole generation, not just the visible answer
For reasoning models, the visible JSON is not necessarily the only token use counted toward generation. OpenAI’s reasoning guidance says total generated tokens can include reasoning, visible output, and non-visible formatting tokens. An output limit sized only for the expected object can therefore leave too little room for completion. Treat an incomplete response as a failed generation to handle or retry according to your application’s policy, not as a complete object to parse.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Check provider-specific behavior in the selected API
Anthropic also documents structured JSON output, but provider features should not be assumed to share identical schema subsets, refusal signals, or interruption semantics. Check the current documentation for the specific provider and API you use before implementing those branches. The OpenAI-specific examples and response signals above should not be generalized to other providers.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
