iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
To convert free-form text into application data, define the record you need as a Pydantic model, have a compatible LLM API return data constrained by that schema, and validate the response in your Python application before using it. Structured output can enforce the response’s shape; it cannot prove that extracted values are true or faithfully reflect the source.
What the workflow does—and what it does not
An LLM can extract details from text such as an invoice, email, or support request. Pydantic gives your application a typed contract for those details, and can generate a JSON Schema representation for APIs that accept one. The resulting pipeline is:
- Define the target record and its validation rules in Pydantic.
- Use a provider-supported structured-output feature when available for the schema you need.
- Check the API outcome, parse the returned data as the Pydantic model, and apply domain-specific checks before acting on it.
The distinction matters: valid JSON is not necessarily data that matches your schema, and schema-conforming data is not necessarily correct. A response can contain a plausible but wrong invoice total while satisfying every type and field requirement.
Define the record in Pydantic
Start from what the downstream application needs, not from every detail that happens to appear in the source. Make required and optional values explicit, choose types that suit the data, and add constraints or enumerated values when they help reject invalid records. Field descriptions can clarify what should be extracted.
#1 Best Overall
from pydantic import BaseModel, Field
class InvoiceFields(BaseModel):
supplier: str = Field(description="Supplier named on the invoice")
invoice_number: str | None = None
total: float | None = None
schema = InvoiceFields.model_json_schema()
# Send the schema through a provider-supported structured-output interface.
# Parse and validate the returned data as InvoiceFields before using it.This is an illustrative model, not a complete invoice specification or a tested provider request. Pydantic documents model_json_schema() as a way to generate JSON Schema, and its official LLM article demonstrates a Pydantic model and generated schema for extraction: Pydantic JSON Schema documentation and Pydantic’s LLM integrations article.
Think carefully about optionality. If the source may omit a value, represent that possibility rather than encouraging the model to invent one. Likewise, a simple type such as float does not establish which currency an amount uses or whether it is a subtotal or final total. Add fields and application rules that preserve those distinctions when they matter.
Choose an output method that matches the API
When your provider and selected model support the schema you need, native structured output is generally the stronger option for controlling response shape. OpenAI distinguishes it from JSON mode: JSON mode is intended to produce valid JSON, while Structured Outputs enforce adherence to a supported supplied schema. Feature support depends on the provider, model, and API surface, so verify all three against the current documentation before implementation: OpenAI Structured Outputs guide.
Free tools Windows power users keep installed
One-click scans. No signup required.
| Approach | What it constrains | What to account for |
|---|---|---|
| Native structured output | Schema adherence, when the provider supports the selected schema and model. | The provider may support only a subset of JSON Schema. Handle refusal, incomplete responses, and application-side validation. |
| JSON mode | Valid JSON, not necessarily adherence to your particular schema. | Parse the response and validate it against your Pydantic model; a parseable object can still have missing or wrongly typed fields. |
| Prompt-only output | Only what the model follows from instructions; schema adherence is requested, not mechanically enforced. | Expect less reliability than native constraints in general, and validate every result. It may be an option when native support is unavailable or unsuitable. |
Function calling serves a different purpose: it connects model output to a tool or application function. A schema-shaped response intended for the caller is usually a response-format structured-output use case, rather than a tool call. OpenAI’s API documentation describes its current structured-output interface and Pydantic support; consult it for the chosen API surface and model’s requirements: OpenAI Structured Outputs guide.
Check and adapt the generated JSON Schema
A Pydantic model is not automatically a drop-in schema for every provider. Pydantic can generate validation and serialization schemas, which may differ when a type’s accepted input differs from its serialized representation. Select the mode appropriate to the data the model is expected to return, inspect the generated schema, and adapt it if the provider’s constrained-output feature supports only a subset of JSON Schema.
Do not assume that every Pydantic constraint, type, or schema keyword is supported by every constrained-decoding implementation. Verify provider-specific limitations and availability in the relevant documentation; these details vary and can change. Pydantic’s versioned 2.12 documentation explains schema generation modes: Pydantic JSON Schema documentation.
Validate the response before using it
Even with native structured output, parse the returned object into the intended Pydantic model. That gives your application an independent check that the data it receives meets its own contract. Then apply checks that the schema cannot settle: whether a value is supported by the source, whether fields agree with each other, and whether an extracted value is plausible for your domain.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
- Preserve uncertainty. Allow missing or ambiguous source values to remain missing instead of treating a required-looking field as permission to guess.
- Check meaning, not just types. A date can be syntactically valid but refer to the wrong event; an amount can be numeric but belong to a different total.
- Keep evidence when it matters. For consequential extraction, consider retaining source text or a supporting excerpt alongside a value so a later step or reviewer can inspect its basis.
- Apply cross-field rules. Validate relationships such as line-item totals and invoice totals in application logic when those relationships matter.
OpenAI warns that schema-constrained output can still contain mistakes in the values. Pydantic AI also documents testing and evaluation approaches for agent behavior; semantic extraction quality needs to be measured separately from whether a response parses: OpenAI Structured Outputs guide and Pydantic AI evaluations documentation.
Handle refusals and incomplete responses explicitly
A refusal or generation stopped by a token limit or another ending condition may not produce a usable schema-conforming object. Check the provider response’s refusal and completion state before attempting to consume the result. Treat those outcomes, and Pydantic validation failures, as distinct from successful extraction.
Rank #4
Choose an explicit policy for each failure: retry when appropriate, send the item for review, use a documented fallback, or return a clear failure to the caller. Do not silently accept partial output or convert a failed extraction into a record that appears complete. OpenAI documents refusal and incomplete-output considerations in its structured-output guidance: OpenAI Structured Outputs guide.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Evaluate extraction quality with real examples
A model that returns valid objects may still misunderstand documents. Build a test set representative of the inputs the application will see, with expected values for each field. Include incomplete, ambiguous, and difficult examples—not only clean documents—and compare extracted values with the source.
- Track schema or parsing failures separately from incorrect values.
- Measure field-level correctness, including whether the system leaves unsupported values unset.
- Review ambiguous cases to see whether the model preserves uncertainty or makes an unjustified choice.
- Re-run evaluations when changing the model, prompt, schema, or provider settings.
Pydantic AI documents unit tests and evaluations for agent behavior at Pydantic AI evaluations documentation. The key operational distinction is to measure both structural reliability and whether the extracted data is actually right.
How much confidence should you place in schema adherence?
OpenAI reported historical results in its 2024 announcement for its own complex JSON Schema-following evaluation: gpt-4o-2024-08-06 scored 100% with Structured Outputs, while gpt-4-0613 scored less than 40% on that evaluation. The announcement also reported 93% for the newer model before deterministic constrained decoding was added, saying that result did not meet its reliability needs.
These are vendor-reported results on a schema-following evaluation, not independent measures of general extraction accuracy or production success. They illustrate the value of constraining output shape, but do not show that a correctly shaped answer contains truthful values. OpenAI’s announcement provides the evaluation context: Introducing Structured Outputs in the API.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →
