LLM Structured Output Practical Guide — JSON Mode, Tool Use, Pydantic Schema Validation

• Introduction
• Structured Output Comparison by Provider
  • OpenAI: response_format + Structured Outputs
  • Anthropic: Structured Output via Tool Use
  • Google Gemini: responseSchema
• What Changed as of August 2026
  • Anthropic — Native Structured Outputs Is GA
  • OpenAI — The Payload Shape Differs by API
  • Google — There Were Two Migrations
• Automating Schema Validation with Pydantic
  • Basic Pattern
  • Retry Pattern (Self-Healing)
• Unifying Providers with LiteLLM
• Using the Instructor Library
• Building a Production Pipeline
  • FastAPI + Structured Output
  • Batch Processing Pipeline
• What Actually Happens When You Run It
  • The First Request Is Slow
  • Fields You Must Always Check on the Response
• End-to-End Example: A Support Ticket Classifier
  • 1. Define the Schema
  • 2. Call It
  • 3. What Comes Back
  • 4. The Same Schema on OpenAI Responses
• Schema Constraints — Support Differs by Provider
• Failure Modes — From Symptom to Cause
  • Latency Still Spikes, Even After a Warmup
  • 400 — Errors About additionalProperties
  • 400 — Using It Alongside Citations
  • JSON Parsing Fails Intermittently
  • Switched to LiteLLM and the Validation Vanished
• When Not to Use a Schema
• Conclusion
• References
• Quiz

Introduction

To programmatically process LLM outputs, structured formats (JSON, XML, etc.) are essential. Simply adding "respond in JSON" to a prompt is not enough — various issues arise such as schema mismatches, missing fields, and incorrect types.

In this article, we compare the Structured Output features of major LLM providers and cover how to use them reliably in production.

Structured Output Comparison by Provider

OpenAI: response_format + Structured Outputs

from openai import OpenAI
from pydantic import BaseModel
from typing import List, Optional

client = OpenAI()

# Method 1: JSON Mode (basic)
response = client.chat.completions.create(
    model="gpt-4o-2024-08-06",
    messages=[
        {"role": "system", "content": "Return your response as JSON."},
        {"role": "user", "content": "Recommend 3 famous restaurants in Seoul"}
    ],
    response_format={"type": "json_object"}
)
# JSON is guaranteed, but schema is not

# Method 2: Structured Outputs (schema guaranteed)
class Restaurant(BaseModel):
    name: str
    cuisine: str
    price_range: str
    rating: float
    address: str

class RestaurantList(BaseModel):
    restaurants: List[Restaurant]
    total_count: int

response = client.beta.chat.completions.parse(
    model="gpt-4o-2024-08-06",
    messages=[
        {"role": "system", "content": "Recommend famous restaurants in Seoul."},
        {"role": "user", "content": "3 Korean cuisine restaurants"}
    ],
    response_format=RestaurantList
)

result = response.choices[0].message.parsed
print(result.restaurants[0].name)  # Type-safe!

Anthropic: Structured Output via Tool Use

import anthropic
from typing import List

client = anthropic.Anthropic()

# Anthropic uses Tool Use for Structured Output
response = client.messages.create(
    model="claude-sonnet-4-20250514",
    max_tokens=1024,
    tools=[
        {
            "name": "extract_restaurants",
            "description": "Extract restaurant information in a structured format",
            "input_schema": {
                "type": "object",
                "properties": {
                    "restaurants": {
                        "type": "array",
                        "items": {
                            "type": "object",
                            "properties": {
                                "name": {"type": "string"},
                                "cuisine": {"type": "string"},
                                "price_range": {
                                    "type": "string",
                                    "enum": ["$", "$$", "$$$", "$$$$"]
                                },
                                "rating": {"type": "number"},
                                "address": {"type": "string"}
                            },
                            "required": ["name", "cuisine", "price_range"]
                        }
                    },
                    "total_count": {"type": "integer"}
                },
                "required": ["restaurants", "total_count"]
            }
        }
    ],
    tool_choice={"type": "tool", "name": "extract_restaurants"},
    messages=[
        {"role": "user", "content": "Recommend 3 Korean restaurants in Seoul"}
    ]
)

# Extract structured data from Tool Use result
tool_use = next(
    block for block in response.content
    if block.type == "tool_use"
)
restaurants = tool_use.input["restaurants"]

Google Gemini: responseSchema

import google.generativeai as genai

genai.configure(api_key="YOUR_API_KEY")

model = genai.GenerativeModel(
    "gemini-2.0-flash",
    generation_config=genai.GenerationConfig(
        response_mime_type="application/json",
        response_schema={
            "type": "object",
            "properties": {
                "restaurants": {
                    "type": "array",
                    "items": {
                        "type": "object",
                        "properties": {
                            "name": {"type": "string"},
                            "cuisine": {"type": "string"},
                            "rating": {"type": "number"}
                        }
                    }
                }
            }
        }
    )
)

response = model.generate_content("3 Korean restaurants in Seoul")
import json
data = json.loads(response.text)

What Changed as of August 2026

The provider examples above were written in March 2026. Most of them still run, which is why they are still here rather than deleted. But in the meantime two of the three providers changed their recommended path. What follows was verified against each provider's docs on 2026-08-16.

Anthropic — Native Structured Outputs Is GA

This is the biggest thing this article got wrong. Above I wrote that Anthropic uses Tool Use as a workaround. That pattern still works, but it is no longer necessary. The output_config parameter, which takes a schema directly, is generally available, and no beta header is required.

from anthropic import Anthropic

client = Anthropic()

response = client.messages.create(
    model="claude-opus-5",
    max_tokens=16000,
    messages=[{"role": "user", "content": "..."}],
    output_config={
        "format": {
            "type": "json_schema",
            "schema": {
                "type": "object",
                "properties": {
                    "name": {"type": "string"},
                    "plan": {"type": "string"},
                },
                "required": ["name", "plan"],
                "additionalProperties": False,
            },
        }
    },
)

The tool definition disappears, and so does the code that digs a tool_use block out of the response. If you use Pydantic there is one more helper. Pass a model class as output_format to client.messages.parse() and the validated instance arrives on response.parsed_output.

from pydantic import BaseModel

class Account(BaseModel):
    name: str
    plan: str

response = client.messages.parse(
    model="claude-opus-5",
    max_tokens=16000,
    messages=[{"role": "user", "content": "..."}],
    output_format=Account,
)

account = response.parsed_output  # a validated Account instance

There is one naming overlap that everyone gets wrong once. The keyword argument on the .parse() helper really is output_format, but the canonical parameter on .create() is format under output_config. The older top-level output_format parameter on .create() was superseded by it and is deprecated.

If you stay on Tool Use, note that strict is a top-level key alongside name, description, and input_schema — not something inside input_schema.

tools = [
    {
        "name": "extract_restaurants",
        "description": "Extract restaurant information in a structured format",
        "strict": True,  # top level, not inside input_schema
        "input_schema": {
            "type": "object",
            "properties": {
                "restaurants": {"type": "array", "items": {"type": "object"}},
                "total_count": {"type": "integer"},
            },
            "required": ["restaurants", "total_count"],
            "additionalProperties": False,
        },
    }
]

The difference that bites most often is required. Anthropic does not demand that every property be listed in required. OpenAI does.

OpenAI — The Payload Shape Differs by API

On the OpenAI side the accidents come from payload shape rather than features. Where the same setting goes differs between the two APIs.

# Chat Completions — json_schema nests one level deeper, name lives inside
"response_format": {
    "type": "json_schema",
    "json_schema": {
        "name": "person",
        "strict": True,
        "schema": {"type": "object", "properties": {}},
    },
}

# Responses API — flat under text.format, name is a sibling of type
"text": {
    "format": {
        "type": "json_schema",
        "name": "person",
        "strict": True,
        "schema": {"type": "object", "properties": {}},
    }
}

Copy an example, switch only the API you call, and the setting is either silently ignored or returns a 400. The official guidance is that Chat Completions remains supported but Responses is recommended for all new projects.

The SDK helpers were tidied up too. The client.beta.chat.completions.parse used in the body above does not exist on the current SDK. Use client.chat.completions.parse. The Responses equivalent is client.responses.parse, and the schema goes in as text_format. I could not confirm which release removed the beta path, so do not pin that claim to a version number — import it and check.

from openai import OpenAI

client = OpenAI()

# Chat Completions — here, not under beta
completion = client.chat.completions.parse(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "3 Korean restaurants"}],
    response_format=RestaurantList,
)
result = completion.choices[0].message.parsed

# Responses API — the recommended path for new projects
response = client.responses.parse(
    model="gpt-4o-mini",
    input=[{"role": "user", "content": "3 Korean restaurants"}],
    text_format=RestaurantList,
)
# The attribute that holds the parsed result varies by SDK version.
# Check the docs for the version you are on for the exact API.

JSON Mode is worth pinning down as well. The json_object type guarantees only valid JSON, not schema adherence, and it requires the word JSON to appear somewhere in the conversation. The official docs describe Structured Outputs as the evolution of JSON mode, but JSON Mode is not formally deprecated.

Turning on strict requires additionalProperties set to false and every property listed in required. Optional fields are emulated by making the type a union of the string type and null.

Google — There Were Two Migrations

Gemini is two generations behind here. The first migration is the SDK. The google-generativeai package used above is not actively maintained, and the legacy libraries were deprecated as of November 30th, 2025. What you install now is google-genai, and the package name and import paths all change. The second is the API. The Interactions API went GA in June 2026, and the structured output docs now lead with it.

from google import genai

client = genai.Client()

interaction = client.interactions.create(
    model="gemini-3.6-flash",
    input=prompt,
    response_format={
        "type": "text",
        "mime_type": "application/json",
        "schema": Recipe.model_json_schema(),  # a Pydantic model cannot be passed directly
    },
)

recipe = Recipe.model_validate_json(interaction.output_text)

The generateContent API remains fully supported but is now explicitly described as legacy, and the recommendation is to use the Interactions API for all new development.

There is a trap here too. On the Interactions path a Pydantic model cannot be passed directly, so you call .model_json_schema() to produce a dict. On the legacy generate_content path, by contrast, a BaseModel can be passed as-is.

from google import genai
from google.genai import types

client = genai.Client(api_key="...")

# Legacy path — still supported, and here a Pydantic model can be passed directly
response = client.models.generate_content(
    model="gemini-3.6-flash",
    contents=prompt,
    config=types.GenerateContentConfig(
        response_mime_type="application/json",
        response_schema=CountryInfo,
    ),
)

Some things I could not confirm, recorded as such. Whether response_schema and response_json_schema are separate coexisting fields is something the SDK docs site and the GitHub README disagree about, so I am not asserting it. The same goes for whether there is an attribute that hands you the parsed object directly. Check the docs for the version you are on for the exact API.

Automating Schema Validation with Pydantic

Basic Pattern

from pydantic import BaseModel, Field, validator
from typing import List, Optional, Literal
from enum import Enum
import json

class PriceRange(str, Enum):
    CHEAP = "$"
    MODERATE = "$$"
    EXPENSIVE = "$$$"
    VERY_EXPENSIVE = "$$$$"

class Restaurant(BaseModel):
    name: str = Field(..., min_length=1, max_length=100)
    cuisine: str = Field(..., description="Type of cuisine")
    price_range: PriceRange
    rating: float = Field(..., ge=0.0, le=5.0)
    address: Optional[str] = None
    tags: List[str] = Field(default_factory=list, max_length=10)

    @validator('rating')
    def round_rating(cls, v):
        return round(v, 1)

class RestaurantResponse(BaseModel):
    restaurants: List[Restaurant] = Field(..., min_length=1, max_length=20)
    query: str
    total_count: int

# Parse + validate LLM response
def parse_llm_response(raw_json: str) -> RestaurantResponse:
    """Parse LLM response and validate with Pydantic"""
    try:
        data = json.loads(raw_json)
        return RestaurantResponse(**data)
    except json.JSONDecodeError as e:
        raise ValueError(f"Invalid JSON: {e}")
    except Exception as e:
        raise ValueError(f"Schema validation failed: {e}")

Retry Pattern (Self-Healing)

from tenacity import retry, stop_after_attempt, retry_if_exception_type

class StructuredOutputParser:
    def __init__(self, client, model: str, schema: type[BaseModel]):
        self.client = client
        self.model = model
        self.schema = schema

    @retry(
        stop=stop_after_attempt(3),
        retry=retry_if_exception_type(ValueError)
    )
    def parse(self, prompt: str) -> BaseModel:
        """Retry with error message included on schema validation failure"""
        schema_json = self.schema.model_json_schema()

        messages = [
            {
                "role": "system",
                "content": f"Respond according to the following JSON schema:\n{json.dumps(schema_json, indent=2)}"
            },
            {"role": "user", "content": prompt}
        ]

        # Include error from previous attempt if available
        if hasattr(self, '_last_error'):
            messages.append({
                "role": "user",
                "content": f"An error occurred in the previous response: {self._last_error}\nPlease respond again with valid JSON."
            })

        response = self.client.chat.completions.create(
            model=self.model,
            messages=messages,
            response_format={"type": "json_object"}
        )

        raw = response.choices[0].message.content
        try:
            data = json.loads(raw)
            result = self.schema(**data)
            if hasattr(self, '_last_error'):
                del self._last_error
            return result
        except Exception as e:
            self._last_error = str(e)
            raise ValueError(str(e))

# Usage
parser = StructuredOutputParser(client, "gpt-4o", RestaurantResponse)
result = parser.parse("Recommend 3 Korean restaurants in Seoul")

Unifying Providers with LiteLLM

import litellm
from pydantic import BaseModel

class ExtractedInfo(BaseModel):
    summary: str
    key_points: list[str]
    sentiment: str
    confidence: float

# OpenAI
response = litellm.completion(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Summarize the Kubernetes 1.35 release"}],
    response_format=ExtractedInfo
)

# Anthropic (automatically converts to Tool Use)
response = litellm.completion(
    model="claude-sonnet-4-20250514",
    messages=[{"role": "user", "content": "Summarize the Kubernetes 1.35 release"}],
    response_format=ExtractedInfo
)

# Gemini
response = litellm.completion(
    model="gemini/gemini-2.0-flash",
    messages=[{"role": "user", "content": "Summarize the Kubernetes 1.35 release"}],
    response_format=ExtractedInfo
)

# Same code works across all 3 providers!

Using the Instructor Library

# pip install instructor
import instructor
from openai import OpenAI
from pydantic import BaseModel
from typing import List

client = instructor.from_openai(OpenAI())

class Step(BaseModel):
    explanation: str
    output: str

class MathSolution(BaseModel):
    steps: List[Step]
    final_answer: str
    confidence: float

# Use Pydantic model directly as response_model
solution = client.chat.completions.create(
    model="gpt-4o",
    response_model=MathSolution,
    messages=[
        {"role": "user", "content": "Solve 2x + 5 = 15"}
    ],
    max_retries=3  # Automatic retries
)

print(solution.steps[0].explanation)
print(f"Answer: {solution.final_answer}")

# Anthropic is also supported the same way
import anthropic

anthropic_client = instructor.from_anthropic(anthropic.Anthropic())

solution = anthropic_client.messages.create(
    model="claude-sonnet-4-20250514",
    response_model=MathSolution,
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Solve 3x - 7 = 20"}
    ]
)

Building a Production Pipeline

FastAPI + Structured Output

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from typing import List
import instructor
from openai import OpenAI

app = FastAPI()
client = instructor.from_openai(OpenAI())

class ProductReview(BaseModel):
    sentiment: str  # positive, negative, neutral
    score: float
    key_phrases: List[str]
    summary: str
    language: str

class ReviewRequest(BaseModel):
    text: str
    model: str = "gpt-4o-mini"

@app.post("/analyze", response_model=ProductReview)
async def analyze_review(request: ReviewRequest):
    try:
        result = client.chat.completions.create(
            model=request.model,
            response_model=ProductReview,
            messages=[
                {
                    "role": "system",
                    "content": "Analyze the product review."
                },
                {"role": "user", "content": request.text}
            ],
            max_retries=2
        )
        return result
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

Batch Processing Pipeline

import asyncio
from typing import List
from openai import AsyncOpenAI
import instructor

async_client = instructor.from_openai(AsyncOpenAI())

class ExtractedEntity(BaseModel):
    name: str
    entity_type: str
    confidence: float

class EntityExtractionResult(BaseModel):
    entities: List[ExtractedEntity]
    text_length: int

async def extract_entities(text: str) -> EntityExtractionResult:
    return await async_client.chat.completions.create(
        model="gpt-4o-mini",
        response_model=EntityExtractionResult,
        messages=[
            {"role": "system", "content": "Extract entities from the text."},
            {"role": "user", "content": text}
        ]
    )

async def batch_extract(texts: List[str], concurrency: int = 5):
    """Batch processing with concurrency limits"""
    semaphore = asyncio.Semaphore(concurrency)

    async def limited_extract(text):
        async with semaphore:
            return await extract_entities(text)

    tasks = [limited_extract(text) for text in texts]
    results = await asyncio.gather(*tasks, return_exceptions=True)

    successes = [r for r in results if not isinstance(r, Exception)]
    failures = [r for r in results if isinstance(r, Exception)]

    print(f"Successes: {len(successes)}, Failures: {len(failures)}")
    return successes

# Run
texts = ["A new AI startup in Seoul...", "Samsung Electronics semiconductor...", ...]
results = asyncio.run(batch_extract(texts))

What Actually Happens When You Run It

A request carrying a schema behaves differently from an ordinary one. Miss that difference and you mistake normal behavior for an incident, and wave a real incident through as normal.

The First Request Is Slow

Both providers say the same thing. OpenAI's wording is that the first request with any schema carries extra latency while the API processes the schema, but subsequent requests do not. Anthropic likewise warns about grammar compilation latency on the first request. The compiled grammar is cached for 24 hours from last use; it is invalidated when the schema structure or the tool set changes, but changing only name or description does not invalidate it.

When you measure latency, throw the first call away. Otherwise your p99 is just reporting schema compilation time.

Fields You Must Always Check on the Response

On Anthropic, a stop_reason of refusal means the output may not match your schema. max_tokens means the JSON was truncated, and parsing will always fail. The answer is not a retry but a higher max_tokens. Throw the same prompt again and it truncates at the same length again.

On OpenAI Chat Completions, finish_reason is one of stop, length, content_filter, tool_calls. The assistant message carries a separate refusal field that is either a string or null. The parse() helper raises LengthFinishReasonError when the output ran long and ContentFilterFinishReasonError when the filter fired.

The quiet trap is the Responses API. It has no finish_reason. Instead you check whether response.status is incomplete and whether response.incomplete_details.reason is max_output_tokens. Port Chat Completions-era code over as-is and that branch is never true.

# Catching the same failure on all three paths

# Chat Completions
if completion.choices[0].finish_reason == "length":
    raise RuntimeError("truncated — raise max_tokens")

refusal = completion.choices[0].message.refusal
if refusal:
    raise RuntimeError(f"model refused: {refusal}")

# Responses API — there is no finish_reason at all
if response.status == "incomplete":
    if response.incomplete_details.reason == "max_output_tokens":
        raise RuntimeError("truncated — raise max_output_tokens")

# Anthropic
if message.stop_reason == "max_tokens":
    raise RuntimeError("JSON truncated — raise max_tokens")
if message.stop_reason == "refusal":
    raise RuntimeError("refusal — output may not match the schema")

End-to-End Example: A Support Ticket Classifier

Let me put the pieces together. This is a classifier that takes a support ticket and produces a category, a severity, a one-line summary, and whether a human needs to look at it. The first two fields have a fixed value set and downstream routing keys off them, so a single typo loses a ticket entirely.

1. Define the Schema

from typing import Literal, Optional
from pydantic import BaseModel, Field

class TicketTriage(BaseModel):
    category: Literal["billing", "bug", "feature_request", "account", "other"]
    severity: Literal["p0", "p1", "p2", "p3"]
    summary: str = Field(description="one sentence, under 60 characters")
    needs_human: bool
    suggested_owner: Optional[str] = None

The two Literal fields are the point. They become enum in the JSON Schema, and constrained decoding blocks any token outside those values. A prompt is a request; an enum is enforcement.

2. Call It

from anthropic import Anthropic

client = Anthropic()

TICKET = """
I was charged twice. My card statement shows two identical charges
dated August 3rd. The order number is A-91823. Please refund one.
"""

response = client.messages.parse(
    model="claude-sonnet-5",
    max_tokens=2000,
    messages=[
        {"role": "user", "content": f"Classify the following support ticket.\n\n{TICKET}"}
    ],
    output_format=TicketTriage,
)

if response.stop_reason == "max_tokens":
    raise RuntimeError("truncated — raise max_tokens")

triage = response.parsed_output
print(triage.category, triage.severity, triage.needs_human)

3. What Comes Back

response.parsed_output is an already-validated TicketTriage instance. Serialized, it looks like this.

{
  "category": "billing",
  "severity": "p1",
  "summary": "Duplicate charge on Aug 3, order A-91823, refund requested",
  "needs_human": true,
  "suggested_owner": "billing-ops"
}

That category falls inside the five defined values is guaranteed by constrained decoding. That severity is a sensible call is not. Only the shape is guaranteed; the quality of the judgment is a separate question.

suggested_owner is where the providers diverge. Hand the same TicketTriage to OpenAI and the strict rules put this field in required too, making its type a union of string and null instead. On OpenAI the key is always present and may be null; on Anthropic the key itself may be absent. Read it with dictionary brackets and you get a KeyError on exactly one of them. Take it as a Pydantic instance and both normalize to None.

4. The Same Schema on OpenAI Responses

from openai import OpenAI

client = OpenAI()

response = client.responses.parse(
    model="gpt-4o-mini",
    input=[{"role": "user", "content": f"Classify the following support ticket.\n\n{TICKET}"}],
    text_format=TicketTriage,
)

if response.status == "incomplete":
    if response.incomplete_details.reason == "max_output_tokens":
        raise RuntimeError("truncated — raise max_output_tokens")

What changed is the client, the argument names (messages versus input, output_format versus text_format), and how truncation is detected. The TicketTriage definition was not touched at all. Keep the schema as a single Pydantic model and wrap the call thinly, and this is what provider portability costs.

Schema Constraints — Support Differs by Provider

JSON Schema is a standard, but the subset each provider supports is different, and the two do not even nest. enum and anyOf work on both. The rest is where it gets interesting.

Anthropic supports the string formats date-time, date, duration, email, hostname, uri, ipv4, ipv6, and uuid, plus const and default. Internal references work, external ones do not, and recursive schemas are unsupported — a self-referential model like a tree needs a different approach. OpenAI publishes hard limits instead: 5000 object properties in total, 10 levels of nesting, 120,000 characters of total string length, and 1000 enum values across all enum properties combined.

The takeaway is the asymmetry. OpenAI supports numeric constraints but not string-length constraints; Anthropic supports neither. A schema that sails through on one provider can be rejected with a 400 on the other. If you plan to run multi-provider, narrow the schema to the intersection of the two and put length and range checks in the Pydantic layer instead — it travels better.

Failure Modes — From Symptom to Cause

What shows up in the log is the symptom. The cause is always behind it.

Latency Still Spikes, Even After a Warmup

The cache is being invalidated every time. The usual culprits are code that rebuilds the schema dict per request so key order shifts, or that fills an enum list dynamically from user input. Build the schema once at startup and hold it as a constant.

400 — Errors About additionalProperties

You turned on strict for OpenAI but additionalProperties is missing or true. It has to be false, set explicitly on every nested object, and every property has to be in required. This is the first thing to blow up when a schema that ran fine on Anthropic is carried over unchanged.

400 — Using It Alongside Citations

On Anthropic, structured outputs and citations cannot be used together and you get a 400. The trouble starts with trying to merge the step that needs provenance and the step that needs structure into a single call.

JSON Parsing Fails Intermittently

If it only fails on long inputs or large result arrays, it is almost certainly truncation. Check the three signals from the previous section in order, raise the output token ceiling, and if that is still not enough, split the batch by pulling fewer items per call. Conversely, if parsing succeeds but the values look wrong, you missed a stop_reason of refusal.

Switched to LiteLLM and the Validation Vanished

Instructor and LiteLLM look similar but hand back different things. Instructor gives you a validated object. Pass a Pydantic model as response_format to LiteLLM's completion and it is accepted, but what comes back is JSON text, and validating it is on you.

from litellm import completion, get_supported_openai_params, supports_response_schema

resp = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "..."}],
    response_format=EventsList,
)

# Still a string at this point. Validation is my job.
events = EventsList.model_validate_json(resp.choices[0].message.content)

Switch over without knowing this and the validation step disappears entirely on the way to production. The type hints are unchanged, so static analysis will not catch it either. LiteLLM ships get_supported_openai_params and supports_response_schema helpers, so check once at startup.

Instructor's currently recommended entrypoint is instructor.from_provider. from_openai and from_litellm are still there, and from_anthropic or from_genai are available when the corresponding provider package is installed. instructor.patch still exists and is not deprecated, but the docs do not recommend manual patching.

When Not to Use a Schema

Schemas are not free. Force one into a place it does not fit and output quality drops while debugging gets harder.

When free-form generation is the point. Put a schema on work whose output is prose — an article, a summary — and the model spends its budget satisfying the format instead of the content. A schema wrapping a single prose field is mostly ceremony.

When the extraction is high-cardinality. Trying to pin a field with hundreds or thousands of possible values into an enum usually fails. The longer the list, the more the schema crowds out the prompt. Pulling a free string and normalizing afterward with a dictionary lookup or embedding match is more accurate.

When a plain prompt plus one retry is cheaper. For a script that runs a few times a day, or analysis a human reviews by eye, the cost of designing and maintaining a schema exceeds what it buys. Schemas earn their keep when the result flows straight into the next system without passing through a person.

When the schema fights the task. Force a field the model cannot know into required and it will not leave the blank empty — it invents something plausible. Constrained decoding only lets grammatical tokens through, so it will always produce a well-formed answer even for a field it does not know. That is a hallucination the schema created. Make uncertain fields optional, or pair them with a confidence field.

When the nesting is deep. A schema approaching the 10-level limit is usually a design mistake. Split it into stages so each call handles a shallow schema — better for both accuracy and debugging.

Conclusion

Structured Output is a key technology for integrating LLMs into production systems:

1. OpenAI: 100% schema guarantee with response_format + Structured Outputs
2. Anthropic: An indirect approach via Tool Use, but reliable
3. Instructor/LiteLLM: Code reuse through provider unification
4. Pydantic: The standard for schema definition + validation
5. Retry Pattern: Stability through self-healing

One more thing: in this area the docs turn over in about six months. The first half of this article had two providers change their recommended path in five.

References

The documents I actually checked while revising this article. The further you are from the verification date, the more you should go to the source first.

• Anthropic Structured Outputs — https://platform.claude.com/docs/en/build-with-claude/structured-outputs (verified 2026-08-16)
• OpenAI Structured Outputs — https://developers.openai.com/api/docs/guides/structured-outputs (verified 2026-08-16). The old platform.openai.com path now 301s here.
• Gemini API libraries — https://ai.google.dev/gemini-api/docs/libraries (verified 2026-08-16)
• Gemini structured output — https://ai.google.dev/gemini-api/docs/structured-output (verified 2026-08-16)

The versions described here are openai 3.1.0, instructor 1.15.4, litellm 1.96.2, and google-genai. The model IDs used are claude-opus-5, claude-sonnet-5, claude-haiku-4-5, and claude-fable-5. For any method or field not listed here, do not guess — check the docs for the version you are on for the exact API.

Quiz