Enforcing Guaranteed Structured Outputs from LLMs with Pydantic
Eliminate JSON parsing errors and hallucinated keys by leveraging OpenAI Structured Outputs and Pydantic schema validation for guaranteed type safety.
Prerequisites
- Python 3.9+
- Basic understanding of Pydantic BaseModel
- OpenAI Python SDK (v1.40.0+)
1. The Vulnerability of Unconstrained JSON Generation
Instructing an LLM to 'Respond in JSON' frequently results in invalid JSON, trailing commas, markdown formatting fences (```json), or randomly omitted fields. OpenAI's Structured Outputs feature uses context-free grammar (CFG) constrained sampling at the token level, guaranteeing 100% adherence to your Pydantic schema.
# Ensure modern SDK installed
pip install "openai>=1.40.0" "pydantic>=2.0.0"2. Modeling Schema with Pydantic V2
Define nested Pydantic models with explicit types, Enum restrictions, and Field descriptions that guide the LLM's reasoning engine.
from enum import Enum
from typing import List, Optional
from pydantic import BaseModel, Field
class SentimentType(str, Enum):
POSITIVE = "positive"
NEUTRAL = "neutral"
NEGATIVE = "negative"
class ExtractedEntity(BaseModel):
name: str = Field(description="Entity or brand name")
category: str = Field(description="Category, e.g., Organization, Person, Metric")
confidence_score: float = Field(ge=0.0, le=1.0, description="Model confidence between 0 and 1")
class ArticleAnalysis(BaseModel):
summary: str = Field(description="A concise 2-sentence executive summary")
sentiment: SentimentType
key_entities: List[ExtractedEntity]
actionable_recommendations: List[str]3. Querying the Model via beta.chat.completions.parse
Use the native parse method in the OpenAI SDK. Under the hood, OpenAI compiles the Pydantic model into a strict JSON Schema and constrains token generation.
import os
from openai import OpenAI
client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))
user_article = """
Acme Corp reported a 24% increase in quarterly revenue to $1.2B, driven by enterprise AI adoption.
However, supply chain bottlenecks in Europe caused a 5% delay in hardware shipments.
CEO Jane Doe stated that operational realignment will be complete by Q4.
"""
completion = client.beta.chat.completions.parse(
model="gpt-4o-2024-08-06",
messages=[
{"role": "system", "content": "You are a senior equity research assistant."},
{"role": "user", "content": f"Extract detailed analysis from this article:\n{user_article}"}
],
response_format=ArticleAnalysis,
)
analysis: ArticleAnalysis = completion.choices[0].message.parsed
# Directly access typed attributes with full IDE autocomplete!
print(f"Sentiment: {analysis.sentiment.value}")
print(f"Summary: {analysis.summary}")
for entity in analysis.key_entities:
print(f"- {entity.name} ({entity.category}): {entity.confidence_score:.2f}")4. Handling Refusals and Error Recovery
If a user prompt violates safety policies or contains adversarial jailbreaks, the model will return a refusal rather than structured output. Always inspect the refusal attribute before consuming the parsed object.
message = completion.choices[0].message
if message.refusal:
print(f"Model refused to process request: {message.refusal}")
else:
structured_data = message.parsed
# Safe to persist to database or forward to downstream APIs
save_to_database(structured_data.model_dump())Best Practices & Architecture Advice
- Use Field(description='...') on every attribute to give the model contextual semantic guidance.
- Prefer Enums over plain strings for finite state values (e.g. status, sentiment, priority).
- Keep schemas modular by nesting sub-models rather than defining monolithic flat models.
- Check for message.refusal before attempting to access message.parsed in production web handlers.
Common Mistakes to Watch Out For
- •Using complex unsupported Pydantic types like Union with arbitrary primitives which cannot compile to strict JSON Schema.
- •Relying on manual regex extraction from unstructured completions when native parse is available.
- •Setting optional fields without providing default values or Optional[T] typing.
Frequently Asked Questions
What is the difference between JSON Mode and Structured Outputs?
JSON Mode ensures the model outputs valid JSON syntax, but does NOT guarantee the keys, types, or nested structure match your schema. Structured Outputs guarantees 100% adherence to your exact schema through constrained decoding.
Does using Structured Outputs slow down response time?
The first request with a new schema has a slight setup latency while the grammar is compiled. Subsequent requests with the same schema have zero latency overhead and benefit from standard inference speeds.