Zero-Shot vs. Few-Shot for JSON & Data Extraction: When Schema Descriptions Fall Short

You wrote a detailed schema, yet the LLM still returns wrong date formats or unexpected null. This highlights a fundamental gap in structured extraction: the difference between telling a model what you want and showing it.

This is the core tension between zero-shot and few-shot prompting for structured data extraction. Both approaches can produce valid JSON. But only one of them reliably handles the messy edge cases that show up the moment your pipeline touches real-world data.

What is JSON?

JSON (JavaScript Object Notation) is a lightweight, text-based data format built on key-value pairs ({"key": "value"}) and lists.

Why JSON Extraction Is Different From Regular Prompting

Most prompting advice is written for open-ended tasks: summarization, brainstorming, creative writing. Structured extraction is a different beast. You're not asking the model to be creative, you're asking it to be consistent. Every field name, every data type, every formatting convention has to match exactly, every single time, because something downstream (a database, an API, a parser) is going to consume that output without a human checking it first.

That consistency requirement is why prompting strategy matters so much more here than in other domains. A model can get the gist of a summary and still be useful. A model that gets the gist of a JSON schema and outputs "true" instead of true, or "$45.00" instead of 45.00, can break an entire pipeline.

Note

Structured extraction failures are rarely about the model misunderstanding your intent. They're almost always about the model not knowing exactly what format you expect, especially for values it hasn't seen an example of.

Zero-Shot Extraction: Instructions Only

Zero-shot prompting means you describe the schema and the rules in plain language (or in a formal schema definition like JSON Schema) and ask the model to extract accordingly, without showing it a single filled-in example.

A typical zero-shot extraction prompt looks like this:

Prompt Template

Extract the following fields from the text below and return valid JSON matching this schema:

- {{field_1}}: {{type_and_description_1}}
- {{field_2}}: {{type_and_description_2}}
- {{field_3}}: {{type_and_description_3}}

Rules:
{{formatting_rules}}

Text:
{{input_text}}

Return only the JSON object, no explanation.

Here's the same template filled in for a product review extraction task:

Prompt Example

Extract the following fields from the text below and return valid JSON matching this schema:

- product_name: string, the name of the product being reviewed
- rating: integer from 1 to 5
- sentiment: one of "positive", "negative", or "mixed"
- purchase_date: date in ISO 8601 format, or null if not mentioned

Rules:
- If a field cannot be determined from the text, use null.
- Do not invent values that are not stated or clearly implied.

Text:
"I bought the Aria noise-cancelling headphones back in March and honestly they're a mixed bag. Sound quality is great but the battery life is disappointing. 3 out of 5 stars."

Return only the JSON object, no explanation.

For straightforward inputs like this one, zero-shot works fine. The model has plenty of prior exposure to product reviews and star ratings, so it can generalize without needing an example. This is the appeal of zero-shot: less prompt engineering, shorter prompts, lower token cost, and faster iteration when you're prototyping.

Where Zero-Shot Breaks Down

The trouble starts when your data has ambiguity that a description can't fully resolve. Consider these situations:

A date is written as "early Q2" instead of a clean calendar date.
A price appears as a range, "$40 to $60," and your schema expects a single number.
A field should be an empty array [] when nothing is found, but the model defaults to null or omits the key entirely.
Nested objects need a specific shape (e.g., {"city": ..., "state": ...}) but the input text mixes address formats.
Enum values need to map from messy real-world language ("kinda satisfied") to a fixed set ("neutral").

Instructions can describe these rules, but instructions are interpreted, not copied. The model has to infer the correct behavior from your wording, and inference is exactly where inconsistency creeps in. You might write "use null for missing values," and the model mostly complies, except when a field is present but ambiguous, where it might invent a plausible-sounding placeholder instead.

Tip

If you're only extracting from clean, well-structured text (like form submissions or structured emails), zero-shot with a precise schema is often good enough. Save few-shot examples for messier, more varied input sources.

Few-Shot Extraction: Showing, Not Just Telling

Few-shot prompting adds one or more worked examples directly in the prompt, pairs of input text and the exact JSON output you expect. Instead of relying on the model to interpret your formatting rules correctly, you demonstrate them.

Prompt Template

Extract the following fields from the text and return valid JSON matching the schema shown in the examples.

Example 1
Input: {{example_input_1}}
Output: {{example_output_1}}

Example 2
Input: {{example_input_2}}
Output: {{example_output_2}}

Now extract from this text:
Input: {{input_text}}
Output:

Applied to the same headphone review task, but now handling a tricky edge case (a vague date reference):

Prompt Example

Extract the following fields from the text and return valid JSON matching the schema shown in the examples.

Example 1
Input: "Got these shoes last week, super comfortable, 5 stars for sure."
Output: {"product_name": "shoes", "rating": 5, "sentiment": "positive", "purchase_date": null}

Example 2
Input: "Picked up the blender sometime in early April, works okay I guess, 3/5."
Output: {"product_name": "blender", "rating": 3, "sentiment": "mixed", "purchase_date": "2026-04"}

Now extract from this text:
Input: "I bought the Aria noise-cancelling headphones back in March and honestly they're a mixed bag. Sound quality is great but the battery life is disappointing. 3 out of 5 stars."
Output:

Notice what the second example is doing beyond just showing valid JSON syntax. It's teaching the model a specific convention: when a date is vague ("early April"), truncate to year-month precision ("2026-04") rather than guessing a day or returning null. That convention would be hard to state as a general rule, but it's trivial to demonstrate once and let the model pattern-match from.

The Real Value of Few-Shot: Edge Case Calibration

The biggest misconception about few-shot prompting is that it's mainly about teaching output format. In practice, a well-written schema and a "return only JSON" instruction already get you 90% of the way to correct formatting in zero-shot. What few-shot actually buys you is edge case calibration, showing the model how to behave when the input doesn't cleanly fit the schema.

Good few-shot examples deliberately include:

At least one example with a missing or ambiguous field, so the model sees exactly how you want gaps handled.
At least one example with a boundary condition, like an empty list, a zero value, or a field at the edge of a valid enum.
At least one example that resembles the messiest, most unusual input you expect in production, not just the easy cases.

Tip

Don't waste your few-shot slots on easy, obvious inputs. The model already handles those well in zero-shot. Spend your examples on the 10% of cases that actually cause failures.

Instructions vs. Concrete Schemas: A Side-by-Side Comparison

Aspect	Zero-Shot (Instructions Only)	Few-Shot (Concrete Examples)
Prompt length	Shorter, cheaper per call	Longer, higher token cost
Setup effort	Low, write the schema once	Higher, requires curated examples
Format compliance on clean data	Generally reliable	Generally reliable
Handling of ambiguous or missing fields	Inconsistent, depends on model's interpretation	Strong, if examples cover the ambiguity
Handling of unusual formatting conventions	Weak, hard to describe precisely in words	Strong, conventions are demonstrated directly
Maintenance	Easy to tweak wording	Requires updating examples as edge cases evolve
Best for	Prototyping, simple/uniform inputs, tight token budgets	Production pipelines, messy or varied real-world inputs

The pattern here is consistent across most extraction tasks: zero-shot and few-shot converge on accuracy for clean, well-behaved input, but diverge sharply as soon as the input gets messy. If your data source is variable (scraped web pages, customer support tickets, scanned documents, freeform user text), few-shot is not a nice-to-have, it's often the difference between a pipeline that works and one that silently produces bad data.

One-Shot vs. Few-Shot: How Many Examples Do You Actually Need

A related question that trips people up is how many examples to include. One-shot (a single example) is often enough to lock in output format and basic structure. But for edge case coverage, one example usually isn't enough, because a single example can only demonstrate one convention at a time.

Here's a practical way to think about it:

One-shot is good for establishing the JSON shape and key naming when the schema itself is the main risk (e.g., you need snake_case keys instead of camelCase, or a specific nesting structure).
Two-to-three-shot is the sweet spot for most production extraction tasks. Use one clean example, one example with a missing field, and one example with a formatting edge case.
Four-plus examples start to pay off when your schema has multiple enum fields, several date or number formats in play, or when different input sources need to be handled differently within the same prompt.

Beyond four or five examples, returns diminish quickly and token cost climbs. If you find yourself wanting ten examples to cover all your edge cases, that's usually a sign you should either simplify the schema, split the extraction into multiple smaller prompts, or move to a fine-tuning or few-shot retrieval approach where relevant examples are selected dynamically per input rather than hardcoded into every prompt.

Note

If your edge cases are numerous and varied, consider retrieval-based few-shot: store a library of example input/output pairs and dynamically select the 2-3 most relevant ones for each new input at runtime. This scales better than a single static few-shot prompt.

A Practical Decision Framework

Zero-Shot vs. Few-Shot Decision Framework

Rather than picking zero-shot or few-shot as a blanket policy, it helps to decide per field, per source, or per pipeline stage:

Start with zero-shot. Write your schema as precisely as you can, including type, description, and default behavior for missing values. Test it against a representative sample of your real data, not just the easy cases.
Log the failures. Whenever the output is wrong, malformed, or inconsistent, save that input alongside the correct expected output. This log becomes your few-shot example bank.
Add targeted examples. Once you have a handful of recurring failure patterns, add one or two examples per pattern to your prompt. Don't add examples speculatively for problems you haven't actually observed.
Re-test and prune. After adding examples, re-run your evaluation set. If accuracy plateaus, stop adding examples, more isn't always better, and longer prompts increase latency and cost without guaranteed benefit.

This workflow keeps your prompt lean while still targeting the exact failure modes your pipeline actually experiences, instead of guessing at hypothetical edge cases upfront.

Combining Both Approaches

In practice, most production-grade extraction prompts aren't purely zero-shot or purely few-shot, they're a hybrid. You keep the precise schema instructions (types, required fields, formatting rules) because they're cheap and effective for the majority of cases, and you layer in a small number of high-value examples specifically for the edge cases that instructions alone can't reliably resolve.

This hybrid structure also makes your prompts easier to maintain. The schema block documents your contract in plain language, which is useful for anyone reading the prompt later, while the examples act as a living record of the tricky cases your team has already debugged. When a new edge case shows up in production, you don't have to rewrite your instructions, you just add one more example.

Tip

Version your few-shot examples the same way you'd version code. When you fix a bug by adding an example, note why that example exists (a short comment above it in your prompt file) so future you understands the edge case it's protecting against.

Conclusion

Zero-shot prompting works well for simple schemas and clean inputs. But when real-world edge cases appear, plain instructions fall short—few-shot examples bridge that gap by showing the model exactly how to handle ambiguity.

The best approach is iterative: start with zero-shot instructions, monitor pipeline failures, and turn recurring edge cases into targeted few-shot examples.

Acluebox

Craft perfect AI prompts and build powerful, reusable systems. Your all-in-one workspace for prompt discovery, organization and management.

Start for Free

FAQs

What's the main difference between zero-shot and few-shot extraction?

Zero-shot extraction relies only on written instructions and a schema description, while few-shot extraction adds one or more worked input/output examples so the model can see exactly how edge cases and formatting conventions should be handled.

Does few-shot prompting always improve JSON extraction accuracy?

Not always. For clean, uniform data, zero-shot with a precise schema often performs just as well. Few-shot mainly helps with ambiguous fields, unusual formatting, and edge cases that are hard to describe in plain instructions.

How many examples should I include in a few-shot extraction prompt?

Two to three examples is typically enough for most tasks: one clean example, one with a missing or null field, and one covering a specific formatting edge case. Beyond four or five examples, returns tend to diminish while token cost keeps rising.

Can I mix zero-shot instructions with few-shot examples in the same prompt?

Yes, and this hybrid approach is common in production pipelines. Keep your schema and formatting rules as clear written instructions, then add a small number of targeted examples specifically for edge cases that instructions alone don't reliably resolve.

What should I do if my edge cases are too varied for a fixed set of examples?

Consider retrieval-based few-shot prompting, where you maintain a larger library of example input/output pairs and dynamically select the most relevant few for each new input at runtime, instead of hardcoding the same examples into every prompt.