LLM JSON Output Kept Failing? Disable Thinking Mode
Summary
Our AutoSEM pipeline's JSON output failures traced to thinking mode leaking into structured responses — content came back empty. Fix: disable thinking, batch about 20 items per call, hard system prompt, layered fallback parsing, and no JSON examples in prompts.
Quick answer
Disable thinking mode for structured calls (e.g. enable_thinking=False) or content returns empty; keep batches small; demand "only a valid JSON" in the system prompt; parse in fallback layers (direct → code block → truncation repair → per-object); and never show JSON examples in user messages.
Our AutoSEM pipeline depends on LLMs returning clean JSON — keyword batches, ad-group structures, search-term analyses. For weeks, the same bug kept showing up: the model returned empty strings, or a JSON array wrapped in natural language, and the parser failed. After twelve logged failure patterns, we traced most of them to one root cause. None of it was exotic — the log was boring, repetitive, and fixable: the same empty content string, the same prose-wrapped array, over and over. The fixes below are the ones that made the log go quiet.
The root cause: thinking mode leaks into content
Many models now enable a "thinking" mode by default. The reasoning is generated internally — but in structured-output calls, the reasoning can end up in the response, or worse, the actual content comes back as an empty string because the model wrote its reasoning where the answer should be. Your JSON parser is not broken; the model never produced JSON.
Five rules that fixed it
- Disable thinking explicitly. Structured calls turn thinking off — with our model, that flag is
enable_thinking=False. Leave it on andcontentcomes back empty. - Batch, don't flood. Long inputs push models back into prose. We call in batches of about 20 items so the model never loses the JSON format.
- Be firm in the system prompt. Something like "MUST respond with ONLY a valid JSON" — no pleasantries, no alternatives.
- Parse in layers. Try direct parsing first; then extract from code blocks; then repair truncation; then pull objects one by one. A fallback chain is cheaper than a retry loop.
- Never show a JSON example to the model. When we pasted an example format into the user message, the model copied the example's explanatory text instead of producing data. Keep examples out of the prompt.
Why thinking mode hurts structured calls
Thinking is a feature for chat, not for data extraction. For extraction, the model's job is translating input to a schema — a narrow task where added reasoning costs tokens, latency, and failure surface without adding accuracy.
Reusable checklist: reliable JSON in 5 lines
- Turn thinking off for structured calls.
- Keep batches small enough that the model never drops to prose.
- State "only valid JSON" firmly in the system prompt.
- Layer your parser: direct → code block → truncation repair → per-object.
- Leave example formats out of user messages.
These rules came out of our own pipeline's failure log. If you are building LLM-driven marketing automation, our growth engine runs on exactly these structured-output practices.
Frequently asked questions
Why does the model return an empty content field?
With thinking mode enabled, the model writes its reasoning into the response and the actual content comes back empty. For structured calls we set enable_thinking=False explicitly.
Why not just show the model a JSON example?
When we pasted an example, the model mimicked the example's explanatory text instead of producing data. Examples in user messages teach format mimicry, not extraction.
What if the response still fails to parse?
Layer the parser: direct JSON parse first, then extract from code blocks, then repair truncation, then pull objects one by one. A fallback chain costs less than retrying the model.