JSON Kit← Blog

Why Your ChatGPT JSON Keeps Breaking (And How to Fix It)

LLMs produce broken JSON constantly — trailing commas, single quotes, Python literals, markdown fences. Here's why it happens and how to fix it in production.

·8 min readllmjsonchatgptdebugging

If you've built anything with the OpenAI API, Anthropic Claude, or any language model, you've hit this wall: you ask for JSON, the model gives you almost JSON. There's a trailing comma. A single quote somewhere. The whole thing wrapped in a markdown code block. JSON.parse throws, your app breaks, and you start wondering if this is just how AI development works.

It is — but there's a systematic fix. This post explains exactly why LLMs produce broken JSON, which errors appear most often, and how to handle them reliably in production.

Why LLMs Can't Help Breaking JSON

Language models don't generate JSON by following a grammar. They predict the next token based on statistical patterns learned from training data. The model has seen vastly more JavaScript and Python code than it has strict JSON — so it bleeds those conventions in without "knowing" it's breaking the spec.

Specifically:

  • Trailing commas are valid in JavaScript object literals and most modern JSON5/JSONC formats. The model has seen millions of examples of {"key": "value",} in JS code.
  • Single quotes are standard in Python and JavaScript. JSON requires double quotes but the model treats them as equivalent.
  • True, False, None are Python's boolean and null literals. JSON uses lowercase true, false, null.
  • Markdown code fences are how the model has been trained to present code in its responses. When you ask for JSON, it wraps the output in ```json ... ``` because that's the format it learned to use.
  • Comments appear throughout real-world configuration files and codebases. The model adds // explanatory comments because it's trying to be helpful.

There's no malice here — the model is doing exactly what its training optimized for. The problem is that strict JSON has no tolerance for any of these.

The 6 Most Common LLM JSON Errors

1. Trailing Commas

{
  "name": "Alice",
  "age": 30,
  "active": true,
}

The last comma after true is illegal in JSON. This is probably the single most common LLM JSON error — it appears in a majority of hand-written JSON too, which is why the model produces it constantly.

Fix: Strip trailing commas before parsing, or use jsonrepair.

2. Single Quotes

{
  'name': 'Alice',
  'role': 'admin'
}

Both keys and string values require double quotes in JSON. Single quotes are valid Python and JavaScript string delimiters — the model uses them interchangeably.

Fix: Replace single-quoted strings with double quotes. Simple regex doesn't work reliably here (strings can contain apostrophes) — use a proper parser.

3. Python / Ruby Literals

{
  "active": True,
  "deleted": False,
  "score": None
}

True, False, and None are Python. JSON requires true, false, null. This happens most often when the model is generating Python-adjacent output — Pydantic schemas, FastAPI responses, LangChain outputs.

Fix: Case-insensitive replacement: True → true, False → false, None → null.

4. Markdown Code Fences

```json
{
  "id": 1,
  "name": "Alice"
}
```

The model wraps JSON in a code fence because that's how it learned to present code. JSON.parse has no idea what to do with the backticks.

Fix: Strip the ```json and ``` lines before parsing. Use the Extract JSON from Markdown tool to pull JSON out of any response.

5. JavaScript Comments

{
  // user record
  "id": 1,
  "name": "Alice" /* primary identifier */
}

Comments are not valid JSON. They appear when the model is trying to explain its output inline. Double-slash // and block /* */ comments both occur.

Fix: Strip comments before parsing. JSONC (JSON with Comments) parsers like jsonc-parser handle this natively.

6. Truncated / Cut-Off JSON

{
  "results": [
    {"id": 1, "title": "First"},
    {"id": 2, "title": "Second"},
    {"id": 3,

This happens when the model hits its max_tokens limit mid-response. The JSON structure is abandoned mid-field. This is particularly nasty because the output looks almost correct.

Fix: jsonrepair can close unmatched brackets and produce a structurally valid (though incomplete) result. The real fix is increasing max_tokens or restructuring your prompt to ask for less data.

The Systematic Production Fix

Don't write custom regex to handle each case. Use jsonrepair — an open-source library specifically designed for this problem. It handles all six cases above plus a dozen more edge cases.

import { jsonrepair } from 'jsonrepair';

async function parseLlmJson<T>(rawOutput: string): Promise<T> {
  // First, try clean parse
  try {
    return JSON.parse(rawOutput) as T;
  } catch {
    // Fall back to repair
    try {
      return JSON.parse(jsonrepair(rawOutput)) as T;
    } catch (e) {
      throw new Error(`Could not parse LLM output as JSON: ${e}`);
    }
  }
}

This pattern — try parse, fall back to repair — handles ~95% of real-world LLM JSON errors with no manual case handling.

For a quick browser-based fix, use the Fix LLM JSON tool which uses the same library and shows exactly what was repaired.

Prevention: Getting Cleaner JSON from the Start

Repair is a safety net. Prevention is better. Here are the approaches that work:

1. Use response_format: { type: "json_object" }

OpenAI's JSON mode constrains the model to always return a valid JSON object. Enable it like this:

const response = await openai.chat.completions.create({
  model: "gpt-4o",
  response_format: { type: "json_object" },
  messages: [
    { role: "system", content: "You are a helpful assistant that responds only in JSON." },
    { role: "user", content: "Give me a user profile for Alice, age 30, role admin." }
  ],
});

const data = JSON.parse(response.choices[0].message.content!);

Limitation: JSON mode guarantees syntactic validity but not schema conformance. The model might return {"user": {...}} when you expected {...} directly.

2. Use Structured Outputs (Strict Mode)

OpenAI's Structured Outputs (available on gpt-4o and later) guarantee the response exactly matches a JSON Schema you provide:

const response = await openai.chat.completions.create({
  model: "gpt-4o",
  response_format: {
    type: "json_schema",
    json_schema: {
      name: "user_profile",
      strict: true,
      schema: {
        type: "object",
        properties: {
          name: { type: "string" },
          age:  { type: "integer" },
          role: { type: "string" }
        },
        required: ["name", "age", "role"],
        additionalProperties: false
      }
    }
  },
  messages: [...]
});

Use the JSON to OpenAI Schema tool to generate the schema from a sample JSON object.

3. Explicit System Prompt Instructions

When using models or endpoints that don't support JSON mode, explicit instructions help significantly:

SYSTEM: You are a data extraction assistant.
Always respond with a raw JSON object only.
Do NOT wrap in markdown code fences.
Do NOT add comments.
Do NOT use trailing commas.
Use double quotes for all strings and keys.
Respond with only the JSON — no explanation before or after.

This doesn't eliminate errors but reduces frequency by 70–80% in practice.

4. Anthropic Claude Tool Use

Claude's tool use feature enforces a strict input_schema:

const response = await anthropic.messages.create({
  model: "claude-sonnet-4-6",
  tools: [{
    name: "create_user",
    description: "Create a user profile",
    input_schema: {
      type: "object",
      properties: {
        name:  { type: "string" },
        age:   { type: "integer" },
        role:  { type: "string" }
      },
      required: ["name", "age", "role"]
    }
  }],
  tool_choice: { type: "tool", name: "create_user" },
  messages: [{ role: "user", content: "Create a profile for Alice, 30, admin" }]
});

The model's response will always match the schema exactly.

Validate What You Get

Even with JSON mode or structured outputs enabled, you should validate the parsed JSON against your expected schema at runtime. TypeScript types disappear at compile time — they won't protect you at runtime.

Use Zod for this:

import { z } from 'zod';

const UserSchema = z.object({
  name: z.string(),
  age:  z.number().int().positive(),
  role: z.enum(["admin", "editor", "viewer"]),
});

type User = z.infer<typeof UserSchema>;

const raw = await parseLlmJson<unknown>(llmOutput);
const user = UserSchema.parse(raw); // throws on schema mismatch

Generate a Zod schema from a sample JSON object using the JSON to Zod Schema tool.

Summary

| Issue | Cause | Fix | |-------|-------|-----| | Trailing commas | JS conventions | jsonrepair | | Single quotes | Python/JS | jsonrepair | | Python literals | Python code in training data | jsonrepair | | Markdown fences | Model's learned presentation format | Strip or use extract tool | | Comments | Model trying to be helpful | jsonrepair or jsonc-parser | | Truncated output | max_tokens limit hit | Increase max_tokens + jsonrepair |

The production pattern: try JSON.parse → fall back to jsonrepair → validate with Zod. Use JSON mode or Structured Outputs when available. Add explicit instructions to your system prompt. Monitor parse failures in production so you can tune your prompts over time.

LLMs will never be perfect JSON generators — the architecture doesn't support it. But with repair + validation, you can make LLM JSON pipelines reliable enough for production.

Tools mentioned in this post:

Try the JSON Kit tools

Everything mentioned in this post is available as a free browser-side tool.

Browse all 30+ tools →