Asking a model for JSON and actually getting JSON

Constrain the output to a schema at the API, validate the result against that schema in your own code, and give the model a list to choose from rather than a blank to fill in. A reply that parses is not a reply that is correct, and the gap between those two is where the bugs live.

Below, seven replies of the kind a model actually returns. Three fail to parse. Three parse cleanly and are wrong.

The problem

You want a structured decision out of unstructured text. Classify an email, extract a field, pick a route. The natural prompt is:

Return your answer as JSON with the keys action, subject_line_index and confidence.

It works. It works in testing, it works in the demo, and it works often enough that you ship it. Then it returns:

Sure! Here is the JSON you asked for:
{"action":"pay_invoice","subject_line_index":0,"confidence":"high"}

and JSON.parse throws in production.

Why the obvious approach fails

Not because of the parse errors. Those are loud, and you will fix them within a day. The obvious approach fails because of the replies that parse.

Here are seven replies through a bare JSON.parse, on Node 23.5.0:

  prose preamble      SyntaxError: Unexpected token 'S', "Sure! Here"... is not val
  markdown fence      SyntaxError: Unexpected token '`', "```json\n{""... is not va
  trailing comma      SyntaxError: Expected double-quoted property name in JSON at 
  invented enum       parsed
  index off the end   parsed
  extra field         parsed
  clean               parsed

Four of seven parsed. One of those four was correct.

• invented enum — "action":"escalate_to_finance". A sensible label. Not one of the four your switch statement handles, so it falls through to the default branch and the message is silently ignored.
• index off the end — "subject_line_index":7 into an array of three. lines[7] is undefined, and undefined propagates a long way before it becomes an error you can trace.
• extra field — an unrequested "note" key. Harmless today. Tomorrow it is written straight into a JSONB column and something downstream reads it.

None of these throw. All of them are wrong. A parse check tells you the reply is syntactically JSON; it tells you nothing about whether it is the JSON you asked for.

The salvage layer is not the fix

The first instinct is to rescue the unparseable replies:

const salvage = (s) => JSON.parse(s.slice(s.indexOf('{'), s.lastIndexOf('}') + 1))

Run it on three real shapes:

fence         {"action":"none"}
two objects   threw: Unexpected non-whitespace character afte
brace in text {"action":"none","note":"the } character confused me"}

Two of three behave. The one that matters is this one:

const wrapped = 'Here you go:\n{"result": {"action":"none","subject_line_index":0,"confidence":"high"}}'
salvage(wrapped)      // {"result":{"action":"none",...}}   parses fine

salvage says:   {"result":{"action":"none","subject_line_index":0,"confidence":"high"}}
got.action is:  undefined
typeof:         undefined

The model wrapped the object one level deeper than usual. Salvage succeeded. The caller reads .action, gets undefined, and the if that was supposed to fire quietly does not. No exception, no log line, no ticket — just a decision that never happened.

Salvage is a patch applied one layer too late. The fix goes above it and below it: constrain the output so the fence and the preamble cannot occur, and validate the object so the wrapper and the invented enum cannot pass.

Constrain the output, then validate it anyway

Every major provider now offers some form of schema-constrained decoding — structured outputs, JSON mode, response schemas, grammar-constrained sampling locally. Use it. It removes the whole first category: no preamble, no fence, no trailing comma, because those tokens cannot be sampled.

It does not remove the second category. Constrained decoding guarantees the shape. Whether index 7 exists in your array, or whether the model picked the right one of four valid labels, is your problem and always was. Schema constraints are also a feature of a specific endpoint on a specific model version — the day you swap providers, add a fallback, or a request degrades to an unconstrained path, your only remaining defence is the one in your own process.

So: constrain at the API, and validate in your code. Both. The validator is forty lines and needs no dependency.

const schema = {
  type: 'object',
  required: ['action', 'subject_line_index', 'confidence'],
  additionalProperties: false,
  properties: {
    action: { enum: ['pay_invoice', 'sign_document', 'reply_needed', 'none'] },
    subject_line_index: { type: 'integer', minimum: 0, maximum: 2 },
    confidence: { enum: ['high', 'low'] },
  },
}

export function validate(value, s, path = '$') {
  const errs = []
  const fail = (msg) => errs.push(`${path}: ${msg}`)

  if (s.enum) {
    if (!s.enum.includes(value)) fail(`${JSON.stringify(value)} not in [${s.enum.join(', ')}]`)
    return errs
  }
  if (s.type === 'object') {
    if (value === null || typeof value !== 'object' || Array.isArray(value)) {
      fail('expected an object'); return errs
    }
    for (const k of s.required ?? []) {
      if (!Object.hasOwn(value, k)) fail(`missing required property "${k}"`)
    }
    for (const k of Object.keys(value)) {
      if (s.properties?.[k]) errs.push(...validate(value[k], s.properties[k], `${path}.${k}`))
      else if (s.additionalProperties === false) fail(`unexpected property "${k}"`)
    }
    return errs
  }
  if (s.type === 'integer') {
    if (!Number.isInteger(value)) { fail(`expected an integer, got ${JSON.stringify(value)}`); return errs }
    if (s.minimum !== undefined && value < s.minimum) fail(`${value} < minimum ${s.minimum}`)
    if (s.maximum !== undefined && value > s.maximum) fail(`${value} > maximum ${s.maximum}`)
    return errs
  }
  if (s.type === 'string' && typeof value !== 'string') fail('expected a string')
  return errs
}

Ajv is the right answer once your schemas grow; the point of showing it in forty lines is that "we didn't want a dependency" is not a reason to skip this. Same seven replies, now through parse and validate:

  prose preamble      REJECT  not JSON
  markdown fence      REJECT  not JSON
  trailing comma      REJECT  not JSON
  invented enum       REJECT  $.action: "escalate_to_finance" not in [pay_invoice, sign_document, reply_needed, none]
  index off the end   REJECT  $.subject_line_index: 7 > maximum 2
  extra field         REJECT  $: unexpected property "note"
  clean               ACCEPT

Three schema details are doing most of that work.

additionalProperties: false is the one people leave out. Run the wrapper case from the salvage section through the validator and you get four errors at once:

  $: missing required property "action"
  $: missing required property "subject_line_index"
  $: missing required property "confidence"
  $: unexpected property "result"

Worth being precise about which line saved you: required is what rejects it, and additionalProperties: false is what names the actual cause. That fourth line is the difference between "the model didn't answer" and "the model wrapped the answer in result" — a five-minute fix instead of an afternoon. Keep both. Rejection is the safety property; the diagnosis is what makes the review queue worth reading.

Assign, don't generate

The strongest constraint is not in the schema. It is in what you ask for.

Note the field name: subject_line_index, not subject_line. The model is not writing a string. It is choosing a position in a list you handed it. Compare:

  index 1 -> "Contract renewal for signature"
  generated string -> "Contract renewal for signing"
  is it one we supplied? false

"for signature" became "for signing". One word, entirely reasonable, and it no longer matches a record in your system. An index cannot do that. An index is either in range or it is not, and the schema already checks which.

Reduce every field you can to one of two shapes:

• An enum — the model picks from a fixed set of labels you defined.
• An index or ID — the model picks from a set of values you supplied in this request.

Then validate membership, not just type. suppliedLines.includes(value) is one line, and it converts an entire class of hallucination into a rejected response. This is the same principle behind handing a business card's ambiguous lines to a model while extracting the email with a regex: the model labels things that exist, and never produces a value that has to be right.

Never ask for a value a regex can produce

Worth stating on its own, because it is the rule that gets broken while everything else is done properly.

If a value has a shape — an email address, a phone number, an invoice reference, an ISO date, an amount — extract it with a pattern. Do not include it in the JSON you ask a model for, even a schema-constrained one, even with a pattern in the schema.

A pattern constraint guarantees the answer looks like an email address. It cannot guarantee it is the one in the document. bill.smith@acme.com and bil.smith@acme.com both satisfy the regex; one of them is a bounced invoice that nobody notices for a month. A regex over the source text cannot invent a character, because it only ever returns a slice of what it was given.

The division is clean:

What to do when validation fails

Not: log a warning and carry on with a partial object. That is how a bad value gets into a database, and a bad value in a database outlives every other kind of bug.

Retry once, then stop:

export function decide(rawReplies, s) {
  for (let attempt = 0; attempt < rawReplies.length && attempt < 2; attempt++) {
    let obj
    try { obj = JSON.parse(rawReplies[attempt]) } catch { continue }
    const errs = validate(obj, s)
    if (errs.length === 0) return { status: 'ok', attempt: attempt + 1, value: obj }
  }
  return { status: 'needs_review', attempt: 2, value: null }
}

  bad then good: {"status":"ok","attempt":2,"value":{"action":"sign_document","subject_line_index":1,"confidence":"high"}}
  bad then bad:  {"status":"needs_review","attempt":2,"value":null}

One retry, and put the validation errors in the second prompt. A model given $.action: "escalate_to_finance" not in [...] often corrects itself; the same model given the same prompt again often produces the same reply, because the failure was rarely random. If the second attempt fails too, the input is genuinely outside what you designed for, and a third attempt is a slower way to reach the same conclusion.

Then needs_review — a real state, in the schema of the row, with a queue behind it. Two things make that bearable:

• A human reviewing an item is not an outage. It is the design working. The alternative is a wrong value that nobody reviews.
• The review queue is your evaluation set. Every item in it is a real input your prompt or schema did not cover. Read it weekly and the enum grows the labels it was missing.

If failures pile up faster than anyone can review them, that is information too, and it says the schema is wrong rather than the model. Give the queue the same durable backing as everything else so a restart does not lose the items waiting on a person.

Check it yourself

Every js block above concatenates, in order, into one file. Append this and you have validate.mjs — no network, no API key, no model call, because every reply in it is a literal string:

const F = '\u0060'.repeat(3)   // ```
const replies = [
  ['prose preamble',  'Sure! Here is the JSON you asked for:\n{"action":"pay_invoice","subject_line_index":0,"confidence":"high"}'],
  // The fence is built from a char code so this line can live inside a
  // markdown code block without ending it. At runtime it is a literal ```.
  ['markdown fence',  F + 'json\n{"action":"reply_needed","subject_line_index":1,"confidence":"low"}\n' + F],
  ['trailing comma',  '{"action":"none","subject_line_index":2,"confidence":"low",}'],
  ['invented enum',   '{"action":"escalate_to_finance","subject_line_index":0,"confidence":"high"}'],
  ['index off the end','{"action":"pay_invoice","subject_line_index":7,"confidence":"high"}'],
  ['extra field',     '{"action":"none","subject_line_index":0,"confidence":"high","note":"looks routine"}'],
  ['clean',           '{"action":"sign_document","subject_line_index":1,"confidence":"high"}'],
]

console.log('=== raw JSON.parse on each reply ===')
for (const [name, raw] of replies) {
  let r
  try { JSON.parse(raw); r = 'parsed' }
  catch (e) { r = 'SyntaxError: ' + e.message.replace(/\n/g, '\\n').slice(0, 48) }
  console.log(`  ${name.padEnd(19)} ${r}`)
}

console.log('\n=== parse + validate ===')
for (const [name, raw] of replies) {
  let obj
  try { obj = JSON.parse(raw) }
  catch { console.log(`  ${name.padEnd(19)} REJECT  not JSON`); continue }
  const errs = validate(obj, schema)
  console.log(`  ${name.padEnd(19)} ${errs.length ? 'REJECT  ' + errs[0] : 'ACCEPT'}`)
}

const suppliedLines = [
  'Invoice BL-2291 is overdue',
  'Contract renewal for signature',
  'Re: lunch',
]

console.log('\n=== assign, do not generate ===')
console.log('  index 1 ->', JSON.stringify(suppliedLines[1]))
const generated = 'Contract renewal for signing'
console.log('  generated string ->', JSON.stringify(generated))
console.log('  is it one we supplied?', suppliedLines.includes(generated))

console.log('\n=== retry policy ===')
console.log('  bad then good:', JSON.stringify(decide([replies[3][1], replies[6][1]], schema)))
console.log('  bad then bad: ', JSON.stringify(decide([replies[3][1], replies[4][1]], schema)))

node validate.mjs

Real output, from Node 23.5.0:

=== raw JSON.parse on each reply ===
  prose preamble      SyntaxError: Unexpected token 'S', "Sure! Here"... is not val
  markdown fence      SyntaxError: Unexpected token '`', "```json\n{""... is not va
  trailing comma      SyntaxError: Expected double-quoted property name in JSON at 
  invented enum       parsed
  index off the end   parsed
  extra field         parsed
  clean               parsed

=== parse + validate ===
  prose preamble      REJECT  not JSON
  markdown fence      REJECT  not JSON
  trailing comma      REJECT  not JSON
  invented enum       REJECT  $.action: "escalate_to_finance" not in [pay_invoice, sign_document, reply_needed, none]
  index off the end   REJECT  $.subject_line_index: 7 > maximum 2
  extra field         REJECT  $: unexpected property "note"
  clean               ACCEPT

=== assign, do not generate ===
  index 1 -> "Contract renewal for signature"
  generated string -> "Contract renewal for signing"
  is it one we supplied? false

=== retry policy ===
  bad then good: {"status":"ok","attempt":2,"value":{"action":"sign_document","subject_line_index":1,"confidence":"high"}}
  bad then bad:  {"status":"needs_review","attempt":2,"value":null}

Three experiments worth doing once it runs:

1. Delete additionalProperties: false and re-run. The extra field case flips from REJECT to ACCEPT. That is the wrapper bug, arriving quietly.
2. Change action from an enum to { type: 'string' }. The invented enum case starts passing. Your switch statement now has a silent default branch.
3. Raise maximum on subject_line_index above the length of the array. The index off the end case passes and the lookup returns undefined — the failure moves from your validator into whatever reads the result.

Each one takes about ten seconds and shows you which specific line was holding which specific bug shut.

Where this goes next

This is how every model-backed step in Workflow Builder returns its result: a constrained schema, a validator between the model and the run state, and a needs_review outcome that a person can act on.

The pattern is not really about JSON. It is the same boundary as refusing to evaluate a user's expression as JavaScript and the same one as not sending an inbox to a model to find out whether it mattered: decide what the untrusted side is allowed to say, then enforce it in code that you control.