Instructions Are Hints: Guardrails for an LLM Tool Loop

I wrote the same rule three times.

Once in the tool description, so the model would read it while deciding what to call. Once in the system prompt, in capitals, because the first one wasn't holding. And once in the executor, as four lines of JavaScript that simply refuse.

Only the third one works. That's most of what I learned building this.

The project was an assistant embedded in an enterprise CRM. A user asks a question in plain language, and the model answers from live records rather than from anything it was trained on. No hardcoded queries, no baked-in field lists. It navigates the data model itself.

It never shipped. The vendor released their own agents partway through, they were better suited to the domain than mine, and I stopped. The guardrails below are the part worth keeping, and they had nothing to do with that decision.

TL;DR

Publish a filtered schema artifact instead of putting the data model in the prompt. Give the model three tools: list the objects, describe named objects, query one object. Then assume every instruction you write will be ignored eventually, and enforce the ones that matter in the executor. Instructions are advisory. Code is not.

The problem: 530 objects

The org's metadata extract describes 530 objects across roughly 13,000 fields.

You cannot put that in a prompt. You also cannot hardcode the queries, because the whole point was that adding an object shouldn't mean a code change. And you can't let the model guess field names, because it will, confidently, and the query will fail in a way that costs you three hops to recover from.

The field counts are lopsided in a way that matters. The median object has 30 fields. The widest has 232. So "just fetch the schema for the object you need" is fine right up until the model picks the wrong one and drops 232 field definitions into the conversation.

Filter first, at build time

Before anything reaches the model, a Node script pulls the org's metadata extract and cuts it down:

An object allowlist. 85 API names, hand-picked, covering the parts of the CRM anyone actually asks about. That's 16% of the org. Adding an object is a one-line edit and a re-fetch, which was the design goal.

A system-field blocklist. Exact names for the obvious ones (created_by, modified_date), plus patterns for the rest: anything ending _share, _history, _feed, _tag, anything starting audit_ or system_. Audit metadata is invisible to the model because no user has ever asked a question that needed it.

Picklist values dropped entirely. This one surprised me. The enumeration of every valid picklist value was roughly 80% of the file, and it's useless. The actual value comes back attached to each record when you query it. All you'd be shipping is a dictionary of cryptic codes for the model to wade through.

What comes out the other side: 85 objects, 3,482 fields, 487 KB. Around 120,000 tokens of data model.

Which never enters a context window. The client fetches it, holds it in memory for the session, and answers tool calls from it. The model sees a list of 85 names, then the fields for the two or three objects it actually needs.

Make the graph closed

One detail I'd repeat on any project like this.

Each field that points at another object gets a references annotation, and after all objects are built, a second pass populates a referencedBy reverse index. 366 edges, with 49 objects carrying at least one back-reference. So when the model reads any object's schema, it can see what connects to it without asking another question.

The part worth stealing: a reference is only attached if the target survived the allowlist.

If an object was filtered out, every pointer to it is silently dropped. The model never sees a link it can't follow. Without that, you get a model that reads a schema, spots a promising relationship, asks for the target object, gets told it doesn't exist, and burns two hops recovering from a dead end you created.

The graph the model sees is closed under the filter. That's four lines in the build script and it removes an entire category of confusion.

Three tools

list_objects       → every object, API name plus human label
get_object_schema  → fields for one or more named objects
query_records      → records from one object

The middle one takes an array, deliberately. The model needs five objects for a typical question, and five separate calls is five hops. One call with five names is one hop. The whole cost model of a tool loop is round trips, so batching is not a micro-optimisation.

And the loop executes every tool call in a single response concurrently. The model routinely groups independent work into one turn: schemas for two objects, a query against a third. If you handle only the first call and ignore the rest, you turn one hop into three, and non-trivial questions hit the cap.

const results = await Promise.all(
  calls.map((c) => executor(c.name, c.args ?? {})),
);

That's the entire mechanism. Easy to miss, expensive to miss.

The part I got wrong: instructions are hints

Here's the rule I wanted enforced: don't ask for a schema you already have.

Obvious, right? The earlier response is still in the conversation. Re-requesting it is pure waste, and the model can see its own history.

So I wrote it in the tool description, where the model reads it while choosing what to call:

DO NOT re-request a schema you already have in this conversation.

It kept happening. So I wrote it again in the system prompt, harder:

DO NOT re-request a schema you already have in this conversation. The tool response you got earlier is still in context, so read from there.

It still happened. Less often, and never on short conversations, which is worse in a way, because it meant the failure only appeared when the loop was already long and I'd stopped watching.

So I wrote it a third time, in the executor:

const schemasReturned = new Set();

case "get_object_schema": {
  const requested = args?.objects ?? [];
  const fresh = requested.filter((n) => !schemasReturned.has(n));
  const alreadyFetched = requested.filter((n) => schemasReturned.has(n));
  fresh.forEach((n) => schemasReturned.add(n));

  result = getObjectSchema(schema, fresh);
  if (alreadyFetched.length > 0) {
    result.alreadyFetched = alreadyFetched;
    result.note = "Objects listed in `alreadyFetched` were returned in an "
      + "earlier tool response. Consult it instead of re-requesting.";
  }
  break;
}

A Set. That's it. Ask twice and you get the names back with a note instead of the payload.

The three layers do genuinely different jobs, and I'd argue you want all three:

The tool description is read at decision time, when the model is choosing what to call. Highest leverage per word, and the surface people most often leave as bare API documentation. Mine tells the model when not to call a tool: "if you already have a good guess, skip this and go straight to get_object_schema."

The system prompt sets policy for the whole conversation. Good for things that aren't about any single tool.

The executor is the only one that's true. Everything above it is a strong suggestion to a system that is, by construction, allowed to ignore you.

I'd got that backwards for a while. I kept editing prose, because editing prose feels like progress and writing a guard feels like admitting the prose didn't work. It didn't work. Four lines of code fixed what two rounds of increasingly emphatic capital letters could not.

Errors that tell the model what to do next

The same idea applies to failures. Every guard returns something the model can act on:

// object not on the allowlist
{ error: "Unknown object 'x'. Call list_objects first." }

// running outside the host application
{ error: "queryRecord unavailable. Not running inside the CRM." }

That second half of each message is the useful part. An error saying "unknown object" gets you a model guessing again. An error saying "unknown object, call list_objects first" gets you a model calling list_objects and recovering in one hop.

You are writing error messages for a reader who will act on them immediately and literally. That's a different audience from the one your usual API errors are written for, and it's worth adjusting the tone.

The token discovery

This one cost me an afternoon.

I'd set maxOutputTokens to 1024. Seemed generous, given the answers are meant to be short, under 250 words, on a tablet screen.

Responses came back truncated. Sometimes mid-sentence, sometimes empty with finishReason: MAX_TOKENS and no text at all. Which makes no sense for a 250-word answer, until you look at the usage numbers the API returns per call:

tokens in/out/thoughts: 6412/240/1180

240 tokens of answer. 1,180 tokens of reasoning. Five times more thinking than output, and on this model family the thinking counts against the same output budget.

So a 1024 cap wasn't a generous budget for a short answer. It was a budget the model could exhaust entirely on internal reasoning before writing a single word the user would see.

I raised it to 8192 and logged finishReason plus the full usage breakdown on every hop. Two things I'd do from the start next time:

Log the token split per call. Not just the total. The in/out/thoughts breakdown is what turns "why is this truncated" into an answer in ten seconds.

Filter the reasoning out of what you render. Those parts arrive flagged, and if you concatenate every text part naively you will ship "let me do a more precise count here…" straight into the user's chat bubble. One filter, easily forgotten:

const textParts = parts
  .filter((p) => typeof p.text === "string" && !p.thought)
  .map((p) => p.text);

Two kinds of budget

The loop has a hard ceiling of 10 hops. Past that it throws, and the error carries a trail of which tools ran on which hop, so you can see what the model was doing when it got stuck.

But the system prompt separately says: aim to finish common questions in 2-3 turns.

Those aren't the same mechanism. The soft budget shapes behaviour, because the model batches more aggressively when it knows it's meant to be quick. The hard cap is there for when shaping fails. One is a nudge, the other is a wall, and a loop with only one of them is either sloppy or unnecessarily rigid.

Same for the ID rule, which is a user-experience guardrail rather than a cost one:

Never show raw record IDs. If a record has an ID pointing at another object, query that object for its name and use that instead. "Dana Whitfield (12 interactions)", never "rec_8f31c0a4: 12 interactions."

Nothing enforces that in code. It's a genuine soft rule and it mostly holds, because unlike the schema-refetch rule it aligns with what the model is naturally inclined to do.

That's the distinction worth internalising: soft rules work when they push in the direction the model already leans. Enforce the ones that push against it.

Keyword tasks over a general assistant

One structural choice that paid off more than expected.

Rather than one assistant answering anything, four keyword-triggered tasks. Type brief and the system prompt gains a fragment naming exactly which objects to fetch, in what order, and what shape the answer should take. Type nothing special and you get the general mode.

Each task prompt is specific to the point of being boring:

Look up the customer id from the page context. Then, in ONE turn,
batch these schema lookups:
  • get_object_schema for: customer, interactions, orders, open_tasks, notes

Then, in the NEXT turn, emit these queries IN PARALLEL:
  • customer where id = '<id>' (limit 1)
  • interactions where customer = '<id>' order by date desc (limit 5)
  • orders where customer = '<id>' order by date desc (limit 10)
  ...

That converts an open-ended retrieval problem into a mostly-determined one. The model still resolves IDs and synthesises the answer, but it isn't rediscovering the data model on every request.

The general mode still exists for anything the four tasks don't cover. It's just slower and less reliable, which is the honest trade.

Do you need a domain-specific model for this?

The question I got asked most, and the one I'd been expecting to answer yes to.

The answer here was no, and I think the threshold sits further out than people assume. 85 objects and 3,482 fields fit comfortably into retrieval over a general model. The model doesn't need to know the data model. It needs to be able to look it up cheaply, and three tools over a 487 KB artifact does that.

What I deferred, with the trigger written down so it isn't a vibe:

Vector search over the schema. Worth it if the allowlist grows past roughly 200 objects and matching a question to the right object starts failing on names alone. At 85 it never did.

Curated domain packs. Named groups of related objects the model could request as a unit. The task prompts already name their objects explicitly, so packs only help if free-form questions start missing cluster boundaries.

A server-side proxy for the model API. Not optional for production. Client-side environment variables compile into the bundle as plain strings. Fine for a prototype key you rotate, wrong for anything real.

Writing the trigger next to the deferral is the part I'd keep. "We'll add embeddings if we need them" decays into either never revisiting it or adding it because it sounded due. "We'll add embeddings past 200 objects" is a decision you can actually check against.

What I'd keep

Strip the domain out and the transferable parts are short:

1. Publish the schema as a versioned artifact. Filter it hard at build time, and never load it into context.
2. Close the graph under your filter. No pointers to things the model can't fetch.
3. Batch by design. Array parameters, parallel execution per hop. Round trips are the cost.
4. Write the rule three times, and mean the third one. Tool description, system prompt, executor. Only the last is enforcement.
5. Make errors instructive. Say what went wrong and what to call next.
6. Log the token split per hop, including reasoning tokens.
7. Two budgets. A soft one to shape behaviour, a hard one for when shaping fails.
8. Write the trigger next to every deferral.

None of it is clever. All of it came from watching a loop misbehave and fixing the specific thing, which is the only method I know that reliably works.

This is the third of three on the same idea

Part 1 built a queryable index over a codebase so a search returns ten lines instead of a repository. Part 2 handed an agent the dependency neighbourhood around a change rather than the whole project. This one hands a model 85 objects out of 530.

Same argument three times: the win is in what you don't send. The context window keeps getting bigger and it keeps not being the answer.

• Build a TypeScript Code Graph with the Compiler API
• Code Graphs for AI Coding Agents

---

I'm Ashish Kr Jha, a software engineer. I build things in production and then have to live with them, which is the part that actually teaches you something. More at ashishkrjha.dev.