How to Write Instructions for a Custom AI Agent (Templates Included)

Most custom AI agents fail for the same boring reason: nobody told them how to behave. Teams spend weeks picking the right model and tools, then write the actual instructions in five rushed minutes — a paragraph of vague hopes instead of a real specification. If you want to know how to write AI agent instructions that actually hold up across hundreds of conversations, the answer isn't a smarter model. It's a better document.

This post gives you a repeatable framework, two ready-to-use templates, and real before/after examples so you can stop debugging agent behavior one weird response at a time.

Why Agent Instructions Are Different From One-Off Prompts

A single prompt asks for one output. Agent instructions have to hold up across dozens or hundreds of future conversations you haven't seen yet. That's a fundamentally different writing problem — closer to writing a job description than sending a text message.

Think about the difference between asking a new hire to "handle customer emails" versus giving them a documented process: who they're representing, what tone to use, what they're allowed to promise, and what to escalate. Vague instructions produce inconsistent agents. Specific, structured instructions produce reliable ones.

This is also why the highest-ROI technique in AI coding work — the specification file (think CLAUDE.md or Cursor rules) — exists at all. A spec file isn't a prompt; it's a standing set of instructions the AI references on every single interaction. The same logic applies to any custom agent, whether it's answering support tickets, drafting reports, or writing code.

The R-C-T-O Framework for Agent Instructions

We teach the R-C-T-O framework — Role, Context, Task, Output format — as the backbone of every strong prompt. For agent instructions specifically, R-C-T-O becomes even more important because you're defining these elements once for potentially unlimited future interactions.

Role — Who is the agent, and whose voice does it speak in? Be specific about expertise level, personality, and boundaries. "You are a helpful assistant" tells the model nothing useful.

Context — What does the agent need to know that isn't obvious? This includes your product, your audience, your constraints, and any facts it should never guess at.

Task — What is the agent actually supposed to do, decision by decision? Good task instructions cover the common path and the edge cases.

Output format — How should responses look? Length, structure, tone, formatting rules, what to include and exclude.

Here's what R-C-T-O looks like in practice, applied to a customer support agent:

Weak instructions:
"You're a support agent for our SaaS product. Be friendly and helpful. Answer customer questions."

R-C-T-O instructions:
Role: You are a Tier 1 support specialist for Northwind Analytics, a B2B dashboard tool. You have deep product knowledge but no access to billing systems.
Context: Customers are usually mid-level marketing managers, not technical users. Common issues involve dashboard filters, data refresh delays, and export formatting. Refunds and cancellations must always be escalated to a human — never promise one yourself.
Task: Diagnose the issue by asking one clarifying question if the request is ambiguous. Offer a solution using only the approved knowledge base. If the issue involves billing, account deletion, or a bug you can't confirm, escalate immediately with a summary.
Output format: Respond in 3-5 sentences, plain conversational tone, no bullet lists unless walking through steps. End every escalation with "I've flagged this for our team — expect a reply within 4 hours."

The second version doesn't just sound better — it produces measurably more consistent behavior because the model isn't guessing at boundaries you never defined.

Template 1: The Agent Spec Document

Borrowing directly from the specification-file pattern used in AI coding tools, we recommend writing every custom agent's instructions as a standalone document, not a buried system prompt. Use this structure:

# Agent Name: [name]

## Role
[Who the agent is, whose voice it uses, expertise level]

## Context
[Product/company facts, audience, known constraints, things it must never guess]

## Task
- Primary responsibility
- Step-by-step decision logic for common scenarios
- Explicit escalation/refusal conditions

## Output Format
- Length
- Tone
- Structure (bullets? prose? headers?)
- Formatting rules (no em dashes, no emojis, etc.)

## Examples
[2-3 few-shot examples of ideal input/output pairs]

## Known Failure Modes
[Things it has gotten wrong before, and the corrected behavior]

That last section — Known Failure Modes — is the one most teams skip, and it's the one that matters most. Every agent will eventually do something wrong. The teams who write it down and fold it back into the instructions get an agent that improves over time. The teams who don't get the same wrong answer, forever.

Template 2: The Few-Shot Behavior Block

Instructions alone often aren't enough — showing the agent examples of ideal behavior is one of the most reliable ways to lock in a specific style or decision pattern. This is the same principle behind few-shot prompting: teaching by example instead of by description.

## Examples

**Input:** "Can I get a refund for last month?"
**Ideal response:** "I don't have access to billing, but I've flagged this for our billing team — expect a reply within 4 hours. In the meantime, is there anything else I can help with?"

**Input:** "My dashboard isn't loading any data."
**Ideal response:** "Let's figure this out. Are you seeing a blank screen, or a specific error message?"

Two or three well-chosen examples usually outperform a full paragraph of abstract description, because they show the model the shape of a good answer instead of just describing it.

Before/After: A Coding Agent Instruction Set

Specification quality matters just as much for coding agents as it does for customer-facing ones. Compare these two approaches to instructing an AI coding assistant working inside a codebase:

Before (vague):
"Write clean code and follow best practices."

After (specific, R-C-T-O applied):
Role: You are a senior backend engineer working in our Python/FastAPI codebase.
Context: We use pytest for testing, follow PEP 8, and never use global variables. All new endpoints require input validation with Pydantic.
Task: When asked to write a function or endpoint, first check for existing utilities in /utils before writing new ones. Always include type hints.
Output format: Return the code first, then a short bullet list of any assumptions made. Include at least one test case for new functions.

The "after" version isn't longer for the sake of it — every line removes a decision the model would otherwise have to guess, which is exactly what learning how to write AI agent instructions well comes down to: replacing guesswork with explicit defaults.

Common Mistakes When Writing Agent Instructions

A quick debugging checklist for instructions that aren't producing consistent results:

• Too abstract. "Be professional" means something different to every reader — and every model. Replace adjectives with concrete rules.
• No escalation path. If you don't tell the agent what it can't do, it will confidently do it anyway.
• Missing output constraints. Length and format drift is one of the most common complaints teams have about agent output — and it's almost always fixable in the instructions, not the model.
• No examples. Description-only instructions leave more room for interpretation than a few well-chosen sample interactions.
• Never updated. Instructions written once and never revisited go stale as your product, policies, or edge cases change. Treat your agent spec like a living document, not a launch artifact.

If you want a deeper dive into structuring reusable prompt frameworks for exactly this kind of work, our Prompt Engineering course covers few-shot prompting and prompt library organization in more depth.

Key Takeaways

• Custom agent instructions are a specification, not a single prompt — write them for hundreds of future conversations, not one.
• Use the R-C-T-O framework (Role, Context, Task, Output format) to eliminate the guesswork that causes inconsistent agent behavior.
• Structure instructions as a standing document (like a spec file), including a "Known Failure Modes" section you update over time.
• Two or three strong few-shot examples often beat a paragraph of abstract description.
• Revisit and refine instructions regularly — the best agents are the ones whose instructions keep learning from real mistakes.

Ready to practice? Try a free scored exercise in the WellPrompted Playground — instant feedback on your prompting skills. Or start with our free AI Foundations course (7 modules, no credit card required).