<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Everyone gives agents skills. I made skills hatch their own agents.]]></title><description><![CDATA[Everyone gives agents skills. I made skills hatch their own agents.]]></description><link>https://agenthatch.hashnode.dev</link><image><url>https://cdn.hashnode.com/uploads/logos/6a3486abb6601d504a18e52d/0be1ff0b-5109-43fc-a542-c61aad1e87f6.png</url><title>Everyone gives agents skills. I made skills hatch their own agents.</title><link>https://agenthatch.hashnode.dev</link></image><generator>RSS for Node</generator><lastBuildDate>Thu, 24 Sep 2026 12:24:49 GMT</lastBuildDate><atom:link href="https://agenthatch.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Hatching a D&D spellbook sage: four RAG bugs and one that almost hid behind an editable install]]></title><description><![CDATA[I spent last week building a fake D&D 5e spellbook agent. Not because I needed one — I needed to break agenthatch v1.0.1's RAG pipeline in ways the existing test suite couldn't catch.
Four bugs fell o]]></description><link>https://agenthatch.hashnode.dev/hatching-a-d-d-spellbook-sage-four-rag-bugs-and-one-that-almost-hid-behind-an-editable-install</link><guid isPermaLink="true">https://agenthatch.hashnode.dev/hatching-a-d-d-spellbook-sage-four-rag-bugs-and-one-that-almost-hid-behind-an-editable-install</guid><category><![CDATA[#agent]]></category><category><![CDATA[skills]]></category><category><![CDATA[cli]]></category><category><![CDATA[GitHub]]></category><dc:creator><![CDATA[EternalRights]]></dc:creator><pubDate>Thu, 23 Jul 2026 14:22:50 GMT</pubDate><content:encoded><![CDATA[<p>I spent last week building a fake D&amp;D 5e spellbook agent. Not because I needed one — I needed to break agenthatch v1.0.1's RAG pipeline in ways the existing test suite couldn't catch.</p>
<p>Four bugs fell out. The last one is the interesting one, because it sailed through three rounds of verification before a fresh virtualenv caught it lying.</p>
<p><strong>Why a spellbook</strong></p>
<p>The agenthatch test suite already had three knowledge bases when I started. One was a fictional Chinese sword sect (a parallel experiment — long story). The other two were real-world topics that LLMs have seen a thousand times: Aetheria, a made-up planet, and a Chinese mythical star. The problem with real-world KBs is that you can never tell whether the agent actually retrieved from the index or just hallucinated from training data. The D&amp;D 5e SRD is technically real, but the long-tail mechanical details (exact damage dice, components, school) are easy to fact-check and easy to get wrong — which makes them a decent probe for "did retrieval actually run."</p>
<p>There's also a softer reason. Programmers and D&amp;D players overlap a lot. If I'm going to spend a week hand-writing markdown about evocation spells, I'd rather write about fireballs than about quarterly revenue benchmarks. Sue me.</p>
<p>The KB itself was small. Twelve spells, one file each: fireball, fire bolt, magic missile, shield, counterspell, cure wounds, healing word, light, darkness, mage armor, misty step, eldritch blast. Plus a <code>draft/secret-unreleased-spell.md</code> to test that the exclude-patterns filter actually excluded things, and a stray <code>api.secret</code>file to make sure secret-file conventions were caught.</p>
<p>That last file matters more than it sounds. If you tell users "we honor exclude patterns" and then quietly don't, you've built a leak. I wanted to know before any user did.</p>
<p><strong>Bug #6: a YAML that lied to itself</strong></p>
<p>The first thing I noticed was weird. After hatching, the agent's <code>knowledge_base.py</code> had constants <code>TOTAL_CHUNKS = 45</code> and <code>INDEX_SIZE_BYTES = 4096</code> baked in. Those come from the actual built index. But the <code>agenthatch.yaml</code> sitting next to the skill said <code>total_chunks: 0, index_size_bytes: 0</code>.</p>
<p>Same agent, same hatch run, two files disagreeing about basic facts.</p>
<p>The cause was a write-ordering bug. The hatch command wrote <code>agenthatch.yaml</code> at step 11, before Phase 3.5 ran the KB index build. Phase 3.5 mutated the spec in memory — bumping chunk counts from 0 to 45 — but only the agent-output copy of the YAML got the updated values. The skill-dir copy stayed stale because nothing went back to refresh it.</p>
<p>The fix was the obvious one: after <code>generate()</code> returns, re-dump the spec to the skill-dir YAML. It's a fifteen-line patch. The interesting part is that nobody had caught it for weeks, because the stale file lived in <code>tests/fixtures/</code> and the test suite never read it back. Tests assert what you tell them to assert.</p>
<p><strong>Bug #7: the dependency that wasn't</strong></p>
<p>Next one. I ran <code>pip install -e .</code> on the generated agent and called <code>retrieve("fireball")</code>. Got back an empty list. Every time. For every query.</p>
<p>The generated <code>knowledge_base.py</code> does this: from agenthatch_core.bricks.knowledge.store import KnowledgeStore</p>
<p>But the generated <code>pyproject.toml</code> didn't list <code>agenthatch-core</code> as a dependency. So <code>pip install -e .</code>succeeded (because the import only fires at runtime, not at install time), and then <code>retrieve()</code> swallowed the <code>ImportError</code> inside <code>_get_store()</code>'s exception handler and returned <code>[]</code> for everything.</p>
<p>This is the worst kind of bug. The user sees no error. They see empty results. They probably blame the LLM, or the chunker, or themselves. They almost never blame the dependency list.</p>
<p>Fix was a Jinja conditional in <code>pyproject.toml.j2</code>: if <code>kb_enabled</code>, add <code>agenthatch-core</code> and <code>sentence-transformers</code> to the dependencies list. Two lines of template, plus a paragraph of comment explaining why.</p>
<p><strong>Bug #8: a top_k that lied about its own contract</strong></p>
<p>This one I found by reading code, not by running it. The <code>KnowledgeStore.search()</code> method had this: if top_k &lt;= 0: top_k = 1</p>
<p>With a comment that said, with a straight face, "preserves the contract of returning at most top_k results" and "surfaces user intent of getting at least one result." Both claims were wrong. Returning 1 result when the user asked for 0 violates "at most top_k." And <code>top_k=0</code> doesn't express intent for "at least one." It expresses intent for zero.</p>
<p>A user passing <code>top_k=0</code> is asking for an empty list. Maybe they're warming up a UI. Maybe they're measuring latency without payload. Maybe they have a reason I can't predict. The job of the API is to honor the contract, not to second-guess it.</p>
<p>Fix: if top_k &lt;= 0:return []</p>
<p>With a warning log so the call is still traceable. Negative <code>top_k</code> hits the same branch — SQLite would treat <code>LIMIT -1</code> as "no limit" without the early return, which is its own kind of bug.</p>
<p>The lesson I keep coming back to: API contracts shouldn't have escape hatches for "what the user probably meant." If they meant 1, they'd pass 1.</p>
<p><strong>Bug #4: the one that almost got away</strong></p>
<p>This is where the story gets good.</p>
<p>The original Bug #4 was straightforward. The generated <code>pyproject.toml</code> declared <code>packages = ["src/&lt;pkg&gt;"]</code> and nothing else. When a user ran <code>pip install</code>, only the Python package shipped. The <code>knowledge/</code> directory — containing the pre-built SQLite index — stayed on the developer's machine. At runtime, <code>_KB_INDEX_DIR</code> resolved to a path inside site-packages that didn't exist, and <code>retrieve()</code> returned <code>[]</code> for every query. Same failure mode as Bug #7, different root cause.</p>
<p>The fix had two parts. First, add <code>force-include = { "knowledge" = "knowledge" }</code> to the generated <code>pyproject.toml</code> so hatchling ships the directory in the wheel. Second, replace the single hardcoded path in <code>knowledge_base.py</code> with a resolver that tried multiple candidate locations — because dev layout and pip layout put <code>knowledge/</code> in different places relative to the package.</p>
<p>I wrote the resolver like this: def _resolve_kb_index_dir() -&gt; Path: here = Path(<strong>file</strong>).resolve().parent # / candidates = [ here.parent.parent / "knowledge", # pip layout (site-packages/knowledge/) here.parent.parent.parent / "knowledge", # dev layout (&lt;agent_dir&gt;/knowledge/)]for c in candidates:if c.exists():return c return candidates[-1]</p>
<p>I tested it three ways. Unit tests in <code>test_kb_regressions.py</code> — twelve of them, all passing. End-to-end Python script that called <code>GenerateEngine.generate()</code> directly, bypassing the LLM harness, then ran <code>retrieve("fireball")</code> on the generated agent. Three results, all from <code>spells/fireball.md</code>. Then a virtualenv with <code>pip install -e .</code> — editable install — and <code>retrieve("fireball")</code> returned 3 results, <code>retrieve("magic missile")</code> returned 2 results, <code>retrieve("fireball", top_k=0)</code> returned 0.</p>
<p>I marked Bug #4 as fixed and moved on.</p>
<p><strong>The fourth verification, and the lie</strong></p>
<p>I had one verification left that I almost skipped. The editable-install venv had passed. The unit tests had passed. The end-to-end script had passed. I was done, by any reasonable standard.</p>
<p>But something bugged me. Editable installs symlink back to the source tree. The <code>__file__</code> of an editable-installed module resolves to the source path, not to a site-packages copy. Which means an editable install's layout is identical to the dev layout — the resolver's second candidate, the one labeled "dev layout," would always hit first for editable installs.</p>
<p>I had never actually tested a non-editable install.</p>
<p>So I built a fresh venv and ran <code>pip install &lt;agent_dir&gt;</code> — no <code>-e</code>. The install succeeded. The <code>knowledge/</code>directory showed up in site-packages, right where <code>force-include</code> said it would. I fired up a Python REPL and called <code>retrieve("fireball")</code>.</p>
<p>Zero results.</p>
<p>The resolver returned <code>_KB_INDEX_DIR = /private/tmp/dnd_venv3/lib/python3.14/site-packages/../knowledge</code> — that's <code>site-packages</code>'s parent, which doesn't exist. The fallback path returned a directory that wasn't there, and <code>_get_store()</code> logged a warning that nobody would see.</p>
<p>The bug was in my candidate list. Look at the resolver again: here.parent.parent / "knowledge" # I called this "pip layout"</p>
<p>For an editable install, <code>here</code> resolves to <code>&lt;agent_dir&gt;/src/&lt;pkg&gt;/</code>. Two levels up is <code>&lt;agent_dir&gt;/</code>, and <code>knowledge/</code> lives there. That's why it worked.</p>
<p>For a non-editable install, <code>here</code> is <code>&lt;site-packages&gt;/&lt;pkg&gt;/</code>. Two levels up is <code>&lt;site-packages&gt;</code>'s parent — typically <code>&lt;python&gt;/</code>, which has no <code>knowledge/</code> directory. The actual <code>knowledge/</code> shipped by <code>force-include</code> lives one level up from <code>&lt;pkg&gt;/</code>, at <code>&lt;site-packages&gt;/knowledge/</code>. That's <code>here.parent / "knowledge"</code>. Not two levels up.</p>
<p>I had an off-by-one error in my "pip layout" candidate, and editable installs had been quietly covering for me.</p>
<p>The fix was a one-line change: candidates = [ here.parent / "knowledge", # non-editable pip layout here.parent.parent / "knowledge", # dev / editable layout]</p>
<p>I added three regression tests that exec the rendered resolver against a fake <code>__file__</code> pointing at each layout. Pip layout passes. Dev layout passes. When both directories exist, the pip-layout candidate wins because it's listed first — which is what we want, because pip layout is what users actually hit.</p>
<p>Then I rebuilt the venv, reinstalled, and ran <code>retrieve("fireball")</code> one more time. Three results. <code>spells/fireball.md</code>, <code>spells/fire-bolt.md</code>, <code>spells/counterspell.md</code>. <code>retrieve("magic missile")</code>returned 2. <code>retrieve("fireball", top_k=0)</code> returned 0.</p>
<p>The off-by-one was a thirty-second fix. Finding it took a fresh venv and a willingness to test the install path that nobody on the team had bothered to test, because editable installs are faster and we're all impatient.</p>
<p><strong>What I actually learned</strong></p>
<p>Editable installs lie. They symlink back to source, which means any path resolution that happens to work in your dev checkout will also work in an editable install — for the wrong reason. The bug only surfaces when you test the install path your users actually use, which is <code>pip install &lt;dir&gt;</code>, not <code>pip install -e &lt;dir&gt;</code>.</p>
<p>This isn't a new lesson. It's the same lesson as "run your tests on CI, not just on your laptop." But it keeps needing to be relearned, because editable installs are convenient and convenience is seductive.</p>
<p>The other thing. When I wrote the resolver, I labeled the candidates with comments — "pip layout" and "dev layout." The comments were wrong about which was which. The tests passed anyway, because I was testing the labels, not the layouts. If I'd named the candidates <code>one_level_up</code> and <code>two_levels_up</code> instead, I would have caught the off-by-one in five seconds. Naming matters. Labels that describe intent rather than mechanism are a tax you pay later.</p>
<p>The whole bug hunt took a week. Four bugs, sixteen regression tests, four verification rounds. The D&amp;D spellbook sage now lives in <code>tests/fixtures/skills/dnd-spell-advisor/</code>, and it'll keep stress-testing the RAG pipeline for as long as anyone runs the test suite. Which, given that this is open source, is probably forever.</p>
<p>Twelve spells. Four bugs. One near-miss. Not bad for a week of fake fireballs.</p>
<hr />
<p><em>agenthatch is open source:</em> <a href="https://github.com/agenthatch/agenthatch"><em>github.com/agenthatch/agenthatch</em></a></p>
]]></content:encoded></item><item><title><![CDATA[Everyone gives agents skills. I made skills hatch their own agents.]]></title><description><![CDATA[I spent two weeks writing a SKILL.md for our internal agent stack at work. Every API endpoint, every MCP config, every token-budget rule I'd bled for over three months, written down in clean markdown.]]></description><link>https://agenthatch.hashnode.dev/everyone-gives-agents-skills-i-made-skills-hatch-their-own-agents</link><guid isPermaLink="true">https://agenthatch.hashnode.dev/everyone-gives-agents-skills-i-made-skills-hatch-their-own-agents</guid><category><![CDATA[AI]]></category><category><![CDATA[skills]]></category><category><![CDATA[agentic AI]]></category><category><![CDATA[#agent]]></category><category><![CDATA[llm]]></category><category><![CDATA[Python]]></category><category><![CDATA[claude]]></category><category><![CDATA[cli]]></category><dc:creator><![CDATA[EternalRights]]></dc:creator><pubDate>Fri, 19 Jun 2026 13:00:42 GMT</pubDate><content:encoded><![CDATA[<p>I spent two weeks writing a SKILL.md for our internal agent stack at work. Every API endpoint, every MCP config, every token-budget rule I'd bled for over three months, written down in clean markdown. I handed it in as a deliverable.</p>
<p>Then I watched Claude Code run it.</p>
<p>My strict requirements, the ones in bold, the ones that said "check environment variables first" and "exit immediately on type X errors"? The model skimmed past them like they were terms of service. It treated the skill as a reference book, not a contract. Every run, the interpretation drifted a little. One run it drifted far enough that a bug which should have died in staging almost made it to production.</p>
<p>That was the night I decided skills shouldn't be interpreted. They should be compiled.</p>
<p>I'm an agent developer intern at DiDi and a junior CS student. I build this stuff for a living. And I'm tired of watching LLMs selectively ignore prose that took me weeks to write. So I built <a href="https://github.com/agenthatch/agenthatch">agenthatch</a>. It turns a SKILL.md into a standalone, pip-installable Python agent with typed tools, a state machine, and its own runtime. Not a prompt wrapper. Actual generated code.</p>
<pre><code class="language-bash">pip install agenthatch
agenthatch init
agenthatch skills add ./my-skill/SKILL.md
agenthatch hatch my-skill
agenthatch run my-skill
</code></pre>
<p>Three commands from markdown to a running agent. The rest of this post is the why and the how, with source file references so you can verify I'm not making this up.</p>
<img src="https://dev-to-uploads.s3.us-east-2.amazonaws.com/uploads/articles/cvlzui9xdm2edldhebwp.jpg" alt="hatch" style="display:block;margin:0 auto" />

<img src="https://dev-to-uploads.s3.us-east-2.amazonaws.com/uploads/articles/02uu32ttgyaos1cdtdfv.png" alt="run" style="display:block;margin:0 auto" />

<hr />
<h2>The problem isn't your skill. It's the format.</h2>
<p>Let me be precise about what's broken.</p>
<p>A SKILL.md file is prose. Human-written prose for humans to read. You then paste it into a system prompt and ask an LLM to figure out what you meant, at runtime, every single turn.</p>
<p>One skill, fine. Three skills, manageable. Five or more? Things break.</p>
<table>
<thead>
<tr>
<th><strong>What happens</strong></th>
<th><strong>Why</strong></th>
</tr>
</thead>
<tbody><tr>
<td>Skills leak into each other</td>
<td>Every skill shares one context window. A file-organizer skill and a git-ops skill cross-contaminate. The agent applies logic from one to the other.</td>
</tr>
<tr>
<td>The agent skim-reads</td>
<td>Long skills get treated as loose suggestions. The model picks the parts that look relevant and ignores the rest.</td>
</tr>
<tr>
<td>Token waste</td>
<td>Every skill lives in the system prompt. 5 skills at 3KB each means 15KB burned before the conversation starts. Long tasks compound this.</td>
</tr>
<tr>
<td>No validation</td>
<td>A typo in a tool name, a missing parameter, an ambiguous instruction. None of it gets caught until runtime, and by then you're 20 turns deep.</td>
</tr>
<tr>
<td>Scale decays</td>
<td>1 to 3 skills works. At 10+ it's chaos. No dependency graph, no conflict detection, no way to know which skill overrides which.</td>
</tr>
</tbody></table>
<p>This isn't an Anthropic bug or an OpenAI bug. It's not your skill being bad either. It's architectural decay. You're asking one LLM to interpret seven pieces of zero-isolation prose in the same context window, and the interpretation shifts every run.</p>
<p>Think about it this way: you hand someone seven operating manuals and ask them a question. They have to flip through all seven books every time and stitch an answer together. That person would lose their mind. The LLM does too, just quietly.</p>
<p>The core issue is that SKILL.md is prompt engineering, not software engineering. There's no compile step, no type checking, no contract between you and the model.</p>
<hr />
<h2>The thesis: skills are source code, agents are binaries</h2>
<p>This is the part I want you to actually think about, even if you skip the rest.</p>
<p>Right now everyone is writing skills. For Claude Code, for Codex CLI, for OpenClaw, for whatever comes next. And in all of these, the skill's role is the same: it's prose stuffed into a system prompt, interpreted at runtime by a host agent. The skill is a prompt accessory. It can't exist on its own.</p>
<p>That's the wrong paradigm.</p>
<p>Skills shouldn't be prompt accessories. Skills should be agent source code.</p>
<p>Java compiles to bytecode for the JVM. TypeScript compiles to JavaScript for the browser. The compile step exists because it converts human expression into a format the machine can execute deterministically, before runtime. Typos, type errors, ambiguity, all of it gets caught at compile time, before the agent ever runs.</p>
<p>SKILL.md has no compile step. It hands raw prose to an LLM and hopes for the best, every turn, forever.</p>
<p>So the endgame for skills isn't "write them better." It's "compile them into agents." A skill should be the hatching input for an agent. You write the skill, a compiler turns it into an independent runtime with its own tools, its own state machine, its own config. The skill stops being a prompt fragment and becomes a program.</p>
<p>That's what agenthatch does. You still write skills in markdown, that part doesn't change. The generated agent runs independently, no host required. agenthatch is the step in between, the compiler. <code>javac</code> is to <code>.java</code> what <code>agenthatch</code> is to <code>SKILL.md</code>.</p>
<p>Once you internalize this, a bunch of things click into place:</p>
<ul>
<li><p>Skills stop burning tokens in the system prompt. Compiled agents carry about 150 bytes of runtime config.</p>
</li>
<li><p>Each skill becomes an isolated agent. No more cross-contamination.</p>
</li>
<li><p>Schema validation happens at compile time. Typos and ambiguity die before runtime.</p>
</li>
<li><p>The output is a real Python package. <code>pip install</code>, <code>import</code>, run it anywhere, no host required.</p>
</li>
</ul>
<hr />
<h2>The pipeline: three phases, six harnesses</h2>
<p>Here's the architecture. Every file path I mention is real, you can open it in the repo.</p>
<pre><code class="language-plaintext">SKILL.md  →  Parse  →  6-Harness LLM Pipeline  →  Code Generation  →  Runnable Agent
  (input)   (Phase 1)    (Phase 2: AI inference)     (Phase 3: Jinja2)     (output)
</code></pre>
<h3>Phase 1: deterministic parse, zero AI</h3>
<p>Phase 1 doesn't use AI. It reads the SKILL.md, pulls out the frontmatter, the body, and every file in the skill directory. Pure filesystem operations. The entry point is <code>assemble_context()</code> in <a href="https://github.com/agenthatch/agenthatch/blob/main/src/agenthatch/skill/parser.py">parser.py</a>:</p>
<pre><code class="language-python">def assemble_context(skill_path: str | Path) -&gt; ContextPack:
    skill_dir = _resolve_skill_directory(Path(skill_path))
    dir_name = skill_dir.name
    manifest = _discover_files(skill_dir)
    frontmatter, body, warnings = _best_effort_parse_yaml(skill_dir)
    return ContextPack(frontmatter, body, manifest, dir_name, warnings, skill_dir)
</code></pre>
<p>The key design decision here: Phase 1 makes no semantic judgment. It doesn't try to guess whether a file is a script, a doc, or a config. That's Phase 2's job. Phase 1 just reads bytes, computes SHA-256 hashes, and does YAML parsing.</p>
<p>There's a small detail I like in the file reader. It checks binary magic numbers to skip PNGs, JPEGs, PDFs, ZIPs, and friends:</p>
<pre><code class="language-python">_BIN_SIGS: list[bytes] = [
    b"\x89PNG\r\n\x1a\n",
    b"\xff\xd8\xff",
    b"GIF89a",
    b"%PDF",
    b"PK\x03\x04",
    b"Rar!\x1a\x07",
    b"\x1f\x8b",
    b"BZh",
    b"\xca\xfe\xba\xbe",
]
</code></pre>
<p>Files over 1MB get skipped. Files with null bytes in the header get skipped. If something can be handled deterministically, don't ask an LLM. That principle shows up over and over in this codebase.</p>
<h3>Phase 1.5: AST signature extraction</h3>
<p>This was added in v0.8 and I think it's the most underrated part of the whole project.</p>
<p>Phase 1.5 uses Python's built-in <code>ast</code> module to parse Python scripts and regex to parse shell scripts, extracting function signatures. Deterministic, zero LLM. This feeds into Harness C for precise interface inference.</p>
<pre><code class="language-python">def extract_python_signatures(file_path: Path) -&gt; list[ToolSchema]:
    """AST-parse a Python script, extract public function signatures.
    Deterministic, zero LLM. Uses Python's built-in ast module.
    Skips private functions (those starting with _).
    """
    try:
        tree = _ast.parse(file_path.read_text(encoding="utf-8"))
    except (SyntaxError, UnicodeDecodeError, OSError):
        return []

    functions: list[ToolSchema] = []
    for node in _ast.walk(tree):
        if isinstance(node, _ast.FunctionDef) and not node.name.startswith("_"):
            args: list[dict[str, str | None]] = []
            for arg in node.args.args:
                arg_type: str | None = None
                if arg.annotation:
                    try:
                        arg_type = _ast.unparse(arg.annotation)
                    except Exception:
                        arg_type = None
                args.append({"name": arg.arg, "type": arg_type})
            functions.append(ToolSchema(...))
    return functions
</code></pre>
<p>Why bother? Because Harness C has to design tool signatures. If it reads raw script contents, it burns tokens and hallucinates. Hand it a 1KB compact signature summary extracted via AST, and the inference quality jumps. This is compiler thinking: extract deterministically whatever you can, leave only the genuinely ambiguous stuff for the LLM.</p>
<h3>Phase 2: six AI harnesses</h3>
<p>This is the heart. Six specialized harnesses process the skill, each with its own persona and temperature. The config is hardcoded in <a href="https://github.com/agenthatch/agenthatch/blob/main/src/agenthatch/skill/engine.py">engine.py</a>:</p>
<pre><code class="language-python">HARNESS_CONFIG: dict[str, dict[str, Any]] = {
    "A": {"thinking": True, "temperature": 0.1,
          "reason": "Identity extraction is deterministic — low temp for consistency"},
    "B": {"thinking": True, "temperature": 0.5,
          "reason": "Intent inference requires creativity for long-tail triggers"},
    "C": {"thinking": True, "temperature": 0.5,
          "reason": "Interface inference is complex — needs SKILL.md + ScriptManifest"},
    "D": {"thinking": True, "temperature": 0.3,
          "reason": "Base detection needs precision — moderate temp"},
    "E": {"thinking": True, "temperature": 0.2,
          "reason": "Assembly validation is structured — low temp for consistency"},
    "F": {"thinking": True, "temperature": 0.3,
          "reason": "MCP config extraction needs exact matching — moderate temp"},
}
</code></pre>
<p>Every temperature has a reason. Identity extraction is deterministic, so temp drops to 0.1. Intent inference needs to cover long-tail triggers, so it gets 0.5 for creativity. Assembly validation is structured, so 0.2.</p>
<table>
<thead>
<tr>
<th><strong>Harness</strong></th>
<th><strong>Job</strong></th>
<th><strong>Model tier</strong></th>
<th><strong>Temp</strong></th>
</tr>
</thead>
<tbody><tr>
<td>A — Identity</td>
<td>Extract name, version, description from frontmatter</td>
<td>small</td>
<td>0.1</td>
</tr>
<tr>
<td>B — Intent</td>
<td>Infer trigger phrases and user intents</td>
<td>small</td>
<td>0.5</td>
</tr>
<tr>
<td>C — Interface</td>
<td>Design tool signatures, parameters, return types</td>
<td>large</td>
<td>0.5</td>
</tr>
<tr>
<td>D — Base</td>
<td>Detect runtime base class and instruction structure</td>
<td>large</td>
<td>0.3</td>
</tr>
<tr>
<td>E — Assembly</td>
<td>Cross-validate all harness outputs, produce AHSSPEC</td>
<td>small</td>
<td>0.2</td>
</tr>
<tr>
<td>F — MCP</td>
<td>Detect and configure MCP server connections</td>
<td>small</td>
<td>0.3</td>
</tr>
</tbody></table>
<p>Why six? Because I tried one giant prompt that did everything, and the output was a lottery. Splitting it up so each harness does one thing, quality went up significantly. Same reason compilers split the frontend into lexer, parser, semantic analysis. Single responsibility.</p>
<p>Each harness runs an Analyze, Infer, Self-Validate, Correct loop with up to two internal retries. Every harness has its own <code>validate_output()</code>. Harness A checks that <code>identity.id</code> is kebab-case:</p>
<pre><code class="language-python">def validate_output(self, result: dict[str, Any]) -&gt; tuple[bool, str]:
    identity = result.get("identity", {})
    identity_id = identity.get("id", "")
    if not identity_id:
        return False, "identity.id is empty"
    if not re.match(r"^[a-z0-9]+(-[a-z0-9]+)*$", identity_id):
        return False, f"identity.id '{identity_id}' is not kebab-case"
    if not identity.get("display_name"):
        return False, "identity.display_name is empty"
    return True, ""
</code></pre>
<p>Harness B checks that triggers count is between 5 and 15, satisfies between 3 and 8, summary at least 20 characters. These constraints aren't the LLM's call. They're enforced by code. If the LLM produces non-compliant output, it gets sent back to redo.</p>
<p>Harness E is the critical one. It cross-validates the other five and produces a unified AHSSPEC (Agent Hatch Standard Specification). E also computes a structural confidence score, and this is important: it's not LLM self-assessment, it's code counting fields:</p>
<pre><code class="language-python">def _compute_structural_confidence(self, ahs_dict: dict[str, Any]) -&gt; float:
    """Compute confidence based on structural checks, not LLM self-assessment."""
    checks = 0
    passed = 0
    id_ = ahs_dict.get("identity", {})
    for f in ("id", "display_name", "version"):
        checks += 1
        if id_.get(f): passed += 1
    iface = ahs_dict.get("interface", {})
    for f in ("provides", "requires"):
        checks += 1
        if iface.get(f): passed += 1
    score = round(passed / max(checks, 1), 2)
    return score
</code></pre>
<p>I don't trust LLM self-reported confidence. The model will cheerfully tell you "0.95 confidence" while missing three required fields. Code counting fields doesn't lie.</p>
<p>There's also a pre-flight classifier that picks model tiers per skill type. A pure-instruction skill skips Harness D entirely (no base class to detect). An integration skill with API calls and scripts upgrades everything to large models. Not every skill deserves the expensive model. The classifier saves real money.</p>
<pre><code class="language-python">MODEL_TIER_MAP: dict[str, dict[str, str]] = {
    "pure_instruction": {
        "A": "small", "B": "small", "C": "large", "D": "skip", "E": "small", "F": "small",
    },
    "script_driven": {
        "A": "small", "B": "small", "C": "large", "D": "large", "E": "small", "F": "small",
    },
    "integration": {
        "A": "small", "B": "large", "C": "large", "D": "large", "E": "large", "F": "small",
    },
    "knowledge": {
        "A": "small", "B": "large", "C": "large", "D": "small", "E": "small", "F": "small",
    },
}
</code></pre>
<h3>Phase 3: code generation</h3>
<p>Phase 3 renders the AHSSPEC into a complete Python package via Jinja2 templates. The engine is <code>GenerateEngine</code> in <a href="https://github.com/agenthatch/agenthatch/blob/main/src/agenthatch/generate/engine.py">generate/engine.py</a>:</p>
<pre><code class="language-python">TEMPLATE_MAP: dict[str, str] = {
    "pyproject.toml.j2": "pyproject.toml",
    "agent.py.j2": "src/{package_name}/agent.py",
    "tools.py.j2": "src/{package_name}/tools.py",
    "references.py.j2": "src/{package_name}/references.py",
    "runtime.toml.j2": "runtime.toml",
    "README.md.j2": "README.md",
}
</code></pre>
<p>But Phase 3 isn't just template rendering. There's an AI code generation step. <code>_ai_generate_tool_impls()</code> reads the full skill directory context (SKILL.md, reference files, script files, templates) and has the LLM generate real Python function bodies for each tool. Not stubs.</p>
<p>The critical part is validation. AI-generated code doesn't get written to disk directly. It gets <code>compile()</code>-checked first. If it fails, the engine tries to auto-fix indentation:</p>
<pre><code class="language-python">wrapper = "def _validate():\n" + indented + "\n"
try:
    compile(wrapper, f"&lt;tool:{func_name}&gt;", "exec")
except SyntaxError as se:
    try:
        fixed = GenerateEngine._normalize_indentation(indented_lines, error_lines)
        fixed_wrapper = "def _validate():\n" + fixed_str + "\n"
        compile(fixed_wrapper, f"&lt;tool:{func_name}&gt;", "exec")
        indented = fixed_str
        valid = True
    except SyntaxError:
        pass

if valid:
    valid_tools[func_name] = indented
else:
    logger.warning(
        "AI-generated code for tool '%s' has syntax errors, skipping. "
        "Tool will use template fallback.", func_name,
    )
</code></pre>
<p>This is the compiler's attitude. Generated code must compile. If it doesn't, fix it. If it can't be fixed, fall back to a template stub. Never let syntactically broken code reach runtime.</p>
<p>There's also a <code>_validate_generated_python()</code> pass that scans every generated <code>.py</code> file for JavaScript keywords that leaked in (<code>null</code>, <code>undefined</code>, <code>true</code>, <code>false</code>). Added in v0.7.15 after I watched an LLM try to write Python like it was TypeScript one too many times.</p>
<hr />
<h2>What comes out</h2>
<p>The output is a real Python package:</p>
<pre><code class="language-yaml">hatched-agent/
├── pyproject.toml          # pip-installable
├── runtime.toml            # LLM provider, model, API keys
├── README.md               # generated usage docs
├── agenthatch.yaml         # AHSSPEC manifest
└── src/{package_name}/
    ├── __init__.py
    ├── agent.py            # Agent class (extends AHCoreAgent)
    ├── tools.py            # type-annotated tool implementations
    └── references.py       # AI-extracted structured data
</code></pre>
<p>You can <code>pip install</code> it. You can <code>import</code> it. You can run it as a CLI. You can wrap it as an MCP server. It doesn't depend on Claude Code, Codex, or any host agent. It's a program.</p>
<p>The generated agent also carries a full copy of its source skill directory. It can read its own SKILL.md at runtime for self-reference, execute its own scripts, and self-repair when things go sideways. Like a compiled binary with debug symbols, not a stripped binary.</p>
<h3>The runtime: PlanLayer state machine</h3>
<p>The generated agent doesn't run a naive ReAct loop. It uses a 6-state PlanLayer state machine, defined in <a href="https://github.com/agenthatch/agenthatch/blob/main/agenthatch-core/src/agenthatch_core/bricks/plan.py">plan.py</a>:</p>
<pre><code class="language-python">class AgentState(str, Enum):
    STARTING = "starting"       # initial state, waiting for plan
    PLANNING = "planning"       # generating/updating plan
    EXECUTING = "executing"     # executing plan steps
    VERIFYING = "verifying"     # checking results
    REPLANNING = "replanning"   # hit a blockage, revising plan
    DONE = "done"               # terminal state
</code></pre>
<p>The agent generates a structured plan via a virtual <code>plan</code> tool at session start, then executes steps with explicit state tracking. Three consecutive tool failures trigger REPLANNING. State transitions are managed by the loop, not the LLM. The LLM is unreliable. State machines are reliable.</p>
<pre><code class="language-python">class PlanLayer:
    MAX_CONSECUTIVE_FAILURES = 3
    VERIFY_EVERY_N_STEPS = 5
</code></pre>
<p>The plan renders into the system prompt so the agent can see its own progress:</p>
<pre><code class="language-markdown">## Plan: Add i18n to the project
  ☐ Step 1: Install next-intl
  ▶ Step 2: Create language packs
  ☐ Step 3: Configure middleware
Progress: 1/3 steps done
</code></pre>
<hr />
<h2>Model support: pick your poison</h2>
<p>agenthatch supports OpenAI, Anthropic, DeepSeek, and any OpenAI-compatible endpoint. The harness system picks model tiers per task, so you can mix providers.</p>
<p>I've been testing with Claude Opus 4.5 for the large-tier harnesses (C and D, the ones doing interface and base detection) and GPT-5.2 Codex for comparison runs. Opus 4.5 is genuinely good at structured interface inference, the kind of task where you want the model to design a clean tool signature without inventing parameters. GPT-5.2 Codex is faster on the small-tier harnesses (A, B, E, F) where the job is extraction and validation, not design.</p>
<p>The point is: you're not locked in. The harness config is just a dict. Swap models per harness, swap providers, run the same SKILL.md through different stacks and diff the output. That's the kind of thing you can't do when your skill is glued to a host agent's system prompt.</p>
<hr />
<h2>What's still broken</h2>
<p>I'm not going to pretend this is finished.</p>
<ul>
<li><p><strong>Python only.</strong> JS/TS support is in progress. If you want a Node agent, not today.</p>
</li>
<li><p><strong>You need an LLM API key.</strong> Phase 2 is AI inference. No key, no hatch.</p>
</li>
<li><p><strong>Single-file skills work.</strong> Multi-file directory skills are in development.</p>
</li>
<li><p><strong>It's v0.9.x.</strong> There are bugs. I find new ones every week.</p>
</li>
<li><p><strong>Windows is untested.</strong> I develop on macOS and Linux. Windows users, tell me what breaks.</p>
</li>
</ul>
<p>I've shipped 8 PRs to pytest and 1 to agent-browser. agenthatch is my first from-scratch project, built nights and weekends. "Ship beats perfect" is the most useful thing I learned from open source. This tool isn't perfect, but it runs, and the rest gets fixed in flight.</p>
<hr />
<h2>Try it</h2>
<p>If you maintain more than three SKILL.md files and feel the friction, this is for you.</p>
<pre><code class="language-bash">pip install agenthatch
agenthatch init
agenthatch skills add ./my-skill/SKILL.md
agenthatch hatch my-skill
agenthatch run my-skill
</code></pre>
<p>Repo: <a href="https://github.com/agenthatch/agenthatch">github.com/agenthatch/agenthatch</a>​</p>
<p>PyPI: <a href="https://pypi.org/project/agenthatch/">pypi.org/project/agenthatch</a>​</p>
<p>If you hit bugs, file an issue. If you want to argue about the paradigm, the discussions tab is open. And if you think skills should stay as prompt fragments, I genuinely want to hear why, because I might be wrong and I'd rather find out now than after writing another 2,000 lines of compiler.</p>
<p>Skills are source code. Agents are binaries, and the compiler is the part everyone's been skipping.</p>
<p>​</p>
]]></content:encoded></item></channel></rss>