This is the fourth piece in a series on designing the Metaglass AIOS kernel. Part 1 was on context management; part 2 on the agent loop; part 3 on the tool system. This one is about prompt composition — the layer that decides what the model "knows" at the moment it's called.

Every other part of the harness eventually serializes through this layer. The loop owns control flow, tools own capability, context management owns history, but prompt composition is where all of it gets stitched into the bytes the model actually reads. Get this layer wrong and the rest of the system can be perfect and the agent still won't do what you want.

Why naive implementations fall short

The first prompt anyone writes is a single hand-tuned string. "You are a helpful assistant. You have access to the following tools…" plus a list. It works for the demo. Then it doesn't:

  • A new tool is added; nobody remembers to mention it in the prompt and the model never calls it.
  • The user adds project rules ("always use kebab-case for filenames") in a config file; the prompt doesn't know how to load them.
  • The agent ignores the system instruction because the actual goal is buried three pages down past tool schemas.
  • The prompt is so verbose that the prompt cache misses on every turn and the cost triples.
  • One user wants a formal tone, another wants a casual one; there's nowhere clean to put that.
  • A directory has special conventions that don't apply elsewhere, and stuffing them globally just confuses the agent in unrelated parts of the codebase.
  • An external file (a README, a note) gets pulled into the prompt and turns out to contain a prompt injection — ignore previous instructions — that the harness happily forwards to the model.
  • The system reminder about "you currently have 3 todos open" never makes it to the model because it's in the wrong message slot.
  • The agent forgets a stable rule after compaction because the rule was buried in conversation history rather than re-injected from a durable source.

Each of these is a separate decision. The naive single-string prompt collapses them all into "rewrite the string." A real prompt composition layer is a pipeline of named, ordered sources that assemble into the final document, with explicit rules for who can write what, when it gets re-loaded, and where it sits in the message structure so the cache can do its job.

The shape the problem keeps taking

Reading enough prompt-composition implementations across the field, the same five-layer skeleton kept emerging:

  1. Identity — who the agent is. Stable across sessions. Defines voice, role, defaults.
  2. Rules and conventions — what the agent must always do (safety, formatting, project standards). Often hierarchical: global → user → project → directory.
  3. Capabilities — what tools exist and how to use them. Changes when tools change.
  4. Context — what's relevant right now: recalled memories, active files, retrieved notes, current state.
  5. Steering — per-turn nudges that influence behavior without altering the agent's identity. Todo reminders, planning prompts, budget warnings.

Every working system implements all five, though they differ in how strictly they separate them. The most opinionated systems put each layer in its own file, with its own loader and its own priority slot. The least opinionated concatenate everything into one string and hope.

That five-layer shape became the spine of AIOS's design. The boundary I cared about most: layers 1–3 are stable (cache-friendly, prepend-once), layer 4 is per-conversation (rebuilt at session start), layer 5 is per-turn (re-injected each iteration). Crossing those boundaries casually is what destroys prompt caching and makes the agent feel inconsistent.

Mechanism by mechanism

Layered sources with explicit priority

The single largest decision in a prompt-composition layer is whether there's one prompt or many sources composed into one. Every system that survived a real workload picked composition.

The composition rules are surprisingly stable across systems:

  • Each source has an explicit position in the assembly order.
  • Layers combine additively (they don't overwrite each other) unless explicitly designed to.
  • More specific sources go later so they can refine, not replace, more general ones.
  • Sources can be conditional — loaded only when relevant — to keep the prompt focused.

What I chose for AIOS: five distinct composition stages. A base system prompt (identity + Metaglass conventions: wikilinks, callouts, hub notes), a memory layer that injects recalled facts by topic, a tool-guidance layer that varies by the available tool set, an optional skill-enrichment layer that fires when a registered skill matches the input, and a kernel-injection layer that adds per-turn steering. Each stage is owned by a named module (AIOSRuntime, MemoryContextProvider, SystemPromptBuilder, SkillPromptProvider, ConversationEngine) and the boundaries between them are deliberate.

Identity as its own layer

The most disciplined systems treat identity as a separate, editable artifact rather than burying it inside the system prompt. Some put it in a SOUL.md file the user can edit. Some pull it from a configured profile. The common pattern: identity is something that should change rarely and can be customized without rewriting the rest of the prompt.

What I chose for AIOS: identity lives in the host's base system prompt today — a long, deliberately-written ~900-line block that defines Metaglass as a knowledge-management agent (notes are first-class, Obsidian conventions, MOC patterns, formatting rules). It's not yet a separate editable file. If AIOS grows toward per-user personas, splitting identity into its own loader is the natural next move; the seam is already there.

Hierarchical rule files (the CLAUDE.md pattern)

A recurring pattern: project-level instruction files (.cursorrules, CLAUDE.md, HERMES.md, .hermes.md) discovered by walking up the directory tree to a project root. Sometimes hierarchical — managed policy at the top, organization defaults, project rules, user rules — all combining additively.

The brilliant move in the best of these is that the file is re-read from disk on every request. Compaction may have summarized it out of conversation history, but the next turn gets a fresh copy. Static rules don't decay.

What I chose for AIOS: no filesystem rule files yet — and this is a real gap. Metaglass is built on a vault, which is exactly the right substrate for this pattern. The natural design: a designated vault note (or notes matching a glob) gets re-loaded into the prompt on every request, with directory-walking from the active note up to the vault root. This is the highest-priority addition I'd make to the prompt layer; the substrate is unusually well-suited and the cost is low.

Path-specific rules

A refinement of the hierarchical pattern: some rules only apply when the agent is touching specific files. Path-specific rules with a paths: frontmatter glob load only when the model is reading files matching that pattern. The model sees React conventions when working in src/components/**/*.tsx but not when editing a Python script in scripts/.

What I chose for AIOS: not yet implemented. Vault tags are a natural analog — load rules tagged for the active note's section, hub, or category. Same shape as path-specific rules, but tag-keyed rather than glob-keyed. Lands well after basic rule files are wired up.

Capabilities described, not just listed

Tool definitions go into every prompt regardless. The interesting question is whether the prompt also includes guidance about how to use them — when to prefer one tool over another, common anti-patterns, error modes — beyond the schema itself.

The disciplined systems do this. They include a "tool usage notes" section in the prompt that's separate from the tool schemas, and they vary that section by which tools are actually available. If only read tools are loaded, the guidance reflects read-only workflows. If write tools are loaded, the guidance discusses verification and confirmation.

What I chose for AIOS: SystemPromptBuilder adds a tool-guidance section keyed off CATEGORY_GUIDANCE — separate blocks for read, edit, and execute tool categories. The section appears only when tools of that category are present. This keeps the prompt focused: a planning-only run doesn't see edit guidance; a read-only run doesn't see execution guidance. The model isn't tempted by capabilities it doesn't have.

Memory injection at session start

A live conversation accumulates memory in two timescales: working memory (this session's history) and durable memory (facts learned across sessions). The durable memory has to get into the model somehow, and the cleanest pattern I saw was snapshot at session start rather than live-update mid-conversation.

The reasoning is subtle but important: every mid-conversation change to the system prompt invalidates the prompt cache. Live-updating memory is appealing in principle and ruinous in practice — every memory write costs you the cache hit on the next turn. Snapshotting memory at session start (so the prompt is stable for the duration) and writing updates to durable storage (so the next session sees them) preserves both correctness and cost.

What I chose for AIOS: MemoryContextProvider extracts topics from the current message, recalls matching memories, and injects them into the prompt with a 5-minute TTL cache. The injection is at session start; mid-conversation memory writes go to durable storage and surface on the next session, not the current one. This is the right tradeoff for AIOS and matches the cache-stable pattern.

Progressive disclosure for skills

Skills are a tricky case. A user might have 50 skills installed, and putting all their content in every prompt is wasteful. But the model needs to know the skills exist to be able to invoke them.

The clean pattern: progressive disclosure. The system prompt contains only skill metadata — names, one-line descriptions, when to use them. The full skill content is loaded on demand when the agent decides to invoke a skill. This keeps the per-turn cost proportional to the skills used, not the skills installed.

What I chose for AIOS: SkillPromptProvider matches user input against registered skills and enriches the prompt only when a trigger matches. The full skill body is loaded on match; non-matching skills cost nothing. This is closer to "matched-and-loaded" than to a full skill index in the prompt — a slight variation on progressive disclosure that fits AIOS's smaller skill catalog. As the catalog grows, moving to the index-then-load pattern becomes more attractive.

Per-turn steering as a separate channel

Per-turn signals — "you have 3 todos open," "you've used 70% of your turn budget," "consider planning before you start" — don't belong in the system prompt. The system prompt should be stable for caching; steering signals should change freely.

The pattern I saw most often: per-turn signals are injected as user messages or as tagged blocks inside tool results, not as system prompt modifications. They influence the current turn without touching the cache-stable prefix. Once the turn is over, they can be stripped from history so they don't pollute future context.

What I chose for AIOS: the kernel injects three kinds of per-turn signals as user messages at the top of each turn: intent classification (once per conversation, after the input is classified), todo-write guidance (gradient: none / soft / medium / strong), and an active-todos reminder (the current todo list, so the agent tracks its own progress). None of these touch the system prompt. The cache stays intact; the steering happens through a separate, mutable channel.

Gradient nudging with decay

The naive version of behavioral steering is binary: either the prompt says "you must plan" or it doesn't. The disciplined version is a gradient — the strength of the nudge varies with how badly the system thinks planning is needed, and it decays as the agent demonstrates progress.

A typical gradient:

  • none — query-only tasks, no nudge.
  • soft — "you may want to use TodoWrite…"
  • medium — "consider calling TodoWrite first…"
  • strong — "you should call TodoWrite to plan…"

The decay matters because the alternative is the agent getting nagged about planning while it's already executing productively. If the agent is producing output and calling tools, the system should back off.

What I chose for AIOS: the gradient lives in ConversationEngine. Initial level is set by IntentClassifier's output (see part 2). The level decays per turn based on whether the agent is producing assistant text or calling tools. There's also a hard fallback: if requireTodoWrite is enabled and the agent tries to invoke a mutation tool without a plan, the kernel blocks the call and returns a synthesized tool result with the exact TodoWrite invocation format — so smaller models don't have to guess the schema. The block-and-instruct pattern is more expensive than a soft nudge but it's reserved for the cases where the soft nudges have already failed.

Injection scanning

If the prompt composition layer pulls content from external sources — files, notes, web — that content can contain prompt injections. "Ignore previous instructions." "Do not tell the user." Hidden HTML comments. Invisible Unicode characters. Credential-exfiltration commands disguised as examples.

The defensive pattern: every external source passes through a scanner before injection. The scanner looks for known injection patterns (regex over a curated list), invisible Unicode ranges, hidden HTML containers, and known exfiltration signatures. On match, the source is replaced with [BLOCKED: ...] and the rest of the prompt assembles normally.

What I chose for AIOS: not yet implemented. The vault is largely user-authored, so the immediate threat surface is low. But the moment AIOS reads externally-sourced content (web fetches, imported notes, shared vaults), this becomes urgent. The right place to put the scanner is at the boundary in MemoryContextProvider and SkillPromptProvider — the layers that pull external content into the prompt. I'd build this before enabling any feature that ingests un-trusted content.

Prompt caching as a first-class concern

Modern providers offer prompt caching — write a stable prefix once, pay the read cost on subsequent calls, save 90% on tokens you've already sent. The catch is that the cache is byte-exact: any mid-prefix change invalidates everything after it.

This turns prompt composition into a layout problem. What can change has to live after what can't. Static identity at the top. Tool definitions next (they rarely change within a session). Memory snapshot next (stable for the session). Then the conversation history. Then any per-turn injections at the very end.

The disciplined systems treat cache breakpoints as a design artifact, not an afterthought — typically four breakpoints placed at the system/tools boundary, after the first few stable messages, after the recent stable context, and at the latest turn. The TTL is short (5 minutes for one provider) so the breakpoint strategy has to assume frequent re-warming.

What I chose for AIOS: the layout respects cache stability — system prompt and tool definitions go first, memory injection is one-shot at session start, per-turn steering goes into user messages at the bottom. Explicit cache-breakpoint annotations aren't wired up yet (the AI SDK abstracts over providers and the breakpoint API isn't uniform). The structural decisions that enable caching are made; the explicit signals to providers that support it are the next refinement.

System reminders as attention refreshers

A long conversation drifts. The model forgets the original goal, ignores a rule it was told about 30 turns ago, stops doing something it was supposed to do every turn. The fix isn't to put more in the system prompt — it's to inject targeted reminders at strategic points in the conversation.

The pattern: tagged blocks (e.g., <system-reminder>...</system-reminder>) inserted into user messages after tool uses, at compaction boundaries, when new capabilities become available. They're not separate messages; they're appendices to existing user messages so they don't break the message-pair structure providers expect.

What I chose for AIOS: the kernel does this implicitly — the active-todos reminder is exactly this pattern, injected as a user-message prefix at each turn. There's no general-purpose reminder system yet (no <system-reminder> tag, no central registry of "things to remind about"), but the mechanism is there. Generalizing it is a future move.

Mode-aware composition

Some prompts should be smaller than others. A subagent doing a focused subtask doesn't need the full identity prompt, the full memory recall, and the full skill index. A scheduled task doesn't need user-interactive guidance.

The cleanest pattern: an explicit mode parameter (full / minimal / none) that controls which layers get composed. The composer assembles different prompts depending on the caller, with the layers themselves unchanged.

What I chose for AIOS: today the engine selection (MinimalEngine vs. MetaglassEngine, see part 2) plays a similar role at a coarser grain. The minimal engine skips intent classification, gradient guidance, and the reflection/verification layers. A finer mode parameter at the prompt-composition layer — mode: 'subagent' skipping memory recall, for example — is a fair next refinement when subagents become more common.

Plugin/hook injection

The prompt composition layer is a natural extension point. Plugins want to add context, conventions, or instructions without modifying the core composer. The clean answer is a before_prompt_build hook that runs after history is loaded but before the prompt is built; plugins can return prependContext or systemPromptAddition strings.

What I chose for AIOS: the host's plugin architecture already has PluginAPI capabilities, and the prompt-composition seam for plugin contributions exists in the provider classes (a plugin can register additional skill triggers via SkillRegistry, or additional context blocks via the ContextPort). A dedicated before_prompt_build hook is on the roadmap; today, plugin influence is mediated through the existing extension points (skills, context blocks, tool registration) rather than a single hook.

Configuration-driven vs. programmatic assembly

Some systems read the prompt structure from config files. Others assemble it programmatically. The tradeoff: config-driven is more transparent and user-modifiable; programmatic is more flexible and type-checked.

The pragmatic answer is usually both — programmatic assembly with config-file inputs at specific named layers (identity, rules, persona). The composer is code; the content is data, and the data lives where users expect it.

What I chose for AIOS: programmatic assembly with no filesystem config layer today. The base prompt is in code; memory and skills are in the vault; everything else is computed. As I mentioned in the rule-files section, the vault is the right substrate for adding a configuration layer — designated vault notes that the composer reads. This is the move from "all programmatic" to "programmatic with vault-driven content layers," and it's a small change with large payoff for user customization.

The problem-to-mechanism table

Problem Mechanism
Single hand-tuned prompt becomes unmaintainable Five-layer composition with named modules per layer
Rules forgotten after compaction Re-read from durable source on every request (vault-backed rule files, deferred)
Tool capabilities not consistently used Per-tool-category guidance section, varies with available tool set
Memory has to influence behavior MemoryContextProvider snapshot at session start, topic-keyed recall
Many skills installed, few used per turn Skill enrichment fires only on trigger match (progressive disclosure variant)
Cache invalidates every turn Stable layers first, per-turn steering as user messages at the bottom
Agent ignores per-turn signals Tagged user-message injections (todos reminder, intent classification)
Nagging when the agent is making progress Gradient TodoWrite guidance with per-turn decay
Soft nudges fail; agent still skips planning Hard tool-call block with synthesized example TodoWrite payload
External content carries prompt injection (deferred) injection scanner at MemoryContextProvider / SkillPromptProvider boundary
Subagent doesn't need the full prompt Engine selection today; finer mode parameter deferred
Plugins want to influence the prompt Extension via skills, context blocks, tool registration; before_prompt_build hook deferred
Per-directory or per-section conventions (deferred) tag-keyed rules — vault analog of path-specific rules
User wants to customize identity (deferred) identity as a separate editable file
Configuration buried in code (deferred) vault-backed rule files for user-visible content layers

What comes next

The AIOS prompt-composition layer ships with the load-bearing pieces: five-layer composition, identity in a stable base prompt, memory injection at session start with TTL-cached topic extraction, dynamic tool guidance keyed by available tool category, skill enrichment via trigger matching, per-turn steering as user-message injections, gradient TodoWrite guidance with decay and a hard-block fallback, intent classification injected once per conversation, and active-todos reminders re-injected each turn.

The deferred items — vault-backed rule files with directory-walking, tag-keyed conditional rules, injection scanning, explicit cache-breakpoint annotations, a generalized system-reminder system, a fine-grained mode parameter, a before_prompt_build plugin hook, identity as a separate editable artifact — are all additive at clean seams. The composer doesn't need rewriting; each deferred item is a new source registered into an existing layer.

The highest-priority deferred item is vault-backed rule files. The substrate is unusually well-suited (Metaglass is a vault), the cost is low, and it unlocks user customization in the way users already expect it (markdown files in the vault). I'd build this next.

The highest-stakes deferred item is injection scanning. As soon as AIOS ingests external content, this becomes a security problem, not a polish problem. I'd build it before enabling any external-content feature.

Three lessons

If I had to compress this survey into advice for someone designing a prompt-composition layer from scratch:

  1. Prompts are not strings; prompts are layouts. The cache-stable prefix has to come first, the per-turn signals have to come last, and the layers in between have to be ordered by how often they change. Every "let me just stick this in the system prompt" decision has a cost in cache invalidation that compounds over a session. Lay the prompt out the way you'd lay out a database schema — with stability and access patterns in mind from the start.

  2. Identity and rules belong outside the conversation. Anything the agent should know regardless of conversation history should be loaded from a durable source on every request, not appended once and hoped to survive compaction. Files re-read from disk, vault notes re-loaded per turn, persona snapshots re-injected at session start. The conversation history is volatile; the things you care about most should live somewhere it can't reach.

  3. Steering is a separate channel from instruction. Per-turn nudges, reminders, budget warnings, and behavioral hints don't belong in the system prompt — they belong in user messages or tagged blocks inside tool results, where they can change freely without invalidating cache or polluting future context. Conflating instruction (stable, identity-shaping) with steering (transient, per-turn) is the single most common cause of bloated prompts that get worse over time.

The rest is tuning. Layer counts, source priorities, guidance gradients, cache breakpoint placement — dials, not decisions. The decisions are the three above, and the prompt-composition layers that get them right end up looking remarkably similar even when their authors didn't talk to each other.

Earlier in this series: Part 1: Context management · Part 2: The agent loop · Part 3: The tool system.