Harness & Governance
Use the runtime environment to control tools, permissions, side effects, budgets, human approval, recovery, and platform differences.
What controls permissions, runtime boundaries, side effects, platform adaptation, and recovery?Concept calibration
Harness
- What it is
- The runtime control environment that hosts models, tools, state, permissions, logs, budgets, human approval, and recovery.
- What it is not
- It can constrain risk and preserve evidence, but it cannot directly prevent model hallucinations.
Skill
- What it is
- A reusable capability package containing instructions, resources, operating boundaries, versioning, and compatibility metadata.
- What it is not
- It is not merely a prompt, and its scripts must not be assumed safe to execute automatically.
Tool
- What it is
- A constrained capability an agent can invoke through explicit arguments, permissions, and result contracts.
- What it is not
- A tool is not the same as MCP; a local function, command, or HTTP client can also be a tool.
Cross-Platform Adapters & Migration
Build capability, directory, and permission adapters for Codex, Claude Code, and WorkBuddy.
Python 3.12 · pytest · Pydantic · SQLite · deterministic MockConcept calibration\n\nPut the concepts used in this module back inside their engineering boundaries before you work with the code.\n\n\n\n### Harness harness\n\n- What it is: The runtime control environment that hosts models, tools, state, permissions, logs, budgets, human approval, isolation, and recovery.\n- What it is not: It can reduce risk, reject actions, and preserve evidence, but it cannot guarantee that model output is always correct or replace domain acceptance checks.\n- Relationship to adjacent concepts: The agent loop runs inside the harness; tool, memory, and MCP connections obey its policies; observability records execution and evaluation judges behavior.\n- Where it lives in HeatStack Forge: The Forge harness manages workspace, tools, approval queue, budget, checkpoints, audit logs, and rollback, separating model suggestions from real side effects.\n- Typical misuse and correction: A common misuse treats a system prompt as a security boundary. Enforce permissions, paths, network access, secrets, budgets, and rollback outside the model.\n\n### Skill skill\n\n- What it is: A reusable capability package containing instructions, resources, operating boundaries, versioning, and compatibility metadata that an agent platform can discover, install, or reference.\n- What it is not: It is not merely a prompt, and bundled scripts are not automatically trusted; execution remains constrained by platform, tool contracts, and permission policy.\n- Relationship to adjacent concepts: A prompt may be part of a Skill, a Skill may reference tools, the harness controls installation and runtime boundaries, and adapters handle platform differences.\n- Where it lives in HeatStack Forge: Forge packages requirement extraction as a Skill with a manifest, tests, resource inventory, and risk notes, then adapts it to three target platforms.\n- Typical misuse and correction: A common misuse copies one prompt file and claims cross-platform compatibility. Declare dependencies, entry points, resources, versions, permissions, and acceptance tests.\n\n### Tool tool\n\n- What it is: A constrained capability an agent invokes through explicit arguments, permissions, execution boundaries, and result contracts, such as a function, command, database query, or HTTP client.\n- What it is not: A tool is not MCP or any arbitrary script. A protocol can expose capability, while the harness still decides whether invocation is allowed and how side effects are handled.\n- Relationship to adjacent concepts: The agent loop selects tools, schemas constrain arguments, the harness enforces permission and isolation, and verification checks whether results advanced the goal.\n- Where it lives in HeatStack Forge: Forge stores tool contracts and risk levels, produces a change plan before execution, waits for approval on high-risk writes, and records results and recovery data.\n- Typical misuse and correction: A common misuse lets the model invent arguments from a tool name. Use strict schemas, deterministic validation, timeouts, idempotency keys, and refusal tests.\n\n
Describe required tools, resources, and approvals with a platform-neutral contract
Start with a concrete problem
How can we declare which capabilities a Skill needs in a platform-neutral way, distinguishing required from optional?
You are building a cross-platform adapter layer for the Forge platform, and you need to let the same Skill determine whether a platform satisfies its capability requirements during installation.
The codebase currently has no forge/adapters package and no capability model, so the tests test_required_capabilities_are_reported and test_optional_capability_is_not_a_blocker fail at import time.
You need to create a platform-neutral contract model: use CapabilityRequirement to represent a single capability need (with a name and whether it is required), use SkillContract to represent all capability needs of a Skill, and implement missing_capabilities to compute missing required capabilities.
The key decision is that when a capability is marked optional, its absence on a platform should not block installation.
Make a prediction
When implementing the required_names method, which approach correctly distinguishes required from optional capabilities?
- Return all capability names regardless of the
requiredflag. - Return only capability names where
required=True. - Return only capability names where
required=False. - Return tuples of capability name and
requiredflag.
Reasoning guide: The correct approach is to return only capability names where required=True, because missing_capabilities should report only missing required capabilities; missing optional capabilities should not be installation blockers.
What it is
What a platform-neutral capability contract is
A platform-neutral capability contract is a set of data structures that declare which external capabilities a Skill needs at runtime, without caring how those capabilities are implemented on a specific platform.
CapabilityRequirement encapsulates a capability name and a boolean flag required, which defaults to True, meaning the capability is required.
SkillContract aggregates all capability requirements of a Skill and provides a required_names method to extract the set of names of all required capabilities.
The missing_capabilities function takes a contract and a set of available capabilities, and returns all missing required capabilities.
What it is not
What a platform-neutral capability contract is not
It is not a platform mapping table; it does not translate capability names into platform-specific APIs or configuration.
It is not an installer; it does not execute any installation logic or check whether the platform actually has these capabilities.
It contains no platform-specific implementation details; it only describes the requirements themselves.
How it relates to neighboring concepts
Relationships among contract components
SkillContract contains multiple CapabilityRequirement objects, each with a name and a required flag.
The required_names method iterates over all requirements, filters out those with required=True, and returns their names as a set.
missing_capabilities calls required_names to get the set of required capabilities, then subtracts the available capabilities set to obtain the missing required capabilities.
Boundary decision for this stage: The contract model only describes requirements; it does not handle platform mapping or installation. Determining whether a platform meets the requirements is done by missing_capabilities, but the actual installation decision is made by the upper-level Harness based on the set of missing capabilities.
Map the canonical Skill contract to Codex paths, tool declarations, and approval policy
Start with a concrete problem
From canonical contract to Codex install plan
You need to implement the plan_codex function in forge/adapters/codex.py to convert a SkillContract into a CodexInstallPlan.
Currently, both tests test_codex_path_and_tools_are_mapped and test_codex_write_approval_is_preserved fail because the module does not exist yet.
The key decision is that the target path must follow Codex’s skills/<skill_id>/SKILL.md structure, the tool list must come from the contract requirements, and the approval mode must be fixed as ask-before-writes.
If the approval mode is incorrectly set to full-access, it bypasses confirmation before writes, leading to permission escalation.
Make a prediction
When implementing plan_codex, how should the approval mode be handled?
- Dynamically decide based on available capabilities
- Always set to
ask-before-writes - Set to
full-accessto simplify installation - Pass as a parameter from the caller
Reasoning guide: The correct option is “Always set to ask-before-writes”. The adapter must preserve security semantics and cannot escalate permissions due to platform differences.
What it is
Adapter is a translator of security semantics
The adapter maps capability requirements from the canonical contract into a platform-specific install plan while keeping permission boundaries unchanged.
For Codex, this means generating the correct directory path, tool declarations, and approval mode.
The approval mode ask-before-writes ensures that any write operation requires user confirmation, which is an implicit security requirement of the contract.
What it is not
Adapter is not a permission decision-maker
The adapter cannot arbitrarily change permission modes, even if installation succeeds, it must not escalate permissions.
It does not perform the installation or verify that the platform actually enforces the approval mode.
The adapter only generates a plan; actual execution is controlled by the Harness.
How it relates to neighboring concepts
Relationship between contract, adapter, and Harness
SkillContract defines capability requirements, the adapter converts them into a platform plan, and the Harness executes and enforces permissions.
The missing_capabilities function checks whether available capabilities satisfy the contract; if not, it should raise an exception.
The target path is constructed from home and skill_id, and the tool list comes from the contract’s required_names().
Boundary decision for this stage: The adapter must preserve security semantics and cannot escalate permissions; the approval mode is fixed as ask-before-writes, even if the platform supports a more permissive mode.
Generate a Claude Code command allowlist and reject required capabilities that cannot be represented
Start with a concrete problem
Generate a command allowlist for Claude Code and reject required capabilities that cannot be represented
The repository currently has only a Codex adapter and no Claude adapter, so the tests fail immediately when importing forge.adapters.claude.
You need to create forge/adapters/claude.py and implement the plan_claude function, which maps capability requirements from a skill contract to a list of tools allowed by Claude Code.
If a required capability cannot be represented in Claude Code, the adapter must raise a ValueError instead of defaulting to generic shell access.
The goal of this stage is to make two tests pass: test_known_capabilities_map_to_tools and test_unknown_required_capability_is_rejected.
Make a prediction
When the adapter encounters a required capability that cannot be mapped to a Claude Code tool, what is the safest action?
- Default to Bash, because Bash can execute any command
- Raise an exception and refuse to generate a plan
- Ignore the capability and continue generating the plan
- Log a warning and map to a read-only tool
Reasoning guide: The correct option is ‘Raise an exception and refuse to generate a plan’. Defaulting to Bash widens permissions and may allow the model to perform unintended actions; ignoring the capability prevents the skill from working correctly; mapping to a read-only tool is also inaccurate. The adapter must explicitly reject required capabilities it cannot represent to ensure safety and semantic correctness.
What it is
Explicit capability mapping table
An explicit capability mapping table is a dictionary from capability names to platform tool names, for example {"read_files": "Read", "write_files": "Edit", "shell": "Bash"}.
The adapter only allows capabilities listed in the mapping table; any capability not in the table is considered unrepresentable and must be rejected.
This design makes capability boundaries clear, avoids accidental permission widening, and facilitates auditing and testing.
What it is not
Not an alias for generic shell access
The explicit mapping table is not a mechanism that defaults unknown capabilities to Bash, which would lose capability constraints.
It is also not an open set that can be arbitrarily extended; adding a new mapping requires considering whether the platform truly supports that capability.
The mapping table only handles name conversion; it does not perform permission checks or execution isolation, which are handled by the Harness layer.
How it relates to neighboring concepts
Relationship with skill contract and platform plan
The skill contract (SkillContract) defines which capabilities a skill needs; the adapter reads these capabilities and converts them into a list of tools allowed by the platform.
The ClaudePlan returned by plan_claude contains the skill file path and the allowed tools tuple for later deployment.
If a capability in the contract is not in the mapping table, the adapter detects the missing capability using the missing_capabilities function and raises a ValueError.
Boundary decision for this stage: The adapter must reject required capabilities it cannot represent rather than widen permissions; this ensures the safety and semantic correctness of the platform plan.
Map capabilities to WorkBuddy workspace permissions while preserving read/write distinctions
Start with a concrete problem
Derive WorkBuddy workspace access level from capability requirements
The repository currently lacks forge/adapters/workbuddy.py, so the tests fail immediately when importing plan_workbuddy with ModuleNotFoundError: No module named 'forge.adapters.workbuddy'.
This stage requires creating that file and implementing the plan_workbuddy function to decide the WorkBuddy workspace access level based on the Skill’s capability requirements.
If the Skill requires the write_files capability, the workspace access level should be write; otherwise it must remain read, without defaulting to elevated permissions.
The tests test_write_skill_receives_write_access and test_read_only_skill_stays_read_only verify the write and read-only scenarios respectively.
Make a prediction
Before implementing plan_workbuddy, predict: what happens if the adapter always requests write access?
- All Skills work correctly because write access is more powerful.
- Read-only Skills unexpectedly receive write access, violating the principle of least privilege.
- All tests pass because tests do not check the access level.
- The adapter throws an exception because it cannot handle read-only Skills.
Reasoning guide: The correct answer is the second option. If the adapter always requests write access, read-only Skills unexpectedly receive write access, violating the principle of least privilege, and the test test_read_only_skill_stays_read_only fails.
What it is
What the WorkBuddy adapter is
The WorkBuddy adapter is a component that maps a Skill’s capability requirements to a WorkBuddy workspace access level.
It receives a SkillContract object, checks whether it includes the write_files capability, and then produces a WorkBuddyPlan containing the configuration path and access level.
The adapter must derive the access level from capability requirements rather than hardcoding write or read.
What it is not
What the WorkBuddy adapter is not
It is not a general permission management system, nor does it perform actual write operations.
It does not automatically elevate permissions or ignore the Skill’s capability requirements.
It does not handle the shell capability; if shell is detected, it raises a ValueError because the WorkBuddy profile does not expose shell.
How it relates to neighboring concepts
Relationships with other components
plan_workbuddy depends on the required_names() method of SkillContract to obtain the set of capability names.
WorkBuddyPlan is a frozen dataclass with two fields: config_path and workspace_access.
The test file tests/test_stage.py imports plan_workbuddy and verifies its behavior, so the adapter must match the test expectations.
Boundary decision for this stage: The adapter must decide the access level based on the presence of the write_files capability, without defaulting to elevated permissions; it must also reject the shell capability because WorkBuddy does not support shell.
Normalize three platform plans into comparable and auditable configuration records
Start with a concrete problem
How do we unify different platform install plans into comparable configuration records?
The file forge/adapters/config.py does not exist yet, so the normalize_plan function is missing, causing the tests to fail at import time.
We need to implement a normalization function that takes a platform identifier and an install plan dataclass, and returns a dictionary containing schemaVersion and configuration.
The key decision is that normalization must validate the platform identifier; otherwise unknown platforms can slip into release configuration, creating audit and deployment risks.
Make a prediction
When implementing normalize_plan, what should the function do if an unknown platform identifier like “mystery-agent” is passed?
- Serialize and return the configuration record directly, because normalization only handles format conversion
- Raise
ValueError, because normalization must validate the platform identifier - Return a configuration record with a default platform field
- Ignore the platform field and only serialize the plan dataclass
Reasoning guide: The correct option is to raise ValueError. Normalization is not just format conversion; it must ensure the platform identifier belongs to the supported adapter registry, otherwise unknown platforms enter release configuration and undermine auditability and deployment safety.
What it is
What normalized platform configuration is
Normalized platform configuration is the process of converting different platform install plans into a unified structure containing platform identifier, schema version, and configuration data.
It must validate that the platform identifier is in the supported adapter registry and check that the plan is a dataclass containing an install path.
The normalized configuration record is comparable, auditable, and can be safely consumed by downstream deployment processes.
What it is not
What normalized platform configuration is not
Normalization is not simple serialization; it cannot accept arbitrary platform names without validation.
Normalization does not perform installation or modify plan content; it only converts the plan to a standard format and ensures basic constraints.
Normalization is not the platform adapter itself; it does not handle platform-specific communication protocols or authentication logic.
How it relates to neighboring concepts
Relationships with other components
The normalization function normalize_plan is directly called by the test file tests/test_stage.py, which expects it to validate the platform and return the correct schemaVersion.
It relies on Python’s dataclasses module to serialize the plan dataclass and checks that the plan contains an install path field.
The normalized configuration record will be used by subsequent deployment processes, so the platform identifier must be valid.
Boundary decision for this stage: Normalization must validate the platform identifier and reject unknown platforms; it must also check that the plan is a dataclass and contains an install path, otherwise raise the appropriate exception.
Distinguish blocking gaps from acceptable degradation in an explainable compatibility report
Start with a concrete problem
A platform missing an optional capability is incorrectly marked incompatible
In the previous stage, you normalized raw platform configurations into a unified SkillContract, where each capability carries a required flag.
Now you need to implement the evaluate function to generate a CompatibilityReport based on the platform’s actually available capabilities, and the report must distinguish blocking gaps from acceptable degradations.
The file forge/adapters/compatibility.py does not exist yet, so running the tests will immediately raise ModuleNotFoundError because the test file tests/test_stage.py tries to import evaluate.
If you naively put every missing capability into blockers, then a platform missing the optional capability git will also be marked incompatible, which violates the contract semantics.
Make a prediction
Before implementing evaluate, predict: if a platform lacks a capability marked optional (e.g., git), what should CompatibilityReport.compatible be?
False, because any missing capability means the platform is incompleteTrue, because missing optional capabilities should not block compatibilityFalse, but put the missing item intodegradationsTrue, but put the missing item intoblockers
Reasoning guide: The correct answer is True: missing optional capabilities should only be recorded as degradations and do not affect overall compatibility. Only missing required capabilities cause compatible=False.
What it is
A compatibility report is a diff analysis between contract and platform capabilities
CompatibilityReport is an immutable dataclass containing a compatible boolean, a blockers tuple, and a degradations tuple.
The evaluate function iterates over each CapabilityRequirement in the SkillContract and checks whether its name is in the available set.
For each missing capability, it decides whether to put it into blockers or degradations based on its required flag.
The final compatible value depends only on whether blockers is empty, not on degradations.
What it is not
A compatibility report is not a simple list of missing capabilities
It does not lump all missing capabilities together but preserves the required/optional semantics of each capability.
It does not modify the platform configuration or automatically install missing capabilities; it only performs a static evaluation.
It does not guarantee that the platform will succeed at runtime; it only reflects static contract-level compatibility.
How it relates to neighboring concepts
Relationship with existing components
SkillContract and CapabilityRequirement come from forge/adapters/capabilities.py, which are not modified in this stage.
The evaluate function is called directly by the test file tests/test_stage.py, and the tests expect the returned CompatibilityReport fields to match exactly.
Later stages may use this report to decide whether to allow deployment or trigger a migration process.
Boundary decision for this stage: Only when a missing capability’s required attribute is True should it be added to blockers; otherwise it goes into degradations, and compatible depends solely on whether blockers is empty.
Use one contract suite to check install paths, non-escalation, and capability coverage before release
Start with a concrete problem
How do we automatically check adapter plans for security and capability requirements?
In the previous stage, we generated a compatibility report, but the report only describes differences and does not prevent unsafe adapter releases.
Now we need to implement a conformance check function check_plan that takes a platform name, a configuration dictionary, and a set of required capabilities, and returns a ConformanceResult.
If the configuration lacks an install path, does not cover the required capabilities, or has approval_mode set to full-access while the required capabilities do not include unattended_write, it must be marked as failed with specific errors.
The codebase currently has no forge/adapters/conformance.py file, so the tests fail at import time.
Make a prediction
Before implementing check_plan, predict: if we only check that the configuration is non-empty, will test_permission_escalation_fails_conformance pass?
- Yes, because the configuration is non-empty and contains capabilities.
- No, because the full-access approval mode grants permissions beyond the required capabilities.
- Yes, because the test only checks that passed is False.
- No, because the configuration lacks target_path.
Reasoning guide: The correct answer is ‘No, because the full-access approval mode grants permissions beyond the required capabilities.’ The test passes approval_mode as full-access but the required capability is only read_files, which is a permission escalation and must be rejected. Checking only non-emptiness cannot detect this semantic violation.
What it is
What conformance checking is
Conformance checking is semantic validation of the configuration plan generated by an adapter to ensure it meets security and capability contracts.
It checks three key dimensions: whether the install path exists and is non-empty, whether the required capabilities are declared by the configuration, and whether the approval mode causes permission escalation.
The check result is returned via the ConformanceResult dataclass, containing the platform name, whether it passed, and a tuple of errors.
What it is not
What conformance checking is not
It is not just checking whether the configuration dictionary is non-empty or fields exist, because correct shape does not imply semantic safety.
It does not perform the actual installation or modify system state; it only statically analyzes the plan.
It cannot replace domain acceptance or guarantee that model output is always correct; it only verifies that the adapter plan conforms to predefined rules.
How it relates to neighboring concepts
Relationship to other components
check_plan uses the configuration dictionary and required capabilities set as inputs, which come from the Skill metadata and the adapter-generated plan.
The errors tuple of ConformanceResult is asserted by tests, for example test_permission_escalation_fails_conformance checks that errors contains “permission escalation”.
This check resides in forge/adapters/conformance.py, and together with capabilities.py, compatibility.py, etc., forms the adapter layer.
Boundary decision for this stage: Conformance checks must validate semantics, including non-escalation and capability coverage, not just configuration shape. If we only check non-emptiness, we would approve an adapter that grants undeclared full-access, leading to security risks.
Complete the chapter
Deploy the same Forge Skill to three platforms through adapters.
Local lab self-check
- Not started
- 2Reading
- 3Lab downloaded
- 4Test result read
- 5Local check passed
verification.json is parsed only in this browser and is never uploaded. This checks the lab contract; it is not server certification or third-party endorsement.
Concept calibration and one-week review
Three questions check the chapter's boundaries, delivery evidence, and recovery path. Answers stay in this browser.