Tools, Sandboxing & Side-Effect Control

Start with a concrete problem

How the harness can know which side effects a tool may produce before invoking it

In the Module 07 single-agent loop, tool calls execute directly, so the harness cannot determine before invocation whether a tool will write files or make network requests, allowing any tool to produce uncontrolled side effects.

This stage requires creating forge/tools/registry.py and introducing the ToolDescriptor and ToolRegistry symbols so that each tool declares its capability set at registration time, letting the harness enforce permission checks before any call.

The current test file tests/test_stage.py tries to import ToolDescriptor and ToolRegistry from forge.tools.registry, but because the forge/tools package does not yet exist, collection fails with ModuleNotFoundError: No module named 'forge.tools' and exit code 2.

Make a prediction

When ToolDescriptor("shell", frozenset({"root"})) is constructed, what does unknown = set(self.capabilities) - set(ALLOWED_CAPABILITIES) evaluate to in __post_init__, and what happens next?

• unknown is an empty set, and construction succeeds
• unknown is {‘root’}, and a ValueError is raised
• unknown is {‘read’, ‘write’}, and a ValueError is raised
• unknown is {‘root’}, and construction succeeds

Reasoning guide: ALLOWED_CAPABILITIES contains read, write, network, and process; it does not contain root, so the difference yields {‘root’}, a non-empty set that triggers ValueError. Select the second option.

What it is

Tool capability declarations are the basis for harness permission decisions

ToolDescriptor is a frozen dataclass with name and capabilities fields, where capabilities is typed frozenset[str] and represents the categories of side effects the tool may produce.

ALLOWED_CAPABILITIES is a module-level frozenset containing the four members read, write, network, and process, forming the harness policy vocabulary; every capability a tool declares must be a subset of it.

ToolRegistry maintains a dict[str, ToolDescriptor], provides a register method that stores tools and raises ValueError on duplicate names, and provides a supporting method that filters tools by a required capability set.

What it is not

Capability declarations are not arbitrary scripts or an MCP protocol

A tool is not an arbitrary script; it must declare its capabilities explicitly through ToolDescriptor, and any capability not in ALLOWED_CAPABILITIES is rejected at construction time rather than discovered at runtime.

A tool is also not equivalent to MCP; a protocol can expose capability, but the harness still decides whether invocation is allowed based on ToolDescriptor.capabilities, so the protocol itself is not the permission basis.

Capability declarations do not replace domain acceptance checks; they only tell the harness which category of side effect a tool may produce, while verification still checks whether the actual result is correct.

How it relates to neighboring concepts

The causal chain from vocabulary to descriptor to registry

ALLOWED_CAPABILITIES constrains the legal capability set for ToolDescriptor; ToolDescriptor.__post_init__ computes set(self.capabilities) - set(ALLOWED_CAPABILITIES) at construction time and raises ValueError if the result is non-empty, ensuring tools registered into ToolRegistry never carry undeclared capabilities.

ToolRegistry.supporting receives a required set and uses required <= set(tool.capabilities) to determine whether each registered tool’s capabilities are a superset of the requirement, returning only tools that satisfy every requested capability.

When the agent loop selects a tool, it first queries ToolRegistry.supporting(required), then the harness performs permission checks and isolation based on the returned descriptors, while observability records the entire process.

Boundary decision for this stage: Capability validation is enforced at construction time, deduplication is enforced at registration time, and filtering uses subset semantics at query time; the harness will not execute any tool carrying undeclared capabilities and will not extend ALLOWED_CAPABILITIES at runtime.

Start with a concrete problem

String concatenation cannot stop path traversal

In the software delivery scenario, the agent needs to modify files inside a frontend repository, and the tool system receives a relative path and concatenates it onto the workspace root. If the code only performs lexical concatenation without canonical resolution, an attacker can pass a path like ../secret.txt so the final path points to a file outside the workspace.

The current test suite in tests/test_stage.py defines three tests: test_relative_path_resolves_inside_workspace, test_parent_traversal_is_rejected, and test_symlink_escape_is_rejected. Because forge/tools/paths.py does not yet exist, the tests abort during collection with ModuleNotFoundError and exit code 2.

The learner must create the forge/tools/paths.py file and implement the resolve_workspace_path function so that it returns a canonical absolute path inside the workspace for valid paths and raises a ValueError containing escapes for traversal or symlink escape attempts.

Make a prediction

When workspace is /tmp/repo and requested is ../secret.txt, what happens if you execute resolved = workspace / Path(requested) and then call resolved.relative_to(workspace)?

• It raises ValueError because ../secret.txt does not start with the workspace prefix
• It does not raise an exception because the lexically concatenated path string still starts with the workspace
• It raises FileNotFoundError because the file does not exist
• It returns /tmp/secret.txt without any error

Reasoning guide: Lexical concatenation workspace / Path(“../secret.txt”) yields /tmp/repo/../secret.txt, and this string still starts with /tmp/repo, so relative_to(workspace) does not raise. Only after calling .resolve() to collapse the .. segments into a canonical path does the path become /tmp/secret.txt, at which point relative_to can detect the escape.

What it is

The meaning of canonical path resolution

Canonical path resolution means using the pathlib.Path.resolve() method to collapse the . and .. segments in a path into a real absolute path while also resolving any symlinks along the way. Only a resolved path represents the true location on the filesystem and can be reliably checked for whether it falls inside the workspace.

The core strategy of resolve_workspace_path is to reject absolute path inputs first, then join the workspace root with the candidate path, then call resolve(strict=False) on the joined result to obtain a canonical path, and finally use relative_to to check whether that canonical path is still inside the workspace.

What it is not

Lexical prefix matching is not a security check

Checking only whether the path string starts with the workspace prefix is not an effective security measure, because ../secret.txt still starts with the workspace prefix after concatenation but actually points to an external file. Lexical matching cannot understand the semantics of .. and cannot discover symlinks that point outside the workspace.

Using resolve(strict=True) is also inappropriate in this scenario because the target file may not exist yet, and strict=True would raise FileNotFoundError when the file is absent, whereas the purpose of the path safety check is to reject illegal paths before any file access occurs.

How it relates to neighboring concepts

The cooperation between resolve and relative_to

resolve converts the path into canonical form, eliminating the ambiguity of .. and symlinks, while relative_to performs a strict containment check on the canonical path. The two must be used together: first resolve then relative_to, and the order cannot be reversed nor can resolve be omitted.

The workspace root itself must also be canonicalized by calling resolve() first, otherwise when the workspace contains a symlink, even a valid candidate path might be falsely rejected by relative_to due to a prefix mismatch.

Boundary decision for this stage: The boundary of path safety checking is: resolve(strict=False) is responsible for collapsing path semantics without requiring the file to exist, and relative_to is responsible for containment checking on the canonical path; both are indispensable, and absolute paths must be rejected before joining.

Start with a concrete problem

A change plan must bind approval to exact bytes before any write occurs

The codebase currently lacks the forge/tools/plans.py file, so the test suite fails at collection time with a ModuleNotFoundError because it cannot import the build_change_plan symbol. The learner must create this file from scratch, defining two immutable dataclasses named PlannedWrite and ChangePlan, and implementing the build_change_plan function.

The core responsibility of this function is to separate the human-readable summary from the bytes to be written before any file write operation occurs, and to compute a SHA-256 hash over those bytes. If the hash is incorrectly computed over the summary string, the approval step will bind to a human-readable description rather than the actual bytes being written, enabling a content substitution attack after approval.

Make a prediction

In the faultySource, build_change_plan computes the hash using sha256(summary.encode(“utf8”)).hexdigest(). When the test asserts plan.writes[0].content_sha256 == sha256(b“A”).hexdigest(), what will happen?

• The assertion fails because hashing the summary string ‘write a’ produces a different result than hashing the byte b“A”
• The assertion passes because the summary and content are conceptually interchangeable
• A TypeError is raised because summary is a string while content is bytes
• The assertion fails because sorting causes the path order to be wrong

Reasoning guide: The correct answer is the first option. SHA-256 produces different outputs for different inputs, so the hash of the encoded string ‘write a’ will never equal the hash of the single byte b’A’. This reveals the root error of mistaking human-readable metadata for the side-effect payload.

What it is

A change plan is an immutable content-bound contract

ChangePlan is a frozen dataclass that captures the operation identifier and all target file paths, summaries, and content hashes before any write operation executes. Once constructed, its fields cannot be modified, which guarantees that the plan seen during the approval stage is the exact same immutable record used during the execution stage.

The content_sha256 field must be computed directly from the bytes to be written, not from the encoded summary string. This means an approver can verify the exact byte content that will be written to disk by checking the hash, establishing a cryptographic binding between approval and execution.

What it is not

Summary text cannot substitute for byte content as an approval basis

The summary field is a human-readable description such as ‘write a’ or ‘write b’, and its purpose is to help an operator understand the write intent, but its encoded result has no correlation with the actual file bytes. Two completely different file contents can share the same summary text, so hashing the summary cannot uniquely identify the write payload.

ChangePlan is also not a generic key-value store or configuration file. It specifically serves the side-effect control flow, and its writes field is an ordered tuple of PlannedWrite objects sorted by relative_path, ensuring that the same set of writes always produces an identical plan regardless of dictionary insertion order.

How it relates to neighboring concepts

The causal chain between plans, harness approval, and verification

The harness requires a ChangePlan to be generated before executing any file write, and the approval step binds to the content_sha256 within the plan. The execution step then recomputes the hash of the actual bytes written and compares it to the hash in the plan, allowing the harness to reject execution if they do not match, thereby preventing content substitution after approval.

Verification relies on the immutability of ChangePlan to check whether results advanced the goal. The test test_plan_hash_changes_when_content_changes verifies that when content changes from b“A” to b“B”, the content_sha256 must differ, ensuring hash sensitivity to content variations.

Boundary decision for this stage: content_sha256 must be computed from the bytes to be written, never from the encoded summary string; the plan is immutable after construction, and writes are sorted by relative_path to guarantee stability.

Start with a concrete problem

Stale Approval Authorizes a New Plan: Missing plan_digest Binding

In the software delivery scenario, an agent must generate a ChangePlan and wait for human approval before modifying a frontend repository. The file forge/tools/approval.py does not exist, so the tests fail immediately with a ModuleNotFoundError when importing Approval and ApprovalGate, causing collection of tests/test_stage.py to fail entirely.

Even after creating the file, if the authorize method only checks approval.operation_id == operation_id, an approval bound to an older plan digest old will still authorize a write operation for a new plan digest new. This means an attacker or a stale process can reuse the approval token of the same operation identifier to execute an unverified new write after the plan content has been replaced.

This stage requires you to create forge/tools/approval.py, define the Approval dataclass and ApprovalGate class, and ensure the authorize method decides whether execution is allowed based on the combination of risk level, operation identifier, and plan digest, thereby closing the vulnerability where a stale approval authorizes new content.

Make a prediction

When risk="reversible_write" and approval=Approval("op", "old"), while the current plan_digest="new", what will the authorize method return if it only checks operation_id?

• Returns False because plan_digest does not match
• Returns True because operation_id matches and plan_digest is ignored
• Raises ValueError because the risk level is unknown
• Returns None because the approval is incomplete

Reasoning guide: The correct answer is that it returns True. Because the faultySource contains return approval.operation_id == operation_id, it only compares the operation identifier and completely ignores the plan_digest field, so a stale approval incorrectly authorizes the new plan. This demonstrates why and approval.plan_digest == plan_digest must also be checked.

What it is

The Approval Binding Model of ApprovalGate

ApprovalGate.authorize is a pure decision function that accepts the risk level risk, the operation identifier operation_id, the current plan digest plan_digest, and an optional Approval object, returning a boolean or raising an exception. It performs no writes itself; it only determines whether the current call is allowed to proceed to the execution phase.

Approval is a frozen dataclass containing operation_id and plan_digest, representing the operation and plan content snapshot bound at the time of human approval. Once created, the approval object is immutable, ensuring the approval token cannot be tampered with to match a different plan.

Risk levels fall into three categories: read has no side effects and requires no approval; reversible_write and irreversible have side effects and require an Approval whose operation_id and plan_digest both match the current call parameters; an unknown risk level raises a ValueError.

What it is not

ApprovalGate Is Not an Executor or a Permission Cache

ApprovalGate is not responsible for executing write operations or modifying filesystem state; it only makes authorization decisions. Actual execution is handled by other components in the Harness, and the return value of ApprovalGate serves only as a gating signal before execution.

It is not a secondary opinion verification of model output, nor is it a general-purpose permission cache. The lifecycle of the approval object is managed by the caller, and ApprovalGate does not store historical approval records or accumulate state across multiple calls.

It does not replace domain acceptance testing. Even if authorize returns True, the execution result must still pass Verification to check whether it truly advanced the goal; the approval gate is just one link in the side-effect control chain.

How it relates to neighboring concepts

The Causal Chain Between ApprovalGate, ChangePlan, and the Harness

ApprovalGate depends on the plan_digest produced by ChangePlan from forge/tools/plans.py, which was passed in Stage 03. The plan digest is a cryptographic fingerprint of the plan content, so when the plan content changes, the digest necessarily changes; binding approval to the digest prevents stale approvals from remaining valid after content replacement.

The Harness calls authorize within the Agent Loop; if it returns False, execution is rejected and log evidence is retained, and if it returns True, the operation is passed to the isolated execution environment. This design separates the authorization decision from execution isolation, reducing the risk of a single point of failure.

The operation_id binding of Approval ensures the approval targets a specific operation rather than a generic token, while the plan_digest binding ensures it targets a specific content version. Both are indispensable: with only operation_id matching, an attacker can swap plan content and reuse the approval; with only plan_digest matching, the approval could be misapplied to a different operation.

Boundary decision for this stage: read risk returns True immediately without checking approval; reversible_write and irreversible risks require Approval with both operation_id and plan_digest matching the current call parameters; unknown risk raises ValueError and never defaults to allowing execution.

Start with a concrete problem

Writing directly to the target file leaves a partial write on crash

In the software delivery scenario, an agent needs to modify the contents of config.txt. If it calls target.write_bytes(content) directly on the target file and the process crashes or is interrupted mid-write, the target file retains a partial set of bytes, causing subsequent reads to return truncated data.

The starting point for this stage is that forge/tools/approval.py exists and passes, but forge/tools/atomic.py does not yet exist, so the test file tests/test_stage.py raises a ModuleNotFoundError when it tries to import forge.tools.atomic.

The learner must create forge/tools/atomic.py and implement the atomic_write function, using a same-directory temporary file and os.replace to atomically swap the target so that the target either updates completely or stays unchanged.

Make a prediction

When the process crashes halfway through target.write_bytes(b“new-content”), what will the contents of config.txt become?

• The complete new content b“new-content”
• The original content b“old”
• Truncated partial content such as b“new-c”
• An empty file

Reasoning guide: Writing directly to the target file is not an atomic operation; the operating system writes bytes progressively, so on crash the target file retains whatever bytes were already written, leaving truncated content rather than complete or original data.

What it is

The mechanism of atomic write

Atomic write requires all bytes to be persisted first to a temporary file in the same directory as the target, calling handle.flush() to push Python buffers to the operating system, then calling os.fsync(handle.fileno()) to force the operating system page cache onto disk.

After the temporary file is fully persisted, os.replace(temporary_name, target) atomically replaces the target file with the temporary file. os.replace is atomic on both POSIX and Windows, so the target file is always accessible and complete during the replacement.

What it is not

Boundaries of atomic write

Atomic write is not writing a temporary file first and then writing the target directly. If you write a temporary file and then call target.write_bytes(content), the target file still faces the risk of a partial write, and the existence of the temporary file does not protect the directly written target.

Atomic write also does not equal merely calling flush(). flush() only passes Python internal buffers to the operating system, but data may still reside in the operating system page cache and be lost on power failure, so os.fsync() is required to guarantee durability.

How it relates to neighboring concepts

Cooperation between temporary file, replacement, and cleanup

The temporary file must be in the same directory as the target because the atomicity of os.replace is only guaranteed within the same file system. A cross-file-system replacement degrades into a non-atomic copy-then-delete operation.

The temporary_name variable in the finally block acts as a cleanup state flag. After os.replace succeeds, setting it to None indicates the temporary file has been replaced and needs no cleanup; if an exception occurs before replacement, temporary_name still points to the temporary file path, and the finally block deletes it to avoid leftover files.

Boundary decision for this stage: The boundary of atomic write is that all bytes must first be persisted through a same-directory temporary file with os.fsync, then the target is atomically replaced via os.replace; on failure the finally block must clean up any leftover temporary file.

Start with a concrete problem

How can you reliably return files to their pre-change state after modifications?

In the software delivery scenario, you are adding a feature to a frontend repository and need to modify existing files and create new ones. If something goes wrong mid-change, you must be able to restore the filesystem to its pre-change state, otherwise subsequent steps will operate on incorrect content.

You already have the atomic_write function from forge/tools/atomic.py, which guarantees atomicity for a single write, but it does not record the pre-change state. You need a new mechanism that captures file existence and original bytes before mutation, so you can roll back when needed.

The core challenge of this stage is: when the same file is modified multiple times, the rollback journal must keep the first snapshot rather than overwriting it with later modified content. If it overwrites, rollback restores an intermediate state instead of the transaction-start state.

Make a prediction

In the RollbackJournal.capture method, if capture is called twice for the same path, what should the second call do?

• Overwrite the first snapshot with current content, because the latest state is more accurate.
• Ignore the second call and keep the first captured snapshot.
• Append a new snapshot and restore all snapshots in reverse order during rollback.
• Raise an exception because duplicate capture is a programming error.

Reasoning guide: The correct answer is to ignore the second call. The rollback journal’s purpose is to record the state at transaction start, not after each modification. If you overwrite the first snapshot, rollback restores an intermediate state instead of the original.

What it is

What a rollback journal is

A rollback journal is a list of pre-change states for filesystem mutations, where each entry records the path, whether it existed at capture time, and the original byte content.

The capture method is called before modifying a file and records the current state; the rollback method restores those states in reverse order, thereby undoing all changes.

What it is not

What a rollback journal is not

A rollback journal is not a full history of every modification; it only cares about the state at transaction start, so it does not need to record intermediate versions.

It is also not a backup system; it does not save modified content, only the pre-change state, for the purpose of undoing operations.

How it relates to neighboring concepts

Relationship with atomic writes

atomic_write guarantees atomicity for a single file write, while the rollback journal manages overall restoration across multiple files; the two are complementary.

The rollback journal calls atomic_write when restoring original content, ensuring that the restoration process itself is atomic.

Boundary decision for this stage: The rollback journal only handles filesystem state, not in-memory objects or database transactions; it assumes all changes are made through filesystem operations and that capture is called before any mutation.

Start with a concrete problem

How does the controller ensure only approved bytes are written and rollback on path escape?

You have already implemented path resolution, plan building, approval gate, atomic writes, and rollback journal in previous stages, but there is no unified entry point to chain these capabilities together. Now you need to create forge/tools/controller.py providing a SideEffectController class whose apply method, in a single call, builds the plan, computes the digest, validates through the approval gate, writes each file atomically, and rolls back any written files on any exception.

The current test file tests/test_stage.py imports from forge.tools.controller import SideEffectController, plan_digest, but this module does not exist, so running python -m pytest -q tests/test_stage.py fails with ModuleNotFoundError: No module named 'forge.tools.controller' during collection. Your task is to create this module and make all three tests pass.

The three tests verify respectively: after normal approval, the specified bytes are written and the audit status is committed; a stale approval (digest mismatch) is denied with no writes and the audit status is denied; a path escape (e.g., ../escape.txt) triggers rollback, the already-written safe file is restored, and the audit status is rolled_back.

Make a prediction

When implementing SideEffectController.apply, what do you think is the key point of approval validation?

• As long as the approval parameter is not None, allow the write
• Must compare the digest computed from the current plan with the digest in the approval, validated through ApprovalGate
• Only need to check if approval.operation_id matches
• No validation needed, just write directly

Reasoning guide: The correct answer is the second option. The test test_stale_approval_is_denied_without_writes specifically verifies that a stale approval (digest mismatch) must be denied, so merely checking that approval is not None is insufficient; you must call ApprovalGate().authorize with the digest computed from the current plan.

What it is

Composition of a safe change transaction

SideEffectController.apply is an orchestration function that chains five independent capabilities in a fixed order: first use build_change_plan to generate the plan, then compute the digest with plan_digest, then validate the approval through ApprovalGate.authorize, then resolve each target path with resolve_workspace_path, and finally write with atomic_write while recording with RollbackJournal for rollback.

The digest is computed by concatenating the content_sha256 of each write item in the plan in order, then hashing the entire string with SHA-256. This way any byte change alters the digest, invalidating stale approvals.

What it is not

It is not a simple existence check

apply does not allow writes just because an approval object exists; it must verify that the approval’s digest exactly matches the current plan. If you only check approval is None, an approval issued for an old plan can pass, which is exactly the defect injected in the failure fixture.

It also does not handle writes and rollback separately; instead, it wraps the entire write loop in try/except, so any exception (including path escape) triggers rollback, ensuring no partial write state remains.

How it relates to neighboring concepts

How components collaborate

build_change_plan generates the plan, plan_digest extracts the digest from the plan, ApprovalGate.authorize validates the approval using the digest and operation_id, resolve_workspace_path ensures the path stays within the workspace, atomic_write performs the write, and RollbackJournal records written files for rollback.

The audit list audit records the result status of each operation, and tests verify behavior by checking controller.audit[-1]["status"].

Boundary decision for this stage: When the approval digest does not match the current plan digest, you must deny the write and record denied, rather than attempting partial writes or ignoring the mismatch; when a path escape occurs, you must roll back all written files and record rolled_back, rather than merely skipping the escaping path.