How do I isolate self-hosted CI/CD runners and build an ephemeral, clean-after-each-run environment?

Short answer: a persistent self-hosted runner that runs arbitrary code on a box with root and the real Docker socket is a server takeover waiting to happen. Build on two principles: ephemerality and isolation.

Short answer

The root of the risk: you’re running code you don’t trust on a fully-privileged, persistent machine. I explain from the ground up what container isolation does and doesn’t give you in the Docker post; this architecture is built on that foundation.

Why

1. On a persistent runner, everything carries across jobs. A file, a cache, or a poisoned workspace left by one job lands in front of the next one; unless the environment is single-use you can’t stop that contamination.

2. Handing over /var/run/docker.sock is equivalent to handing over root. A job that can see the socket can start any container on the host with any mount it likes; --privileged or sensitive mounts open the same door.

3. A PR from a fork means arbitrary code on your infrastructure. This is the most overlooked and most dangerous vector; the runner you set up to protect your source code becomes the attacker’s entry point if you leave it unguarded.

What to do

1. Ephemerality: each job runs in a fresh, throwaway environment. Switch the runner to a “register → run one job → deregister” model; each job runs in a single-use VM or container that’s destroyed when it finishes.

2. Isolation: don’t give the runner host root or the real Docker socket. Run jobs in unprivileged containers (or rootless Docker / a VM per job); drop capabilities, never use --privileged, and mount nothing sensitive.

3. Never run untrusted fork PRs on a self-hosted runner. Put these behind required approval; no outside contribution should run code on your machine without a human approving it first.

4. Network-segment runners away from prod. Even if a job escapes, it shouldn’t reach prod resources. On tooling: ephemeral runners via Actions Runner Controller on Kubernetes, or autoscaling VMs recreated for each job.

Bottom line: I’d set up ephemeral one-job runners + unprivileged isolation + a fork-PR ban + network segmentation, and I’d treat every job as hostile. The power of a self-hosted runner is control, but the cost is responsibility — I explain how container isolation works and why handing over the socket is handing over root, from the ground up, in the hub Docker post.