Lab Lifecycle
A lab you cannot reset is a lab you use once. Three operations carry the whole lifecycle — build a known-good artifact, return to it, destroy everything the lab created — and the third one is only finished when its verification output is on disk.
Write the Destroy Command First
The lab lifecycle: build, reset, prove the teardown
Scroll sideways to see the whole diagram
The Reset Contract
"Clean" is a property of an artifact, not a memory of how the VM looked on Tuesday. State it once, then make every reset return to that artifact instead of trying to undo changes by hand.
| Operation | Question It Answers | What You Keep |
|---|---|---|
| build | What exactly is the known-good state? | An exported baseline plus a build note and a hash |
| reset | How do I get back to it between exercises? | One command, or one clone, measured in seconds |
| destroy | How do I prove nothing is left? | The teardown output, not the teardown command |
What a Snapshot Captures
A hypervisor snapshot records the VM's virtual disks, its configuration, and — if it was taken while the machine was running — its memory state. It does not copy the disks. VirtualBox creates a differencing image containing only the writes made after the snapshot; restoring throws that image away and returns to the previous state. The image starts empty and grows with every write, which is why a week-old snapshot can be larger than the base disk file it was taken from.
Three things follow from that design:
- A restore is destructive: the current state you are reverting from is discarded, not merged.
- Anything written outside the virtual disks is untouched.
- A disk attached in a mode that bypasses snapshots is untouched too — VirtualBox documents that write-through disks are not included in snapshots and remain unaltered when a machine is reverted.
A Snapshot Is Not a Backup
What It Does Not Cover
- Files you exported or copied to the host — notes, PCAPs, screenshots, credential dumps, installers.
- Host-side network objects: host-only adapters, DHCP servers, virtual switches, port forwards, firewall rules.
- Cloud resources, IAM users and access keys created during the exercise.
- Docker volumes, images and networks created outside the Compose project.
- Device state: an unlocked bootloader stays unlocked, and a flashed boot image stays flashed.
Restore Semantics Differ by Product
- VirtualBox discards the current state and returns the VM to the snapshot. Deleting a snapshot merges its differencing data back into the parent, which can be slow and can temporarily need extra disk; the running VM's current state is unaffected.
- VMware Workstation reverts the VM to the named snapshot.
- libvirt/KVM documents the revert as destructive: changes since the snapshot are lost. A disk snapshot with no saved VM state leaves the domain inactive.
- Proxmox rolls back to the named snapshot; if the snapshot includes RAM, the VM can be started as part of the rollback.
Snapshot Naming
You will read these names under pressure, usually while deciding whether the current state is worth keeping. Use
a fixed shape: lab-state-YYYYMMDD,
ASCII only, no spaces. The state vocabulary should be small:
base— OS installed and patched, before tools.clean— tools installed and validated, nothing executed. This is the one you keep.wip— a working snapshot during an exercise. Disposable by definition.
Put the reason in the description, not in your head: VirtualBox takes --description,
libvirt and Proxmox take a description argument, and VMware snapshot names are all you get — so encode the date and
state there. One clean
snapshot per VM is the goal; every additional snapshot lengthens the differencing chain and slows disk I/O.
Per-Hypervisor Commands
The core operations are the same — take, list, revert, delete — but no two products spell them the same way.
VirtualBox
# Take a named snapshot before the exercise, not after it breaks
VBoxManage snapshot "win11-lab" take clean-20261017 --description "patched; tools installed; no samples run"
# List what exists, then revert to it
VBoxManage snapshot "win11-lab" list --details
VBoxManage snapshot "win11-lab" restore clean-20261017
# Revert to whichever snapshot is marked current, without naming it
VBoxManage snapshot "win11-lab" restorecurrent
# Delete merges the differencing data back into its parent: it can be slow
# and can temporarily need extra disk space
VBoxManage snapshot "win11-lab" delete clean-20261017VMware Workstation
# vmrun ships with Workstation; pass the .vmx path, quoted if it has spaces
VMX="/vms/win11-lab/win11-lab.vmx"
vmrun snapshot "$VMX" clean-20261017
vmrun listSnapshots "$VMX"
vmrun revertToSnapshot "$VMX" clean-20261017
vmrun deleteSnapshot "$VMX" clean-20261017
On Windows the same commands run from the Workstation install directory against a
.vmx
path; quote it when the path contains spaces.
KVM / libvirt
# name and description are positional; quote the description
virsh snapshot-create-as win11-lab clean-20261017 "patched; tools installed; no samples run"
virsh snapshot-list win11-lab --tree
# Reverting is destructive: changes since the snapshot are discarded
virsh snapshot-revert win11-lab clean-20261017
virsh snapshot-delete win11-lab clean-20261017
Internal snapshots live inside a single qcow2 file. A raw or block-backed disk needs an external snapshot instead
(--disk-only),
which writes a new overlay per disk. Quiescing with
--quiesce
requires a guest agent and only works with --disk-only;
without the agent the snapshot fails rather than silently skipping the freeze.
Proxmox
# --vmstate 1 includes RAM, which makes the snapshot larger and slower
qm snapshot 101 clean-20261017 --description "patched; tools installed" --vmstate 1
qm listsnapshot 101
qm rollback 101 clean-20261017 --start 1
qm delsnapshot 101 clean-20261017
Proxmox also has a stronger primitive for baselines:
qm template <vmid>
converts a VM into a template you clone from, so the baseline itself is never booted or modified.
Containers: Rebuild, Don't Commit
A container's writable layer is not a VM disk, and the data that matters usually is not in it. Mounted volumes
outlive the container, so stopping or removing the container does not reset the application. That is why the
reset is down -v
and a rebuild, not a restart.
# See what the exercise actually changed in the writable layer
docker diff dvwa-web
# commit captures that delta, not mounted volumes and not the build steps
docker commit dvwa-web lab/dvwa:after-exercise
# Reset instead: discard the writable layer and the project's volumes
docker compose down -v --remove-orphans
docker compose up -d --build
docker commit
is not a baseline. It creates an image from the container's filesystem changes; its own documentation states that
commits do not include data in mounted volumes, and that the container and its processes are paused while the
commit is taken. What you get is an opaque image with no record of how it was built and no way to reproduce it
after the fact — the opposite of a baseline.
Pin the image instead. Resolve the tag to a digest once, put the digest in the Compose file, and every
up
starts from the same bytes on any host:
image: vulnerables/web-dvwa@sha256:<digest>.
# Resolve a tag to the digest that identifies its bytes
docker inspect --format '{{index .RepoDigests 0}}' vulnerables/web-dvwaExternal Volumes Survive down -v
external, and plain
down leaves anonymous volumes in place. If the lab defines either, add an explicit
docker volume rm or docker network rm to the teardown — and then verify with
docker volume ls, because a successful down -v does not prove they are gone.
Baseline Images
A baseline is built once and cloned many times: install the OS, patch it fully, install and validate the tools,
shut it down, snapshot that state as clean,
then export it. The export is what you keep — snapshots are a convenience on the build host, not the artifact you
archive.
# VirtualBox: export the clean state, not whatever state the VM is in now
VBoxManage snapshot "win11-base" restore clean-20261017
VBoxManage export "win11-base" --output win11-base-20261017.ova --manifest
# libvirt: convert the shut-down disk to a compressed qcow2 baseline
qemu-img convert -O qcow2 -c win11-base.qcow2 win11-base-20261017.qcow2
qemu-img info win11-base-20261017.qcow2
# One overlay per lab; the baseline file is only ever read
qemu-img create -f qcow2 -b win11-base-20261017.qcow2 -F qcow2 win11-lab-a.qcow2
# VirtualBox clone from the clean snapshot
VBoxManage clonevm "win11-base" --snapshot clean-20261017 --name "win11-lab-a" --register
# Store the artifact hash beside the build note
sha256sum win11-base-20261017.ova
The VirtualBox export needs care: OVF cannot describe snapshots, so exporting a VM that has snapshots writes only
the current state, flattened. Restore the clean
snapshot before exporting, or your baseline silently captures whatever you were doing last. Write the
artifact hash and the tool versions into the build note; a baseline without a version list is a rumour.
A Baseline That Has Been Booted Is No Longer a Baseline
Clone per lab rather than reverting one shared VM. On libvirt that is a qcow2 overlay with the baseline as its
backing file; on VirtualBox it is
clonevm --snapshot;
on Proxmox it is a clone of the template. The baseline file is then read-only in practice, and deleting one lab
cannot damage the next.
What Rollback Does Not Undo
A rollback restores virtual disks. Everything the lab caused on the host or in a provider account is outside that boundary, and this is where most "clean" labs are actually dirty.
Host Files and Evidence
Exported PCAPs, screenshots, notes, dumped hashes and downloaded installers all survive a revert. That is usually what you want — evidence has to leave the VM before teardown — but it also means malware samples, credentials and client data are now sitting on your host. Move them deliberately, then handle them under the same rules as the lab.
Host Network Objects
Host-only adapters, DHCP servers, virtual switches, NAT port forwards and firewall rules are created on the
host, not in the guest. VBoxManage list hostonlyifs
and virsh net-list --all
show what is still there after the VM is gone.
Cloud Resources Outside State
Terraform destroys what is in its state file and nothing else. Console clicks, one-off CLI resources, IAM users and access keys created by hand all remain — and so does the state bucket itself, which normally lives outside the scenario's configuration.
Container Artefacts
Images you built by hand, external volumes, manually created networks and anything you pushed to a registry are not part of the Compose project's lifecycle. Neither is a container started outside Compose on the same Docker host.
Device State
Re-flashing restores the OS, but the bootloader unlock happened before that. Re-locking resets the device and restores the locked state, though on some vendors the unlock leaves a permanent record that re-locking does not clear — check your vendor's documentation rather than assuming a factory-fresh attestation.
The Snapshot Chain Itself
Reverting does not delete the snapshot you reverted from, and it does not shrink the differencing files. Left alone, the chain grows until the host disk fills — which fails the next exercise in a way that looks like a hypervisor bug.
The working rule: if it was created by a command on the host, rollback does not see it. Put that command in the teardown script, next to the snapshot revert.
Reset Cadence
Snapshot at state transitions, not on a timer. Each row below is a point where the previous state stops being a valid reference.
| Trigger | Action | What You Keep |
|---|---|---|
| First build | Patch, install tools, validate, shut down, snapshot clean, export |
Baseline artifact and hash |
| Between exercises | Revert to clean, delete the working snapshot, re-check isolation if the NIC changed |
clean unchanged |
| Before a new target or sample | Clone from the exported baseline instead of reusing a VM that has already run something | New clone; baseline still unbooted |
| After installing or updating tools | Build a new baseline with a new date, export it, delete clones made from the old one | Baseline version bump |
| After changing host networking | Re-snapshot only after proving the new interface; the old snapshot still carries the old NIC configuration | One new clean |
| End of session | Run the teardown script and save its verification output | Evidence folder |
Teardown by Lab Type
Teardown is a sequence, not a single command. Stop the workload, export anything you need, remove the lab's objects, then prove the objects are gone. The examples below assume the lab is the only thing on the host that matches the filter — during a shared session, use names and labels that cannot match someone else's work.
Hypervisor Labs
# VirtualBox: unregister and delete the files, then remove the host-only network
VBoxManage controlvm "win11-lab" poweroff
VBoxManage unregistervm "win11-lab" --delete
VBoxManage hostonlyif remove vboxnet1
# libvirt: stop, drop snapshot metadata, then remove the storage libvirt knows about
virsh destroy win11-lab
virsh undefine win11-lab --remove-all-storage --snapshots-metadata
# Proxmox: stop and destroy, including VM-specific permissions and firewall rules
qm stop 101
qm destroy 101 --purge
unregistervm
without --delete
leaves the disk images on the host. libvirt refuses to undefine an inactive domain that still has snapshot
metadata, so either delete the snapshots or pass
--snapshots-metadata;
--remove-all-storage
only covers volumes libvirt knows about.
Docker Labs
docker compose down -v --remove-orphans
docker network rm lab-net
docker volume rm lab-data # external or hand-created volumes; down -v removed the rest
docker image rm lab/web:local # only images built on this host
The two trailing comments are the traps. A volume declared in the Compose file is already removed by
down -v;
an external one is not, and removing it may be destructive outside the lab. Only delete images you built here —
deleting a pulled base image just costs you the next pull.
Cloud Labs
# Preview the destroy plan, then apply it
terraform plan -destroy
terraform destroy
# CloudGoat deletes only what its own state created
./cloudgoat destroy lab-scenario
CloudGoat's documentation is explicit that it can only manage resources it creates, and that a scenario which
asks you to create resources by hand requires you to remove them before running
destroy.
The same limit applies to Terraform: destroy
deprovisions the objects managed by the configuration, not the ones you clicked into existence.
Device Labs
# Pull evidence off the device first: re-locking resets it
adb pull /sdcard/lab-capture ./2026-10-17-wireless/
# Pixel factory image: flash every partition, then lock the bootloader
./flash-all.sh
fastboot flashing lock
fastboot flashing lock
both locks the bootloader and resets the device, so the evidence pull has to happen first. Devices that do not
implement the standard flashing lock use a vendor-specific path; treat the exact command as device- and
firmware-dependent and follow the vendor's unlock guide in reverse.
Proof of Teardown
"I ran the destroy command" is not evidence. Destroy paths fail partially: Terraform refuses to remove a bucket
that is not empty, CloudFormation can end in
DELETE_FAILED,
unregistervm
can leave disks, and every destroy only knows about the objects it tracks. A zero exit code is not a clean account.
The evidence is the listing command that returns nothing. Run it after the destroy, not instead of it.
| Lab Type | Verification Command | What Clean Looks Like |
|---|---|---|
| Hypervisor | VBoxManage list vms, virsh list --all, qm list |
Target absent; no leftover host-only interface |
| Docker | docker ps -a, docker volume ls, docker network ls |
No project containers, volumes or networks |
| Cloud | terraform state list, filtered list-stacks, IAM key listing |
Empty state; no live stacks; no lab keys |
| Device | Boot to stock image; read the bootloader's lock state | Stock build, bootloader locked, no lab accounts |
For cloud labs, the listing needs a status filter. CloudFormation keeps summary information for deleted stacks
for 90 days, and
aws cloudformation list-stacks
with no filter returns deleted stacks too — so an unfiltered query makes a finished teardown look like a failure,
and a careless filter can hide a live stack.
# Terraform's view of what it still owns: empty output means empty
terraform state list
# CloudFormation keeps deleted summaries for 90 days, so filter by live status
aws cloudformation list-stacks \
--stack-status-filter CREATE_COMPLETE UPDATE_COMPLETE UPDATE_ROLLBACK_COMPLETE \
--query "StackSummaries[].[StackName,StackStatus]" \
--output table
# Keys created by hand survive every destroy
aws iam list-access-keys --user-name lab-userWhere the Output Goes
Keep the verification output in the same evidence folder as the isolation proof and the benign test event, and keep that folder outside the VM or container being destroyed. A snapshot rollback deletes everything inside the guest, including the notes that said what you did.
2026-10-17-adlab/
01-isolation/network-list.txt
02-validation/benign-event.txt
03-notes/
04-teardown/vbox-list-vms.txt
04-teardown/hostonlyifs.txtThis is the hub's evidence convention applied to the end of the lab: one dated folder per lab, proof of isolation, proof it worked, and proof it is gone. The teardown output is the third proof.
Failure Modes Worth Expecting
- The snapshot was taken too late. If you snapshot after installing the tool you are about to test, the "clean" state already contains half the exercise.
- The host disk fills mid-exercise. Differencing images grow with writes, and a running VM's snapshot also stores RAM. Check free space before a long session, not after a failure.
- The restore breaks networking. A snapshot taken before a host-only adapter change carries the old NIC configuration. The guest may also rename the interface after a restore; that is a udev/NetworkManager symptom, not an isolation failure.
- The cloud stack will not delete. Non-empty buckets, retained resources and deletion protection all stop
terraform destroy; the stack then sits in a failed state and keeps billing. - The container "reset" kept the data. A bind mount to a host directory, an external volume, or a database container started outside Compose all survive
down -v. - The lab is gone but the credentials are not. Access keys, API tokens and certificates created for the lab do not disappear with the VM. Revoke them as part of teardown, not next month.
The lab is finished when the verification output is in the folder — not when the VM window closes.