8 hours ago8 hr Someone wants to try a self-hosted tool, and of course it ships as a Docker image. A whole VM for it feels like overkill, so most of the time we run Docker inside an LXC container on Proxmox instead. Containers start in seconds, share the host kernel and use far less RAM. It works well once the right features are turned on. Here's how we set it up, how we give a container a TUN device so Tailscale or WireGuard can run inside it, and when I give up and build a VM.Before you startA Proxmox VE host. We're on the 8.x and 9.x series.A Debian or Ubuntu container template downloaded to local storageRoot shell access to the Proxmox hostStay unprivilegedIn an unprivileged container, root inside the container maps to a high, unprivileged UID on the host (UID 100000 by default). If something breaks out, it lands on the host as a nobody user. In a privileged container, root inside is root on the host, so a container escape is a host compromise.Privileged mode does avoid some friction, like bind-mounted host directories with odd ownership, certain device access, or NFS mounts inside the container. We still treat it as a last resort. If an app only works in privileged mode, that usually tells me it belongs in a VM.When I build a VM insteadThe workload needs its own kernel, kernel modules or a different kernel version.You need live migration without downtime. Containers restart when they migrate.The service is internet-facing and untrusted, and you want the stronger isolation boundary.The stack leans on Docker features that fight with nesting, or it's Kubernetes.It's Windows, or anything else that isn't Linux.For a handful of internal services, Docker in LXC is fine, and we run plenty of them.Creating the containerTwo features matter. nesting=1 lets the container create its own nested namespaces and cgroups, which Docker needs. keyctl=1 allows the keyctl system call in unprivileged containers. Docker and runc use it, and without it you get failures that are confusing to track down.pveam update pveam available --section system | grep debian-12 pveam download local debian-12-standard_12.7-1_amd64.tar.zst pct create 210 local:vztmpl/debian-12-standard_12.7-1_amd64.tar.zst \ --hostname docker01 \ --unprivileged 1 \ --features nesting=1,keyctl=1 \ --cores 2 --memory 2048 --swap 512 \ --rootfs local-lvm:16 \ --net0 name=eth0,bridge=vmbr0,ip=10.10.20.21/24,gw=10.10.20.1 \ --onboot 1 pct start 210Use the template filename that pveam available actually lists, because the version suffix changes over time. For a container you already have, turn the features on under Options, Features in the GUI, or run pct set 210 --features nesting=1,keyctl=1 and restart it.Installing DockerGet in with pct enter 210 and install Docker from Docker's official repository using their documented Debian steps, or use the distribution's docker.io package if you prefer. Then run a test container:docker run --rm hello-world docker info | grep -i 'storage driver'You want the overlay2 storage driver. If you see vfs, every image layer is a full copy and disk use balloons. That used to be common on ZFS-backed container storage. Newer ZFS releases support overlayfs, so check what your host is running.Giving it /dev/net/tunTailscale and WireGuard userspace implementations need /dev/net/tun, and unprivileged containers don't get it by default. Stop the container and add two lines to its config file on the host:pct stop 210 nano /etc/pve/lxc/210.conflxc.cgroup2.devices.allow: c 10:200 rwm lxc.mount.entry: /dev/net/tun dev/net/tun none bind,create=fileThe first line allows access to the character device with major 10, minor 200, which is TUN. The second bind-mounts the host's device node into the container and creates the file if it doesn't exist. Start the container again and install Tailscale the normal way:pct start 210 pct enter 210 ls -l /dev/net/tun curl -fsSL https://tailscale.com/install.sh | sh tailscale upIf the container is going to be a subnet router or exit node, also enable IP forwarding inside it (net.ipv4.ip_forward=1 in a sysctl file) and approve the routes in the Tailscale admin console. That last bit is the part I always forget.Quick sanity checksdocker run --rm hello-world finishes without permission errors.docker info shows overlay2.Inside the container, ls -l /dev/net/tun shows a character device, and tailscale status shows the node online.On the host, pct config 210 shows unprivileged: 1 and the features line.Where this has bitten usBind-mounted host folders show up as nobody:nogroup. That's UID mapping doing its job. Either chown the host directory to the mapped UID (container UID plus 100000) or set up a custom idmap. Please don't switch to privileged just to fix permissions.Updates can break nesting. We've seen Docker, containerd or runc updates inside containers start failing with AppArmor or permission errors until the host side caught up. Take a pct snapshot before upgrading Docker in any container that matters.vzdump snapshot mode doesn't quiesce databases running in Docker. For anything with a database, run its native dump before the backup or use stop mode.Moving the container to another node restarts it, along with every Docker service inside.I try to keep it to one service per container. Once a single container is running twenty stacks, you've rebuilt a VM with weaker isolation, and that's usually my cue to move it to an actual VM.
Create an account or sign in to comment