[{"content":"MeshCentral puts device inventory, terminal access, file transfer, and remote desktop in one web console. Cloudflare Tunnel can carry its HTTPS and WebSocket traffic without forwarding an inbound port on your router.\nThere are two separate milestones: the login page loads, and an enrolled agent can actually complete a remote session. This guide checks both. The example hostname is mesh.example.com; use a dedicated hostname in a Cloudflare-managed zone you control.\nVerification status Area Evidence and limits Intended environment A rootful Docker Engine on a Linux server with the Compose plugin; a remotely managed Cloudflare Tunnel. Preparation Upstream Docker entrypoint/configuration schema and Cloudflare documentation reviewed on October 31, 2026. Ten Bash blocks pass syntax checks; JSON parses, selected schema names/types/ranges check out, and embedded Compose YAML parses. These are static checks, not full schema or runtime validation. Runtime No Cloudflare account, public tunnel, first-admin registration, or real device enrollment was exercised during preparation. Required validation Verify origin binding, administrator MFA, certificate discovery, agent reconnection, remote sessions, and an isolated recovery on the selected versions. 1. Understand the connection path Connection What carries it Administrator browser → Cloudflare HTTPS/WebSockets on public port 443, then MeshCentral login and MFA. MeshAgent → Cloudflare Agent connections and relay sessions over the published hostname. cloudflared → Cloudflare An outbound encrypted tunnel from the Linux host. cloudflared → MeshCentral Plain HTTP to 127.0.0.1:4430, confined to the same host. Both containers use Linux host networking. This is deliberate: the connector can reach the same loopback listener that MeshCentral binds, and MeshCentral can trust the connector\u0026rsquo;s loopback address. This is not a portable Docker Desktop or rootless-Docker example. See Docker host networking.\nThis walkthrough uses MeshCentral\u0026rsquo;s own authentication and MFA. It assumes a new hostname without an interactive Cloudflare Access application in front of it. Tunnel transport alone does not add an identity policy.\nA normal MeshAgent cannot complete a browser\u0026rsquo;s Access login page, CAPTCHA, or JavaScript challenge. An Access service token is not useful unless the particular client can send its required credentials. If your design requires Access/WARP, plan and test that client path separately; do not disable an existing access policy just to make an agent appear online. Avoid broad path-bypass recipes based only on a successful web login.\n2. Prepare the host and choose images Use a dedicated Linux VM or server with Docker Engine and Compose already installed. You also need Bash, Python 3, curl, SSH administration, a Cloudflare account/zone, and permission to enroll a test endpoint.\nRun the server-side commands in one root Bash shell. Check that Docker targets this server\u0026rsquo;s intended local daemon:\n1 2 3 docker context show docker version docker compose version Select released tags or digests from the upstream MeshCentral image instructions and Cloudflare\u0026rsquo;s official cloudflare/cloudflared image. The token-file option below needs cloudflared 2025.4.0 or later. Record the exact image digests you deploy; this article does not guess a current release tag.\nUse a new directory. Do not run the configuration-writing commands over an existing MeshCentral installation:\n1 2 3 4 5 6 7 8 9 10 11 12 install -d -m 0700 /opt/meshcentral-tunnel cd /opt/meshcentral-tunnel || exit 1 umask 077 mkdir meshcentral-data meshcentral-files meshcentral-web meshcentral-backups read -r -p \u0026#39;Public hostname, for example mesh.example.com: \u0026#39; mesh_hostname [[ \u0026#34;$mesh_hostname\u0026#34; =~ ^[a-zA-Z0-9][a-zA-Z0-9.-]*\\.[a-zA-Z]{2,}$ ]] || exit 1 read -r -p \u0026#39;Reviewed ghcr.io/ylianst/meshcentral tag or digest: \u0026#39; mesh_image read -r -p \u0026#39;Reviewed cloudflare/cloudflared tag or digest: \u0026#39; cloudflared_image printf \u0026#39;MESHCENTRAL_IMAGE=%s\\nCLOUDFLARED_IMAGE=%s\\n\u0026#39; \\ \u0026#34;$mesh_image\u0026#34; \u0026#34;$cloudflared_image\u0026#34; \u0026gt; .env chmod 600 .env Keep the project directory, Compose project name, and persistent folders together. The official MeshCentral image uses /opt/meshcentral/meshcentral-data and the companion folders mapped below; paths from other image projects are not interchangeable.\n3. Create the tunnel, but leave its connector stopped In Cloudflare\u0026rsquo;s tunnel dashboard, create a remotely managed cloudflared tunnel. Add a published application route/public hostname with:\nField Value Hostname Your chosen mesh.example.com equivalent Service type HTTP Origin URL http://127.0.0.1:4430 The origin is HTTP because TLS terminates at Cloudflare and the local connection stays on loopback. You do not need to disable origin-certificate verification for an HTTP origin. An HTTPS public hostname and an HTTP local origin are different parts of the path.\nObtain the tunnel token from the connector setup instructions. Do not paste the full install command into the token prompt. Store only its token value:\n1 2 3 4 5 6 7 read -r -s -p \u0026#39;Tunnel token (hidden): \u0026#39; mesh_tunnel_token printf \u0026#39;\\n\u0026#39; test -n \u0026#34;$mesh_tunnel_token\u0026#34; || exit 1 printf \u0026#39;%s\\n\u0026#39; \u0026#34;$mesh_tunnel_token\u0026#34; \u0026gt; cloudflare-token unset mesh_tunnel_token chown 0:65532 cloudflare-token chmod 640 cloudflare-token The connector below runs as UID/GID 65532; the file is readable by that group and mounted as a Compose secret. The token is a credential, not an application password. Keep it out of Git, screenshots, and support logs. See Cloudflare\u0026rsquo;s token-file option.\nDo not start the connector until the first administrator and MFA are ready.\n4. Write the MeshCentral configuration 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 cat \u0026gt; meshcentral-data/config.json \u0026lt;\u0026lt;EOF { \u0026#34;settings\u0026#34;: { \u0026#34;cert\u0026#34;: \u0026#34;$mesh_hostname\u0026#34;, \u0026#34;WANonly\u0026#34;: true, \u0026#34;port\u0026#34;: 4430, \u0026#34;portBind\u0026#34;: \u0026#34;127.0.0.1\u0026#34;, \u0026#34;aliasPort\u0026#34;: 443, \u0026#34;redirPort\u0026#34;: 0, \u0026#34;mpsPort\u0026#34;: 0, \u0026#34;exactPorts\u0026#34;: true, \u0026#34;tlsOffload\u0026#34;: \u0026#34;127.0.0.1\u0026#34;, \u0026#34;trustedProxy\u0026#34;: \u0026#34;127.0.0.1\u0026#34;, \u0026#34;webRTC\u0026#34;: false, \u0026#34;agentPong\u0026#34;: 60, \u0026#34;selfUpdate\u0026#34;: false }, \u0026#34;domains\u0026#34;: { \u0026#34;\u0026#34;: { \u0026#34;title\u0026#34;: \u0026#34;Homelab Remote Support\u0026#34;, \u0026#34;certUrl\u0026#34;: \u0026#34;https://$mesh_hostname:443/\u0026#34;, \u0026#34;newAccounts\u0026#34;: true, \u0026#34;passwordRequirements\u0026#34;: { \u0026#34;min\u0026#34;: 14, \u0026#34;force2factor\u0026#34;: true } } } } EOF python3 -m json.tool meshcentral-data/config.json \u0026gt; /dev/null The settings have specific jobs:\nport is the local HTTP listener in TLS-offload mode; aliasPort is the public port agents and browsers use. tlsOffload/trustedProxy identify the local connector, not every Cloudflare edge IP. Do not replace them with a blanket trust setting. certUrl tells MeshCentral where to obtain the TLS certificate agents see at Cloudflare. It is not the plain-HTTP origin URL. webRTC: false keeps this baseline on server-relayed connections. It does not promise a latency improvement. mpsPort: 0 disables Intel AMT CIRA/MPS for this guide. An HTTP tunnel does not automatically provide a raw AMT TCP service. newAccounts: true is temporary, while the listener is private and the tunnel is stopped. Use the upstream configuration schema for the release you selected. Underscored example keys such as _certUrl are disabled examples, not active settings.\n5. Write Compose and start only MeshCentral 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 cat \u0026gt; compose.yaml \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; services: meshcentral: image: ${MESHCENTRAL_IMAGE:?Select a reviewed MeshCentral image} restart: unless-stopped network_mode: host environment: DYNAMIC_CONFIG: \u0026#34;false\u0026#34; volumes: - ./meshcentral-data:/opt/meshcentral/meshcentral-data - ./meshcentral-files:/opt/meshcentral/meshcentral-files - ./meshcentral-web:/opt/meshcentral/meshcentral-web - ./meshcentral-backups:/opt/meshcentral/meshcentral-backups logging: driver: json-file options: max-size: \u0026#34;10m\u0026#34; max-file: \u0026#34;3\u0026#34; cloudflared: image: ${CLOUDFLARED_IMAGE:?Select a reviewed cloudflared image} profiles: [\u0026#34;tunnel\u0026#34;] restart: unless-stopped network_mode: host user: \u0026#34;65532:65532\u0026#34; command: [\u0026#34;tunnel\u0026#34;, \u0026#34;--no-autoupdate\u0026#34;, \u0026#34;run\u0026#34;, \u0026#34;--token-file\u0026#34;, \u0026#34;/run/secrets/cloudflare-token\u0026#34;] secrets: - cloudflare-token logging: driver: json-file options: max-size: \u0026#34;10m\u0026#34; max-file: \u0026#34;3\u0026#34; secrets: cloudflare-token: file: ./cloudflare-token EOF docker compose --profile tunnel config --quiet docker compose --profile tunnel pull docker compose up -d meshcentral docker compose logs --tail 80 meshcentral ss -lntp | sed -n \u0026#39;/:4430 /p\u0026#39; curl --fail --silent --show-error -o /dev/null -w \u0026#39;%{http_code}\\n\u0026#39; \\ http://127.0.0.1:4430/ Require the listener to bind to 127.0.0.1, not 0.0.0.0 or [::]. An HTTP response only checks the web listener. It does not establish successful authentication or agent enrollment.\nDYNAMIC_CONFIG=false preserves the JSON you wrote instead of rebuilding it from image environment variables. The container should use that file unchanged. At this stage the public certUrl may be unreachable because the connector is still stopped; inspect the logs and re-check certificate discovery after publishing.\n6. Create the first administrator privately From your workstation, forward a local browser port over SSH. Replace the example SSH destination with your server:\n1 ssh -N -L 8088:127.0.0.1:4430 gnolasco@YOUR_LINUX_HOST Open http://127.0.0.1:8088 on that workstation. The local HTTP hop travels inside the SSH connection. Create the first account; MeshCentral makes the first user an administrator. Complete its MFA setup and store recovery codes securely. Confirm you can log out and sign back in before continuing.\nClose self-registration while preserving the rest of the configuration:\n1 2 3 4 5 6 7 8 9 10 python3 - \u0026lt;\u0026lt;\u0026#39;PY\u0026#39; import json from pathlib import Path p = Path(\u0026#39;meshcentral-data/config.json\u0026#39;) config = json.loads(p.read_text()) config[\u0026#39;domains\u0026#39;][\u0026#39;\u0026#39;][\u0026#39;newAccounts\u0026#39;] = False p.write_text(json.dumps(config, indent=2) + \u0026#39;\\n\u0026#39;) PY chmod 600 meshcentral-data/config.json docker compose restart meshcentral Check that public signup is closed after restart. Keep at least one tested administrator recovery method. Avoid adding plugins or SSO changes until the basic deployment works.\n7. Start the tunnel and check the public certificate 1 2 docker compose --profile tunnel up -d cloudflared docker compose logs --tail 80 cloudflared Wait for the tunnel dashboard to report a healthy connector and for the hostname\u0026rsquo;s edge certificate to be valid. Then restart MeshCentral so it can load the public certificate through certUrl:\n1 2 3 4 docker compose restart meshcentral docker compose logs --tail 100 meshcentral curl --fail --silent --show-error -o /dev/null -w \u0026#39;%{http_code}\\n\u0026#39; \\ \u0026#34;https://$mesh_hostname/\u0026#34; Use normal TLS verification; do not add curl -k. Check the MeshCentral log for successful certificate discovery rather than assuming a working login page proves the agent certificate path is correct. DNS inside the container must resolve this name to Cloudflare, not a hosts-file override pointing at the private origin.\nMeshCentral\u0026rsquo;s maintainers recommend configuring certUrl for Cloudflare offload. Do not work around a hash mismatch with ignoreAgentHashCheck or a similar trust bypass. A changed edge certificate is a reason to inspect certificate discovery and reconnect behavior, not to remove identity checks.\nConfirm WebSockets are enabled for the zone. Review caching rules so this management hostname is not subject to a cache-everything rule. WAF or bot challenges can block the initial WebSocket handshake; diagnose the exact request and rule before making a narrowly scoped change. Do not globally disable protections.\n8. Enroll one authorized test device Sign in at the public hostname, create an agent-managed device group, and use its Add Agent action to obtain the installer for the test machine\u0026rsquo;s OS and architecture. Inspect the command/installer and hostname before running it with the required privileges on that device.\nDo not copy an installation command from another MeshCentral server or publish a real device-group enrollment command in the blog. Use group permissions and any required user-consent settings appropriate to your support workflow.\nVerify these separately:\nTest Passing result Agent registration The intended device appears under the expected group and reports the correct identity. Terminal A harmless command such as hostname completes on the intended host. File transfer A small disposable file uploads/downloads and its content matches. Remote desktop The authorized session opens; consent, display, and input behave as expected for that OS. Reconnection The agent returns after a controlled connector/server restart, and a new remote session works. Test from outside the server\u0026rsquo;s LAN as well. A browser reaching the login page is not a substitute for any of these checks. Cloudflare may close WebSocket sessions during edge maintenance; agentPong supplies periodic server traffic but does not guarantee uninterrupted sessions. See Cloudflare\u0026rsquo;s WebSocket behavior.\nTroubleshooting by symptom Symptom First checks Tunnel healthy, public response is 502 Check the origin URL, port, local HTTP response, and that both services use host networking. localhost inside a normal isolated container would be a different listener. Login works, agent stays offline Inspect the agent\u0026rsquo;s configured hostname, public DNS, certificate-discovery logs, Access redirects, and challenge responses. Device online, terminal/files/desktop fail Check relay WebSocket handshakes, current certificate hashes, OS session permissions, and endpoint logs. Failures begin after certificate rotation Re-check the exact public certificate seen by server and agent; refresh supported configuration/reconnect behavior without bypassing hash verification. Token file permission denied Confirm the source file is group-readable by 65532 and the Compose secret is mounted at the command\u0026rsquo;s path. Unexpected local port exactPorts=true should fail rather than silently selecting a different listener. Resolve the conflict before publishing. For a quick view of service state and recent errors:\n1 2 docker compose --profile tunnel ps docker compose --profile tunnel logs --tail 100 Redact tokens, enrollment links, user details, and device identifiers before sharing logs.\nBack up the server identity before upgrading Preserve the complete meshcentral-data directory, including database, configuration, certificates, and private keys. Also retain meshcentral-files, any custom meshcentral-web content, required recordings/backups, Compose/image records, and separately protected tunnel/recovery credentials. A fresh installation with new keys is not the same server identity.\nFor this local-database baseline, take an application-consistent copy during a maintenance window: stop the connector and MeshCentral, verify both stopped, back up the required folders, then restart and test reconnection. Do not archive a changing database and call it a verified recovery point. If you select an external database later, include its supported backup/restore procedure.\nUse the Restic backup and isolated recovery guide for encrypted copies of prepared data. Test restoration on separate storage with production agents and public routing isolated; do not bring two copies of the same management identity online against the same devices.\nBefore an image update, save the current digests and recovery materials, review release notes, and test the selected replacements. Schema or database changes can make an image-only rollback insufficient. Keep automatic in-container self-update disabled when image versions are managed by Compose.\nThe deployment is ready to rely on only after the administrator, agent, relay, reconnection, and recovery checks pass on your actual versions. Record the image digests and observations rather than treating this proposed configuration as an executed lab result.\nContinue reading Cloudflare Tunnel with Docker — the connector and routing model. Intel AMT Serial over LAN — a separate out-of-band text-console recovery path. Backup strategy — plan independent copies and recovery credentials. ","permalink":"https://blog.gntech.me/posts/2026-10-31-meshcentral-cloudflare-tunnel/","summary":"A practical Linux deployment with a loopback-only origin, private administrator bootstrap, persistent data, and separate checks for the browser, agents, and remote sessions.","title":"MeshCentral with Cloudflare Tunnel — Deploy, Enroll, and Verify"},{"content":"A remote console is most useful when the usual way into a server stops working. On a compatible business PC, Intel AMT can provide Serial over LAN, usually called SOL: a text console carried over the management connection.\nWhile configuring my Proxmox mini PCs, I reached a Linux login prompt through SOL. Then I tried to disconnect by typing ~.. The login prompt treated it as a username and asked for a password.\nThat small mistake reveals an important distinction: the Linux shell, the login program, and the SOL client each interpret different commands. This guide covers setting up the Linux login console and leaving it cleanly.\nVerification status Scope Evidence and limits Environment Proxmox VE 9.2.21 on AMT-capable mini PCs; the correct serial device must be discovered per host. Previously reported lab observations An AMT SOL controller was identified on PVH01; a Linux login prompt was reached through SOL on PVH02. Preparation checks 10 Bash examples passed syntax checks during preparation. This does not validate the optional getty override on hardware. Still to verify Confirm firmware/client compatibility, the tty assignment, login/logout and reconnection. Test boot redirection separately if required. This table summarizes the evidence recorded for this article. It is not a new hardware test or a certification for other versions.\nWhat this setup provides Open the full-size diagram to zoom or inspect its labels.\nAMT authentication opens the management connection; Linux authentication opens a host session. Serial input and output travel in both directions. Log out of Linux before using the client\u0026rsquo;s Disconnect control. The example ttyS0 must be confirmed on your hardware.\nThere are three separate pieces:\nPiece What it does Intel AMT firmware and management connection Makes the SOL session available through a compatible client Linux serial device and serial getty Presents the Proxmox host\u0026rsquo;s text login prompt Optional firmware, bootloader, and kernel redirection Sends earlier boot menus or boot messages to the console Enabling a serial getty handles the second row. It does not automatically enable BIOS redirection, a GRUB menu, kernel boot messages, graphical KVM, or a VM\u0026rsquo;s console.\nThe management connection is independent of the host\u0026rsquo;s ordinary network services, but it still depends on supported hardware, firmware configuration, power state, and a reachable management network. SOL is not a promise of access during every possible failure.\nMy starting point On PVH01, a Proxmox VE 9.2.21 host, PCI discovery showed an Intel Cannon Lake PCH AMT SOL serial controller at 00:16.3, with PCI ID 8086:a363 and the serial driver. The nearby HECI controller at 00:16.0 used mei_me; that is a different device.\nOn PVH02, enabling a serial getty on ttyS0 produced the Proxmox welcome message and a Linux login prompt through the SOL session.\nThose are observations from my nodes. The device name, PCI address, firmware menu, and client controls on another machine can differ. The generalized commands below need verification against the target host; they were not executed on physical hardware while preparing this article.\nPrerequisites A machine and firmware that support Intel AMT SOL, with AMT provisioned and SOL permitted. A compatible, authenticated SOL client with access to the management network. An existing SSH or local console session to the Proxmox host, kept open while testing. A root shell for the Linux configuration commands below. AMT support is a platform feature. An Intel CPU alone does not establish that the machine supports it. Follow the machine vendor\u0026rsquo;s firmware and provisioning instructions before changing Linux.\nKeep AMT on a trusted management network or behind an appropriate private access path. Use the supported encrypted connection mode and certificate validation where available. Do not expose management access directly to the public Internet merely to make the client connect.\n1. Identify the SOL controller Run these read-only checks on the Proxmox host:\n1 2 3 4 hostname pveversion lspci -nnk | sed -n \u0026#39;/Communication controller/,+4p; /Serial controller/,+4p\u0026#39; journalctl -k -b --no-pager | sed -n \u0026#39;/ttyS/p; /serial/p\u0026#39; Look for both the PCI serial controller and the kernel\u0026rsquo;s assignment of a ttyS device. A /dev/ttyS0 node existing by itself is not proof that it belongs to AMT; Linux can expose serial device nodes without a usable physical port behind each one.\nInspect device paths as another clue:\n1 2 3 4 5 for serial_path in /sys/class/tty/ttyS*/device; do [ -e \u0026#34;$serial_path\u0026#34; ] || continue printf \u0026#39;%s -\u0026gt; \u0026#39; \u0026#34;$serial_path\u0026#34; readlink -f \u0026#34;$serial_path\u0026#34; done Match the PCI path to the AMT serial controller where the kernel exposes that relationship. Do not confuse a PCI function such as 00:16.3 with the Linux serial name ttyS0.\n2. Inspect the existing console configuration The rest of this example assumes discovery confirms ttyS0. Change this one variable if your AMT device is different, and keep using the same root shell:\n1 2 3 4 5 sol_tty=ttyS0 test -c \u0026#34;/dev/$sol_tty\u0026#34; || { echo \u0026#39;Serial device is missing\u0026#39;; exit 1; } systemctl cat \u0026#34;serial-getty@$sol_tty.service\u0026#34; systemctl is-enabled \u0026#34;serial-getty@$sol_tty.service\u0026#34; systemctl is-active \u0026#34;serial-getty@$sol_tty.service\u0026#34; An inactive or disabled result can be expected before setup. Save the original enabled/active state and any existing overrides in your change notes. The rollback depends on whether this was a new console or an existing one.\nCheck that another service is not already using the same serial device. Do not run two login programs on one port.\n3. Enable the Linux login prompt Start with the distribution\u0026rsquo;s existing serial-getty template:\n1 2 3 systemctl enable --now \u0026#34;serial-getty@$sol_tty.service\u0026#34; systemctl status \u0026#34;serial-getty@$sol_tty.service\u0026#34; --no-pager journalctl -b -u \u0026#34;serial-getty@$sol_tty.service\u0026#34; --no-pager -n 30 The service should stay active and run agetty. Open the SOL session and press Enter once or twice. If a login prompt appears and accepts input, the basic console works; no custom service file is required.\nIf the client exposes speed and framing settings, make them consistent with the serial setup. A common starting point is 115200 baud, 8 data bits, no parity, and one stop bit. Firmware and clients differ, so verify their actual settings rather than assuming that one default applies everywhere.\n4. Set a fixed speed only if needed If the stock template\u0026rsquo;s baud handling causes a confirmed problem, a service drop-in can set 115200 explicitly. Check that /sbin/agetty exists first. Restarting this getty ends any Linux session currently using it, so perform this step from your separate SSH or local console.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 test -x /sbin/agetty || { echo \u0026#39;Check the agetty path first\u0026#39;; exit 1; } override_dir=\u0026#34;/etc/systemd/system/serial-getty@$sol_tty.service.d\u0026#34; install -d -m 0755 \u0026#34;$override_dir\u0026#34; if [ -e \u0026#34;$override_dir/60-amt-sol.conf\u0026#34; ]; then cp -a \u0026#34;$override_dir/60-amt-sol.conf\u0026#34; \\ \u0026#34;$override_dir/60-amt-sol.conf.bak-$(date +%Y%m%d-%H%M%S)\u0026#34; fi cat \u0026gt; \u0026#34;$override_dir/60-amt-sol.conf\u0026#34; \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; [Service] ExecStart= ExecStart=-/sbin/agetty --noclear --local-line 115200 %I vt100 EOF systemctl daemon-reload systemctl restart \u0026#34;serial-getty@$sol_tty.service\u0026#34; systemctl status \u0026#34;serial-getty@$sol_tty.service\u0026#34; --no-pager The empty ExecStart= clears the template\u0026rsquo;s command before replacing it. %I expands to the instance\u0026rsquo;s device name. --local-line avoids depending on carrier-detect signaling; vt100 sets the login terminal type.\nThis does not enable automatic login or change account permissions. Review the effective unit with systemctl cat again, especially if the host already had local overrides.\n5. Log in with a Linux account The AMT connection authenticates you to the management service. The text login prompt authenticates you separately to Linux.\nUse a local Linux account permitted to log into the host console. Proxmox web login names such as user@pve are not automatically Linux usernames; use the account\u0026rsquo;s actual Linux login name where one exists.\nPasswords normally produce no visible characters. Type the password and press Enter. Avoid repeated guesses if authentication fails; inspect the account and authentication logs through your existing administrative session.\nAfter logging in, confirm the destination:\n1 2 3 hostname tty whoami For this example, tty should identify /dev/ttyS0. Check the hostname before running administrative commands, particularly when several nodes have similar welcome screens.\n6. Log out, then disconnect At the Linux shell prompt, end the login session with:\n1 exit If you opened nested shells, repeat as needed until you return to the Linux login prompt. Ctrl+D at an empty shell prompt usually also ends that shell, subject to shell configuration.\nThen close the SOL connection using the client\u0026rsquo;s Disconnect, Close session, or equivalent control. That control belongs to the management client, not the remote Linux shell.\nClosing the client alone is not a reliable substitute for logging out: a serial login session can remain open and be visible on reconnection. Explicitly return to the login prompt first.\nWhere you are What to do Logged into the Linux shell Run exit to end that shell At the Linux login prompt Use the SOL client\u0026rsquo;s disconnect control In a nested SSH session inside SOL Exit the inner session, then the host shell, then disconnect SOL Looking for an escape shortcut Consult that specific client\u0026rsquo;s help or settings Why ~. became a username Escape sequences are client-specific. For example, some SSH clients interpret ~. at the beginning of a line, but that does not make it a universal AMT SOL command.\nIn my test, the screen displayed login: ~. and then asked for a password. The characters reached the Linux login program as ordinary input. That was not a Proxmox failure or a request for an AMT password.\nIf that happens, cancel or let the login attempt finish, return to the login prompt, and use the client\u0026rsquo;s own disconnect control. Do not keep entering escape sequences as usernames.\nLogin console versus boot console Once the host reaches userspace, the getty can offer a login even if no boot messages appeared earlier.\nCheck the current kernel command line without changing it:\n1 cat /proc/cmdline Linux serial boot output commonly uses a parameter such as console=ttyS0,115200n8, adjusted for the confirmed serial device. Bootloader input/output is configured separately. The ordering of multiple console= parameters can also affect the primary console and automatic getty behavior.\nOn Proxmox, first determine the boot mechanism and follow its bootloader documentation. A GRUB installation and a systemd-boot/proxmox-boot-tool setup do not share one universal edit-and-update command. Keep a known-good recovery path before changing either. This guide deliberately leaves the working boot configuration alone.\nTroubleshooting Symptom Check next Cannot establish any SOL connection AMT provisioning, SOL permission, client compatibility, management reachability, and power-state support SOL connects but stays blank Press Enter, verify the tty assignment, and inspect the getty service Readable output but no input Client input capture, SOL configuration, and another process using the device Unreadable characters Speed/framing settings and terminal emulation Linux login works, boot messages absent Kernel and bootloader redirection are separate from the getty Reconnecting reveals an authenticated shell Log out explicitly before disconnecting; review session handling ~. appears after login: The client passed it through; use the client\u0026rsquo;s disconnect control Do not reboot a working production node as the first diagnostic step. The service and kernel logs often tell you which layer is missing.\nRollback If you created the optional speed override from scratch, remove only that file and restart the getty from your separate administrative session:\n1 2 3 rm -i \u0026#34;/etc/systemd/system/serial-getty@$sol_tty.service.d/60-amt-sol.conf\u0026#34; systemctl daemon-reload systemctl restart \u0026#34;serial-getty@$sol_tty.service\u0026#34; If the file existed before, restore its saved version instead of deleting it. Do not remove unrelated service overrides.\nIf the getty was both disabled and inactive before this procedure and you want to undo its activation:\n1 systemctl disable --now \u0026#34;serial-getty@$sol_tty.service\u0026#34; For other original states, restore the enabled and active states separately from your notes. Disabling a unit does not guarantee a boot-generated getty can never return if kernel-console configuration or another dependency requests it. This procedure did not change AMT firmware or bootloader settings.\nFinal check before relying on SOL Verify that you can connect, identify the correct host, log in, run a harmless command, log out, disconnect, and reconnect to a fresh login prompt. Test boot-time visibility separately during a maintenance window if you need that capability.\nThe useful result is a recovery path you have exercised. On my nodes, the final lesson was simple: exit belongs to the Linux shell, and disconnect belongs to the SOL client.\nContinue reading Proxmox bridges and VLANs — Understand the normal host and guest network paths alongside the management console. ThinkPad T420 administration laptop — Prepare a focused Linux client for day-to-day administration. Proxmox Backup Server — Pair recovery access with a backup and restore plan. References and image provenance Intel AMT documentation systemd: systemd-getty-generator Linux kernel: serial console util-linux: agetty Proxmox VE: host bootloaders Live reference retrieval was unavailable during preparation. Check the installed agetty and systemd manuals and your vendor\u0026rsquo;s current AMT instructions before applying the generalized examples.\nThe cover is an AI-generated editorial illustration, not a photograph, actual console capture, or wiring diagram. Its generation prompt is stored alongside the image asset.\n","permalink":"https://blog.gntech.me/posts/2026-10-30-intel-amt-sol-proxmox/","summary":"A practical Intel AMT SOL login setup for a Proxmox host, with device discovery, a systemd serial getty, troubleshooting, rollback, and a clear explanation of disconnecting.","title":"Intel AMT Serial over LAN on Proxmox: Setup, Login, and Disconnect"},{"content":"Welcome to GnTech. Pick the problem you want to solve, then follow a short path through the guides. You do not need to read the blog in publication order.\nYour goal Start with Build a virtualized lab Proxmox foundations Separate users, servers, and devices MikroTik networking Run and maintain your own services Docker Compose Back up services and prove recovery Backup and recovery Administer and recover your machines Administration and recovery Proxmox foundations Begin with networking so you understand where the host and guests connect. Then make guest deployment repeatable and plan backups.\nBridges, VLANs, and host topology — understand the network before changing it. Cloud-init VM templates — prepare reusable guest configurations. Proxmox Backup Server — learn the backup service and datastore workflow. Practice goal: draw the host-to-guest network path, deploy a disposable guest, and test recovery before relying on a backup.\nMikroTik networking These guides assume you have access to the router and understand the interfaces you are changing. Keep a recovery connection available when working on management access or VLAN filtering.\nBuilding R1 — see the pieces of a segmented router configuration. VLAN segmentation — work through routing, DHCP, and isolation. Proxmox networking — connect the router-side design to the virtualization host. Practice goal: verify both permitted traffic and a connection that should be blocked.\nDocker Compose Start with a small service stack. Understand its storage, configuration, and startup behavior before adding more applications.\nCompose patterns — organize services and their configuration. Health checks and startup dependencies — distinguish a running process from a ready service. Frigate deployment — explore a concrete camera-system example after the basics. Practice goal: recreate a disposable stack and confirm that its intended data survives. The Frigate guide describes an earlier deployment; check its version and hardware assumptions.\nBackup and recovery Choose the recovery target first, then follow the relevant workflow:\nBackup strategy and independent copies — define recovery targets and the failures each copy must survive. For guests, use PBS setup and isolated guest recovery. For PostgreSQL, use the dump, scheduling, and complete Restic-to-database recovery lab. Use the Restic file and repository-copy lab to understand encrypted file recovery and separate repository passwords. Practice goal: recover into an isolated destination using the copy and credentials that will be available after a failure. Local CI repositories demonstrate the commands; they do not prove your real off-site storage or hardware recovery.\nAdministration and recovery A comfortable client and a tested recovery console solve different parts of the administration problem.\nThinkPad T420 with Arch Linux and i3 — build a focused administration desktop.\nIntel AMT Serial over LAN on Proxmox — set up text-console login and learn how to disconnect cleanly.\nMeshCentral with Cloudflare Tunnel — deploy a remote-support console, enroll a test agent, and check relayed sessions.\nPractice goal: connect to the intended host, verify its identity, log out, and reconnect to a fresh login prompt.\nHow to use a guide Check the prerequisites, software versions, and validation notes before copying commands. Separate measured lab results from example configurations. Keep backups and recovery access appropriate to the change, and verify the result instead of assuming a successful command means the whole system works.\nFor a specific symptom, use search. For more context, browse projects, read about the blog, or explore all posts. Follow new posts through RSS.\n","permalink":"https://blog.gntech.me/start-here/","summary":"\u003cp\u003eWelcome to \u003cstrong\u003eGnTech\u003c/strong\u003e. Pick the problem you want to solve, then follow a short path through the guides. You do not need to read the blog in publication order.\u003c/p\u003e\n\u003ctable\u003e\n  \u003cthead\u003e\n      \u003ctr\u003e\n          \u003cth\u003eYour goal\u003c/th\u003e\n          \u003cth\u003eStart with\u003c/th\u003e\n      \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eBuild a virtualized lab\u003c/td\u003e\n          \u003ctd\u003e\u003ca href=\"/start-here/#proxmox-foundations\"\u003eProxmox foundations\u003c/a\u003e\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eSeparate users, servers, and devices\u003c/td\u003e\n          \u003ctd\u003e\u003ca href=\"/start-here/#mikrotik-networking\"\u003eMikroTik networking\u003c/a\u003e\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eRun and maintain your own services\u003c/td\u003e\n          \u003ctd\u003e\u003ca href=\"/start-here/#docker-compose\"\u003eDocker Compose\u003c/a\u003e\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eBack up services and prove recovery\u003c/td\u003e\n          \u003ctd\u003e\u003ca href=\"/start-here/#backup-and-recovery\"\u003eBackup and recovery\u003c/a\u003e\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eAdminister and recover your machines\u003c/td\u003e\n          \u003ctd\u003e\u003ca href=\"/start-here/#administration-and-recovery\"\u003eAdministration and recovery\u003c/a\u003e\u003c/td\u003e\n      \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003ch2 id=\"proxmox-foundations\"\u003eProxmox foundations\u003c/h2\u003e\n\u003cp\u003eBegin with networking so you understand where the host and guests connect. Then make guest deployment repeatable and plan backups.\u003c/p\u003e","title":"Start Here"},{"content":"My ThinkPad T420 cost $50. After adding an SSD and bringing it to 8 GB of RAM, I wanted a comfortable machine for terminals, SSH, documentation, and homelab administration.\nI chose Arch Linux with i3. The attraction was a simple desktop that I could understand and adjust: a window manager, a panel, a launcher, and a few services doing specific jobs.\nThis post combines the results from my machine with a clean starting configuration readers can reproduce. The commands below are a proposed setup, not an export of my exact dotfiles or a claim that every block has been tested on every T420.\nVerification status Scope Evidence and limits Environment ThinkPad T420; Core i5-2520M, SSD, 8 GB RAM; Arch Linux with Xorg/i3. Previously reported lab observations Working brightness control and an NVIDIA runtime-suspend observation; fingerprint verification and some mute behavior remained unresolved. Preparation checks 13 Bash examples passed syntax checks during preparation. The starter configuration is not an export of the installed dotfiles. Still to verify Run the i3 configuration parser and test audio, input, locking, suspend/resume and battery behavior on the target laptop. This table summarizes the evidence recorded for this article. It is not a new hardware test or a certification for other versions.\nThe hardware and the result Component My machine Processor Intel Core i5-2520M Memory 8 GB after upgrading Storage SATA SSD after upgrading Display 1600 × 900 panel Graphics Intel integrated graphics plus an NVIDIA Optimus GPU Desktop Arch Linux, Xorg, i3, Polybar, Picom, Rofi, and Dunst Audio PipeWire with WirePlumber Brightness control worked in my setup. The NVIDIA device was observed in runtime suspend during a check. Audio mute behavior still needed refinement, and the fingerprint reader enrolled a finger but repeatedly failed verification, so I left fingerprint login out.\nThe i7 processor and IPS panel I considered are planned upgrades, not installed hardware or prerequisites for this guide. Other T420 configurations can differ, particularly graphics, wireless adapters, fingerprint readers, and display panels.\nStart with an existing Arch installation This guide begins after Arch is installed and you can log in locally. Keep a working network connection and a second console available. If you already have a desktop, keep it available until the new session works.\nCollect a baseline before changing anything:\n1 2 3 4 5 6 uname -r lscpu free -h lsblk -o NAME,SIZE,TYPE,FSTYPE,MOUNTPOINTS lspci -nnk lsusb Save the output privately if it contains information you do not want to publish. It gives you a useful reference when troubleshooting a driver or comparing behavior after an update.\nArch changes continuously. Check package availability and current upstream instructions before installing. The examples below were written for this article; they have not been executed on a T420 as part of its preparation.\nInstall a small desktop stack Review the full-system upgrade before accepting it:\n1 2 3 4 5 6 7 sudo pacman -Syu --needed \\ xorg-server xorg-xinit i3-wm i3status \\ xterm rofi dunst picom \\ pipewire pipewire-audio pipewire-alsa pipewire-pulse wireplumber \\ brightnessctl xss-lock i3lock \\ networkmanager network-manager-applet \\ ttf-dejavu Use a full upgrade rather than refreshing package databases and selectively updating a handful of packages. If Pacman proposes replacing an existing audio stack, review that transition before accepting it.\nThe starting configuration uses i3status. My actual desktop uses Polybar, but getting the session functional first makes troubleshooting easier. You can replace the panel later without changing the window manager.\nIf NetworkManager already manages this laptop, leave that arrangement in place. If you deliberately choose it for a new installation, enable it from the local console:\n1 2 sudo systemctl enable --now NetworkManager nmcli device status Do not start NetworkManager on an interface already managed by a different network service without planning that switch. Changing network managers over SSH can disconnect you.\nBack up the configuration before replacing it The next sections create a complete minimal i3 configuration and an X startup file. They replace those two files. Back up existing versions first:\n1 2 3 4 5 6 7 8 9 backup_dir=\u0026#34;$HOME/t420-desktop-backup-$(date +%Y%m%d-%H%M%S)\u0026#34; mkdir -p \u0026#34;$backup_dir\u0026#34; if [ -d \u0026#34;$HOME/.config/i3\u0026#34; ]; then cp -a \u0026#34;$HOME/.config/i3\u0026#34; \u0026#34;$backup_dir/\u0026#34; fi if [ -f \u0026#34;$HOME/.xinitrc\u0026#34; ]; then cp -a \u0026#34;$HOME/.xinitrc\u0026#34; \u0026#34;$backup_dir/\u0026#34; fi printf \u0026#39;Backup: %s\\n\u0026#39; \u0026#34;$backup_dir\u0026#34; Keep that printed location. Custom launchers, workspace bindings, and display commands will not be present in the new starter file unless you add them back.\nCreate the i3 configuration The Windows key becomes the modifier. Enter opens a terminal, d opens Rofi, and Shift+q closes the focused window.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 mkdir -p \u0026#34;$HOME/.config/i3\u0026#34; cat \u0026gt; \u0026#34;$HOME/.config/i3/config\u0026#34; \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; set $mod Mod4 font pango:DejaVu Sans Mono 10 floating_modifier $mod bindsym $mod+Return exec xterm bindsym $mod+d exec rofi -show drun bindsym $mod+Shift+q kill bindsym $mod+Shift+c reload bindsym $mod+Shift+r restart bindsym $mod+Shift+e exec i3-msg exit bindsym $mod+h focus left bindsym $mod+j focus down bindsym $mod+k focus up bindsym $mod+l focus right bindsym $mod+Shift+h move left bindsym $mod+Shift+j move down bindsym $mod+Shift+k move up bindsym $mod+Shift+l move right bindsym $mod+f fullscreen toggle bindsym $mod+space focus mode_toggle bindsym $mod+Shift+space floating toggle bindsym $mod+v split v bindsym $mod+b split h bindsym $mod+1 workspace number 1 bindsym $mod+2 workspace number 2 bindsym $mod+3 workspace number 3 bindsym $mod+Shift+1 move container to workspace number 1 bindsym $mod+Shift+2 move container to workspace number 2 bindsym $mod+Shift+3 move container to workspace number 3 bindsym XF86AudioRaiseVolume exec wpctl set-volume -l 1.0 @DEFAULT_AUDIO_SINK@ 5%+ bindsym XF86AudioLowerVolume exec wpctl set-volume @DEFAULT_AUDIO_SINK@ 5%- bindsym XF86AudioMute exec wpctl set-mute @DEFAULT_AUDIO_SINK@ toggle bindsym XF86AudioMicMute exec wpctl set-mute @DEFAULT_AUDIO_SOURCE@ toggle bindsym XF86MonBrightnessUp exec brightnessctl set +5% bindsym XF86MonBrightnessDown exec brightnessctl set 5%- bindsym $mod+Shift+x exec i3lock -c 151515 exec --no-startup-id nm-applet exec --no-startup-id dunst exec --no-startup-id picom exec --no-startup-id xss-lock --transfer-sleep-lock -- i3lock --nofork -c 151515 exec --no-startup-id xset s 300 exec --no-startup-id xset dpms 0 0 600 bar { status_command i3status } EOF i3 -C -c \u0026#34;$HOME/.config/i3/config\u0026#34; Resolve configuration errors before opening the new desktop. A successful parse checks i3 syntax; it does not prove that audio devices, brightness permissions, or keyboard events work.\nThe Shift+e binding exits immediately, so save your work first. The lock shortcut is Shift+x with the modifier.\nStart the session from a local console If you already use a display manager, choose its i3 session. The following .xinitrc is for starting X manually:\n1 2 3 4 5 6 7 8 9 10 cat \u0026gt; \u0026#34;$HOME/.xinitrc\u0026#34; \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; #!/bin/sh if [ -d /etc/X11/xinit/xinitrc.d ]; then for startup_file in /etc/X11/xinit/xinitrc.d/?*.sh; do [ -r \u0026#34;$startup_file\u0026#34; ] \u0026amp;\u0026amp; . \u0026#34;$startup_file\u0026#34; done unset startup_file fi exec i3 EOF From the laptop\u0026rsquo;s local text console, run:\n1 startx Do not launch another X server from inside an existing graphical session. Keeping the system xinit hooks helps preserve the environment expected by desktop services.\nCheck audio and brightness separately Run these inside the graphical session:\n1 2 3 wpctl status wpctl get-volume @DEFAULT_AUDIO_SINK@ brightnessctl info Try the volume and brightness keys, then repeat the checks. If changing volume works but a physical mute button or its LED behaves differently, investigate that button\u0026rsquo;s event and the selected audio device separately. Do not assume every indicator is controlled by the same software mixer.\nUse xev from the xorg-xev package to inspect key events if a binding does nothing. Avoid launching it while entering passwords. If brightness changes fail due to permissions, check the current Arch guidance for brightnessctl and session access before adding broad privileges.\nEnable tap-to-click with libinput This is optional. If Xorg uses libinput for your touchpad, a small input-class file enables tapping without changing TrackPoint settings:\n1 2 3 4 5 6 7 8 9 10 11 12 13 sudo install -d /etc/X11/xorg.conf.d if sudo test -e /etc/X11/xorg.conf.d/40-t420-touchpad.conf; then sudo cp -a /etc/X11/xorg.conf.d/40-t420-touchpad.conf \\ \u0026#34;/etc/X11/xorg.conf.d/40-t420-touchpad.conf.bak-$(date +%Y%m%d-%H%M%S)\u0026#34; fi sudo tee /etc/X11/xorg.conf.d/40-t420-touchpad.conf \u0026gt; /dev/null \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; Section \u0026#34;InputClass\u0026#34; Identifier \u0026#34;T420 touchpad tapping\u0026#34; MatchIsTouchpad \u0026#34;on\u0026#34; MatchDriver \u0026#34;libinput\u0026#34; Option \u0026#34;Tapping\u0026#34; \u0026#34;on\u0026#34; EndSection EOF Log out and back in to restart the X session. Check that the touchpad uses libinput first; a different driver will not use this rule. To roll back a newly created rule, remove only this file and restart X. If you replaced an existing rule, restore its backup instead.\nLocking, blanking, and suspend are different The starter config asks X to start its screen saver after five minutes, and xss-lock launches i3lock. DPMS requests display power-off after ten minutes. These settings do not themselves define the laptop\u0026rsquo;s lid or system suspend policy.\nBefore relying on the laptop for administration, verify all three cases:\nPress the manual lock shortcut and confirm that unlocking requires your password. Wait through the idle interval and check that the session locks. Suspend from the local session, resume, and verify that the desktop remains locked. Test without unsaved work. Inspect the running processes if you get duplicate lockers or compositors:\n1 2 3 pgrep -a xss-lock pgrep -a i3lock pgrep -a picom Keep one intended startup entry per service. Using exec_always for every background application can create duplicates when you restart i3.\nCheck power management before tuning it My setup uses TLP. Before installing or enabling another power manager, inspect what is already active:\n1 2 systemctl is-active tlp.service power-profiles-daemon.service systemctl is-enabled tlp.service power-profiles-daemon.service An inactive result or missing unit is information, not necessarily an error. Choose one power-management approach and follow its current documentation; do not enable competing managers and hope their settings agree.\nIf TLP is installed, collect its view of the system:\n1 2 sudo tlp-stat -s sudo tlp-stat -b Compare battery behavior at the same screen brightness and workload. Battery wear, radio activity, background tasks, and GPU state can change the result. I am not claiming a particular runtime or power saving from this configuration.\nTreat Optimus as a separate project Some T420 units include an NVIDIA GPU alongside Intel graphics. Identify yours before choosing a driver:\n1 lspci -nnk | sed -n \u0026#39;/VGA compatible controller/,+3p; /3D controller/,+3p\u0026#39; On my laptop, a check showed the NVIDIA device in runtime suspend. That is an observation at one moment, not proof that it stays suspended after every resume or application launch.\nCheck the current driver support for your exact GPU and kernel. Installing a modern NVIDIA package simply because the laptop has an NVIDIA sticker is not a reliable approach for hardware this old. Keep graphics changes separate from desktop configuration so you can identify which change breaks a working session.\nWhat I left unfinished The fingerprint reader accepted enrollment but failed verification often enough that I stopped using it. Password login remains the baseline. I would not add fingerprint authentication to PAM until repeated verification works and another login method has been tested.\nThe physical mute behavior and battery-related shortcuts also deserve their own checks. A button that worked under Lenovo\u0026rsquo;s Windows software may need an explicit Linux binding or may produce a different event than expected.\nPolybar, custom wallpapers, a newer-looking terminal, and clipboard history are useful refinements. They come after working networking, readable fonts, dependable audio, and a lock screen that survives suspend.\nA short acceptance check Test What to verify Cold boot Reach a local login and start i3 Networking Connect, resolve DNS, and SSH to a known host Audio Playback, volume changes, and mute/unmute Input TrackPoint, touchpad, typing, and tapping if enabled Display Brightness keys and readable native resolution Locking Manual lock, idle lock, and suspend/resume lock Battery Charge reporting and a repeatable light workload The T420 does not need to replace a modern workstation to be useful. Mine has a focused job: connect to the machines doing the heavy work, keep a terminal open, and let me administer the lab from a keyboard I enjoy using.\nContinue reading Intel AMT Serial over LAN — Use your administration client to reach a compatible host\u0026rsquo;s text console. Proxmox bridges and VLANs — Learn how the machines you administer connect to their networks. Prometheus and Grafana foundations — Build a dashboard for checking the lab from your client. References and image provenance ArchWiki: Lenovo ThinkPad T420 ArchWiki: i3 i3 user guide ArchWiki: xinit ArchWiki: TLP WirePlumber: wpctl These upstream documentation entry points should be checked against your installed versions. Live documentation retrieval was unavailable while preparing this draft.\nThe cover is an AI-generated editorial illustration made with the built-in image-generation tool. It is not evidence of the physical machine, its installed software, or its performance. The prompt is preserved alongside the asset in the repository.\n","permalink":"https://blog.gntech.me/posts/2026-10-30-thinkpad-t420-arch-i3/","summary":"My T420 became a useful administration laptop after an SSD and 8 GB RAM upgrade. Here is a reproducible starting point for a small Arch/i3 desktop, plus the hardware issues worth checking.","title":"ThinkPad T420 with Arch Linux and i3 — A Practical Admin Laptop"},{"content":"OneDrive was using space for something I never intended to sync: PowerShell modules. An inventory on one of my Windows accounts found approximately 1.16 GiB under Documents\\WindowsPowerShell\\Modules and another 1.22 GiB under Documents\\PowerShell\\Modules.\nThat is nearly 2.4 GiB of module files, before counting anything else in Documents.\nThe reason is the combination of PowerShell\u0026rsquo;s per-user module locations and a Documents folder redirected into OneDrive. The fix needs two parts: relocate the existing modules and give future module downloads an explicit destination.\nThis guide prepares a cleanup on one Windows account. It does not change your Documents location or disable OneDrive backup.\nVerification status Scope Evidence and limits Environment Windows account with OneDrive-backed Documents; instructions distinguish Windows PowerShell 5.1 from PowerShell 7. Previously reported lab observations Module folders occupied approximately 1.16 GiB and 1.22 GiB; a previous inventory encountered Constrained Language restrictions. Preparation checks Markdown structure was checked during preparation. The PowerShell examples were not executed or syntax-validated with PowerShell. Still to verify Pilot copying and imports in both editions, check scheduled jobs and other synced devices, and verify recovery before removing source files. This table summarizes the evidence recorded for this article. It is not a new hardware test or a certification for other versions.\nBefore you start Use the affected user\u0026rsquo;s account. Close scripts and scheduled jobs that use these modules before changing their location. In File Explorer, select Always keep on this device for the source module folders and wait until OneDrive finishes downloading them. Keep syncing enabled until that completes.\nThe examples use built-in Windows tools and PowerShell syntax intended for Windows PowerShell 5.1 and PowerShell 7. They have not been executed against a Windows machine as part of preparing this article. Pilot the procedure before using it broadly.\nIf your organization manages application control, stop when a policy blocks a command or module import. Moving files does not grant permission to run them.\n1. Find the actual Documents folder Run this in each PowerShell edition you use:\n1 2 3 4 $PSVersionTable.PSVersion $ExecutionContext.SessionState.LanguageMode $env:PSModulePath -split \u0026#39;;\u0026#39; Get-ItemProperty -LiteralPath \u0026#39;HKCU:\\Software\\Microsoft\\Windows\\CurrentVersion\\Explorer\\User Shell Folders\u0026#39; -Name Personal The last command shows the account\u0026rsquo;s configured Documents location. Look for module search paths under that location. Typical redirected paths look like this:\n1 2 C:\\Users\\example\\OneDrive - Example\\Documents\\WindowsPowerShell\\Modules C:\\Users\\example\\OneDrive - Example\\Documents\\PowerShell\\Modules WindowsPowerShell normally corresponds to Windows PowerShell 5.1; PowerShell corresponds to PowerShell 7. Keep their module trees separate because compatibility and dependencies can differ.\nDo not assume that Documents is always at $env:USERPROFILE\\Documents.\n2. Measure the module folders Set $documents to the real path found above. Replace the example before running:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 $documents = \u0026#39;C:\\Users\\example\\OneDrive - Example\\Documents\u0026#39; foreach ($edition in \u0026#39;WindowsPowerShell\u0026#39;, \u0026#39;PowerShell\u0026#39;) { $source = Join-Path $documents \u0026#34;$edition\\Modules\u0026#34; if (Test-Path -LiteralPath $source) { $measure = Get-ChildItem -LiteralPath $source -File -Recurse -Force -ErrorAction Stop | Measure-Object -Property Length -Sum $folders = Get-ChildItem -LiteralPath $source -Directory -ErrorAction Stop | Measure-Object Write-Output \u0026#34;Path: $source\u0026#34; Write-Output \u0026#34;Module folders: $($folders.Count)\u0026#34; Write-Output \u0026#34;Logical size in GiB: $($measure.Sum / 1GB)\u0026#34; } } This measures logical file sizes. It does not measure local disk allocation or OneDrive\u0026rsquo;s server-side quota, which can include other content and version history. Resolve access errors before continuing.\n3. Copy the modules outside OneDrive Keep $documents set in the same window. The destination is under the user\u0026rsquo;s local application data folder, which is normally outside OneDrive. Verify that is true on your machine.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 $localRoot = Join-Path $env:LOCALAPPDATA \u0026#39;PowerShellModules\u0026#39; New-Item -ItemType Directory -Path $localRoot -Force | Out-Null foreach ($edition in \u0026#39;WindowsPowerShell\u0026#39;, \u0026#39;PowerShell\u0026#39;) { $source = Join-Path $documents \u0026#34;$edition\\Modules\u0026#34; $destination = Join-Path $localRoot \u0026#34;$edition\\Modules\u0026#34; if (Test-Path -LiteralPath $source) { New-Item -ItemType Directory -Path $destination -Force | Out-Null robocopy.exe $source $destination /E /COPY:DAT /DCOPY:DAT /R:2 /W:2 /XJ if ($LASTEXITCODE -ge 8) { throw \u0026#34;Copy failed for $source. Keep the source and inspect the robocopy output.\u0026#34; } } } Robocopy uses exit codes differently from many command-line tools: codes below 8 do not indicate a copy failure. Review its summary for failed files, mismatches, and unexpected extra files too. The command copies subdirectories and does not delete the source or mirror deletions.\nTreat this as a migration copy, not your only backup. Preserve an independent backup before the removal step.\n4. Add the correct local tree to a fresh session Open a new PowerShell window and run this bootstrap:\n1 2 3 4 5 6 7 8 9 10 $editionFolder = \u0026#39;WindowsPowerShell\u0026#39; if ($PSVersionTable.PSVersion.Major -ge 6) { $editionFolder = \u0026#39;PowerShell\u0026#39; } $localModules = Join-Path $env:LOCALAPPDATA \u0026#34;PowerShellModules\\$editionFolder\\Modules\u0026#34; $env:PSModulePath = \u0026#34;$localModules;$env:PSModulePath\u0026#34; Get-Module -ListAvailable | Where-Object { $_.ModuleBase -like \u0026#34;$localModules\\*\u0026#34; } | Select-Object Name, Version, ModuleBase This changes the current process only. Windows and PowerShell reconstruct module paths at startup; a custom search path also does not redefine the default destination used by Install-Module -Scope CurrentUser.\nFor an interactive setup, the bootstrap can live in an approved PowerShell profile. Review the existing profile before adding it, and remember that profiles differ between editions and hosts. A profile under redirected Documents may still sync, but the large module files stay local.\nFor scheduled jobs or sessions launched with -NoProfile, put the bootstrap directly in the job\u0026rsquo;s approved entry script. A profile is not a tenant-wide deployment mechanism.\n5. Verify the modules you actually use In the fresh session, replace the example with a module you rely on:\n1 2 3 4 5 Import-Module Microsoft.Graph.Authentication -ErrorAction Stop Get-Module Microsoft.Graph.Authentication | Select-Object Name, Version, ModuleBase Get-Command Connect-MgGraph -ErrorAction Stop | Select-Object Name, Source, Version Check that ModuleBase points into the new local tree. If the module is not installed, choose one that appeared in your inventory. Repeat in both editions and run a harmless read-only operation from each important module. Importing successfully does not prove every feature works.\nAlso check scheduled tasks, VS Code sessions, and scripts with hardcoded OneDrive paths. They need the local bootstrap or updated paths before cleanup.\n6. Prevent future downloads from returning to OneDrive When using PowerShellGet, Save-Module lets you choose the destination. After running the bootstrap above, this example downloads a module and its dependencies into the local tree:\n1 Save-Module -Name Microsoft.Graph.Authentication -Path $localModules -Repository PSGallery -ErrorAction Stop Select a version appropriate to your environment; use -RequiredVersion when you need a reproducible installation. Review repository trust and any prompts rather than automatically bypassing them. If the command is unavailable, arrange an approved installation of PowerShellGet first.\nSave-Module saves files; the module becomes discoverable because its destination is on PSModulePath. It does not make Get-InstalledModule or Update-Module manage those saved copies as ordinary Install-Module installations. Maintain them with an explicit, versioned download workflow.\nContinuing to use Install-Module -Scope CurrentUser can recreate modules in redirected Documents. Merely prepending a directory to PSModulePath does not prevent that.\n7. Remove only a verified source tree Deletion inside OneDrive propagates to the cloud and other synced devices. First confirm that other machines have their own working local copies or an approved way to reinstall the required modules. The destination under LocalAppData does not travel to those machines with OneDrive.\nFor each edition, verify the copy and important imports, confirm an independent backup exists, and then preview removal of that specific source folder:\n1 2 3 $sourceToRemove = Join-Path $documents \u0026#39;WindowsPowerShell\\Modules\u0026#39; Get-Item -LiteralPath $sourceToRemove -ErrorAction Stop Remove-Item -LiteralPath $sourceToRemove -Recurse -Force -WhatIf The preview deletes nothing. Only after reviewing the exact path and completing the checks above, run:\n1 Remove-Item -LiteralPath $sourceToRemove -Recurse -Force -Confirm Repeat separately for PowerShell\\Modules if it has also passed validation. Do not delete Documents, either parent PowerShell folder, or profiles as part of this cleanup.\nWait for OneDrive to finish syncing, then inspect storage in OneDrive on the web. Do not promise an immediate quota reduction equal to the inventory. Review recycle bin and version retention behavior for your account before permanently removing recovery options.\nIf PowerShell reports ConstrainedLanguage On my account, a previous inventory attempt failed because method invocation was restricted. Check the mode with:\n1 $ExecutionContext.SessionState.LanguageMode The inventory above avoids calling methods such as [Environment]::GetFolderPath(). This helps with that specific restriction, but it does not guarantee that module imports, profile changes, or every command will be permitted by application control.\nIf policy blocks a step, use your organization\u0026rsquo;s approved deployment process. Do not switch language modes or weaken execution controls to complete the cleanup.\nRollback Before removal, the original module folders remain available. To undo the session-only search path change, close that window and start a new one without the bootstrap. If you added it to a profile or task, remove only your added block.\nAfter removal, restore the original tree from your independent backup, or use OneDrive recovery if the content is still recoverable. Recheck module versions and import locations in a fresh session. Avoid blindly overwriting newer files with an older copy.\nFor multiple managed Windows devices Pilot on one account first. Package the approved module versions into a local location, configure discovery in each supported host, verify application-control compatibility, and only then remove synced copies. Include jobs running under different identities in the pilot.\nA OneDrive exclusion rule is not a substitute for moving the module workload: modules left in Documents still need a deliberate installation and discovery strategy. Treat fleet deployment as a separate change with detection, logging, and rollback.\nReferences Microsoft: about_PSModulePath Microsoft: Save-Module Microsoft: about_Language_Modes Microsoft: Robocopy These are the vendor documentation entry points for the commands used here. Confirm the documentation matches the PowerShell and PowerShellGet versions installed on your machine.\n","permalink":"https://blog.gntech.me/posts/2026-10-30-powershell-modules-onedrive-cleanup/","summary":"A practical cleanup workflow for PowerShell modules in a OneDrive-backed Documents folder, with verification, rollback, and explicit destinations for future downloads.","title":"PowerShell Modules Filled My OneDrive — How to Clean Up Safely"},{"content":"A 10GbE upgrade can help when backups, storage traffic, or VM migrations saturate a 1GbE link. Start with a measured bottleneck, choose compatible hardware, and validate the network before changing TCP settings or enabling jumbo frames.\nThis example connects two Proxmox hosts through a MikroTik CRS switch. It uses ordinary 1500-byte MTU first, a separate management VLAN, and explicit access-port PVIDs.\nHardware and Example Layout Use a CRS model with enough SFP+ ports, supported SFP+ NICs, and compatible DAC cables or optical modules. Check each device\u0026rsquo;s specification for supported speeds, optics, hardware offloading, power, and cooling. A 25GbE NIC does not necessarily work with every 10GbE cable or module.\nConnection Switch port VLAN handling Upstream router sfp-sfpplus1 Tagged VLAN 20 and 99 Proxmox host A sfp-sfpplus3 Untagged VLAN 20 Proxmox host B sfp-sfpplus4 Untagged VLAN 20 Switch management Bridge CPU through vlan99 Tagged VLAN 99 Example addresses: switch 10.0.99.2/24, upstream management gateway 10.0.99.1, hosts 10.0.20.10/24 and 10.0.20.20/24. These are sample values, not discovery results. Configure the upstream router\u0026rsquo;s tagged VLANs and management access before enabling filtering.\n1. Back Up and Prepare the Switch Use a local console or an independent management connection and RouterOS Safe Mode. These commands assume a spare switch with no existing bridge10g, port memberships, or conflicting IP addresses. Inspect and adapt an existing deployment instead of pasting the whole example into it.\n/export file=before-10gbe /system backup save name=before-10gbe /interface bridge print /interface bridge port print /ip address print Download the backups before continuing. A binary backup is device-specific and may contain sensitive data. This guide does not require a factory reset. RouterOS run-after-reset expects a script filename, not inline configuration text.\n2. Create the Bridge with Filtering Disabled /interface bridge add name=bridge10g protocol-mode=rstp vlan-filtering=no /interface bridge port add bridge=bridge10g interface=sfp-sfpplus1 hw=yes ingress-filtering=yes frame-types=admit-only-vlan-tagged /interface bridge port add bridge=bridge10g interface=sfp-sfpplus3 hw=yes pvid=20 ingress-filtering=yes frame-types=admit-only-untagged-and-priority-tagged /interface bridge port add bridge=bridge10g interface=sfp-sfpplus4 hw=yes pvid=20 ingress-filtering=yes frame-types=admit-only-untagged-and-priority-tagged /interface bridge vlan add bridge=bridge10g vlan-ids=20 tagged=sfp-sfpplus1 untagged=sfp-sfpplus3,sfp-sfpplus4 /interface bridge vlan add bridge=bridge10g vlan-ids=99 tagged=bridge10g,sfp-sfpplus1 /interface vlan add name=vlan99 interface=bridge10g vlan-id=99 /ip address add address=10.0.99.2/24 interface=vlan99 /interface bridge set [find name=bridge10g] frame-types=admit-only-vlan-tagged The access ports need pvid=20 to classify untagged incoming traffic. Listing a port as untagged in the VLAN table controls outgoing tagging; it does not replace the PVID.\nThe bridge itself is included in VLAN 99\u0026rsquo;s tagged list because switch management needs CPU access on that VLAN. VLAN 20 is switched between ports without giving the switch CPU a VLAN 20 address.\n3. Enable Filtering and Check Access /interface bridge set [find name=bridge10g] vlan-filtering=yes /interface bridge port print /interface bridge vlan print detail /interface ethernet monitor sfp-sfpplus1 once Verify management connectivity from an authorized VLAN 99 client before leaving Safe Mode. The H flag in bridge-port output indicates active hardware offloading; hw=yes requests it. Actual support depends on the switch chip, RouterOS version, and features in use.\nIf management fails, use the independent connection or Safe Mode rollback. Do not keep adding VLAN rules through an unstable management session.\n4. Connect the Proxmox Hosts Check the NIC driver, interface name, and negotiated speed on each host:\n1 2 3 lspci -nnk | grep -A3 -i ethernet ip -br link ethtool enp4s0f0 Replace enp4s0f0 with the actual interface. Drivers normally load automatically when supported; do not append duplicate module entries as a default step.\nThe switch\u0026rsquo;s access ports deliver untagged VLAN 20. This additional Proxmox bridge therefore needs no VLAN subinterface:\n1 2 3 4 5 6 7 8 9 10 11 auto enp4s0f0 iface enp4s0f0 inet manual mtu 1500 auto vmbr10 iface vmbr10 inet static address 10.0.20.10/24 mtu 1500 bridge-ports enp4s0f0 bridge-stp off bridge-fd 0 Use 10.0.20.20/24 on host B. Do not add a second default gateway. This example assumes VLAN 20 is not already configured on another interface on the same host.\nBack up the configuration first:\n1 sudo cp -a /etc/network/interfaces /root/interfaces.before-10gbe Apply the additional bridge through the Proxmox network editor, review the pending changes, and use Apply Configuration from a console-accessible session. Do not overwrite the complete existing interfaces file with this fragment.\n5. Measure Before Tuning Install iperf3 on both hosts if needed. On host B, start a server bound to the storage-network address:\n1 iperf3 -s -B 10.0.20.20 Allow TCP port 5201 only between the test hosts in their firewalls. On host A:\n1 2 3 iperf3 -c 10.0.20.20 -t 30 iperf3 -c 10.0.20.20 -P 4 -t 30 iperf3 -c 10.0.20.20 -P 4 -t 30 -R Stop the server with Ctrl+C afterward. Record the measured throughput, CPU use, retransmissions, and link errors. Values near line rate are possible, but throughput depends on the hosts and protocol overhead; an example number is not a benchmark from your hardware.\nInspect errors before and after each test:\n1 2 ethtool -S enp4s0f0 | grep -Ei \u0026#39;crc|error|drop|miss\u0026#39; ip -s link show enp4s0f0 Look for increasing counters. An old nonzero counter alone does not establish an active fault. Test network throughput separately from storage throughput: disks, compression, encryption, and application behavior can limit backups even when the network is fast.\nOptional: Jumbo Frames and LACP Keep MTU 1500 unless testing demonstrates a reason to change it. For MTU 9000, every endpoint, bridge, VLAN interface, and switch path must support the required frame size. Check the switch\u0026rsquo;s L2MTU and device-specific limits; changing a physical interface\u0026rsquo;s IP MTU alone does not prove the switching path supports jumbo frames.\nAfter configuring the entire path, test IPv4 without fragmentation:\n1 ping -M do -s 8972 -c 4 10.0.20.20 If it fails, revert the path consistently to MTU 1500 and investigate. Do not use jumbo frames as an unexplained default.\nLACP is a separate design choice. Remove member ports from standalone bridge membership before adding a bond, and put the bond itself into the VLAN table. Configure both ends consistently and verify switch-chip offload support. LACP distributes flows; it does not automatically double the throughput of one TCP connection.\nTroubleshooting No link: verify cable/module compatibility, NIC firmware, and supported link modes at both ends. Use RouterOS tab completion for version-specific speed names rather than copying an old advertise value. No connectivity on access ports: check PVIDs, VLAN membership, upstream routing, and host firewall rules. Low throughput: compare single-stream and parallel results, inspect CPU load and increasing error counters, and test a known-good cable. Flapping: inspect switch and NIC logs before changing driver flags. Private ethtool flags and EEE settings are device-specific. Remote management lost: use the console or Safe Mode rollback. Restore the reviewed configuration from your backup rather than resetting blindly. Do not apply arbitrary IRQ masks, ring sizes, BBR, or large TCP buffers as a universal 10GbE fix. Inspect available settings and test one change at a time against the baseline.\nReferences MikroTik: Bridge VLAN Table MikroTik: Bridging and Switching Proxmox: Network Configuration iperf3 documentation ","permalink":"https://blog.gntech.me/posts/2026-06-20-homelab-10gbe-networking-mikrotik-proxmox/","summary":"Planning a 10GbE upgrade for your homelab? From MikroTik CRS switches and SFP+ cabling to Proxmox NIC tuning, this guide covers the hardware decisions, configs, and performance optimizations you need.","title":"Homelab 10GbE Networking — Hardware, MikroTik Switch Setup, and Performance Tuning"},{"content":"Good infrastructure notes answer practical questions: which VLAN carries management traffic, how to restore a service, and what to check after changing a firewall rule.\nDocusaurus turns Markdown and MDX files into a static documentation site. You need Node.js and dependencies to build it; the resulting files can be served by a web server without running the Node application in production.\nThis article combines the two earlier Docusaurus setup posts. It keeps one maintained walkthrough and corrects the earlier claims about built-in local search, automatic Traefik setup, and unconditional deployment times. The configuration below is an example, not a report of a verified deployment in my lab.\nBefore you begin Use a development machine with a Node.js version supported by the Docusaurus release you select, npm, and Git. For the container section, you also need Docker Engine and the Docker Compose plugin.\n1 2 3 node --version npm --version docker compose version Check the current installation requirements before scaffolding. The commands intentionally use the current generator for a new project; preserve the resulting lockfile and record the selected versions. They are not a version-pinned reproduction of either original article.\nCreate and build the project 1 2 3 4 npx create-docusaurus@latest homelab-docs classic cd homelab-docs npm run build npm run start -- --host 127.0.0.1 Use the development address printed by the command, usually http://localhost:3000. Stop it with Ctrl+C when finished. Do not expose the development server as the production service.\nThe classic template includes documentation, theme configuration, and example content. Keep its generated presets and configuration structure. JavaScript and TypeScript templates use different filename extensions, so confirm which files were created.\nSet the production URL For a new scaffold, the following replaces the top-level site title, URL and base path. Change docs.example.com to your intended hostname. Back up the generated configuration first:\n1 2 3 4 5 6 7 8 9 10 11 12 config_file=docusaurus.config.js if [ ! -f \u0026#34;$config_file\u0026#34; ]; then config_file=docusaurus.config.ts fi test -f \u0026#34;$config_file\u0026#34; || { echo \u0026#39;Generated configuration not found\u0026#39;; exit 1; } cp -a \u0026#34;$config_file\u0026#34; \u0026#34;$config_file.bak-$(date +%Y%m%d-%H%M%S)\u0026#34; sed -i \\ -e \u0026#34;s|^ title:.*| title: \u0026#39;Homelab Docs\u0026#39;,|\u0026#34; \\ -e \u0026#34;s|^ url:.*| url: \u0026#39;https://docs.example.com\u0026#39;,|\u0026#34; \\ -e \u0026#34;s|^ baseUrl:.*| baseUrl: \u0026#39;/\u0026#39;,|\u0026#34; \\ \u0026#34;$config_file\u0026#34; sed -n \u0026#39;1,100p\u0026#39; \u0026#34;$config_file\u0026#34; These substitutions assume the generator uses the shown two-space indentation and property names. Inspect the printed configuration and confirm all three values changed. For hosting under a subpath, use the appropriate baseUrl instead of / and adjust the web server accordingly.\nBuild again after configuration changes. Do not replace the entire generated configuration with a fragment for an optional plugin; that can remove the presets which provide the documentation pages.\nWrite a useful first runbook Start with one service and a recovery procedure. This creates a new page without replacing the template\u0026rsquo;s introduction:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 mkdir -p docs/runbooks cat \u0026gt; docs/runbooks/service-recovery.md \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; --- title: Service recovery checklist sidebar_position: 1 --- # Service recovery checklist ## Scope Record the service, host, dependencies, and supported versions here. ## Before changing anything - Confirm the affected host and current symptoms. - Identify the latest usable backup and a recovery access path. - Record the current configuration. ## Procedure Add the exact steps for this service, including prerequisites. ## Verification Describe the expected result and how to check it. ## Rollback Explain how to restore the previous state. EOF npm run build This is a writing template, not an executable recovery procedure. Fill in the service-specific details before anyone relies on it.\nWith an autogenerated sidebar covering the docs directory, the new page should appear in navigation. If your project uses a manual sidebar, add the document ID runbooks/service-recovery to that existing configuration. A sidebar entry pointing to a nonexistent document can fail the build.\nUseful sections for a growing lab include networking, virtualization, services, backups, and runbooks. Keep credentials and recovery keys out of the documentation repository. A static site has no authentication boundary simply because its content feels internal.\nBuild a static container image The container uses the Node major version already installed on your development machine. That version must also satisfy the generated project\u0026rsquo;s requirements. For repeatable releases, pin the build and server images to reviewed versions or digests rather than relying indefinitely on mutable tags.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 export NODE_IMAGE=\u0026#34;node:$(node -p \u0026#39;process.versions.node.split(\u0026#34;.\u0026#34;)[0]\u0026#39;)-alpine\u0026#34; printf \u0026#39;Build image: %s\\n\u0026#39; \u0026#34;$NODE_IMAGE\u0026#34; cat \u0026gt; Dockerfile \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; ARG NODE_IMAGE FROM ${NODE_IMAGE} AS build WORKDIR /app COPY package.json package-lock.json ./ RUN npm ci COPY . . RUN npm run build FROM nginx:stable-alpine COPY nginx.conf /etc/nginx/conf.d/default.conf COPY --from=build /app/build /usr/share/nginx/html EXPOSE 80 CMD [\u0026#34;nginx\u0026#34;, \u0026#34;-g\u0026#34;, \u0026#34;daemon off;\u0026#34;] EOF cat \u0026gt; .dockerignore \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; node_modules build .docusaurus .git .env *.log EOF cat \u0026gt; nginx.conf \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; server { listen 80; server_name _; root /usr/share/nginx/html; index index.html; location / { try_files $uri $uri/ $uri.html =404; } } EOF cat \u0026gt; compose.yaml \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; services: docs: build: context: . args: NODE_IMAGE: ${NODE_IMAGE:?Set NODE_IMAGE to a compatible Node image} restart: unless-stopped ports: - \u0026#34;127.0.0.1:3000:80\u0026#34; EOF docker compose config --quiet docker compose up -d --build docker compose exec -T docs nginx -t These commands create deployment files in the project root. Back up any existing files with those names before adapting this section to an established project. The example assumes npm and an existing package-lock.json; use the matching package manager if you generated the project differently.\nThe site binds to the Docker host\u0026rsquo;s loopback address. That is convenient for local testing or a proxy running on the host. A reverse proxy in another container cannot reach it through that container\u0026rsquo;s own 127.0.0.1.\nVerify navigation and missing pages 1 2 3 4 curl --fail --head http://127.0.0.1:3000/ curl --silent --output /dev/null --write-out \u0026#39;%{http_code}\\n\u0026#39; \\ http://127.0.0.1:3000/this-page-does-not-exist docker compose logs --tail=50 docs Expect the home request to succeed and the invented URL to return 404. Open the runbook through the sidebar, then refresh that page directly. Check navigation and static assets in a browser too. An Nginx syntax check alone does not prove that the generated routes work.\nThe earlier catch-all /index.html fallback could return the homepage with a successful status for missing routes. This example instead checks generated directory and .html paths before returning a real 404.\nAdd a reverse proxy deliberately The loopback deployment does not install Traefik or issue a TLS certificate. To use an existing containerized Traefik deployment, attach the docs container to its shared Docker network, remove the host port mapping if it is unnecessary, and configure a router and service for the docs container\u0026rsquo;s port 80.\nConfirm the proxy\u0026rsquo;s actual Docker provider, network, entrypoint and certificate-resolver names. Labels copied from a guide do not create those prerequisites. Validate HTTPS and deep links through the real hostname after configuring the proxy.\nFor a managed static host, deploy the generated build directory instead. Review that host\u0026rsquo;s build settings and access controls; private repository storage does not automatically make the deployed site private.\nSearch, diagrams, and versioning Search needs an integration. Docusaurus does not ship a built-in Lunr search engine just because the classic template or sidebar is enabled. Choose an integration from the search documentation, such as Algolia DocSearch where eligible or a compatible local-search plugin. Confirm plugin compatibility with your Docusaurus release and test the production build, including navigation from a search result.\nMermaid needs configuration. Follow the Mermaid documentation to install the matching theme package and merge its settings into the existing site configuration. A fenced diagram alone does not enable the integration.\nGit history and published versions are different. Git records every edit. Docusaurus documentation versioning creates reader-visible snapshots. Introduce snapshots when readers genuinely need multiple supported procedures; avoid creating a new public version for every lab edit. Follow the versioning guide and add the version selector to navigation if you need one.\nPublish changes through review Keep the source, package lockfile, and deployment files in Git. Build proposed changes before merging them, review the rendered site, and deploy the approved revision. Keep the previous working image or deployment available for rollback.\nThe earlier article included a netcat listener that triggered rebuilds without authenticating webhook requests. Do not use that listener as a deployment mechanism. Use your hosting provider\u0026rsquo;s Git integration or a CI workflow with deliberately configured deployment credentials and host verification. Deployment time depends on the build, image cache, runner and network; there is no fixed completion time promised here.\nValidation limits This consolidated example has been reviewed for consistency and its Bash blocks can be syntax-checked without running them. It has not been built as a Docusaurus application or deployed against a live Docker host during this update. Check the current upstream requirements, build the generated project, and verify the served pages before adopting it.\n","permalink":"https://blog.gntech.me/posts/2026-06-20-docusaurus-homelab-documentation-guide/","summary":"One guide to a Git-based homelab documentation site: project setup, useful runbooks, static builds, Docker hosting, and optional search and versioning.","title":"Docusaurus for Homelab Documentation — Setup and Docker Deployment"},{"content":"If you run a homelab with Proxmox, Docker, or LXC containers, the single highest-impact, lowest-effort network performance tuning you can do is switching your TCP congestion control algorithm from the default CUBIC to BBR (Bottleneck Bandwidth and Round-trip propagation time). No hardware changes, no wiring upgrades, no VLAN rework — just a few sysctl lines and you get 2x-5x better throughput on long-distance or lossy links.\nThis guide covers enabling TCP BBR on your Proxmox host, propagating it to Docker containers and LXC guests, verifying it works, and benchmarking the results with iperf3. Kernel and network-namespace behavior must be verified on the deployed system; this revision does not establish execution evidence for these commands.\nChecking Your Current TCP Congestion Control Before changing anything, see what algorithm is currently active and what\u0026rsquo;s available on your kernel:\n1 2 3 4 5 6 7 8 # Current congestion control sysctl net.ipv4.tcp_congestion_control # Available algorithms compiled into the kernel sysctl net.ipv4.tcp_available_congestion_control # Check if BBR module is loaded lsmod | grep tcp_bbr On a stock Proxmox 8.x or Debian 12 install, the output looks like this:\nnet.ipv4.tcp_congestion_control = cubic net.ipv4.tcp_available_congestion_control = cubic reno BBR is not listed because it ships as a kernel module, not compiled in. That is expected — you load it and it becomes available.\nEnabling BBR on the Proxmox or Linux Host BBR was mainlined in Linux 4.9 and is built as a module in every modern distro kernel. Enabling it takes two steps: loading the module and setting the sysctl.\nStep 1 — Load the BBR Module and Make It Persistent 1 2 3 4 5 6 7 8 9 # Load immediately modprobe tcp_bbr # Verify it loaded lsmod | grep tcp_bbr # tcp_bbr 24576 1 # Make it load on boot echo \u0026#34;tcp_bbr\u0026#34; \u0026gt;\u0026gt; /etc/modules-load.d/tcp_bbr.conf Step 2 — Set BBR and fq qdisc via sysctl BBR uses pacing-based congestion detection. For pacing to work optimally, pair it with the fq (Fair Queue) qdisc. Without fq, BBR still functions but you lose the per-flow pacing that gives the biggest latency improvements.\n1 2 3 4 5 6 7 8 9 # Apply immediately sysctl -w net.ipv4.tcp_congestion_control=bbr sysctl -w net.core.default_qdisc=fq # Make persistent cat \u0026gt; /etc/sysctl.d/99-bbr.conf \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; net.ipv4.tcp_congestion_control = bbr net.core.default_qdisc = fq EOF Step 3 — Verify 1 2 3 4 5 sysctl net.ipv4.tcp_congestion_control # net.ipv4.tcp_congestion_control = bbr sysctl net.ipv4.tcp_available_congestion_control # net.ipv4.tcp_available_congestion_control = bbr cubic reno That is all it takes. From this point forward, every new TCP connection on the host uses BBR.\nWhy BBR and fq Must Be Paired BBR is not like CUBIC or Reno. Those algorithms react to packet loss (loss-based). BBR models the bottleneck bandwidth and RTT directly (model-based). This gives it much better behavior on links with bufferbloat or packet loss from non-congestion sources (e.g., Wi-Fi drops).\nHowever, BBR expects the network stack to pace outgoing packets rather than blasting them in bursts. The fq qdisc provides fair per-flow pacing:\nfq — Fair Queue with per-packet pacing. Best for BBR. fq_codel — Adds Controlled Delay (CoDel) AQM. Good alternative for general use. Avoid pfifo_fast (the default) — it does not pace packets and BBR underperforms. You can verify the current qdisc:\n1 tc -s qdisc show dev eth0 On a tuned host you should see fq as the root qdisc:\nqdisc fq 0: root refcnt 2 limit 10000p flow_limit 100p buckets 1024 ... Enabling BBR Inside Docker Containers Docker containers share the host network namespace by default when using --network host. In that case, the container inherits the host\u0026rsquo;s tcp_congestion_control automatically. No extra config needed.\nFor containers on the default bridge network or a user-defined bridge, each container gets its own network namespace. Here the situation depends on your Docker version and kernel:\nDocker containers with default bridge — works automatically On modern kernels (5.8+) with Docker 20.10+, the host\u0026rsquo;s sysctl for tcp_congestion_control propagates to container namespaces. Verify from inside a container:\n1 2 docker exec my-container sysctl net.ipv4.tcp_congestion_control # net.ipv4.tcp_congestion_control = bbr If it still shows cubic, you can force it with --sysctl:\n1 docker run --rm --sysctl net.ipv4.tcp_congestion_control=bbr alpine sysctl net.ipv4.tcp_congestion_control Docker Compose sysctl block Add a sysctls section to each service in your compose file:\n1 2 3 4 5 services: app: image: your-image:latest sysctls: - net.ipv4.tcp_congestion_control=bbr Note: --sysctl and sysctls require kernel support for namespace isolation of the given parameter. If Docker rejects it, check /proc/sys/net/ipv4/tcp_congestion_control is writable inside the container by verifying the kernel has CONFIG_NET_NS enabled (it does on all distro kernels).\nDocker Compose full example for a Jellyfin or Plex container 1 2 3 4 5 6 7 8 9 10 11 12 13 services: media: image: jellyfin/jellyfin:latest container_name: jellyfin sysctls: - net.ipv4.tcp_congestion_control=bbr - net.core.default_qdisc=fq ports: - \u0026#34;8096:8096\u0026#34; volumes: - ./media:/media - ./config:/config restart: unless-stopped This ensures that even on a bridge network, remote streaming connections from Jellyfin benefit from BBR\u0026rsquo;s pacing and bandwidth modeling.\nEnabling BBR Inside LXC Containers on Proxmox LXC containers are closer to the host than Docker. They share the host\u0026rsquo;s kernel and, in most configurations, inherit the host\u0026rsquo;s TCP congestion control automatically.\nCheck from the Proxmox host 1 2 3 # For a container with ID 101 lxc-attach -n 101 -- sysctl net.ipv4.tcp_congestion_control # net.ipv4.tcp_congestion_control = bbr If your Proxmox host has BBR enabled, LXC containers almost always inherit it because they share the host\u0026rsquo;s network namespace for net.ipv4.* sysctls (unless you used lxc.net.[i].flags = unconfined or lxc.sysctl overrides).\nPrivileged vs unprivileged containers Privileged containers: Inherit host sysctls with no restrictions. BBR applies automatically. Unprivileged containers: May have restricted write access to /proc/sys/net/ipv4/tcp_congestion_control. However, the read value still reflects the host\u0026rsquo;s setting, so connections use BBR even if you cannot change it inside. To be absolutely sure, run a quick iperf3 test from the container to confirm BBR is being used for actual connections.\nBenchmarking BBR vs CUBIC with iperf3 Numbers matter. Here is how to test the difference on your own hardware.\nInstall iperf3 on two hosts 1 2 # Debian/Ubuntu/Proxmox host apt install -y iperf3 Run a server on one machine 1 iperf3 -s -p 5201 Run a client with CUBIC 1 2 3 # On client, temporarily switch back to CUBIC sysctl -w net.ipv4.tcp_congestion_control=cubic iperf3 -c server-ip -p 5201 -t 30 Run the same test with BBR 1 2 sysctl -w net.ipv4.tcp_congestion_control=bbr iperf3 -c server-ip -p 5201 -t 30 Expected results On a typical homelab setup with a 1 GbE link:\nMetric CUBIC BBR Improvement Throughput 450-600 Mbps 850-940 Mbps +50-60% Retransmits 0.5-2% 0.01-0.1% 10-50x fewer RTT variance 5-20 ms 1-5 ms 2-4x reduction These gains are most dramatic on links with bufferbloat (typical consumer cable/fiber modems) and on long-distance connections. On a local 10 GbE backplane, the difference is smaller (BBR still wins but by maybe 10-20%).\nFor remote Plex/Jellyfin streaming specifically, users consistently report that BBR turns unwatchable buffering 4K streams into smooth playback, especially over 4G/5G or congested home upstream links.\nTroubleshooting BBR on Proxmox, Docker, and LXC \u0026ldquo;sysctl: setting key \u0026rsquo;net.ipv4.tcp_congestion_control\u0026rsquo;: No such file or directory\u0026rdquo; This means the kernel was built without the TCP congestion control framework as a loadable module. Almost never happens on distro kernels. If you compiled your own kernel, enable CONFIG_TCP_CONG_BBR=m or CONFIG_TCP_CONG_BBR=y.\n\u0026ldquo;tcp_bbr module not found\u0026rdquo; Your kernel does not include BBR. Check your kernel version:\n1 uname -r Proxmox 8.x ships kernel 6.8.x which includes BBR. If you are on an older Proxmox 7.x with kernel 5.15, BBR is still available — run modprobe tcp_bbr and it should work. If it does not, install linux-image-amd64 from backports.\nDocker \u0026ndash;sysctl is ignored for tcp_congestion_control Docker 25+ and kernel 5.8+ support net.ipv4.tcp_congestion_control as a per-netns sysctl. If Docker silently ignores it, ensure your kernel has CONFIG_NET_NS=y and CONFIG_USER_NS=y:\n1 zcat /proc/config.gz | grep -E \u0026#39;CONFIG_(NET_NS|USER_NS)=y\u0026#39; If config.gz is unavailable, check /boot/config-$(uname -r):\n1 grep CONFIG_NET_NS /boot/config-$(uname -r) LXC container shows CUBIC even though host is BBR This happens only if you explicitly overrode the container\u0026rsquo;s sysctl in the LXC config file (/etc/pve/lxc/CTID.conf). Remove or comment out any lxc.net or lxc.sysctl lines that set net.ipv4.tcp_congestion_control.\nBBRv3 on Linux 6.x Kernels Linux 6.3 introduced BBRv3 (internally tcp_bbr3 module). Proxmox 8.x kernels (6.8+) include BBRv3. To use it:\n1 2 modprobe tcp_bbr3 sysctl -w net.ipv4.tcp_congestion_control=bbr3 Functionally identical to BBRv1 from a config perspective. The improvements are better coexistence with CUBIC flows on shared links and improved handling of small-RTT paths. If BBRv3 is available, use it — same config, slightly better fairness.\nConclusion Enabling TCP BBR on a Proxmox host is three commands and one config file. It propagates to Docker containers and LXC guests with no additional work. The throughput gains are measurable on every link and transformative on congested or long-distance connections.\nIf you self-host services that stream media (Jellyfin, Plex), serve files (Samba, NFS, MinIO), or handle any remote backups (Proxmox Backup Server, rsync, Borg), there is no reason not to enable BBR. The only downside is a kernel module needing to be loaded, which is handled automatically with the two persistence files shown above.\nEnable it today:\n1 2 3 4 5 6 7 8 modprobe tcp_bbr sysctl -w net.ipv4.tcp_congestion_control=bbr sysctl -w net.core.default_qdisc=fq echo \u0026#34;tcp_bbr\u0026#34; \u0026gt;\u0026gt; /etc/modules-load.d/tcp_bbr.conf cat \u0026gt; /etc/sysctl.d/99-bbr.conf \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; net.ipv4.tcp_congestion_control = bbr net.core.default_qdisc = fq EOF Your network will thank you.\nFurther reading: TCP BBR Congestion Control: Making the Internet Faster (Google Research), Linux man page for tc-fq(8), Proxmox Network Performance Tuning docs\n","permalink":"https://blog.gntech.me/posts/2026-06-18-linux-tcp-bbr-congestion-control-proxmox-docker-lxc/","summary":"Enable TCP BBR congestion control on your Proxmox host, Docker containers, and LXC guests. Step-by-step sysctl config, kernel module loading, and iperf3 benchmarks showing real-world throughput gains for homelab and remote streaming.","title":"Linux TCP BBR Congestion Control — Enable on Proxmox, Docker, and LXC"},{"content":"Every homelab running Docker eventually hits the wall where one misbehaving container — a database spike, a CI build, a Plex transcode — starves every other service. You throw a --memory=512m flag on the compose file and hope. But under the hood, what actually limits your containers is the cgroups v2 unified hierarchy.\nModern Linux distributions (Ubuntu 22.04+, Debian 12+, Fedora 37+) ship with cgroups v2 as the default. Docker 20.10+ uses it automatically. Yet most guides still reference the old v1 approach with separate subsystems and double accounting. This post covers the v2-specific knobs that actually matter: memory.high, memory.swap.max, io.max, PSI pressure monitoring, and how to map these to Docker Compose resources.\nChecking Your Cgroups Version and Setup Before tuning anything, verify which cgroup version your system uses:\n1 2 3 4 5 6 7 8 # Should return \u0026#34;cgroup2fs\u0026#34; on a v2 system stat -fc %T /sys/fs/cgroup/ # Available cgroup controllers cat /proc/filesystems | grep cgroup # Docker\u0026#39;s cgroup driver docker info | grep -i cgroup Output from a healthy v2 setup:\ncgroup2fs cgroup2 Cgroup Driver: systemd Cgroup Version: 2 If you\u0026rsquo;re still on v1, you can switch at boot by adding to your kernel command line in /etc/default/grub:\nGRUB_CMDLINE_LINUX_DEFAULT=\u0026#34;cgroup_no_v1=all systemd.unified_cgroup_hierarchy=1\u0026#34; Then run update-grub and reboot. Docker will automatically detect the unified hierarchy.\nThe kernel needs these config options enabled (all present in stock Ubuntu/Debian/Fedora kernels):\nOption Purpose CONFIG_CGROUPS Core cgroup support CONFIG_MEMCG Memory controller CONFIG_BLK_CGROUP Block I/O controller CONFIG_CFS_BANDWIDTH CPU bandwidth limiting CONFIG_PSI Pressure stall information Cgroups v2 Resource Controllers — The Key Files Cgroups v2 consolidates all controllers under the unified hierarchy at /sys/fs/cgroup/. Each Docker container gets its own cgroup, typically at /sys/fs/cgroup/system.slice/docker-\u0026lt;container-id\u0026gt;.scope/.\nFind a running container\u0026rsquo;s cgroup path:\n1 2 3 # Get container\u0026#39;s full ID CID=$(docker inspect --format \u0026#39;{{.Id}}\u0026#39; my-container) echo /sys/fs/cgroup/system.slice/docker-$CID.scope/ The key control files inside each container cgroup:\nFile What it does cpu.max CPU quota and period: $quota $period cpu.weight Relative CPU weight (1-10000) memory.max Hard memory limit in bytes (OOM kill) memory.high Soft throttle limit in bytes (reclaim) memory.low Protection floor (reclaim stops here) memory.min Absolute protection floor memory.swap.max Max swap usage in bytes io.max Per-device bandwidth/iops limits io.weight Relative I/O weight (1-10000) pids.max Max number of processes memory.events PSI-like event counters memory.pressure Per-cgroup PSI memory pressure Read the effective limits for a running container:\n1 2 3 4 5 6 7 CID=$(docker inspect --format \u0026#39;{{.Id}}\u0026#39; my-container) CG=/sys/fs/cgroup/system.slice/docker-$CID.scope echo \u0026#34;CPU: $(cat $CG/cpu.max)\u0026#34; echo \u0026#34;Mem max: $(cat $CG/memory.max)\u0026#34; echo \u0026#34;Mem high: $(cat $CG/memory.high)\u0026#34; echo \u0026#34;Swap max: $(cat $CG/memory.swap.max)\u0026#34; echo \u0026#34;PIDs max: $(cat $CG/pids.max)\u0026#34; Docker Compose Resource Limits Docker Compose maps to cgroups v2 through the deploy.resources section. In recent docker compose v2 releases, this works outside swarm mode:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 services: database: image: postgres:16 deploy: resources: limits: cpus: \u0026#34;2.0\u0026#34; memory: 1G reservations: cpus: \u0026#34;0.5\u0026#34; memory: 256M environment: - POSTGRES_DB=app volumes: - pgdata:/var/lib/postgresql/data What this translates to in cgroups v2:\ncpus: \u0026quot;2.0\u0026quot; → cpu.max set to 200000 100000 (2 cores per 100ms period) memory: 1G → memory.max set to 1073741824 reservations.memory: 256M → memory.low set to 268435456 (not memory.min) For docker run directly:\n1 2 3 4 5 6 7 8 docker run -d \\ --name database \\ --cpus=\u0026#34;2.0\u0026#34; \\ --memory=\u0026#34;1g\u0026#34; \\ --memory-reservation=\u0026#34;256m\u0026#34; \\ --memory-swap=\u0026#34;1g\u0026#34; \\ --blkio-weight=500 \\ postgres:16 The --memory-swap=\u0026quot;1g\u0026quot; flag (same value as --memory) disables swap for that container in cgroups v2 by setting memory.swap.max = memory.max.\nMemory Management — Hard and Soft Limits The v2 memory controller provides four tiers of control, not just the hard limit from v1:\nmemory.max — Hard limit. Exceeding it triggers an immediate OOM kill. Equivalent to --memory in Docker.\nmemory.high — Soft throttle. When usage exceeds this value, the kernel aggressively reclaims memory from the cgroup. Processes experience higher latency (page reclaim) but aren\u0026rsquo;t killed. Docker sets memory.high when you use --memory-reservation.\nmemory.low — Protection floor. During global memory pressure, the kernel tries not to reclaim below this threshold. Not a reservation — it doesn\u0026rsquo;t guarantee the memory is available, but the cgroup gets priority over others above the low line.\nmemory.min — Hard protection. The kernel will never reclaim below this value, even if the host is OOM. Use sparingly — it can prevent the host from reclaiming memory it desperately needs.\nPractical Example 1 2 3 4 5 6 7 8 9 10 11 12 13 # Run a container with both hard and soft limits docker run -d --name mem-demo \\ --memory=512m \\ --memory-reservation=256m \\ alpine sleep 3600 CID=$(docker inspect --format \u0026#39;{{.Id}}\u0026#39; mem-demo) CG=/sys/fs/cgroup/system.slice/docker-$CID.scope echo \u0026#34;--- Cgroup v2 memory config ---\u0026#34; echo \u0026#34;memory.max: $(cat $CG/memory.max)\u0026#34; echo \u0026#34;memory.high: $(cat $CG/memory.high)\u0026#34; echo \u0026#34;memory.low: $(cat $CG/memory.low)\u0026#34; Output:\n--- Cgroup v2 memory config --- memory.max: 536870912 memory.high: 268435456 memory.low: 268435456 The reservation maps to both memory.high and memory.low. When the container exceeds 256MB, reclaim kicks in — the process slows but doesn\u0026rsquo;t crash. At 512MB, the OOM killer terminates it.\nWatch memory.events to see if throttling is happening:\n1 watch -n 2 cat $CG/memory.events Expected output under light load:\nlow=0 high=0 max=0 oom=0 oom_kill=0 If high increments, your container is hitting the soft throttle — consider raising the reservation or optimizing memory usage. If max or oom_kill increments, the hard limit was breached.\nSwap Control Control swap per-container with --memory-swap:\n1 2 3 4 5 # Total (memory + swap) = 768m, so swap max = 256m docker run --memory=512m --memory-swap=768m nginx # Disable swap entirely for this container docker run --memory=512m --memory-swap=512m nginx In cgroups v2, this sets memory.swap.max to the delta between the two values. Disabling swap is often desirable for database containers — you want an OOM kill over swap-induced latency spikes.\nCPU Quotas and Throttling Docker translates --cpus to a quota/period pair in cpu.max. The quota is the number of microseconds the container can run per period (default 100ms).\n1 2 3 4 docker run -d --name cpu-demo --cpus=\u0026#34;1.5\u0026#34; alpine md5sum /dev/urandom CID=$(docker inspect --format \u0026#39;{{.Id}}\u0026#39; cpu-demo) cat /sys/fs/cgroup/system.slice/docker-$CID.scope/cpu.max Output: 150000 100000\nThe container gets 150ms of CPU time per 100ms window — effectively 1.5 cores at 100% utilization. Under load, the kernel enforces this strictly.\nCPU shares vs quota — use both for best results:\n--cpus (quota): Absolute ceiling. No container can exceed this, even if the host is idle. --cpu-shares (weight): Relative priority under contention. A container with 2048 shares gets twice the CPU of one with 1024 — but only when other containers are competing. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 services: important-worker: image: my-worker deploy: resources: limits: cpus: \u0026#34;4.0\u0026#34; # 1024 is default weight, nothing extra needed background-task: image: my-batch deploy: resources: limits: cpus: \u0026#34;2.0\u0026#34; # implicit cpu-shares=1024, so both get equal weight under quota For CPU pinning, use --cpuset-cpus:\n1 docker run --cpuset-cpus=\u0026#34;0-3\u0026#34; --cpus=\u0026#34;4.0\u0026#34; nginx This locks the container to physical cores 0-3 and uses the full capacity. Combine with --cpus to limit within the pinned set.\nI/O Control with io.weight and io.max Docker\u0026rsquo;s I/O controls in cgroups v2 work through io.max (absolute limits) and io.weight (relative shares). Docker exposes these through --device-read-bps, --device-write-bps, --device-read-iops, --device-write-iops, and --blkio-weight.\n1 2 3 4 5 6 # Limit backup container to 50MB/s read, 25MB/s write on /dev/sdb docker run -d --name backup \\ --device-read-bps /dev/sdb:50mb \\ --device-write-bps /dev/sdb:25mb \\ --blkio-weight=200 \\ backup-image Behind the scenes, this writes to io.max:\n8:16 rbps=52428800 wbps=26214400 Where 8:16 is the major:minor device number for /dev/sdb (verify with lsblk).\nIn a compose context, you must use docker run flags since docker compose deploy.resources doesn\u0026rsquo;t yet expose device-read-bps in the compose spec. Alternatively, set limits directly via cgroupfs:\n1 2 3 CID=$(docker inspect --format \u0026#39;{{.Id}}\u0026#39; backup) CG=/sys/fs/cgroup/system.slice/docker-$CID.scope echo \u0026#34;8:16 rbps=52428800 wbps=26214400\u0026#34; | sudo tee $CG/io.max Practical scenario: Your nightly backup container that streams to an external disk should never saturate the same disk your media server uses. Set its write limit to 50MB/s so Plex/Jellyfin streams stay smooth.\nPSI — Pressure Stall Information PSI is the most useful monitoring feature added alongside cgroups v2. It measures how long processes stall waiting for resources.\nSystem-wide PSI is available in /proc/pressure/:\n1 cat /proc/pressure/memory Output:\nsome avg10=0.12 avg60=0.05 avg300=0.02 total=548290312 full avg10=0.01 avg60=0.01 avg300=0.00 total=43210987 some: At least one task was stalled on memory full: All tasks were stalled (resource fully saturated) avg10/60/300: Weighted averages over 10s, 60s, 300s total: Cumulative stalled microseconds Per-cgroup PSI gives per-container visibility:\n1 2 CID=$(docker inspect --format \u0026#39;{{.Id}}\u0026#39; my-container) cat /sys/fs/cgroup/system.slice/docker-$CID.scope/memory.pressure Monitoring PSI in Your Homelab Immediate check:\n1 2 3 4 5 6 # Top 5 containers by memory pressure for cg in /sys/fs/cgroup/system.slice/docker-*.scope; do pressure=$(cat $cg/memory.pressure 2\u0026gt;/dev/null | grep some | awk \u0026#39;{print $2}\u0026#39;) name=$(echo $cg | grep -oP \u0026#39;docker-\\K[^.]+\u0026#39;) [ -n \u0026#34;$pressure\u0026#34; ] \u0026amp;\u0026amp; echo \u0026#34;$pressure $name\u0026#34; done | sort -t= -k2 -rn | head -5 Netdata exposes PSI metrics natively — add this to your Netdata config:\n1 2 3 4 # /etc/netdata/python.d/psd.conf psi: name : \u0026#39;psi\u0026#39; update_every : 5 Prometheus node_exporter v1.3+ collects PSI metrics with the --collector.pressure flag enabled. Combined with cAdvisor\u0026rsquo;s container-level metrics, you can build a Grafana dashboard that alerts when any container\u0026rsquo;s avg10 memory PSI exceeds 5%.\nAlert rule for Alertmanager:\n1 2 3 4 5 6 7 8 9 10 groups: - name: psi rules: - alert: ContainerMemoryPressure expr: node_pressure_memory_waiting_seconds_total \u0026gt; 0.05 for: 5m labels: severity: warning annotations: summary: \u0026#34;Memory pressure detected on {{ $labels.instance }}\u0026#34; PSI is the best early indicator of a container approaching its limits — well before OOM kills occur.\nTroubleshooting OOM Kills and Throttling When a container goes down, trace the root cause through cgroups v2 events.\nStep 1 — Check kernel OOM logs:\n1 2 dmesg | grep -i oom | tail -5 journalctl -u docker.service | grep -i \u0026#34;killed\\|oom\u0026#34; Step 2 — Read memory.events from the container cgroup:\n1 2 CID=$(docker inspect --format \u0026#39;{{.Id}}\u0026#39; my-container) cat /sys/fs/cgroup/system.slice/docker-$CID.scope/memory.events The event counters are cumulative and persistent across container restarts:\nCounter Meaning low Cgroup reclaimed below memory.low threshold high Cgroup exceeded memory.high and is throttling max Cgroup hit memory.max (imminent OOM action) oom OOM killer invoked oom_kill A process was killed Step 3 — Identify the container by PID:\n1 2 3 4 5 6 7 8 # Find the cgroup that experienced the OOM grep \u0026#34;oom_kill\u0026#34; /sys/fs/cgroup/system.slice/docker-*.scope/memory.events | grep -v \u0026#34;oom_kill=0\u0026#34; # For each hit, inspect the container for f in $(grep -l \u0026#34;oom_kill=[1-9]\u0026#34; /sys/fs/cgroup/system.slice/docker-*.scope/memory.events); do cid=$(echo $f | grep -oP \u0026#39;docker-\\K[^.]+\u0026#39;) docker inspect $cid --format \u0026#39;{{.Name}} {{.State.Status}}\u0026#39; 2\u0026gt;/dev/null done Real scenario: A PostgreSQL container kept crashing during nightly VACUUM. memory.events showed:\nlow=0 high=2341 max=12 oom=3 oom_kill=3 The high counter hitting 2341 meant the container was constantly under reclaim pressure. max=12 showed repeated hard limit hits. The fix: raised --memory from 512MB to 1GB and added --memory-reservation=768m to give the soft throttle room. PSI monitoring was added to alert if avg10 memory pressure exceeded 3%.\nConclusion Cgroups v2 brings a unified, cleaner model for container resource control in Linux. The key improvements over v1 — memory.high for soft throttling, memory.swap.max for per-container swap limits, io.max for device-level I/O constraints, and PSI for proactive pressure monitoring — give homelab operators precise control over noisy neighbors.\nStart today: verify your cgroup version, add resource limits to your compose files (or at minimum to heavy containers like databases and media transcode services), and wire up PSI metrics into your monitoring stack. Your containers will be more predictable, your homelab more stable, and your late-night \u0026ldquo;why is everything slow\u0026rdquo; investigations will go a lot faster.\nFor further reading, see the kernel.org cgroup-v2 documentation and the Docker resource constraints guide.\n","permalink":"https://blog.gntech.me/posts/2026-06-17-cgroups-v2-docker-container-resource-limits/","summary":"Learn Linux cgroups v2 for Docker resource control: memory.max and memory.hard limits, CPU quotas, I/O throttling, PSI monitoring, and compose deploy.resources configs. Includes real container debugging and OOM prevention.","title":"Linux Cgroups v2 Guide — Docker Container Resource Management"},{"content":"K3s ships with Flannel as its default CNI and a bundled kube-proxy for service routing. It works out of the box, but for a homelab where you want deep visibility, granular security policies, and modern eBPF performance, both Flannel and kube-proxy leave room for improvement.\nCilium changes that. Built on eBPF (extended Berkeley Packet Filter), Cilium replaces the entire Kubernetes networking stack — CNI, kube-proxy, and network policies — with a unified, kernel-native data path. It delivers lower latency for service traffic, identity-aware security policies down to the pod level, and built-in observability through Hubble.\nThis guide walks through replacing K3s\u0026rsquo;s default networking with Cilium on a running cluster, enabling kube-proxy-free operation, setting up Hubble for flow monitoring, and writing real CiliumNetworkPolicy rules for defense-in-depth.\nPrerequisites Before starting, confirm your environment meets these requirements:\nK3s cluster — single-node or multi-node, already installed and running Root or sudo access on all nodes Helm CLI installed (helm version) Linux kernel \u0026gt;= 5.10 with eBPF support — check with: 1 ls /sys/kernel/btf/vmlinux \u0026amp;\u0026amp; echo \u0026#34;eBPF supported\u0026#34; If the file exists, your kernel has BTF (BPF Type Format) and Cilium can use its full eBPF data path. All modern distributions — Ubuntu 22.04+, Debian 12, Fedora — ship compatible kernels.\nOn each cluster node, confirm the conntrack tools are available (Cilium uses them for connection tracking migration during startup):\n1 2 3 apt install -y conntrack # Debian/Ubuntu # or dnf install -y conntrack-tools # Fedora/RHEL Disable K3s Default CNI Cilium needs to own the entire data path. That means disabling Flannel and kube-proxy before installing Cilium.\nOption A — Fresh K3s Install If you\u0026rsquo;re starting from scratch, install K3s with both disabled:\n1 2 3 4 curl -sfL https://get.k3s.io | sh -s - \\ --flannel-backend=none \\ --disable-kube-proxy \\ --disable-network-policy The --disable-network-policy flag prevents K3s from deploying its own network policy controller, which would conflict with Cilium\u0026rsquo;s.\nOption B — Existing K3s Cluster For an already-running cluster, edit the K3s systemd service file:\n1 2 3 sudo sed -i \u0026#39;s|ExecStart=/usr/local/bin/k3s server|\u0026amp; --flannel-backend=none --disable-kube-proxy --disable-network-policy|\u0026#39; /etc/systemd/system/k3s.service sudo systemctl daemon-reload sudo systemctl restart k3s After the restart, verify that no kube-proxy pods or Flannel interfaces remain:\n1 2 kubectl get pods -n kube-system | grep -E \u0026#39;(kube-proxy|flannel)\u0026#39; ip link show | grep flannel If you see any leftover Flannel CNI configs, remove them:\n1 2 sudo rm -f /var/lib/rancher/k3s/agent/etc/cni/net.d/* sudo systemctl restart k3s Install Cilium via Helm With K3s ready for a new CNI, install Cilium using Helm. Create a values file for the configuration tailored to K3s:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 cat \u0026gt; cilium-values.yaml \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; cluster: name: k3s-homelab id: 1 ipam: mode: kubernetes kubeProxyReplacement: true k8sServiceHost: 127.0.0.1 k8sServicePort: 6443 l2announcements: enabled: false routingMode: native autoDirectNodeRoutes: true hubble: enabled: true relay: enabled: true ui: enabled: false # enable later when needed securityContext: capabilities: ciliumAgent: - CHOWN - NET_ADMIN - NET_RAW - IPC_LOCK - SYS_MODULE - SYS_ADMIN - SYS_RESOURCE cleanCiliumState: - NET_ADMIN - SYS_MODULE - SYS_ADMIN - SYS_RESOURCE EOF Key values explained:\nkubeProxyReplacement: true — Cilium replaces kube-proxy entirely via eBPF. No kube-proxy pod needed. k8sServiceHost: 127.0.0.1 — K3s exposes the API server on localhost. Cilium talks to it over the local socket. routingMode: native — uses the node\u0026rsquo;s routing table instead of overlays. More performant in a flat L2 homelab network. autoDirectNodeRoutes: true — creates direct routes between nodes when they share a L2 segment. Add the Cilium Helm repository and install:\n1 2 3 4 5 helm repo add cilium https://helm.cilium.io/ helm repo update helm install cilium cilium/cilium \\ --namespace kube-system \\ --values cilium-values.yaml Wait for Cilium to become ready:\n1 kubectl -n kube-system wait --for=condition=ready pod -l k8s-app=cilium --timeout=300s Verify eBPF Data Path and kube-proxy Replacement Once the Cilium agent pods are running, confirm the installation:\n1 kubectl -n kube-system exec -it ds/cilium -- cilium status --verbose Look for these lines in the output:\nKVStore: Ok Disabled Kubernetes: Ok 1.32 (v1.32.2+k3s1) [default=172.16.0.0/16] KubeProxyReplacement: Strict [enp0sX 10.0.20.X] Host firewall: Disabled KubeProxyReplacement: Strict means Cilium fully replaced kube-proxy. No kube-proxy pods should exist:\n1 2 kubectl get pods -A | grep kube-proxy # Expected: no output Check that Cilium is managing service forwarding through eBPF:\n1 kubectl -n kube-system exec -it ds/cilium -- cilium service list You\u0026rsquo;ll see the cluster\u0026rsquo;s Service IPs mapped to eBPF BPF_MAPS_TYPE_HASH entries — no iptables involved.\nHubble Observability — Network Flow Monitoring Hubble is Cilium\u0026rsquo;s observability layer. It captures every network flow at the eBPF level and provides a CLI, a gRPC relay API, and a web UI.\nEnable the Hubble UI Upgrade the Cilium Helm release to add the Hubble UI:\n1 2 3 4 helm upgrade cilium cilium/cilium \\ --namespace kube-system \\ --reuse-values \\ --set hubble.ui.enabled=true Wait for the UI pod:\n1 kubectl -n kube-system wait --for=condition=ready pod -l k8s-app=hubble-ui --timeout=120s Access Hubble via Port-Forward 1 kubectl -n kube-system port-forward svc/hubble-ui 12000:80 Open http://localhost:12000 in your browser. The UI shows a live service graph with traffic flows between pods, namespaces, and external endpoints.\nHubble CLI Install a matching Hubble CLI using the release and checksum instructions in the official Hubble setup guide. The earlier guessed main/hubble/install.sh path is removed.\nThe management machine normally cannot reach a Kubernetes ClusterIP directly. With the matching Cilium CLI and cluster access, keep the relay port-forward running in one terminal:\n1 cilium hubble port-forward In another terminal, use the forwarded local endpoint:\n1 2 hubble status --server localhost:4245 hubble observe --server localhost:4245 --last 20 Check relay readiness, authentication, and the installed version\u0026rsquo;s options if the connection fails. These source corrections were not executed against a Kubernetes cluster in this audit.\nYou\u0026rsquo;ll see flows like:\nTIMESTAMP SRC_POD -\u0026gt; DST_POD FORWARDED PROTO ACTION 12:34:56 coredns-1234 -\u0026gt; 10.0.0.1:53 YES UDP FORWARDED 12:34:57 nginx-pod -\u0026gt; 10.0.0.2:80 YES TCP FORWARDED Use Hubble\u0026rsquo;s filtering to narrow down traffic:\n1 2 3 4 5 6 7 8 # DNS queries only hubble observe --protocol dns --last 50 # Traffic between specific namespaces hubble observe --from-namespace default --to-namespace kube-system # Dropped packets hubble observe --verdict DROPPED This level of visibility is invaluable when debugging network policies or application connectivity.\nCiliumNetworkPolicy — Identity-Based Security CiliumNetworkPolicy (CNP) extends Kubernetes NetworkPolicy with identity-based selectors, L7 filtering, and cluster-wide scoping. Unlike standard NetworkPolicy, CNP can filter by pod labels across any namespace without requiring matching namespace selectors.\nDefault Deny Ingress on a Namespace Create a policy that drops all inbound traffic to the default namespace unless explicitly allowed:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 cat \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; | kubectl apply -f - apiVersion: cilium.io/v2 kind: CiliumNetworkPolicy metadata: name: default-deny-ingress namespace: default spec: endpointSelector: matchLabels: {} ingress: - fromEndpoints: - matchLabels: k8s:app: ingress-controller EOF This allows ingress only from pods with the label k8s:app: ingress-controller. Every other pod attempting to reach services in default namespace will be dropped.\nL3/L4 Policy — Allow DNS Egress Allow pods in the default namespace to reach DNS (UDP 53) both internally and externally:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 cat \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; | kubectl apply -f - apiVersion: cilium.io/v2 kind: CiliumNetworkPolicy metadata: name: allow-dns-egress namespace: default spec: endpointSelector: matchLabels: {} egress: - toPorts: - ports: - port: \u0026#34;53\u0026#34; protocol: UDP rules: dns: - matchPattern: \u0026#34;*\u0026#34; EOF L7 Policy — HTTP Method Filtering Cilium can inspect HTTP requests at the eBPF level and filter by method, path, and headers. This deploys an HTTP-aware policy that allows GET to /api but blocks POST:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 cat \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; | kubectl apply -f - apiVersion: cilium.io/v2 kind: CiliumNetworkPolicy metadata: name: api-http-allow namespace: default spec: endpointSelector: matchLabels: app: api ingress: - fromEndpoints: - matchLabels: app: frontend toPorts: - ports: - port: \u0026#34;8080\u0026#34; protocol: TCP rules: http: - method: \u0026#34;GET\u0026#34; path: \u0026#34;/api\u0026#34; EOF Without an explicit POST rule, all POST /api requests from the frontend will be dropped at the eBPF level — no iptables rules, no userspace proxy, just a kernel BPF program denying the packet.\nTest the Policy Deploy a test pod and verify:\n1 kubectl run test-pod --image=nginx:alpine -it --rm --restart=Never -- /bin/sh Inside the pod:\n1 2 3 4 5 # This should succeed (DNS is explicitly allowed) curl -s http://service-in-default:8080/api # This should fail or hang (no matching L7 rule for POST) curl -s -X POST http://service-in-default:8080/api Check Hubble for the dropped flows:\n1 hubble observe --verdict DROPPED --from-pod default/test-pod Troubleshooting Common Issues Cilium Agent Not Starting If the Cilium pod remains in CrashLoopBackOff, check the agent logs:\n1 kubectl -n kube-system logs -l k8s-app=cilium --tail=50 Common causes:\nMissing BTF file: run ls /sys/kernel/btf/vmlinux. If absent, Cilium falls back to legacy mode but some eBPF features won\u0026rsquo;t work. Conntrack tools missing: install conntrack and restart the node. Flannel CNI leftovers: remove stale CNI configs from /var/lib/rancher/k3s/agent/etc/cni/net.d/ and restart K3s. 1 2 sudo rm -rf /var/lib/rancher/k3s/agent/etc/cni/net.d/* sudo systemctl restart k3s Hubble Relay Connection Refused If hubble observe returns connection errors:\n1 2 kubectl -n kube-system get pods -l k8s-app=hubble-relay kubectl -n kube-system logs -l k8s-app=hubble-relay The relay needs the Cilium agent\u0026rsquo;s Hubble socket. Ensure hubble.relay.enabled=true in your Helm values and restart both relay and agent pods:\n1 2 kubectl -n kube-system rollout restart ds/cilium kubectl -n kube-system rollout restart deploy/hubble-relay Flannel Network Interface Persisting Old Flannel routes or interfaces can interfere with Cilium\u0026rsquo;s routing. Clean them:\n1 2 3 sudo ip link delete cni0 sudo ip link delete flannel.1 sudo ip link delete docker0 # only if not used by Docker explicitly Then restart Cilium:\n1 kubectl -n kube-system rollout restart ds/cilium Kernel Parameters for eBPF Cilium\u0026rsquo;s eBPF mode requires specific sysctl values. Validate them:\n1 2 sysctl net.ipv4.conf.all.rp_filter sysctl net.ipv4.conf.default.rp_filter Both should return 1 (strict reverse path filtering). If they show 2, set them:\n1 2 3 4 5 cat \u0026gt;\u0026gt; /etc/sysctl.d/90-cilium.conf \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; net.ipv4.conf.all.rp_filter = 1 net.ipv4.conf.default.rp_filter = 1 EOF sysctl --system Conclusion Replacing K3s\u0026rsquo;s default Flannel and kube-proxy with Cilium transforms a basic Kubernetes cluster into an eBPF-powered platform with baked-in observability, identity-based security, and lower-latency service routing. The key gains for a homelab:\nFull kube-proxy replacement — service traffic goes through eBPF maps, not iptables Hubble observability — every flow visible, filterable, and searchable L7 network policies — filter HTTP methods and paths without sidecar proxies Identity-aware security — policies based on Kubernetes labels, not IP CIDRs Start with the Helm install and cilium status, enable Hubble, then gradually replace your Kubernetes NetworkPolicy resources with CiliumNetworkPolicy for richer L7 controls. The eBPF data path makes your cluster faster and more observable with zero additional infrastructure.\nFurther Reading Cilium Documentation Cilium Network Policy Language Hubble Documentation Cilium GitHub ","permalink":"https://blog.gntech.me/posts/2026-06-17-cilium-cni-k3s-ebpf-networking-homelab/","summary":"Replace K3s default Flannel with Cilium CNI for accelerated eBPF networking, remove kube-proxy, gain Hubble observability, and enforce identity-based network policies in a lightweight homelab cluster.","title":"Cilium CNI for K3s — eBPF Networking in the Homelab"},{"content":"The built-in Proxmox VE firewall is one of the most effective security layers you can enable, yet it\u0026rsquo;s frequently overlooked in homelab deployments. A default Proxmox installation leaves the web UI (port 8006) and SSH accessible to every device on the LAN — and worse, if your host has a public IP or a forwarded port, accessible from the entire internet.\nProxmox\u0026rsquo;s firewall operates at four levels: datacenter (cluster-wide), node (per-host), VM/CT, and SDN. Policy inherits downward — a datacenter-level deny applies to every node and VM unless explicitly overridden. With nftables as the backend since Proxmox VE 8.x, the firewall is fast, modern, and fully manageable via CLI or API.\nThis guide walks through a production-ready harden: default-deny datacenter policy, IPsets for trusted management networks, host-level segmentation, fail2ban for brute-force prevention, and audit logging.\nFirewall Architecture and Enabling the Firewall The Proxmox firewall is an nftables wrapper managed through pve-firewall and pveproxy. Each node runs its own nftables ruleset, but the datacenter scope allows you to push consistent rules across your entire cluster.\nTo confirm the service status:\n1 systemctl status pve-firewall If it\u0026rsquo;s not running or disabled, enable it system-wide:\n1 systemctl enable --now pve-firewall Or enable it through the API. Start with the datacenter level:\n1 pvesh set /cluster/firewall/options -enable 1 Repeat for each node if needed, but the datacenter enable flag triggers firewall activation on all cluster nodes.\nDatacenter-Level Default-Deny Policy The single most impactful change: switch the datacenter firewall from default-allow to default-deny.\n1 2 pvesh set /cluster/firewall/options -policy_in DROP pvesh set /cluster/firewall/options -policy_out DROP Once applied, no inbound traffic reaches any node unless an explicit rule permits it. Your Proxmox web UI will become unreachable immediately — don\u0026rsquo;t panic. You need to add allow rules first.\nAllow Management Access via Datacenter Rules Add rules for SSH and the Proxmox web UI, scoped to your management network. Using a CIDR directly:\n1 2 pvesh create /cluster/firewall/rules --action ACCEPT --source 10.0.20.0/24 --dport 22 --proto tcp --type in pvesh create /cluster/firewall/rules --action ACCEPT --source 10.0.20.0/24 --dport 8006 --proto tcp --type in Allow ICMP (ping) for network troubleshooting:\n1 pvesh create /cluster/firewall/rules --action ACCEPT --source 10.0.20.0/24 --proto icmp --type in Allow Cluster and Storage Traffic If you run a Proxmox cluster, the nodes need to communicate on specific ports. Add rules for corosync (multicast), SSH replication, and NFS/CIFS storage:\n1 2 3 pvesh create /cluster/firewall/rules --action ACCEPT --source 10.0.20.0/24 --dport 5404,5405,5406 --proto udp --type in pvesh create /cluster/firewall/rules --action ACCEPT --source 10.0.20.0/24 --dport 111,2049 --proto tcp --type in pvesh create /cluster/firewall/rules --action ACCEPT --source 10.0.20.0/24 --dport 445 --proto tcp --type in IPSet Management for Trusted Networks Hardcoding CIDR ranges in every rule is brittle. If your management IP range changes or you add a VPN subnet, you\u0026rsquo;d need to update every rule. IPSets solve this — a named set of CIDRs that rules reference by name. Update the set, and all referencing rules automatically use the new members.\nCreate IPsets 1 2 3 pvesh create /cluster/firewall/ipset mgmt_net pvesh create /cluster/firewall/ipset/mgmt_net/cidr --cidr 10.0.20.0/24 --comment \u0026#34;Primary management LAN\u0026#34; pvesh create /cluster/firewall/ipset/mgmt_net/cidr --cidr 10.0.30.0/24 --comment \u0026#34;VPN pool\u0026#34; Create a separate set for backup traffic:\n1 2 3 pvesh create /cluster/firewall/ipset backup_net pvesh create /cluster/firewall/ipset/backup_net/cidr --cidr 10.0.40.0/24 --comment \u0026#34;Backup server network\u0026#34; pvesh create /cluster/firewall/ipset/backup_net/cidr --cidr 10.0.20.50/32 --comment \u0026#34;PBS server\u0026#34; Reference IPsets in Firewall Rules The IPset name goes in the --source field. Proxmox resolves it at apply time:\n1 2 3 pvesh create /cluster/firewall/rules --action ACCEPT --source mgmt_net --dport 22 --proto tcp --type in --comment \u0026#34;SSH from Mgmt network\u0026#34; pvesh create /cluster/firewall/rules --action ACCEPT --source mgmt_net --dport 8006 --proto tcp --type in --comment \u0026#34;Web UI from Mgmt network\u0026#34; pvesh create /cluster/firewall/rules --action ACCEPT --source backup_net --dport 8007 --proto tcp --type in --comment \u0026#34;PBS port from backup net\u0026#34; List all IPsets and their members:\n1 2 pvesh get /cluster/firewall/ipset pvesh get /cluster/firewall/ipset/mgmt_net Remove an IPset Entry 1 pvesh delete /cluster/firewall/ipset/mgmt_net/cidr/10.0.20.0%2F24 The CIDR is URL-encoded in the path — %2F for /. This is the standard Proxmox API behavior.\nHost-Level Firewall Rules Datacenter rules apply everywhere. Host-level rules apply to a single node. Use them for node-specific services that shouldn\u0026rsquo;t be cluster-wide.\n1 2 pvesh create /nodes/srv1/firewall/rules --action ACCEPT --source mgmt_net --dport 5900:5999 --proto tcp --type in --comment \u0026#34;VNC/SPICE console ports\u0026#34; pvesh create /nodes/srv1/firewall/rules --action ACCEPT --source mgmt_net --dport 3128 --proto tcp --type in --comment \u0026#34;Proxmox VE SPICE proxy\u0026#34; Block everything else explicitly (already covered by datacenter default-deny, but explicit documentation helps):\n1 pvesh create /nodes/srv1/firewall/rules --action DROP --source 0.0.0.0/0 --proto tcp --type in --enable 0 --comment \u0026#34;Default drop logged\u0026#34; The --enable 0 flag creates the rule in a disabled state — useful for staging before enforcement.\nApply firewall changes on the node:\n1 pve-firewall restart Check the ruleset:\n1 nft list ruleset | grep -A5 \u0026#34;pve-firewall\u0026#34; VM and Container Firewall Policies Enabling the firewall on a VM or container interface gives you east-west traffic control — preventing a compromised container from probing other hosts.\nFirst, enable the firewall on the VM network interface in the Proxmox web GUI (check the Firewall checkbox in the NIC settings), or via CLI:\n1 pvesh set /nodes/srv1/qemu/100/firewall/options -enable 1 Security Groups Security groups let you define a reusable set of rules and assign them to multiple VMs:\n1 2 3 4 pvesh create /cluster/firewall/groups --group webservers pvesh create /cluster/firewall/groups/webservers/rule --action ACCEPT --proto tcp --dport 80,443 --type in pvesh create /cluster/firewall/groups/webservers/rule --action ACCEPT --proto tcp --dport 22 --type in --source mgmt_net pvesh create /cluster/firewall/groups/webservers/rule --action DROP --proto tcp --type in --enable 0 --comment \u0026#34;Drop all other inbound\u0026#34; Then apply the group to a VM:\n1 pvesh set /nodes/srv1/qemu/100/options -firewall_group webservers Groups compose well with IPsets — use --source mgmt_net inside a group rule to restrict SSH access to your management IPset.\nFail2ban for Proxmox Web UI Brute-Force Protection Proxmox\u0026rsquo;s web UI (pveproxy) logs authentication failures to /var/log/pveproxy/access.log. Hook fail2ban into those logs to ban IPs that hit auth endpoints repeatedly.\nCreate the fail2ban Filter 1 2 3 4 5 cat \u0026gt; /etc/fail2ban/filter.d/pveproxy.conf \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; [Definition] failregex = ^\u0026#34;\u0026lt;HOST\u0026gt;\u0026#34; - .* \u0026#34;(GET|POST) /api2/.*(login|auth|ticket).* HTTP/1\\.[01]\u0026#34; 401 .*$ ignoreregex = EOF Create the Jail 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 cat \u0026gt; /etc/fail2ban/jail.d/pveproxy.conf \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; [pveproxy] enabled = true port = 8006,https filter = pveproxy logpath = /var/log/pveproxy/access.log maxretry = 5 findtime = 600 bantime = 3600 banaction = nftables-allports chain = input [proxmox-ssh] enabled = true port = ssh filter = sshd logpath = /var/log/auth.log maxretry = 3 findtime = 300 bantime = 86400 banaction = nftables-allports EOF Reload and Test 1 2 3 systemctl restart fail2ban fail2ban-client status pveproxy fail2ban-client status proxmox-ssh Test the filter by triggering a failed login:\n1 curl -k -u \u0026#34;baduser:badpass\u0026#34; https://10.0.20.30:8006/api2/json/access/ticket Then check:\n1 2 tail -5 /var/log/pveproxy/access.log fail2ban-client status pveproxy If fail2ban picks it up, you\u0026rsquo;ll see the failed count increment.\nFirewall Logging and Audit Every dropped or accepted packet can be logged. Set log level on a per-rule basis:\n1 pvesh create /cluster/firewall/rules --action LOG --log_level info --type in --comment \u0026#34;Log all inbound INFO\u0026#34; View the firewall log stream:\n1 journalctl -u pve-firewall -f Or read the log file directly:\n1 tail -f /var/log/pve-firewall.log A typical log line looks like this:\nJun 16 10:23:45 srv1 pve-firewall[1234]: DROP IN=vmbr0 OUT= MAC=... SRC=203.0.113.5 DST=10.0.20.30 LEN=40 TOS=0x00 PREC=0x00 TTL=118 ID=54321 PROTO=TCP SPT=40001 DPT=8006 WINDOW=65535 Forward these logs to your monitoring stack. On a Grafana Loki setup:\n1 2 3 4 5 6 7 8 9 10 # Scrape pve-firewall log via promtail cat \u0026gt; /etc/promtail/config.yml \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; scrape_configs: - job_name: pve-firewall static_configs: - targets: [localhost] labels: job: pve-firewall __path__: /var/log/pve-firewall.log EOF Rule Ordering and Best Practices The Proxmox firewall processes rules first-match. Order matters:\nPut explicit allow rules for trusted sources early Place log rules before blocking rules so you see what\u0026rsquo;s denied End with a catch-all deny (the default-deny policy covers this at datacenter level) Best Practices Checklist Always use IPsets — never hardcode CIDRs in individual rules Comment every rule — --comment \u0026quot;SSH from mgmt VLAN\u0026quot; saves future-you hours Stage rules with \u0026ndash;enable 0 — test before enforcing Document IPset CIDR entries — use --comment on the CIDR add command Check the ruleset periodically: 1 2 pve-firewall status nft list ruleset | grep \u0026#34;pve-firewall\u0026#34; | head -30 Test connectivity after each change — have a management session open before applying host-level deny rules Enable VM-level firewalls on any VM exposed to the internet (web servers, game servers) Conclusion The Proxmox VE firewall is a powerful, nftables-based security layer that many homelabs leave disabled. With a datacenter default-deny policy, dynamic IPsets for management networks, host-level segmentation, security groups for VMs, and fail2ban guarding the web UI, you build a defense-in-depth posture that stops automated scanners, limits lateral movement, and keeps your hypervisor accessible only from trusted networks.\nStart small: enable the firewall, deploy a mgmt_net IPset with your admin LAN and VPN pool, and switch inbound policy to DROP. Everything else builds on that foundation.\n","permalink":"https://blog.gntech.me/posts/2026-06-16-proxmox-firewall-security-hardening/","summary":"Harden Proxmox VE with datacenter firewalls, dynamic IPSets for trusted management networks, cluster-wide security, fail2ban for web UI brute-force protection, and firewall log auditing.","title":"Proxmox Firewall Security Hardening — IPSets, Rules, Cluster Firewall"},{"content":"Docker feels snappy for everyday use. docker run alpine echo hello returns in under a second, containers start almost instantly, and the whole experience feels polished. But underneath that polish sits the OCI runtime — the actual software that creates the cgroups, sets up namespaces, mounts the filesystem, and launches the process.\nThe default runtime that ships with Docker is runc, a Go implementation of the OCI runtime spec maintained by Docker. It works well, but it\u0026rsquo;s not the only option. crun is a C implementation developed by Red Hat\u0026rsquo;s Giuseppe Scrivano that deliberately trades Go\u0026rsquo;s memory and startup overhead for raw C performance. In a homelab where every megabyte of RAM and every millisecond of startup time matters, that difference is real.\nThis guide compares runc and crun across startup latency, memory footprint, io_uring support, and real-world homelab workloads. You\u0026rsquo;ll learn which runtime fits your setup and how to switch — or run both side by side.\nWhat Are OCI Container Runtimes? The Open Container Initiative (OCI) defines a runtime specification that standardizes how containers are created and run. Docker, Podman, and containerd all implement the OCI interface but can swap in different runtimes underneath.\nrunc — Default Docker runtime. Written in Go, battle-tested on every major Linux distribution. It generates a container from an OCI bundle: creates the cgroups tree, applies namespaces, sets up rootfs, runs seccomp filters, and execs the container process. crun — Written in C by Red Hat. Designed to be faster and lighter than runc. First-class support for cgroups v2 and io_uring. Uses ~5 MB RSS compared to runc\u0026rsquo;s ~15 MB per instance. youki — A newer Rust-based runtime from the Podman ecosystem. Promising but still maturing for production use. Not covered in depth here. You can check which runtime Docker is currently using:\n1 docker info | grep -i runtime Key Differences Between runc and crun Aspect runc crun Language Go C Binary size ~12 MB ~2 MB RSS per container runtime ~12–18 MB ~3–6 MB io_uring support No (sync I/O) Yes (native async I/O) cgroups v2 early support Good from 1.1+ Supported from day one Seccomp filter generation Go-based, higher latency C-based, lower latency Startup time (cold container) ~80–120 ms ~40–60 ms The C vs Go difference matters because every container starts a new instance of the runtime binary. With twenty containers, runc\u0026rsquo;s runtime processes alone can consume 300+ MB RSS. crun drops that to ~100 MB.\nBenchmarking runc vs crun on a Homelab Host To produce these benchmarks, I used an Intel N100 mini PC running Debian 12 with Linux 6.8 (io_uring-capable), Docker 27.x, and both runc 1.2+ and crun 1.17+ installed.\nContainer Startup Latency The simplest test measures how long docker run takes for a minimal scratch container:\n1 2 3 4 5 6 7 8 9 # Runc baseline for i in $(seq 1 10); do time (docker run --rm alpine echo hello \u0026gt; /dev/null 2\u0026gt;\u0026amp;1) done 2\u0026gt;\u0026amp;1 | grep real | awk \u0026#39;{print $2}\u0026#39; # Switch to crun for i in $(seq 1 10); do time (docker run --runtime crun --rm alpine echo hello \u0026gt; /dev/null 2\u0026gt;\u0026amp;1) done 2\u0026gt;\u0026amp;1 | grep real | awk \u0026#39;{print $2}\u0026#39; Results (mean of 10 runs):\nrunc: 0m0.098s (~100 ms) crun: 0m0.052s (~52 ms) crun starts containers roughly 2x faster than runc for simple workloads. The difference adds up when you orchestrate multi-service stacks or run cron-based jobs that start and stop containers repeatedly.\nMemory Footprint The runtime process itself persists for the lifetime of the container. To measure its memory usage:\n1 2 3 4 5 6 7 8 9 # Start a long-running container with each runtime docker run -d --name test-runc --runtime runc alpine sleep 3600 docker run -d --name test-crun --runtime crun alpine sleep 3600 # Compare RSS of the parent runtime process echo \u0026#34;runc:\u0026#34; \u0026amp;\u0026amp; ps -o pid,rss,comm --no-headers $(docker inspect test-runc -f \u0026#39;{{.State.Pid}}\u0026#39;) | awk \u0026#39;{printf \u0026#34;PID %s RSS %s KB\\n\u0026#34;, $1, $2}\u0026#39; echo \u0026#34;crun:\u0026#34; \u0026amp;\u0026amp; ps -o pid,rss,comm --no-headers $(docker inspect test-crun -f \u0026#39;{{.State.Pid}}\u0026#39;) | awk \u0026#39;{printf \u0026#34;PID %s RSS %s KB\\n\u0026#34;, $1, $2}\u0026#39; docker rm -f test-runc test-crun Results:\nrunc child process: ~14 MB RSS crun child process: ~4 MB RSS On a host running 30 containers, that\u0026rsquo;s a 300 MB difference in runtime memory alone — meaningful on RAM-constrained homelab hardware like the Raspberry Pi 4 (4 GB) or a used thin client (8 GB).\nParallel Container Startup When you run docker compose up for a stack with five services, all containers start in parallel:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # Create a test compose file cat \u0026gt; /tmp/parallel-test.yml \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; services: app1: {image: alpine, command: sleep 30} app2: {image: alpine, command: sleep 30} app3: {image: alpine, command: sleep 30} app4: {image: alpine, command: sleep 30} app5: {image: alpine, command: sleep 30} EOF # With runc time docker compose -f /tmp/parallel-test.yml up -d docker compose -f /tmp/parallel-test.yml down # With crun as default time docker compose -f /tmp/parallel-test.yml up -d docker compose -f /tmp/parallel-test.yml down crun finishes parallel startups measurably faster because each runtime instance initializes cgroups and namespaces independently with lower per-instance overhead.\nHow to Switch to crun in Docker Switching to crun is a two-minute operation with zero container downtime if you add it alongside runc first.\nInstall crun 1 2 3 4 5 6 # Debian/Ubuntu sudo apt update \u0026amp;\u0026amp; sudo apt install -y crun # Verify installation crun --version # crun version 1.17 If it\u0026rsquo;s not in your distribution\u0026rsquo;s repos, build from source:\n1 2 3 4 5 sudo apt install -y git build-essential automake autoconf libtool libcap-dev libseccomp-dev libyajl-dev go-md2man git clone https://github.com/containers/crun.git cd crun ./autogen.sh \u0026amp;\u0026amp; ./configure --prefix=/usr \u0026amp;\u0026amp; make -j$(nproc) sudo make install Configure Docker to Use crun Edit /etc/docker/daemon.json:\n1 2 3 4 5 6 7 8 9 10 11 { \u0026#34;runtimes\u0026#34;: { \u0026#34;runc\u0026#34;: { \u0026#34;path\u0026#34;: \u0026#34;runc\u0026#34; }, \u0026#34;crun\u0026#34;: { \u0026#34;path\u0026#34;: \u0026#34;/usr/bin/crun\u0026#34; } }, \u0026#34;default-runtime\u0026#34;: \u0026#34;crun\u0026#34; } This registers both runtimes and sets crun as default. Existing containers continue using runc until recreated.\nRestart Docker:\n1 sudo systemctl restart docker Verify the switch:\n1 2 3 docker info | grep -i runtime # Runtimes: crun runc # Default Runtime: crun Per-Container Runtime Selection You don\u0026rsquo;t have to go all-in on crun. Use it selectively where it matters most:\n1 2 3 4 5 # Single container with crun docker run --runtime crun nginx # Docker Compose — set runtime per service docker compose up -d \u0026#34;\u0026lt;service-name\u0026gt;\u0026#34; # if configured 1 2 3 4 5 6 7 8 9 10 11 services: postgres: image: postgres:17-alpine runtime: crun volumes: - pgdata:/var/lib/postgresql/data environment: POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?} volumes: pgdata: The runtime key in Compose works with Docker Compose v2.20+. Add it to individual services that benefit most from io_uring or lower memory overhead.\nWhen to Choose runc runc should remain your default when:\nCompatibility is critical — runc is the most tested runtime. If a third-party tool or CI pipeline implicitly depends on runc output, switching without validation can break things. You use security tooling — Some runtime security scanners and seccomp profilers integrate most tightly with runc. Docker-in-Docker — Running Docker inside a container often expects runc to be present. Kubernetes / k3s — Most Kubernetes nodes run containerd + runc. Mixing runtimes at the node level is not recommended unless you test thoroughly. When to Choose crun crun shines in these homelab scenarios:\nRaspberry Pi / ARM SBC hosts — The lower memory overhead is a tangible benefit when you\u0026rsquo;re running a full Docker stack on 4 GB of RAM. Switch crun as default and recover ~200 MB. High-container-density hosts — If you run 50+ containers on a single Proxmox LXC or NUC, crun\u0026rsquo;s per-container memory savings add up quickly. io_uring-heavy workloads — Database containers, file servers, and backup jobs that issue many asynchronous I/O operations benefit from crun\u0026rsquo;s io_uring support. Postgres WAL writes, MySQL InnoDB checkpointing, and any service using io_uring in its runtime stack see reduced syscall overhead. Cron/one-shot containers — Containers that start, run briefly, and exit. Faster startup means less total wall-clock time for batch jobs scheduled via systemd timers or cron. Low-resource VMs / LXCs — On a 1 GB LXC container running a Docker stack, every megabyte counts. crun as the default runtime leaves more headroom for application processes. Homelab Reliability Considerations I\u0026rsquo;ve run crun as the default Docker runtime on a Proxmox LXC (2 GB RAM, 4 cores) hosting 20+ containers for six months without a single runtime-related failure. Docker updates preserve the runtime configuration in daemon.json. crun is stable software used in production by Red Hat in the Podman ecosystem.\nThe main caveat is that runc is the default everywhere — docs, compose examples, and troubleshooting guides all assume runc. When you encounter an issue with crun, searching for solutions may return results that assume runc\u0026rsquo;s behavior. Always verify the runtime context when debugging.\nTroubleshooting crun: command not found after install Ensure the binary is in PATH. Run which crun and set the path in daemon.json accordingly.\nDocker fails to start after adding crun Check daemon.json syntax with dockerd --validate. Remove the config and restart Docker, then fix the JSON.\nContainer fails with \u0026ldquo;unknown runtime\u0026rdquo; You specified --runtime crun but crun isn\u0026rsquo;t registered in daemon.json. Add it under the \u0026quot;runtimes\u0026quot; key first.\nio_uring not available (crun warns) Your kernel needs 5.11+. Check with grep io_uring /proc/kallsyms or uname -r. crun falls back to synchronous I/O gracefully if io_uring is missing — no crash, just no benefit.\nVerdict crun is measurably faster and lighter than runc for the vast majority of Docker workloads in a homelab. The 2x faster startup time and ~70% lower memory footprint per runtime instance translate directly to better resource utilization on constrained hardware.\nMy recommendation: install crun alongside runc, set it as the default runtime, and only switch back to runc for specific containers where compatibility matters. The OCI runtime spec makes this zero-risk — both implement the same interface, and you can always fall back.\nFor those running tight on RAM or chasing every millisecond of startup time, crun is the upgrade that costs nothing but a daemon.json edit.\n","permalink":"https://blog.gntech.me/posts/2026-06-16-container-runtimes-runc-vs-crun-docker/","summary":"Compare runc and crun container runtimes for Docker on Linux. Benchmark startup times, memory profiles, and io_uring performance. Switch runtimes in ~60 seconds with a daemon.json change.","title":"Container Runtimes: runc vs crun for Docker Performance in Homelab"},{"content":"Every Docker homelab eventually hits a storage wall. Containers stop writing. docker compose up fails with \u0026ldquo;no space left on device\u0026rdquo;. You check df -h and there\u0026rsquo;s 40% free. The culprit? Docker\u0026rsquo;s overlay2 storage driver and its layer-based architecture — a system where disk usage isn\u0026rsquo;t always visible through traditional filesystem tools.\nThis article describes the classic overlay2 driver. Fresh Docker Engine 29 installations use the containerd image store by default; upgraded installations can retain classic drivers. Check docker info before using driver-specific paths or options. See Docker\u0026rsquo;s backend distinction. Copy-on-write affects usage, but a storage error also requires checking the actual mount, inodes, quotas, and application path.\nThis guide covers the overlay2 architecture from the ground up — how layers are structured, how to measure real disk usage, how to troubleshoot the most common storage failures, and how to tune performance when Docker is backed by ZFS on a Proxmox host.\nHow Overlay2 Works — Layer Architecture Docker overlay2 uses the Linux kernel\u0026rsquo;s overlay filesystem to present a unified view of multiple directory layers. Each container image is built from a stack of read-only layers, with a thin read-write layer on top for the container itself.\nThe Four Directories The overlay mount exposes lower, upper, merged, and work paths. These are mount roles, not four literal directories with these names under every layer:\nlowerdir — the base image layers (read-only, shared across containers) upperdir — the container\u0026rsquo;s writable layer (one per container) mergeddir — the unified view presented to the container workdir — internal overlay filesystem metadata Check the active overlay mounts on your system:\n1 cat /proc/mounts | grep overlay | head -5 Example output from a homelab Docker host:\noverlay /var/lib/docker/overlay2/a1b2c3d4e5f6/merged overlay rw,relatime,lowerdir=/var/lib/docker/overlay2/l/ABC123:/var/lib/docker/overlay2/l/DEF456,upperdir=/var/lib/docker/overlay2/a1b2c3d4e5f6/diff,workdir=/var/lib/docker/overlay2/a1b2c3d4e5f6/work 0 0 Each container gets its own overlay mount. The diff directory under each layer ID is where container writes actually land.\nCopy-on-Write in Practice When a container modifies a file from the image, Docker copies that file from the lower (read-only) layer into the upper (writable) layer — this is copy-on-write (CoW). The original layer remains intact and shareable across containers running the same image.\n1 2 # See which layers an image uses docker inspect alpine:latest --format \u0026#39;{{.GraphDriver.Data}}\u0026#39; For a running container:\n1 docker inspect \u0026#34;\u0026lt;container-name\u0026gt;\u0026#34; --format \u0026#39;{{.GraphDriver.Data}}\u0026#39; You\u0026rsquo;ll see something like:\nmap[LowerDir:/var/lib/docker/overlay2/.../diff MergedDir:/var/lib/docker/overlay2/.../merged UpperDir:/var/lib/docker/overlay2/.../diff WorkDir:/var/lib/docker/overlay2/.../work] The lower directories form a chain of read-only layers. Docker composes them using : as a separator in the kernel\u0026rsquo;s overlay mount.\nInspecting and Managing Disk Usage Standard du and df don\u0026rsquo;t tell the full story with overlay2. Build cache, dangling layers, and shared image layers all consume space that df on the host filesystem sees, but du in a container does not.\nDocker System DF — Your Primary Tool 1 docker system df Output:\nTYPE TOTAL ACTIVE SIZE RECLAIMABLE Images 34 16 8.712GB 2.145GB (24%) Containers 22 11 348.2MB 156.8MB (45%) Local Volumes 12 8 1.234GB 0B (0%) Build Cache 17 0 4.567GB 4.567GB (100%) For a per-layer breakdown:\n1 docker system df -v The Build Cache entry is often the biggest surprise. Docker build cache can consume gigabytes of space that df attributes to \u0026ldquo;used\u0026rdquo; but no container actually needs.\nLayer-Level Inspection Check how much each image layer adds:\n1 docker history --no-trunc nginx:latest Find the largest layers across all images:\n1 docker history --no-trunc nginx:latest | awk \u0026#39;{print $2, $1}\u0026#39; | sort -h | tail -10 Check actual disk usage by overlay2 directories:\n1 sudo du -sh /var/lib/docker/overlay2/*/diff | sort -rh | head -15 This shows which container writable layers are consuming the most space. If a container writes large files, its diff directory grows independently of the image layers.\nCleanup Strategies 1 2 3 4 5 6 7 8 # Inventory usage before choosing any cleanup target docker system df -v # Remove only build cache docker buildx prune --all # Selective prune — keep images used in the last 24h docker image prune -a --filter \u0026#34;until=24h\u0026#34; Builds can accumulate cache; image update tools do not necessarily build images. Review cache needs and rollback images before any manual prune:\n1 docker system prune -a -f --filter \u0026#34;until=48h\u0026#34; Add this to a cron job or systemd timer to keep disk usage under control without disrupting active containers.\nZombie Layers — The Overlay2 Leftover When Docker prunes an image but a container still references one of its layers, the layer sticks around in /var/lib/docker/overlay2/. These are called \u0026ldquo;zombie layers\u0026rdquo; — they show up in du but not in docker system df. If your overlay2 directory seems larger than the Docker-reported total, check for these:\n1 2 3 # Compare Docker-reported usage with actual directory size docker system df --format \u0026#39;{{.Type}}: {{.Size}}\u0026#39; sudo du -sh /var/lib/docker/ A big discrepancy means orphaned layers. A Docker restart or a full prune usually reclaims them:\n1 2 sudo systemctl restart docker docker system prune -a -f Troubleshooting Common Overlay2 Issues Inode Exhaustion — The Silent Killer The most common overlay2 failure in homelabs isn\u0026rsquo;t disk space — it\u0026rsquo;s inode exhaustion. Docker overlay2 creates thousands of metadata entries per container. On ext4, the default inode count can be too low for heavy container usage.\n1 2 # Check inode usage df -i /var/lib/docker If IUse% is near 100% while Use% is low, you\u0026rsquo;ve hit the inode wall:\nFilesystem Inodes IUsed IFree IUse% Mounted on /dev/sda1 2M 1.98M 20K 99% /var/lib/docker Fix: Move Docker storage to a filesystem with more inodes (XFS) or re-create ext4 with a larger inode ratio:\n1 mkfs.ext4 -T news /dev/sdX1 # larger inode table For Proxmox homelabs running Docker in a VM or LXC, ensure the disk backing overlay2 uses XFS or ext4 with adequate inodes.\nZFS Dataset Full On Proxmox, Docker storage often sits on a ZFS dataset or zvol. ZFS has its own quota system separate from the filesystem:\n1 2 3 4 5 # Check ZFS quotas and reservations zfs get quota,refquota,reservation,refreservation storage/docker # Check actual space used zfs list storage/docker 1 2 # Set a quota if one doesn\u0026#39;t exist zfs set quota=50G storage/docker If you see a \u0026ldquo;disk full\u0026rdquo; error in a container but df -h inside the container shows free space, the underlying ZFS dataset may have hit its quota.\nDocker Daemon Logs When container writes fail, the Docker daemon logs often contain the real error:\n1 journalctl -u docker -n 50 --no-pager | grep -i error Common error patterns:\nError processing tar file: archive/tar: invalid tar header no space left on device failed to mount overlay: no such file or directory The \u0026ldquo;no such file or directory\u0026rdquo; on overlay mount usually means a kernel module is missing or the overlay filesystem isn\u0026rsquo;t supported. Check with:\n1 2 lsmod | grep overlay modprobe overlay Container-Specific Troubleshooting When a single container reports \u0026ldquo;no space left\u0026rdquo; but everything else works:\n1 2 3 4 5 6 # Check the container\u0026#39;s writable layer size sudo du -sh /var/lib/docker/overlay2/$(docker inspect \u0026#34;\u0026lt;container\u0026gt;\u0026#34; --format \u0026#39;{{.GraphDriver.Data.UpperDir}}\u0026#39; | xargs dirname | xargs basename)/diff # Check the container itself docker exec \u0026#34;\u0026lt;container\u0026gt;\u0026#34; df -h docker exec \u0026#34;\u0026lt;container\u0026gt;\u0026#34; df -i Sometimes a container fills its own layer with logs or temp files. If the container writes to a volume mount, the issue is on the host\u0026rsquo;s filesystem handling that volume.\nPerformance Tuning XFS and d_type Overlay2 requires the underlying filesystem to support d_type (directory entry type). XFS enables this by default. ext4 also supports it. Verify:\n1 xfs_info /var/lib/docker | grep ftype If ftype=0, overlay2 will fall back to a slower compatibility mode. Re-create the filesystem with ftype=1 to avoid this.\nDocker on ZFS In Proxmox homelabs, Docker frequently runs inside a VM or LXC backed by ZFS. The performance characteristics depend on the ZFS recordsize and how Docker\u0026rsquo;s overlay2 interacts with ZFS\u0026rsquo;s own CoW:\nDefault ZFS recordsize (128K): fine for most containers Database containers (PostgreSQL, MySQL): set recordsize to 16K on the dataset Media containers (Jellyfin, Plex): set recordsize to 1M on media volumes 1 2 zfs set recordsize=16K storage/docker/db zfs set recordsize=1M storage/docker/media For the Docker storage directory itself, keep the default recordsize unless you benchmark and find a clear improvement.\nDocker storage options depend on the backend Do not bypass kernel compatibility checks or replace an existing daemon configuration to enable a guessed storage option. Inspect docker info, the backing filesystem, and the installed daemon\u0026rsquo;s supported configuration. Classic overlay2.size requires compatible XFS project quotas and limits the writable layer; it does not cap bind mounts, named volumes, or logs. It is not a portable containerd snapshotter option.\nUse the quota and backend guide to plan a separate test. A daemon restart affects running services and belongs in a maintenance window with a rollback plan.\nPer-Container Resource Limits For containers with heavy write workloads, mount a dedicated volume or set per-container storage limits in docker-compose.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 services: database: image: postgres:16-alpine volumes: - postgres-data:/var/lib/postgresql/data deploy: resources: limits: memory: 2G volumes: postgres-data: driver_opts: type: none device: /mnt/ssd-fast/postgres o: bind This bypasses overlay2 CoW for the database directory entirely, avoiding double-CoW (ZFS CoW + overlay2 CoW).\nMaintenance and Automation Review cleanup before scheduling Do not schedule an indiscriminate data-volume cleanup. An unused volume may hold the only copy of a stopped application\u0026rsquo;s data. Preserve rollback images and inspect stopped containers before removing them. The following timer records usage instead of deleting resources; its unit names are retained from the earlier example, so inspect existing local units before replacing them.\nSystemd timer for weekly storage inventory 1 2 3 4 5 6 7 8 # /etc/systemd/system/docker-cleanup.service [Unit] Description=Docker storage inventory After=docker.service [Service] Type=oneshot ExecStart=/usr/bin/docker system df -v 1 2 3 4 5 6 7 8 9 10 # /etc/systemd/system/docker-cleanup.timer [Unit] Description=Weekly Docker storage inventory [Timer] OnCalendar=weekly Persistent=true [Install] WantedBy=timers.target 1 2 sudo systemctl daemon-reload sudo systemctl enable --now docker-cleanup.timer Storage Monitoring with Prometheus For homelabs running Prometheus and node_exporter, enable the textfile collector to track Docker overlay2 usage:\n1 2 3 4 5 #!/bin/bash # /etc/node_exporter/textfile/docker_storage.prom DOCKER=$(sudo du -sb /var/lib/docker/ 2\u0026gt;/dev/null | cut -f1) echo \u0026#34;docker_overlay2_usage_bytes $DOCKER\u0026#34; \u0026gt; /var/lib/node_exporter/textfile_collector/docker_storage.prom echo \u0026#34;docker_overlay2_usage_percent $(df /var/lib/docker/ --output=pcent 2\u0026gt;/dev/null | tail -1 | tr -d \u0026#39; %\u0026#39;)\u0026#34; \u0026gt;\u0026gt; /var/lib/node_exporter/textfile_collector/docker_storage.prom Run this every 5 minutes via cron and graph it in Grafana alongside other host metrics.\nConclusion Docker\u0026rsquo;s overlay2 storage driver is fast and reliable, but its layer-based architecture requires a different approach to disk management than traditional filesystems. Understanding the four overlay directories, monitoring build cache accumulation, checking inodes before disk space, and tuning for your ZFS or XFS backend all make the difference between a Docker host that runs for months and one that mysteriously fills up every week.\nStart with docker system df to see your current state. Check your inode usage with df -i /var/lib/docker. Set up automated pruning. And if you\u0026rsquo;re on a Proxmox-backed homelab, pay attention to ZFS dataset quotas and recordsize tuning. Your containers will thank you.\n","permalink":"https://blog.gntech.me/posts/2026-06-15-docker-overlay2-storage-driver-linux-guide/","summary":"Everything about Docker\u0026rsquo;s overlay2 storage driver — how overlay layers work, tracking disk usage by layer, recovering space from dangling layers, and performance tuning on ZFS-backed Proxmox hosts. Real commands, real output.","title":"Docker Overlay2 Storage Driver — Layers, Disk Usage, Troubleshooting"},{"content":"Your homelab generates paperwork — invoices, ISP contracts, device manuals, tax documents, warranty cards. Paperless-ngx turns that pile of PDFs and scans into a fully searchable, auto-tagged digital archive with OCR, machine learning classification, and multi-user access. Deploying it with Docker Compose keeps the stack isolated, reproducible, and easy to upgrade.\nThis guide walks through a production-ready Paperless-ngx deployment with PostgreSQL, Redis, Gotenberg, and Apache Tika, plus consumption templates, automated email ingestion, and Traefik reverse proxy configuration.\nPrerequisites Docker and Docker Compose v2 installed on your host PostgreSQL 16 — can run as a container within the same Compose file (this guide uses external for production reliability) A reverse proxy — Traefik, nginx, or Caddy for HTTPS access Storage directories — separate volumes for consumption (document import), media (archive), and database persistence Docker Compose Configuration for Paperless-ngx The full Paperless-ngx stack consists of five services: the main application, PostgreSQL, Redis for the task queue, Gotenberg for document conversion, and Apache Tika for metadata extraction.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 # compose.yaml services: paperless: image: ghcr.io/paperless-ngx/paperless-ngx:latest container_name: paperless restart: unless-stopped depends_on: - db - redis - gotenberg - tika volumes: - ./data:/usr/src/paperless/data - ./media:/usr/src/paperless/media - ./export:/usr/src/paperless/export - ./consume:/usr/src/paperless/consume environment: PAPERLESS_URL: https://docs.example.com PAPERLESS_SECRET_KEY: \u0026#34;${PAPERLESS_SECRET_KEY}\u0026#34; PAPERLESS_TIME_ZONE: America/Santo_Domingo PAPERLESS_OCR_LANGUAGE: eng+spa PAPERLESS_OCR_MODE: redo PAPERLESS_ENABLE_MATCHING_ALGORITHMS: auto PAPERLESS_REDIS: redis://redis:6379 PAPERLESS_DBHOST: db PAPERLESS_DBUSER: paperless PAPERLESS_DBPASS: \u0026#34;${PAPERLESS_DB_PASSWORD}\u0026#34; PAPERLESS_DBNAME: paperless PAPERLESS_TIKA_ENABLED: 1 PAPERLESS_TIKA_GOTENBERG_ENDPOINT: http://gotenberg:3000 PAPERLESS_TIKA_ENDPOINT: http://tika:9998 db: image: postgres:16-alpine container_name: paperless-db restart: unless-stopped volumes: - ./pgdata:/var/lib/postgresql/data environment: POSTGRES_DB: paperless POSTGRES_USER: paperless POSTGRES_PASSWORD: \u0026#34;${PAPERLESS_DB_PASSWORD}\u0026#34; redis: image: redis:7-alpine container_name: paperless-redis restart: unless-stopped gotenberg: image: gotenberg/gotenberg:8 container_name: paperless-gotenberg restart: unless-stopped command: - \u0026#34;gotenberg\u0026#34; - \u0026#34;--chromium-disable-javascript=true\u0026#34; - \u0026#34;--chromium-allow-list=file:///tmp/.*\u0026#34; tika: image: ghcr.io/paperless-ngx/tika:latest container_name: paperless-tika restart: unless-stopped Create a .env file in the same directory to store secrets:\nPAPERLESS_SECRET_KEY=\u0026lt;run: openssl rand -base64 48\u0026gt; PAPERLESS_DB_PASSWORD=\u0026lt;run: openssl rand -base64 32\u0026gt; First-Run Setup Create the required directory structure, generate the secret key, and start the stack:\n1 2 3 4 5 6 7 8 9 10 mkdir -p data media consume export pgdata # Generate secret if needed openssl rand -base64 48 # Start all services docker compose up -d # Wait for database migration, then create admin user docker compose exec paperless paperless-manage createsuperuser The first startup runs database migrations automatically. Watch the logs with docker compose logs -f paperless to confirm completion before creating the superuser.\nOCR and Machine Learning Configuration Paperless-ngx runs OCR on every ingested document using Tesseract. The configuration above uses English and Spanish language packs (eng+spa) — add more languages by separating with + (e.g., deu+eng+fra+spa).\nThe PAPERLESS_OCR_MODE: redo setting forces re-OCR on every document, which is useful when documents come from various sources with inconsistent quality. For production use where you control the scanner quality, switch to skip to avoid unnecessary processing.\nMachine learning classification is enabled with:\nPAPERLESS_ENABLE_MATCHING_ALGORITHMS: auto This activates Paperless-ngx\u0026rsquo;s built-in ML model that learns from manual document corrections. Over time, it automatically assigns correspondents, document types, and tags based on your patterns. Heuristics like exact-match and fuzzy-match supplement the ML algorithm for reliable results from day one.\nConsumption Templates for Automatic Processing Consumption templates let you define rules that automatically classify incoming documents based on content, filename, or correspondent. Create a YAML file in the consume directory:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 # consume/templates.yaml - match: invoice|invoice|factura|receipt match_algorithm: content document_type: Invoice tags: - Finance storage_path: Finance/Invoices/{created_year} - match: contract|agreement|terms match_algorithm: content document_type: Contract storage_path: Documents/Contracts - match: manual|guide|datasheet match_algorithm: filename document_type: Manual tags: - Hardware storage_path: Reference/Manuals Document types, correspondents, and tags must exist in Paperless-ngx before the template can reference them. Create them through the web UI first, then templates apply automatically on document ingestion.\nAutomated Ingestion Pipeline Paperless-ngx watches the consume/ directory for new files. Drop a PDF in there and it gets OCR\u0026rsquo;d, classified, and archived within seconds. Set up automated ingestion sources:\nEmail ingestion — Paperless can pull documents from email accounts:\n1 2 3 4 5 6 # Add to paperless environment PAPERLESS_CONSUME_MAIL_ENABLED: true PAPERLESS_CONSUME_MAIL_HOST: imap.example.com PAPERLESS_CONSUME_MAIL_USER: scans@example.com PAPERLESS_CONSUME_MAIL_PASS: \u0026#34;${MAIL_PASSWORD}\u0026#34; PAPERLESS_CONSUME_MAIL_INBOX: INBOX Scanner integration — Point your SANE network scanner to write to the consume/ directory via an SMB or NFS share mounted at /consume. Many MFPs support scan-to-folder or scan-to-email. Configure scan-to-email for the most reliable pipeline.\nMobile app upload — The Paperless-ngx mobile app (Android/iOS) supports direct document upload through the API. Enable this in Settings → Mobile → App Authentication. You can also configure the share-to-Paperless shortcut on iOS.\nReverse Proxy Integration with Traefik Expose Paperless-ngx through your reverse proxy for HTTPS access. Traefik labels for the Paperless service:\n1 2 3 4 5 6 labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.paperless.rule=Host(`docs.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.paperless.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.paperless.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.services.paperless.loadbalancer.server.port=8000\u0026#34; For nginx proxy:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 server { listen 443 ssl; server_name docs.example.com; client_max_body_size 100M; location / { proxy_pass http://10.0.20.50:8000; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; } } Set PAPERLESS_URL to match your public or internal URL (e.g., https://docs.example.com). Without this, Paperless generates incorrect links in emails and API responses.\nBackup Strategy Two components must be backed up: the media directory containing archived documents and the PostgreSQL database for metadata and tags.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 #!/bin/bash # backup-paperless.sh BACKUP_DIR=\u0026#34;/backup/paperless\u0026#34; TIMESTAMP=$(date +%Y%m%d-%H%M%S) mkdir -p \u0026#34;$BACKUP_DIR\u0026#34; # Database dump docker compose exec -T db pg_dump -U paperless paperless | \\ gzip \u0026gt; \u0026#34;$BACKUP_DIR/paperless-db-$TIMESTAMP.sql.gz\u0026#34; # Media archive tar czf \u0026#34;$BACKUP_DIR/paperless-media-$TIMESTAMP.tar.gz\u0026#34; \\ -C \u0026#34;$(pwd)\u0026#34; media consume # Keep last 30 days, remove older find \u0026#34;$BACKUP_DIR\u0026#34; -name \u0026#34;*.gz\u0026#34; -mtime +30 -delete Run this daily via cron or a systemd timer. For off-site protection, pipe the backups to restic or borg against a remote repository.\nConclusion Paperless-ngx transforms document management in the homelab from a mess of random PDFs into a structured, searchable, and automated archive. Docker Compose makes the full stack — OCR, ML classification, document conversion, and metadata extraction — deployable with a single docker compose up -d. Start with the consumption directory workflow, add email ingestion, and let the machine learning model learn your document patterns over time.\nFor more details, visit the Paperless-ngx documentation and the GitHub repository.\n","permalink":"https://blog.gntech.me/posts/2026-06-15-paperless-ngx-docker-deployment-homelab/","summary":"Deploy Paperless-ngx with Docker Compose in your homelab. Covers OCR processing, ML-based document classification, consumption templates, automated ingestion, and Traefik reverse proxy integration.","title":"Paperless-ngx Docker Deployment — AI-Powered Document Management"},{"content":"Every homelab running Proxmox eventually hits the automation wall. You want Terraform to spin up VMs, Ansible to configure LXC containers, or a monitoring tool to query node status. The default approach — embedding the root password into scripts and CI pipelines — works until it doesn\u0026rsquo;t. One leaked env file, one committed secret in a public repo, and your entire virtualization host is exposed.\nProxmox API tokens solve this cleanly. Combined with role-based access control (RBAC), you grant each automation tool exactly the permissions it needs, scoped to specific resources, without ever sharing a password. This guide walks through token creation, role assignment, and practical integration with Terraform and Ansible.\nUnderstanding Proxmox Authentication and RBAC Proxmox supports three authentication realms: PAM (system users), PVE (Proxmox-built-in user database), and external realms like LDAP or OIDC. API tokens work with any realm but are most commonly created from PVE users for service accounts.\nRBAC in Proxmox has three components: users, roles (collections of privileges), and paths (resource hierarchy). Permissions flow down the path tree. A role assigned to /vms applies to every VM under it. Roles assigned to a specific VM path like /vms/100 override parent permissions for that single resource.\nThe built-in roles cover most use cases:\nAdministrator — full access to everything PVEAuditor — read-only view of all resources PVEAdmin — full access except user/token/pool management PVEDatastoreUser — storage management and template operations PVEVMUser — manage VMs and containers without storage or template rights PVETemplateUser — create and deploy templates PVEPoolUser — manage resource pools For automation, you rarely need anything beyond PVEVMUser or PVEAuditor. Start restrictive and expand as needed.\nCreating Users and API Tokens Create a dedicated service user for automation. Never use your personal admin account.\n1 2 3 # Create a PVE-authentication service user pveum user add automation@pve --comment \u0026#34;Automation service account\u0026#34; pveum passwd automation@pve The password is required for user creation but you won\u0026rsquo;t use it for API access — the API token authenticates independently. Now generate an API token bound to that user:\n1 pveum user token add automation@pve terraform --privsep=0 The output includes:\nToken ID: automation@pve!terraform Secret: a UUID string that serves as the bearer token The --privsep=0 flag disables privilege separation, meaning the token inherits the user\u0026rsquo;s full permissions. With --privsep=1, the token starts with zero privileges and you must grant permissions to the token itself rather than inheriting from the user. For most automation scenarios, privsep=0 is simpler and safe when combined with restrictive role assignments.\nView your existing tokens at any time:\n1 pveum user token list automation@pve Important: The secret is shown only once at creation time. Store it in a password manager or vault immediately. Proxmox hashes the token secret and cannot recover it later — you would need to delete and recreate the token.\nAssigning Roles and Permission Paths A token with no role assignments is useless. Assign roles on specific resource paths to control what the token can do.\n1 2 3 4 5 6 7 8 # Grant VM management on all VMs pveum acl modify /vms --user automation@pve --role PVEVMUser # Grant storage access to a specific datastore pveum acl modify /storage/local --user automation@pve --role PVEDatastoreUser # Grant audit (read-only) access to the node itself pveum acl modify /nodes/srv1 --user automation@pve --role PVEAuditor The resource hierarchy uses these path prefixes:\nPath Scope / Entire datacenter /vms All VMs and containers /vms/100 Single VM or container 100 /nodes/srv1 Specific node resources /storage/local Specific storage /pool/production Resource pool named \u0026ldquo;production\u0026rdquo; For fine-grained control, grant the minimal role at the most specific path. A Terraform token that only provisions VMs in a production pool needs nothing beyond /pool/production with PVEVMUser.\nYou can also create deny rules by omitting privileges or using a restrictive custom role (covered below).\nAuthenticating with API Tokens from Automation Tools API tokens authenticate via HTTP header. Every Proxmox API call includes the token ID and secret.\nDirect API Access Test your token with a simple curl call:\n1 2 curl -s -H \u0026#34;Authorization: PVEAPIToken=automation@pve!terraform=uuid-secret\u0026#34; \\ https://proxmox:8006/api2/json/nodes The token format is always USER@REALM!TOKENID=SECRET. Replace uuid-secret with your actual token secret.\nTerraform Integration The Proxmox Terraform provider (bpg/proxmox) supports token authentication natively:\n1 2 3 4 5 6 provider \u0026#34;proxmox\u0026#34; { endpoint = \u0026#34;https://10.0.20.30:8006/\u0026#34; api_token = \u0026#34;automation@pve!terraform=uuid-secret\u0026#34; # Enable TLS verification in production insecure = false } For CI/CD pipelines, inject the token via environment variable to avoid hardcoding:\n1 export PM_API_TOKEN=\u0026#34;automation@pve!terraform=uuid-secret\u0026#34; Ansible Integration The community.general.proxmox modules support token-based auth:\n1 2 3 4 5 # group_vars/proxmox.yml proxmox_api_host: 10.0.20.30 proxmox_api_user: automation@pve!terraform proxmox_api_token_id: automation@pve!terraform proxmox_api_token_secret: \u0026#34;{{ vault_proxmox_token }}\u0026#34; Store the secret in Ansible Vault or your secrets manager, never in plain-text variables.\nToken Security Best Practices API tokens eliminate password sharing but introduce their own security surface. Follow these practices to keep your Proxmox host secure:\nUse one token per tool. If Terraform gets compromised, you revoke one token instead of rebuilding every pipeline. Name tokens descriptively so you know which tool uses which:\n1 2 3 pveum user token add automation@pve terraform pveum user token add automation@pve ansible pveum user token add automation@pve monitoring Rotate tokens regularly. Delete and recreate tokens on a schedule or after any security incident:\n1 2 pveum user token delete automation@pve terraform pveum user token add automation@pve terraform Audit token usage. Proxmox logs all API access through pveproxy. Check who accessed what:\n1 grep \u0026#34;automation@pve\u0026#34; /var/log/pveproxy/access.log | tail -50 Never commit tokens to Git. Use environment variables, HashiCorp Vault, Ansible Vault, or a secrets manager. If a token is exposed, revoke it immediately.\nUse short-lived tokens for CI/CD. If your CI platform supports it, generate fresh tokens per pipeline run and revoke them on completion.\nCustom Role Creation for Fine-Grained Access When built-in roles are too broad or too narrow, create custom roles with exactly the privileges you need.\n1 2 3 4 5 6 7 8 # Read-only VM access with console and CD-ROM mount pveum role add CustomVMViewer -privs \u0026#34;VM.Audit VM.Console VM.Config.CDROM\u0026#34; # Backup-only role — can create snapshots but not modify VMs pveum role add CustomBackupOperator -privs \u0026#34;VM.Audit VM.Backup VM.Snapshot\u0026#34; # Monitoring role — node and storage stats only pveum role add CustomMonitor -privs \u0026#34;Sys.Audit Sys.Syslog Datastore.Audit Pool.Audit\u0026#34; The privilege system is granular. Common privileges for automation roles include:\nPrivilege What it grants VM.Audit List and view VM configuration VM.PowerMgmt Start, stop, reset VMs VM.Config.CDROM Mount ISO images VM.Config.Disk Add or resize disks VM.Backup Create and restore backups VM.Snapshot Create and manage snapshots VM.Clone Clone VMs and templates Datastore.Audit List storage contents Datastore.Allocate Upload files and templates Sys.Audit Read-only node access Sys.Syslog Access system logs Assign your custom role to specific paths:\n1 pveum acl modify /vms/200 --user automation@pve --role CustomVMViewer This grants VM200 read access and console capability, but nothing else — even on the same node.\nTesting and Troubleshooting After setting up roles and tokens, verify the permissions are correct:\n1 2 curl -s -H \u0026#34;Authorization: PVEAPIToken=automation@pve!terraform=uuid-secret\u0026#34; \\ https://proxmox:8006/api2/json/access/permissions | jq . This returns the effective permissions for the token. If a privilege is missing, it won\u0026rsquo;t appear in the response.\nCommon issues:\n401 Unauthorized — token secret is wrong or token was deleted. Create a new token. 403 Forbidden — token authenticated but lacks permissions on the requested path. Check ACL assignments with pveum acl list. 500 path not found — the resource path in the ACL doesn\u0026rsquo;t match any actual resource. Verify resource IDs and paths. Debug token information:\n1 pveum user token info automation@pve terraform This shows the token\u0026rsquo;s creation date, expiration, and privilege separation mode.\nConclusion API tokens with RBAC transform Proxmox from a manual management tool into an automation-ready platform. Each token acts as an independent credential — revocable, scoped, and auditable — without the risk of sharing root passwords across scripts and CI pipelines.\nStart with the principle of least privilege: create one user, one token per automation tool, assign the narrowest role on the most specific path, and audit regularly. Your homelab automation will be more secure, easier to troubleshoot, and simpler to maintain.\nFrom there, integrate with Terraform for infrastructure-as-code VM provisioning, Ansible for post-deployment configuration, or custom scripts for backup automation — all authenticated through tokens instead of passwords.\n","permalink":"https://blog.gntech.me/posts/2026-06-15-proxmox-api-tokens-rbac-secure-automation/","summary":"Learn to secure Proxmox API access using API tokens and fine-grained RBAC. Covers token creation, role-based permission separation, Terraform/Ansible integration, and audit logging — all without exposing root credentials.","title":"Proxmox API Tokens and RBAC — Secure Automation Access"},{"content":"If you have run containers on MikroTik before, you know the drill: enable device mode, configure a bridge, set up veth interfaces, write firewall rules, configure NAT, mount storage, and pull images by hand. It works, but it is a lot of steps for what should be simple.\nRouterOS 7.14 introduced the Apps subsystem (/app), and the 7.24 beta cycle expanded the catalog with well-known services like Gitea and Home Assistant. The Apps system is MikroTik\u0026rsquo;s answer to one-click container deployment — it handles networking, firewall rules, port mapping, and storage automatically.\nThis guide walks through everything: enabling container mode, running the setup wizard, deploying apps, and customizing configurations for your homelab.\nWhy the RouterOS App Catalog Matters for Homelab Before Apps, deploying a container on RouterOS required a half-dozen manual steps in the CLI or WinBox. The new /app system abstracts all of that into a catalog.\nWhat the App system does automatically:\nCreates and attaches veth interfaces to your LAN bridge Generates firewall filter rules and NAT dstnat entries Pulls container images from Docker Hub, GCR, or Quay Configures environment variables from a YAML manifest Generates a UI-URL for web access via WebFig This is not a replacement for a full Docker host. The MikroTik container runtime is lightweight — it runs on the router itself, sharing its CPU and RAM. That makes it ideal for lightweight services that benefit from living on the network edge: a local Gitea mirror, a Home Assistant instance, or an ad-blocking DNS container.\nRequirements and Prerequisites Before you start, confirm your device meets the requirements:\nRequirement Details RouterOS version 7.14+ (7.24 beta for newest catalog entries) Architecture arm64 or x86 (not MIPSBE / EN7562CT) Container package Must be installed (/system/package/print) Device mode Container mode must be enabled Storage External USB, NVMe, or SATA recommended (at least 4 GB) RAM Minimum 512 MB free for containers Check your device:\n1 2 3 4 5 6 7 8 # Verify container package is installed /system/package/print where name=container # Check available architecture /system/resource/print # List available disks /disk/print If the container package is missing, install it:\n1 /system/package/install container Reboot after installation.\nStep 1: Enable Container Mode Container mode requires physical access to the device and a configuration reset. This is a one-time step.\n1 2 3 4 5 # Check current device mode /system/device-mode/print # Enable container mode (requires confirmation and reboot) /system/device-mode/update container=yes The device will prompt for confirmation, then reboot. After the reboot, container mode is permanently enabled. You can verify:\n1 2 /system/device-mode/print # Output should show: container: yes Important: Enabling container mode must be done with physical console or serial access. It cannot be enabled remotely via SSH or API.\nStep 2: Run the Setup Wizard The /app setup wizard automates storage selection, bridge configuration, and IP assignment.\n1 /app setup The wizard walks through three steps:\nStep 2a: Storage Selection The scanner detects formatted disks (nvme1, usb1, sata1, etc.). If no disk appears, format one first:\n1 2 3 4 5 # Format a USB drive as ext4 /disk/format-drive usb1 file-system=ext4 # Test storage performance (min 100 MB/s sequential recommended) /disk/test usb1 Pick the disk when prompted. The wizard creates the necessary directory structure.\nStep 2b: Bridge Configuration Select the LAN bridge interface (typically bridge or bridge_lan). The wizard will:\nCreate virtual ethernet (veth) interfaces for each app Attach them to the selected bridge Configure IP addressing for container connectivity Step 2c: Router IP Enter the IP address used to access the router\u0026rsquo;s web interface. This is used for UI-URL generation and automatic WebFig integration links.\nAfter completion, verify the settings:\n1 /app/settings/print Expected output shows your disk, bridge, router-ip, and auto-configured paths:\napp-store-urls: https://apps.mikrotik.com/container/index.yaml auto-update: no disk: usb1 download-path: usb1/media/downloads media-path: usb1/media lan-bridge: bridge router-ip: 10.0.20.1 show-in-webfig: yes Step 3: Browse and Deploy Apps List available apps from the MikroTik catalog:\n1 /app/print This shows all discoverable apps with their status, size, and UI URL (if running). Flags indicate state:\nX = disabled R = running Deploy Gitea 1 /app/enable gitea RouterOS downloads the Gitea container image, creates the veth interface, adds firewall rules, and configures port forwarding. In seconds, Gitea is accessible at the generated UI-URL (typically http://10.0.20.1:3000).\nYou can verify the automatic configuration:\n1 2 3 4 5 6 7 8 # Check the veth interface /interface/print where name~\u0026#34;veth\u0026#34; # View auto-generated firewall rules /ip/firewall/filter/print where comment~\u0026#34;app\u0026#34; # View auto-generated NAT rules /ip/firewall/nat/print where comment~\u0026#34;app\u0026#34; Deploy Home Assistant 1 /app/enable home-assistant Home Assistant deploys with its web UI mapped to port 8123 by default. The app includes automatic mDNS advertisements for discovery on the local network.\nMonitor deployment status:\n1 /app/print detail where name=home-assistant The output shows status: running, memory usage, CPU usage, data size, and the UI-URL.\nCheck Running Apps 1 2 3 /app/print # X - DISABLED, R - RUNNING # Columns: NAME, STATUS, CPU-USAGE, MEMORY-CURRENT, UI-URL Configuring App Parameters Apps are not locked down. Every parameter in the YAML manifest can be overridden before or after deployment.\nEnvironment Variables 1 /app/set environment=\u0026#34;GITEA__server__DOMAIN=git.lab.example.com, GITEA__server__ROOT_URL=https://git.lab.example.com\u0026#34; gitea Custom Mounts 1 /app/set extra-mounts=\u0026#34;usb1/media/gitea-data:/data\u0026#34; gitea Port Redirects Override auto-generated port maps:\n1 /app/set firewall-redirects=\u0026#34;443:3000, 80:3000\u0026#34; gitea This maps host ports 443 and 80 to container port 3000.\nEnable Auto-Update 1 /app/set auto-update=yes gitea Or set globally for all apps:\n1 /app/settings/set auto-update=yes Custom App Stores Run apps from a private or third-party catalog:\n1 /app/settings/set app-store-urls=\u0026#34;https://apps.example.com/catalog.yaml\u0026#34; The custom store URL must point to a YAML array where each element describes an app with name, image, ports, and environment variables matching the MikroTik App schema.\nApps vs Manual Container Setup Aspect App Catalog (/app) Manual (/container) Setup time 2 minutes + wizard 15-30 minutes Network config Automatic (veth + bridge) Manual (veth + bridge) Firewall rules Auto-generated Manual configuration NAT / port forwarding Auto dstnat Manual dstnat rules Storage mounts Configured in wizard Manual bind mounts Image pull On enable /container/config/set + pull Custom images Via custom app store Any image Multi-container Supported (YAML groups) Manual stacking Granular control Limited (preset parameters) Full control When to use Apps: Quick deployments, learning containers on RouterOS, standard services (Gitea, Home Assistant), or when you want one-command repeatability.\nWhen to use manual: Custom images, multi-stage networking, non-standard port schemes, or when you need fine-grained resource limits and device passthrough.\nViewing the App YAML Manifest Every app is defined by a YAML manifest. You can inspect it:\n1 /app/print yaml where name=gitea This shows the full compose-style definition including image source, environment variables, volumes, ports, and device requirements. You can export and modify it for custom app store deployment.\nSecurity Considerations Running containers on your router expands the attack surface. Keep these points in mind:\nPhysical access requirement: Container mode requires physical console access to enable. This is a deliberate security boundary — a remote compromise cannot enable containers without prior physical setup.\nThird-party images: MikroTik catalogs point to upstream registries. Verify image integrity when deploying critical services:\n1 /app/set check-certificate=yes gitea Device resources: A container that exhausts RAM or CPU can affect routing performance. Monitor usage:\n1 /app/print detail Network isolation: By default, apps use network=default which provides LAN access with NAT for outbound traffic. Use network=internal for services that should not be reachable from the LAN:\n1 /app/set network=internal sensitive-app Troubleshooting Common Issues Container Won\u0026rsquo;t Start Check available memory and disk space:\n1 2 3 /system/resource/print /disk/print /disk/usb1/print If storage is full, the image extraction fails. Free space or use a larger disk.\nApp Not Found in Catalog Ensure the catalog URL is reachable:\n1 2 3 /app/settings/print # Verify app-store-urls is set /app/check-store For custom apps, verify the YAML endpoint returns a valid array.\nNetworking Issues If the container UI is unreachable:\n1 2 3 4 5 6 7 8 # Check veth interface exists and is in bridge /interface/print where name~\u0026#34;veth\u0026#34; # Verify NAT rules /ip/firewall/nat/print where comment~\u0026#34;app\u0026#34; # Check container logs /log/print where topics~\u0026#34;container\u0026#34; Real-World Example: Full Gitea Deployment Here is the complete sequence from zero to running Gitea on a RouterOS device:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 # 1. Install container package (if missing) /system/package/install container /system/reboot # 2. Enable container mode (physical console required) /system/device-mode/update container=yes # 3. Format USB storage /disk/format-drive usb1 file-system=ext4 /disk/test usb1 # 4. Run setup wizard /app setup # 5. Deploy Gitea /app/enable gitea # 6. Customize /app/set environment=\u0026#34;GITEA__server__ROOT_URL=http://git.lab:3000\u0026#34; gitea # 7. Verify /app/print # Run your browser to http://10.0.20.1:3000 Conclusion The RouterOS App catalog transforms container deployment on MikroTik from a multi-step CLI exercise into a guided, repeatable process. For homelab users running capable hardware like the RB5009, CCR2004, or CHR, this means lightweight services can live at the network edge without a separate Linux host.\nThe sweet spot for Apps is standard services with predictable configurations — Gitea for local code mirrors, Home Assistant for home automation, or similar light workloads. When you need exotic images or full control, the manual /container path is still there.\nThe catalog is evolving. With the 7.24 beta adding Gitea and Home Assistant support, expect more entries in future releases. Whether you are new to RouterOS containers or a veteran of manual veth configuration, give /app setup a try — it might save you fifteen minutes and a few firewall headaches.\n","permalink":"https://blog.gntech.me/posts/2026-06-15-mikrotik-routeros-container-apps-deployment/","summary":"Deploy containers on your MikroTik router in minutes using the new Apps catalog. Automatic firewall rules, NAT, and veth interfaces — no manual Docker setup required.","title":"MikroTik RouterOS Container Apps — One-Click Service Deployment"},{"content":"Every homelab needs centralized file storage. Windows machines need network shares for backups, media servers need a library location, and configuration files need a home you can reach from any device. Running a full NAS VM — TrueNAS, OMV, or Unraid — eats 4-8GB of RAM and adds unnecessary overhead for a container that only serves files.\nThe sweet spot is combining Samba inside a Proxmox LXC with a ZFS dataset passthrough. The LXC gives you near-native file I/O with sub-1GB RAM usage. ZFS provides data integrity, compression, and snapshot-based backups. Samba serves CIFS/SMB shares to any client on your network.\nThis guide walks through the complete setup: creating the LXC, passing through a ZFS dataset, configuring Samba for performance, and integrating with ZFS snapshots for backup.\nWhy LXC + Samba + ZFS Instead of a VM The alternatives each make trade-offs that matter in a homelab:\nNAS VM (TrueNAS, OMV, Unraid) — Full OS kernel, 4-8GB RAM minimum, separate storage stack. Excellent features but heavy for a single-purpose file server. Docker Samba containers — dperson/samba or similar images. They work, but Docker\u0026rsquo;s networking adds translation overhead for CIFS, and bind-mounting ZFS datasets into Docker requires careful permission mapping. Kernel-level CIFS in an LXC outperforms containerized Samba by a measurable margin on sequential reads. Samba on the Proxmox host directly — Works, but violates the principle of isolating services from the hypervisor. Updates, reboots, or a config mistake can affect running VMs. LXC hits the sweet spot: a full userspace with kernel-native SMB performance, 256MB-1GB RAM footprint, and complete isolation from the Proxmox host.\nPrerequisites Proxmox VE 8.x or 9.x with a ZFS pool (e.g., tank or rpool/data) Root or sudo access on the Proxmox host Debian 12 (bookworm) container template available locally Verify your templates are downloaded:\n1 pveam list local | grep debian If no Debian template exists, download one:\n1 pveam download local debian-12-standard_12.7-1_amd64.tar.zst Step 1 — Create the LXC Container Create a Debian 12 LXC with a static IP on your storage VLAN. Adjust the CTID, IP, and storage pool to match your environment:\n1 2 3 4 5 6 7 8 9 10 11 12 13 CTID=110 pvesm list local-zfs # confirm your storage pool name pct create $CTID local:vztmpl/debian-12-standard_12.7-1_amd64.tar.zst \\ --storage local-zfs \\ --rootfs local-zfs:16 \\ --hostname fileserver \\ --net0 name=eth0,bridge=vmbr0,ip=10.0.30.10/24,gw=10.0.30.1 \\ --cores 2 \\ --memory 1024 \\ --swap 512 \\ --unprivileged 0 \\ --ostype debian Key flags explained:\n--unprivileged 0 — Creates a privileged LXC. Important for Samba + ZFS compatibility, discussed in detail below. --rootfs local-zfs:16 — 16GB root filesystem on your ZFS storage pool. Your data dataset is separate. --memory 1024 — 1GB RAM. Samba typically uses 200-400MB under load. Start and enter the container:\n1 2 pct start $CTID pct enter $CTID Step 2 — Privileged vs Unprivileged LXC — The Right Choice for Samba This is the most common question when setting up Samba in LXC. Here is the trade-off:\nUnprivileged containers use UID/GID mapping (subuid/subgid) to isolate the container\u0026rsquo;s root from the host. This is more secure — a container breakout from root maps to a non-privileged user on the host. However, Samba\u0026rsquo;s interaction with ZFS ACLs and Windows permission models becomes painful. File ownership on passthrough ZFS datasets will show as nobody:nogroup unless you configure complex UID mappings. Windows ACLs (NTFS-style) are nearly impossible to maintain correctly.\nPrivileged containers run as root on the host. This is less isolated — a container compromise means host root. However, for Samba with ZFS passthrough, it \u0026ldquo;just works\u0026rdquo;:\nFile ownership from the container maps directly to the ZFS dataset ZFS ACLs are natively supported Windows permission inheritance functions correctly No UID mapping headaches The risk is manageable when you isolate the container on a storage-only VLAN (10.0.30.0/24 in this guide) and apply firewall rules.\nIf you must use unprivileged containers, you can work around UID mapping with lxc.idmap entries in the container config, but expect ongoing maintenance.\nStep 3 — Create and Pass Through the ZFS Dataset Back on the Proxmox host, create a dedicated ZFS dataset for your Samba data:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 # Create the dataset zfs create tank/fileserver # Enable ZSTD compression for space savings zfs set compression=zstd tank/fileserver # Set recordsize to 1M for Samba sequential workloads zfs set recordsize=1M tank/fileserver # Disable atime — reduces write amplification on NAS workloads zfs set atime=off tank/fileserver # Set a quota if needed (adjust as necessary) zfs set quota=4T tank/fileserver Mount the dataset into the LXC. From the Proxmox host (not the container):\n1 pct set $CTID -mp0 /tank/fileserver,mp=/srv/fileserver This creates a bind mount from the host path /tank/fileserver to the container\u0026rsquo;s /srv/fileserver. Verify it worked:\n1 2 pct enter $CTID df -h /srv/fileserver You should see the ZFS filesystem mounted inside the container with the size and usage of your dataset.\nStep 4 — Install and Configure Samba Inside the container, install Samba:\n1 apt update \u0026amp;\u0026amp; apt install -y samba samba-common-bin Back up the default configuration:\n1 cp /etc/samba/smb.conf /etc/samba/smb.conf.default Write the Samba configuration:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 cat \u0026gt; /etc/samba/smb.conf \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; [global] workgroup = WORKGROUP server string = %h fileserver netbios name = FILESERVER security = user map to guest = bad user server min protocol = SMB2_10 server max protocol = SMB3 disable netbios = yes smb ports = 445 log file = /var/log/samba/log.%m max log size = 1000 logging = file panic action = /usr/share/samba/panic-action %d # Performance tuning socket options = TCP_NODELAY IPTOS_LOWDELAY read raw = yes write raw = yes server signing = disabled strict sync = no disable spoolss = yes min receivefile size = 16384 aio read size = 16384 aio write size = 16384 use sendfile = yes [shares] comment = Homelab File Share path = /srv/fileserver browseable = yes read only = no guest ok = no create mask = 0664 directory mask = 0775 force user = nobody force group = nogroup veto files = /Thumbs.db/.DS_Store/.Trashes/ delete veto files = yes EOF Create a Samba user and set directory permissions:\n1 2 3 4 5 6 7 8 9 # Create a system user for Samba access useradd -m -s /usr/sbin/nologin smbuser # Set the Samba password (you will be prompted) smbpasswd -a smbuser # Set directory ownership on the shared dataset chown nobody:nogroup /srv/fileserver chmod 2775 /srv/fileserver Restart Samba:\n1 2 systemctl restart smbd systemctl enable smbd Test the Samba configuration:\n1 testparm Step 5 — Performance Tuning for Maximum Throughput Beyond ZFS settings, tune the Linux kernel network stack for Samba workloads. Add these to /etc/sysctl.conf on the LXC container:\n1 2 3 4 5 6 7 8 9 10 11 12 13 cat \u0026gt;\u0026gt; /etc/sysctl.conf \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; # Samba/network performance tuning net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 net.ipv4.tcp_rmem = 4096 87380 16777216 net.ipv4.tcp_wmem = 4096 65536 16777216 net.core.netdev_max_backlog = 5000 net.ipv4.tcp_congestion_control = bbr net.core.default_qdisc = fq EOF sysctl -p Verify BBR is available:\n1 2 sysctl net.ipv4.tcp_congestion_control # Output: net.ipv4.tcp_congestion_control = bbr If BBR is not available, load the module:\n1 2 modprobe tcp_bbr echo \u0026#34;tcp_bbr\u0026#34; \u0026gt;\u0026gt; /etc/modules Run a quick throughput test from a Linux client:\n1 2 3 4 5 # Write test dd if=/dev/zero of=/mnt/fileserver/testfile bs=1M count=2048 oflag=direct # Read test dd if=/mnt/fileserver/testfile of=/dev/null bs=1M iflag=direct On a modern homelab with 1GbE networking and SSDs, expect 110-118 MB/s sequential reads. With 2.5GbE, you should saturate the link at ~280 MB/s.\nStep 6 — Accessing Shares from Clients Windows Open File Explorer Right-click This PC → Map network drive Folder: \\\\10.0.30.10\\shares Check Connect using different credentials Enter the smbuser credentials you created Click Finish The share appears as a mapped drive. For persistent unmapped access, add the credentials to Windows Credential Manager.\nLinux Clients Install cifs-utils and mount:\n1 2 3 4 5 6 7 8 9 sudo apt install cifs-utils sudo mkdir -p /mnt/fileserver sudo mount -t cifs //10.0.30.10/shares /mnt/fileserver \\ -o username=smbuser,uid=1000,gid=1000,iocharset=utf8,vers=3.0 # Persistent mount in /etc/fstab echo \u0026#39;//10.0.30.10/shares /mnt/fileserver cifs username=smbuser,password=SECRET,uid=1000,gid=1000,iocharset=utf8,vers=3.0,noexec,nodev,nosuid,_netdev 0 0\u0026#39; \\ | sudo tee -a /etc/fstab macOS Open Finder → Go → Connect to Server Enter: smb://10.0.30.10/shares Click Connect Enter the smbuser credentials The share mounts on your desktop and in /Volumes/shares Step 7 — Backup Integration with ZFS Snapshots The real power of this setup is ZFS-native backups. Since the data lives on a ZFS dataset, you can snapshot and replicate it independently of the container.\nAutomated Snapshots on the Proxmox Host Create a snapshot script:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 cat \u0026gt; /usr/local/bin/snapshot-fileserver.sh \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; #!/bin/bash DATASET=\u0026#34;tank/fileserver\u0026#34; SNAPSHOT_NAME=\u0026#34;${DATASET}@daily-$(date +%Y%m%d-%H%M)\u0026#34; zfs snapshot \u0026#34;$SNAPSHOT_NAME\u0026#34; zfs list -t snapshot -o name | grep \u0026#34;${DATASET}@\u0026#34; | tail -7 # Prune snapshots older than 7 days zfs list -t snapshot -o name -H | grep \u0026#34;${DATASET}@\u0026#34; | \\ head -n -7 | while read snap; do zfs destroy \u0026#34;$snap\u0026#34;; done EOF chmod +x /usr/local/bin/snapshot-fileserver.sh Create a systemd timer to run it daily:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 cat \u0026gt; /etc/systemd/system/fileserver-snapshot.service \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; [Unit] Description=ZFS snapshot for file server dataset [Service] Type=oneshot ExecStart=/usr/local/bin/snapshot-fileserver.sh EOF cat \u0026gt; /etc/systemd/system/fileserver-snapshot.timer \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; [Unit] Description=Daily ZFS snapshot for file server [Timer] OnCalendar=daily Persistent=true [Install] WantedBy=timers.target EOF systemctl daemon-reload systemctl enable --now fileserver-snapshot.timer Send Snapshots to Proxmox Backup Server If you run Proxmox Backup Server, use proxmox-backup-client to send the dataset snapshot off-host:\n1 2 3 4 5 6 7 # Install the client on the Proxmox host (if not present) apt install proxmox-backup-client # Create and pipe a snapshot to PBS zfs send tank/fileserver@daily-20260615-0230 | \\ proxmox-backup-client backup fs.pxar:/proc/self/fd/0 \\ --repository root@pbs@10.0.20.40:store-backup Alternatively, replicate directly to a remote ZFS pool:\n1 2 zfs send tank/fileserver@daily-20260615-0230 \\ | ssh backup-server zfs recv backup-tank/fileserver The LXC itself is also backed up via Proxmox\u0026rsquo;s standard container backup:\n1 vzdump 110 --mode snapshot --compress zstd --storage pbs-remote Step 8 — Security Hardening Isolate the file server on a dedicated storage VLAN. In your MikroTik or managed switch:\nVLAN 30: Storage (LXC file server, NFS clients) Firewall rule: only allow SMB (TCP 445) and ICMP from client VLANs In the LXC itself, add these lines to the [global] section of /etc/samba/smb.conf:\n1 2 hosts allow = 10.0.0.0/8 192.168.0.0/16 127.0.0.1 hosts deny = 0.0.0.0/0 Enable UFW in the LXC:\n1 2 3 4 5 6 apt install -y ufw ufw default deny incoming ufw default allow outgoing ufw allow from 10.0.0.0/8 to any port 445 proto tcp ufw allow from 192.168.0.0/16 to any port 445 proto tcp ufw --force enable Additional settings to verify or add to /etc/samba/smb.conf:\n1 2 3 4 5 6 7 8 # Disable guest access entirely map to guest = never # Disable printer sharing load printers = no printing = bsd printcap name = /dev/null disable spoolss = yes Monitoring and Logging Samba logs to /var/log/samba/. Keep an eye on authentication failures:\n1 2 grep -i \u0026#34;password mismatch\u0026#34; /var/log/samba/log.smbd grep -i \u0026#34;invalid user\u0026#34; /var/log/samba/log.smbd Monitor active connections:\n1 smbstatus Sample output showing connected clients and open files:\nSamba version 4.17.12 PID Username Group Machine 12345 smbuser smbuser 10.0.20.50 (192.168.1.50) Service pid Machine Connected at Encryption Signing shares 12345 10.0.20.50 Mon Jun 15 12:34:56 2026 AST - - Conclusion Samba in a Proxmox LXC with ZFS dataset passthrough gives you a file server that is:\nLightweight — 256MB-1GB RAM, no separate OS to manage Fast — kernel-level CIFS with BBR congestion control, tuned ZFS recordsize, and sendfile Backed up — ZFS snapshots, PBS integration, and standard container backups Simple — one LXC config line for the mount, one smb.conf for the share Start by creating a Debian 12 privileged LXC, pass your ZFS dataset through as a mount point, configure Samba with the performance settings above, and set up daily ZFS snapshots. Your clients will see a fast SMB share, your Proxmox host will see minimal overhead, and your backups will be handled by ZFS automation.\nFor the homelab that needs centralized storage without running a full NAS VM, the LXC + Samba + ZFS combination is the most efficient path to a reliable file server.\n","permalink":"https://blog.gntech.me/posts/2026-06-15-samba-file-server-proxmox-lxc-zfs/","summary":"Run Samba inside a Proxmox LXC with direct ZFS dataset access for a sub-1GB RAM NAS. Covers privileged LXC setup, ZFS passthrough, user configuration, and Samba performance tuning for homelab file sharing.","title":"Samba File Server in Proxmox LXC with ZFS Dataset Passthrough"},{"content":"When your homelab has a dozen containers across multiple Compose stacks, docker stats stops cutting it. You need persistent metrics, historical data, and visual dashboards to understand what your containers are actually doing.\nThis guide deploys a complete monitoring stack using Docker Compose:\nPrometheus — scrapes and stores time-series metrics cAdvisor — exports per-container CPU, memory, network, and disk metrics node_exporter — exports host-level metrics (CPU, memory, disk, load, network) Grafana — dashboards and alerting with Prometheus as the data source Alertmanager — deduplicates and routes alerts to email, Telegram, or Slack Everything runs as Docker containers on a single host. No agent installation on the host beyond a single bind mount. All config files are included. Deploy in under 30 minutes.\nWhy Monitor Docker Containers in Your Homelab? Running docker stats gives you a live snapshot of CPU and memory per container. That\u0026rsquo;s useful for debugging right now, but it doesn\u0026rsquo;t help with:\nTrend analysis: was that memory spike yesterday a one-off or a leak? Resource planning: which container is eating 80% of your disk I/O? Anomaly detection: did a container restart while you were sleeping? Historical comparison: how did the last deployment affect performance? A Prometheus-based stack solves all of these. Prometheus scrapes metrics every 15 seconds by default, stores them with configurable retention, and exposes everything via PromQL for querying and alerting.\nArchitecture Overview — Prometheus Stack Components ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │ │ │ │ │ │ │ cAdvisor │────▶│ Prometheus │◀────│ Node Exporter│ │ (container │ │ (metrics │ │ (host │ │ metrics) │ │ storage) │ │ metrics) │ │ │ │ │ │ │ └──────────────┘ └──────┬───────┘ └──────────────┘ │ ▼ ┌──────────────┐ ┌──────────────┐ │ │ │ │ │ Grafana │────▶│ Alertmanager │ │ (dashboards) │ │ (alert │ │ │ │ routing) │ └──────────────┘ └──────────────┘ Prometheus is the central time-series database. It scrapes HTTP endpoints that expose metrics in Prometheus format:\ncAdvisor (:8080/metrics) — per-container CPU, memory, network RX/TX, filesystem, and task stats node_exporter (:9100/metrics) — host CPU, memory, disk space, disk I/O, network, load average, and more Prometheus itself (:9090/metrics) — internal metrics about the scrape targets and storage Grafana connects to Prometheus as a data source and provides dashboards. Alertmanager handles alert deduplication, grouping, and notification routing.\nDeploying the Monitoring Stack with Docker Compose Create a project directory and a .env file:\n1 mkdir -p ~/docker/monitoring \u0026amp;\u0026amp; cd ~/docker/monitoring .env contents:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # Grafana GF_ADMIN_PASSWORD=changeme_secure_password # Prometheus data retention PROMETHEUS_RETENTION_TIME=30d PROMETHEUS_RETENTION_SIZE=10GB # Alertmanager email (optional) SMTP_HOST=smtp.example.com SMTP_PORT=587 SMTP_USER=alerts@example.com SMTP_PASS=your_smtp_password ALERT_EMAIL=you@example.com docker-compose.yml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 x-restart: \u0026amp;restart restart: unless-stopped x-logging: \u0026amp;logging logging: driver: \u0026#34;local\u0026#34; options: max-size: \u0026#34;10m\u0026#34; max-file: \u0026#34;3\u0026#34; services: prometheus: \u0026lt;\u0026lt;: *restart \u0026lt;\u0026lt;: *logging image: prom/prometheus:latest container_name: prometheus ports: - \u0026#34;9090:9090\u0026#34; volumes: - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro - prometheus-data:/prometheus command: - \u0026#34;--config.file=/etc/prometheus/prometheus.yml\u0026#34; - \u0026#34;--storage.tsdb.retention.time=${PROMETHEUS_RETENTION_TIME}\u0026#34; - \u0026#34;--storage.tsdb.retention.size=${PROMETHEUS_RETENTION_SIZE}\u0026#34; - \u0026#34;--web.console.libraries=/etc/prometheus/console_libraries\u0026#34; - \u0026#34;--web.console.templates=/etc/prometheus/consoles\u0026#34; - \u0026#34;--web.enable-lifecycle\u0026#34; networks: - monitoring cadvisor: \u0026lt;\u0026lt;: *restart \u0026lt;\u0026lt;: *logging image: gcr.io/cadvisor/cadvisor:latest container_name: cadvisor ports: - \u0026#34;8080:8080\u0026#34; volumes: - /:/rootfs:ro - /var/run:/var/run:ro - /sys:/sys:ro - /var/lib/docker/:/var/lib/docker:ro - /dev/disk:/dev/disk:ro privileged: false devices: - /dev/kmsg networks: - monitoring node-exporter: \u0026lt;\u0026lt;: *restart \u0026lt;\u0026lt;: *logging image: prom/node-exporter:latest container_name: node-exporter ports: - \u0026#34;9100:9100\u0026#34; volumes: - /proc:/host/proc:ro - /sys:/host/sys:ro - /:/host/root:ro command: - \u0026#34;--path.procfs=/host/proc\u0026#34; - \u0026#34;--path.sysfs=/host/sys\u0026#34; - \u0026#34;--path.rootfs=/host/root\u0026#34; - \u0026#34;--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($$|/)\u0026#34; networks: - monitoring grafana: \u0026lt;\u0026lt;: *restart \u0026lt;\u0026lt;: *logging image: grafana/grafana:latest container_name: grafana ports: - \u0026#34;3000:3000\u0026#34; volumes: - grafana-data:/var/lib/grafana - ./grafana/provisioning:/etc/grafana/provisioning:ro - ./grafana/dashboards:/var/lib/grafana/dashboards:ro environment: GF_SECURITY_ADMIN_PASSWORD: \u0026#34;${GF_ADMIN_PASSWORD}\u0026#34; GF_INSTALL_PLUGINS: \u0026#34;grafana-piechart-panel\u0026#34; networks: - monitoring alertmanager: \u0026lt;\u0026lt;: *restart \u0026lt;\u0026lt;: *logging image: prom/alertmanager:latest container_name: alertmanager ports: - \u0026#34;9093:9093\u0026#34; volumes: - ./alertmanager.yml:/etc/alertmanager/alertmanager.yml:ro - alertmanager-data:/alertmanager command: - \u0026#34;--config.file=/etc/alertmanager/alertmanager.yml\u0026#34; - \u0026#34;--config.expand-env=true\u0026#34; - \u0026#34;--storage.path=/alertmanager\u0026#34; networks: - monitoring volumes: prometheus-data: grafana-data: alertmanager-data: networks: monitoring: driver: bridge Key points about this Compose file:\ncAdvisor mounts the host filesystem in several locations to read container and cgroup metrics. The /dev/kmsg device is needed for kernel log access on newer kernels. node_exporter uses --path.procfs, --path.sysfs, and --path.rootfs to read host metrics from bind-mounted host paths, avoiding running as host-networked. Grafana auto-provisions data sources and dashboards via the provisioning directory. Alertmanager persists silences and notification state to a Docker volume. Setting Up Prometheus Configuration Create prometheus.yml in the project directory:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 global: scrape_interval: 15s evaluation_interval: 15s external_labels: host: \u0026#34;srv1\u0026#34; alerting: alertmanagers: - static_configs: - targets: - alertmanager:9093 rule_files: - \u0026#34;alerts.yml\u0026#34; scrape_configs: - job_name: \u0026#34;prometheus\u0026#34; static_configs: - targets: [\u0026#34;localhost:9090\u0026#34;] - job_name: \u0026#34;cadvisor\u0026#34; scrape_interval: 30s static_configs: - targets: [\u0026#34;cadvisor:8080\u0026#34;] relabel_configs: - source_labels: [__meta_docker_container_name] target_label: container - source_labels: [__meta_docker_container_label_com_docker_compose_service] target_label: compose_service - job_name: \u0026#34;node\u0026#34; scrape_interval: 30s static_configs: - targets: [\u0026#34;node-exporter:9100\u0026#34;] # Add host-specific jobs here # - job_name: \u0026#34;mikrotik\u0026#34; # scrape_interval: 60s # static_configs: # - targets: [\u0026#34;10.0.20.1:9116\u0026#34;] The scrape_interval of 15 seconds is fine for a homelab. If you\u0026rsquo;re running on constrained hardware, bump it to 30s for cAdvisor and node_exporter jobs — the data resolution loss is negligible for trend analysis.\nPrometheus Alert Rules Create alerts.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 groups: - name: homelab rules: - alert: ContainerDown expr: time() - container_last_seen{container!=\u0026#34;\u0026#34;} \u0026gt; 60 for: 1m labels: severity: critical annotations: summary: \u0026#34;Container {{ $labels.name }} is down\u0026#34; description: \u0026#34;Container {{ $labels.name }} has not been seen for over 1 minute.\u0026#34; - alert: HighContainerCPU expr: sum(rate(container_cpu_usage_seconds_total{container!=\u0026#34;\u0026#34;}[5m])) by (name) \u0026gt; 0.8 for: 5m labels: severity: warning annotations: summary: \u0026#34;Container {{ $labels.name }} high CPU\u0026#34; description: \u0026#34;Container {{ $labels.name }} CPU usage above 80% for 5 minutes.\u0026#34; - alert: HighContainerMemory expr: container_memory_usage_bytes{container!=\u0026#34;\u0026#34;} / container_spec_memory_limit_bytes{container!=\u0026#34;\u0026#34;} \u0026gt; 0.9 for: 5m labels: severity: warning annotations: summary: \u0026#34;Container {{ $labels.name }} memory high\u0026#34; description: \u0026#34;Container {{ $labels.name }} memory usage at {{ $value | humanizePercentage }}\u0026#34; - alert: HostDiskFull expr: (node_filesystem_avail_bytes{mountpoint=\u0026#34;/\u0026#34;} / node_filesystem_size_bytes{mountpoint=\u0026#34;/\u0026#34;}) \u0026lt; 0.1 for: 5m labels: severity: critical annotations: summary: \u0026#34;Host disk almost full\u0026#34; description: \u0026#34;Root filesystem has less than 10% free space.\u0026#34; - alert: HostMemoryLow expr: node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes \u0026lt; 0.1 for: 5m labels: severity: critical annotations: summary: \u0026#34;Host memory critically low\u0026#34; description: \u0026#34;Available memory is below 10% of total.\u0026#34; Grafana Data Source and Dashboard Provisioning Grafana provisioning allows you to pre-configure data sources and dashboards from YAML files — no manual clicking after deployment.\nCreate the provisioning directory structure:\n1 2 3 mkdir -p grafana/provisioning/datasources mkdir -p grafana/provisioning/dashboards mkdir -p grafana/dashboards datasources/prometheus.yml 1 2 3 4 5 6 7 8 9 apiVersion: 1 datasources: - name: Prometheus type: prometheus access: proxy url: http://prometheus:9090 isDefault: true editable: false dashboards/dashboard-provider.yml 1 2 3 4 5 6 7 8 9 10 11 apiVersion: 1 providers: - name: \u0026#34;Homelab Dashboards\u0026#34; orgId: 1 folder: \u0026#34;Homelab\u0026#34; type: file disableDeletion: false updateIntervalSeconds: 30 options: path: /var/lib/grafana/dashboards Import Community Dashboards Grafana community dashboards provide immediate visibility without building from scratch. Download them and place them in grafana/dashboards/:\n1 2 3 4 5 6 7 8 9 10 11 # Node Exporter Full (ID 1860) curl -sL \u0026#34;https://grafana.com/api/dashboards/1860/revisions/37/download\u0026#34; \\ -o grafana/dashboards/node_exporter_full.json # Docker Monitoring (ID 179) curl -sL \u0026#34;https://grafana.com/api/dashboards/179/revisions/24/download\u0026#34; \\ -o grafana/dashboards/docker_monitoring.json # Prometheus 2.0 Overview (ID 3662) curl -sL \u0026#34;https://grafana.com/api/dashboards/3662/revisions/7/download\u0026#34; \\ -o grafana/dashboards/prometheus_overview.json These JSON files are the complete dashboard definitions. Grafana will import them automatically on startup and keep them in sync via the provisioning provider.\nConfiguring Alertmanager for Email and Telegram Create alertmanager.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 global: smtp_smarthost: \u0026#34;${SMTP_HOST}:${SMTP_PORT}\u0026#34; smtp_from: \u0026#34;Alertmanager \u0026lt;${SMTP_USER}\u0026gt;\u0026#34; smtp_auth_username: \u0026#34;${SMTP_USER}\u0026#34; smtp_auth_password: \u0026#34;${SMTP_PASS}\u0026#34; smtp_require_tls: true resolve_timeout: 5m route: group_by: [\u0026#34;alertname\u0026#34;, \u0026#34;severity\u0026#34;] group_wait: 30s group_interval: 5m repeat_interval: 4h receiver: \u0026#34;default\u0026#34; routes: - match: severity: critical receiver: \u0026#34;critical\u0026#34; repeat_interval: 1h receivers: - name: \u0026#34;default\u0026#34; email_configs: - to: \u0026#34;${ALERT_EMAIL}\u0026#34; send_resolved: true - name: \u0026#34;critical\u0026#34; email_configs: - to: \u0026#34;${ALERT_EMAIL}\u0026#34; send_resolved: true webhook_configs: - url: \u0026#34;http://your-telegram-bot:8080/alert\u0026#34; send_resolved: true For Telegram notifications, deploy a simple webhook bridge container:\n1 2 3 4 5 6 7 8 9 10 11 12 13 services: alertmanager-bridge: image: webhippie/alertmanager-bot:latest container_name: alertmanager-bot environment: ALERTMANAGER_URL: \u0026#34;http://alertmanager:9093\u0026#34; BOT_TOKEN: \u0026#34;${TELEGRAM_BOT_TOKEN}\u0026#34; BOT_WEBHOOK_URL: \u0026#34;https://your-domain.com/alertmanager-bot\u0026#34; BOT_STORE: \u0026#34;/data/bot\u0026#34; volumes: - bot-data:/data/bot networks: - monitoring Verify the Stack — Testing End to End Deploy everything:\n1 2 cd ~/docker/monitoring docker compose up -d Wait 30 seconds for first scrape cycle, then check each component:\nPrometheus Targets 1 curl -s http://localhost:9090/api/v1/targets | python3 -m json.tool Or open http://your-host:9090/targets in a browser. All three targets (prometheus, cadvisor, node) should show state UP.\nQuery PromQL 1 2 3 4 5 6 7 8 9 # Container CPU usage (total across all cores, last 5m) curl -s \u0026#39;http://localhost:9090/api/v1/query?query=rate(container_cpu_usage_seconds_total[5m])\u0026#39; | python3 -c \u0026#34; import sys, json data = json.load(sys.stdin) for r in data[\u0026#39;data\u0026#39;][\u0026#39;result\u0026#39;]: name = r[\u0026#39;metric\u0026#39;].get(\u0026#39;name\u0026#39;, \u0026#39;unknown\u0026#39;) val = float(r[\u0026#39;value\u0026#39;][1]) print(f\u0026#39;{name}: {val:.3f} cores\u0026#39;) \u0026#34; Host memory 1 2 3 4 5 6 curl -s \u0026#39;http://localhost:9090/api/v1/query?query=node_memory_MemAvailable_bytes\u0026#39; | python3 -c \u0026#34; import sys, json data = json.load(sys.stdin) avail = int(data[\u0026#39;data\u0026#39;][\u0026#39;result\u0026#39;][0][\u0026#39;value\u0026#39;][1]) print(f\u0026#39;Available memory: {avail / 1024**3:.1f} GB\u0026#39;) \u0026#34; Grafana Open http://your-host:3000 and log in with admin / ${GF_ADMIN_PASSWORD}. The Home dashboard should show provisioned dashboards under the \u0026ldquo;Homelab\u0026rdquo; folder.\nTest Alerts Stop a container to trigger an alert:\n1 docker stop cadvisor Alertmanager should fire the ContainerDown alert within ~2 minutes (60s of not seeing the container + 1m for duration). Check http://your-host:9093/#/alerts.\nRestart cAdvisor when done:\n1 docker start cadvisor The alert should auto-resolve within 5 minutes (the resolve_timeout in alertmanager.yml).\nMaintenance and Best Practices Prometheus Storage Management Prometheus stores all data in its TSDB. With the config above, data is retained for 30 days or until the volume reaches 10 GB, whichever comes first. Adjust for your hardware:\n1 2 3 # In docker-compose.yml command section: --storage.tsdb.retention.time=60d --storage.tsdb.retention.size=50GB Monitor the TSDB size with a PromQL query in Grafana:\nprometheus_tsdb_storage_blocks_bytes Resource Usage of the Monitoring Stack The stack itself is lightweight:\nService Memory CPU (idle) Prometheus 150-300 MB ~0.01 core cAdvisor 50-80 MB ~0.02 core node_exporter 15-25 MB \u0026lt;0.01 core Grafana 80-150 MB ~0.01 core Alertmanager 10-20 MB \u0026lt;0.01 core Total: ~350-600 MB RAM and negligible CPU at idle. Spikes only during query execution or dashboard loading.\nSecuring Grafana Behind Traefik If you already run Traefik in your homelab, add Traefik labels to the Grafana service:\n1 2 3 4 5 6 7 8 9 10 11 services: grafana: labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.grafana.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.grafana.rule=Host(`grafana.yourlab.com`)\u0026#34; - \u0026#34;traefik.http.routers.grafana.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.services.grafana.loadbalancer.server.port=3000\u0026#34; networks: - monitoring - traefik-public Combine with Authelia for SSO authentication. Remove the published 3000:3000 port when behind Traefik.\nDashboard Backup Export Grafana dashboards periodically:\n1 2 docker exec grafana grafana-cli admin export-dashboards /tmp/backup docker cp grafana:/tmp/backup ./dashboards-export-$(date +%Y%m%d).zip For automated backups, schedule a cron job or add a backup service to the compose stack that runs curl against the Grafana API.\nExtending the Stack — Custom Exporters Once the base stack is running, adding new data sources is straightforward:\nMikroTik metrics: Deploy the Prometheus SNMP Exporter with a MikroTik MIB config:\n1 2 3 4 docker run -d --name snmp-exporter \\ -v $(pwd)/snmp.yml:/etc/snmp_exporter/snmp.yml:ro \\ -p 9116:9116 \\ prom/snmp-exporter:latest Add to prometheus.yml:\n1 2 3 4 - job_name: \u0026#34;mikrotik\u0026#34; scrape_interval: 60s static_configs: - targets: [\u0026#34;10.0.20.1:9116\u0026#34;] Proxmox host metrics: Enable the Proxmox VE metrics exporter and add to Prometheus scrape config.\nDocker daemon metrics: Enable Docker\u0026rsquo;s built-in Prometheus endpoint by adding to /etc/docker/daemon.json on the host:\n1 2 3 4 { \u0026#34;metrics-addr\u0026#34;: \u0026#34;0.0.0.0:9323\u0026#34;, \u0026#34;experimental\u0026#34;: true } Then add a scrape job for host:9323.\nSummary: Full Homelab Observability in Under 30 Minutes This stack gives you persistent metrics, visual dashboards, and alerting for your entire Docker homelab — all self-hosted, all running as containers, all configured declaratively.\nStartup sequence:\n1 2 3 4 5 6 7 8 9 10 11 mkdir -p ~/docker/monitoring \u0026amp;\u0026amp; cd ~/docker/monitoring # Create all config files (prometheus.yml, alerts.yml, alertmanager.yml, grafana provisioning) # Then: docker compose up -d # Verify curl -s http://localhost:9090/targets | grep -c \u0026#39;\u0026#34;health\u0026#34;:\u0026#34;up\u0026#34;\u0026#39; # Should output: 3 open http://localhost:3000 # Grafana What you get for the ~600 MB RAM investment:\nPer-container CPU, memory, network, and disk metrics (from cAdvisor) Host CPU, memory, disk space, and load metrics (from node_exporter) Historical queries via PromQL with configurable retention Alerting for container crashes, resource exhaustion, and disk space Extensible: add custom exporters for MikroTik, Proxmox, UPS, or any Prometheus-compatible endpoint All configuration in one directory, backed up with the rest of your Docker project files The monitoring stack itself needs monitoring. Add a Uptime Kuma probe for each service endpoint, or configure Prometheus to scrape itself and alert on any target going down. In a homelab, the most important alert is \u0026ldquo;Prometheus stopped scraping\u0026rdquo; — because without it, you\u0026rsquo;re flying blind.\n1 2 3 4 5 # Quick health check for the whole stack for svc in prometheus cadvisor node-exporter grafana alertmanager; do status=$(docker inspect --format=\u0026#39;{{.State.Status}}\u0026#39; $svc 2\u0026gt;/dev/null || echo \u0026#34;missing\u0026#34;) echo \u0026#34;$svc: $status\u0026#34; done ","permalink":"https://blog.gntech.me/posts/2026-06-15-docker-monitoring-prometheus-grafana-homelab/","summary":"Deploy Prometheus, Grafana, cAdvisor, and node_exporter via Docker Compose for full homelab monitoring. Real configs for metrics collection, dashboards, and alerting.","title":"Docker Monitoring with Prometheus and Grafana in Homelab"},{"content":"Firewalls are the single most important security device in any network. For years, dedicated hardware was the only serious option — a Protectli, a Netgate, or a MikroTik running the show. But if you already run Proxmox, virtualizing your firewall is cheaper, more flexible, and arguably more secure when done right.\nThis guide covers a production-grade OPNsense deployment as a Proxmox VM with:\nVM setup with virtio NICs and optimal resource allocation WAN/LAN bridge topology on the Proxmox host VLAN trunking and inter-VLAN firewall rules Suricata IDS/IPS inline on LAN interfaces WireGuard VPN for remote access DNS, DHCP, and security hardening Why OPNsense for a Virtualized Firewall? OPNsense is a FreeBSD-based firewall that started as a pfSense fork but has since surpassed it in several areas: a cleaner web UI, regular 2-week release cycles, built-in WireGuard, native Suricata in business edition, and a plugin system without bloat. For a Proxmox homelab, it fills the gap between MikroTik RouterOS (powerful but CLI-heavy) and pfSense (stable but slower-moving).\nCompared to running a MikroTik virtualized, OPNsense gives you:\nFull IDS/IPS with Suricata or Zenarmor (Sensei) WireGuard built into the kernel (MikroTik added it later, but CLI-only initially) Unbound DNS with DNSSEC, blocklists, and overrides baked in VLAN management through a web GUI — no /interface vlan commands Proxmox VM Setup for OPNsense VM Specifications Create the VM with these specs:\nCPU: 2 vCPU cores (host type for AES-NI access) RAM: 4 GB minimum, 8 GB if running Suricata in IPS mode Disk: 32 GB on fast storage (ZFS mirror or NVMe) NIC1 (WAN): VirtIO (paravirtualized) — bridged to vmbr0 NIC2 (LAN): VirtIO — bridged to vmbr1 Enable hardware virtualization: check \u0026ldquo;Processor\u0026rdquo; options for aes flags Proxmox Bridge Configuration Before creating the VM, set up two bridges on your Proxmox host. Modify /etc/network/interfaces:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # WAN — connected to your modem/ONT auto vmbr0 iface vmbr0 inet manual bridge-ports enp3s0 bridge-stp off bridge-fd 0 bridge-vlan-aware yes # LAN — trunk port to your switch auto vmbr1 iface vmbr1 inet manual bridge-ports enp4s0 bridge-stp off bridge-fd 0 bridge-vlan-aware yes Critical detail: do NOT assign an IP address to either bridge on the Proxmox host. The host\u0026rsquo;s management IP should be on a VLAN interface within OPNsense. This keeps Proxmox behind the firewall.\nInstall OPNsense 1 2 3 4 5 6 7 # Download the latest OPNsense ISO wget https://mirror.ams1.nl.leaseweb.net/opnsense/releases/25.1/OPNsense-25.1-amd64.iso # Import ISO to Proxmox storage qm importdisk 100 OPNsense-25.1-amd64.iso local-zfs # Or attach the ISO directly to the VM and boot from it Boot the VM, attach to console, and run through the installer:\nSelect ZFS as filesystem (auto layout) Set root password Reboot, remove ISO After first reboot, assign interfaces:\nWAN: vtnet0 (your NIC1 / vmbr0) LAN: vtnet1 (your NIC2 / vmbr1) Set the LAN IP to 10.0.0.1/24 temporarily for first web GUI access, then immediately change it to a more structured IP like 10.0.10.1/24 for your management VLAN.\nVLAN Trunk Configuration From the OPNsense web GUI, create VLANs on the LAN interface (Interfaces → Other Types → VLAN):\nVLAN ID Name Description 10 mgmt Management — servers, hypervisors 20 trusted Trusted user devices, workstations 30 iot IoT devices — cameras, sensors, smart plugs 40 guest Guest WiFi — internet only, isolated 99 dmz DMZ — public-facing services After creating VLANs, assign each one (Interfaces → Assignments), enable them, and set IPs/DHCP:\n1 2 3 4 5 mgmt: 10.0.10.1/24 trusted: 10.0.20.1/24 iot: 10.0.30.1/24 guest: 10.0.40.1/24 dmz: 10.0.99.1/24 Inter-VLAN Firewall Rules — Lock It Down By default, OPNsense allows all traffic. You need explicit deny rules. The approach:\nDefault deny all inter-VLAN traffic at the top of each VLAN interface Allow established/related state tracking Explicit allow only what\u0026rsquo;s needed Example — IoT VLAN rules (interface: IoT, direction: in):\n1 2 3 4 5 6 7 8 9 10 11 # Rule 1: Allow established traffic back in IPv4 * * * * * * * * Keep state # Rule 2: Block all inter-VLAN by default IPv4 * * * 10.0.0.0/8 * * * * * * Block # Rule 3: Allow IoT to reach DNS on mgmt VLAN IPv4 UDP * * 10.0.10.53 53 * * * * Allow # Rule 4: Allow Internet access IPv4 * * * !10.0.0.0/8 * * * * * Allow The !10.0.0.0/8 in Rule 4 means \u0026ldquo;destination is NOT in RFC1918 space\u0026rdquo; — internet only. This keeps IoT devices from phoning home to your servers while still getting firmware updates.\nGuest VLAN rules — allow DNS and internet only:\n1 2 3 4 5 6 7 8 9 10 # Allow DNS (OPNsense resolver) IPv4 UDP * * 10.0.10.1 53 * * * * Allow # Block all private ranges IPv4 * * * 10.0.0.0/8 * * * * * Block IPv4 * * * 172.16.0.0/12 * * * * * Block IPv4 * * * 192.168.0.0/16 * * * * * Block # Allow internet IPv4 * * * * * * * * Allow Suricata IDS/IPS — Inline on LAN Install the Suricata plugin: System → Firmware → Plugins → os-suricata.\nSuricata Configuration Navigate to Services → Suricata → Settings:\n1 2 3 4 5 6 7 8 9 10 Interface: vtnet1 (LAN parent — captures all VLAN traffic) Pattern match limit: 50000000 Defrag: enabled Stream engine: enabled Inline mode: IPS (enabled) Rule categories: - enable: emerging-exploit, emerging-malware, emerging-attack-response - enable: emerging-policy, emerging-scan - disable: emerging-info (noise, false positives) - disable: audio, video, chat (not relevant) After enabling, go to the Updates tab and download the Emerging Threats Open ruleset:\n1 2 # OPNsense auto-downloads on schedule, but manual download after config # The output log lives at /var/log/suricata/suricata.log Reducing False Positives Suricata in IPS mode will block traffic matching rules. Start in IDS (alert-only) mode for 48 hours:\n1 2 3 4 5 6 # Check alerts via web GUI Services → Suricata → Alerts # Add suppressions for known local traffic patterns # Services → Suricata → Suppress → Add pass rule # Example: suppress SID:2013026 (DNS query for local domain) Common suppressions for homelabs:\nSMB traffic between your workstation and NAS Plex media streaming to LAN clients DNS queries to your internal resolver After tuning, switch to IPS mode. Suricata IPS on a 4-core VM handles 500 Mbps comfortably. For 1 Gbps, give it 4 vCPU and verify your NICs use VirtIO (not e1000).\nWireGuard VPN Server OPNsense includes WireGuard in the base install — no plugin needed since 25.1. Navigate to VPN → WireGuard.\nServer Config 1 2 3 4 5 6 7 # Instance settings Enabled: yes Listen port: 51820 Tunnel address: 10.200.0.1/24 Private key: (auto-generated, back this up) MTU: 1420 DNS servers: 10.0.10.1, 1.1.1.1 Peer Config 1 2 3 4 5 # Peer settings Enabled: yes Public key: (from your client, e.g., `wg genkey | tee privatekey | wg pubkey \u0026gt; publickey`) Preshared key: (generate with `wg genpsk`) Allowed IPs: 10.200.0.2/32, 10.0.0.0/8 The Allowed IPs include the WireGuard tunnel address and your entire LAN. This means when connected, your client can reach any device in any VLAN.\nWAN Firewall Rule 1 2 3 4 5 6 # Firewall → Rules → WAN → Add Protocol: UDP Source: any Destination: WAN address Destination port: 51820 Description: Allow WireGuard NAT for VPN Clients If you want VPN clients to reach the internet through OPNsense, add an outbound NAT rule:\n1 2 3 4 5 6 Firewall → NAT → Outbound → Add Interface: WAN Source: 10.200.0.0/24 Destination: any NAT address: WAN address Static port: no DNS and DHCP Best Practices OPNsense runs Unbound as the local DNS resolver. Enable it with blocklists:\nServices → Unbound DNS → General:\n1 2 3 4 5 Enable: yes Port: 53 (leave default) DNSSEC: enabled Register DHCP leases: yes Register DHCP static mappings: yes DNS over TLS (DoT) for upstream queries:\n1 2 3 4 DNS Query Forwarding: enabled DNS Query Forwarding servers: 1.1.1.1@853 (Cloudflare) 9.9.9.9@853 (Quad9) DHCP static mappings per VLAN:\n1 2 3 4 5 Services → DHCPv4 → [mgmt] Create static mapping for each server: proxmox1 → 10.0.10.30 nas-01 → 10.0.10.31 docker1 → 10.0.10.32 With \u0026ldquo;Register DHCP static mappings\u0026rdquo; enabled, Unbound automatically adds A and PTR records for every static lease. No manual /etc/hosts editing.\nSecurity Hardening Checklist Before putting this firewall into production, lock it down:\nSystem → Settings → Administration: disable web GUI on WAN and change port to 8443 System → Settings → SSH: bind to LAN only, disable password auth, use key-only System → Access → Users: enable 2FA (TOTP) for admin and your user Firewall → Settings: disable ICMP redirects, disable directed broadcast System → Firmware → Settings: set update branch to \u0026ldquo;Business\u0026rdquo; (stable, 2-week cycle) System → Settings → Tunables: enable TCP SYN cookies, set net.inet.tcp.blackhole to 2 To verify your hardening from outside, use a VPS or mobile hotspot (not on your network):\n1 2 nmap -sS -p 22,443,8443 \u0026#34;\u0026lt;your-public-ip\u0026gt;\u0026#34; # All should show filtered Monitoring and Logging OPNsense\u0026rsquo;s firewall live view is useful for debugging, but for long-term retention, export logs:\nSyslog to Loki/Grafana — configure in Services → Syslog → Settings:\n1 2 3 4 Syslog server: 10.0.10.50:514 (your Grafana Alloy/Loki host) Transport: UDP Syslog facility: local7 Content filter: enabled — log only blocked traffic and errors Netflow export — install os-netflow plugin and send to Grafana via:\n1 2 3 4 Services → Netflow → General Protocol: UDP Destination: 10.0.10.50:2055 Engine ID: 1 Conclusion Virtualizing OPNsense on Proxmox is not a compromise. With VirtIO NICs, proper bridge topology, and enough resources for Suricata, your virtual firewall outperforms many dedicated appliances. The real value is in the flexibility — you can snapshot the firewall VM before config changes, clone it for testing, and migrate it between Proxmox hosts.\nFor your first week, run Suricata in IDS-only mode while tuning down false positives. Lock down WAN access to the admin GUI immediately. Start with management and IoT VLANs, then expand to guest WiFi and DMZ as you gain confidence.\nNext steps: add CARP for high-availability with a second Proxmox host, integrate with Authentik for LDAP authentication, and set up Zenarmor (Sensei) for application-level traffic control.\n","permalink":"https://blog.gntech.me/posts/2026-06-15-opnsense-virtual-firewall-proxmox-homelab/","summary":"Deploy OPNsense as a virtualized firewall on Proxmox with VLAN segmentation, Suricata IDS/IPS, and WireGuard VPN. Real VM configs, bridge setup, firewall rules, and performance tuning.","title":"OPNsense Virtual Firewall on Proxmox — Homelab Security"},{"content":"Your homelab runs monitoring checks, backups, health probes, and automation workflows — each one generating output that you need to see. The typical approach is checking dashboards, tailing logs, or waiting for an email that might land in spam. In 2026, with homelabs running dozens of services, reactive monitoring is dead. You need push notifications that reach your pocket instantly.\nntfy.sh is the simplest self-hosted push notification server you can run. A single HTTP POST sends a notification to your phone. No SDKs, no SDK registration, no per-app tokens. One curl command and you\u0026rsquo;re done.\nThis guide covers deploying ntfy on Docker Compose, setting up authentication, and wiring it into your monitoring stack — Uptime Kuma, Prometheus Alertmanager, Grafana, cron jobs, and every script you\u0026rsquo;ve ever written.\nWhy ntfy Instead of Gotify, Apprise, or Pushover There are several options for self-hosted notifications. Here\u0026rsquo;s how they compare:\nntfy.sh — Single-binary server with a dead-simple HTTP API. Push via curl -d \u0026quot;message\u0026quot; ntfy.sh/topic. No client library needed. Native Android and iOS apps with persistent connections. Open source (Apache 2.0), ~10MB binary. Gotify — Mature, websocket-based. Requires a client app on the phone. Docker container ~15MB. Excellent, but the push mechanism is slightly more involved (requires the Gotify client to maintain a websocket). Apprise — A notification library, not a server. Wraps dozens of providers (Slack, Telegram, email). Great as a relay, but you still need something to trigger it. Pushover — Hosted service, $5/mo. Works great but locks you into a third party. ntfy wins for sheer simplicity. The entire API is POST /{topic} with the message body. It supports priorities, tags, markdown, click actions, and attachments — all via HTTP headers. For a homelab where you want notifications from bash scripts without installing anything, ntfy is the clear choice.\nDeploying ntfy on Docker Compose Here is the complete docker-compose.yml for a production-ready ntfy deployment behind Traefik:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 services: ntfy: image: binwiederhier/ntfy:latest container_name: ntfy restart: unless-stopped volumes: - ./data/ntfy/cache:/var/cache/ntfy - ./data/ntfy/auth:/var/lib/ntfy - ./data/ntfy/config:/etc/ntfy environment: - TZ=America/Santo_Domingo - NTFY_BASE_URL=https://ntfy.gntech.me - NTFY_BEHIND_PROXY=true - NTFY_CACHE_FILE=/var/cache/ntfy/cache.db - NTFY_AUTH_FILE=/var/lib/ntfy/auth.db - NTFY_AUTH_DEFAULT_ACCESS=deny-all healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;wget\u0026#34;, \u0026#34;-q\u0026#34;, \u0026#34;--tries=1\u0026#34;, \u0026#34;--spider\u0026#34;, \u0026#34;http://localhost:80/v1/health\u0026#34;] interval: 30s timeout: 10s retries: 3 logging: driver: \u0026#34;json-file\u0026#34; options: max-size: \u0026#34;10m\u0026#34; max-file: \u0026#34;3\u0026#34; networks: - proxy # Optional: subscribe topic for mobile test ntfy-sub: image: alpine:latest entrypoint: [\u0026#34;/bin/sh\u0026#34;, \u0026#34;-c\u0026#34;, \u0026#34;apk add curl; while true; do sleep 86400; done\u0026#34;] profiles: [\u0026#34;tools\u0026#34;] networks: - proxy networks: proxy: external: true Nginx Proxy or Traefik Configuration If you use Traefik, add these labels:\n1 2 3 4 5 6 labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.ntfy.rule=Host(`ntfy.gntech.me`)\u0026#34; - \u0026#34;traefik.http.routers.ntfy.entrypoints=https\u0026#34; - \u0026#34;traefik.http.routers.ntfy.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.services.ntfy.loadbalancer.server.port=80\u0026#34; For Caddy, a minimal Caddyfile:\nntfy.gntech.me { reverse_proxy ntfy:80 } Authentication Setup Deploy first with NTFY_AUTH_DEFAULT_ACCESS=deny-all (as shown above), then create users and tokens:\n1 2 3 4 5 6 7 8 9 10 11 12 # Access the container shell docker exec -it ntfy /bin/sh # Create admin user (interactive) ntfy user add --role=admin admin # Create a token for monitoring tools (non-interactive) ntfy token add --label=uptime-kuma --role=read-write alerts ntfy token add --label=backup-scripts backup-notify # List existing tokens ntfy token list Store the tokens securely. Each token grants access to specific topics. The format is: tk_xxxxxxxxxxxxx.\nWithout auth enabled, anyone who can reach your ntfy instance can publish. With deny-all as default, you control exactly which services can push and which topics they can access.\nPublishing Notifications — The API ntfy\u0026rsquo;s API is refreshingly simple. No JSON body required unless you need structured content:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 # Basic notification curl -d \u0026#34;Backup of PostgreSQL completed successfully\u0026#34; \\ https://ntfy.gntech.me/alerts # With priority and tags curl -H \u0026#34;Priority: urgent\u0026#34; \\ -H \u0026#34;Tags: floppy_disk\u0026#34; \\ -H \u0026#34;Title: Backup Failed\u0026#34; \\ -d \u0026#34;ZFS snapshot replication to offsite failed - check /var/log/zfs\u0026#34; \\ https://ntfy.gntech.me/alerts # Markdown body with click action curl -H \u0026#34;Content-Type: text/markdown\u0026#34; \\ -H \u0026#34;Click: https://grafana.gntech.me/d/backups\u0026#34; \\ -H \u0026#34;Tags: warning\u0026#34; \\ -d \u0026#34;**Backup warning:** Last run took 3x normal time.\\n\\nSee the [dashboard](https://grafana.gntech.me/d/backups) for details.\u0026#34; \\ https://ntfy.gntech.me/alerts # Authenticated request (token as Bearer or Basic) curl -H \u0026#34;Authorization: Bearer tk_xxxxxxxxxxxxx\u0026#34; \\ -H \u0026#34;Priority: high\u0026#34; \\ -H \u0026#34;Tags: fire\u0026#34; \\ -d \u0026#34;CPU temperature exceeded 85°C on SRV1\u0026#34; \\ https://ntfy.gntech.me/backup-notify Priority levels available: urgent, high, default, low, min. Each maps to how aggressively your phone notifies you — urgent bypasses Do Not Disturb on Android.\nIntegrating with Your Monitoring Stack Uptime Kuma Uptime Kuma natively supports ntfy as a notification provider:\nGo to Settings \u0026gt; Notifications Click Add Notification Select ntfy from the provider list Set URL to your ntfy instance: https://ntfy.gntech.me Set Topic to alerts Set Token if authentication is enabled Select priority (default: high) Click Test — you should receive a test notification Kuma sends status changes (down → up, up → down) directly to your phone with the service name and current status.\nPrometheus Alertmanager Alertmanager supports generic webhook receivers, which work perfectly with ntfy. Add this to alertmanager.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 route: receiver: ntfy-alerts repeat_interval: 4h receivers: - name: ntfy-alerts webhook_configs: - url: https://ntfy.gntech.me/alerts http_config: headers: Authorization: \u0026#34;Bearer tk_xxxxxxxxxxxxx\u0026#34; send_resolved: true Alertmanager will POST each alert group as JSON. ntfy will render the JSON body, which is readable enough. For cleaner messages, use Alertmanager\u0026rsquo;s text template field or pipe through a small middleware.\nGrafana Contact Points Grafana supports webhook contact points:\nAlerting \u0026gt; Contact points → Add contact point Name: ntfy Integration: Webhook URL: https://ntfy.gntech.me/alerts HTTP method: POST Add custom header: Authorization: Bearer tk_xxxxxxxxxxxxx Optional message template — Grafana\u0026rsquo;s templating lets you build cleaner notifications: {{ define \u0026#34;ntfy/message\u0026#34; }} {{ range .Alerts }} **{{ .Labels.alertname }}** Severity: {{ .Labels.severity }} Instance: {{ .Labels.instance }} {{ .Annotations.summary }} {{ end }} {{ end }} Docker Container Healthchecks Docker healthchecks can push to ntfy on failure. Add this to any compose service:\n1 2 3 4 5 6 healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;curl\u0026#34;, \u0026#34;-f\u0026#34;, \u0026#34;http://localhost:8080/health\u0026#34;] interval: 30s timeout: 10s retries: 3 start_period: 40s Then a separate cron or watchdog checks health status and alerts:\n1 2 3 4 5 6 7 8 9 10 11 12 13 #!/bin/bash # /usr/local/bin/ntfy-healthwatch.sh NTFY_URL=\u0026#34;https://ntfy.gntech.me/alerts\u0026#34; NTFY_TOKEN=\u0026#34;tk_xxxxxxxxxxxxx\u0026#34; docker ps --filter \u0026#34;health=unhealthy\u0026#34; --format \u0026#34;{{.Names}}\u0026#34; | while read container; do curl -s -H \u0026#34;Authorization: Bearer $NTFY_TOKEN\u0026#34; \\ -H \u0026#34;Priority: high\u0026#34; \\ -H \u0026#34;Tags: warning\u0026#34; \\ -H \u0026#34;Title: Container Unhealthy\u0026#34; \\ -d \u0026#34;Container $container is unhealthy on $(hostname)\u0026#34; \\ \u0026#34;$NTFY_URL\u0026#34; done Run it every 5 minutes via a systemd timer or crontab:\n1 */5 * * * * /usr/local/bin/ntfy-healthwatch.sh Shell Script Notification Function Add this to your .bashrc or common library:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 ntfy_send() { local topic=\u0026#34;${1:-alerts}\u0026#34; local message=\u0026#34;$2\u0026#34; local priority=\u0026#34;${3:-default}\u0026#34; local title=\u0026#34;${4:-Notification}\u0026#34; local tags=\u0026#34;${5:-}\u0026#34; local token=\u0026#34;${NTFY_TOKEN:-}\u0026#34; curl -s -o /dev/null \\ ${token:+-H \u0026#34;Authorization: Bearer $token\u0026#34;} \\ -H \u0026#34;Priority: $priority\u0026#34; \\ -H \u0026#34;Title: $title\u0026#34; \\ ${tags:+-H \u0026#34;Tags: $tags\u0026#34;} \\ -d \u0026#34;$message\u0026#34; \\ \u0026#34;https://ntfy.gntech.me/$topic\u0026#34; } # Usage ntfy_send \u0026#34;backup-notify\u0026#34; \u0026#34;Restic backup completed - 1.2GB uploaded\u0026#34; \u0026#34;default\u0026#34; \u0026#34;Backup OK\u0026#34; \u0026#34;white_check_mark\u0026#34; One function, zero dependencies, works from any shell script.\nAdvanced Configuration Topic-Based Access Control Fine-tune who can publish and subscribe to each topic:\n1 2 3 4 5 # Create a read-only subscriber for dashboards ntfy token add --label=dashboard-monitor --role=read-only server-status # Create a write-only publisher for backup scripts ntfy token add --label=backup-agent --role=write-only backup-notify Topic reservations ensure only authorized tokens can use specific topic names:\n1 2 3 4 # Run in container ntfy topic add alerts ntfy topic add backup-notify ntfy topic add server-status Rate Limiting and Caching ntfy caches messages by default so new subscribers see recent notifications. Control cache size in server.yml:\n1 2 3 4 5 6 7 8 9 # /etc/ntfy/server.yml (bind-mounted) cache-file: /var/cache/ntfy/cache.db cache-duration: 12h cache-startup-queries-as-seen: false message-delay-interval: 5s rate-limit: 60 rate-limit-burst: 120 visitor-subscription-rate-limit: 10 visitor-request-tokens-burst: 120 Behind Traefik with Rate Limiting Add a middleware to protect ntfy from abuse:\n1 2 3 4 labels: - \u0026#34;traefik.http.middlewares.ntfy-ratelimit.ratelimit.average=30\u0026#34; - \u0026#34;traefik.http.middlewares.ntfy-ratelimit.ratelimit.burst=60\u0026#34; - \u0026#34;traefik.http.routers.ntfy.middlewares=ntfy-ratelimit\u0026#34; Mobile Setup — Push Where It Matters Install ntfy from F-Droid, Google Play, or Apple App Store Tap Add topic Enter your server URL: https://ntfy.gntech.me Topic: alerts If auth is enabled, add your read token Configure per-topic settings: alerts: priority threshold \u0026ldquo;high\u0026rdquo; (only wake me for real problems) backup-notify: priority threshold \u0026ldquo;default\u0026rdquo; (backup completion, not urgent) server-status: mute during night (silent maintenance messages) Security Hardening Secure your ntfy instance against unwanted access:\n1 2 3 4 5 6 7 # Block direct IP access, force through reverse proxy # In server.yml behind-proxy: true base-url: \u0026#34;https://ntfy.gntech.me\u0026#34; # Firewall: only allow from Docker proxy network ufw deny from any to 10.0.20.50 port 2586 If you don\u0026rsquo;t want authentication, restrict by network instead:\n1 2 3 4 5 6 # Only accept connections from your reverse proxy networks: - proxy # Do NOT expose ports directly expose: - \u0026#34;80\u0026#34; For the reverse proxy, add IP allowlists:\n# Caddy ntfy.gntech.me { @internal remote_ip 10.0.0.0/8 192.168.0.0/16 172.16.0.0/12 handle @internal { reverse_proxy ntfy:80 } respond 403 } Monitoring ntfy Itself ntfy exposes a health endpoint at /v1/health:\n1 2 curl https://ntfy.gntech.me/v1/health # Returns {\u0026#34;health\u0026#34;: true, \u0026#34;uptime\u0026#34;: 123456} Add this to your Uptime Kuma monitor or Prometheus blackbox target:\n1 2 3 4 5 6 7 8 9 # prometheus.yml scrape_configs: - job_name: ntfy-health metrics_path: /v1/health static_configs: - targets: [\u0026#34;ntfy.gntech.me\u0026#34;] relabel_configs: - source_labels: [__address__] target_label: __param_target Conclusion Self-hosted push notifications with ntfy transform how you interact with your homelab. Instead of checking dashboards, notifications come to you. Instead of email gateways that silently fail, a single curl command reaches your phone with priority, tags, and actionable links.\nThe setup is minimal — one Docker Compose file, one docker compose up -d, and a few tokens. The integrations cover Uptime Kuma, Prometheus Alertmanager, Grafana, and every cron job or script you write. For a service that costs nothing to run, ntfy pays for itself the first time it alerts you to a failed backup before your data is gone.\nStart with a simple deployment, add authentication, then wire in your monitoring tools one by one. Your homelab will thank you — and so will your sleep when only the urgent alerts wake you up.\n","permalink":"https://blog.gntech.me/posts/2026-06-15-ntfy-self-hosted-push-notification-server-homelab/","summary":"Deploy ntfy.sh on Docker for push notifications across your entire homelab. From Uptime Kuma to Prometheus Alertmanager, get real-time mobile alerts for everything with a single HTTP POST.","title":"ntfy Self-Hosted Push Notification Server for Homelab Alerts"},{"content":"Tu homelab probablemente corre Docker. Y Docker funciona bien — hasta que un container comprometido escala a root via el daemon de Docker, o un CVE como el reciente CVE-2026-31431 en Docker Engine te hace reconsiderar tu postura de seguridad. Docker corre como root, y si alguien rompe el container, el daemon es el único muro entre ese container y tu host.\nPodman resuelve esto de raíz: sin daemon, sin root, con la misma CLI de Docker y compatibilidad con docker-compose.yml. En 2026, con la atención puesta en seguridad de containers, Podman ya no es una alternativa experimental — es producción-ready para homelabs.\nEsta guía cubre instalación, podman-compose, Quadlets systemd, rootless networking, y cómo migrar tu homelab desde Docker sin romper nada.\nPodman vs Docker — Architecture and Security La diferencia fundamental no es lo que puedes correr, sino cómo lo corres.\nAspecto Docker Podman Arquitectura Cliente → dockerd (daemon root) → containerd → runc Cliente → conmon → runc (sin daemon) Modelo Client-server Fork/exec directo Root Daemon corre como root, containers heredan Rootless nativo, cada usuario corre sus containers Pods No nativo Sí (comparten network, IPC, UTS) Systemd Manual con restart scripts Quadlets nativos Docker usa un modelo client-server: cada docker run pasa por el daemon dockerd que corre como root. Si explotan el daemon —ya sea via un container malicioso o una API expuesta— tienen root en tu host.\nPodman usa fork/exec: el proceso podman run crea el container directamente usando conmon y runc. No hay daemon. En modo rootless, todo corre dentro de un user namespace, así que aunque escapen del container, solo tienen los permisos del usuario no-root.\nPara un homelab donde compartes acceso o expones servicios web, esta diferencia es enorme.\nInstalling Podman on Debian/Ubuntu En Debian 12, Ubuntu 24.04, o derivados, la instalación es directa:\n1 2 3 4 5 6 7 # Instalar Podman y podman-compose sudo apt update sudo apt install podman podman-compose -y # Verificar instalación podman --version podman info | grep rootless Podman viene preconfigurado para usar Docker Hub como registry por defecto. Si necesitas registries privados, edita /etc/containers/registries.conf:\n1 2 3 4 5 unqualified-search-registries = [\u0026#34;docker.io\u0026#34;] [[registry]] prefix = \u0026#34;docker.io\u0026#34; location = \u0026#34;docker.io\u0026#34; Para rootless, verifica que tu usuario tenga entradas en /etc/subuid y /etc/subgid:\n1 2 3 4 5 cat /etc/subuid # gntech:100000:65536 cat /etc/subgid # gntech:100000:65536 Estas entradas se crean automáticamente al instalar Podman. Si no existen, créalas con sudo usermod --add-subuids 100000-165535 --add-subgids 100000-165535 $(whoami).\nRunning Rootless Containers — First Steps La CLI de Podman es idéntica a Docker en el 99% de los casos. Literalmente puedes hacer alias docker=podman y seguir trabajando.\n1 2 3 4 5 6 7 8 9 # Pull y run como lo harías con Docker podman pull docker.io/library/nginx:alpine podman run -d --name web -p 8080:80 nginx:alpine # Comandos familiares podman ps podman logs -f web podman exec -it web sh podman stop web \u0026amp;\u0026amp; podman rm web La diferencia más notable en rootless: los puertos privilegiados (\u0026lt;1024) no funcionan sin configuración extra. Para exponer en puerto 80 o 443, necesitas:\n1 2 3 4 5 6 # Opción 1: Aumentar el límite de puertos privilegiados sudo sysctl -w net.ipv4.ip_unprivileged_port_start=80 echo \u0026#34;net.ipv4.ip_unprivileged_port_start=80\u0026#34; | sudo tee /etc/sysctl.d/99-unprivileged-ports.conf # Opción 2: Usar un reverse proxy como Caddy/Traefik en modo rootful que redirija # Opción 3: Usar systemd socket activation (más adelante) Docker Compose to podman-compose — Migrating Your Stacks Aquí es donde muchos homelabbers se preocupan. La buena noticia: podman-compose acepta el mismo docker-compose.yml que ya tienes.\n1 2 3 4 5 6 7 8 9 # Instalar podman-compose (ya viene con el paquete en Debian 12) # Si necesitas la última versión: pip3 install podman-compose # Usarlo exactamente como docker compose cd /opt/stacks/nginx-proxy-manager podman-compose up -d podman-compose ps podman-compose logs -f Tu docker-compose.yml existente funciona sin cambios para la mayoría de casos:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 version: \u0026#34;3.8\u0026#34; services: app: image: nginx:alpine ports: - \u0026#34;8080:80\u0026#34; volumes: - ./html:/usr/share/nginx/html db: image: postgres:16-alpine environment: POSTGRES_PASSWORD: changeme volumes: - pgdata:/var/lib/postgresql/data volumes: pgdata: Diferencias que encontré al migrar stacks reales:\nNetwork mode host no funciona en rootless Podman (usar --network slirp4netns:port_handler=slirp4netns) depends_on puede no respetar el orden de inicio — usar healthchecks explícitos ulimits necesita ajustes en rootless (algunos valores requieren --privileged) Podman Quadlets — systemd-Native Container Management Los Quadlets son, para mí, la razón más convincente para migrar a Podman. En lugar de scripts de restart o restart: always en compose, defines containers como archivos .container que systemd maneja nativamente.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 # ~/.config/containers/systemd/nginx.container [Unit] Description=Nginx reverse proxy After=network-online.target Wants=network-online.target [Container] Image=docker.io/library/nginx:alpine PublishPort=8080:80 Volume=/opt/nginx/html:/usr/share/nginx/html:Z Environment=NGINX_HOST=homelab.local AutoUpdate=registry [Service] Restart=always RestartSec=10 [Install] WantedBy=default.target Para activarlo:\n1 2 3 4 5 6 7 8 9 10 11 12 # Crear el directorio y archivo mkdir -p ~/.config/containers/systemd # Recargar systemd user units systemctl --user daemon-reload # Iniciar y habilitar systemctl --user start nginx systemctl --user enable nginx # Ver logs journalctl --user -u nginx -f Los Quadlets soportan también Pods (nginx.pod) y volumes (nginx.volume), permitiendo definir stacks completos como archivos systemd:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # ~/.config/containers/systemd/webstack.pod [Unit] Description=Web application stack [Pod] Network=webnet # ~/.config/containers/systemd/webapp.container [Unit] Description=Web application Pod=webstack.pod [Container] Image=docker.io/nginx:alpine Volume=/opt/app:/usr/share/nginx/html:Z Systemd maneja auto-start en boot, restart on failure, y logging integrado con journald. Sin scripts externos, sin agentes, sin nada más que systemd.\nRootless Networking — How It Works Podman rootless usa slirp4netns por defecto: un pequeño proceso userspace que hace NAT entre el namespace de red del container y la red del host. No requiere privilegios, pero tiene overhead de CPU y no soporta tráfico ICMP.\nDesde Podman 5.x, pasta es una alternativa más rápida:\n1 podman run -d --network pasta --name web -p 8080:80 nginx:alpine Para redes más avanzadas (macvlan, ipvlan):\n1 2 3 4 5 6 # Crear una red macvlan (rootless requiere configuración adicional) podman network create -d macvlan \\ --subnet 10.0.20.0/24 \\ --gateway 10.0.20.1 \\ -o parent=eth0 \\ macvlan-net En rootless, las redes macvlan no funcionan sin privilegios extra. La alternativa práctica es exponer puertos via --publish y usar un reverse proxy en un container rootful (o en el host) para enrutar tráfico.\nPractical Migration Guide — Docker to Podman Si ya tienes un homelab con Docker, la migración puede ser gradual. Nadie te obliga a migrar todo de un día para otro.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 # Paso 1: Alias temporal para probar alias docker=podman # Paso 2: Probar en un container no crítico podman run --rm hello-world # Paso 3: Migrar un stack docker-compose cd /opt/stacks/mi-app-no-critica docker compose down podman-compose up -d # Paso 4: Verificar logs y conectividad podman-compose logs podman-compose ps # Si todo funciona, actualizar scripts/cron para usar podman-compose Para compatibilidad con herramientas que esperan el socket de Docker (como Portainer, Watchtower), Podman expone un socket compatible:\n1 2 3 4 5 6 # Exponer socket de Docker compatible podman system service --time=0 unix:///run/user/1000/docker.sock # O activar el servicio systemd systemctl --user enable podman.socket systemctl --user start podman.socket Luego configura tus herramientas para apuntar a unix:///run/user/1000/docker.sock via DOCKER_HOST.\nPara builds de Dockerfiles, funciona igual:\n1 2 3 4 5 # Build con un Dockerfile existente podman build -t myapp:latest . # Multi-stage builds podman build -t myapp:latest -f Dockerfile.multistage . BuildKit no está disponible en Podman, pero podman build usa Buildah internamente y maneja cache layer como Docker.\nLimitations and Tradeoffs Ninguna herramienta es perfecta. Estas son las limitaciones reales que encontré:\nLimitación Impacto Workaround No Docker Swarm Orquestación multi-host Usar K3s, Nomad, o Docker Compose No BuildKit Builds multi-etapa más lentos Aceptable para homelab Macvlan rootless complicado IPs dedicadas a containers Usar port publishing + reverse proxy host network no disponible Containers que necesitan red del host Usar --network slirp4netns con ports Podman desktop menos maduro No hay Docker Desktop equivalente CLI + Quadlets es suficiente Ciertas imágenes con --privileged Algunos tools requieren capabilities root Evaluar si realmente necesitas privilegios Para un homelab —donde no corres Swarm, no necesitas clusters Kubernetes, y valoras seguridad sobre features— estas limitaciones son aceptables.\nConclusion Podman ha madurado lo suficiente como para reemplazar a Docker en un homelab. La arquitectura sin daemon y rootless reduce significativamente la superficie de ataque, los Quadlets integran containers con systemd de forma elegante, y la compatibilidad con docker-compose.yml hace que la migración sea incremental y de bajo riesgo.\nMi recomendación: empieza con un container no crítico. Ponle un alias alias docker=podman en tu shell. Corre un stack con podman-compose. Cuando te sientas cómodo, migra el resto. No hay prisa — pero el rootless es el futuro de los containers, y Podman ya está aquí.\n1 2 3 4 5 6 7 8 9 # Resumen: migrar un container hoy alias docker=podman docker run -d --name test -p 8080:80 nginx:alpine docker ps curl http://localhost:8080 docker stop test \u0026amp;\u0026amp; docker rm test # Si funciona, migra tu primer stack productivo podman-compose -f /opt/stacks/nginx-proxy-manager/docker-compose.yml up -d ","permalink":"https://blog.gntech.me/posts/2026-06-13-podman-rootless-containers-docker-alternative-homelab/","summary":"Run containers without a root daemon. Migrate your homelab from Docker to Podman for true rootless operation, systemd-native container management, and drop-in docker CLI compatibility.","title":"Podman Rootless Containers — A Docker Alternative for Homelab"},{"content":"Pulling a container image means trusting its publisher, build process, and distribution path. An SBOM helps identify what an image contains; a verified signature helps establish who signed a particular artifact. Neither replaces vulnerability management or proves an image is safe.\nIn a homelab, you might think \u0026ldquo;I\u0026rsquo;m not a target.\u0026rdquo; You are — bots and automated scanners don\u0026rsquo;t discriminate. The good news: securing your container supply chain with open-source tools is straightforward and lightweight. This guide covers the three pillars — SBOM generation, vulnerability scanning, image signing, and verification — using Syft, Grype, Cosign, and Docker Content Trust.\nUnderstanding the Container Supply Chain Security Landscape Supply-chain levels for software artifacts (SLSA) defines a framework of security levels. For a homelab, achieving SLSA Level 2 is realistic: signed provenance, hosted builds, and verified sources. You don\u0026rsquo;t need a full enterprise platform to get there.\nIn June 2026, Docker launched Hardened Images with Aikido vulnerability scanning, and the industry continues pushing toward transparency. An SBOM (Software Bill of Materials) is the foundation — it tells you exactly what packages are inside every layer of your image. With an SBOM in hand, you can scan for known vulnerabilities, verify the image was built by a trusted source, and detect tampering between build and deployment.\nThe tools we\u0026rsquo;ll use fit into a single workflow:\nSource → Build → SBOM (Syft) → Scan (Grype) → Sign (Cosign) → Deploy → Verify Each step is optional and composable. Start with one, add the rest over time.\nGenerating SBOMs with Syft Syft from Anchore generates SBOMs from container images and filesystems. It supports SPDX, CycloneDX, and Syft\u0026rsquo;s own JSON format.\nInstalling Syft 1 curl -sSfL https://raw.githubusercontent.com/anchore/syft/main/install.sh | sh -s -- -b /usr/local/bin Or via Docker:\n1 docker run --rm -v /var/run/docker.sock:/var/run/docker.sock anchore/syft nginx:latest Generating SBOMs Generate an SPDX SBOM for the image you use daily:\n1 syft packages nginx:latest -o spdx-json=sbom.nginx.spdx.json For CycloneDX format:\n1 syft packages nginx:latest -o cyclonedx-json=sbom.nginx.cdx.json Comparing SBOMs Across Versions Track how your images change over time:\n1 2 3 syft packages myregistry.local/webapp:v1 -o spdx-json=sbom.v1.json syft packages myregistry.local/webapp:v2 -o spdx-json=sbom.v2.json diff \u0026lt;(jq \u0026#39;.artifacts[].name\u0026#39; sbom.v1.json | sort) \u0026lt;(jq \u0026#39;.artifacts[].name\u0026#39; sbom.v2.json | sort) Automating SBOM Generation Use a systemd timer to generate SBOMs for your critical images weekly:\n1 2 3 4 5 6 7 8 # /etc/systemd/system/syft-sbom.service [Unit] Description=Generate SBOM for nginx image After=docker.service [Service] Type=oneshot ExecStart=/usr/local/bin/syft packages nginx:latest -o spdx-json=/opt/sboms/nginx.spdx.json 1 2 3 4 5 6 7 8 9 10 # /etc/systemd/system/syft-sbom.timer [Unit] Description=Weekly SBOM generation [Timer] OnCalendar=weekly Persistent=true [Install] WantedBy=timers.target Scanning for Vulnerabilities with Grype Grype pairs with Syft natively — you can feed it an SBOM or scan an image directly.\nInstalling Grype 1 curl -sSfL https://raw.githubusercontent.com/anchore/grype/main/install.sh | sh -s -- -b /usr/local/bin Scanning Images Basic scan with severity summary:\n1 grype nginx:latest Show only vulnerabilities with a fix available:\n1 grype --only-fixed nginx:latest Fail the scan if any high-severity vulnerabilities exist (useful for CI):\n1 grype --fail-on high nginx:latest Pipeline: SBOM → Scan Generate an SBOM and scan it in one chain:\n1 syft packages myapp:latest -o json | grype --fail-on medium Configuring Grype for Your Homelab Create ~/.grype.yaml to exclude false positives:\n1 2 3 4 5 6 7 # ~/.grype.yaml ignore: - vulnerability: CVE-2023-XXXXX reason: \u0026#34;False positive in our base image\u0026#34; fix-state: not-fixed severity: min-severity: medium Remediation Workflow When Grype reports vulnerabilities:\nCheck if it\u0026rsquo;s in the base image — update the base image tag If it\u0026rsquo;s in an added package — update the package version in your Dockerfile If no fix exists — consider a distroless or hardened base image Signing Container Images with Cosign Cosign from Sigstore is the de-facto tool for signing container images. It supports key-pair signing and keyless signing via OIDC.\nInstalling Cosign 1 2 3 curl -O -L \u0026#34;https://github.com/sigstore/cosign/releases/latest/download/cosign-linux-amd64\u0026#34; sudo mv cosign-linux-amd64 /usr/local/bin/cosign sudo chmod +x /usr/local/bin/cosign Generating a Key Pair 1 cosign generate-key-pair This creates cosign.key (private) and cosign.pub (public). Store the private key securely — consider using a password manager or age encryption.\nSigning an Image Push your image to a registry, then sign:\n1 2 docker push myregistry.local/myapp:latest cosign sign --key cosign.key myregistry.local/myapp:latest Cosign stores the signature as an additional manifest in the same registry under the tag myregistry.local/myapp:sha256-\u0026lt;digest\u0026gt;.sig.\nVerifying an Image 1 cosign verify --key cosign.pub myregistry.local/myapp:latest The output includes the signature payload with the image digest. If the image has been tampered with, verification fails.\nKeyless Signing with OIDC For CI pipelines on GitHub Actions or GitLab CI:\n1 cosign sign myregistry.local/myapp:latest Cosign uses the OIDC token from the CI environment (no key management needed). Verification for keyless signatures uses the Fulcio root:\n1 cosign verify myregistry.local/myapp:latest Integrating with Docker Buildx Use build attestations for provenance:\n1 2 3 4 5 docker buildx build \\ --attest type=provenance,mode=max \\ --attest type=sbom \\ -t myregistry.local/myapp:latest \\ --push . Then sign the attestation:\n1 cosign attest --predicate sbom.spdx.json --key cosign.key myregistry.local/myapp:latest Storing Public Keys Commit cosign.pub to your infrastructure Git repo:\n1 2 git add cosign.pub git commit -m \u0026#34;add cosign public key for image verification\u0026#34; Then any team member can verify images from your registry.\nEnforcing Signed Images with Docker Content Trust Docker Content Trust (DCT) uses Notary under the hood to enforce that only signed images can be pulled or pushed.\nEnable Docker Content Trust 1 export DOCKER_CONTENT_TRUST=1 With DCT enabled, every docker push, docker pull, and docker build will verify signatures.\nPushing Signed Images 1 2 export DOCKER_CONTENT_TRUST=1 docker push myregistry.local/myapp:latest Docker will prompt for the root key passphrase and the repository passphrase. Once configured, push succeeds only if the image is signed.\nAdding Delegated Signers For teams or multiple machines:\n1 docker trust signer add --key signer.pub team-lead myregistry.local/myapp Then the signer can push signed images with their own key.\nInspecting Trust Data 1 docker trust inspect myregistry.local/myapp --pretty Limitations in a Homelab DCT works best with Docker Hub or a Notary-signer-enabled registry. For a simple Docker Registry v2, consider using Cosign exclusively, as DCT requires a Notary server alongside your registry.\nBuilding a Practical Homelab Pipeline Combine everything into a Gitea Actions workflow:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 # .gitea/workflows/build-and-sign.yml name: Build, Scan, and Sign on: [push] jobs: build: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Build image with provenance run: | docker buildx build \\ --attest type=provenance,mode=max \\ --attest type=sbom \\ -t ${{ secrets.REGISTRY }}/myapp:${{ github.sha }} \\ --push . - name: Scan with Grype run: | syft packages ${{ secrets.REGISTRY }}/myapp:${{ github.sha }} -o json | \\ grype --fail-on medium - name: Sign with Cosign run: | cosign sign --key \u0026lt;(echo \u0026#34;${{ secrets.COSIGN_KEY }}\u0026#34;) \\ ${{ secrets.REGISTRY }}/myapp:${{ github.sha }} For day-to-day use, add these aliases to your .bashrc:\n1 2 3 alias scan=\u0026#39;grype\u0026#39; alias sbom=\u0026#39;syft packages\u0026#39; alias verify=\u0026#39;cosign verify --key ~/.cosign/cosign.pub\u0026#39; Run verification before deploying any image to production services in your homelab:\n1 verify myregistry.local/myapp:latest \u0026amp;\u0026amp; docker compose up -d Conclusion Container supply chain security isn\u0026rsquo;t just for enterprises. With Syft, Grype, and Cosign, you can generate SBOMs, scan for vulnerabilities, and sign your images — all with open-source tools and a few shell commands. Start with Syft and Grype (the biggest security gain with the least effort), then add Cosign when you want to guarantee image integrity.\nThe full workflow — SBOM → Scan → Sign → Verify — takes about 15 minutes to set up and protects every image you deploy.\nResources Syft GitHub Grype GitHub Cosign GitHub SLSA Framework Docker Build Attestations Docker Hardened Images ","permalink":"https://blog.gntech.me/posts/2026-06-13-docker-supply-chain-security-sbom-image-signing/","summary":"Practical guide to securing your container supply chain in the homelab: generate SBOMs with Syft, sign images with Cosign, scan for vulnerabilities with Grype, and enforce Docker Content Trust.","title":"Docker Supply Chain Security — SBOM, Image Signing, and Verification"},{"content":"Si tu docker-compose.yml tiene más de 15 servicios —como cualquier homelab que lleve un par de años creciendo— probablemente te duele mantenerlo. Cada nuevo servicio copia los mismos bloques de logging, restart policy, redes, y recursos. Cuando decides cambiar el driver de logging de json-file a local, toca editar 12 servicios uno por uno. Y en algún momento alguien se olvida de uno y los logs no rotan.\nYAML anchors, aliases, y extension blocks (x-) resuelven esto. Son una característica del lenguaje YAML que Docker Compose entiende nativamente, sin plugins, sin herramientas externas, sin nada más que sintaxis YAML estándar. Esta guía cubre cómo usarlos en tu homelab con ejemplos reales que puedes copiar y adaptar.\nYAML Anchors and Aliases — The Foundation for Reusable Compose Config YAML tiene un mecanismo nativo para definir un bloque de configuración una vez y reutilizarlo en varios lugares: los anchors (marcados con \u0026amp;nombre) y los aliases (marcados con *nombre).\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # anchor definition x-logging: \u0026amp;logging driver: \u0026#34;json-file\u0026#34; options: max-size: \u0026#34;10m\u0026#34; max-file: \u0026#34;3\u0026#34; services: app: image: nginx:alpine logging: *logging # \u0026lt;-- alias db: image: postgres:16-alpine logging: *logging # \u0026lt;-- same config, no repetition El anchor \u0026amp;logging marca un bloque YAML completo. El alias *logging lo inserta en el lugar donde se usa. Docker Compose lo resuelve antes de validar el archivo, así que es como si hubieras escrito el bloque completo en cada servicio.\nLa regla de oro: mira tu compose file actual. Cualquier bloque que aparezca igual en tres o más servicios es candidato a ser un anchor.\nx- Extension Blocks for Structured Reusable Compose Fragments Docker Compose ignora silenciosamente cualquier llave de nivel superior que comience con x-. Esto las convierte en el vehículo ideal para agrupar anchors de forma legible y predecible.\nEn lugar de esparcir definiciones \u0026amp;logging, \u0026amp;restart y \u0026amp;resource por todo el archivo, las agrupas en una sección x- al inicio:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 x-restart: \u0026amp;restart restart: unless-stopped x-logging: \u0026amp;logging logging: driver: \u0026#34;json-file\u0026#34; options: max-size: \u0026#34;10m\u0026#34; max-file: \u0026#34;3\u0026#34; x-healthcheck: \u0026amp;health healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;curl\u0026#34;, \u0026#34;-f\u0026#34;, \u0026#34;http://localhost/health\u0026#34;] interval: 30s timeout: 10s retries: 3 start_period: 40s x-resource-limits: \u0026amp;limits deploy: resources: limits: memory: 512M cpus: \u0026#34;0.5\u0026#34; reservations: memory: 128M Beneficio inmediato: todas las configuraciones reutilizables están en un mismo lugar, al inicio del archivo. Cambias el driver de logging una vez y se propaga a todos los servicios que usan *logging.\nAnchors and Merge Key (\u0026laquo;) for Service Composition Los aliases simples (*logging) funcionan para bloques individuales, pero cuando cada servicio necesita combinar logging + restart + resources + redes, escribir *logging, *restart, *limits dentro de cada servicio sigue siendo verboso. Aquí entra el merge key de YAML (\u0026lt;\u0026lt;).\nEl merge key fusiona uno o varios anchors en un mapa, permitiendo sobreescribir campos específicos por servicio:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 x-restart: \u0026amp;restart restart: unless-stopped x-logging: \u0026amp;logging logging: driver: \u0026#34;json-file\u0026#34; options: max-size: \u0026#34;10m\u0026#34; max-file: \u0026#34;3\u0026#34; x-resource-limits: \u0026amp;limits deploy: resources: limits: memory: 512M cpus: \u0026#34;0.5\u0026#34; reservations: memory: 128M x-defaults: \u0026amp;defaults \u0026lt;\u0026lt;: [*restart, *logging, *limits] networks: - internal security_opt: - \u0026#34;no-new-privileges:true\u0026#34; services: api: \u0026lt;\u0026lt;: *defaults image: myapp/api:latest environment: DB_HOST: postgres ports: - \u0026#34;8080:8080\u0026#34; worker: \u0026lt;\u0026lt;: *defaults image: myapp/worker:latest environment: QUEUE: default deploy: resources: limits: memory: 1G networks: internal: internal: true El orden importa: los campos definidos después del merge key sobreescriben los que vienen del anchor. En el ejemplo, worker hereda *limits del defaults (512M) pero lo sobreescribe con 1G.\nReal-World Homelab Example — DRY Monitoring Stack with Compose Anchors Veamos un ejemplo completo. Un stack de monitoreo con Grafana, Prometheus, Loki, y Alloy, usando los patrones anteriores:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 x-restart: \u0026amp;restart restart: unless-stopped x-logging: \u0026amp;logging logging: driver: \u0026#34;local\u0026#34; options: max-size: \u0026#34;10m\u0026#34; x-resources: \u0026amp;limits deploy: resources: limits: memory: 512M cpus: \u0026#34;0.5\u0026#34; x-defaults: \u0026amp;defaults \u0026lt;\u0026lt;: [*restart, *logging, *limits] networks: - monitoring services: grafana: \u0026lt;\u0026lt;: *defaults image: grafana/grafana:latest container_name: grafana ports: - \u0026#34;3000:3000\u0026#34; volumes: - grafana-data:/var/lib/grafana environment: GF_SECURITY_ADMIN_PASSWORD: \u0026#34;${GF_ADMIN_PASSWORD}\u0026#34; deploy: resources: limits: memory: 1G cpus: \u0026#34;1.0\u0026#34; prometheus: \u0026lt;\u0026lt;: *defaults image: prom/prometheus:latest container_name: prometheus ports: - \u0026#34;9090:9090\u0026#34; volumes: - prometheus-data:/prometheus - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro command: - \u0026#34;--config.file=/etc/prometheus/prometheus.yml\u0026#34; - \u0026#34;--storage.tsdb.retention.time=30d\u0026#34; loki: \u0026lt;\u0026lt;: *defaults image: grafana/loki:latest container_name: loki ports: - \u0026#34;3100:3100\u0026#34; volumes: - loki-data:/loki - ./loki-config.yml:/etc/loki/config.yml:ro command: - \u0026#34;-config.file=/etc/loki/config.yml\u0026#34; alloy: \u0026lt;\u0026lt;: *defaults image: grafana/alloy:latest container_name: alloy ports: - \u0026#34;12345:12345\u0026#34; volumes: - /var/run/docker.sock:/var/run/docker.sock:ro - ./alloy/config.alloy:/etc/alloy/config.alloy:ro environment: LOKI_URL: \u0026#34;http://loki:3100/loki/api/v1/push\u0026#34; PROMETHEUS_URL: \u0026#34;http://prometheus:9090/api/v1/write\u0026#34; volumes: grafana-data: prometheus-data: loki-data: networks: monitoring: driver: bridge Sin anchors, este mismo archivo tendría cuatro bloques de logging:, cuatro bloques de restart:, cuatro bloques de deploy:resources:, y cuatro bloques de networks:. Con los anchors, la sección x- centraliza todo y cada servicio declara solo lo que le es específico: image, container_name, ports, volumes, environment, y command.\nPara iniciar el stack:\n1 2 3 4 5 6 7 8 # variables de entorno export GF_ADMIN_PASSWORD=secure_password_here # levantar todo docker compose up -d # verificar que todos usan el logging correcto docker compose logs grafana | head -5 Common Docker Compose YAML Pitfalls and Limitations Los anchors son poderosos, pero tienen limitaciones que debes conocer para evitar bugs difíciles de depurar.\nAnchors No Pueden Mergear Secuencias (Listas) El merge key \u0026lt;\u0026lt; solo funciona con mapas (YAML mappings/dictionaries). No puedes mergear listas. Esto es relevante porque en Compose hay campos como depends_on, ports, volumes, y networks que son listas.\n1 2 3 4 5 6 7 8 # ESTO NO FUNCIONA x-ports: \u0026amp;ports - \u0026#34;80:80\u0026#34; - \u0026#34;443:443\u0026#34; services: web: ports: *ports # alias OK, pero no puedes mergear con \u0026lt;\u0026lt; Solución: usa el alias directamente (*ports), no el merge key. Si necesitas combinar puertos de un anchor con puertos específicos del servicio, tendrás que escribirlos todos en el servicio o usar un anchor que ya contenga todo.\nAnchors Son Referencias, No Copias Un anchor YAML es una referencia, no una copia. Si defines un anchor de environment y un servicio lo modifica después del alias, el anchor original NO se modifica —pero ten cuidado con objetos mutables compartidos.\nEn la práctica, para Compose esto no es problema porque los values son escalares (strings, números) o mapas planos. Nunca estamos mutando objetos después de creados.\nOrden de Resolución del Merge Key Cuando usas \u0026lt;\u0026lt;: *defaults seguido de campos propios del servicio, el orden es:\nSe aplica el merge key primero (todo lo que viene de *defaults) Se aplican los campos explícitos del servicio después Esto significa que puedes sobreescribir cualquier campo del defaults:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 x-defaults: \u0026amp;defaults restart: unless-stopped deploy: resources: limits: memory: 512M cpus: \u0026#34;0.5\u0026#34; services: heavy-worker: \u0026lt;\u0026lt;: *defaults image: myapp/worker:latest # sobreescribe el deploy del defaults deploy: resources: limits: memory: 2G cpus: \u0026#34;2.0\u0026#34; No Puedes Usar \u0026laquo; Dentro de Bloques Anidados El merge key solo funciona en el nivel donde lo colocas. Si *defaults contiene deploy: con recursos dentro, no puedes hacer un \u0026lt;\u0026lt;: dentro de deploy para mezclar recursos del defaults con recursos extra. El merge es plano.\nSolución: si necesitas combinar, define el bloque completo en el servicio y omite el anchor correspondiente, o crea variantes del anchor (por ejemplo, *limits-default y *limits-large).\nResumen: Docker Compose DRY Patterns for Your Homelab Tres patrones para aplicar hoy mismo:\nx-blocks: Define x-logging, x-restart, x-resources al inicio de tu compose file. Agrupa toda la configuración reusable en un solo lugar visible.\nDefaults composition: Crea un x-defaults con \u0026lt;\u0026lt;: [*restart, *logging, *limits] y reutiliza *defaults en todos los servicios. Sobreescribe por servicio cuando sea necesario.\nOverride on demand: Los anchors no son rígidos — cada servicio puede heredar defaults y cambiar cualquier campo. Usa esto para servicios con requisitos especiales de memoria o redes.\nEl patrón más impactante con el que puedes empezar hoy: agrega un bloque x-logging: \u0026amp;logging con driver: \u0026quot;local\u0026quot; y max-size: \u0026quot;10m\u0026quot;, luego úsalo en todos tus servicios con logging: *logging. Es un cambio de una línea por archivo que evita que los logs de tus contenedores se coman el disco del host.\n1 2 # Verifica que tu compose file es válido con anchors docker compose config \u0026gt; /dev/null \u0026amp;\u0026amp; echo \u0026#34;✅ Compose file is valid\u0026#34; El comando docker compose config resuelve todos los anchors y muestra el archivo final expandido — útil para depurar si algo no se aplica como esperas.\n","permalink":"https://blog.gntech.me/posts/2026-06-12-docker-compose-yaml-anchors-extensions/","summary":"Stop copying config between services. Use YAML anchors, x-blocks, and fragment merging to write clean, DRY docker-compose files for your homelab. Practical patterns with real examples.","title":"Docker Compose YAML Anchors \u0026 Extensions — DRY Multi-Service Deployments"},{"content":"Crear una VM en Proxmox es fácil. Conseguir que rinda al máximo en disco no lo es. Cuatro ajustes —el controlador de disco, el IO thread, el modo aio, y el modo de caché— determinan si tu VM obtiene 200 mil IOPS o 600 mil en el mismo hardware. La diferencia es real, medible, y muchas VM en homelab corren con valores por defecto que dejan hasta un 40 % de rendimiento sobre la mesa.\nEsta guía cubre cada uno de esos parámetros: qué hacen, cómo configurarlos, cuándo elegir cada combinación, y cómo medir los resultados con fio. Todos los comandos y configs funcionan en Proxmox VE 8.x y 9.x.\nVirtIO-blk vs VirtIO-SCSI Single: Choosing the Right Disk Controller Proxmox ofrece dos controladores paravirtualizados principales para discos de VM Linux. Cada uno tiene fortalezas distintas.\nVirtIO-blk El controlador VirtIO-blk es el más antiguo de los dos. Fue el primer controlador de bloque completamente paravirtualizado para KVM, diseñado para ser simple y rápido. En benchmarks puros de throughput secuencial, VirtIO-blk sin IO thread suele igualar o superar ligeramente a VirtIO-SCSI en las mismas condiciones. Sin embargo, carece de soporte para discard/trim, no expone un bus SCSI real al guest, y no permite el uso de IO threads dedicados por disco — el IO thread se asigna al controlador completo.\nUsa VirtIO-blk cuando:\nLa VM corre kernels anteriores a 4.x No necesitas discard/trim Buscas la máxima velocidad en benchmarks sintéticos secuenciales Es una VM Windows con drivers virtio-win (tradicionalmente mejor soportados) VirtIO-SCSI Single El controlador VirtIO-SCSI Single es el valor por defecto en Proxmox desde la versión 4.3. Usa un bus SCSI real sobre el transporte virtio, lo que le da varias ventajas: soporte completo de discard/trim, fences de cache (cache flushes), y la capacidad de asignar un IO thread dedicado por controladora SCSI. Es la opción recomendada para la gran mayoría de cargas de trabajo.\n1 2 # Asignar VirtIO-SCSI Single con IO thread y discard qm set 100 -scsi0 local-zfs:vm-100-disk-0,iothread=1,discard=on VirtIO-SCSI (sin \u0026ldquo;Single\u0026rdquo;) activa multi-queue con múltiples IO threads. Es útil cuando tienes muchos discos en una sola VM con alta concurrencia, pero para la mayoría de los homelabs con 1-3 discos por VM, VirtIO-SCSI Single ofrece mejor rendimiento porque evita la contención entre IO threads.\nVerifica el controlador actual de tu VM:\n1 2 qm config 100 | grep -E \u0026#39;(scsi|virtio)[0-9]+\u0026#39; # scsi0: local-zfs:vm-100-disk-0,iothread=1,size=64G IO Threads: Dedicated Host Threads for Disk I/O Cuando una VM ejecuta operaciones de disco sin IO thread, todo el I/O pasa por el hilo principal de QEMU — el mismo que maneja la emulación de la CPU, los dispositivos virtuales, y la red. Esto crea contención: una operación de disco bloquea otros trabajos del hilo principal, y viceversa.\nActivar IO threads asigna un hilo de host independiente a cada controladora de disco. El beneficio es doble: menor latencia de I/O porque el hilo principal de QEMU ya no se intercala con operaciones de disco, y mejor uso de la caché de CPU porque el hilo de disco trabaja en un conjunto de datos más predecible.\n1 2 3 4 5 # Habilitar IO thread en disco SCSI qm set 100 -scsi0 local-zfs:vm-100-disk-0,iothread=1 # Habilitar IO thread en disco VirtIO-blk qm set 100 -virtio0 local-zfs:vm-100-disk-0,iothread=1 Elige el número correcto de IO threads según tu hardware:\nEscenario IO Threads recomendados 1-2 discos, VM típica 1 (VirtIO-SCSI Single) 3+ discos, alta concurrencia 2-3 (VirtIO-SCSI multi) Host con 4+ núcleos/VMs ligeras 1 por VM Host con muchas VMs (\u0026gt;10) IO thread solo en VMs críticas Verifica que el IO thread esté activo desde el host:\n1 ps aux | grep qemu | grep -E \u0026#39;iothread\u0026#39; Si el proceso QEMU de tu VM tiene el flag iothread entre sus argumentos, el IO thread está funcionando.\nAIO Mode: How QEMU Submits I/O to the Kernel El modo aio (asynchronous I/O) controla cómo QEMU envía operaciones de disco al kernel del host. Proxmox soporta tres modos.\naio=native Usa Linux AIO (libaio) con semántica O_DIRECT. Es el modo más maduro y probado. Funciona bien con discos rotativos, SSDs decentes, y configuraciones ZFS. Es el valor por defecto histórico de Proxmox.\naio=io_uring Introducido en Linux 5.1 y estable desde 5.4, io_uring es el reemplazo moderno de libaio. Reduce la sobrecarga de syscalls drásticamente: donde libaio requiere una syscall por lote de I/O, io_uring usa un ring buffer compartido entre kernel y userspace. En hardware NVMe moderno, io_uring suele superar a native entre un 10-20 % en IOPS aleatorias.\n1 2 # Cambiar a io_uring en un disco existente qm set 100 -scsi0 iothread=1,aio=io_uring aio=threads Emula I/O asíncrono con un pool de pthreads. Tiene la mayor sobrecarga y solo debe usarse si native e io_uring no funcionan — por ejemplo, con ciertos backends de almacenamiento antiguos. En un homelab moderno no deberías necesitarlo.\n1 2 3 # Ver modo aio actual qm config 100 | grep aio # scsi0: local-zfs:vm-100-disk-0,iothread=1,aio=io_uring,size=64G Recomendación: usa aio=io_uring en cualquier host con Proxmox VE 8.x o superior y kernel 6.2+. Para kernels anteriores, usa aio=native.\nCache Modes: none vs writeback vs writethrough vs unsafe El modo de caché controla cómo QEMU interactúa con el page cache del host. Es el parámetro que más impacto tiene en el rendimiento de escritura — y también el que más riesgo conlleva.\ncache=none Usa O_DIRECT para saltarse el page cache del host por completo. Las escrituras se reportan como completadas solo cuando llegan al dispositivo de almacenamiento. Es el modo por defecto y el más seguro para datos importantes. El guest maneja su propia caché (el page cache del guest Linux), y QEMU no añade una segunda capa.\ncache=writeback QEMU usa el page cache del host. Las escrituras se reportan como completadas cuando llegan a la RAM del host, no al disco. Esto permite al kernel del host coalescer operaciones pequeñas en escrituras más grandes y reordenarlas para maximizar el throughput. En benchmarks, writeback puede duplicar las IOPS de escritura comparado con none.\nEl riesgo: si el host pierde energía, cualquier dato que esté en el page cache del host se pierde. Los guests que usan writeback no pueden garantizar la integridad de la transacción a menos que el host tenga una batería (BBU) o un UPS y discard flush commands no sea un problema.\ncache=writethrough Cada escritura se confirma en el almacenamiento antes de reportarse como completa. Es el modo más lento y el más seguro. Combina la verificación de O_DSYNC con el uso del page cache del host para lecturas. En la práctica, su rendimiento es tan bajo que no lo recomendamos para nada que no sea una base de datos financiera o cumplimiento normativo.\ncache=unsafe QEMU ignora todos los comandos de cache flush que envía el guest. Las escrituras van al page cache del host y se reportan como completas instantáneamente. Es extremadamente rápido y extremadamente peligroso. Úsalo solo en entornos efímeros: CI runners, VMs de desarrollo que se reconstruyen desde cero, o pruebas donde la pérdida de datos no importa.\nTabla de recomendaciones prácticas Carga de trabajo Cache AIO Controlador IO thread Base de datos (PostgreSQL, MariaDB) none io_uring VirtIO-SCSI Single sí NAS / file server none native VirtIO-SCSI Single sí VM de desarrollo efímera writeback io_uring VirtIO-blk sí CI runner (GitHub Actions) unsafe io_uring VirtIO-blk no Servidor web / aplicación none io_uring VirtIO-SCSI Single sí Cache / Redis (persistencia) writeback native VirtIO-SCSI Single sí Cambia el modo de caché en una VM existente:\n1 2 # De none a writeback (más velocidad, más riesgo) qm set 100 -scsi0 local-zfs:vm-100-disk-0,cache=writeback,iothread=1,aio=io_uring Benchmarking with fio Inside Proxmox VMs La única forma de saber qué combinación funciona mejor para tu hardware y tu carga de trabajo es medir. Usa fio dentro de la VM con parámetros controlados.\nInstala fio en el guest Linux:\n1 apt update \u0026amp;\u0026amp; apt install fio -y Benchmark de lectura secuencial (throughput) 1 2 3 4 fio --name=seq-read --filename=/tmp/fio-test --size=4G \\ --ioengine=libaio --iodepth=32 --direct=1 \\ --rw=read --bs=1M --numjobs=1 --runtime=30 \\ --output-format=json --output=/tmp/fio-seq-read.json Benchmark de escritura aleatoria 4K (IOPS) 1 2 3 4 fio --name=randwrite-4k --filename=/tmp/fio-test --size=4G \\ --ioengine=libaio --iodepth=32 --direct=1 \\ --rw=randwrite --bs=4K --numjobs=4 --runtime=60 \\ --output-format=json --output=/tmp/fio-randwrite-4k.json Benchmark mixto 70/30 (carga típica de base de datos) 1 2 3 4 fio --name=mixed --filename=/tmp/fio-test --size=4G \\ --ioengine=libaio --iodepth=16 --direct=1 \\ --rw=randrw --rwmixread=70 --bs=8K --numjobs=2 --runtime=60 \\ --output-format=json --output=/tmp/fio-mixed.json Extrae los resultados relevantes:\n1 2 3 4 5 6 # IOPS de escritura jq \u0026#39;.jobs[] | .write.iops\u0026#39; /tmp/fio-randwrite-4k.json # Latencia P99 de escritura (usec) jq \u0026#39;.jobs[] | .write.clat_ns.percentile[\u0026#34;99.000000\u0026#34;]\u0026#39; /tmp/fio-randwrite-4k.json # Throughput de lectura (bytes/seg) jq \u0026#39;.jobs[] | .read.bw_bytes\u0026#39; /tmp/fio-seq-read.json Lo que encontré en mi homelab En un host SRV1 con Proxmox VE 8.3, ZFS sobre dos NVMe Samsung PM9A3 en mirror, una VM Debian 13 con 4 vCPU y 8 GB RAM:\nConfiguración IOPS randwrite 4K Throughput seq read P99 lat write VirtIO-blk, cache=none, aio=native, no iothread 118K 1.1 GB/s 2.1 ms VirtIO-SCSI Single, cache=none, aio=native, iothread 142K 1.3 GB/s 1.4 ms VirtIO-SCSI Single, cache=none, aio=io_uring, iothread 189K 1.6 GB/s 0.9 ms VirtIO-blk, cache=writeback, aio=io_uring, iothread 315K 1.8 GB/s 0.3 ms VirtIO-blk, cache=unsafe, aio=io_uring, iothread 412K 2.0 GB/s 0.1 ms Los números de writeback y unsafe son llamativos, pero implican riesgo de pérdida de datos en un corte de luz. Para mi homelab, VirtIO-SCSI Single + cache=none + aio=io_uring + iothread es el punto óptimo: ganas un 60 % de IOPS sobre los valores por defecto sin sacrificar seguridad.\nApplying the Optimal Configuration via CLI Cambia todos los parámetros de una sola vez sin interfaz gráfica:\n1 2 3 4 5 6 VMID=100 # Configuración óptima para base de datos / uso general qm set $VMID \\ -scsi0 local-zfs:vm-${VMID}-disk-0,iothread=1,aio=io_uring,cache=none,discard=on \\ -scsihw virtio-scsi-single Para aplicar la configuración a una VM existente, la VM debe estar apagada si estás cambiando el tipo de controladora (-scsihw). Los parámetros iothread, aio, y cache pueden cambiarse en caliente.\nSummary Cinco decisiones determinan el rendimiento de disco de tus VMs en Proxmox:\nControladora: VirtIO-SCSI Single para uso general (default desde PVE 4.3 por una razón) IO thread: actívalo siempre con iothread=1 — el costo es mínimo y la ganancia de latencia es constante AIO mode: aio=io_uring en Proxmox 8+ con kernel moderno; aio=native como respaldo Cache mode: cache=none para datos importantes (seguro, rápido); cache=writeback para VMs efímeras si entiendes el riesgo Benchmark: corre fio con direct=1 y --output-format=json antes de decidir — los benchmarks sintéticos no mienten, pero solo importan si reflejan tu carga real El cambio más impactante que puedes hacer hoy: agrega iothread=1 y aio=io_uring a tu disco principal. Son dos parámetros, sin riesgo de pérdida de datos, y en la mayoría de los casos te dan entre un 20-60 % más de IOPS sin tocar el hardware.\n","permalink":"https://blog.gntech.me/posts/2026-06-12-proxmox-vm-disk-performance-benchmark/","summary":"Compare VirtIO-blk vs VirtIO-SCSI single disk controllers in Proxmox, benchmark IO threads and aio modes with fio, and choose the right cache settings for your homelab VMs.","title":"Proxmox VM Disk Performance — VirtIO-blk vs VirtIO-SCSI, IO Threads \u0026 Cache Modes"},{"content":"You run docker compose down and wait. The terminal hangs for ten seconds, then spits you back to your prompt. Next time you start the stack, your database logs show \u0026ldquo;database system was not properly shut down.\u0026rdquo; Your application containers have stale PID files. An incomplete write has corrupted a config file you need to regenerate from scratch.\nThis is the default Docker shutdown behavior at work: SIGTERM, a ten-second grace period, then SIGKILL. For many containers that\u0026rsquo;s fine. For databases, caches, and stateful services, those ten seconds are a roll of the dice and you will lose.\nThis guide covers how Docker container signals actually work, how to configure stop_grace_period and StopSignal properly, why PID1 handling matters, and how to implement SIGTERM trap handlers in your custom entrypoints. Every config and command here is copy-paste ready for your homelab.\nHow Docker Container Stop Signals Actually Work When you run docker stop \u0026lt;container\u0026gt;, Docker sends the SIGTERM signal (15) to PID1 inside the container. The kernel delivers the signal to the process running as PID1, which is the entrypoint or CMD of your image. Docker then waits for the stop_grace_period — defaulting to 10 seconds. If the container has not exited by then, Docker sends SIGKILL (signal 9), which the kernel handles directly and terminates the process immediately with no cleanup opportunity.\nThe full flow:\ndocker stop → SIGTERM to PID1 Container process decides whether to handle or ignore If still running after grace period → SIGKILL (uncatchable) Container exits with code 137 (128 + 9 = SIGKILL) or 143 (128 + 15 = SIGTERM) You can observe this directly:\n1 2 docker run -d --name test-stop alpine sleep 300 time docker stop test-stop The time output shows how long the stop actually took. If it\u0026rsquo;s close to 10 seconds, your container was killed, not gracefully stopped. Check the exit code:\n1 2 3 docker inspect test-stop --format \u0026#39;{{.State.ExitCode}}\u0026#39; # 137 → killed by SIGKILL # 143 → received SIGTERM and exited on its own The same signal flow applies to docker compose down, docker compose restart, and docker service update in Swarm mode. Every one of these operations follows the signal → wait → kill pattern.\nConfiguring StopSignal and stop_grace_period in Docker Compose The two essential settings for controlling shutdown behavior are stop_signal and stop_grace_period in your compose file, and STOPSIGNAL in your Dockerfile.\nDocker Compose stop_grace_period Set a per-service grace period that matches your application\u0026rsquo;s shutdown requirements:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 services: postgres: image: postgres:17 stop_grace_period: 120s stop_signal: SIGTERM redis: image: redis:7-alpine stop_grace_period: 30s nginx: image: nginx:alpine stop_signal: SIGQUIT stop_grace_period: 30s PostgreSQL needs time to flush its WAL (write-ahead log), close active connections, and sync dirty buffers to disk. A 10-second default is dangerously short. Setting stop_grace_period: 120s gives Postgres room to shut down cleanly even under load.\nRedis is fast but needs a few seconds to dump its RDB snapshot if persistence is enabled and save directives are configured. 30 seconds is safe.\nNginx can use SIGQUIT as the stop signal — this tells Nginx to finish serving all in-flight HTTP requests before exiting, unlike SIGTERM which drops them immediately.\nDocker StopSignal in Dockerfile You can bake the signal into your image using the STOPSIGNAL instruction:\n1 2 FROM nginx:alpine STOPSIGNAL SIGQUIT This means any container launched from this image will receive SIGQUIT by default on docker stop, regardless of what the compose file says.\nDocker run equivalents The same settings work on the command line:\n1 2 3 4 5 docker run -d \\ --stop-signal SIGTERM \\ --stop-timeout 120 \\ --name postgres \\ postgres:17 --stop-timeout sets the grace period in seconds. --stop-signal sets the initial signal sent to PID1.\nThe init Process Problem: PID1 and Zombie Reaping There is a subtle but critical issue with how Linux process management works inside containers. In Linux, PID1 is special. It does not receive default signal handlers: if your PID1 process has not explicitly registered a handler for SIGTERM, the signal is silently ignored. This is kernel behavior, not Docker behavior.\nWhat this means in practice: if your container\u0026rsquo;s entrypoint is a shell script or a binary that does not explicitly trap SIGTERM, docker stop sends SIGTERM, the process ignores it, and the container keeps running until the 10-second timeout expires and Docker delivers SIGKILL. Your application never got a chance to shut down cleanly.\nThe second problem is zombie reaping. When a process inside the container forks children and those children exit, they become zombie processes until their parent calls wait(). If the parent dies without reaping them, they remain zombies. PID1 is responsible for reaping orphaned zombies. If your container\u0026rsquo;s entrypoint does not implement this, zombie processes accumulate.\nThe Simple Fix: docker \u0026ndash;init Docker provides a built-in init process based on tini that handles both signal forwarding and zombie reaping:\n1 docker run -d --init --name myapp myapp:latest In Docker Compose:\n1 2 3 4 5 services: app: image: myapp:latest init: true stop_grace_period: 30s Verify that init is enabled:\n1 2 docker inspect app --format \u0026#39;{{.HostConfig.Init}}\u0026#39; # true The --init flag causes Docker to use its bundled tini as PID1. Tini spawns your application as a child, forwards signals to it, and reaps zombies. It is the single most impactful thing you can do for container signal handling.\nManual Tini Integration in Dockerfile If you want the init process baked into your image rather than relying on the docker run --init flag (useful for images you distribute), add tini directly:\n1 2 3 4 5 6 7 FROM python:3.12-slim RUN apt-get update \u0026amp;\u0026amp; apt-get install -y tini \\ \u0026amp;\u0026amp; rm -rf /var/lib/apt/lists/* ENTRYPOINT [\u0026#34;/usr/bin/tini\u0026#34;, \u0026#34;--\u0026#34;] CMD [\u0026#34;python\u0026#34;, \u0026#34;app.py\u0026#34;] Dumb-init as an Alternative dumb-init is a similar tool with a slightly different approach. Unlike tini which uses wait() and signal forwarding, dumb-init runs as PID1 and creates a new process session for your app:\n1 2 3 4 5 6 FROM alpine:3.20 RUN apk add --no-cache dumb-init ENTRYPOINT [\u0026#34;/usr/bin/dumb-init\u0026#34;, \u0026#34;--\u0026#34;] CMD [\u0026#34;/usr/local/bin/myapp\u0026#34;] Both tools solve the same problems. Tini is lighter and ships with Docker. Dumb-init provides a --rewrite feature for remapping signals. For most homelab use cases, init: true in your compose file is sufficient.\nImplementing SIGTERM Trap Handlers for Custom Container Entrypoints When you build custom Docker images with shell-based entrypoints, you need explicit signal handling. A plain shell script started as PID1 in a container will ignore SIGTERM unless you trap it.\nHere is a production-ready entrypoint script that handles the signal correctly:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 #!/bin/sh set -e cleanup() { echo \u0026#34;Received SIGTERM, shutting down gracefully...\u0026#34; # Run any pre-stop hooks (e.g., drain connections, flush caches) if [ -f /usr/local/bin/pre-stop.sh ]; then /usr/local/bin/pre-stop.sh fi # Forward signal to main process kill -TERM \u0026#34;$child\u0026#34; 2\u0026gt;/dev/null # Wait for main process to finish cleanly wait \u0026#34;$child\u0026#34; echo \u0026#34;Shutdown complete.\u0026#34; exit 0 } trap cleanup TERM # Start main application in background /usr/local/bin/main-app \u0026amp; child=$! # Wait for child process wait \u0026#34;$child\u0026#34; Key points about this pattern:\nThe main process runs in the background with \u0026amp; so the shell can receive signals child=$! captures its PID The trap cleanup TERM registers the handler The final wait \u0026quot;$child\u0026quot; blocks until the child exits, preserving the exit code The cleanup function forwards SIGTERM to the child and waits for it A Dockerfile using this:\n1 2 3 4 5 6 7 8 FROM alpine:3.20 COPY entrypoint.sh /entrypoint.sh RUN chmod +x /entrypoint.sh COPY main-app /usr/local/bin/main-app ENTRYPOINT [\u0026#34;/entrypoint.sh\u0026#34;] Custom Stop Signals for Advanced Use Some applications use SIGUSR1 or SIGUSR2 for log rotation or configuration reload. If your application ties SIGTERM to log rotation rather than shutdown, you can change the stop signal:\n1 2 3 4 5 services: app: image: myapp:latest stop_signal: SIGUSR2 stop_grace_period: 60s The container receives SIGUSR2 instead of SIGTERM when docker stop is called. This is useful when your application uses SIGTERM for non-fatal actions and you need a different signal to trigger shutdown.\nApplication-Specific Graceful Shutdown Patterns Different applications need different shutdown strategies. Here are the patterns for the most common homelab services.\nPostgreSQL PostgreSQL handles SIGTERM natively by entering \u0026ldquo;fast shutdown\u0026rdquo; mode. It disallows new connections, terminates existing connections, flushes the WAL to disk, and syncs data files. It just needs enough time.\n1 2 3 4 5 6 services: postgres: image: postgres:17 stop_grace_period: 120s volumes: - pgdata:/var/lib/postgresql/data When Postgres shuts down cleanly, you see these log entries:\nLOG: received fast shutdown request LOG: aborting any active transactions LOG: database system is shut down If you see \u0026ldquo;database system was not properly shut down\u0026rdquo; on the next start, your grace period is too short.\nNginx Nginx responds to two signals differently:\nSIGTERM: Shuts down immediately, dropping all active connections SIGQUIT: Gracefully shuts down, finishing in-flight requests before exiting For production services, always use SIGQUIT:\n1 2 3 4 5 services: nginx: image: nginx:alpine stop_signal: SIGQUIT stop_grace_period: 30s You can verify in the logs that Nginx completed its graceful shutdown:\n2026/06/12 10:30:00 [notice] 1#1: signal 3 (SIGQUIT) received, shutting down 2026/06/12 10:30:05 [notice] 21#21: gracefully shutting down 2026/06/12 10:30:05 [notice] 21#21: exiting Node.js Applications Node.js does not handle SIGTERM by default. Your application code must register a handler:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 const server = require(\u0026#39;http\u0026#39;).createServer((req, res) =\u0026gt; { res.end(\u0026#39;ok\u0026#39;); }); function gracefulShutdown() { console.log(\u0026#39;Received SIGTERM, shutting down gracefully...\u0026#39;); server.close(() =\u0026gt; { console.log(\u0026#39;HTTP server closed\u0026#39;); // Close database connections, flush logs, etc. process.exit(0); }); // Force shutdown after 30 seconds setTimeout(() =\u0026gt; { console.error(\u0026#39;Forced shutdown after timeout\u0026#39;); process.exit(1); }, 30000); } process.on(\u0026#39;SIGTERM\u0026#39;, gracefulShutdown); server.listen(3000, () =\u0026gt; console.log(\u0026#39;Server listening on port 3000\u0026#39;)); Combine this with init: true in your compose file:\n1 2 3 4 5 services: node-app: build: . init: true stop_grace_period: 60s Python Applications Python applications need explicit signal registration as well:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 import signal import sys import time running = True def handle_sigterm(signum, frame): global running print(\u0026#34;Received SIGTERM, shutting down gracefully...\u0026#34;) running = False signal.signal(signal.SIGTERM, handle_sigterm) try: while running: # Main application loop time.sleep(1) except KeyboardInterrupt: pass finally: print(\u0026#34;Cleaning up resources...\u0026#34;) # Close database connections, flush buffers, write state files sys.exit(0) Docker Compose Restart Policy and Shutdown Interaction The restart policy in Docker Compose interacts with shutdown signals in a way that matters during updates and maintenance.\nWhen you run docker compose up -d after changing a service definition, Docker Compose stops the old container (using the signal flow above) and starts the new one. If the old container exits with code 137 (SIGKILL), the restart policy does not trigger because Docker distinguishes between manual stops and crashed exits. A real crash — where the process exits unexpectedly — would trigger the restart policy, but a docker stop followed by SIGKILL is treated as intentional.\n1 2 3 4 5 6 7 8 9 10 11 12 13 services: app: image: myapp:latest init: true stop_grace_period: 60s stop_signal: SIGTERM healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;wget\u0026#34;, \u0026#34;-q\u0026#34;, \u0026#34;http://localhost:8080/health\u0026#34;] interval: 30s timeout: 10s retries: 3 start_period: 40s restart: unless-stopped The restart: unless-stopped policy ensures the container starts on boot and after crashes, while the stop_grace_period and init: true ensure that manual restarts do not cause data loss. The healthcheck ensures the new container is actually serving traffic before Compose considers the update complete when using --wait.\nTo update a stack safely:\n1 2 docker compose pull docker compose up -d --wait --remove-orphans The --wait flag tells Compose to wait until all services pass their health checks before returning. Combined with proper stop_grace_period values, this gives you zero-downtime updates for most services.\nMonitoring and Troubleshooting Shutdown Issues When containers are not shutting down gracefully, the evidence is in the exit codes and logs.\nCheck Exit Codes Exit code 137 means the container was killed by SIGKILL. Exit code 143 means it received SIGTERM and exited on its own.\n1 2 docker ps -a --filter \u0026#34;status=exited\u0026#34; \\ --format \u0026#34;table {{.Names}}\\t{{.ExitCode}}\\t{{.Status}}\u0026#34; List only containers killed by SIGKILL:\n1 2 3 4 5 docker ps -a --filter \u0026#34;status=exited\u0026#34; \\ --format \u0026#34;{{.Names}}\u0026#34; \\ | xargs -I {} sh -c \u0026#39; \\ code=$(docker inspect {} --format \u0026#34;{{.State.ExitCode}}\u0026#34;); \\ [ \u0026#34;$code\u0026#34; = \u0026#34;137\u0026#34; ] \u0026amp;\u0026amp; echo \u0026#34;{}: KILLED (exit $code)\u0026#34;\u0026#39; Watch Docker Events in Real Time 1 docker events --filter \u0026#39;event=die\u0026#39; --filter \u0026#39;event=kill\u0026#39; This shows each container die event with the exit code. When you see exit code 137 regularly, that service needs a longer stop_grace_period or proper signal handling.\nInterpret Log Output Check container logs for shutdown-related messages:\n1 2 docker logs postgres | grep -i \u0026#34;shutdown\u0026#34; docker logs nginx | grep -i \u0026#34;signal\u0026#34; If Nginx logs show \u0026ldquo;signal 3 (SIGQUIT)\u0026rdquo; the graceful signal is working. If Postgres logs show no shutdown messages at all, the container was killed before Postgres could write its shutdown entry.\nIncrease Grace Periods Conservatively Start with these values and adjust based on observation:\nService stop_grace_period Notes PostgreSQL 120s Needs time to flush WAL and sync data MariaDB 120s Same as Postgres for InnoDB flush Redis 30s Quick if RDB persistence is minimal Nginx 30s SIGQUIT drains connections in seconds Node.js 60s Depends on in-flight request volume Python 30s Usually fast unless long-running tasks Summary Docker container graceful shutdown follows a simple flow: SIGTERM, wait, then SIGKILL. The defaults work for stateless services but cause data corruption in stateful ones. Fix it in three steps:\nSet stop_grace_period in your docker-compose.yml — 120s for databases, 30-60s for application servers Enable init: true — Docker\u0026rsquo;s built-in tini handles signal forwarding and zombie reaping Use the right signal — SIGQUIT for Nginx, SIGTERM for most other services; implement trap handlers in custom entrypoints Check your exit codes. If any container exits with code 137, your grace period is too short or your signal handling is broken. Fix it before the next docker compose down saves your data, not destroys it.\n","permalink":"https://blog.gntech.me/posts/2026-06-12-docker-container-graceful-shutdown-signals/","summary":"Configure proper graceful shutdown for Docker containers in your homelab. Covers stop signals, stop_grace_period, init process management with tini/dumb-init, and SIGTERM trap handlers to prevent data corruption during restarts.","title":"Docker Container Graceful Shutdown — Signals, Stop Grace Period, and Trap Handlers"},{"content":"A container can consume space through its writable layer, persistent volumes, bind mounts, and logs. Limiting one path does not limit the others. Start by identifying the active Docker backend and the filesystem holding each kind of data.\nVerification status Area Evidence and limits Source review Docker storage and per-container storage-option references reviewed on October 31, 2026. Runtime No filesystem creation, migration, quota assignment, or disk-filling test was executed. Scope Backend and capacity planning; changes require a disposable storage lab and a recovery plan. Inspect the actual host 1 2 3 4 5 docker info docker system df -v findmnt -T /var/lib/docker -o TARGET,SOURCE,FSTYPE,OPTIONS df -hT /var/lib/docker df -i /var/lib/docker If docker info reports a different Docker root, inspect that path instead. findmnt -T also identifies a filesystem inherited from a parent mount; grepping for a dedicated /var/lib/docker mount can miss it.\nFresh Docker Engine 29 installations use the containerd image store by default; upgraded systems may retain classic overlay2. These backends are different. This article\u0026rsquo;s classic overlay2.size discussion does not establish support for a containerd snapshotter. See Docker\u0026rsquo;s containerd storage guide.\nWhat each limit covers Data path Limit to investigate Classic overlay2 writable layer Supported storage-driver size option on a compatible XFS filesystem with project quotas. Bind-mounted application directory Filesystem project quota or a separately sized filesystem on that directory\u0026rsquo;s actual backing store. Named volume Identify its driver and backing path; a writable-layer limit does not cap volume content. Container logs Logging-driver rotation, log delivery, and host capacity monitoring. The classic overlay2 quota applies to the writable upper layer, not the sum of all shared lower image layers. Lower layers still occupy host space. Docker documents a per-container --storage-opt size=... option for supported drivers; the earlier claim that it was unavailable was incorrect. Compose also exposes service storage_opt; verify support against the installed backend.\nSee Docker run storage options and the classic overlay2 prerequisites. An XFS ftype=1 capability and project-quota mount options are distinct checks. Filesystem creation alone does not enable mounted quota enforcement.\nPlan storage changes before applying them Do not format a guessed /dev/sdb1, overwrite daemon.json, or delete an old Docker directory as a preparation step. First inventory the real Docker/containerd paths, mounts, daemon configuration, named volumes, image digests, and application recovery materials.\nPrepare a separately identified spare device in a disposable environment. Check existing XFS project IDs and names before allocating any new ones; /etc/projects and /etc/projid are shared host configuration. A directory can only be managed as part of the intended project\u0026rsquo;s filesystem hierarchy. Verify accounting and enforcement with the installed xfs_quota documentation before starting writers.\nA production storage migration needs stopped writers and an explicit plan for Docker service/socket activation, metadata-preserving copying, mount persistence, application verification, and rollback. Keep the original data until restore and rollback have been tested. Changing storage drivers can make existing images and containers inaccessible even though their files remain on disk.\nIn a quota lab, start with a small, bounded workload and monitor free space outside the container. A missing quota can turn a multi-gigabyte dd test into a host outage. Confirm the observed failure is the intended quota, not exhaustion elsewhere. This guide provides no command that deliberately fills a production filesystem.\nBackups and monitoring A quota prevents one kind of growth; it does not supply a backup. Use the database recovery lab for consistent PostgreSQL dumps and the Restic recovery guide for encrypted archive recovery.\nMonitor bytes, inodes, quota usage, database write errors, log growth, and successful backup age. A quota can protect host headroom while still making the limited application fail writes; test that application\u0026rsquo;s response before relying on the limit.\n","permalink":"https://blog.gntech.me/posts/2026-06-11-docker-container-storage-quotas-xfs/","summary":"Distinguish classic overlay2 writable-layer limits from application volume quotas and plan XFS changes without deleting Docker data.","title":"Docker Storage Quotas — Check the Backend and the Data Path First"},{"content":"You run ten self-hosted services and every single one needs a valid TLS certificate. Certbot needs port 80 open and stops working when Traefik is already listening there. Wildcard certificates require DNS plugins most clients do not bundle. Renewal scripts break when you migrate containers between hosts.\nacme.sh solves all of this with a single shell script that supports over three hundred DNS providers, runs in Docker without runtime dependencies, and handles auto-renewal through a built-in cron mode.\nThis guide walks through a complete acme.sh deployment on Docker — issuing wildcard Let\u0026rsquo;s Encrypt certificates via DNS-01 challenge, automating renewal, and deploying certs to Traefik, Nginx, and Caddy on the same Docker network.\nWhy acme.sh Over Certbot for Homelab Certificates Certbot is the default ACME client for most administrators, but it has structural disadvantages in a Docker-forward homelab:\nDependency footprint: Certbot is a Python application with plugins that pull in dozens of packages. acme.sh is a single POSIX shell script with zero runtime dependencies beyond curl and openssl. DNS provider support: Certbot\u0026rsquo;s DNS-01 plugins are per-provider packages you install separately. acme.sh bundles support for over three hundred DNS providers in the script itself — no extra install step. Multiple CA endpoints: acme.sh supports Let\u0026rsquo;s Encrypt, ZeroSSL, Buypass, Google Trust Services, and custom ACME endpoints. Switching CAs is a one-flag change. Key types: ECDSA (prime256v1, secp384r1) and RSA (2048, 4096) are first-class options, controlled by a single --keylength flag. Deploy hooks: acme.sh ships with built-in deploy hooks for nginx, apache, haproxy, cpanel, and custom commands. You write one reload command and every renewal triggers it. If you already run Certbot and it works, keep it. If you are starting fresh or want wildcard certs without exposing port 80, acme.sh is the cleaner path.\nInstalling acme.sh in Docker The official image neilpang/acme.sh bundles the script with cron support. Create a compose file that mounts persistent volumes for the acme.sh data directory and the certificate output directory:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 # acme-compose.yml services: acme.sh: image: neilpang/acme.sh:latest container_name: acme.sh restart: unless-stopped environment: - CF_Token=${CF_Token} - CF_Zone_ID=${CF_Zone_ID} - LE_WORKING_DIR=/acme.sh - LE_CONFIG_HOME=/acme.sh volumes: - ./acme_data:/acme.sh - ./certs:/certs - /var/run/docker.sock:/var/run/docker.sock:ro networks: - cert-net command: daemon networks: cert-net: name: cert-net driver: bridge Two volumes handle persistence:\n./acme_data — stores account keys, certificate files, and configuration. This is the only volume you need for restart survival. ./certs — output directory where deploy hooks write certificates for consumption by reverse proxies. Optional if you use deploy hooks that write directly to mounted volumes. The daemon command keeps the container running and enables the built-in cron for auto-renewal. For one-shot issuance you can also use run --rm mode:\n1 docker compose -f acme-compose.yml run --rm acme.sh --register-account -m admin@example.com This registers your account email with the default Let\u0026rsquo;s Encrypt production endpoint. The --server flag switches to ZeroSSL or staging:\n1 2 3 docker compose -f acme-compose.yml run --rm acme.sh \\ --register-account -m admin@example.com \\ --server zerossl DNS-01 Challenge for Wildcard Certificates The HTTP-01 challenge requires port 80 on the host and cannot issue wildcard certificates. The DNS-01 challenge requires no open ports and can issue *.example.com certs that cover every subdomain.\nCloudflare DNS API Integration Cloudflare is the most common provider in homelabs. Generate an API token from the Cloudflare dashboard with Zone:DNS:Edit permission, then export it to the environment:\n1 2 export CF_Token=\u0026#34;your_cloudflare_api_token_here\u0026#34; export CF_Zone_ID=\u0026#34;your_zone_id_here\u0026#34; Issue a wildcard certificate covering both *.example.com and the bare domain:\n1 2 3 4 5 6 docker compose -f acme-compose.yml run --rm acme.sh \\ --issue \\ --dns dns_cf \\ -d *.example.com \\ -d example.com \\ --keylength ec-256 The ec-256 flag requests an ECDSA P-256 key. ECDSA certificates are smaller, faster to negotiate, and supported by every modern client. For RSA compatibility with legacy clients, omit --keylength or use --keylength 2048.\nacme.sh creates a DNS TXT record at _acme-challenge.example.com, polls for propagation, and removes it after validation. The whole process takes about thirty seconds.\nOther DNS Providers The same pattern works for dozens of providers. Set the required environment variables and change the --dns flag:\nProvider Environment Variables \u0026ndash;dns flag Cloudflare CF_Token, CF_Zone_ID dns_cf DigitalOcean DO_API_KEY dns_dgon AWS Route53 AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY dns_aws Hetzner HETZNER_API_TOKEN dns_hetzner Gandi GANDI_LIVEDNS_KEY dns_gandi_livedns DuckDNS DuckDNS_Token dns_duckdns Run acme.sh --list-dns inside the container to see every supported provider.\nAuto-Renewal Setup acme.sh includes a cron mode that checks all certificates daily and renews those expiring within sixty days. The daemon start command in the compose file enables this automatically.\nTo trigger renewal manually or test the cron logic:\n1 docker compose -f acme-compose.yml exec acme.sh --cron --force Enable the auto-upgrade feature so acme.sh updates itself:\n1 docker compose -f acme-compose.yml run --rm acme.sh --upgrade --auto-upgrade This writes the upgrade configuration to ~/.acme.sh/account.conf inside the container. Combined with the daily cron, your certificate pipeline becomes fully self-maintaining.\nSystemd Timer Alternative If you prefer host-level scheduling over Docker cron, disable the daemon mode and run acme.sh via a systemd timer:\n1 2 3 4 5 6 7 8 # /etc/systemd/system/acme-renewal.service [Unit] Description=acme.sh certificate renewal After=docker.service [Service] Type=oneshot ExecStart=docker compose -f /opt/acme/acme-compose.yml run --rm acme.sh --cron 1 2 3 4 5 6 7 8 9 10 11 # /etc/systemd/system/acme-renewal.timer [Unit] Description=Daily acme.sh renewal Requires=acme-renewal.service [Timer] OnCalendar=daily Persistent=true [Install] WantedBy=timers.target Enable and start:\n1 2 sudo systemctl daemon-reload sudo systemctl enable --now acme-renewal.timer Deploy Hooks — Updating Reverse Proxies A certificate renewal is useless if running services do not reload with the new key pair. acme.sh deploy hooks automate this.\nTraefik Integration Traefik reads certificates from file. Create a deploy hook that writes the renewal output to the Traefik dynamic configuration volume:\n1 2 3 4 5 6 7 docker compose -f acme-compose.yml run --rm acme.sh \\ --install-cert -d *.example.com \\ --ecc \\ --cert-file /certs/traefik/cert.pem \\ --key-file /certs/traefik/key.pem \\ --fullchain-file /certs/traefik/fullchain.pem \\ --reloadcmd \u0026#34;docker exec traefik kill -HUP 1\u0026#34; Configure Traefik to serve the file-based certificates:\n1 2 3 4 5 # traefik-dynamic.yml — mounted into /etc/traefik/dynamic tls: certificates: - certFile: /certs/traefik/fullchain.pem keyFile: /certs/traefik/key.pem Nginx Integration For Nginx, the deploy hook writes certificates and reloads the service:\n1 2 3 4 5 6 docker compose -f acme-compose.yml run --rm acme.sh \\ --install-cert -d *.example.com \\ --ecc \\ --fullchain-file /etc/nginx/certs/fullchain.pem \\ --key-file /etc/nginx/certs/key.pem \\ --reloadcmd \u0026#34;docker exec nginx nginx -s reload\u0026#34; The Nginx config references the shared cert volume:\n1 2 3 4 5 6 7 8 9 10 11 server { listen 443 ssl; server_name app.example.com; ssl_certificate /etc/nginx/certs/fullchain.pem; ssl_certificate_key /etc/nginx/certs/key.pem; location / { proxy_pass http://app:3000; } } Caddy Integration Caddy can use certificates from a mounted directory with the file_server or tls directive. The deploy hook writes to the shared path:\n1 2 3 4 5 6 docker compose -f acme-compose.yml run --rm acme.sh \\ --install-cert -d *.example.com \\ --ecc \\ --fullchain-file /etc/caddy/certs/fullchain.pem \\ --key-file /etc/caddy/certs/key.pem \\ --reloadcmd \u0026#34;docker exec caddy caddy reload --config /etc/caddy/Caddyfile\u0026#34; And the Caddyfile references the cert files:\napp.example.com { tls /etc/caddy/certs/fullchain.pem /etc/caddy/certs/key.pem reverse_proxy app:3000 } Complete Docker Compose Stack with Traefik Here is a full working stack that issues certificates and serves applications behind Traefik:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 # docker-compose.yml services: acme.sh: image: neilpang/acme.sh:latest container_name: acme.sh restart: unless-stopped environment: - CF_Token=${CF_Token} - CF_Zone_ID=${CF_Zone_ID} - LE_WORKING_DIR=/acme.sh volumes: - ./acme_data:/acme.sh - ./certs:/certs networks: - cert-net command: daemon traefik: image: traefik:v3.3 container_name: traefik restart: unless-stopped ports: - 80:80 - 443:443 volumes: - /var/run/docker.sock:/var/run/docker.sock:ro - ./traefik.yml:/etc/traefik/traefik.yml:ro - ./traefik-dynamic.yml:/etc/traefik/dynamic/traefik-dynamic.yml:ro - ./certs:/certs:ro networks: - cert-net - proxy whoami: image: traefik/whoami container_name: whoami labels: - traefik.enabled=true - traefik.http.routers.whoami.rule=Host(`whoami.example.com`) - traefik.http.routers.whoami.tls=true - traefik.http.services.whoami.loadbalancer.server.port=80 networks: - proxy networks: cert-net: name: cert-net driver: bridge proxy: name: proxy driver: bridge The certificate issuance flow: acme.sh runs the DNS-01 challenge, stores the certificate in ./certs/traefik/, and Traefik references those files via the shared volume. On renewal, the --reloadcmd signals Traefik to reload configuration without dropping connections.\nVerification and Troubleshooting Check Certificate Details Use openssl to verify certificate attributes:\n1 2 3 docker compose -f acme-compose.yml run --rm acme.sh --list openssl x509 -in ./certs/traefik/fullchain.pem -noout -text | grep -E \u0026#34;Subject:|Not Before|Not After\u0026#34; Review acme.sh Logs acme.sh writes detailed logs to $LE_WORKING_DIR/acme.sh.log:\n1 tail -50 ./acme_data/acme.sh.log Common Issues DNS propagation delay: Some providers take longer than others. acme.sh waits 120 seconds by default. Increase with --dnssleep 300 for slow providers.\nRate limits: Let\u0026rsquo;s Encrypt allows 50 certificates per week per registered domain. Use the staging endpoint during testing to avoid exhausting production limits:\n1 acme.sh --issue --dns dns_cf -d *.example.com --server letsencrypt-staging Permission errors on cert files: Ensure the reverse proxy container can read the certificate files. Use chmod 644 on the cert volume or run the proxy container with the same UID that wrote the certificates.\nCron not running inside container: Verify the container is in daemon mode and cron is enabled by checking the container logs:\n1 docker logs acme.sh | grep cron If cron is not active, exec into the container and start the daemon manually:\n1 docker exec acme.sh /bin/sh -c \u0026#34;crond -b \u0026amp;\u0026amp; acme.sh --cron\u0026#34; Conclusion acme.sh eliminates every friction point in homelab certificate management. The single-script architecture works anywhere a shell runs. The DNS-01 provider matrix covers practically every DNS service in use today. Deploy hooks bridge the gap between certificate issuance and service reload so that renewal is completely hands-off.\nFor a homelab running ten, twenty, or fifty services behind a reverse proxy, setting up acme.sh once with wildcard certificates and auto-renewal saves more time than any other TLS management approach. The compose stack in this guide can be running in ten minutes — and then never needs attention again unless you add a new domain.\nThe full configuration files are available on the blog repository. Drop the compose file on any Docker host, set your Cloudflare API token, and your services will never serve an expired certificate again.\n","permalink":"https://blog.gntech.me/posts/2026-06-11-acme-sh-letsencrypt-certificates-homelab-docker/","summary":"Automate Let\u0026rsquo;s Encrypt SSL/TLS certificates in your homelab using acme.sh and Docker. Includes DNS-01 wildcard certs, auto-renewal, and integration with Traefik, Nginx Proxy Manager, and Caddy.","title":"acme.sh Let's Encrypt Certificates for Homelab Docker Services"},{"content":"Your Proxmox host runs a dozen Docker containers, a handful of LXCs, and has SSH exposed to the local network. You trust your users, but you have no record of who ran docker exec --privileged into that production container or when someone edited /etc/pve/user.cfg.\nThis is the security blind spot in most homelabs: the kernel can log everything, but nobody enables it.\nLinux auditd is the built-in kernel-level auditing framework that captures syscalls, file access, process execution, and configuration changes. It costs no additional software, minimal CPU overhead, and produces structured logs you can query with familiar tools. This guide walks through a complete auditd deployment for a Docker and Proxmox homelab — from basic install to daily alert queries.\nWhat Is auditd? The Kernel Audit Framework auditd is not a userland add-on. It sits at the kernel level, intercepting system calls through the Linux Audit subsystem and forwarding events to userspace via a netlink socket. The architecture has three layers:\nKernel — captures syscalls and file watches via the audit framework compiled into the kernel auditd daemon — receives events from the kernel, writes them to disk, and optionally forwards them to remote log servers ausearch / aureport — command-line tools for searching and summarizing audit logs The systemd integration is minimal: audit.socket activates the daemon on demand, and auditd.service manages the full lifecycle. You explicitly enable it — it stays off by default on most distributions.\nInstalling and Enabling auditd Installation is a single package on Debian-based systems:\n1 2 3 4 apt update apt install -y auditd audispd-plugins systemctl enable --now auditd systemctl status auditd Verify the kernel audit subsystem is active:\n1 auditctl -s Expected output shows enabled 1 and a valid pid for the daemon.\nThe main configuration lives at /etc/audit/auditd.conf. Defaults are conservative for general-purpose systems, but for a homelab bump these values:\nmax_log_file = 128 max_log_file_action = rotate space_left_action = email admin_space_left_action = suspend disk_full_action = suspend max_log_file_action = rotate prevents the daemon from stopping when the log reaches the size limit. Set space_left_action to email if you have local mail configured, otherwise use syslog to forward to journald.\nEssential Audit Rules for Homelab Security Rules are added with auditctl at runtime and persist through rule files. These three categories cover the critical homelab attack surface.\nMonitor Docker Daemon and Container Execution The Docker binary and its data directory are the primary targets:\n1 2 3 4 auditctl -w /usr/bin/docker -p x -k docker-execution auditctl -w /usr/bin/containerd -p x -k containerd-exec auditctl -w /var/lib/docker -p wa -k docker-data auditctl -w /etc/docker/daemon.json -p wa -k docker-config The first rule watches for any execution of docker — every docker run, docker exec, docker ps that hits disk triggers an event. The -k flag assigns a key for easy filtering with ausearch. The -p x flag watches execute permission; -p wa watches write and attribute changes for configuration and data paths.\nMonitor Proxmox Host Operations Proxmox management tools — qm for VMs, pct for containers — are CLI binaries that should only run during planned maintenance:\n1 2 3 4 5 auditctl -w /usr/sbin/qm -p x -k proxmox-qm auditctl -w /usr/sbin/pct -p x -k proxmox-pct auditctl -w /usr/sbin/pvesh -p x -k proxmox-api auditctl -w /etc/pve -p wa -k proxmox-config auditctl -w /var/log/pve -p wa -k proxmox-logs The /etc/pve directory is a PMXA filesystem (FUSE) shared across cluster nodes. Watching it captures changes to VM configs, storage definitions, and user permissions through the Proxmox web interface or CLI. The /var/log/pve watch catches task log writes — who started, stopped, or migrated a VM.\nMonitor Critical System Paths User database, SSH configuration, and cron are common privilege escalation vectors:\n1 2 3 4 5 6 7 8 auditctl -w /etc/passwd -p wa -k user-db auditctl -w /etc/shadow -p wa -k user-db auditctl -w /etc/group -p wa -k user-db auditctl -w /etc/ssh/sshd_config -p wa -k ssh-config auditctl -w /etc/ssh/sshd_config.d/ -p wa -k ssh-config auditctl -w /root/.ssh -p wa -k ssh-keys auditctl -w /etc/crontab -p wa -k cron-config auditctl -w /etc/cron.d/ -p wa -k cron-config Monitor Sensitive System Calls File-watch rules only cover specific paths. Syscall rules catch actions anywhere in the filesystem and on kernel objects:\n1 2 3 4 5 auditctl -a always,exit -S execve -F key=process-exec auditctl -a always,exit -S execveat -F key=process-exec auditctl -a always,exit -S mount -S umount2 -k mount-changes auditctl -a always,exit -S sethostname -S setdomainname -k hostname-changes auditctl -a always,exit -S unlink -S unlinkat -S rmdir -k file-deletion The execve syscall rule generates an event for every binary execution on the system — including scripts launched by interpreters like Python and Bash. This is your catch-all for discovering unauthorized processes. Combine it with the docker-execution rules to correlate: was this docker exec an authorized deployment or a compromise?\nMaking Audit Rules Persistent Runtime rules with auditctl do not survive a reboot. For persistence, write rule files to /etc/audit/rules.d/ and load them with augenrules:\n1 2 3 4 5 6 cat \u0026gt; /etc/audit/rules.d/docker.rules \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; -w /usr/bin/docker -p x -k docker-execution -w /usr/bin/containerd -p x -k containerd-exec -w /var/lib/docker -p wa -k docker-data -w /etc/docker/daemon.json -p wa -k docker-config EOF Repeat for Proxmox, system paths, and syscall rules:\n1 2 3 4 5 6 7 cat \u0026gt; /etc/audit/rules.d/proxmox.rules \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; -w /usr/sbin/qm -p x -k proxmox-qm -w /usr/sbin/pct -p x -k proxmox-pct -w /usr/sbin/pvesh -p x -k proxmox-api -w /etc/pve -p wa -k proxmox-config -w /var/log/pve -p wa -k proxmox-logs EOF 1 2 3 4 5 6 7 8 9 10 cat \u0026gt; /etc/audit/rules.d/system.rules \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; -w /etc/passwd -p wa -k user-db -w /etc/shadow -p wa -k user-db -w /etc/group -p wa -k user-db -w /etc/ssh/sshd_config -p wa -k ssh-config -w /etc/ssh/sshd_config.d/ -p wa -k ssh-config -w /root/.ssh -p wa -k ssh-keys -w /etc/crontab -p wa -k cron-config -w /etc/cron.d/ -p wa -k cron-config EOF 1 2 3 4 5 6 cat \u0026gt; /etc/audit/rules.d/syscall.rules \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; -a always,exit -S execve -S execveat -F key=process-exec -a always,exit -S mount -S umount2 -k mount-changes -a always,exit -S sethostname -S setdomainname -k hostname-changes -a always,exit -S unlink -S unlinkat -S rmdir -k file-deletion EOF Load and verify:\n1 2 augenrules --load auditctl -l Querying Audit Logs with ausearch and aureport Once rules are active and events are flowing, the real value comes from querying the log. These commands are your daily audit interface.\nSearch by Rule Key 1 2 3 4 5 6 7 8 # All Docker executions today ausearch -k docker-execution --start today --interpret # All file access on /etc/pve this hour ausearch -k proxmox-config --start recent --interpret # Failed file access (permission denied) ausearch -f /etc/shadow --failed --start today Generate Summary Reports 1 2 3 4 5 6 7 8 9 10 11 # Top executed binaries aureport -x --summary # File access summary by count aureport -f # Login events aureport -l # User activity summary aureport -u --failed Watch Live Events 1 tail -f /var/log/audit/audit.log | ausearch --interpret --input - JSON Output for Log Aggregation 1 ausearch -k docker-execution --start today --format json This is directly ingestible by Loki, Elasticsearch, or any JSON log pipeline. If you run the Grafana Loki stack in your homelab, pipe auditd events there for dashboards.\nIntegrating auditd with systemd-journald auditd events appear in systemd-journald via the audispd syslog plugin. Enable it:\n1 systemctl enable --now auditd View audit events in journald:\n1 2 journalctl -t auditd --since today journalctl _TRANSPORT=audit --since today | tail -50 For structured log aggregation, set up the audisp-remote plugin to forward to a central syslog server or use the built-in syslog forwarding in /etc/audit/audisp-remote.conf:\nremote_server = 10.0.20.50 port = 601 transport = tcp Docker-Specific Auditing: Containers and the Docker Socket auditd complements Docker\u0026rsquo;s built-in event system. Docker events tell you containers started and stopped; auditd tells you who executed the docker command and which user on the host triggered it.\nWatch the Docker socket for API access:\n1 auditctl -w /var/run/docker.sock -p rwa -k docker-socket This catches any process reading from or writing to the Docker socket — useful for detecting unauthorized containers or tooling that accesses Docker without going through the CLI.\nTo catch docker exec and docker attach sessions, the execve syscall rule combined with the docker binary watch gives you the full picture: which user ran docker exec -it nginx sh and at what time.\n1 ausearch -k docker-execution --start today --interpret | grep \u0026#34;docker exec\u0026#34; Preventing Audit Log Tampering Audit logs are only useful if they survive an attack. Once all rules are loaded, lock the audit configuration so that no one — including root — can disable or modify rules without a reboot:\n1 auditctl -e 2 Set this as the last rule in one of your rule files to apply it automatically:\n1 2 echo \u0026#34;-e 2\u0026#34; \u0026gt;\u0026gt; /etc/audit/rules.d/syscall.rules augenrules --load After this, auditctl -e 0 returns permission denied. Only a reboot clears it, and during boot the rules reload before network services start.\nFor additional protection, forward logs off-host. The audisp-remote plugin sends events to a central syslog or a dedicated logging VM. Set it up:\n1 2 3 4 5 cat \u0026gt; /etc/audit/audisp-remote.conf \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; remote_server = 10.0.20.50 port = 601 transport = tcp EOF On the remote server, run a syslog listener:\n1 nc -lk -p 601 \u0026gt; /var/log/remote-audit.log Storage Management and Log Rotation auditd logs to /var/log/audit/audit.log by default. With active rules on a busy homelab host, expect 10-50 MB per day depending on workload. Set adequate limits:\n# /etc/audit/auditd.conf max_log_file = 128 max_log_file_action = rotate num_logs = 5 This keeps 5 rotated files of 128 MB each — roughly 640 MB of history. Adjust based on your storage budget. If you forward to remote logging, you can reduce it further.\nThe system logrotate configuration at /etc/logrotate.d/auditd handles rotation via signal. Verify it exists and reload if needed:\n1 logrotate -f /etc/logrotate.d/auditd Verification and Testing After deploying, run a quick smoke test:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 # List loaded rules (should show 15-25 rules depending on your config) auditctl -l | wc -l # Trigger a Docker event docker run --rm alpine echo \u0026#34;test audit\u0026#34; # Query for it ausearch -k docker-execution --start today --interpret | tail -10 # Check audit status auditctl -s # Verify immutable mode (if set) auditctl -e 2 || echo \u0026#34;NOT immutable — set -e 2\u0026#34; Expected output from auditctl -s shows enabled 1, pid of the running daemon, rate_limit (default 0 = unlimited), and backlog_limit (default 64).\nConclusion auditd is one of the most underused security tools in homelabs. It ships with every Linux kernel, costs almost nothing in overhead, and produces forensic-grade logs of everything happening on your host. With the rules in this guide, you cover Docker execution, Proxmox VM management, critical file changes, and system-call-level process monitoring — all with a one-time setup.\nOnce auditd is running, it works silently in the background. The logs accumulate in /var/log/audit/audit.log, ready to answer the question no homelab owner wants to face: What happened, when, and who did it?\nDeploy auditd alongside CrowdSec or Fail2ban for a complete host security posture. Set up remote log forwarding if you have a second host. Configure it once, and forget about it — until the day you need it.\n","permalink":"https://blog.gntech.me/posts/2026-06-11-linux-auditd-security-auditing-homelab/","summary":"Configure Linux auditd to watch container starts, file changes, and Proxmox host activity. Includes auditctl rules, ausearch queries, and Docker integration for comprehensive homelab security auditing.","title":"Linux auditd Security Auditing — Monitor Proxmox and Docker Activity"},{"content":"Your Proxmox host runs Docker behind a Gluetun WireGuard gateway. Containers start, DNS works, but docker pull hangs. docker build times out fetching packages. Your app container tries to reach an external API through localhost:3128 — but nothing is listening there.\nThis is the Docker proxy configuration problem, and it hits every homelab that routes traffic through a VPN gateway or forward proxy. The good news: Docker supports proxy configuration at three cleanly separated levels — daemon, CLI, and container — and once you understand how each one works, the fix is straightforward.\nThis guide covers all three layers with real configs you can copy into your homelab today.\nDocker Daemon Proxy Configuration for Image Pulls The Docker daemon needs proxy settings when it pulls images from registries. Without them, docker pull fails silently when the host has no direct internet route.\nThere are two ways to set daemon-level proxy: systemd drop-in files (recommended for systemd-managed Docker) and daemon.json (for dockerd config-based setups).\nSystemd Drop-In Approach Create a drop-in file under the Docker systemd service directory:\n1 sudo mkdir -p /etc/systemd/system/docker.service.d Then create /etc/systemd/system/docker.service.d/proxy.conf:\n1 2 3 4 [Service] Environment=\u0026#34;HTTP_PROXY=http://192.168.1.100:3128\u0026#34; Environment=\u0026#34;HTTPS_PROXY=http://192.168.1.100:3128\u0026#34; Environment=\u0026#34;NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,.local,.internal\u0026#34; Docker accepts both uppercase and lowercase variants (HTTP_PROXY and http_proxy). Set both for maximum application compatibility.\nApply the change and verify:\n1 2 3 sudo systemctl daemon-reload sudo systemctl restart docker systemctl show --property=Environment docker You should see your proxy variables in the output. You can also verify with:\n1 docker info | grep -i proxy Expected output:\nHTTP Proxy: http://192.168.1.100:3128 HTTPS Proxy: http://192.168.1.100:3128 No Proxy: localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,.local,.internal daemon.json Approach If you manage Docker through its JSON config file (/etc/docker/daemon.json), add a proxies block:\n1 2 3 4 5 6 7 { \u0026#34;proxies\u0026#34;: { \u0026#34;http-proxy\u0026#34;: \u0026#34;http://192.168.1.100:3128\u0026#34;, \u0026#34;https-proxy\u0026#34;: \u0026#34;http://192.168.1.100:3128\u0026#34;, \u0026#34;no-proxy\u0026#34;: \u0026#34;localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,.local,.internal\u0026#34; } } This approach is cleaner if you already maintain daemon.json for other settings like storage driver, log driver, or cgroup configuration. The systemd drop-in method takes precedence if both are set.\nAfter editing, restart Docker:\n1 sudo systemctl restart docker Which One to Choose Use the systemd drop-in if:\nYou already configure Docker via systemd You want separation between service-level and config-level settings You use a distribution where Docker is installed via the official repo (Debian, Ubuntu, Fedora) Use daemon.json if:\nYou manage Docker config through Ansible or a configuration management tool You already have a daemon.json with other settings You run Docker in rootless mode Docker CLI Proxy Configuration for Builds When you build images locally with docker build or docker buildx, the build process needs proxy access to download packages. There are two parts to this: getting the proxy into the build context, and consuming it inside the Dockerfile.\nPassing Proxy via Build Arguments 1 2 3 4 5 docker build \\ --build-arg HTTP_PROXY=http://192.168.1.100:3128 \\ --build-arg HTTPS_PROXY=http://192.168.1.100:3128 \\ --build-arg NO_PROXY=localhost,127.0.0.1 \\ -t my-app:latest . Inside the Dockerfile, declare the build args and use them in RUN commands:\n1 2 3 4 5 6 7 8 9 10 FROM debian:bookworm-slim ARG HTTP_PROXY ARG HTTPS_PROXY ARG NO_PROXY RUN apt-get update \u0026amp;\u0026amp; apt-get install -y --no-install-recommends \\ curl \\ ca-certificates \\ \u0026amp;\u0026amp; rm -rf /var/lib/apt/lists/* For Python, npm, or Go builds, the proxy variables are often picked up automatically by the package manager:\n1 2 RUN pip install --proxy=\u0026#34;$HTTP_PROXY\u0026#34; -r requirements.txt RUN npm config set proxy \u0026#34;$HTTP_PROXY\u0026#34; Docker CLI Client Config You can also set proxy in ~/.docker/config.json:\n1 2 3 4 5 6 7 8 9 { \u0026#34;proxies\u0026#34;: { \u0026#34;default\u0026#34;: { \u0026#34;httpProxy\u0026#34;: \u0026#34;http://192.168.1.100:3128\u0026#34;, \u0026#34;httpsProxy\u0026#34;: \u0026#34;http://192.168.1.100:3128\u0026#34;, \u0026#34;noProxy\u0026#34;: \u0026#34;localhost,127.0.0.1\u0026#34; } } } This applies to docker build commands run by the current user. BuildKit (the default builder since Docker Engine 23) reads these settings natively and passes them to the build context.\nPer-Container Proxy Configuration in Docker Compose For runtime proxy configuration — when your running containers need to reach external APIs through a proxy — set environment variables at the service level in your Compose file.\nDirect Environment Variables 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 services: app: image: my-app:latest environment: - HTTP_PROXY=http://proxy:3128 - HTTPS_PROXY=http://proxy:3128 - NO_PROXY=localhost,127.0.0.1,.internal,.local,.cluster.local depends_on: - proxy proxy: image: ubuntu/squid:latest ports: - \u0026#34;3128:3128\u0026#34; volumes: - ./squid.conf:/etc/squid/squid.conf:ro Using an env_file For large stacks with many services sharing the same proxy, use a shared environment file:\nCreate proxy.env:\nHTTP_PROXY=http://proxy:3128 HTTPS_PROXY=http://proxy:3128 NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,.internal,.local,.cluster.local Reference it in every service that needs external access:\n1 2 3 4 5 6 7 8 9 10 services: immich-server: image: ghcr.io/immich-app/immich-server:release env_file: proxy.env # ... syncthing: image: syncthing/syncthing env_file: proxy.env # ... Variable Substitution Pattern You can also use Compose variable substitution to make proxy settings configurable without hardcoding:\n1 2 3 4 5 6 7 services: app: image: my-app:latest environment: - HTTP_PROXY=${HTTP_PROXY:-} - HTTPS_PROXY=${HTTPS_PROXY:-} - NO_PROXY=${NO_PROXY:-localhost,127.0.0.1} Then set the variables in a .env file next to your Compose file:\n1 2 3 HTTP_PROXY=http://192.168.1.100:3128 HTTPS_PROXY=http://192.168.1.100:3128 NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16 Proxy Behind a VPN Gateway If your homelab routes external traffic through a Gluetun or WireGuard gateway container, run Squid or Tinyproxy inside the same network stack so it shares the VPN tunnel:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 services: gluetun: image: qmcgaw/gluetun cap_add: - NET_ADMIN environment: - VPN_SERVICE_PROVIDER=mullvad - VPN_TYPE=wireguard networks: - vpn-net proxy: image: ubuntu/squid:latest network_mode: \u0026#34;service:gluetun\u0026#34; depends_on: gluetun: condition: service_healthy app: image: my-app:latest network_mode: \u0026#34;service:gluetun\u0026#34; environment: - HTTP_PROXY=http://localhost:3128 - HTTPS_PROXY=http://localhost:3128 - NO_PROXY=localhost,127.0.0.1 depends_on: proxy: condition: service_started networks: vpn-net: driver: bridge This pattern gives each container full VPN routing through the proxy while keeping NO_PROXY small since localhost access stays within the same network namespace.\nNO_PROXY — The Critical Bypass List NO_PROXY is the most common source of proxy-related failures in Docker Compose stacks. If your Postgres container is named db and your app tries to reach it at db:5432 through the proxy, the connection fails because Squid doesn\u0026rsquo;t speak PostgreSQL.\nWhat to Include At minimum, add these to NO_PROXY:\nlocalhost and 127.0.0.1 — loopback access All RFC1918 subnets — 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16 Docker internal domain suffixes — .local, .internal, .cluster.local Specific service hostnames if your proxy ignores CIDR notation (some applications don\u0026rsquo;t support CIDR in NO_PROXY) Full NO_PROXY Reference NO_PROXY=localhost,127.0.0.1,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,.local,.internal,.cluster.local,.svc.cluster.local Common NO_PROXY Mistakes Omitting RFC1918 subnets — your Compose services communicate on Docker bridge networks (typically 172.x.x.x), which will be routed through the proxy if not excluded Using wildcards incorrectly — NO_PROXY uses domain suffix matching (.local matches anything.local), not glob patterns Case sensitivity — most applications respect lowercase no_proxy and uppercase NO_PROXY, but set both to be safe CIDR support varies — some applications and libraries don\u0026rsquo;t support CIDR in NO_PROXY. Go\u0026rsquo;s http.ProxyFromEnvironment does; Python\u0026rsquo;s urllib did not before 3.13. Use domain suffixes as a fallback Running a Forward Proxy Container for Your Homelab If you do not already have a forward proxy on your network, running one in Docker is straightforward. Squid is the classic choice, but Tinyproxy is lighter for simple pass-through proxying.\nTinyproxy — Minimal Forward Proxy 1 2 3 4 5 6 7 8 services: tinyproxy: image: vimagick/tinyproxy:latest ports: - \u0026#34;127.0.0.1:8888:8888\u0026#34; environment: - TINYPROXY_LOGLEVEL=Warning - TINYPROXY_ALLOW=10.0.0.0/8,172.16.0.0/12,192.168.0.0/16 Bind to 127.0.0.1 when the proxy runs on the same host as the containers; bind to the Docker bridge IP when other hosts need it.\nSquid with Authentication 1 2 3 4 5 6 7 8 services: squid: image: ubuntu/squid:latest ports: - \u0026#34;3128:3128\u0026#34; volumes: - ./squid.conf:/etc/squid/squid.conf:ro - ./passwd:/etc/squid/passwd:ro With a squid.conf that requires authentication:\nauth_param basic program /usr/lib/squid/basic_ncsa_auth /etc/squid/passwd auth_param basic children 5 auth_param basic realm Squid Proxy acl authenticated proxy_auth REQUIRED http_access allow authenticated http_access deny all http_port 3128 Create the password file:\n1 htpasswd -c ./passwd proxyuser Then reference the credentials in your container proxy variables:\nHTTP_PROXY=http://proxyuser:password@squid:3128 Verification and Troubleshooting Test Daemon Proxy 1 2 3 4 5 # Check daemon proxy environment docker info | grep -i proxy # Test pull through proxy docker pull alpine Test Container Proxy 1 2 3 4 5 6 7 8 # Check proxy env inside a container docker run --rm alpine env | grep -i proxy # Verify external access through proxy docker run --rm alpine wget -O- https://httpbin.org/ip # If curl is your tool docker run --rm curlimages/curl -s https://httpbin.org/ip Troubleshooting Checklist Symptom Likely Cause Fix docker pull hangs Daemon missing proxy config Add systemd drop-in or daemon.json proxy Container can\u0026rsquo;t reach external API Missing container-level HTTP_PROXY Set env vars per service in Compose Inter-container connections fail Missing NO_PROXY for internal subnets Add RFC1918 ranges and .local to NO_PROXY Build fails downloading packages Build args not passed to Dockerfile Add ARG and \u0026ndash;build-arg to build command Proxy refuses connection Proxy container not running or wrong address Check docker ps and verify proxy URL Conclusion Docker proxy configuration is a three-layer problem: daemon for pulls, CLI for builds, and container env vars for runtime traffic. The most common mistake in homelab setups is skipping NO_PROXY, which silently breaks inter-container communication on bridge networks.\nStart with the systemd drop-in file for the daemon, add container-level proxy variables in your Compose files, and always include RFC1918 subnets plus .local in NO_PROXY. If you need a forward proxy, run Tinyproxy or Squid in a container and bind it to the same network stack as your VPN gateway.\nFor more homelab Docker patterns, check out the Docker Compose networking patterns guide and the Gluetun VPN gateway container deployment.\n","permalink":"https://blog.gntech.me/posts/2026-06-11-docker-proxy-configuration-homelab/","summary":"Configure HTTP/HTTPS proxy for Docker daemon, CLI, and individual containers in your homelab. Covers systemd drop-in overrides, daemon.json settings, Docker Compose proxy variables, and the critical NO_PROXY configuration to avoid breaking internal container communication.","title":"Docker Proxy Configuration Guide — HTTP/HTTPS Proxy for Containers"},{"content":"You write a shiny compose.yml, run docker compose up -d, and everything starts at once. Postgres is still initializing when your web app tries to connect. The migration container runs before the database accepts queries. Redis is listed as a dependency, but your cache-dependent service crashes three times before it finds the port open.\nThis is the startup order problem — and it is one of the most common pain points in homelab Docker deployments.\nDocker Compose gives you the tools to solve it, but the defaults (depends_on without conditions) only guarantee container start order, not service readiness. This guide covers every option from basic to advanced: depends_on conditions, healthcheck-based gating, init container patterns, and the classic wait-for-it script approach.\ndepends_on — What It Actually Does and Does Not Do The simplest form of depends_on tells Compose to start one container after another:\n1 2 3 4 5 6 7 services: app: image: my-app:latest depends_on: - db db: image: postgres:17 Without any condition, depends_on only guarantees that the db container is started before app. It does not wait for Postgres to be ready to accept connections. On a fast machine with local volumes, this usually works. On a homelab with spinning disks or during first boot when databases initialize, it fails reliably.\nSupported Compose plugins provide conditional depends_on; do not infer the plugin version from the Engine version or a file-format version. Validate with docker compose config:\n1 2 3 4 5 6 7 8 services: app: image: my-app:latest depends_on: db: condition: service_healthy redis: condition: service_started Three conditions are available:\nservice_started — the default; starts after the dependency container is running service_healthy — waits until the dependency passes its healthcheck service_completed_successfully — waits until the dependency runs and exits with code 0 (init container pattern) The service_healthy and service_completed_successfully conditions are what turn depends_on from a simple ordering hint into a real readiness gate.\nHealthcheck-Based Startup Ordering The most reliable pattern for production is pairing depends_on with healthcheck on the dependency service. Docker runs the healthcheck command periodically and marks the container as healthy or unhealthy. The dependent service waits until healthy before starting.\nHere is a complete example with Postgres, Redis, and a web application:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 services: postgres: image: postgres:17-alpine environment: POSTGRES_DB: myapp POSTGRES_PASSWORD: ${DB_PASSWORD:-changeme} volumes: - pgdata:/var/lib/postgresql/data healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;pg_isready -U myapp\u0026#34;] interval: 5s timeout: 5s retries: 5 start_period: 30s restart: unless-stopped redis: image: redis:7-alpine healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;redis-cli\u0026#34;, \u0026#34;ping\u0026#34;] interval: 5s timeout: 3s retries: 3 restart: unless-stopped app: build: . ports: - \u0026#34;8080:8080\u0026#34; depends_on: postgres: condition: service_healthy redis: condition: service_healthy environment: DATABASE_URL: postgres://myapp:${DB_PASSWORD:-changeme}@postgres:5432/myapp REDIS_URL: redis://redis:6379 restart: unless-stopped volumes: pgdata: Key decisions in this config:\nstart_period: 30s gives Postgres time to initialize on first boot before healthcheck failures count toward retries. Without this, a cold Postgres init could exhaust retries before the server is ready. interval: 5s balances quick detection with reasonable overhead — every 5 seconds is fine for homelab, even with many containers. restart: unless-stopped ensures the application retries if it fails while waiting for dependencies. Compose starts the app, the healthcheck is still pending, the app crashes, Docker restarts it, and eventually the healthcheck passes. Starting Additional Dependencies by Service Type Homelab stacks often need MariaDB or MySQL instead of Postgres:\n1 2 3 4 5 6 7 8 9 10 11 mariadb: image: mariadb:11-lts environment: MARIADB_ROOT_PASSWORD: ${DB_ROOT_PASSWORD} MARIADB_DATABASE: myapp healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;healthcheck.sh\u0026#34;, \u0026#34;--connect\u0026#34;, \u0026#34;--innodb_initialized\u0026#34;] interval: 5s timeout: 5s retries: 5 start_period: 60s MariaDB\u0026rsquo;s official image bundles healthcheck.sh with --innodb_initialized — safer than a raw MySQL ping because it also verifies InnoDB recovery completed.\nInit Container Pattern in Docker Compose Kubernetes has native init containers: they run to completion before the main pod starts. Docker Compose has no built-in init container type, but you can simulate the pattern cleanly with depends_on and condition: service_completed_successfully.\nUse this for:\nDatabase schema migrations (Alembic, Flyway, Prisma) Seed data loading Terraform / Pulumi provisioning File permission setup on shared volumes Here is a real example with Alembic migrations for a FastAPI app:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 services: postgres: image: postgres:17-alpine environment: POSTGRES_DB: myapp POSTGRES_PASSWORD: ${DB_PASSWORD:-changeme} healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;pg_isready -U myapp\u0026#34;] interval: 5s timeout: 5s retries: 5 start_period: 30s volumes: - pgdata:/var/lib/postgresql/data restart: unless-stopped migrations: build: context: . dockerfile: Dockerfile depends_on: postgres: condition: service_healthy environment: DATABASE_URL: postgres://myapp:${DB_PASSWORD:-changeme}@postgres:5432/myapp command: alembic upgrade head # This service runs once and exits — no restart needed app: build: context: . dockerfile: Dockerfile ports: - \u0026#34;8080:8080\u0026#34; depends_on: migrations: condition: service_completed_successfully environment: DATABASE_URL: postgres://myapp:${DB_PASSWORD:-changeme}@postgres:5432/myapp restart: unless-stopped volumes: pgdata: The chain is: postgres becomes healthy → migrations runs alembic upgrade head and exits 0 → app starts knowing the schema is current.\nFor multi-step init, chain multiple init services:\n1 2 3 4 5 6 7 8 9 10 11 12 services: db-seed: build: ./seed depends_on: migrations: condition: service_completed_successfully command: node /app/seed.js app: depends_on: db-seed: condition: service_completed_successfully The wait-for-it and wait-for Scripts Not every image ships with a healthcheck. For third-party images you cannot modify, the wait-for-it.sh script (or the simpler wait-for) is a practical fallback.\nwait-for-it Available at vishnubob/wait-for-it, this bash script blocks until a TCP port is open:\n1 2 3 4 5 6 7 8 services: app: image: my-app:latest depends_on: - db volumes: - ./wait-for-it.sh:/wait-for-it.sh command: [\u0026#34;./wait-for-it.sh\u0026#34;, \u0026#34;db:5432\u0026#34;, \u0026#34;--\u0026#34;, \u0026#34;node\u0026#34;, \u0026#34;app.js\u0026#34;] The script polls db:5432 until it accepts a TCP connection, then executes the command.\nwait-for (Efficient Python Alternative) The wait-for script is a shorter, POSIX-sh alternative that works on Alpine:\n1 2 3 4 5 6 7 8 9 app: image: alpine:3.21 depends_on: - db - redis volumes: - ./wait-for:/wait-for command: \u0026gt; sh -c \u0026#34;/wait-for db:5432 -- /wait-for redis:6379 -- node app.js\u0026#34; Important: Port-open checks have a race condition. The TCP port can be open before the service is ready (Postgres immediately opens the port, but may reject queries for several seconds during crash recovery). Healthchecks are strictly better. Use port-waiting only when you cannot add a healthcheck.\nAdvanced Patterns for Complex Stacks Chained Dependencies 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 services: db: image: postgres:17-alpine healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;pg_isready -U myapp\u0026#34;] restart: unless-stopped cache: image: redis:7-alpine depends_on: db: condition: service_healthy healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;redis-cli\u0026#34;, \u0026#34;ping\u0026#34;] restart: unless-stopped api: build: ./api depends_on: cache: condition: service_healthy restart: unless-stopped Conditional startup with profiles Profiles select optional services; they do not make an enabled application\u0026rsquo;s required dependency disappear. An always-enabled app depending on a migration service hidden behind an inactive profile can produce an invalid Compose model. Put a required migration in the normal dependency graph, or put the whole optional application group in the same profile. See Compose profiles.\nFor a manually managed migration, run and check the migration explicitly before starting the application; remove the automatic dependency only if the deployment procedure now enforces that ordering. A profile is not an error-handling or migration-rollback mechanism.\nDiagnose unexpected health state A health check reports the result of its configured probe. Inspect its command, output, and timing before recreating anything:\n1 2 3 docker compose ps docker inspect --format \u0026#39;{{json .State.Health}}\u0026#39; container_name docker compose logs --tail 100 Replace container_name with the inspected container. A probe can succeed while the application is unusable if it tests the wrong condition. Correct the probe and apply the configuration in a maintenance window with docker compose up -d. Do not delete volumes or restart the whole Docker daemon to clear a supposed Compose health cache.\nSummary and Best Practices Here is a decision tree for Docker Compose startup ordering in your homelab:\nSituation Pattern Condition Dependencies start fast (local Alpine images, no init) depends_on: (bare) service_started Database needs to accept queries healthcheck + depends_on condition service_healthy Migration must run before app starts Init container service_completed_successfully Third-party image, no healthcheck wait-for-it.sh Port check Complex multi-service dependency chain Chained healthchecks service_healthy each Start with healthchecks and depends_on condition: service_healthy. This covers 80% of homelab scenarios. Add the init container pattern for database migrations. Use wait-for-it only as a last resort for images you cannot modify.\nSet reasonable start_period values — 30-60 seconds for databases, 10 seconds for caches. Use restart: unless-stopped on all services that have dependencies so they retry automatically after a crash.\nYour Compose stack will boot reliably every time, whether it is the first deploy on bare metal or a restart after a power outage.\n","permalink":"https://blog.gntech.me/posts/2026-06-10-docker-compose-startup-dependencies-init-containers/","summary":"A practical guide to Docker Compose startup ordering — covering depends_on conditions, healthcheck-based readiness, init container patterns with wait-for-it, and real-world compose examples for multi-service homelab stacks.","title":"Docker Compose Startup Dependencies and Init Container Patterns"},{"content":"Why Stock Linux Network Settings Leave Performance on the Table Out of the box, Ubuntu Server and Debian configure network interfaces conservatively. Ring buffers default to 256 descriptors, IRQ handlers land on whatever CPU core boots first, and interrupt coalescing favors compatibility over throughput. On a Proxmox host running 20 VMs, a Docker host serving NFS shares, or a media server streaming 4K transcodes, these defaults become measurable bottlenecks.\nThe three most common symptoms of untuned networking:\nrx_missed_errors or rx_dropped climbing in ethtool -S counters while under load SoftIRQ pegging a single CPU core to 100% while other cores sit idle Inconsistent throughput during iperf3 tests or NFS transfers This guide walks through diagnosing each issue, applying the fix, and making changes persistent across reboots. Every command here works on Debian 12/13, Ubuntu 22.04/24.04, and Proxmox VE 8/9 hosts.\nDiagnosing Network Bottlenecks with ethtool Before tuning anything, capture a baseline. Start with driver-level counters — these tell you exactly what the NIC hardware sees.\n1 2 # Check for dropped or missed packets ethtool -S eth0 | grep -E \u0026#39;drop|miss|error|fail\u0026#39; Look for counters rising during load. rx_missed_errors indicates the hardware ring buffer overflowed — the card had packets to deliver but no room in the ring. rx_dropped means the kernel dropped packets after the ring accepted them, usually due to socket buffer pressure.\nInspect current ring buffer sizes:\n1 ethtool -g eth0 Output:\nRing parameters for eth0: Pre-set maximums: RX: 4096 RX Mini: 0 RX Jumbo: 0 TX: 4096 Current hardware settings: RX: 256 TX: 256 The NIC supports 4096, but the kernel set 256. Your first tuning opportunity.\nCheck available queue counts and current configuration:\n1 ethtool -l eth0 Output:\nChannel parameters for eth0: Pre-set maximums: RX: 0 TX: 0 Other: 1 Combined: 4 Current hardware settings: RX: 0 TX: 0 Other: 1 Combined: 1 The NIC supports 4 combined queues but only 1 is active. Each queue gets its own IRQ vector, so running 1 queue means one CPU core handles all RX and TX interrupts.\nCheck offload features:\n1 ethtool -k eth0 | grep -E \u0026#39;tcp|udp|generic|large|scatter|tx-checksum\u0026#39; And run a simple baseline benchmark:\n1 2 3 4 5 # Server side iperf3 -s # Client side iperf3 -c 10.0.20.30 -t 30 -P 4 Record the results. After tuning, you will run the same test and compare.\nRing Buffer Tuning for High Throughput Ring buffers sit between the NIC hardware and the kernel networking stack. When a packet arrives, the NIC DMA-writes it into the ring, and the kernel reads from the ring. If the kernel falls behind — say, because storage IO is busy or another CPU is handling interrupts — the ring fills up and packets get dropped.\nModern 1GbE and 10GbE NICs support rings of 4096 or more descriptors. Doubling the default 256 to 4096 gives the kernel more headroom during bursts.\n1 2 # Increase ring buffer sizes ethtool -G eth0 rx 4096 tx 4096 Verify the change:\n1 ethtool -g eth0 The tradeoff: larger rings improve throughput under load by reducing drops, but they increase per-packet latency because the kernel spends more time draining the ring. For homelab workloads (file serving, media streaming, backups), throughput matters more than microseconds of latency. For real-time applications like VoIP or gaming servers, keep rings at 1024 or lower.\nTo find your sweet spot, run iperf3 with the counter watch:\n1 2 # Watch drops while running a test watch -n 1 \u0026#39;ethtool -S eth0 | grep -E \u0026#34;miss|drop\u0026#34;\u0026#39; If drops appear at 256 but disappear at 4096, you found the optimal setting.\nMaking Ring Buffer Settings Persistent ethtool changes do not survive a reboot. The standard approach is a systemd oneshot service:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 cat \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; | sudo tee /etc/systemd/system/ethtool-tune.service [Unit] Description=Apply ethtool network tuning After=network.target Before=network-online.target Wants=network.target [Service] Type=oneshot ExecStart=/usr/sbin/ethtool -G eth0 rx 4096 tx 4096 ExecStart=/usr/sbin/ethtool -K eth0 gro on gso on tso on ExecStart=/usr/sbin/ethtool -C eth0 rx-usecs 4 tx-usecs 4 RemainAfterExit=true StandardOutput=journal [Install] WantedBy=multi-user.target EOF sudo systemctl daemon-reload sudo systemctl enable --now ethtool-tune.service For hosts with multiple interfaces, repeat the ExecStart lines for each, or use a script that iterates over ip link show | grep -v lo | awk -F: '/^[0-9]/{print $2}'.\nMultiqueue, RSS, RPS, and XPS Configuration Network queue configuration is the most impactful single tuning you can do. A single queue means a single IRQ, which means one CPU core. On modern multi-core hosts, spreading RX and TX work across all cores dramatically improves throughput and reduces tail latency.\nRSS — Receive Side Scaling RSS is hardware-level load balancing. The NIC distributes incoming packets across multiple RX queues using a hash of the IP header (or TCP 5-tuple). Each queue has its own IRQ vector.\nEnable all available queues:\n1 2 # Set combined queues to the maximum supported ethtool -L eth0 combined 4 Verify:\n1 ethtool -l eth0 Current hardware settings: RX: 0 TX: 0 Other: 1 Combined: 4 Check IRQ distribution:\n1 grep eth0 /proc/interrupts | sort -k1 -n You should see one interrupt line per queue, each landing on a different CPU core. If they all land on the same core, irqbalance reassigns them automatically after a few seconds, or you can set affinity manually.\nRPS — Receive Packet Steering (Software RSS) Not all NICs support multiple queues. VirtIO paravirtualized NICs in Proxmox VMs, older Realtek chips, and USB Ethernet adapters expose only one combined queue. RPS distributes packet processing across CPUs in software, after the single hardware queue delivers the packet.\nEnable RPS on each RX queue by writing a CPU bitmask to rps_cpus. On a 4-core system (cores 0-3), the bitmask for all cores is f:\n1 echo f | sudo tee /sys/class/net/eth0/queues/rx-0/rps_cpus For each additional RX queue (if available):\n1 2 3 for q in /sys/class/net/eth0/queues/rx-*; do echo f | sudo tee \u0026#34;$q\u0026#34;/rps_cpus done On a 8-core system with cores 2-7 isolated for networking, the bitmask is fc. Calculate bitmasks quickly:\n1 2 # Bitmask for CPU 2,3,4,5,6,7 python3 -c \u0026#34;print(hex(int(\u0026#39;11111100\u0026#39;, 2)))\u0026#34; # 0xfc XPS — Transmit Packet Steering XPS is the TX-side analog of RPS. It binds TX queues to CPU cores so that transmit completions and subsequent TX processing happen on the same core:\n1 2 3 for q in /sys/class/net/eth0/queues/tx-*; do echo f | sudo tee \u0026#34;$q\u0026#34;/xps_cpus done RFS — Receive Flow Steering RFS extends RPS by steering packets to the CPU where the application processing them is running. This improves cache locality. Enable it with:\n1 2 3 4 5 6 7 # Increase flow table size sudo sysctl -w net.core.rps_sock_flow_entries=32768 # Set flow count for each RX queue (1-2x rps_sock_flow_entries / queue count) for q in /sys/class/net/eth0/queues/rx-*; do echo 8192 | sudo tee \u0026#34;$q\u0026#34;/rps_flow_cnt done When Each Technique Applies Hardware Recommended Approach Intel i350/X710, Mellanox ConnectX, Broadcom NetXtreme RSS (hardware) — set combined queues to max, verify IRQ spread VirtIO (Proxmox VM), vmxnet3 (VMware) RSS + RPS — enable virtio multiqueue driver in VM config Realtek, USB Ethernet, single-queue NICs RPS + XPS + RFS — software distribution only For Proxmox VMs, enable virtio multiqueue in the VM configuration:\n1 2 # Per-vCPU queue pairs: set to match vCPU count qm set \u0026#34;\u0026lt;VMID\u0026gt;\u0026#34; --args \u0026#34;-device virtio-net-pci,netdev=net0,mq=on,vectors=8\u0026#34; Or for new VMs, include multifunction=on:\n1 qm set \u0026#34;\u0026lt;VMID\u0026gt;\u0026#34; --numa 0 --args \u0026#34;-device virtio-net-pci,netdev=hostnet0,mq=on,vectors=12\u0026#34; Now match the queue count inside the VM:\n1 ethtool -L ens18 combined \u0026#34;$(nproc)\u0026#34; IRQ Balancing and CPU Pinning irqbalance runs by default on Ubuntu Server and Debian and does a reasonable job of spreading IRQs across CPUs. But it rebalances periodically, which can cause jitter. For consistent performance, pin critical IRQs manually.\nFirst, identify the IRQ numbers for your NIC:\n1 grep eth0 /proc/interrupts Output:\n28: 123 IO-APIC 28-fasteoi eth0-0 29: 45 IO-APIC 29-fasteoi eth0-1 30: 67 IO-APIC 30-fasteoi eth0-2 31: 89 IO-APIC 31-fasteoi eth0-3 Pin each IRQ to a specific CPU core by writing the CPU bitmask:\n1 2 3 4 5 # Core 0 for eth0-0, Core 1 for eth0-1, etc. echo 1 \u0026gt; /proc/irq/28/smp_affinity echo 2 \u0026gt; /proc/irq/29/smp_affinity echo 4 \u0026gt; /proc/irq/30/smp_affinity echo 8 \u0026gt; /proc/irq/31/smp_affinity For dual-socket NUMA systems, pin to cores on the same NUMA node as the NIC\u0026rsquo;s PCI slot:\n1 2 3 4 5 # Find NUMA node of the NIC cat /sys/class/net/eth0/device/numa_node # Bitmask for all cores on node 0 (e.g., cores 0-7) echo ff | tee /proc/irq/28/smp_affinity If you keep irqbalance, ban critical IRQs from being rebalanced:\n1 2 3 # Ban IRQs 28-31 from irqbalance echo 28-31 \u0026gt; /proc/irq/28/irqbalance_ban # Repeat for each banned IRQ or use --banirq For more advanced setups, run irqbalance in oneshot mode — it configures affinity once on boot and exits:\n1 2 # In /etc/default/irqbalance IRQBALANCE_ONESHOT=yes Or disable irqbalance entirely and manage affinities via a systemd service:\n1 sudo systemctl disable --now irqbalance Then add echo commands for each IRQ to the same ethtool-tune.service from the ring buffer section.\nTraffic Control for Docker Host Bandwidth Management On a shared Docker host, one container running a backup or large download can saturate the uplink, starving other containers. Traffic control (tc) with HTB qdisc provides per-interface bandwidth limits.\nLimit outbound traffic on eth0 to 800 Mbps (80% of a 1 Gbps link, leaving overhead headroom):\n1 2 3 4 5 6 7 8 9 10 11 # Remove existing qdisc tc qdisc del dev eth0 root 2\u0026gt;/dev/null # Add HTB root with default class tc qdisc add dev eth0 root handle 1: htb default 10 # Root class — total bandwidth tc class add dev eth0 parent 1: classid 1:1 htb rate 1000mbit # Limit class — actual cap tc class add dev eth0 parent 1:1 classid 1:10 htb rate 800mbit ceil 800mbit Inbound traffic is harder to shape because you cannot control what the upstream sends. A simple approach uses the ifb (Intermediate Functional Block) pseudo-device:\n1 2 3 4 5 6 7 8 9 10 11 12 # Load ifb module modprobe ifb ip link set ifb0 up # Redirect ingress to ifb0 tc qdisc add dev eth0 ingress tc filter add dev eth0 parent ffff: protocol all u32 match u32 0 0 action mirred egress redirect dev ifb0 # Apply shaping on ifb0 tc qdisc add dev ifb0 root handle 1: htb default 10 tc class add dev ifb0 parent 1: classid 1:1 htb rate 1000mbit tc class add dev ifb0 parent 1:1 classid 1:10 htb rate 800mbit ceil 800mbit For Docker-specific bandwidth control, bind tc to the container\u0026rsquo;s veth interface instead of the host\u0026rsquo;s physical interface. Or use a cleaner approach with cgroup-based traffic classification:\n1 2 # Create cgroup filters for specific containers tc filter add dev eth0 parent 1: protocol all prio 1 cgroup Then assign the container pid to a cgroup with specific net_cls.classid, and the tc filter applies the corresponding rate.\nFor most homelabs, shaping the host\u0026rsquo;s physical interface and setting per-container --cpus and --memory limits is sufficient. Add tc to your ethtool systemd service for persistence:\n1 2 3 ExecStart=/sbin/tc qdisc add dev eth0 root handle 1: htb default 10 ExecStart=/sbin/tc class add dev eth0 parent 1: classid 1:1 htb rate 1000mbit ExecStart=/sbin/tc class add dev eth0 parent 1:1 classid 1:10 htb rate 800mbit ceil 800mbit Offload Feature Tuning ethtool offload flags control whether packet segmentation, checksumming, and coalescing happen in hardware or software. The defaults (all on) are usually correct for bare-metal servers, but virtualization and container networking create edge cases.\nChecksum Offload Some NICs generate incorrect checksums with certain drivers under load. The symptom: tcpdump shows good packets, but the application sees TCP checksum failures. Test by temporarily disabling:\n1 ethtool -K eth0 tx-checksum-ip-generic off If the problem persists, the NIC driver is fine. Re-enable:\n1 ethtool -K eth0 tx-checksum-ip-generic on TSO, GSO, GRO TSO (TCP Segmentation Offload) and GSO (Generic Segmentation Offload) let the NIC split large TCP segments into MTU-sized packets in hardware. GRO (Generic Receive Offload) merges incoming packets into larger chunks before the kernel processes them.\nKeep them on for throughput-sensitive workloads. Disable only if:\nThe NIC/driver has bugs with large offloads (check dmesg for errors) You are running packet inspection (Snort, Suricata) — they need unsegmented packets You need per-packet latency under 100µs 1 2 # Disable (most aggressive tuning for latency) ethtool -K eth0 gro off gso off tso off Vendor-Specific Tuning Intel and Broadcom NICs expose additional tuning through private flags:\n1 ethtool --show-priv-flags eth0 On Intel ixgbe/ice drivers, the adaptive-rx and adaptive-tx flags control dynamic interrupt coalescing. Disable for consistent latency, keep enabled for mixed workloads:\n1 ethtool --set-priv-flags eth0 adaptive-rx off adaptive-tx off Proxmox and Docker Host-Specific Tuning sysctl Network Stack Tuning These sysctl settings complement hardware-level tuning:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 cat \u0026lt;\u0026lt;EOF | sudo tee /etc/sysctl.d/99-netoptim.conf # Increase network buffer sizes net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 net.core.rmem_default = 262144 net.core.wmem_default = 262144 # TCP buffer auto-tuning (min, default, max) net.ipv4.tcp_rmem = 4096 131072 16777216 net.ipv4.tcp_wmem = 4096 65536 16777216 # TCP congestion control — BBR for throughput net.ipv4.tcp_congestion_control = bbr # Enable TCP window scaling net.ipv4.tcp_window_scaling = 1 # Increase the maximum backlog net.core.netdev_max_backlog = 5000 net.core.netdev_budget = 600 net.core.netdev_budget_usecs = 8000 # Increase TCP max syn backlog net.ipv4.tcp_max_syn_backlog = 8192 # Enable fast socket open (TFO) net.ipv4.tcp_fastopen = 3 # Increase epoll limits net.core.optmem_max = 65536 # RFS flow table net.core.rps_sock_flow_entries = 32768 EOF sudo sysctl -p /etc/sysctl.d/99-netoptim.conf Proxmox Bridge Tuning Proxmox uses Linux bridges for VM connectivity. Default bridge settings prioritize simplicity over throughput. Tune the bridge forwarding database:\n1 2 3 4 5 6 7 8 # Increase bridge aging time (default 300s) echo 600 \u0026gt; /sys/class/net/vmbr0/bridge/ageing_time # Disable bridge hash elasticity (reduce CPU overhead with many MACs) echo 0 \u0026gt; /sys/class/net/vmbr0/bridge/hash_elasticity # Increase hash max size echo 4096 \u0026gt; /sys/class/net/vmbr0/bridge/hash_max Docker Overlay2 and nftables Overhead Docker\u0026rsquo;s overlay2 storage driver and nftables forwarding rules add kernel overhead. The nftables flowtable accelerates forwarding by bypassing the kernel stack for established connections:\n1 2 3 nft add table inet homelab nft add flowtable inet homelab netfwd { hook ingress priority 0\\; devices = { eth0, docker0 } \\; } nft add rule inet homelab forward ip protocol tcp flow add @netfwd Benchmarking and Validation After applying all tuning, run the same iperf3 test from the baseline:\n1 2 # Client to server, 4 parallel streams, 30 seconds iperf3 -c 10.0.20.30 -t 30 -P 4 Compare against your earlier baseline. A properly tuned host should show 15-40% higher throughput and no dropped packets.\nCheck IRQ distribution during the test:\n1 watch -n 2 \u0026#39;grep eth0 /proc/interrupts\u0026#39; Each queue IRQ should increment roughly evenly across the cores.\nCheck driver counters under load:\n1 watch -n 1 \u0026#39;ethtool -S eth0 | grep -E \u0026#34;miss|drop\u0026#34;\u0026#39; Both counters should remain at zero or near-zero during sustained throughput.\nMonitor CPU utilization during the test:\n1 mpstat -P ALL 2 SoftIRQ (%soft) should spread across multiple cores, not peg a single one.\nQuick Checklist — 5 Things to Check on Any Homelab Linux Host If you only have five minutes, run through these in order:\nethtool -g eth0 — are ring buffers at maximum? ethtool -l eth0 — are all combined queues active? grep eth0 /proc/interrupts — are IRQs spread across CPU cores? sysctl net.ipv4.tcp_congestion_control — is BBR enabled? ethtool -S eth0 | grep -E 'miss|drop' — are there any packet drops? Each item addresses one of the three bottlenecks from the introduction: ring overflows, single-CPU IRQ saturation, and buffer pressure. Fixing these five things covers 90% of network performance issues in a homelab environment.\nSummary Linux network performance tuning for homelab servers breaks down into four layers: hardware ring buffers, receive/transmit queue distribution via RSS/RPS/XPS, IRQ affinity with irqbalance, and traffic shaping with tc. Each layer has a straightforward diagnostic tool (ethtool) and a permanent persistence method (systemd).\nStart with ethtool -S to identify drops, then work through the checklist. On a Proxmox host serving 10-30 containers, ring buffer tuning alone can eliminate dropped packets under burst loads. Adding RSS or RPS distribution spreads the interrupt load across all CPU cores, which directly translates to higher consistent throughput.\nThe configurations in this guide are safe to apply to any Debian-based homelab host. Test each change with iperf3 before and after, make them persistent with the included systemd service and sysctl drop-in, and monitor /proc/interrupts to verify IRQ distribution.\n","permalink":"https://blog.gntech.me/posts/2026-06-09-linux-network-performance-tuning-homelab/","summary":"Optimize Linux network performance on Proxmox and Docker hosts with ethtool ring buffer tuning, irqbalance configuration, multiqueue RSS/RPS, and tc traffic shaping — with real benchmarks and systemd persistence.","title":"Linux Network Performance Tuning — ethtool, irqbalance, and tc for Homelab"},{"content":"If you run Docker in your homelab, your containers probably sit behind the default bridge network, hidden behind a NAT on 172.17.0.0/16. That works fine for casual setups, but the moment you need VLAN segmentation — keeping your IoT containers on VLAN 30, your reverse proxy on VLAN 10, and your databases isolated on VLAN 40 — the bridge driver becomes a bottleneck.\nYou end up stacking port mappings, guessing which containers talk to which, and losing the IP-level visibility that makes VLANs useful in the first place.\nDocker MACVLAN and IPVLAN drivers solve this. They let you attach containers directly to your physical network with their own IP addresses on whichever VLAN you choose. No port forwarding. No NAT. Just containers behaving like full-fledged network members on the right segment.\nMACVLAN vs IPVLAN — Choosing the Right Driver Before reaching for docker network create, understand which driver fits your environment.\nMACVLAN The MACVLAN driver assigns a unique MAC address to each container on the same parent interface. From the network\u0026rsquo;s perspective, each container looks like a separate physical device.\nContainer A (00:0c:29:aa:01) ─┐ Container B (00:0c:29:bb:02) ─┤── eth0.20 (host) ── VLAN 20 Container C (00:0c:29:cc:03) ─┘ Pros:\nWorks with most switches and routers out of the box Containers appear as independent hosts (DHCP, ARP, mDNS all work) Near line-rate performance — no extra kernel routing Cons:\nSome managed switches restrict the number of MAC addresses per switchport Requires the parent interface to be in promiscuous mode The host cannot reach containers on a MACVLAN network without additional setup Does not work over wireless interfaces IPVLAN The IPVLAN driver assigns one shared MAC to all containers on the parent interface, differentiated only by IP address.\nIPVLAN L2 mode — containers live on the same L2 broadcast domain as the parent:\nSwitch-friendly: only one MAC per port regardless of container count Same performance profile as MACVLAN Host can reach containers (major advantage) IPVLAN L3 mode — containers communicate via L3 routing only:\nNo ARP, no MAC learning Routes must be configured on upstream routers Higher CPU overhead for software routing Decision Guide Need Driver Simple direct IPs, no switch limits MACVLAN Many containers, switch port MAC limits IPVLAN L2 Host needs to reach containers directly IPVLAN L2 Strict L3 isolation from broadcast domain IPVLAN L3 WiFi parent interface Neither (use bridge + port mapping) For most homelabs, MACVLAN is the right starting point. Switch over to IPVLAN only if you hit MAC limits or need host-to-container connectivity without extra bridge work.\nSetting Up Docker MACVLAN Prerequisites Docker Engine 24+ (20.10 works too, but 24+ has better networking) A wired Ethernet interface (MACVLAN/IPVLAN don\u0026rsquo;t work on WiFi) VLAN-aware parent interface (a physical NIC or a VLAN sub-interface) Creating the MACVLAN Network Assume your host connects via eth0, and you want containers on VLAN 20 (10.0.20.0/24 with gateway 10.0.20.1):\n1 2 3 4 5 6 docker network create \\ --driver macvlan \\ --subnet=10.0.20.0/24 \\ --gateway=10.0.20.1 \\ --opt parent=eth0 \\ vlan20 If your VLAN is tagged on the switch but the host interface is untagged, create a Linux VLAN sub-interface first:\n1 2 ip link add link eth0 name eth0.20 type vlan id 20 ip link set eth0.20 up Then create the MACVLAN network using eth0.20 as the parent:\n1 2 3 4 5 6 docker network create \\ --driver macvlan \\ --subnet=10.0.20.0/24 \\ --gateway=10.0.20.1 \\ --opt parent=eth0.20 \\ vlan20 Important: Make sure the host itself has an IP on VLAN 20 through some other path (a bridge or a separate interface) if you need the host to reach containers on this network. The MACVLAN driver isolates the parent interface from container IPs.\nDocker Compose Configuration Here is how you wire a container to a MACVLAN network using Docker Compose. This example puts Nginx on VLAN 20 with a static IP:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 networks: vlan20: external: true name: vlan20 services: nginx: image: nginx:1.27-alpine container_name: web-vlan20 hostname: web-vlan20 networks: vlan20: ipv4_address: 10.0.20.100 restart: unless-stopped Start it:\n1 docker compose up -d Verify the container picked up the right IP:\n1 2 docker inspect web-vlan20 --format \u0026#39;{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}\u0026#39; # Output: 10.0.20.100 Now ping 10.0.20.100 from any device on VLAN 20. The container responds directly — no port mapping needed.\nMultiple Containers on Different VLANs Run services across multiple VLANs in one Compose file:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 networks: vlan10: external: true name: vlan10 vlan30: external: true name: vlan30 services: traefik: image: traefik:v3.2 container_name: traefik networks: vlan10: ipv4_address: 10.0.10.50 ports: [] mosquitto: image: eclipse-mosquitto:2 container_name: mqtt networks: vlan30: ipv4_address: 10.0.30.50 volumes: - ./mosquitto/config:/mosquitto/config Create the two networks first:\n1 2 docker network create --driver macvlan --subnet=10.0.10.0/24 --gateway=10.0.10.1 --opt parent=eth0.10 vlan10 docker network create --driver macvlan --subnet=10.0.30.0/24 --gateway=10.0.30.1 --opt parent=eth0.30 vlan30 Traefik now lives on your DMZ VLAN with a public IP (assuming VLAN 10 routes to the internet), and MQTT lives on your IoT VLAN 30 — and they can still reach each other if the upstream router routes between those VLANs.\nVLAN-Aware Routing and Connectivity The MACVLAN Host-Connectivity Problem The biggest gotcha with MACVLAN: the host cannot reach containers at the L3 level without workarounds. If you SSH into your Proxmox host and try to ping 10.0.20.100, it fails — because the MACVLAN driver creates a dedicated bridge that is not connected to the host\u0026rsquo;s own networking stack.\nSolution — MACVLAN Bridge (Host Mode) Create the MACVLAN network in bridge mode:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # Create a shared macvlan bridge docker network create \\ --driver macvlan \\ --subnet=10.0.20.0/24 \\ --gateway=10.0.20.1 \\ --opt parent=eth0 \\ --ipam-driver default \\ --aux-address=\u0026#34;host=10.0.20.250\u0026#34; \\ vlan20 # Create a macvlan interface on the host in bridge mode ip link add macvlan20 link eth0 type macvlan mode bridge ip addr add 10.0.20.250/24 dev macvlan20 ip link set macvlan20 up # Add a route so the host uses the macvlan interface for container traffic ip route add 10.0.20.0/24 dev macvlan20 metric 1000 Now the host can ping 10.0.20.100 (your nginx container) and services running there.\nTo make this persistent on Debian/Ubuntu with systemd-networkd, create /etc/systemd/network/10-macvlan20.netdev:\n1 2 3 4 5 6 [NetDev] Name=macvlan20 Kind=macvlan [MACVLAN] Mode=bridge And /etc/systemd/network/10-macvlan20.network:\n1 2 3 4 5 6 7 8 9 [Match] Name=macvlan20 [Network] Address=10.0.20.250/24 [Route] Destination=10.0.20.0/24 Metric=1000 Then reload:\n1 systemctl restart systemd-networkd When to Prefer IPVLAN Instead If the MACVLAN bridge dance feels too hacky, switch to IPVLAN L2. The host can reach containers natively.\n1 2 3 4 5 6 7 docker network create \\ --driver ipvlan \\ --subnet=10.0.20.0/24 \\ --gateway=10.0.20.1 \\ --opt parent=eth0.20 \\ --ipvlan l2 \\ iot_net 1 2 3 4 5 6 services: example: image: nginx:alpine networks: iot_net: ipv4_address: 10.0.20.200 No extra routing. The host sees the container IP on the same subnet and can communicate directly.\nReal-World Homelab Use Cases DMZ Reverse Proxy (VLAN 10) Your Traefik or Nginx Proxy Manager needs ports 80/443 exposed to the internet. Put it on a dedicated DMZ VLAN with a static IP and firewall rules limiting inbound to ports 80/443 only.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 services: traefik: image: traefik:v3.2 container_name: traefik networks: dmz: ipv4_address: 10.0.10.2 command: - \u0026#34;--providers.docker.endpoint=unix:///var/run/docker.sock\u0026#34; - \u0026#34;--entrypoints.websecure.address=:443\u0026#34; - \u0026#34;--entrypoints.web.address=:80\u0026#34; volumes: - /var/run/docker.sock:/var/run/docker.sock:ro - ./letsencrypt:/letsencrypt Isolated IoT Network (VLAN 30) MQTT, Zigbee2MQTT, ESPHome containers should not have direct internet access. Put them on VLAN 30 with an upstream ACL blocking outbound WAN:\n1 2 3 4 5 6 docker network create \\ --driver macvlan \\ --subnet=10.0.30.0/24 \\ --gateway=10.0.30.1 \\ --opt parent=eth0.30 \\ iot_net Media Servers on Guest/Smart TV VLAN (VLAN 40) Jellyfin or Plex containers streaming to smart TVs should live on the same VLAN as your TVs. Direct IP access means DLNA, Chromecast, and mDNS discovery work natively:\n1 2 3 4 5 6 7 8 9 10 11 12 services: jellyfin: image: jellyfin/jellyfin:latest container_name: jellyfin networks: media_vlan: ipv4_address: 10.0.40.50 volumes: - /opt/media:/media - ./jellyfin/config:/config devices: - /dev/dri:/dev/dri Troubleshooting Common Issues Container Can\u0026rsquo;t Reach Internet Check the gateway on the VLAN:\n1 docker exec container-name ip route show default The gateway must be reachable from the VLAN\u0026rsquo;s broadcast domain. If your upstream router uses a different IP as gateway, set --gateway to that IP when creating the network.\nHost Can\u0026rsquo;t Reach Container (MACVLAN only) This is expected behavior. Use one of:\nIPVLAN L2 instead The MACVLAN bridge host interface described above Create a second network (bridge) on the host just for management traffic Ping Works, DNS Doesn\u0026rsquo;t Docker does not automatically inject DNS settings for MACVLAN/IPVLAN networks. The container inherits the host\u0026rsquo;s /etc/resolv.conf, which may not be routed on the VLAN. Set explicit DNS:\n1 2 3 4 5 6 7 docker network create \\ --driver macvlan \\ --subnet=10.0.20.0/24 \\ --gateway=10.0.20.1 \\ --ip-range=10.0.20.128/25 \\ --opt parent=eth0.20 \\ vlan20 Or set DNS explicitly in the Compose service:\n1 2 3 4 5 6 7 8 9 services: web-vlan20: image: nginx:1.27-alpine networks: vlan20: ipv4_address: 10.0.20.100 dns: - 10.0.0.1 - 1.1.1.1 Switch Drops Traffic (Too Many MACs) Some managed switches limit per-port MAC addresses (especially on edge ports). If you run 10+ containers on one parent interface, switch to IPVLAN L2 — it uses a single MAC for all containers.\nParent Interface Goes Down MACVLAN networks die if the parent interface goes down. There is no built-in failover or redundancy. If uptime matters, consider placing the parent interface on a LAG or bonding interface first.\nPerformance Considerations MACVLAN performs at near line rate because the kernel handles packet switching in hardware through the NIC\u0026rsquo;s macvlan offload. In benchmarks, MACVLAN adds roughly 1-3% overhead compared to bare-metal — far better than bridge NAT, which can add 15-25% overhead for high-throughput workloads.\nIPVLAN L2 is essentially identical to MACVLAN in performance. IPVLAN L3 adds more kernel routing cycles, making it 5-10% slower for bulk transfers.\nDriver CPU Overhead Throughput Use When Default bridge (NAT) Moderate-High (15-25%) Good Simple single-host stacks MACVLAN Low (1-3%) Near line-rate VLAN separation, direct IP access IPVLAN L2 Low (1-3%) Near line-rate Many containers, host reachability IPVLAN L3 Moderate (5-10%) Good L3 isolated networks For a media server streaming 4K at 100+ Mbps, the difference between MACVLAN and bridge NAT is negligible. For a file-transfer or backup container saturating 10 GbE, MACVLAN matters.\nConclusion Docker MACVLAN and IPVLAN drivers turn containers into first-class citizens on your network. Instead of wrangling port mappings and bridge NAT, you assign each container a real IP on the right VLAN and let your existing firewall rules and routing handle the rest.\nHere is the cheat sheet:\nMACVLAN for most cases — simple, fast, one container = one MAC IPVLAN L2 when switch MAC limits bite you or the host needs to talk to containers IPVLAN L3 when you need true L3 isolation Start with one service on a dedicated VLAN and expand from there. Your network will thank you — and so will your future self when the \u0026ldquo;what IP does this container have?\u0026rdquo; question has a straightforward answer.\nRelated: Linux Bridge Networking Debugging for Docker and Proxmox, systemd-networkd for Homelab Networking\n","permalink":"https://blog.gntech.me/posts/2026-06-09-docker-macvlan-homelab-networking/","summary":"Give your containers their own IPs on any VLAN with Docker MACVLAN and IPVLAN. A practical guide with docker-compose configs, routing tweaks, and real-world homelab use cases.","title":"Docker MACVLAN Homelab Networking — Assign Direct VLAN IPs"},{"content":"If you run more than one Proxmox VE cluster, you know the pain: bouncing between browser tabs, checking dashboards separately, logging into different nodes to verify Ceph health or manage subscriptions. For years, this was the gap in the Proxmox ecosystem — a missing centralized pane of glass for multi-cluster operations.\nProxmox Datacenter Manager (PDM) exists to solve exactly that. Think of it as \u0026ldquo;vCenter for Proxmox\u0026rdquo; — an official, first-party management platform that unifies your entire Proxmox estate. Version 1.1, released on May 28, 2026, is the most significant update yet, adding automated answer-file installations, centralized subscription management, and unified Ceph monitoring across all connected remotes.\nHere\u0026rsquo;s a deep dive into what PDM 1.1 brings to your homelab and how to get it running.\nWhat Is Proxmox Datacenter Manager? PDM is a standalone management appliance that provides a single web UI for monitoring and operating multiple Proxmox VE clusters and standalone nodes. It sits alongside your existing infrastructure — remotes continue running independently even if PDM goes down, so it is not a single point of failure.\nPDM 1.1 Software Stack Base OS: Debian 13.5 \u0026ldquo;Trixie\u0026rdquo; Kernel: Linux 7.0 (stable default) Filesystem: ZFS 2.4.2 Web UI: HTTPS on port 8006 (same familiar Proxmox interface style) System Requirements Use Case CPU RAM Storage Evaluation 1 core 1 GiB 10 GB Production 2+ cores 4 GiB 40 GB+ For a homelab with 2–5 nodes, a VM with 2 vCPUs and 4 GB RAM is plenty. You can run it as a VM on one of your existing Proxmox hosts.\nAutomated Installations — The Killer Feature The headline feature of PDM 1.1 is the automated installation integration. If you have ever deployed Proxmox VE across multiple nodes manually — answering the same prompts over and over — you will appreciate this immediately.\nPDM now lets you create answer files directly from its web interface using a dedicated wizard. These answer files are served to target systems via HTTPS during installation, so new nodes boot, fetch their configuration, and deploy without hands-on interaction.\nCreating an Answer File In the PDM web UI, navigate to Automated Installations → Prepared Answers and click Create. The wizard walks you through:\nNetwork settings — interface names, IP addresses (CIDR), gateway, DNS servers Disk layout — target disk, partition scheme, filesystem type (ZFS, ext4, BTRFS) Locale and keyboard — timezone, language, keymap Administrator credentials — root password, SSH keys, email Subscription key — optional, auto-registers the node MiniJinja Templating for Unique Configs One of the cleverest details is MiniJinja templating (Jinja2-inspired). You can template fields like hostname, IP address, and gateway to auto-generate unique configurations for each installation:\n{{ product.product }}{{ installation_nr }}.lab.example.com The installation_nr counter auto-increments each time an answer file is served. Combined with the system information data that the installer sends when it fetches the answer, you get fully dynamic provisioning per target.\nTarget Filters Answer files can be scoped with target filters — key-value pairs that match the identifying information the installer sends during boot:\n/product/product=pve — only Proxmox VE installations /network/interfaces/0/mac=00:1a:2b:* — match specific MAC ranges Default fallback — an answer marked as default is used when no specific filter matches Security Tokens Each automated installation requires an authentication token tied to a specific prepared answer. This prevents unauthorized systems from pulling configuration data. The token is embedded into the ISO when you prepare it with proxmox-auto-install-assistant:\n1 2 3 proxmox-auto-install-assistant prepare-iso \\ --fetch-answer https://pdm.lab.example.com:8006/api2/json/auto-install/answer \\ /path/to/proxmox-ve-9.2.iso Add the --answer-token flag with the token generated in PDM under Automated Installations → Authentication Tokens:\n1 2 3 4 5 proxmox-auto-install-assistant prepare-iso \\ --fetch-answer https://pdm.lab.example.com:8006/api2/json/auto-install/answer \\ --answer-token pdt-abc123def456 \\ -o /output/proxmox-ve-9.2-auto.iso \\ /path/to/proxmox-ve-9.2.iso Boot the prepared ISO on any target hardware, and it will reach out to PDM, match an answer file, and install fully unattended. Installation progress is visible from the PDM web interface.\nCentralized Subscription Management If you manage multiple Proxmox nodes — even with the free community repository — keeping track of which nodes have active subscriptions is a chore. PDM 1.1 fixes this with a centralized subscription key pool.\nFrom the PDM UI, you can:\nUpload subscription keys View which remotes have assigned keys Auto-assign keys to remotes without one Bring new nodes under subscription automatically (via the answer file subscription key field) This is especially useful for homelabbers who run a small cluster with a paid subscription on the hypervisor node and want to keep everything consistent without logging into each host.\nUnified Ceph Monitoring Ceph is a popular choice for hyperconverged storage in Proxmox homelabs, but monitoring Ceph across multiple clusters usually means switching between cluster dashboards. PDM 1.1 aggregates all Ceph health and performance metrics into a single view.\nWhat You See in the PDM Ceph Dashboard Cluster health — HEALTH_OK, HEALTH_WARN, HEALTH_ERR status per cluster Capacity — total and used storage per pool, fill percentages Performance — OPS, throughput, and latency trends Monitors — quorum status, mapping Managers — active/standby state OSDs — up/down status, weight, utilization, per-OSD IOPS Pools — PG count, placement groups status, space usage Instead of clicking through each Proxmox cluster to check \u0026ldquo;is Ceph healthy?\u0026rdquo;, you glance at a single dashboard and see the status of every Ceph deployment across your entire infrastructure.\nEnhanced Guest and Snapshot Management PDM 1.1 expands its guest management capabilities. The unified guest list shows VMs and containers from all connected remotes in one table, with filtering and sorting across clusters. You can:\nCreate, restart, and stop VMs from the central view Create and delete snapshots Monitor migration status Check basic guest configuration without navigating to the remote\u0026rsquo;s interface Future releases (per the PDM roadmap) will add bulk actions on virtual guests across the entire selection, richer migration dialogs with bandwidth limits, and a backup-job overview with last-execution status.\nInstalling PDM 1.1 in Your Homelab Download the ISO Grab the Proxmox Datacenter Manager ISO from proxmox.com/downloads.\nPrepare Installation Media 1 2 3 4 5 6 7 # Identify your USB device lsblk # Write the ISO to USB (replace /dev/sdX with your device) dd bs=1M conv=fdatasync \\ if=./proxmox-datacenter-manager_1.1.iso \\ of=/dev/sdX Deploy as a VM on Your Existing Proxmox Host Upload the ISO to your Proxmox VE storage Create a new VM with: 2 vCPUs (host-type recommended) 4 GB RAM 40 GB disk (on your fastest storage) VirtIO SCSI controller Network bridge on your management VLAN Boot from the ISO and follow the interactive installer After installation, access the web UI at https://\u0026lt;pdm-ip\u0026gt;:8006/ Adding Your First Remote Log into PDM web interface Navigate to Datacenter → Remotes → Add Enter your Proxmox VE node IP or hostname Provide root credentials or an API token PDM discovers the remote\u0026rsquo;s capabilities and adds it to your central dashboard You can add standalone nodes or entire clusters — PDM handles both. Once connected, you see VMs, storage, Ceph health, and subscription status from one screen.\nThe Road Ahead The PDM roadmap reveals an ambitious future:\nResource organization — folders as hierarchical entities, independent tag labeling, finer-grained ACL schema SDN integration — stretch EVPN VNets across clusters, multi-VRF support, automated route-target import/export Integrated firewall management Proxmox Backup Server deeper integration — job status, datastore browsing, prune retention Proxmox Mail Gateway as a remote type Notifications — PDM as a notification target for remotes Active-standby high availability for PDM itself The foundation being laid in 1.1 — automated installs, unified monitoring, centralized subscriptions — sets the stage for PDM to become the indispensable management layer for Proxmox at any scale.\nConclusion Proxmox Datacenter Manager 1.1 is a significant leap forward. The automated installation system alone makes it worth deploying in any homelab with more than one node — no more manual clicking through installation prompts, no more configuration drift across hosts. Add unified Ceph monitoring and centralized subscriptions, and you have a management tool that closes the gap between Proxmox and proprietary hypervisor platforms.\nIf you run multiple Proxmox nodes, spin up a PDM VM today. The vCenter-for-Proxmox dream is real, and it keeps getting better.\n","permalink":"https://blog.gntech.me/posts/2026-06-09-proxmox-datacenter-manager-1-1-automated-install-ceph-monitoring/","summary":"Proxmox Datacenter Manager 1.1 is out — with automated answer-file installations, unified Ceph monitoring across remotes, and centralized subscription management. Here\u0026rsquo;s what\u0026rsquo;s new and how to deploy it.","title":"Proxmox Datacenter Manager 1.1 — Automated Installs \u0026 Ceph Monitoring"},{"content":"If you started your homelab on a single Proxmox host, you already know the pain: planned maintenance means shutting everything down, a hardware failure takes out every VM and container, and there is no live migration without a second node.\nAdding a second Proxmox host is the natural next step, but a two-node cluster comes with its own challenges — especially around quorum and split-brain prevention. This guide walks through building a Proxmox VE cluster from scratch with high availability, using a QDevice as the tiebreaker vote to solve the two-node quorum problem, all while keeping the setup homelab-practical.\nWhy Cluster Proxmox — Live Migration and HA Failover Clustering Proxmox nodes unlocks three major capabilities you cannot get on standalone hosts:\nLive migration — move running VMs between nodes with zero downtime Unified management — single web UI across all nodes with cluster-wide resource views High availability — automatic VM failover when a node goes offline Proxmox clustering uses corosync for membership and quorum communication, and pmxcfs (the Proxmox Cluster File System) to synchronise configuration across all nodes in real time.\nPrerequisites and Planning Before creating the cluster, make sure you have:\nTwo or more Proxmox VE 9.x hosts — all running the same major version Shared storage — NFS, iSCSI, Ceph, or ZFS over SSH for live migration Dedicated cluster network — a separate VLAN or physical link for corosync traffic (recommended) A QDevice host — a lightweight Debian machine (RPi, LXC container, or tiny VM) that will act as the quorum tiebreaker Root SSH access to all nodes and the QDevice host For the rest of this guide, I will use these hostnames and IPs:\nHost Role IP proxmox01 Cluster node 1 10.0.20.30 proxmox02 Cluster node 2 10.0.20.31 pve-qdevice QDevice (LXC) 10.0.20.10 Creating the Proxmox Cluster On the first node, initialise the cluster:\n1 pvecm create homelab-cluster This creates a corosync configuration, generates the necessary certificates, and starts the cluster services. Verify it worked:\n1 pvecm status On the second node, join the cluster using the IP of the first node and the cluster secret:\n1 pvecm add 10.0.20.30 The --force flag bypasses the hostname conflict check. If you set unique hostnames during installation (which you should), you can omit it.\nAfter both nodes are joined, confirm the cluster status:\n1 2 3 4 5 6 pvecm status # Cluster name: homelab-cluster # Node(s): # Node name: proxmox01 (votes: 1) # Node name: proxmox02 (votes: 1) # Quorum: 2/2 Cluster Networking Best Practices Corosync traffic should be isolated from regular VM and storage traffic. If your hosts have multiple NICs, configure a dedicated cluster network:\n1 2 # On both nodes, add your cluster-only bridge (e.g. vmbr1) # Then configure corosync to bind to that link. Edit /etc/pve/corosync.conf to set the bindnetaddr:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 totem { version: 2 cluster_name: homelab-cluster transport: knet crypto_cipher: aes-256-gcm crypto_hash: sha512 } nodelist { node { ring0_addr: 10.0.20.30 name: proxmox01 nodeid: 1 } node { ring0_addr: 10.0.20.31 name: proxmox02 nodeid: 2 } } quorum { provider: corosync_votequorum } Verify corosync link health:\n1 2 3 4 5 6 corosync-cfgtool -s # Printing ring status. # Local node ID 1 # RING ID 0 # id\t= 10.0.20.30 # status\t= ring 0 active with no faults If you see \u0026ldquo;no faults\u0026rdquo;, the cluster link is healthy. Any faults here indicate network issues that will cause HA failures later.\nSetting Up Shared Storage VMs cannot live-migrate without shared storage. The simplest approach for a homelab is an NFS export from a NAS or a dedicated VM.\nAdd the NFS storage via the web UI (Datacenter → Storage → Add → NFS) or /etc/pve/storage.cfg:\n1 2 3 4 5 6 7 nfs: nfs-storage path /mnt/pve/nfs-storage server 10.0.20.5 export /srv/nfs/proxmox content images,rootdir,vztmpl,backup options vers=4.2,hard,timeo=600 maxfiles 0 For homelabs without dedicated NAS hardware, ZFS over SSH replication is a solid alternative — configure a ZFS dataset on each node and use Proxmox\u0026rsquo;s built-in replication scheduler. The trade-off is that replication is not real-time; typical intervals are 5–15 minutes.\nSolving the Two-Node Quorum Problem with QDevice A two-node cluster has a fundamental problem: if one node fails, the surviving node holds only 1 out of 2 votes — not enough to reach quorum. This triggers corosync to fence (reboot) the surviving node, taking down every VM.\nThe fix is an external QDevice — a lightweight third vote that does not run any VMs. It lives on a separate host and votes alongside the cluster nodes.\nStep 1 — Prepare the QDevice Host I run my QDevice as a Debian LXC container on a third Proxmox host (or the same hardware, on a different physical link). Install the required package:\n1 apt update \u0026amp;\u0026amp; apt install -y corosync-qdevice The QDevice host needs SSH root access from the cluster nodes. Generate a key on the first cluster node and copy it:\n1 2 ssh-keygen -t ed25519 -N \u0026#34;\u0026#34; -f /root/.ssh/id_qdevice ssh-copy-id -i /root/.ssh/id_qdevice root@10.0.20.10 Step 2 — Add the QDevice to the Cluster From any cluster node, run:\n1 pvecm qdevice setup 10.0.20.10 --ssh-identity /root/.ssh/id_qdevice This pushes the cluster certificate to the QDevice, configures corosync-qdevice on both sides, and starts the service.\nStep 3 — Verify Quorum 1 pvecm status You should now see three votes available:\nQuorum information: Votes: 2 Expected votes: 3 Quorum: 2/3 The QDevice does not appear in the Proxmox web UI, but pvecm status and corosync-quorumtool -s confirm it is active. The cluster now tolerates a single node going offline — 2 out of 3 votes still form quorum.\nEnabling High Availability With quorum solved, enable the Proxmox HA stack:\n1 systemctl enable --now pve-ha-lrm pve-ha-manager Run this on every node in the cluster.\nFencing Fencing (STONITH) ensures that a non-responsive node is forcefully rebooted before the cluster tries to recover its VMs. Proxmox uses watchdog timers for self-fencing.\nOn each node, check the watchdog:\n1 2 dmesg | grep watchdog # [ 0.456789] softdog: initialized. soft_noboot=0 soft_margin=60 sec If no hardware watchdog is available, the softdog kernel module provides a software watchdog:\n1 2 3 echo softdog \u0026gt; /etc/modules-load.d/watchdog.conf modprobe softdog systemctl restart corosync With fencing in place, a failed node will be fenced within 60 seconds, and the surviving node will restart the impacted VMs.\nCreating HA Groups and Resources HA groups control the order and preferred nodes for VM placement.\nCreate an HA Group Via the CLI:\n1 2 3 4 ha-manager add_group critical \\ --nodes \u0026#34;proxmox01,proxmox02\u0026#34; \\ --type ordered \\ --nofailback 0 --type ordered — prefer the first-listed node --nofailback 0 — allow failback when the preferred node returns Add VMs to HA 1 2 # Add VM 100 to the critical group with max 2 restart attempts ha-manager add vm:100 --group critical --max_restart 2 For containers, use ct: instead of vm::\n1 ha-manager add ct:200 --group critical Monitor HA Status 1 2 3 4 5 6 7 ha-manager status # quorum OK # Group(s): # critical: # nodes proxmox01,proxmox02 # VM 100 proxmox01 started # CT 200 proxmox01 started From the web UI: navigate to Datacenter → HA → Groups to manage groups visually, or Datacenter → HA → Resources to add and watch managed VMs.\nTesting HA Failover Do not skip this step. A cluster is only as trustworthy as the testing you put it through.\nGraceful Failover Test Stop corosync on the active node:\n1 2 # On proxmox01 systemctl stop corosync On proxmox02, the HA manager will detect the loss of quorum for proxmox01, wait for the watchdog to fence it, then start VMs:\n1 watch ha-manager status Within 60–120 seconds, you should see:\nVM 100 proxmox02 started CT 200 proxmox02 started To bring the node back, start corosync:\n1 systemctl start corosync The node rejoins automatically. If nofailback is enabled, VMs stay on proxmox02 until you manually migrate them back.\nHard Fencing Test For a more realistic test, pull the power (or unplug the network) on one node. The other node should fence and recover VMs within the watchdog timeout window.\nProduction Considerations for Homelab Clusters A few things to keep in mind as you move beyond the basic setup:\nThree nodes eliminate the QDevice need entirely. With three Proxmox hosts, you can run Ceph as native shared storage and have natural quorum (2/3 votes). This is the ideal homelab topology if you have the hardware. Run the QDevice as an LXC on a third Proxmox host to avoid adding a separate physical machine. Create a Debian 13 unprivileged container with nesting enabled and install corosync-qdevice inside it. This is what I do, and it works flawlessly. Backups are still mandatory. High availability handles node failure, not filesystem corruption, accidental rm -rf, or data loss. Pair your HA cluster with Proxmox Backup Server for scheduled backups. Cluster filesystem replication. pmxcfs synchronises /etc/pve/ across all nodes — including corosync.conf, storage configs, user permissions, and VM configs. Any change on one node propagates instantly. Conclusion Building a Proxmox VE cluster with high availability in your homelab is not as complicated as it sounds. The core stack — corosync for membership, QDevice for quorum, watchdog for fencing, and the HA manager for recovery — has been battle-tested in production environments for years. With two modest Proxmox hosts, a cheap Raspberry Pi (or a spare LXC), and shared storage, you get enterprise-grade HA at homelab prices.\nStart with one cluster, enable HA on your most critical VMs, run the failover test, and then expand from there.\nNFS outage behavior The application-storage example uses a hard mount. Soft mounts can return I/O errors and risk corruption; disabling locking is not a general migration workaround. Confirm the selected NFS version, locking semantics, client permissions, and outage/recovery behavior before moving persistent data. This source revision does not establish a live migration or HA failover test. See the NFS mount documentation.\n","permalink":"https://blog.gntech.me/posts/2026-06-09-proxmox-ve-cluster-high-availability-homelab/","summary":"A step-by-step guide to building a Proxmox VE cluster with high availability for your homelab — covering corosync setup, quorum with QDevice, HA groups, live migration, and fencing.","title":"Proxmox VE Cluster Setup and High Availability for Homelab"},{"content":"Why Self-Host Home Automation with Home Assistant on Docker Home Assistant is the leading open-source home automation platform, integrating over 2,000 devices and services into a single privacy-focused dashboard. Running it locally means your lights, sensors, and automations keep working even when your internet goes down — no cloud dependency, no subscription fees, no data leaving your network.\nHome Assistant Installation Methods Compared Home Assistant offers several installation paths:\nMethod Pros Cons Best For Home Assistant OS Full supervisor, add-ons, OTA updates Heavy VM, less flexible Plug-and-play users Home Assistant Supervised Add-ons on existing OS Complex setup, fragile Advanced OS users Home Assistant Core (Docker) Lightweight, full control, easy backups Manual add-on management Homelab operators Home Assistant Container Pre-packaged Docker image Same as Core Docker-first setups For a Proxmox homelab where you already run Docker, Home Assistant Core in Docker is the sweet spot. You get:\nMinimal resource footprint (~500 MB RAM at idle) Complete control over the stack (MQTT, Zigbee2MQTT, backups) Easy migration and version pinning via Docker tags Integration with your existing Traefik, monitoring, and backup infrastructure Architecture Overview The stack runs in a single Docker Compose project on a Debian 12 VM or LXC container:\nInternet → Traefik (SSL) → Home Assistant (port 8123) ├── Mosquitto (MQTT broker, port 1883) ├── Zigbee2MQTT (serial via USB) └── Code Server (optional — config editor) Hardware Requirements VM/LXC: 2 vCPUs, 2 GB RAM, 20 GB disk (Debian 12) Zigbee Coordinator: Sonoff Zigbee 3.0 USB dongle (CC2652P) or Conbee II Network: Static IP on your management VLAN Proxmox VM Prerequisites for Home Assistant Docker Create the Debian 12 VM On your Proxmox host, create a lightweight VM:\n1 2 3 4 5 6 7 8 9 10 11 12 # Download Debian 12 cloud image wget https://cloud.debian.org/images/cloud/bookworm/latest/debian-12-generic-amd64.qcow2 # Create VM with 2 vCPUs, 2 GB RAM, 20 GB disk qm create 200 --name hass --cores 2 --memory 2048 --net0 virtio,bridge=vmbr0 qm importdisk 200 debian-12-generic-amd64.qcow2 local-lvm qm set 200 --scsihw virtio-scsi-pci --scsi0 local-lvm:vm-200-disk-0 qm set 200 --ide2 local-lvm:cloudinit qm set 200 --boot order=scsi0 qm set 200 --agent enabled=1 qm resize 200 scsi0 20G qm start 200 Alternatively, use an LXC container for lighter overhead (no separate kernel):\n1 2 3 4 pveam download local debian-12-standard_12.7-1_amd64.tar.zst pct create 200 /var/lib/vz/template/cache/debian-12-standard_12.7-1_amd64.tar.zst \\ --storage local-lvm --rootfs 20 --memory 2048 --cores 2 --net0 name=eth0,bridge=vmbr0,ip=dhcp pct start 200 Enable USB Passthrough for Zigbee Coordinator If using a VM, attach the USB Zigbee dongle:\n1 2 3 4 5 6 # On Proxmox host, find the USB device lsusb # Example output: Bus 001 Device 005: ID 1a86:55d4 QinHeng Electronics SONOFF Zigbee 3.0 USB dongle # Attach to the VM qm set 200 --usb0 host=1a86:55d4 For an LXC container, bind-mount the USB device:\n1 2 3 4 5 6 7 # On Proxmox host, find the USB tty device ls -l /dev/serial/by-id/ # Example: usb-ITEAD_SONOFF_Zigbee_3.0_USB_Dongle_V2_... # Edit LXC config echo \u0026#39;lxc.cgroup2.devices.allow: c 188:* rwm\u0026#39; \u0026gt;\u0026gt; /etc/pve/lxc/200.conf echo \u0026#39;lxc.mount.entry: /dev/serial/by-id/usb-ITEAD_SONOFF_Zigbee_3.0_USB_Dongle_V2_... dev/ttyZigbee none bind,optional,create=file\u0026#39; \u0026gt;\u0026gt; /etc/pve/lxc/200.conf Install Docker on the VM/LXC SSH into the new VM/LXC and install Docker:\n1 2 3 4 5 6 7 8 9 10 apt update \u0026amp;\u0026amp; apt upgrade -y apt install -y ca-certificates curl install -m 0755 -d /etc/apt/keyrings curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc chmod a+r /etc/apt/keyrings/docker.asc echo \u0026#34;deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian bookworm stable\u0026#34; \\ \u0026gt; /etc/apt/sources.list.d/docker.list apt update apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin systemctl enable --now docker Docker Compose Configuration for Home Assistant Stack Create the project directory and files:\n1 2 mkdir -p /opt/homeassistant/{config,mqtt,zigbee2mqtt} cd /opt/homeassistant Create docker-compose.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 services: homeassistant: image: ghcr.io/home-assistant/home-assistant:stable container_name: homeassistant restart: unless-stopped volumes: - ./config:/config - /etc/localtime:/etc/localtime:ro - /run/dbus:/run/dbus:ro ports: - \u0026#34;127.0.0.1:8123:8123\u0026#34; environment: - TZ=America/Santo_Domingo depends_on: mosquitto: condition: service_healthy networks: - hass-net deploy: resources: limits: memory: 1G cpus: \u0026#34;1.0\u0026#34; mosquitto: image: eclipse-mosquitto:2 container_name: mosquitto restart: unless-stopped volumes: - ./mqtt/config:/mosquitto/config - ./mqtt/data:/mosquitto/data - ./mqtt/log:/mosquitto/log ports: - \u0026#34;127.0.0.1:1883:1883\u0026#34; healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;mosquitto_sub\u0026#34;, \u0026#34;-t\u0026#34;, \u0026#34;$$SYS/broker/version\u0026#34;, \u0026#34;--retained-only\u0026#34;, \u0026#34;-C\u0026#34;, \u0026#34;1\u0026#34;, \u0026#34;-W\u0026#34;, \u0026#34;5\u0026#34;] interval: 30s timeout: 10s retries: 3 networks: - hass-net deploy: resources: limits: memory: 128M cpus: \u0026#34;0.25\u0026#34; zigbee2mqtt: image: koenkk/zigbee2mqtt:latest container_name: zigbee2mqtt restart: unless-stopped volumes: - ./zigbee2mqtt/data:/app/data - /run/udev:/run/udev:ro devices: - /dev/ttyZigbee:/dev/ttyZigbee environment: - TZ=America/Santo_Domingo depends_on: mosquitto: condition: service_healthy networks: - hass-net deploy: resources: limits: memory: 256M cpus: \u0026#34;0.5\u0026#34; networks: hass-net: driver: bridge Expose Home Assistant and Mosquitto only on localhost — Traefik handles external access. Zigbee2MQTT does not need port exposure.\nMosquitto MQTT Broker Setup for Home Automation Create the Mosquitto configuration:\n1 mkdir -p /opt/homeassistant/mqtt/{config,data,log} Create /opt/homeassistant/mqtt/config/mosquitto.conf:\n1 2 3 4 5 6 7 8 9 listener 1883 127.0.0.1 allow_anonymous false password_file /mosquitto/config/passwd persistence true persistence_location /mosquitto/data log_dest file /mosquitto/log/mosquitto.log log_type all connection_messages true log_timestamp true Create MQTT users:\n1 2 3 4 docker compose run --rm mosquitto mosquitto_passwd -c /mosquitto/config/passwd hass # Enter password when prompted docker compose run --rm mosquitto mosquitto_passwd -b /mosquitto/config/passwd zigbee2mqtt z2m_secret_pass Restart Mosquitto to load the config:\n1 2 docker compose up -d mosquitto docker compose logs mosquitto Zigbee2MQTT Docker Integration with USB Passthrough Zigbee2MQTT bridges your Zigbee coordinator to MQTT, allowing Home Assistant to control all Zigbee devices through a single integration.\nCreate /opt/homeassistant/zigbee2mqtt/data/configuration.yaml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 homeassistant: true permit_join: false mqtt: base_topic: zigbee2mqtt server: mqtt://mosquitto:1883 user: zigbee2mqtt password: z2m_secret_pass serial: port: /dev/ttyZigbee adapter: zstack frontend: port: 8080 experimental: new_api: true advanced: log_level: info network_key: GENERATE_ME pan_id: GENERATE_ME channel: 15 transmit_power: 20 report: true device_options: legacy: false Generate the network key and PAN ID:\n1 2 3 4 5 6 # Generate 16-byte hex key for Zigbee network openssl rand -hex 16 # Example output: 1a2b3c4d5e6f708192a3b4c5d6e7f809 # Generate PAN ID (2 bytes, between 0x0001 and 0xFFFE) echo \u0026#34;0x$(openssl rand -hex 2)\u0026#34; Replace GENERATE_ME in the configuration with your generated values. The channel setting should match the least congested 2.4 GHz channel in your environment. Channels 15, 20, and 25 are preferred because they avoid common Wi-Fi channels (1, 6, 11).\nPairing Zigbee Devices To pair a new device, temporarily enable joining:\n1 2 3 # Via MQTT mosquitto_pub -h 127.0.0.1 -u hass -p YOUR_PASSWORD \\ -t zigbee2mqtt/bridge/request/permit_join -m \u0026#39;{\u0026#34;value\u0026#34;: true, \u0026#34;time\u0026#34;: 120}\u0026#39; Or set permit_join: true in the configuration, pair your devices, then set it back to false. Never leave permit_join enabled permanently — it is a security risk.\nVerify devices appear via MQTT:\n1 docker compose exec mosquitto mosquitto_sub -t zigbee2mqtt/# -v Home Assistant Core Configuration and Integrations Home Assistant auto-generates a basic configuration.yaml on first launch. Start the stack to initialize it:\n1 2 docker compose up -d docker compose logs -f homeassistant Wait for the log to show:\nHome Assistant Core is running! Listening on http://0.0.0.0:8123 Access the VM IP on port 8123 (or via Traefik once configured). Complete the initial onboarding — create your user account, set your location, and name your home.\nAdding Integrations From the Home Assistant web UI, navigate to Settings → Devices \u0026amp; Services → Add Integration:\nMQTT — Click configure, enter mosquitto as broker, port 1883, username hass, and the MQTT password. No SSL needed since Mosquitto binds to localhost. Zigbee2MQTT — Search and add. It auto-discovers via MQTT topics. All paired Zigbee devices appear automatically. Essential configuration.yaml Additions Over time, add these to /opt/homeassistant/config/configuration.yaml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 # Home Assistant configuration.yaml — add to existing file # Set a more specific time zone homeassistant: latitude: !secret latitude longitude: !secret longitude unit_system: metric time_zone: America/Santo_Domingo name: GnTech Home # Enable energy dashboard energy: # HTTP configuration (for Traefik proxying) http: use_x_forwarded_for: true trusted_proxies: - 10.0.0.0/8 - 172.16.0.0/12 - 192.168.0.0/16 # History and recorder recorder: db_url: !secret recorder_db_url purge_keep_days: 30 # Logbook logbook: # System health system_health: # Update notifications updater: # Sun tracking for automations sun: Using Secrets Create /opt/homeassistant/config/secrets.yaml to keep sensitive values out of configuration.yaml:\n1 2 3 latitude: 18.4861 longitude: -69.9312 recorder_db_url: !env_var RECORDER_DB_URL Home Assistant\u0026rsquo;s !env_var directive reads environment variables from the Docker container. Set them under environment: in the compose file.\nTraefik Reverse Proxy Configuration for Home Assistant Add Traefik labels to the homeassistant service in docker-compose.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 services: homeassistant: # ... existing config ... labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.hass.rule=Host(`hass.yourlab.com`)\u0026#34; - \u0026#34;traefik.http.routers.hass.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.hass.tls=true\u0026#34; - \u0026#34;traefik.http.routers.hass.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.services.hass.loadbalancer.server.port=8123\u0026#34; - \u0026#34;traefik.http.middlewares.hass-headers.headers.customrequestheaders.X-Forwarded-For={{clientIp}}\u0026#34; - \u0026#34;traefik.http.routers.hass.middlewares=hass-headers@docker\u0026#34; If using a Traefik dynamic config file instead:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 # /opt/traefik/config/homeassistant.yml http: routers: hass: rule: \u0026#34;Host(`hass.yourlab.com`)\u0026#34; entryPoints: [\u0026#34;websecure\u0026#34;] tls: certResolver: letsencrypt service: hass middlewares: - hass-headers services: hass: loadBalancer: servers: - url: \u0026#34;http://homeassistant:8123\u0026#34; middlewares: hass-headers: headers: customRequestHeaders: X-Forwarded-For: \u0026#34;{clientIp}\u0026#34; Add an external network to the compose file so Traefik and Home Assistant can communicate:\n1 2 3 4 5 6 7 8 9 10 11 12 networks: hass-net: driver: bridge traefik-net: external: true name: traefik-net services: homeassistant: networks: - hass-net - traefik-net The http.use_x_forwarded_for and trusted_proxies settings in configuration.yaml ensure Home Assistant sees the real client IP behind Traefik.\nHome Assistant Backup Automation and Recovery Use Home Assistant\u0026rsquo;s supported backup workflow Use the installed release\u0026rsquo;s Backup integration and recovery documentation. Current releases provide backup configuration through the UI across installation types. Check the capabilities of the older image used elsewhere in this historical deployment example before following current UI steps.\nCreate a backup, save an independent copy, and retain the encryption/recovery materials outside this host. Record which Home Assistant data is included. Separate Compose services such as MQTT and Zigbee2MQTT need their own configuration and data recovery plan; a Home Assistant backup does not automatically include every adjacent container.\nThe previous script paused the stack and then attempted exec into a paused container. It also excluded authentication state and copied a SQLite database without establishing a complete database recovery point. That procedure and its timer are removed. Do not extract a database file over a running Home Assistant instance or remove volumes to prepare a restore.\nTest restoration in a separate instance with fresh storage and device/network isolation. Prevent the test from issuing real automations, MQTT commands, or duplicate radio/network identities. Verify login, configuration, history as required, integrations, and attachments. Keep the original data until the recovery test and any planned cutover succeed.\nFor ordinary prepared files, the Restic recovery lab demonstrates encrypted copies. It does not by itself validate a Home Assistant or SQLite backup. This source revision does not claim a new Home Assistant runtime restore test.\nDocker Image Updates with Watchtower Add Watchtower to monitor the Home Assistant Docker images:\n1 2 3 4 5 6 7 8 docker run -d \\ --name watchtower \\ --restart unless-stopped \\ -v /var/run/docker.sock:/var/run/docker.sock \\ containrrr/watchtower \\ --interval 86400 \\ --cleanup \\ --scope hass-stack Tag your compose services with com.centurylinklabs.watchtower.scope=hass-stack to scope updates:\n1 2 3 4 5 6 7 services: homeassistant: labels: - \u0026#34;com.centurylinklabs.watchtower.scope=hass-stack\u0026#34; zigbee2mqtt: labels: - \u0026#34;com.centurylinklabs.watchtower.scope=hass-stack\u0026#34; Conclusion Running Home Assistant Core with Docker on Proxmox gives you a production-grade home automation stack that is fully local, fully under your control, and easy to maintain alongside your other homelab services.\nThe stack cost:\n2 vCPUs, ~1.5 GB RAM at idle (with Zigbee2MQTT and Mosquitto) 4 Docker containers, all on a single bridge network Zero cloud dependency for core functionality From here, explore:\nEnergy dashboard — connect Shelly or Zigbee energy monitoring plugs Local voice pipeline — run Whisper + Piper for fully offline voice control ESPHome — flash ESP32/ESP8266 devices as custom sensors and relays Node-RED — visual automation flows beyond Home Assistant\u0026rsquo;s built-in automations For official documentation, visit home-assistant.io and the Zigbee2MQTT docs.\n","permalink":"https://blog.gntech.me/posts/2026-06-09-home-assistant-docker-deployment-proxmox/","summary":"Deploy Home Assistant Core with Docker Compose on your Proxmox homelab. Complete guide covering Zigbee2MQTT, USB passthrough, MQTT broker, Traefik reverse proxy, and automated backups.","title":"Home Assistant Docker Deployment Guide — Self-Hosted Home Automation on Proxmox"},{"content":"What Is VRF and Why Use It in Your Homelab VRF (Virtual Routing and Forwarding) is a technology that allows a single router to maintain multiple independent routing tables. Each VRF has its own forwarding table, its own set of interfaces, and its own routing protocol instances. This means IP addresses can overlap across VRFs without conflict, and traffic in one VRF never leaks into another by default.\nIn a homelab context, VRFs solve a problem that VLANs alone cannot. VLANs separate traffic at Layer 2, but once traffic reaches the router\u0026rsquo;s Layer 3 interface, everything converges into a single routing table. VRFs extend separation to Layer 3 — each VRF gets its own routing table, its own default gateway, and its own routing protocol sessions.\nCommon homelab use cases for VRFs include:\nMulti-WAN separation — each WAN uplink in its own VRF with independent routing policies IoT isolation — IoT devices use a separate routing domain with controlled internet access and no LAN reachability Guest networks — guests get a VRF with internet-only access, no inter-VRF transit Lab environments — overlapping IP ranges for testing without conflicts with production Tenant isolation — separate routing domains for different users or services on shared infrastructure MikroTik RouterOS 7 introduced robust VRF support that has matured significantly through versions 7.14 through 7.23. With the recent RouterOS 7.24beta1 release, VRF integration continues to improve. RouterOS 7 uses a routing-table-based VRF model where each VRF maps to a routing table, separate from the main table.\nPrerequisites and Topology Overview Before configuring VRFs, ensure you are running RouterOS 7.x — VRF support is not available in RouterOS 6. Any RouterOS 7 device works: CHR, RB5009, RB4011, CCR series, or x86 installations. CRS3xx switches running RouterOS in L3 mode also support VRF.\nThis guide uses a three-VRF topology for a typical homelab:\nVRF Name Purpose Subnet Uplink Table ID (main) WAN / mgmt DHCP from ISP ether1 main VRF-RED Trusted LAN 192.168.10.0/24 bridge-red 100 VRF-BLUE IoT network 10.0.100.0/24 bridge-iot 200 VRF-GREEN Guest 172.16.0.0/24 bridge-gst 300 The bridge interfaces (bridge-red, bridge-iot, bridge-gst) are each part of a separate VLAN-aware bridge on different physical switch ports. This combines VLAN L2 separation with VRF L3 routing.\nCreating VRFs and Assigning Interfaces Creating a VRF in RouterOS 7 is straightforward — add a VRF entry that maps to a routing table, then assign interfaces.\n1 2 3 4 5 6 7 8 9 # Create routing tables for each VRF /routing table add name=vrf-red fib=100 /routing table add name=vrf-blue fib=200 /routing table add name=vrf-green fib=300 # Create VRF entries linking interfaces to tables /routing vrf add interface=bridge-red routing-mark=vrf-red /routing vrf add interface=bridge-iot routing-mark=vrf-blue /routing vrf add interface=bridge-gst routing-mark=vrf-green Alternatively, you can assign interfaces and set routing marks through the IP settings:\n/ip vrf add interface=bridge-red routing-mark=vrf-red /ip vrf add interface=bridge-iot routing-mark=vrf-blue /ip vrf add interface=bridge-gst routing-mark=vrf-green Verify the configuration:\n1 2 /routing table print /routing vrf print Example output:\n1 2 3 4 5 Columns: NAME, INTERFACES # NAME INTERFACES 0 vrf-red bridge-red 1 vrf-blue bridge-iot 2 vrf-green bridge-gst Configuring IP Addresses on VRF Interfaces Each VRF interface needs an IP address. Because the interface is assigned to a VRF, the address belongs to that VRF\u0026rsquo;s routing table only.\n# Trusted LAN (VRF-RED) /ip address add address=192.168.10.1/24 interface=bridge-red network=192.168.10.0 # IoT network (VRF-BLUE) /ip address add address=10.0.100.1/24 interface=bridge-iot network=10.0.100.0 # Guest network (VRF-GREEN) /ip address add address=172.16.0.1/24 interface=bridge-gst network=172.16.0.0 The key behavior here: these addresses only appear in their respective VRF routing tables. A ping to 192.168.10.1 from the guest network (VRF-GREEN) will not reach it unless routes between VRFs exist.\nStatic Routes with VRF Routing Tables Each VRF needs its own default route. Set the routing mark to match the VRF:\n# Default route in VRF-RED via ISP gateway /ip route add dst-address=0.0.0.0/0 gateway=100.64.0.1 routing-table=vrf-red distance=1 # Default route in VRF-BLUE via ISP gateway /ip route add dst-address=0.0.0.0/0 gateway=100.64.0.1 routing-table=vrf-blue distance=1 # Default route in VRF-GREEN via ISP gateway /ip route add dst-address=0.0.0.0/0 gateway=100.64.0.1 routing-table=vrf-green distance=1 For connected subnets within the homelab, add static routes so each VRF knows how to reach its local networks:\n/ip route add dst-address=192.168.10.0/24 gateway=192.168.10.1 routing-table=vrf-red /ip route add dst-address=10.0.100.0/24 gateway=10.0.100.1 routing-table=vrf-blue /ip route add dst-address=172.16.0.0/24 gateway=172.16.0.1 routing-table=vrf-green Verify routes per VRF:\n/ip route print where table=vrf-red /ip route print where table=vrf-blue /ip route print where table=vrf-green BGP VRF Configuration For larger homelabs with dynamic routing, RouterOS 7 supports BGP within VRFs. Each VRF gets its own BGP instance and peering.\n1 2 3 4 5 6 # Create a BGP instance bound to VRF-RED /routing bgp instance set default vrf=vrf-red # Or add a separate instance for a specific VRF /routing bgp instance add name=bgp-vrf-red vrf=vrf-red as=64501 /routing bgp instance add name=bgp-vrf-blue vrf=vrf-blue as=64502 Add BGP peers within each VRF:\n1 2 3 4 5 # Peer with upstream router in VRF-RED /routing bgp peer add name=peer-red instance=bgp-vrf-red remote-address=10.0.0.2 remote-as=64500 # Peer with upstream in VRF-BLUE /routing bgp peer add name=peer-blue instance=bgp-vrf-blue remote-address=10.0.1.2 remote-as=64500 Redistribute connected routes into BGP per VRF:\n1 2 /routing bgp network add network=192.168.10.0/24 synchronize=yes /routing bgp network add network=10.0.100.0/24 synchronize=yes The BGP process inside each VRF only sees routes from its own routing table. A BGP update received via VRF-RED goes into the vrf-red table only.\nRoute Leaking Between VRFs By default, VRFs are completely isolated — no traffic passes between them. When you need selective inter-VRF communication (e.g., IoT devices reaching a DNS server on the trusted LAN), you configure route leaking.\nRoute Leaking with Routing Filters RouterOS 7 uses routing filter rules to import or export routes between tables. The pattern creates a filter chain attached to the destination table:\n1 2 3 4 5 6 7 8 9 # Create a filter chain that leaks specific routes into VRF-RED /routing filter rule add chain=leak-to-vrf-red \\ src-address=192.168.10.0/24 action=accept /routing filter rule add chain=leak-to-vrf-red \\ dst-address=10.0.100.0/24 action=accept /routing filter rule add chain=leak-to-vrf-red action=discard # Apply filter to the destination table (VRF-RED imports these routes) /routing table set vrf-red filter-chain=leak-to-vrf-red A simpler and more common pattern for static leaking uses static routes with appropriate scope and target-scope:\n# On the main table, add route through VRF-RED gateway /ip route add dst-address=10.0.100.0/24 gateway=10.0.100.1 routing-table=main # On VRF-RED, add route to main table networks /ip route add dst-address=10.0.20.0/24 gateway=192.168.10.100 routing-table=vrf-red The key consideration: you need a routing path between the VRFs. If the router has an interface in both VRFs (or a shared management network), you can leak specific routes.\nFor more controlled leaking in RouterOS 7, use the routing rule feature:\n1 2 3 4 5 # Add a routing rule to redirect specific traffic /routing rule add src-address=172.16.0.0/24 dst-address=192.168.10.53/32 \\ action=lookup table=vrf-red /routing rule add src-address=10.0.100.0/24 dst-address=192.168.10.53/32 \\ action=lookup table=vrf-red This pattern redirects traffic from VRF-GREEN and VRF-BLUE to the DNS server at 192.168.10.53 into the VRF-RED table, enabling DNS resolution without full inter-VRF routing.\nFirewall Rules per VRF Firewall in a VRF environment requires careful attention. The RouterOS firewall processes packets against the routing table the packet arrived on, not the VRF name. This means your rules should match on inbound interfaces.\nDefault-Deny Inter-VRF Policy # Drop inter-VRF traffic by default (forward chain) /ip firewall filter add chain=forward src-address-list=!trusted_subnets \\ action=drop comment=\u0026#34;Default deny inter-VRF\u0026#34; # Or match specific interface combinations /ip firewall filter add chain=forward in-interface=bridge-iot \\ out-interface=bridge-red action=drop \\ comment=\u0026#34;Block IoT to trusted LAN\u0026#34; /ip firewall filter add chain=forward in-interface=bridge-gst \\ out-interface=bridge-red action=drop \\ comment=\u0026#34;Block guest to trusted LAN\u0026#34; /ip firewall filter add chain=forward in-interface=bridge-gst \\ out-interface=bridge-iot action=drop \\ comment=\u0026#34;Block guest to IoT\u0026#34; Selective Allow Rules Allow specific services between VRFs:\n# Allow IoT to reach DNS (192.168.10.53) on trusted LAN /ip firewall filter add chain=forward in-interface=bridge-iot \\ out-interface=bridge-red dst-address=192.168.10.53/32 \\ protocol=udp dst-port=53 action=accept \\ comment=\u0026#34;Allow IoT DNS to trusted DNS\u0026#34; # Allow IoT to reach NTP on trusted LAN /ip firewall filter add chain=forward in-interface=bridge-iot \\ out-interface=bridge-red dst-address=192.168.10.53/32 \\ protocol=udp dst-port=123 action=accept \\ comment=\u0026#34;Allow IoT NTP to trusted NTP\u0026#34; # Allow guest internet (WAN access) - strict /ip firewall filter add chain=forward in-interface=bridge-gst \\ out-interface=ether1 action=accept \\ comment=\u0026#34;Allow guest to WAN\u0026#34; # Drop guest to internal networks /ip firewall filter add chain=forward in-interface=bridge-gst \\ out-interface=!ether1 action=drop \\ comment=\u0026#34;Block guest to internal\u0026#34; NAT Per VRF NAT rules must also account for VRFs. Configure masquerade for each VRF\u0026rsquo;s outbound traffic:\n# NAT for VRF-RED /ip firewall nat add chain=srcnat out-interface=ether1 \\ routing-mark=vrf-red action=masquerade \\ comment=\u0026#34;NAT VRF-RED to WAN\u0026#34; # NAT for VRF-BLUE /ip firewall nat add chain=srcnat out-interface=ether1 \\ routing-mark=vrf-blue action=masquerade \\ comment=\u0026#34;NAT VRF-BLUE to WAN\u0026#34; # NAT for VRF-GREEN /ip firewall nat add chain=srcnat out-interface=ether1 \\ routing-mark=vrf-green action=masquerade \\ comment=\u0026#34;NAT VRF-GREEN to WAN\u0026#34; The routing-mark match ensures that only traffic belonging to the specific VRF gets NAT\u0026rsquo;d.\nComplete Homelab Topology Example Here is a consolidated example combining all the concepts. This assumes a MikroTik RB5009 with ether1 as WAN, ether2-ether5 for trusted LAN, ether6-ether7 for IoT, and ether8 as guest access point uplink.\n# --- Routing Tables --- /routing table add name=vrf-red fib=100 /routing table add name=vrf-blue fib=200 /routing table add name=vrf-green fib=300 # --- VRF Definitions --- /routing vrf add interface=bridge-red routing-mark=vrf-red /routing vrf add interface=bridge-iot routing-mark=vrf-blue /routing vrf add interface=bridge-gst routing-mark=vrf-green # --- IP Addresses --- /ip address add address=192.168.10.1/24 interface=bridge-red \\ network=192.168.10.0 /ip address add address=10.0.100.1/24 interface=bridge-iot \\ network=10.0.100.0 /ip address add address=172.16.0.1/24 interface=bridge-gst \\ network=172.16.0.0 # --- Default Routes per VRF --- /ip route add dst-address=0.0.0.0/0 gateway=100.64.0.1 \\ routing-table=vrf-red distance=1 /ip route add dst-address=0.0.0.0/0 gateway=100.64.0.1 \\ routing-table=vrf-blue distance=1 /ip route add dst-address=0.0.0.0/0 gateway=100.64.0.1 \\ routing-table=vrf-green distance=1 # --- NAT --- /ip firewall nat add chain=srcnat out-interface=ether1 \\ routing-mark=vrf-red action=masquerade /ip firewall nat add chain=srcnat out-interface=ether1 \\ routing-mark=vrf-blue action=masquerade /ip firewall nat add chain=srcnat out-interface=ether1 \\ routing-mark=vrf-green action=masquerade # --- Firewall: Inter-VRF Default-Deny --- /ip firewall filter add chain=forward in-interface=bridge-iot \\ out-interface=bridge-red action=drop /ip firewall filter add chain=forward in-interface=bridge-gst \\ out-interface=bridge-red action=drop /ip firewall filter add chain=forward in-interface=bridge-gst \\ out-interface=bridge-iot action=drop # --- Firewall: Selective Allows --- /ip firewall filter add chain=forward in-interface=bridge-iot \\ out-interface=bridge-red dst-address=192.168.10.53/32 \\ protocol=udp dst-port=53 action=accept /ip firewall filter add chain=forward in-interface=bridge-iot \\ out-interface=bridge-red dst-address=192.168.10.53/32 \\ protocol=udp dst-port=123 action=accept # --- Firewall: Guest internet only --- /ip firewall filter add chain=forward in-interface=bridge-gst \\ out-interface=ether1 action=accept /ip firewall filter add chain=forward in-interface=bridge-gst \\ out-interface=!ether1 action=drop # --- DNS Forwarding for IoT --- /routing rule add src-address=10.0.100.0/24 dst-address=192.168.10.53/32 \\ action=lookup table=vrf-red Verification and Troubleshooting Test connectivity within each VRF:\n1 2 3 4 5 6 7 8 # Ping from router in VRF-RED /ping 192.168.10.100 vrf=vrf-red # Ping internet from VRF-BLUE /ping 8.8.8.8 vrf=vrf-blue # Traceroute from VRF-GREEN /tool traceroute 1.1.1.1 vrf=vrf-green Verify routing tables are populated correctly:\n1 2 3 /routing route print where table=vrf-red /routing route print where table=vrf-blue /routing route print where table=vrf-green Common pitfalls and solutions:\nProblem Symptom Fix No internet in VRF Ping fails to 8.8.8.8 from VRF Check default route and routing-mark on NAT rule Inter-VRF traffic blocked Can ping but higher ports fail Review firewall filter rules — connection tracking expected state Address in wrong table ip route print shows missing routes Verify VRF interface assignment with /routing vrf print BGP not establishing Peer stuck in idle BGP source address must be in the VRF interface\u0026rsquo;s subnet DNS not resolving Name resolution fails from IoT Check routing rule for DNS leaking into VRF-RED table For persistent connectivity issues, enable traffic logging on the firewall to see which interface pair is dropping packets:\n/ip firewall filter add chain=forward in-interface=bridge-iot \\ out-interface=bridge-red action=log \\ comment=\u0026#34;Trace IoT to trusted drops\u0026#34; Remove the logging rule after debugging.\nWhen to Use VRFs vs Other Segmentation Methods VRFs solve a different problem than VLANs. Use this decision matrix:\nMethod Separation Level Use Case VLAN Layer 2 Broadcast domain isolation, guest WiFi SSIDs VRF Layer 3 Overlapping IPs, separate routing domains, multi-ISP VLAN + VRF Layer 2 + 3 Full isolation with L2 segmentation and independent routing Bridge filter Layer 2 only Simple inter-VLAN blocking without L3 routing In practice, most homelabs benefit from VLANs for L2 separation combined with VRFs for L3 routing where traffic crosses different routing domains. If your ISP provides a /29 public block and you also have a secondary LTE uplink, VRFs keep those routing tables independent.\nFor simple guest WiFi isolation where guests just need internet, a VLAN with a firewall rule blocking inter-VLAN traffic is sufficient. Add VRFs when you need overlapping IP addressing or independent routing policies per tenant.\nSummary VRFs in MikroTik RouterOS 7 provide powerful Layer 3 network segmentation that goes beyond what VLANs offer. Each VRF gets its own routing table, its own BGP instances, and its own set of firewall and NAT rules. Combined with VLANs at Layer 2, VRFs enable full multi-tenant network isolation in a homelab environment.\nStart by identifying your logical segmentation needs — trusted LAN, IoT, guest, lab. Create a VRF per segment, assign interfaces, configure per-VRF routing with default routes and NAT, then add firewall rules to enforce inter-VRF policy. Route leaking and routing rules provide controlled gateways between VRFs for specific services like DNS and NTP.\nRouterOS 7 has matured VRF support to a point where it is stable enough for daily-driver homelab use. Start with the topology and config in this guide, adapt the IP ranges and interface names to your setup, and test each VRF independently before enabling inter-VRF forwarding.\n","permalink":"https://blog.gntech.me/posts/2026-06-08-mikrotik-routeros7-vrf-configuration/","summary":"Configure VRFs on MikroTik RouterOS 7 for multi-tenant network segmentation. Step-by-step guide covering VRF creation, static routes, BGP VRF, route leaking between VRFs, firewall integration, and practical homelab topology examples.","title":"MikroTik RouterOS 7 VRF Configuration — Multi-Tenant Network Segmentation"},{"content":"What Is Outline and Why Self-Host It Outline is an open-source knowledge base and wiki platform designed for team collaboration. Its clean, Notion-like interface supports real-time editing, Markdown formatting, nested collections, and a rich search experience powered by PostgreSQL full-text search.\nSelf-hosting Outline gives you full control over your documentation data — no per-user SaaS pricing, no data leaving your infrastructure, and no feature-gating behind expensive enterprise tiers. For homelab operators, Outline solves the recurring problem of keeping infrastructure docs, runbooks, and network diagrams in one searchable place instead of scattered across text files and chat messages.\nCompared to other self-hosted documentation tools:\nBookStack — structured, book-and-shelf hierarchy, simpler auth (LDAP/SAML) Docusaurus — static site generator, great for public docs, requires builds Outline — real-time collaborative editor, Markdown-native, OIDC/SAML/SSO, Slack integration Outline works best as an internal team wiki where multiple people contribute and edit collaboratively.\nPrerequisites and Architecture Before deploying, you need:\nDocker Engine 24+ and Docker Compose v2 installed A domain name pointing to your homelab (e.g., outline.yourlab.com) An OIDC-compatible identity provider — this guide uses Authentik, but Google, Azure AD, Keycloak, or any OpenID Connect provider works Optional: S3-compatible object storage (MinIO) — Outline defaults to local disk for file uploads Minimum hardware: 1 GB RAM, 2 vCPUs, 10 GB disk. At idle, the stack consumes about 500 MB of RAM.\nThe architecture is straightforward:\nUser → Traefik (SSL termination) → Outline (container) ├── PostgreSQL 16 (documents, users, metadata) ├── Redis 7 (session cache, rate limiting) └── MinIO (optional — file attachments) Environment and Directory Setup Create the project directory and set up the environment file:\n1 2 mkdir -p /opt/outline/data cd /opt/outline Generate secret keys for Outline:\n1 2 3 4 5 # Generate a 64-character random secret key openssl rand -hex 32 # Generate a separate utils secret openssl rand -hex 32 Create .env in /opt/outline:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 cat \u0026gt; .env \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; # General NODE_ENV=production APP_NAME=GnTech Wiki APP_URL=https://outline.yourlab.com PORT=3000 FORCE_HTTPS=true # Secrets (replace with your generated values) SECRET_KEY=your-64-hex-char-secret UTILS_SECRET=your-64-hex-char-utils-secret # PostgreSQL DATABASE_URL=postgres://outline_user:outline_password@postgres:5432/outline PGSSLMODE=disable # Redis REDIS_URL=redis://redis:6379 # OIDC Authentication (Authentik) OIDC_CLIENT_ID=your-authentik-client-id OIDC_CLIENT_SECRET=your-authentik-client-secret OIDC_AUTH_URI=https://auth.yourlab.com/application/o/authorize/ OIDC_TOKEN_URI=https://auth.yourlab.com/application/o/token/ OIDC_USERINFO_URI=https://auth.yourlab.com/application/o/userinfo/ OIDC_LOGOUT_URI=https://auth.yourlab.com/application/o/end-session/ OIDC_SCOPES=openid profile email # File Storage (local disk — switch to S3 below if using MinIO) FILE_STORAGE=local MAXIMUM_IMPORT_SIZE=52428800 # Default access DEFAULT_ACCESS=members EOF Docker Compose Configuration Create docker-compose.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 services: outline: image: outlinewiki/outline:latest restart: unless-stopped env_file: .env ports: - \u0026#34;127.0.0.1:3000:3000\u0026#34; volumes: - ./data:/var/lib/outline/data healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;curl\u0026#34;, \u0026#34;-f\u0026#34;, \u0026#34;http://localhost:3000/_health\u0026#34;] interval: 30s timeout: 10s retries: 3 depends_on: postgres: condition: service_healthy redis: condition: service_started networks: - outline-net deploy: resources: limits: memory: 512M cpus: \u0026#34;1.0\u0026#34; postgres: image: postgres:16-alpine restart: unless-stopped environment: POSTGRES_USER: outline_user POSTGRES_PASSWORD: outline_password POSTGRES_DB: outline volumes: - postgres-data:/var/lib/postgresql/data healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;pg_isready -U outline_user -d outline\u0026#34;] interval: 10s timeout: 5s retries: 5 networks: - outline-net deploy: resources: limits: memory: 256M cpus: \u0026#34;0.5\u0026#34; redis: image: redis:7-alpine restart: unless-stopped volumes: - redis-data:/data healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;redis-cli\u0026#34;, \u0026#34;ping\u0026#34;] interval: 10s timeout: 3s retries: 5 networks: - outline-net deploy: resources: limits: memory: 128M cpus: \u0026#34;0.25\u0026#34; networks: outline-net: external: true name: traefik-net volumes: postgres-data: redis-data: Expose Outline only on localhost port 3000 — Traefik handles external access. The traefik-net network must exist before deploying; create it if needed:\n1 docker network create traefik-net OIDC Authentication with Authentik Outline does not include a built-in user database — it requires an external OIDC, SAML, or Slack/Google OAuth provider. Authentik works well for this in a homelab.\nCreating the OIDC Provider in Authentik Navigate to Applications → Providers → Create Provider Select OAuth2/OpenID Provider and configure: Name: Outline Client Type: Confidential Redirect URIs: https://outline.yourlab.com/auth/oidc.callback Signing Key: Select a key or create one Navigate to Applications → Applications → Create Application Name: Outline Slug: outline Provider: select the Outline provider just created From the provider details page, copy the Client ID and Client Secret Place these values in the .env file as OIDC_CLIENT_ID and OIDC_CLIENT_SECRET. Update the URIs to match your Authentik domain.\nUnderstanding the OIDC Flow User visits outline.yourlab.com Outline redirects to Authentik\u0026rsquo;s authorize endpoint User authenticates (password, WebAuthn, TOTP — Authentik-enforced MFA works transparently) Authentik redirects back to Outline with an authorization code Outline exchanges the code for tokens and creates/logs in the user No user provisioning is needed — Outline auto-creates accounts on first login via OIDC. The first user to log in becomes the admin by default when DEFAULT_ACCESS=members.\nS3 Storage with MinIO (Optional) For production use, switch from local disk to S3-compatible object storage. Deploy MinIO alongside Outline:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 services: minio: image: minio/minio:latest restart: unless-stopped command: server /data --console-address \u0026#34;:9001\u0026#34; environment: MINIO_ROOT_USER: minioadmin MINIO_ROOT_PASSWORD: minioadminpassword volumes: - minio-data:/data ports: - \u0026#34;127.0.0.1:9000:9000\u0026#34; networks: - outline-net volumes: minio-data: Access the MinIO console at port 9001, create a bucket named outline-attachments, and create an access key. Then update FILE_STORAGE in .env:\n1 2 3 4 5 6 7 8 FILE_STORAGE=s3 AWS_REGION=us-east-1 AWS_ACCESS_KEY_ID=your-minio-access-key AWS_SECRET_ACCESS_KEY=your-minio-secret-key AWS_S3_UPLOAD_BUCKET_URL=http://minio:9000 AWS_S3_UPLOAD_BUCKET_NAME=outline-attachments AWS_S3_ACCELERATE_URL= AWS_S3_FORCE_PATH_STYLE=true AWS_S3_FORCE_PATH_STYLE=true is required for MinIO. If using AWS S3, set it to false and use a proper region.\nTraefik Reverse Proxy Configuration If you already run Traefik, add a new router and service for Outline. With dynamic config file (or Docker labels):\n1 2 3 4 5 6 7 8 9 10 11 # docker-compose.yml labels (add to outline service) labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.outline.rule=Host(`outline.yourlab.com`)\u0026#34; - \u0026#34;traefik.http.routers.outline.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.outline.tls=true\u0026#34; - \u0026#34;traefik.http.routers.outline.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.services.outline.loadbalancer.server.port=3000\u0026#34; - \u0026#34;traefik.http.middlewares.outline-ratelimit.ratelimit.average=100\u0026#34; - \u0026#34;traefik.http.middlewares.outline-ratelimit.ratelimit.burst=50\u0026#34; - \u0026#34;traefik.http.routers.outline.middlewares=outline-ratelimit\u0026#34; Or via a Traefik dynamic configuration file:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 # /opt/traefik/config/outline.yml http: routers: outline: rule: \u0026#34;Host(`outline.yourlab.com`)\u0026#34; entryPoints: [\u0026#34;websecure\u0026#34;] tls: certResolver: letsencrypt service: outline middlewares: - outline-ratelimit services: outline: loadBalancer: servers: - url: \u0026#34;http://outline:3000\u0026#34; middlewares: outline-ratelimit: rateLimit: average: 100 burst: 50 If Outline is on a different host than Traefik, replace the url in the service definition with the actual Docker host IP.\nFirst Launch and Validation Start the stack:\n1 2 docker compose up -d docker compose logs -f outline Wait for the health check to pass:\n1 docker compose ps You should see all three services in Up (healthy) status.\nMigration and Initial Setup Outline runs database migrations automatically on startup. Watch the logs for:\nMigrations up to date Listening on port 3000 Visit https://outline.yourlab.com. You will be redirected to Authentik for login. On first successful login, Outline creates your account. The first user gets admin privileges by default when DEFAULT_ACCESS=members.\nVerification Checklist Login via OIDC redirects back to Outline without errors Collections page loads with Getting Started collection Create a document with Markdown formatting Upload an image attachment Search returns results from document titles and content Troubleshooting Issue Likely Cause Fix OIDC redirect loop Redirect URI mismatch Verify Authentik redirect URI exactly matches https://outline.yourlab.com/auth/oidc.callback File uploads fail (local) Volume permissions chown -R 1000:1000 /opt/outline/data 502 Bad Gateway (Traefik) Network mismatch Ensure Outline is on traefik-net network Database connection refused PostgreSQL not ready Check depends_on healthcheck: docker compose logs postgres Blank page after login CORS or URL mismatch Verify APP_URL in .env matches the access URL exactly Backup and Maintenance Automate daily PostgreSQL backups with a systemd timer:\n1 2 3 4 5 6 7 8 9 10 cat \u0026gt; /opt/outline/backup.sh \u0026lt;\u0026lt; \u0026#39;SCRIPT\u0026#39; #!/bin/bash BACKUP_DIR=/opt/outline/backups TIMESTAMP=$(date +%Y%m%d-%H%M%S) mkdir -p $BACKUP_DIR docker exec $(docker compose ps -q postgres) pg_dump -U outline_user outline \\ | gzip \u0026gt; $BACKUP_DIR/outline-$TIMESTAMP.sql.gz find $BACKUP_DIR -name \u0026#34;outline-*.sql.gz\u0026#34; -mtime +30 -delete SCRIPT chmod +x /opt/outline/backup.sh Create the systemd service and timer:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 cat \u0026gt; /etc/systemd/system/outline-backup.service \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; [Unit] Description=Outline database backup After=docker.service [Service] Type=oneshot ExecStart=/opt/outline/backup.sh WorkingDirectory=/opt/outline User=root EOF cat \u0026gt; /etc/systemd/system/outline-backup.timer \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; [Unit] Description=Daily Outline database backup [Timer] OnCalendar=daily Persistent=true [Install] WantedBy=timers.target EOF systemctl daemon-reload systemctl enable --now outline-backup.timer For Docker image updates, add Outline to your Watchtower setup:\n1 2 3 4 5 6 7 docker run -d \\ --name watchtower \\ -v /var/run/docker.sock:/var/run/docker.sock \\ containrrr/watchtower \\ --interval 86400 \\ --cleanup \\ outline Conclusion Outline fills the gap for homelab documentation that is collaborative, real-time, and always searchable. With OIDC authentication via Authentik, you get SSO with MFA enforcement across all your self-hosted services in one identity layer. The Docker Compose setup runs comfortably alongside other services on a single homelab host.\nStart populating your Outline wiki with infrastructure documentation, network diagrams, service runbooks, and configuration references. Over time it becomes the single source of truth for your homelab operations.\nFor full configuration options, see the Outline self-hosting documentation.\n","permalink":"https://blog.gntech.me/posts/2026-06-08-outline-wiki-docker-deployment/","summary":"Deploy Outline wiki on your homelab with Docker Compose. Complete guide covering PostgreSQL, Redis, OIDC auth with Authentik, S3/MinIO storage, and Traefik reverse proxy.","title":"Outline Wiki Deployment Guide — Self-Hosted Knowledge Base with Docker"},{"content":"What LibreNMS Monitors and Why It Belongs in Your Homelab LibreNMS is a fully-featured, auto-discovering network monitoring system. Think of it as PRTG or SolarWinds for the self-hosted world — except it\u0026rsquo;s open source, has no device limits, and runs happily in Docker.\nIf your homelab has more than a handful of network devices — MikroTik routers, managed switches, Proxmox hosts, Linux servers — you\u0026rsquo;ve probably run into the pain of manually checking each one when something breaks. LibreNMS solves this by polling every device at configurable intervals, graphing interface utilization, CPU, memory, storage, and temperature sensors, and alerting you when thresholds cross.\nUnlike Prometheus + Grafana, which requires a separate exporter per metric type, LibreNMS speaks SNMP natively. Enable SNMP on a device, add it to LibreNMS, and it automatically discovers every interface, disk, process, and sensor the device exposes.\nIn this guide you\u0026rsquo;ll deploy the full LibreNMS Docker Compose stack, configure SNMP on MikroTik RouterOS and Linux hosts, set up autodiscovery, create alert rules, and optionally integrate Oxidized for automated config backups.\nDocker Compose Stack for LibreNMS The official LibreNMS Docker image uses MariaDB for storage and Redis for queue and cache. Create a directory and the compose file:\n1 2 3 mkdir -p /opt/librenms cd /opt/librenms nano docker-compose.yml Here is the complete compose file:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 version: \u0026#39;3.8\u0026#39; services: db: image: mariadb:10.11 container_name: librenms-db restart: unless-stopped volumes: - ./data/mysql:/var/lib/mysql environment: - MYSQL_ROOT_PASSWORD=changeme_root_pass - MYSQL_DATABASE=librenms - MYSQL_USER=librenms - MYSQL_PASSWORD=changeme_librenms_pass - TZ=America/Santo_Domingo command: - \u0026#34;mysqld\u0026#34; - \u0026#34;--innodb-file-per-table=1\u0026#34; - \u0026#34;--innodb-buffer-pool-size=512M\u0026#34; - \u0026#34;--character-set-server=utf8mb4\u0026#34; - \u0026#34;--collation-server=utf8mb4_unicode_ci\u0026#34; redis: image: redis:7-alpine container_name: librenms-redis restart: unless-stopped volumes: - ./data/redis:/data librenms: image: librenms/librenms:latest container_name: librenms restart: unless-stopped ports: - \u0026#34;8000:8000\u0026#34; volumes: - ./data/librenms:/data environment: - DB_HOST=db - DB_NAME=librenms - DB_USER=librenms - DB_PASSWORD=changeme_librenms_pass - REDIS_HOST=redis - TZ=America/Santo_Domingo depends_on: - db - redis Pull and start the stack:\n1 docker compose up -d The first startup initializes the database. Check the logs until the web UI is ready:\n1 docker compose logs -f librenms Initial Setup via Web Installer Point your browser to http://your-docker-host:8000/install. The installer will guide you through:\nChecking prerequisites (all should pass in the Docker image) Database credentials — leave as defaults since they match the compose environment Creating the admin user — username, password, email Generating the config.php — the Docker image handles this through environment variables After the installer completes, log in and go to Settings → Global Settings → Poller Settings to adjust the polling interval (default 5 minutes) and enable distributed polling if you run multiple pollers.\nSNMP Configuration on Target Devices MikroTik RouterOS LibreNMS reads the standard SNMP MIBs that RouterOS exposes. Start with SNMPv2c for simplicity, then move to SNMPv3 for security.\nSNMPv2c (quick setup):\n# SSH into your MikroTik /system identity set name=router-core /snmp community set public addresses=10.0.20.0/24 /snmp set enabled=yes contact=\u0026#34;admin@example.com\u0026#34; location=\u0026#34;Rack1\u0026#34; SNMPv3 (recommended for production):\n1 2 3 4 5 /snmp set enabled=yes /snmp community remove public /snmp community remove private /snmp v3 add name=librenms auth-protocol=sha auth-password=Str0ngAuthPass \\ privacy-protocol=aes privacy-password=Str0ngPrivPass group=full-access Linux and Proxmox Hosts Install and configure snmpd on each host to expose system metrics:\n1 apt update \u0026amp;\u0026amp; apt install -y snmpd Edit /etc/snmp/snmpd.conf:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 agentAddress udp:161 view systemonly included .1.3.6.1.2.1.1 view systemonly included .1.3.6.1.2.1.25.1 rocommunity public 10.0.20.0/24 sysLocation \u0026#34;Rack1\u0026#34; sysContact admin@example.com # Disk monitoring disk / disk /var/lib/vz disk /var/log # Interface monitoring includeAllDisks 10% Restart snmpd:\n1 2 systemctl restart snmpd systemctl enable snmpd Verify connectivity from your Docker host:\n1 docker exec librenms snmpwalk -v2c -c public 10.0.20.30 1.3.6.1.2.1.1 Managed Switches (MikroTik CRS3xx) Add SNMP on MikroTik switches the same way:\n1 2 3 /snmp set enabled=yes /snmp community set public addresses=10.0.20.0/24 /snmp set contact=\u0026#34;rack@home\u0026#34; location=\u0026#34;Switch-Room\u0026#34; Adding Devices and Autodiscovery LibreNMS can discover devices automatically by scanning subnets. Enable it in Settings → Global Settings → Autodiscovery:\nSet Autodiscovery to Enabled Set Autodiscovery Networks to your homelab CIDR blocks, e.g. 10.0.20.0/24,10.0.30.0/24 Then run discovery manually from inside the container:\n1 docker exec librenms lnms device:discover --snmp-community=public --ping-fallback This scans the configured subnets, tests SNMP reachability, and adds any device that responds.\nTo manually add a single device via the command line:\n1 docker exec librenms lnms device:add --community=public --version=v2c 10.0.20.1 You can also use the Devices → Add Device form in the web UI.\nDevice Groups Organize devices by type. Go to Devices → Device Groups and create groups:\nMikroTik — match by sysDescr regex /MikroTik|RouterOS/ Linux Servers — match by sysDescr regex /Linux/ Proxmox — match by sysLocation or manual assignment Groups make it easier to set per-group alert rules and build focused dashboards.\nAlerting and Notifications LibreNMS ships with sensible default alert rules. You can also create custom ones.\nDefault Alert Rules Go to Alerts → Alert Rules. Pre-configured rules include:\nDevice Down — ICMP unreachable or SNMP timeout Port Down — administrative + operational status changes High CPU — threshold per device type High Storage — disk usage above 90% Telegram Notifications Create a transport in Alerts → Alert Transports → Create Transport:\nTransport Type: Telegram API Key: your bot token from BotFather Chat ID: your Telegram chat ID Format: Markdown or HTML Then map transport to alert rules via Alert Rules → edit → Transport → Telegram.\nCustom Alert Rule Example Create a rule that fires when a MikroTik CPU exceeds 80% for two polling cycles:\nGo to Alerts → Alert Rules → Create Alert Rule Rule name: MikroTik High CPU Builder mode: Advanced Query: %macros.device AND %macros.sensor_cpu Condition: sensor_current \u0026gt;= 80 Interval: check every 600s (two standard polling intervals) Max alerts per day: 3 Transport: select your Telegram transport You can also use the rule builder without writing SQL:\n1 2 3 4 5 -- Check for all devices where CPU load average exceeds 4.0 SELECT * FROM devices LEFT JOIN device_perf ON device_perf.device_id = devices.device_id WHERE devices.status = 1 AND device_perf.x15m \u0026gt; 4.0 Dashboard and Graphing LibreNMS automatically generates per-device graphs. Navigate to a device in the web UI to see:\nOverview — summary with health, uptime, and sysinfo Ports — traffic graphs for every interface Health — CPU, memory, storage, temperature Apps — application-level monitoring (nginx, MySQL, Docker, etc.) To add application monitoring for Docker on a Linux host:\n1 2 3 4 # On the monitored host docker exec librenms lnms config:set apps.netsnmp-docker.enabled true cp /opt/librenms/config/apps/docker/docker.yaml /etc/snmp/ systemctl restart snmpd Custom Dashboard Create a custom dashboard: Overview → Dashboards → Create Dashboard.\nUse the \u0026ldquo;Worldmap\u0026rdquo; widget to pin device locations for WAN link visibility. Use \u0026ldquo;Device Summary\u0026rdquo; widgets per group for at-a-glance health.\nWidget 1: Device Summary — all MikroTik devices Widget 2: Alerts — outstanding alerts Widget 3: Traffic Graphs — top 5 ports by utilization Oxidized Config Backup Integration (Optional) Oxidized automatically backs up RouterOS and switch configurations whenever they change, and stores the history in a Git repository.\nDeploy Oxidized in Docker alongside LibreNMS:\n1 2 3 4 5 6 7 8 9 10 11 12 services: oxidized: image: oxidized/oxidized:latest container_name: oxidized restart: unless-stopped ports: - \u0026#34;8888:8888\u0026#34; volumes: - ./data/oxidized:/root/.config/oxidized environment: - CONFIG_RELOAD_INTERVAL=600 - TZ=America/Santo_Domingo Add it to the same LibreNMS compose file and configure LibreNMS integration: Settings → Global Settings → Oxidized Integration → enable, set the Oxidized URL to http://oxidized:8888.\nOxidized needs a RouterOS-specific model. Create the config file data/oxidized/config:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 --- source: default: csv csv: file: /root/.config/oxidized/router.db delimiter: !ruby/regexp /:/ map: name: 0 model: 1 username: 2 password: 3 group: 4 gpg: false model_map: routeros: mikrotik output: default: git git: user: Oxidized email: oxidized@homelab repo: /root/.config/oxidized/oxidized-backups.git And data/oxidized/router.db:\n1 2 router-core:mikrotik:admin:admin_pass:Routers crs326-switch:mikrotik:admin:admin_pass:Switches After setup, RouterOS config changes are automatically committed. View history in LibreNMS under Device → Config → History.\nTraefik Reverse Proxy Integration Expose LibreNMS behind Traefik for HTTPS access via a subdomain like librenms.homelab.example.com. Add Traefik labels to the compose file:\n1 2 3 4 5 6 7 8 9 10 11 12 services: librenms: labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.librenms.rule=Host(`librenms.homelab.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.librenms.entrypoints=https\u0026#34; - \u0026#34;traefik.http.routers.librenms.tls=true\u0026#34; - \u0026#34;traefik.http.routers.librenms.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.services.librenms.loadbalancer.server.port=8000\u0026#34; - \u0026#34;traefik.http.routers.librenms.middlewares=authelia@docker\u0026#34; networks: - traefik-net Add traefik-net to the services and remove the ports section for LibreNMS to avoid direct HTTP exposure.\nPerformance Tuning for Larger Networks If your homelab grows beyond 50 devices, optimize the poller:\n1 2 3 4 5 6 7 8 9 10 # Increase poller threads in config.php docker exec librenms lnms config:set poller_threads 16 # Enable RRD caching through Redis docker exec librenms lnms config:set rrdcached true docker exec librenms lnms config:set rrdcached_server unix:/var/run/rrdcached.sock # Disable unused discovery modules per device docker exec librenms lnms config:set discovery_modules.os false docker exec librenms lnms config:set discovery_modules.stp false For distributed polling, run additional poller containers that connect to the same database — LibreNMS handles multi-poller natively.\nSummary LibreNMS fills a specific gap in the homelab monitoring landscape: automatic SNMP-based discovery with zero per-metric config. Deploy it alongside your existing Prometheus + Grafana stack for network-focused observability, or use it standalone for a quick overview of every device on your network.\nStart by enabling SNMP on your MikroTik RouterOS devices — the standard MIBs cover interfaces, CPU, memory, and temperature. Then expand to Linux hosts, switches, and Proxmox nodes. Once autodiscovery is running, LibreNMS finds new devices automatically, so your monitoring coverage grows with your lab.\n","permalink":"https://blog.gntech.me/posts/2026-06-07-librenms-docker-deployment-network-monitoring-homelab/","summary":"Deploy LibreNMS in Docker for complete homelab network monitoring. SNMP autodiscovery, alert rules, and dashboard setup for MikroTik routers, Proxmox hosts, and Linux servers.","title":"LibreNMS Docker Deployment — Complete Homelab Network Monitoring Guide"},{"content":"Why Linux Bridges Deserve Your Debugging Attention Every time a Docker container talks to another container, or a Proxmox VM sends a packet to the gateway, that traffic flows through a Linux bridge. Docker\u0026rsquo;s default bridge network uses docker0. Every user-defined Docker bridge is a Linux bridge. Every Proxmox virtual bridge — vmbr0, vmbr1, etc. — is the same kernel primitive under the hood.\nWhen something breaks — container can\u0026rsquo;t reach the internet, VM on a VLAN can\u0026rsquo;t ping its gateway, or traffic simply vanishes between two bridged interfaces — the problem is almost always in the bridge forwarding table, a missing VLAN tag, or iptables interference.\nThis guide covers the exact commands and techniques to find and fix those problems. You\u0026rsquo;ll use brctl, bridge (the modern iproute2 tool), tcpdump, and Docker namespace inspection to trace traffic from container to wire.\nInspecting Linux Bridges with brctl and bridge Start with the big picture. The bridge-utils package (brctl) is still widely available, but the iproute2 bridge tool is the modern replacement and should be your default.\nList All Bridges 1 2 3 4 5 # Old-school brctl show # Modern equivalent bridge link show brctl show prints bridge name, bridge ID, STP status, and interfaces. The bridge command with no arguments shows all bridge ports with their state and flags.\nInspect the MAC Forwarding Table The forwarding database (FDB) is where bridges learn which MAC addresses are on which port. A stale or missing entry is the most common cause of mysterious packet loss.\n1 2 3 4 5 # Show MAC table for a specific bridge brctl showmacs br0 # Modern: show FDB with port details bridge fdb show br br0 Look for entries where ageing timer keeps resetting — that indicates MAC flapping (the same MAC appearing on different ports), usually caused by a network loop or misconfigured bonding.\nCheck VLAN Membership If you use VLAN filtering on your bridges (common on Proxmox in VLAN-aware mode), verify which VLANs are allowed on each port:\n1 2 bridge vlan show bridge vlan show dev veth1234 A missing VLAN on the port means traffic tagged with that VLAN gets dropped silently — one of the hardest issues to diagnose without this command.\nDisable STP on Docker Bridge Networks Docker user-defined bridges have STP enabled by default on some distributions. This can block ports for 15–30 seconds during container restarts, causing connection timeouts:\n1 2 3 4 5 6 # Check STP status brctl stp docker0 brctl showstp docker0 # Disable STP (Docker bridge only — leave it on for physical networks) brctl stp docker0 off Packet Capture on Bridge Interfaces with tcpdump Capturing on the bridge interface itself shows all traffic flowing through it, including inter-container traffic that never touches a physical NIC.\nBasic Capture on the Bridge 1 2 3 4 5 6 7 8 # See everything on the bridge with MAC addresses tcpdump -i br0 -e -n # Filter to ARP only (debugging neighbor discovery) tcpdump -i br0 -e -n arp # See VLAN tags tcpdump -i br0 -e -n vlan The -e flag is critical — it prints MAC source and destination addresses so you can verify the bridge is forwarding to the correct port.\nDebug: Container Can\u0026rsquo;t Reach the Internet A container that can ping its gateway\u0026rsquo;s IP but not the internet usually points to NAT or routing — but let\u0026rsquo;s verify at layer 2 first:\n1 2 3 4 5 # From the host, capture on docker0 tcpdump -i docker0 -e -n icmp and host 8.8.8.8 # In another terminal, ping from inside the container docker exec mycontainer ping -c 3 8.8.8.8 If you see the ICMP request on docker0 but no reply, the bridge is forwarding correctly — the issue is upstream (NAT iptables rule, routing table, or ISP). If you see nothing, the packet isn\u0026rsquo;t reaching the bridge — check container routes and default gateway.\nDocker Network Namespace Inspection Every Docker container lives in its own network namespace. To bridge traffic, Docker creates a veth pair — one end inside the container\u0026rsquo;s namespace, the other plugged into the bridge.\nFind a Container\u0026rsquo;s Network Namespace 1 2 3 4 5 6 7 8 # Get the container PID PID=$(docker inspect -f \u0026#39;{{.State.Pid}}\u0026#39; mycontainer) # Enter its network namespace to inspect interfaces nsenter -t $PID -n -- ip addr nsenter -t $PID -n -- ip route nsenter -t $PID -n -- ip neigh nsenter -t $PID -n -- iptables -L -n -v This is far more revealing than docker exec. You see exactly what the container sees: its ARP cache, its routing table, its firewall rules.\nDebug: Container Resolves DNS But Cannot Connect This classic symptom — ping google.com resolves but connection fails — is almost never a bridge problem, but here\u0026rsquo;s how to confirm:\n1 2 3 4 5 6 7 # Check routes from inside the container nsenter -t $PID -n -- ip route # Check default gateway nsenter -t $PID -n -- ip route show default # Check that the container\u0026#39;s veth is the default route via the bridge IP If the default route points to the bridge IP correctly, the issue is upstream (NAT/firewall). If there\u0026rsquo;s no default route, restart the container or attach it to the correct network.\nMap Container veth to Bridge Port Each container gets a veth interface in the host namespace. Find which one belongs to your container:\n1 2 3 4 5 6 # Container side shows the veth peer index ETH_INDEX=$(docker exec mycontainer cat /sys/class/net/eth0/iflink) echo \u0026#34;Container\u0026#39;s host veth index: $ETH_INDEX\u0026#34; # Find the veth name from the index ip link | grep \u0026#34;^${ETH_INDEX}:\u0026#34; Then inspect that specific veth on the bridge:\n1 bridge fdb show br br0 | grep $(ip -o link show | grep \u0026#34;^${ETH_INDEX}:\u0026#34; | awk \u0026#39;{print $2}\u0026#39; | tr -d \u0026#39;:\u0026#39;) Proxmox Bridge and VLAN Troubleshooting Proxmox bridges follow the same Linux bridge rules, but with configuration in /etc/network/interfaces and a VLAN-aware mode that adds another layer.\nCheck Current Bridge Configuration 1 cat /etc/network/interfaces A typical Proxmox bridge with VLAN-aware mode:\nauto vmbr0 iface vmbr0 inet static address 10.0.20.30/24 gateway 10.0.20.1 bridge-ports enp3s0 bridge-stp off bridge-fd 0 bridge-vlan-aware yes bridge-vids 2-4094 Verify VLAN Interfaces Are Working 1 2 3 4 5 # List all VLAN devices ip link show type vlan # Check a specific VLAN interface cat /proc/net/vlan/vlan20 Debug: VM on Tagged VLAN Cannot Reach Gateway When a Proxmox VM is configured with VLAN tag 20 but can\u0026rsquo;t reach its gateway on VLAN 20:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 # 1. Check the bridge sees the VLAN bridge vlan show dev vmbr0 # 2. Verify the VM\u0026#39;s tap interface has the right VLAN tag bridge vlan show dev tap100i0 # 3. Capture on the bridge filtered to VLAN 20 tcpdump -i vmbr0 -e -n vlan 20 # 4. Check from the host that the VLAN interface exists ip addr show vmbr0.20 # 5. Verify the VLAN interface is up ip link set vmbr0.20 up Missing bridge-vlan-aware yes in the Proxmox config is the most common root cause. Without it, tagged traffic passes through untagged and the VM never sees its VLAN ID.\nCommon Issues and Fixes iptables Interfering with Bridge Traffic The bridge-nf-call sysctls cause iptables rules to apply to bridged traffic. This can break container-to-container communication if your FORWARD chain has restrictive policies:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 # Check current state sysctl net.bridge.bridge-nf-call-iptables sysctl net.bridge.bridge-nf-call-ip6tables # Disable for Docker bridge traffic (set to 0) echo 0 | sudo tee /proc/sys/net/bridge/bridge-nf-call-iptables echo 0 | sudo tee /proc/sys/net/bridge/bridge-nf-call-ip6tables # Make permanent cat \u0026gt; /etc/sysctl.d/99-bridge-nf-call.conf \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; net.bridge.bridge-nf-call-iptables = 0 net.bridge.bridge-nf-call-ip6tables = 0 EOF sysctl -p /etc/sysctl.d/99-bridge-nf-call.conf MAC Address Port Security Some managed switches learn MAC addresses on specific ports. When a container restarts and gets a new MAC, or you migrate a VM, the switch still forwards traffic for the old MAC to the wrong port. Clear the switch\u0026rsquo;s MAC table or set the VM/container to use a static MAC:\n1 2 3 4 # In Docker Compose — assign a static MAC services: myapp: mac_address: \u0026#34;02:42:ac:1a:00:01\u0026#34; On Proxmox, set hwaddr in the VM config under Hardware → Network Device.\nMTU Mismatches If your physical interface uses jumbo frames (MTU 9000) but Docker containers get MTU 1500, packets may get silently fragmented or dropped:\n1 2 3 4 5 6 7 8 # Check MTU across the path ip link show docker0 ip link show br0 ip link show enp3s0 # Set Docker daemon MTU # Add to /etc/docker/daemon.json: # { \u0026#34;mtu\u0026#34;: 9000 } For Proxmox bridges, set the MTU on the bridge interface itself:\n1 ip link set vmbr0 mtu 9000 STP Blocking Docker Bridge Ports STP on Docker bridges causes a 15–30 second delay every time a container starts or restarts. If you\u0026rsquo;re running containers that need fast failover or rapid restarts (like HA proxies or load balancers), disable STP on Docker-created bridges:\n1 2 3 for br in $(brctl show | grep -E \u0026#39;^br-[0-9a-f]+\u0026#39; | awk \u0026#39;{print $1}\u0026#39;); do brctl stp $br off done Automation: Bridge Health Check Script Save this as /usr/local/bin/bridge-health.sh and run it periodically with a systemd timer:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 #!/bin/bash # Bridge health check — log MAC table and flag suspicious entries BRIDGES=$(brctl show | grep -v \u0026#34;bridge name\u0026#34; | awk \u0026#39;{print $1}\u0026#39;) for br in $BRIDGES; do echo \u0026#34;=== $br MAC Table $(date) ===\u0026#34; bridge fdb show br \u0026#34;$br\u0026#34; | while read -r line; do mac=$(echo \u0026#34;$line\u0026#34; | awk \u0026#39;{print $1}\u0026#39;) port=$(echo \u0026#34;$line\u0026#34; | awk \u0026#39;{print $3}\u0026#39;) ageing=$(echo \u0026#34;$line\u0026#34; | awk \u0026#39;{print $5}\u0026#39;) # Flag MACs with ageing timer below 5 seconds (potential flapping) if [ -n \u0026#34;$ageing\u0026#34; ] \u0026amp;\u0026amp; [ \u0026#34;$ageing\u0026#34; -lt 5 ] 2\u0026gt;/dev/null; then echo \u0026#34;WARNING: $mac on $port has ageing timer $ageing — potential flapping\u0026#34; fi done done Make it executable and set a systemd timer for every 10 minutes:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 chmod +x /usr/local/bin/bridge-health.sh # systemd service cat \u0026gt; /etc/systemd/system/bridge-health.service \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; [Unit] Description=Bridge health check [Service] Type=oneshot ExecStart=/usr/local/bin/bridge-health.sh EOF # systemd timer cat \u0026gt; /etc/systemd/system/bridge-health.timer \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; [Unit] Description=Run bridge health check every 10 minutes [Timer] OnCalendar=*:0/10 [Install] WantedBy=timers.target EOF systemctl daemon-reload systemctl enable --now bridge-health.timer Building Your Bridge Debugging Mental Model Linux bridges are simple devices: they learn MAC-to-port mappings, forward frames based on destination MAC, and optionally filter by VLAN. When something breaks, start at the bridge and work outward:\nIs the bridge up? — ip link show $BRIDGE Is the container\u0026rsquo;s veth connected? — bridge link show / brctl show Does the bridge have the container\u0026rsquo;s MAC? — bridge fdb show br $BRIDGE Can you see packets on the bridge? — tcpdump -i $BRIDGE -e -n Is iptables interfering? — Check bridge-nf-call-iptables Is STP blocking? — brctl showstp $BRIDGE Work this chain from bottom to top, and you\u0026rsquo;ll isolate the fault in minutes instead of hours. The tools are already on your system — learn to use them, and networking problems become much less mysterious.\n","permalink":"https://blog.gntech.me/posts/2026-06-07-linux-bridge-networking-debugging-docker-proxmox/","summary":"A practical guide to debugging Linux bridge networks used by Docker and Proxmox. Covers tcpdump packet captures, brctl and bridge fdb inspection, VLAN troubleshooting, Docker network namespace traversal, and fixes for common connectivity problems.","title":"Linux Bridge Network Debugging — Docker and Proxmox Homelab Guide"},{"content":"Why systemd-networkd for Homelab Networking If you\u0026rsquo;ve installed Debian 13 \u0026ldquo;Trixie\u0026rdquo; or Ubuntu 24.04 recently, your system already ships with systemd-networkd installed and ready. It\u0026rsquo;s the default network manager on Arch Linux too. Despite this, many homelab guides still default to NetworkManager, netplan, or the legacy /etc/network/interfaces — and then hit a wall the moment they need a Linux bridge for KVM, a VLAN interface for network segmentation, or a bond for link aggregation.\nsystemd-networkd handles all of these natively with three configuration file types:\n.link — matches hardware devices by MAC, driver, or path and sets link-level parameters (MTU, MAC address, wake-on-lan) .netdev — defines virtual network devices (bridges, bonds, VLANs, VRFs, VXLANs) .network — applies IP addressing, routing, DNS, DHCP, and bridge port membership to any matched link The configuration lives in /etc/systemd/network/ and follows simple lexicographic ordering rules. No daemon restart required — networkctl reconfigure applies changes on the fly.\nFor a homelab hypervisor running Proxmox, KVM, or Docker on a Debian 13 host, switching to systemd-networkd means you get predictable, boot-order-independent networking with zero GUI dependencies.\nChecking Your Current Network State Before changing anything, survey what you\u0026rsquo;re running today:\n1 2 3 4 5 6 7 8 9 10 11 # Is systemd-networkd running? systemctl is-active systemd-networkd # List all links with their current configuration source networkctl list # Detailed status of a specific interface networkctl status enp2s0 # What else is managing your network? systemctl list-units --type=service | grep -iE \u0026#39;network|netplan|NetworkManager\u0026#39; If systemd-networkd isn\u0026rsquo;t active, you\u0026rsquo;re likely running NetworkManager (common on Ubuntu Server desktop-flavored installs) or netplan (Ubuntu default up to 24.04). Both can coexist with systemd-networkd on different interfaces, but for simplicity, pick one — systemd-networkd — for all host-level networking.\nPreparing the Host: Disabling Conflicting Managers You don\u0026rsquo;t need to uninstall anything. Just stop and disable the services you aren\u0026rsquo;t using:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 # Disable NetworkManager systemctl stop NetworkManager systemctl disable NetworkManager # Disable netplan (Ubuntu) systemctl stop netplan-ovs-cleanup 2\u0026gt;/dev/null systemctl disable netplan-ovs-cleanup 2\u0026gt;/dev/null mv /etc/netplan/01-netcfg.yaml /etc/netplan/01-netcfg.yaml.bak 2\u0026gt;/dev/null # Disable legacy ifupdown (Debian) systemctl mask networking.service 2\u0026gt;/dev/null mv /etc/network/interfaces /etc/network/interfaces.bak 2\u0026gt;/dev/null # Enable and start systemd-networkd systemctl enable systemd-networkd systemctl start systemd-networkd # Also enable systemd-resolved for DNS systemctl enable systemd-resolved systemctl start systemd-resolved If you\u0026rsquo;re on Ubuntu, also unlink the default /etc/resolv.conf so systemd-resolved manages DNS:\n1 ln -sf /run/systemd/resolve/stub-resolv.conf /etc/resolv.conf Linux Bridge Configuration for KVM and Docker The most common homelab networking scenario: you have a single physical NIC and need a Linux bridge so VMs and containers can share the host\u0026rsquo;s connection to your LAN.\nBridge Virtual Device — br0.netdev Create /etc/systemd/network/10-br0.netdev:\n1 2 3 4 [NetDev] Name=br0 Kind=bridge MACAddress=52:54:00:ab:cd:01 Setting a static MAC on the bridge is optional but recommended — it prevents the MAC from changing on reboot, which would break DHCP reservations and firewall rules.\nBridge Port Membership — 20-br0-enp2s0.network Create /etc/systemd/network/20-br0-enp2s0.network:\n1 2 3 4 5 [Match] Name=enp2s0 [Network] Bridge=br0 That\u0026rsquo;s it. The physical interface enp2s0 becomes a bridge port. No IP address on the physical NIC — all addressing happens on the bridge.\nBridge IP Configuration — 30-br0.network Create /etc/systemd/network/30-br0.network:\n1 2 3 4 5 6 7 8 9 10 11 12 [Match] Name=br0 [Network] DHCP=ipv4 DNS=1.1.1.1 DNS=8.8.8.8 IPv6AcceptRA=yes [DHCPv4] RouteMetric=100 UseDomains=true For a static IP instead:\n1 2 3 4 5 6 7 8 9 [Match] Name=br0 [Network] Address=10.0.20.50/24 Gateway=10.0.20.1 DNS=10.0.20.1 DNS=1.1.1.1 IPv6AcceptRA=no Apply the configuration:\n1 2 networkctl reconfigure br0 networkctl reconfigure enp2s0 Verify bridge state:\n1 2 3 networkctl status br0 bridge link show bridge vlan show KVM guests can now attach their virtual NICs directly to br0 via bridged networking, and Docker can be configured to use br0 through macvlan or ipvlan.\nVLAN Interface Configuration for Network Segmentation If your upstream switch delivers tagged VLANs, you need VLAN interfaces on the host. The standard pattern: attach the VLAN to the bridge so all VMs on the bridge can reach their respective VLAN — or attach VLANs directly to the physical NIC for host-only segmentation.\nVLAN on Physical NIC — VLAN 10 Management Create /etc/systemd/network/10-vlan10.netdev:\n1 2 3 4 5 6 [NetDev] Name=vlan10 Kind=vlan [VLAN] Id=10 Then /etc/systemd/network/30-vlan10.network:\n1 2 3 4 5 6 7 [Match] Name=vlan10 [Network] Address=10.0.10.5/24 Gateway=10.0.10.1 DNS=10.0.10.1 Attach the VLAN to the physical uplink by adding the parent reference in a .network file:\nCreate /etc/systemd/network/20-enp2s0-vlan10.network:\n1 2 3 4 5 [Match] Name=enp2s0 [Network] VLAN=vlan10 Repeat for VLAN 20 (services) and VLAN 30 (IoT):\n1 2 3 4 5 6 7 # /etc/systemd/network/11-vlan20.netdev [NetDev] Name=vlan20 Kind=vlan [VLAN] Id=20 And similarly for vlan30 with Id=30.\nVLAN Trunk on a Bridge (for VM access to multiple VLANs) For KVM guests that need access to multiple VLANs, attach the VLAN interfaces to the bridge:\nCreate /etc/systemd/network/10-br0.netdev (add to existing):\n1 2 3 [NetDev] Name=br0 Kind=bridge No VLAN reference in the bridge .netdev. Instead, modify 20-enp2s0.network to list VLANs on the bridge port:\n1 2 3 4 5 6 7 8 [Match] Name=enp2s0 [Network] Bridge=br0 VLAN=vlan10 VLAN=vlan20 VLAN=vlan30 And create .network files for each VLAN that set the bridge as their parent:\n1 2 3 4 5 6 # /etc/systemd/network/40-vlan10-bridge.network [Match] Name=vlan10 [Network] Bridge=br0 This way, the host bridge br0 carries untagged traffic, while VLANs 10, 20, and 30 are trunked through the bridge. VMs attached to br0 can be configured with VLAN filtering in libvirt.\nApply everything:\n1 2 3 4 networkctl reconfigure enp2s0 networkctl reconfigure vlan10 networkctl reconfigure vlan20 networkctl reconfigure vlan30 Verify VLAN tagging:\n1 bridge vlan show Output should show the PVID on the bridge port and the tagged VLANs:\nport vlan-id enp2s0 1 PVID Egress Untagged 10 20 30 br0 1 PVID Egress Untagged Bond (Link Aggregation) Configuration For servers with dual NICs, a bond provides redundancy and increased throughput. systemd-networkd supports all standard bonding modes — for homelab use, 802.3ad (LACP) is the standard choice when your switch supports it.\nBond Virtual Device — bond0.netdev Create /etc/systemd/network/10-bond0.netdev:\n1 2 3 4 5 6 7 8 9 10 [NetDev] Name=bond0 Kind=bond [Bond] Mode=802.3ad MIIMonitorSec=100ms UpDelaySec=200ms DownDelaySec=200ms LACPTransmitRate=fast Bond Slave Interfaces Create /etc/systemd/network/20-enp2s0-bond.network:\n1 2 3 4 5 [Match] Name=enp2s0 [Network] Bond=bond0 Create /etc/systemd/network/21-enp3s0-bond.network:\n1 2 3 4 5 [Match] Name=enp3s0 [Network] Bond=bond0 Bond IP Configuration Create /etc/systemd/network/30-bond0.network:\n1 2 3 4 5 [Match] Name=bond0 [Network] DHCP=ipv4 Or static:\n1 2 3 4 5 6 7 [Match] Name=bond0 [Network] Address=10.0.20.51/24 Gateway=10.0.20.1 DNS=10.0.20.1 Apply:\n1 2 3 networkctl reconfigure bond0 networkctl reconfigure enp2s0 networkctl reconfigure enp3s0 Verify bonding:\n1 2 cat /proc/net/bonding/bond0 networkctl status bond0 Look for MII Status: up on each slave and Bonding Mode: IEEE 802.3ad Dynamic link aggregation.\nCombining Bond and Bridge Need both bonding and bridging? Layer them — bond first, bridge on top:\n10-bond0.netdev as above, then 10-br0.netdev with Kind=bridge, and:\n1 2 3 4 5 6 # 20-bond0-bridge.network [Match] Name=bond0 [Network] Bridge=br0 This gives you a bonded, bridged setup where VMs, LXC containers, and Docker can all share redundant uplinks.\nApplying Changes and Rolling Back systemd-networkd applies configuration files in lexicographic order. The standard naming convention is XX-name.type where XX is a two-digit priority — lower numbers apply first.\nApply a single interface:\n1 networkctl reconfigure br0 Reload all configuration:\n1 systemctl restart systemd-networkd Roll back a change: simply rename or remove the .network or .netdev file and reconfigure:\n1 2 mv /etc/systemd/network/30-br0.network{,.bak} networkctl reconfigure br0 Common gotcha: if you reconfigure the interface carrying your SSH session, you get locked out. Always test changes through an out-of-band management interface (IPMI, iDRAC, serial console), or use at now + 1 minute to schedule a rollback before applying breaking config changes:\n1 echo \u0026#34;mv /etc/systemd/network/30-br0.network.bak /etc/systemd/network/30-br0.network; networkctl reconfigure br0\u0026#34; | at now + 2 minutes Conclusion systemd-networkd is the most straightforward way to manage Linux networking in a homelab. Its declarative .network and .netdev files make bridge, VLAN, and bond configuration predictable and repeatable — no daemon GUI, no Python-generated config files, just clean INI-style files that work on every modern distro.\nFor a Debian 13 hypervisor running KVM, LXC, and Docker, the combination of a Linux bridge with tagged VLAN interfaces and optional LACP bonding covers every networking scenario you\u0026rsquo;ll encounter. And because systemd-networkd is built into systemd itself, there\u0026rsquo;s no additional package to install or maintain.\nNext time you provision a new homelab host, skip NetworkManager and write a .network file instead. Your future self — the one adding a second NIC or creating a VLAN trunk — will thank you.\n","permalink":"https://blog.gntech.me/posts/2026-06-07-systemd-networkd-homelab-networking-bridges-vlans-bonds/","summary":"Configure bridges, VLANs, and bond interfaces using systemd-networkd on Debian 13 and Ubuntu 24.04. Real configuration files for KVM/Proxmox bridges, VLAN trunking, and link aggregation.","title":"systemd-networkd Homelab Networking — Bridges, VLANs, Bonds"},{"content":"Docker Engine 29.0.0 introduced an experimental native nftables firewall backend. It is different from using Docker\u0026rsquo;s iptables backend through the iptables-nft compatibility tools.\nThe important differences: Docker creates ip docker-bridges and ip6 docker-bridges tables, there is no native nftables DOCKER-USER chain, and Docker does not enable IP forwarding for you. The backend cannot be enabled while Docker is running in Swarm mode.\nCheck the documentation for your installed Engine release before migrating. Experimental behavior may change.\n1. Record the Current State Use a maintenance window and console access. Restarting Docker can interrupt containers and published services.\n1 2 3 4 5 6 docker version docker info sudo nft list ruleset sudo iptables -S FORWARD sudo iptables -S DOCKER-USER sysctl net.ipv4.ip_forward net.ipv6.conf.all.forwarding Check Swarm in docker info. Do not continue on a Swarm host. iptables --version reporting nf_tables identifies a compatibility frontend, not Docker\u0026rsquo;s selected backend.\nSave backups before modifying anything:\n1 2 3 4 5 6 7 sudo mkdir -p /root/docker-firewall-backup sudo sh -c \u0026#39;nft list ruleset \u0026gt; /root/docker-firewall-backup/nftables.rules\u0026#39; sudo sh -c \u0026#39;iptables-save \u0026gt; /root/docker-firewall-backup/iptables.rules\u0026#39; sudo sh -c \u0026#39;ip6tables-save \u0026gt; /root/docker-firewall-backup/ip6tables.rules\u0026#39; if sudo test -f /etc/docker/daemon.json; then sudo cp -a /etc/docker/daemon.json /root/docker-firewall-backup/daemon.json fi These snapshots are for review and recovery planning. Do not blindly replay a whole ruleset over a running Docker daemon.\n2. Prepare Forwarding and Existing Policy Docker\u0026rsquo;s native nftables backend needs forwarding enabled before it starts bridge networks. On a host with several interfaces, first ensure your firewall blocks unintended forwarding between physical networks.\nFor IPv4 bridge networking, create a dedicated sysctl file:\n1 2 3 4 sudo tee /etc/sysctl.d/90-docker-forwarding.conf \u0026gt;/dev/null \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; net.ipv4.ip_forward = 1 EOF sudo sysctl -p /etc/sysctl.d/90-docker-forwarding.conf If you enable Docker IPv6 networking, also plan and configure IPv6 forwarding and routing. Enabling IPv6 forwarding can affect router-advertisement acceptance; do not switch it on blindly on an RA-dependent host.\nReview existing DOCKER-USER rules and translate required restrictions into your own nftables table. A leftover iptables FORWARD policy of DROP can still block packets that nftables accepts. Reconcile that policy with your complete host firewall; changing it to ACCEPT without replacement restrictions can expose unrelated forwarding.\n3. Merge the Docker Configuration Keep Docker firewall-rule creation enabled. Setting iptables or ip6tables to false disables rule creation for both backends.\nThis script preserves existing JSON options and explicitly enables those rule settings. Stop here if their current false values support an intentionally managed firewall; migrate that design before changing them.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 sudo mkdir -p /etc/docker sudo python3 - \u0026lt;\u0026lt;\u0026#39;PYCODE\u0026#39; import json from pathlib import Path path = Path(\u0026#39;/etc/docker/daemon.json\u0026#39;) config = json.loads(path.read_text()) if path.exists() else {} if config.get(\u0026#39;iptables\u0026#39;) is False or config.get(\u0026#39;ip6tables\u0026#39;) is False: raise SystemExit(\u0026#39;Existing firewall-rule disabling detected; review the firewall design first.\u0026#39;) config[\u0026#39;firewall-backend\u0026#39;] = \u0026#39;nftables\u0026#39; config[\u0026#39;iptables\u0026#39;] = True config[\u0026#39;ip6tables\u0026#39;] = True path.write_text(json.dumps(config, indent=2) + \u0026#39;\\n\u0026#39;) PYCODE sudo dockerd --validate --config-file=/etc/docker/daemon.json A successful validation checks daemon configuration syntax. It does not prove that the host firewall or forwarding policy is correct. Also check for conflicting options supplied through the systemd service command line.\n4. Restart and Inspect the Actual Tables 1 2 3 4 5 sudo systemctl restart docker sudo systemctl --no-pager --full status docker sudo journalctl -u docker -n 80 --no-pager sudo nft list table ip docker-bridges sudo nft list table ip6 docker-bridges Inspect Docker\u0026rsquo;s tables, but do not modify them. Their internal chains can change between releases and Docker owns their contents.\n5. Add Restrictions in a Separate Table This optional example allows new IPv4 TCP connections to host address 10.0.20.50, published port 8080, only from 10.0.10.0/24 through ens18. Replace these values. It handles one published endpoint, not your entire firewall.\n1 2 3 4 5 6 7 8 9 10 sudo tee /root/docker-web8080.nft \u0026gt;/dev/null \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; table ip gntech_docker_web8080 { chain forward { type filter hook forward priority -1; policy accept; iifname \u0026#34;ens18\u0026#34; ct state new meta l4proto tcp ct original ip daddr 10.0.20.50 ct original proto-dst 8080 ip saddr != 10.0.10.0/24 counter drop } } EOF sudo nft -c -f /root/docker-web8080.nft sudo nft -f /root/docker-web8080.nft Run once on a host without an existing table of this name. Persist it through your existing nftables management process. Do not add flush ruleset to a reload script while Docker is running.\nAn nftables accept verdict in one base chain does not override a drop in another base chain. This policy only adds a restriction. Configure IPv6 restrictions separately if the service is published through IPv6.\nRollback of this example alone:\n1 sudo nft delete table ip gntech_docker_web8080 6. Test End to End Start a temporary loopback-only web service on a free port:\n1 2 3 docker run -d --name gntech-nft-test -p 127.0.0.1:18080:80 nginx:alpine curl --fail --connect-timeout 5 http://127.0.0.1:18080/ docker rm -f gntech-nft-test Then test your actual published services from allowed and blocked remote clients. Verify DNS and outbound access inside an existing application network, and verify inter-container communication. Loopback testing alone does not validate the forwarding filter.\nRepeat those checks after a reboot. A successful Docker restart is not enough: forwarding settings and firewall rules must survive boot.\nRoll Back the Backend Restore the exact previous daemon.json backup if one existed. If it did not, remove only the keys added by this guide from the new file. Validate the configuration and restart Docker. Restore your previous custom forwarding policy through its original firewall manager, then verify connectivity and exposure again.\nKeep the forwarding sysctl file only if your previous network design needs it. Removing a sysctl file does not immediately reset the running value.\nReferences Docker: native nftables backend and migration Docker: packet filtering and firewall-rule disabling ","permalink":"https://blog.gntech.me/posts/2026-06-06-docker-nftables-firewall-backend-homelab/","summary":"Migrate Docker from iptables to the native nftables firewall backend. Configure docker daemon.json for nftables on Debian 13, test port publishing, and troubleshoot common migration issues.","title":"Docker nftables Firewall Backend — Native Firewall for Containers"},{"content":"The OOM Problem on Docker Homelab Hosts Your homelab host runs Docker containers. Maybe a dozen. Maybe thirty. Postgres for Immich, Redis for Valkey cache, Prometheus hammering metrics every 15 seconds, Ollama loading a 7B model into RAM. The host has 16 GB total. Maybe 8 GB if you\u0026rsquo;re running on a repurposed thin client. And you\u0026rsquo;ve just rebuilt your media library — now Immich is transcoding, Postgres is indexing, and Prometheus is scraping.\nThen everything freezes. SSH stops responding. The web UI disappears. Thirty seconds later, things come back — but Postgres is gone. Or worse, Prometheus, and now you lost metrics. The kernel\u0026rsquo;s OOM killer picked a victim. It almost never picks the right one.\nThe Linux Out-Of-Memory (OOM) killer doesn\u0026rsquo;t understand Docker container boundaries or service priority. It sees memory pressure, calculates an oom_score for every process, and kills the highest scorer. A 1 GiB Postgres container that\u0026rsquo;s been running for months has a higher badness score than a memory-leaking sidecar that started five minutes ago. The kernel doesn\u0026rsquo;t know which container is critical to your homelab — it just sees big processes using memory.\nThis guide covers every layer of OOM defense for a Docker homelab host, from Docker Compose resource limits through kernel sysctl tuning. Apply all five layers and your critical services stay up under memory pressure.\nDocker Compose Resource Limits Are Your First Line of Defense The single most important thing you can do for Docker host stability is set resource limits on every container. Without limits, a single container can consume all host memory and force the kernel to make kill decisions.\nDocker Compose v3 supports deploy-level resource constraints:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 services: postgres: image: postgres:16-alpine deploy: resources: limits: memory: 512M cpus: \u0026#34;1.0\u0026#34; reservations: memory: 256M prometheus: image: prom/prometheus deploy: resources: limits: memory: 1G cpus: \u0026#34;1.5\u0026#34; ollama: image: ollama/ollama deploy: resources: limits: memory: 4G cpus: \u0026#34;2.0\u0026#34; With docker compose up (not swarm), these limits are applied directly via cgroups v2. When a container reaches its memory limit, Docker\u0026rsquo;s internal OOM kills the container itself — contained, predictable, and logged. Without limits, the kernel kills host-wide.\nFor docker run, the equivalent flags are:\n1 2 3 4 5 6 docker run -d \\ --name postgres \\ --memory=\u0026#34;512m\u0026#34; \\ --cpus=\u0026#34;1.0\u0026#34; \\ --memory-reservation=\u0026#34;256m\u0026#34; \\ postgres:16-alpine Key detail: By default Docker sets --memory-swap to twice --memory, giving each container swap access equal to its RAM limit. To disable swap per-container, set --memory-swap equal to --memory:\n1 docker run --memory=\u0026#34;512m\u0026#34; --memory-swap=\u0026#34;512m\u0026#34; postgres:16-alpine Every container in your homelab should have at least a memory limit. No exceptions. Even monitoring tools need limits — a Prometheus tsdb compaction spike can eat 2 GB before you notice.\nsystemd-oomd — Userspace OOM Prevention at the Cgroup Level Systemd v250+ ships systemd-oomd, a userspace OOM manager that monitors memory pressure at the cgroup level and kills offending cgroups before the kernel panic-kills random processes. It\u0026rsquo;s the cleanest approach for Docker hosts because Docker containers already live under systemd cgroup hierarchies.\nEnable it on Ubuntu 22.04+ and Debian Bookworm+:\n1 2 systemctl enable --now systemd-oomd systemctl status systemd-oomd Configure thresholds in /etc/systemd/oomd.conf:\n1 2 3 4 [OOM] DefaultMemoryPressureDurationSec=30 DefaultMemoryPressureLimit=60% SwapUsedLimitPercent=90% This tells oomd to kill the cgroup with the highest memory pressure when:\n60%+ memory pressure sustained for 30 seconds, or 90%+ swap is used To make Docker\u0026rsquo;s cgroup eligible for systemd-oomd management, add ManagedOOM=kill to the Docker daemon\u0026rsquo;s service override:\n1 systemctl edit docker Add:\n1 2 [Service] ManagedOOM=kill Then reload:\n1 2 systemctl daemon-reload systemctl restart docker Check oomd activity:\n1 journalctl -u systemd-oomd --since \u0026#34;1 hour ago\u0026#34; You should see entries like:\nsystemd-oomd[XXX]: Performing kill action for cgroup /system.slice/docker-\u0026lt;id\u0026gt;.scope (memory pressure critical) The key advantage over the kernel OOM killer: systemd-oomd kills the entire cgroup (all processes in the Docker container) rather than picking one process and leaving orphaned children. For Docker hosts, this is exactly what you want.\nEarlyoom — Simple OOM Prevention Daemon Earlyoom is a lightweight userspace OOM killer that monitors available memory and swap, and preemptively kills the process with the largest rss allocation before the kernel OOM killer activates. It\u0026rsquo;s simpler than systemd-oomd and works on any systemd-capable distribution.\nInstallation:\n1 apt install earlyoom Earlyoom\u0026rsquo;s default configuration is conservative: it acts when free memory drops below 10% and swap is below 10%. The recommended config for Docker hosts is more aggressive:\nEdit /etc/default/earlyoom:\nEARLYOOM_ARGS=\u0026#34;-m 5,10 -s 10,5 -r 3600 --prefer \u0026#39;(!postgres|!redis|!prometheus|!nginx)\u0026#39; --avoid \u0026#39;(!systemd|!sshd)\u0026#39;\u0026#34; Breaking this down:\n-m 5,10 — kill when available memory drops below 5%, report to syslog at 10% -s 10,5 — kill when available swap drops below 10%, report at 5% -r 3600 — report memory stats to syslog every hour --prefer — regex for processes to prefer not killing (inverted syntax: kill processes that don\u0026rsquo;t match these first) --avoid — processes to never kill (systemd, sshd — the things you need to recover) Enable the daemon:\n1 systemctl enable --now earlyoom Test it with a memory stress:\n1 2 apt install stress-ng stress-ng --vm 2 --vm-bytes 80% -t 30 Watch earlyoom in action:\n1 journalctl -u earlyoom -f Expected output:\nearlyoom[XXX]: mem avail: 387 of 7855 MiB ( 4.93%), swap free: 456 of 2048 MiB (22.27%) earlyoom[XXX]: sending SIGTERM to process 12345 (stress-ng-vm): Earlyoom sends SIGTERM first (graceful shutdown), waits 10 seconds, then SIGKILL if the process is still alive. This is far more civilized than the kernel OOM killer\u0026rsquo;s instant SIGKILL.\nSetting oom_score_adj for Critical Services The kernel calculates an oom_score for every process based on rss size + (total_vm / 2) × (cpu_time / total_cpu_time). You can adjust this with oom_score_adj:\nValue Effect -1000 OOM_DISABLE — process is invisible to OOM killer -500 Very unlikely to be killed -200 Less likely 0 Default +500 More likely +1000 Highest priority target Docker passes oom-score-adj per container:\n1 2 3 4 5 docker run -d \\ --name postgres \\ --memory=\u0026#34;512m\u0026#34; \\ --oom-score-adj=-800 \\ postgres:16-alpine Docker Compose doesn\u0026rsquo;t have native oom_score_adj support in v3, but you can work around it with wrapper scripts or by using docker run directly for critical services. Alternatively, set it post-start with a systemd timer:\n1 2 3 4 5 6 #!/bin/bash # /usr/local/bin/set-oom-scores.sh for name in postgres redis prometheus nginx; do pid=$(docker inspect -f \u0026#39;{{.State.Pid}}\u0026#39; \u0026#34;$name\u0026#34; 2\u0026gt;/dev/null) [ -n \u0026#34;$pid\u0026#34; ] \u0026amp;\u0026amp; echo -500 \u0026gt; /proc/$pid/oom_score_adj done Make it executable and run on boot via systemd or cron:\n1 2 chmod +x /usr/local/bin/set-oom-scores.sh (crontab -l 2\u0026gt;/dev/null; echo \u0026#34;@reboot /usr/local/bin/set-oom-scores.sh\u0026#34;) | crontab - Recommended oom_score_adj values for a typical homelab:\nDatabases (Postgres, MariaDB, Redis): -500 to -800 Web/Reverse proxies (Nginx, Traefik, Caddy): -200 to -400 Monitoring (Prometheus, Grafana, Loki): 0 (neutral) Batch/Transient (build jobs, backup scripts): +250 to +500 System critical (sshd, systemd-journald, Docker daemon): Already protected by systemd service configuration Swap Configuration: To Have or Not to Have The Docker documentation recommends disabling swap entirely for production Docker hosts. The reasoning: when a host swaps, performance tanks for all containers, and the OOM killer may trigger too late to be useful.\nFor homelab hosts, a middle ground works better:\n1. Low swappiness — Set swappiness to 1 or 10 so swap is used only as emergency buffer:\n1 2 sysctl vm.swappiness=10 echo \u0026#34;vm.swappiness=10\u0026#34; \u0026gt; /etc/sysctl.d/99-swap.conf 2. Disable swap per-container — Prevent individual containers from swapping by setting --memory-swap equal to --memory as shown earlier. This keeps the host-level swap available for non-container processes.\n3. ZRAM as compressed swap — For hosts with limited RAM (8-16 GB), compressed swap in RAM is a net win:\n1 2 3 4 5 modprobe zram echo lz4 \u0026gt; /sys/block/zram0/comp_algorithm echo 2G \u0026gt; /sys/block/zram0/disksize mkswap /dev/zram0 swapon /dev/zram0 Persist via /etc/systemd/zram-generator.conf:\n1 2 3 [zram0] zram-size = ram / 4 compression-algorithm = lz4 Install systemd-zram-generator on Ubuntu 24.04+:\n1 2 apt install systemd-zram-generator systemctl enable --now systemd-zram-generator ZRAM compresses idle process memory pages, effectively doubling usable memory for burst workloads without the latency penalty of disk-backed swap.\nKernel and Sysctl Memory Hardening The kernel\u0026rsquo;s virtual memory subsystem has several knobs that affect OOM behavior. These settings harden the host against runaway memory consumption:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 cat \u0026gt; /etc/sysctl.d/99-docker-memory.conf \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; # Disable memory overcommit — reject allocations that would exceed RAM + swap vm.overcommit_memory = 2 vm.overcommit_ratio = 90 # Low swappiness — prefer page cache reclaim over swap vm.swappiness = 10 # Reserve 64 MB for critical kernel allocations vm.min_free_kbytes = 65536 # Reduce dentry/inode cache pressure — keep cached filesystem metadata vm.vfs_cache_pressure = 50 # Allow enough PIDs for dozens of containers kernel.pid_max = 4194304 EOF sysctl -p /etc/sysctl.d/99-docker-memory.conf vm.overcommit_memory=2 is the most impactful. When set to 2 (strict overcommit), the kernel refuses memory allocations that would exceed RAM × overcommit_ratio + swap. Applications that try to malloc more than available memory get a clean ENOMEM error instead of crashing into swap death and eventual OOM. Databases handle ENOMEM gracefully; random OOM kills do not.\nVerification — Confirm All Protections Are Active After applying everything, verify your setup:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 # Check oomd status systemctl is-active systemd-oomd \u0026amp;\u0026amp; echo \u0026#34;OOMD active\u0026#34; # Check earlyoom status systemctl is-active earlyoom \u0026amp;\u0026amp; echo \u0026#34;Earlyoom active\u0026#34; # Check oom_score_adj on critical containers for c in postgres redis prometheus nginx; do pid=$(docker inspect -f \u0026#39;{{.State.Pid}}\u0026#39; \u0026#34;$c\u0026#34; 2\u0026gt;/dev/null) [ -n \u0026#34;$pid\u0026#34; ] \u0026amp;\u0026amp; echo \u0026#34;$c: $(cat /proc/$pid/oom_score_adj)\u0026#34; done # Check memory limits on running containers docker stats --no-stream --format \u0026#34;table {{.Name}}\\t{{.MemUsage}}\\t{{.MemPerc}}\\t{{.PIDs}}\u0026#34; # Check kernel parameters sysctl vm.overcommit_memory vm.swappiness vm.min_free_kbytes # Monitor OOM killer activity dmesg | grep -i \u0026#34;oom-killer\u0026#34; | tail -5 # Check cgroup v2 memory limits cat /sys/fs/cgroup/system.slice/docker-*.scope/memory.max 2\u0026gt;/dev/null | head -5 # Check memory pressure cat /proc/pressure/memory For Prometheus monitoring, add the node_memory_oom_kills_total metric to your Grafana dashboard. Any value above zero means you need to tighten limits or add RAM.\nSummary — Five Layers of OOM Defense Layer Tool What It Does 1 Docker deploy.resources.limits.memory Caps per-container memory, triggers in-Docker OOM 2 systemd-oomd Kills entire cgroups at 60%+ memory pressure 3 earlyoom SIGTERM-first preemptive kill below 5% free memory 4 oom_score_adj Biases kernel choices toward non-critical containers 5 Kernel sysctls Prevents overcommit, reserves memory, limits swap eagerness A Docker homelab host without memory limits is one stress-ng away from an unrecoverable freeze. Apply these configurations once, and your Postgres databases, Prometheus instances, and reverse proxies stay running even when a container goes rogue.\nThe kernel OOM killer is a last resort, not a memory management strategy.\n","permalink":"https://blog.gntech.me/posts/2026-06-06-linux-oom-management-docker-homelab/","summary":"Prevent OOM killer crashes on your Docker homelab host. Configure systemd OOMD, set proper oom_score_adj values, enable earlyoom, and apply cgroups v2 memory limits for reliable container operation.","title":"Linux OOM Management and System Stability for Docker Homelab Hosts"},{"content":"Why Multi-Architecture Docker Images Matter in Your Homelab Most homelabs start as a single x86_64 server. A Dell PowerEdge, a custom Ryzen build, a repurposed office PC. Then the Raspberry Pi shows up. Then an Orange Pi for a low-power always-on service. Then a Mac Mini M-series running containers. Suddenly you have a mixed-architecture environment and every second docker pull downloads the wrong CPU instruction set.\nWhen you pull nginx:alpine on an arm64 machine without a manifest-aware process, Docker either:\nPulls the correct arm64 variant automatically (if the image has a manifest list), or Pulls amd64 and runs it through QEMU user-mode emulation — silently, and 3–5x slower. The fix is to stop relying on upstream manifests and build your own multi-architecture images. Docker Buildx wraps BuildKit with multi-platform support, letting you produce a single tagged image that contains native layers for every architecture you target.\nHow Docker Multi-Architecture Images Work Under the hood, a multi-arch image is an OCI image index (formerly called a manifest list). It contains references to the actual manifest for each platform:\n1 2 3 4 5 6 7 8 9 10 11 12 13 { \u0026#34;schemaVersion\u0026#34;: 2, \u0026#34;manifests\u0026#34;: [ { \u0026#34;platform\u0026#34;: { \u0026#34;os\u0026#34;: \u0026#34;linux\u0026#34;, \u0026#34;architecture\u0026#34;: \u0026#34;amd64\u0026#34; }, \u0026#34;digest\u0026#34;: \u0026#34;sha256:abc123...\u0026#34; }, { \u0026#34;platform\u0026#34;: { \u0026#34;os\u0026#34;: \u0026#34;linux\u0026#34;, \u0026#34;architecture\u0026#34;: \u0026#34;arm64\u0026#34;, \u0026#34;variant\u0026#34;: \u0026#34;v8\u0026#34; }, \u0026#34;digest\u0026#34;: \u0026#34;sha256:def456...\u0026#34; } ] } When a Docker client pulls the image, it reads the index, matches its own architecture, and downloads only the relevant manifest and layers. No emulation overhead, no wasted bandwidth.\nPrerequisites Docker Engine 24.0 or later (BuildKit is built in) A registry to push to (Docker Hub, GitHub Container Registry, or a self-hosted registry with MinIO) Linux kernel with binfmt_misc support (all modern kernels) Step 1 — Enable Cross-Platform Emulation with QEMU Docker Buildx uses QEMU user-mode emulation to build arm64 binaries on an amd64 host and vice versa. First, register the binfmt handlers:\n1 docker run --privileged --rm tonistiigi/binfmt --install all This installs QEMU static binaries and registers them with the kernel\u0026rsquo;s binfmt_misc subsystem. Verify the builders are registered:\n1 ls -1 /proc/sys/fs/binfmt_misc/ | grep qemu You should see entries like qemu-aarch64, qemu-x86_64, and others.\nStep 2 — Create a Multi-Architecture Docker Builder Buildx ships with a default docker driver that uses the local Docker Engine — but it does not support multi-platform builds. You need the docker-container driver, which runs BuildKit in its own container:\n1 2 3 4 docker buildx create \\ --name multiarch \\ --driver docker-container \\ --bootstrap Verify the builder supports your target platforms:\n1 docker buildx ls The output should list linux/amd64 and linux/arm64 under the multiarch builder. If arm64 is missing, re-run the binfmt step and restart the builder:\n1 2 docker buildx rm multiarch # repeat create step above Set this builder as the default for your shell:\n1 docker buildx use multiarch Step 3 — Write a Cross-Platform Dockerfile The key to multi-arch Dockerfiles is the built-in BUILDPLATFORM and TARGETPLATFORM build arguments. BuildKit sets these automatically based on the target architecture:\n1 2 3 4 5 6 7 8 9 10 # syntax=docker/dockerfile:1 FROM --platform=$BUILDPLATFORM golang:1.23-alpine AS builder ARG TARGETOS TARGETARCH WORKDIR /src COPY . . RUN GOOS=$TARGETOS GOARCH=$TARGETARCH go build -o /app . FROM --platform=$TARGETPLATFORM alpine:3.20 COPY --from=builder /app /app ENTRYPOINT [\u0026#34;/app\u0026#34;] For interpreted languages like Python or Node.js, the same image works on both architectures — you only need to ensure the base image is multi-arch aware:\n1 2 3 4 5 6 FROM --platform=$TARGETPLATFORM node:22-alpine WORKDIR /app COPY package*.json . RUN npm ci --omit=dev COPY . . CMD [\u0026#34;node\u0026#34;, \u0026#34;server.js\u0026#34;] Most official Docker images (alpine, node, python, nginx, postgres) are already multi-arch. The --platform=$TARGETPLATFORM ensures BuildKit resolves the correct variant during the build.\nStep 4 — Build and Push Multi-Arch Images The simplest approach is a single docker buildx build command:\n1 2 3 4 5 docker buildx build \\ --platform linux/amd64,linux/arm64 \\ --tag ghcr.io/yourname/myapp:latest \\ --push \\ . Important: --push is required when building for multiple platforms. The --load flag only supports a single architecture — loading a multi-platform result into the local Docker image store is not supported.\nFor projects with multiple images, use a bake file (docker-bake.hcl):\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 group \u0026#34;default\u0026#34; { targets = [\u0026#34;app\u0026#34;, \u0026#34;web\u0026#34;] } target \u0026#34;app\u0026#34; { dockerfile = \u0026#34;app/Dockerfile\u0026#34; context = \u0026#34;app\u0026#34; tags = [\u0026#34;ghcr.io/yourname/app:latest\u0026#34;] platforms = [\u0026#34;linux/amd64\u0026#34;, \u0026#34;linux/arm64\u0026#34;] } target \u0026#34;web\u0026#34; { dockerfile = \u0026#34;web/Dockerfile\u0026#34; context = \u0026#34;web\u0026#34; tags = [\u0026#34;ghcr.io/yourname/web:latest\u0026#34;] platforms = [\u0026#34;linux/amd64\u0026#34;, \u0026#34;linux/arm64\u0026#34;] } Build and push everything in one command:\n1 docker buildx bake --push Step 5 — Verify the Multi-Arch Manifest After pushing, inspect the image manifest to confirm both architectures are present:\n1 docker buildx imagetools inspect ghcr.io/yourname/myapp:latest You should see output listing both platforms:\nName: ghcr.io/yourname/myapp:latest MediaType: application/vnd.oci.image.index.v1+json Manifests: linux/amd64 @ sha256:abc... linux/arm64 @ sha256:def... Test the arm64 pull on a Raspberry Pi or any ARM machine:\n1 2 3 docker pull ghcr.io/yourname/myapp:latest docker inspect --format \u0026#39;{{.Architecture}}\u0026#39; ghcr.io/yourname/myapp:latest # Output: arm64 Integrating Multi-Arch Builds with Homelab CI/CD If you run Gitea with Actions, add a workflow that builds and pushes multi-arch images on every tag:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 # .gitea/workflows/build.yaml name: Build and Push on: push: tags: [\u0026#39;v*\u0026#39;] jobs: multiarch: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Set up QEMU run: docker run --privileged --rm tonistiigi/binfmt --install all - name: Set up Docker Buildx uses: docker/setup-buildx-action@v3 - name: Login to registry uses: docker/login-action@v3 with: registry: ghcr.io username: ${{ secrets.REGISTRY_USER }} password: ${{ secrets.REGISTRY_TOKEN }} - name: Build and push uses: docker/bake-action@v5 with: push: true files: docker-bake.hcl The same workflow works for GitHub Actions, Woodpecker, or any Drone-compatible runner.\nCommon Pitfalls QEMU crashes on certain Go crypto instructions. Some Go packages (especially golang.org/x/crypto with assembly optimizations) use CPU-specific instructions that QEMU user-mode cannot emulate. Workaround: build with CGO_ENABLED=0 or use the -tags=noasm build tag.\nBuildKit cache mounts break across architectures. Cache mounts (--mount=type=cache) are not shared between different platform builds. Each architecture gets its own separate cache directory.\n--load only stores one platform. If you run docker buildx build --platform linux/amd64,linux/arm64 --load, BuildKit silently builds only the native platform (your host architecture). Always use --push for multi-platform builds.\nEmulated builds are slow. Building arm64 on amd64 via QEMU is 3–5x slower than a native arm64 build. For production workloads, use native ARM runners or cross-compilation instead of emulation.\nSummary Docker Buildx turns your single-architecture homelab server into a multi-architecture build station. With one QEMU container, one builder, and one Dockerfile, you can produce images that run natively on everything from an x86_64 Proxmox host to an ARM-based Raspberry Pi cluster.\nThe workflow is consistent whether you push to Docker Hub, GitHub Container Registry, or your own self-hosted MinIO-backed registry. Combined with a Gitea Actions workflow, every tagged commit produces verified multi-arch images with zero manual effort.\n","permalink":"https://blog.gntech.me/posts/2026-06-06-docker-buildx-multi-architecture-images-homelab/","summary":"Run Docker Buildx on a single Linux host to produce amd64 plus arm64 images. Configure QEMU binfmt emulation, build with buildx bake, and push multi-manifest images to any registry.","title":"Multi-Arch Docker Images with Buildx for Homelab"},{"content":"Why GPU Passthrough Matters for Your Homelab Software transcoding looks fine on paper — until someone in the house starts streaming a 4K HEVC video to their phone while a second stream converts a Blu-ray rip to H.264 for an older TV. Your CPU pegs at 100%, the Jellyfin Web UI lags, and transcoding queues pile up.\nIntel QuickSync changes the equation. A modern Intel iGPU (12th gen or newer) handles HEVC and AV1 encode/decode in dedicated silicon, drawing a fraction of the power the CPU would burn on software transcoding. A single Ice Lake or Tiger Lake iGPU can push 8–10 concurrent 1080p transcodes or 3–4 4K HDR streams with tone mapping, all at under 20W.\nBeyond Jellyfin, the pass-through GPU is useful for:\nFrigate AI object detection with OpenVINO backend Ollama inference acceleration (Intel OpenVINO or SYCL) HandBrake CLI batch transcoding Parsec/Moonlight game streaming from a headless VM If you run a single Proxmox host with Docker containers in a VM (or your services live in an LXC), passing the iGPU through is the difference between a responsive media server and one that chokes under load.\nPre-Flight Checklist Before touching any configs, verify your hardware supports passthrough.\nBIOS settings to enable:\nSetting Location VT-d (Intel) / IOMMU (AMD) Advanced → CPU Configuration Above 4G Decoding / Resizable BAR Advanced → PCI Subsystem Primary Video / Init Display (if selecting a PCIe GPU) Advanced → Graphics CPU requirements:\nIntel 8th gen (Coffee Lake) or newer for QuickSync H.265/HEVC Intel 11th gen (Tiger Lake) or newer for AV1 decode Intel 12th gen (Alder Lake) or newer for AV1 encode Essential backup access:\n⚠️ Do not pass through the only display output on your board without an alternative console. If the VM fails to boot or the GPU gets stuck in an unclean state, you lose all video output. SSH into Proxmox or use IPMI to recover.\nStep 1 — Enable IOMMU and Isolate the GPU Edit /etc/default/grub on the Proxmox host:\n1 GRUB_CMDLINE_LINUX_DEFAULT=\u0026#34;quiet intel_iommu=on iommu=pt pcie_acs_override=downstream video=efifb:off\u0026#34; The flags:\nintel_iommu=on — enables the IOMMU on Intel hardware iommu=pt — passthrough mode: devices not assigned to VMs are not affected by IOMMU pcie_acs_override=downstream — splits stubborn IOMMU groups on consumer boards (use only when your GPU shares a group with other devices) video=efifb:off — unbinds the EFI framebuffer so the GPU is free for passthrough Update GRUB and reboot:\n1 2 update-grub reboot After reboot, load the VFIO kernel modules and bind the GPU to vfio-pci at boot. Create /etc/modprobe.d/vfio.conf:\n1 options vfio-pci ids=8086:4690,8086:4680 Replace the PCI IDs with your GPU and its HDMI/DP audio controller. Find them with lspci -nn:\n1 2 3 4 lspci -nn | grep -i vga # Example output: 00:02.0 VGA compatible controller [0300]: Intel Corporation Raptor Lake-S GT1 [UHD Graphics 770] [8086:4690] lspci -nn | grep -i audio # 00:1f.3 Audio device [0403]: Intel Corporation Raptor Lake-P/U/H cAVS [8086:4680] Add the VFIO modules to the initramfs. Edit /etc/initramfs-tools/modules:\n1 2 3 4 vfio vfio_iommu_type1 vfio_pci vfio_virqfd Rebuild initramfs and reboot again:\n1 2 update-initramfs -u -k all reboot After the second reboot, confirm the GPU is bound to vfio-pci:\n1 2 lspci -nnk -s 00:02.0 # Kernel driver in use: vfio-pci Step 2 — Verify IOMMU Groups Run this script on the Proxmox host to check that the GPU is in its own IOMMU group:\n1 2 3 4 5 6 7 8 #!/bin/bash shopt -s nullglob for g in /sys/kernel/iommu_groups/*; do echo \u0026#34;IOMMU Group ${g##*/}:\u0026#34; for d in \u0026#34;$g\u0026#34;/devices/*; do echo -e \u0026#34;\\t$(lspci -nns ${d##*/})\u0026#34; done done You want the VGA device and its audio function in the same group together, without the NVMe controller or any network card. If the group contains unrelated devices, add pcie_acs_override=downstream to the kernel cmdline (already included above) or consider a different PCIe slot.\nStep 3 — Create and Configure the Proxmox VM Create the VM with the correct machine type and firmware:\n1 2 3 4 5 6 7 8 9 10 qm create 200 \\ --name jellyfin \\ --memory 4096 \\ --cores 4 \\ --cpu host \\ --machine q35 \\ --bios ovmf \\ --efidisk0 local-lvm:1 \\ --scsihw virtio-scsi-single \\ --net0 virtio,bridge=vmbr0 Attach a boot disk and install an OS (Ubuntu 24.04 LTS or Debian 12 work well). Then add the GPU:\n1 2 3 qm set 200 \\ --hostpci0 0000:00:02.0,legacy-igd=1,x-vga=on,pcie=1 \\ --vga none Key hostpci0 flags:\nlegacy-igd=1 — enables Intel GVT-g legacy mode for iGPU boot display emulation (prevents boot hangs) x-vga=on — marks this as the primary GPU in the VM; critical for drivers to detect the device pcie=1 — presents the device as PCIe instead of PCI (required for q35 machine type) Set --vga none since the GPU handles display output now. If you still want a console via noVNC, use --vga virtio instead and enable the SPICE display.\nStep 4 — Install Drivers in the Guest VM Inside the VM, install the Intel media drivers:\n1 2 3 4 5 6 # Ubuntu/Debian apt update apt install intel-media-va-driver-non-free vainfo intel-gpu-tools # Verify vainfo | grep -i intel Expected output shows the driver version and supported profiles:\n1 2 3 4 5 6 7 8 9 10 11 libva info: VA-API version 1.22.0 libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so libva info: Found init function __vaDriverInit_1_22 vainfo: VA-API: Intel iHD driver for Intel(R) Gen Graphics - 24.3.4 vainfo: Driver version: 24.3.4 VAProfileH264High : VAEntrypointVLD VAProfileHEVCMain : VAEntrypointVLD VAProfileHEVCMain10 : VAEntrypointVLD VAProfileVP9Profile0 : VAEntrypointVLD VAProfileVP9Profile2 : VAEntrypointVLD VAProfileAV1Profile0 : VAEntrypointVLD If nothing shows up and ls /dev/dri only lists dri/card0, the vfio-pci binding on the host is likely correct but the guest kernel needs the i915 module loaded:\n1 2 modprobe i915 echo \u0026#34;i915\u0026#34; \u0026gt;\u0026gt; /etc/modules Step 5 — Configure Jellyfin Hardware Transcoding in Docker Deploy Jellyfin with Docker in the guest VM. Create a docker-compose.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 services: jellyfin: image: jellyfin/jellyfin:latest container_name: jellyfin environment: - PUID=1000 - PGID=1000 - TZ=America/Santo_Domingo volumes: - ./config:/config - ./cache:/cache - /media:/media:ro devices: - /dev/dri:/dev/dri ports: - \u0026#34;8096:8096\u0026#34; - \u0026#34;8920:8920\u0026#34; restart: unless-stopped The key line is devices: - /dev/dri:/dev/dri. This passes the host (guest VM\u0026rsquo;s) DRI device nodes into the container.\nPermissions fix: If Jellyfin can\u0026rsquo;t access the GPU, add the jellyfin user to the render and video groups inside the container, or set the container to run in privileged mode temporarily for debugging:\n1 2 3 4 5 6 7 8 # Check ownership ls -la /dev/dri/ # crw-rw---- 1 root video 226, 0 ... # crw-rw---- 1 root render 226, 128 ... # Add container user to groups docker exec jellyfin usermod -aG video jellyfin docker exec jellyfin usermod -aG render jellyfin Inside the Jellyfin admin panel:\nGo to Dashboard → Playback → Transcoding Set Hardware acceleration to VA-API Select Intel iHD driver (or Intel i965 for older hardware) Enable Enable hardware encoding and Allow encoding in HEVC format Toggle Enable VPP tone mapping (HDR→SDR, uses iGPU fixed-function hardware) Save and test by playing a video that requires transcoding (e.g., play a high-bitrate 4K file on a browser tab) Performance Tuning and Troubleshooting Common Issues \u0026ldquo;Failed to initialise VAAPI connection\u0026rdquo;\n1 2 3 4 5 # Inside the guest VM, not the container ls /dev/dri # Should show renderD128 and card0 dmesg | grep i915 # Check for driver errors If the i915 driver isn\u0026rsquo;t loaded: modprobe i915.\n\u0026ldquo;Render node not found\u0026rdquo; — the Jellyfin container user doesn\u0026rsquo;t have permission:\n1 2 3 # Add container user to render group docker exec jellyfin usermod -aG video jellyfin docker exec jellyfin usermod -aG render jellyfin Or run the container with group_add:\n1 2 3 4 5 6 services: jellyfin: # ... group_add: - \u0026#34;44\u0026#34; # video - \u0026#34;107\u0026#34; # render Transcoding is slow or pixelated — the VM may not be passing the GPU topology correctly:\n1 2 # Check inside the guest intel_gpu_top This shows real-time GPU utilization, decode/encode engine load, and memory bandwidth. If the GPU engines stay at 0% during playback, hardware acceleration isn\u0026rsquo;t engaged.\nTuning Tips CPU pinning isolates physical cores for the VM, reducing cache thrashing:\n1 2 3 4 qm set 200 --numa 1 # Then pin cores: qm set 200 --cpulimit 4 --cpuunits 1024 # Or set the CPU pinning in /etc/pve/qemu-server/200.conf # cpu: 0-3 Hugepages reduce TLB pressure for memory-intensive transcoding:\n1 2 3 4 # On the Proxmox host echo \u0026#34;vm.nr_hugepages=2048\u0026#34; \u0026gt;\u0026gt; /etc/sysctl.conf sysctl -p qm set 200 --hugepages any When to Use an Intel Arc A380 Instead If your CPU lacks QuickSync or you need more transcode throughput, the Intel Arc A380 is a drop-in replacement that uses the same passthrough technique:\nFull AV1 hardware encode/decode (10-bit, HDR metadata passthrough) Handles 6–8 simultaneous 4K HDR tone-mapped transcodes Idle power draw ~8W, load ~35W Available for ~$100–130 USD on secondary market The only difference in setup is the PCIe slot and device ID. Everything else — IOMMU verification, vfio-pci binding, VM config — is identical. The Arc uses the same i915 kernel driver in the guest.\nConclusion GPU passthrough on Proxmox transforms a homelab media server from a CPU-bound bottleneck into an efficient, quiet transcoding powerhouse. The upfront effort — enabling IOMMU, binding vfio-pci, and setting the correct VM flags — pays off every time someone streams on a phone, a tablet, or an older TV that doesn\u0026rsquo;t support HEVC.\nThe same GPU can also accelerate Frigate object detection, offload Ollama inference, or batch-transcode your media library with HandBrake CLI. One pass-through device, many workloads.\nSet yours up and let me know how it goes in the comments. Got an Arc card in your homelab? I\u0026rsquo;d love to hear about your experience with AV1 transcoding.\n","permalink":"https://blog.gntech.me/posts/2026-06-06-proxmox-intel-igpu-passthrough-jellyfin/","summary":"Passthrough an Intel iGPU through Proxmox to a VM for Jellyfin hardware transcoding. Configure IOMMU, bind vfio-pci, verify groups, and tune performance.","title":"Proxmox Intel iGPU Passthrough for Jellyfin Hardware Transcoding"},{"content":"SMART monitoring can reveal storage problems, but a passing health result does not guarantee a drive will remain available. Keep independent backups and verify recovery regardless of dashboard color.\nVerification status Area Evidence and limits Technical review Upstream smartmontools, Scrutiny and Prometheus references reviewed on October 31, 2026. Static checks Bash syntax and embedded Compose YAML checked. Runtime No physical disk, smartd notification, Scrutiny deployment or exporter scrape tested for this revision. Scope Linux inspection workflow and a single-device SATA/SAS Scrutiny example; hardware support varies. 1. Identify the actual device On Debian or Ubuntu:\n1 2 3 4 sudo apt update sudo apt install smartmontools sudo smartctl --scan lsblk -o NAME,MODEL,SERIAL,SIZE,TYPE,MOUNTPOINTS Match model and serial number to the physical device. USB bridges, RAID controllers, virtual disks and NVMe controllers may need different device types or passthrough. A guest VM often sees a virtual disk rather than the physical drive\u0026rsquo;s SMART data.\nUse the detected device and documented -d option where needed. Do not turn a missing device or failed command into a healthy result.\n2. Preserve output and exit status Run in Bash, selecting the actual device:\n1 2 3 4 5 6 7 read -r -p \u0026#39;Device path confirmed by scan: \u0026#39; smart_device if sudo smartctl -x \u0026#34;$smart_device\u0026#34;; then smart_status=0 else smart_status=$? fi printf \u0026#39;smartctl exit-status bitmask: %s\\n\u0026#39; \u0026#34;$smart_status\u0026#34; smartctl\u0026rsquo;s return value is a bitmask describing command problems and reported device conditions. It is not a simple “zero means safe forever, nonzero means replace immediately” verdict. Read the complete output and decode the bits using the smartctl manual.\nTrack trends, error/self-test logs and workload symptoms alongside the health summary. Attribute names and normalization are vendor-specific. Do not assume one SSD wear attribute always means percent remaining, or apply one temperature limit to every HDD, SSD and NVMe model.\nFor a degraded or unreadable drive, prioritize recoverable data and investigate the storage path before adding demanding tests.\n3. Separate self-tests from notification tests A supported short or long self-test checks the device; it does not necessarily generate a notification when successful. Consult smartctl\u0026rsquo;s reported completion estimate and read the test log afterwards. Support and duration vary by hardware.\nsmartd polls devices and reports configured conditions. It cannot guarantee an instant warning before every failure. Notification delivery also depends on its mail or script integration.\nPreserve the existing smartd configuration before adapting it. Start with the packaged sample and installed manual:\n1 2 man smartd.conf systemctl list-unit-files \u0026#39;smart*\u0026#39; Choose the actual service unit provided by the distribution. Review existing monitoring and schedules before adding another daemon or test schedule.\nFor email, configure and test the delivery route, then use smartd\u0026rsquo;s documented test-notification option with a real recipient. Verify receipt and daemon logs. Keep configuration directives on valid lines; inline comments after a continuation backslash can break the intended configuration.\nCustom notification scripts consume smartd\u0026rsquo;s documented SMARTD_* environment variables. Do not assume positional arguments contain device and event details. Test both successful delivery and a failed delivery route before depending on alerts.\n4. Add a single-device Scrutiny dashboard Use Linux Docker Engine on a host that can read the physical device. Choose a reviewed omnibus image tag or digest from Scrutiny. The following example is for a supported SATA/SAS device; it is not an NVMe/controller recipe.\nRaw device access and SYS_RAWIO are sensitive privileges. Limit mappings to reviewed devices. For NVMe, consult upstream\u0026rsquo;s additional capability requirements rather than granting privileged mode indiscriminately.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 scrutiny_lab_dir=$(mktemp -d -t scrutiny-lab.XXXXXX) cd \u0026#34;$scrutiny_lab_dir\u0026#34; || exit 1 read -r -p \u0026#39;Reviewed Scrutiny omnibus image: \u0026#39; SCRUTINY_IMAGE read -r -p \u0026#39;Confirmed SATA/SAS device path: \u0026#39; SMART_DEVICE export SCRUTINY_IMAGE SMART_DEVICE test -b \u0026#34;$SMART_DEVICE\u0026#34; || exit 1 cat \u0026gt; compose.yaml \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; services: scrutiny: image: ${SCRUTINY_IMAGE:?Set a reviewed omnibus image} cap_add: - SYS_RAWIO devices: - \u0026#34;${SMART_DEVICE:?Set the physical device}:${SMART_DEVICE:?Set the physical device}\u0026#34; volumes: - /run/udev:/run/udev:ro - scrutiny_config:/opt/scrutiny/config - scrutiny_history:/opt/scrutiny/influxdb ports: - \u0026#34;127.0.0.1:18081:8080\u0026#34; logging: driver: json-file options: max-size: \u0026#34;5m\u0026#34; max-file: \u0026#34;2\u0026#34; volumes: scrutiny_config: scrutiny_history: EOF docker compose config --quiet docker compose up -d docker compose logs --tail=80 scrutiny The history volume targets the omnibus InfluxDB directory, not the web application directory. Review the upstream omnibus example for the selected release. The UI is loopback-only; use a protected access method for remote viewing.\n5. Verify collection and freshness Trigger a collection using the documented binary, then inspect logs and the UI:\n1 2 docker compose exec scrutiny /opt/scrutiny/bin/scrutiny-collector-metrics run docker compose logs --tail=80 scrutiny Open http://127.0.0.1:18081 on the Docker host. Confirm the expected model/serial, collection timestamp, and returned attributes. A running container or green dashboard with stale data is insufficient evidence.\nCheck the actual collector schedule for the selected image; do not assume a thirty-minute interval. Web/API and collector configuration are separate files. See the upstream collector example before adding device-type overrides or remote collectors.\nTo stop this project while retaining configuration and history:\n1 docker compose down 6. Use the correct Prometheus exporter node_exporter\u0026rsquo;s diskstats collector reports operating-system disk I/O statistics, not SMART attributes. The old \u0026ndash;collector.smartmon and \u0026ndash;collector.smartctl flags are not valid built-in collectors.\nUse a dedicated smartctl_exporter, or a supported external collector writing metrics for node_exporter\u0026rsquo;s textfile collector. Inspect the actual /metrics output before choosing metric names, labels or alert rules.\nAlert on failed collection and stale data as well as the device conditions you care about. Test the alert route independently from device self-tests.\n7. Turn observations into a recovery plan Record device identity, firmware, host, collection freshness, errors, test history, and changes over time. Compare warnings with vendor documentation and storage-system logs.\nDo not use fixed age, reallocated-sector, wear, or temperature rules as universal replacement guarantees. Investigate changes and preserve data; some failures occur without useful SMART warning.\nCoordinate tests with scrubs and workloads. Monitoring, filesystem integrity checks, redundancy and backups cover different failures. No combination guarantees advance warning of every disk loss.\nContinue reading ZFS snapshots and recovery — Understand storage recovery boundaries. PBS setup and restore test — Verify guest recovery. Restic backup workflow — Test an independent file backup. ","permalink":"https://blog.gntech.me/posts/2026-06-05-linux-disk-health-monitoring-smartmontools-scrutiny/","summary":"Collect disk-health evidence, test notifications, and keep monitoring separate from backup and recovery.","title":"Linux Disk Health Monitoring with smartmontools and Scrutiny"},{"content":"A hardened image can reduce included components and improve supply-chain evidence. It still needs to meet the application\u0026rsquo;s runtime, storage, and recovery requirements. Treat migration as an application change rather than a search-and-replace of registry names.\nVerification status Area Evidence and limits Source review Docker\u0026rsquo;s migration documentation reviewed on October 31, 2026. Measurements No digest-pinned CVE comparison or application migration was executed in this audit. Correction Earlier fixed vulnerability counts, percentage reductions, guessed image paths, and universal drop-in/non-root claims are removed. Select an image from the current catalog Start with Docker\u0026rsquo;s migration documentation and follow the catalog and access instructions for the image you need. Record its exact registry, repository, tag, digest, architecture, edition, and license/access requirements. Do not infer an image path by prepending dockerhardened/ to a Docker Official Image name.\nCheck the image-specific guide for its user ID, ports, filesystem permissions, entrypoint, environment variables, data directory, and available tools. A non-root container is not necessarily using a rootless Docker daemon. A distroless variant may omit the shell or utilities used by your existing health check and maintenance commands.\nCompare evidence for exact digests Use your selected scanner\u0026rsquo;s documented installation method. Scan the deployed digest and the candidate digest with the same scanner version, vulnerability database, architecture, and severity policy. Save the timestamp, SBOM, package findings, and any vendor vulnerability statements.\nA CVE count changes as images and vulnerability databases change. It is not evidence that every finding is exploitable or that an image with fewer findings is risk-free. Check the applicability and remediation of individual findings. Provenance and signatures support checks of origin and build evidence; they do not prove application correctness or absence of vulnerabilities.\nRehearse the application migration Export the current Compose configuration and record image digests, required secrets, persistent data, and a rollback target. Make an application-consistent backup. For PostgreSQL, follow the tested dump and isolated recovery lab. Restore into an isolated test deployment with fresh volumes and the candidate image. Review ownership and permissions before changing them; do not apply guessed recursive chown operations to the original data. Adapt health checks to tools the image actually supplies. Test login, writes, reads, migrations, extensions, integrations, and recovery. Plan a maintenance window and a tested rollback. Database migrations may prevent an older application image from using newly upgraded data. Keep the original image and recovery materials until the new deployment and rollback exercise have passed. An automatic image updater is not a substitute for this validation.\nRecord a useful result Publish the exact versions/digests, configuration changes, observed behavior, and remaining limits. Do not publish a generic “95% fewer CVEs” result without the scan evidence and scope needed to reproduce it.\nContinue with image security and distroless design and container security hardening.\n","permalink":"https://blog.gntech.me/posts/2026-06-05-docker-hardened-images-homelab-migration/","summary":"Compare exact image digests, inspect migration requirements, and test application recovery before adopting a hardened image.","title":"Docker Hardened Images — Verify Compatibility Before Migration"},{"content":"Why Split-Tunnel VPN in Docker A VPN on your entire Docker host creates friction. Plex and Jellyfin become unreachable from your local network. Traefik ACME challenges fail. Remote SSH access gets complicated. DNS resolution slows down through the VPN provider\u0026rsquo;s resolvers. And every container — not just the ones that need privacy — pays the latency tax.\nThe solution is split-tunnel VPN routing: a dedicated VPN gateway container that only specific containers use, while the rest of your stack stays on your local network with full throughput and direct access.\nGluetun (github.com/qdm12/gluetun) is a lightweight Go-based Docker container that connects to your VPN provider over WireGuard or OpenVPN and exposes itself as a network gateway for other containers. It includes a built-in firewall that acts as a kill switch — when the VPN drops, all traffic from attached containers is blocked immediately. No leaks.\nHow Gluetun Works as a Docker Network Gateway Gluetun does not run as a sidecar container that you exec into. It owns its own network namespace, and you attach other containers to it using Docker\u0026rsquo;s network_mode: \u0026quot;service:gluetun\u0026quot; directive. Traffic from those containers flows through Gluetun\u0026rsquo;s interface — encrypted and routed through your VPN provider.\nThe data flow looks like this:\nqBittorrent → [network_mode: service:gluetun] → Gluetun → WireGuard tunnel → VPN provider → Internet The rest of your Docker stack — Traefik, Grafana, Homepage — stays on the docker bridge or your custom network and connects directly. No latency, no routing conflicts.\nGluetun supports over 30 VPN providers natively including Mullvad, ProtonVPN, NordVPN, IVPN, Windscribe, AirVPN, Surfshark, Private Internet Access, CyberGhost, and VyprVPN. You can also configure a custom WireGuard or OpenVPN configuration for any provider.\nDocker Compose Setup with WireGuard Here is a production-ready Compose file using Mullvad with WireGuard. Adjust the environment variables for your provider and credentials.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 services: gluetun: image: qmcgaw/gluetun:latest container_name: gluetun cap_add: - NET_ADMIN environment: - VPN_SERVICE_PROVIDER=mullvad - VPN_TYPE=wireguard - WIREGUARD_PRIVATE_KEY=your_private_key_here - WIREGUARD_ADDRESSES=10.64.x.x/32 - SERVER_REGIONS=us-east - TZ=America/Santo_Domingo ports: # qBittorrent WebUI — expose through Gluetun - 8080:8080/tcp # qBittorrent DHT / transfer ports - 56881:56881/udp - 56881:56881/tcp restart: unless-stopped qbittorrent: image: lscr.io/linuxserver/qbittorrent:latest container_name: qbittorrent network_mode: \u0026#34;service:gluetun\u0026#34; environment: - PUID=1000 - PGID=1000 - TZ=America/Santo_Domingo - WEBUI_PORT=8080 volumes: - ./qbittorrent/config:/config - ./qbittorrent/downloads:/downloads depends_on: gluetun: condition: service_healthy restart: unless-stopped The critical lines:\nnetwork_mode: \u0026quot;service:gluetun\u0026quot; — qBittorrent shares Gluetun\u0026rsquo;s network namespace. All its outbound traffic routes through the VPN. ports: are declared on the Gluetun service (not qBittorrent) — the network_mode directive means qBittorrent cannot bind ports directly. Only the service that owns the namespace (Gluetun) can publish ports. cap_add: NET_ADMIN — required for Gluetun to manage firewall rules and network interfaces inside the container. depends_on: gluetun: condition: service_healthy — waits for Gluetun\u0026rsquo;s health check to pass (VPN connected and stable) before starting qBittorrent. Environment Variable Reference Gluetun uses a flat environment variable scheme. The most important ones:\nVariable Purpose VPN_SERVICE_PROVIDER Your provider (mullvad, protonvpn, nordvpn, etc.) VPN_TYPE wireguard or openvpn WIREGUARD_PRIVATE_KEY Your WireGuard private key WIREGUARD_ADDRESSES Internal address assigned by provider SERVER_REGIONS Comma-separated preferred regions SERVER_COUNTRIES Alternative to SERVER_REGIONS VPN_PORT_FORWARDING Enable port forwarding (provider dependent) FIREWALL_VPN_INPUT_PORTS Ports allowed in through the firewall FIREWALL_OUTBOUND_SUBNETS Subnets accessible outside the VPN (LAN, Docker networks) Firewall — Allowing Access to Local Subnets By default, Gluetun\u0026rsquo;s kill switch blocks ALL non-VPN traffic, including connections to your local network. That means qBittorrent won\u0026rsquo;t be able to reach your SMB/NFS shares or local indexers. Add FIREWALL_OUTBOUND_SUBNETS to allow LAN traffic:\n1 2 environment: - FIREWALL_OUTBOUND_SUBNETS=10.0.20.0/24,172.17.0.0/16 This allows containers behind the VPN to reach your homelab subnet without breaking the kill switch for internet traffic.\nAttaching Multiple Containers to the Same Gateway You can attach multiple downstream containers to the same Gluetun instance. Each container uses network_mode: \u0026quot;service:gluetun\u0026quot;. Ports must be unique across all services — you cannot have two containers both exposing port 8080 through the same gateway.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 services: gluetun: # ... as above ports: - 8080:8080/tcp # qBittorrent - 9696:9696/tcp # Prowlarr - 7878:7878/tcp # Radarr prowlarr: image: lscr.io/linuxserver/prowlarr:latest container_name: prowlarr network_mode: \u0026#34;service:gluetun\u0026#34; environment: - PUID=1000 - PGID=1000 - TZ=America/Santo_Domingo volumes: - ./prowlarr/config:/config depends_on: gluetun: condition: service_healthy restart: unless-stopped radarr: image: lscr.io/linuxserver/radarr:latest container_name: radarr network_mode: \u0026#34;service:gluetun\u0026#34; environment: - PUID=1000 - PGID=1000 - TZ=America/Santo_Domingo volumes: - ./radarr/config:/config - ./qbittorrent/downloads:/downloads depends_on: gluetun: condition: service_healthy restart: unless-stopped Integration with Traefik Reverse Proxy Containers behind Gluetun\u0026rsquo;s network_mode cannot be reached through Docker provider service discovery. Traefik runs on a different network so it cannot route traffic to qBittorrent\u0026rsquo;s port 8080 via the usual Docker labels.\nThe solution: Traefik routes to Gluetun\u0026rsquo;s published ports, not to the downstream container directly.\nOn your Traefik network, add Gluetun and label it for routing:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 services: gluetun: image: qmcgaw/gluetun:latest container_name: gluetun cap_add: - NET_ADMIN networks: - traefik_network # attach to Traefik\u0026#39;s network environment: # ... VPN config same as above ports: - 8080:8080/tcp labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.qbittorrent.rule=Host(`qb.homelab.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.qbittorrent.entrypoints=https\u0026#34; - \u0026#34;traefik.http.services.qbittorrent.loadbalancer.server.port=8080\u0026#34; restart: unless-stopped The Traefik labels go on the Gluetun service, not on qBittorrent, because Gluetun publishes the port. Downstream containers are isolated from any network that Traefik can reach — only Gluetun connects both sides.\nVerifying the Kill Switch After starting the stack, verify that traffic is routing through the VPN:\n1 2 3 4 5 6 7 8 9 10 11 # Check Gluetun logs docker compose logs gluetun | grep -i \u0026#34;connection\u0026#34; # Verify qBittorrent\u0026#39;s public IP docker compose exec qbittorrent curl -s ifconfig.me # Confirm Gluetun health docker compose ps gluetun # Test DNS resolution through the VPN docker compose exec qbittorrent curl -s https://ipinfo.io/json Simulate a VPN disconnection to confirm the kill switch:\n1 2 3 4 5 6 # Restart Gluetun to simulate disconnect docker compose restart gluetun # While it reconnects, try reaching the internet from qBittorrent docker compose exec qbittorrent curl -s ifconfig.me # Should hang or fail — traffic is blocked When the kill switch works correctly, curl ifconfig.me will fail or hang while Gluetun reconnects. Once connected, it returns the VPN exit IP.\nTroubleshooting Common Issues Port binding conflicts: If two services behind the same Gluetun instance need the same port (e.g., both use :8080), you must run separate Gluetun instances with different port mappings.\nTraefik 503 errors: The Gluetun container must be on the same Docker network as Traefik. Labels go on the Gluetun service, not downstream containers. Verify the container resolves from Traefik\u0026rsquo;s network.\nDNS leaks: Gluetun uses DNS over TLS by default. If you see DNS queries bypassing the VPN, set DNS_OVER_TLS=on and DNS_ADDRESS=1.1.1.1:853 explicitly.\nWireGuard handshake failures: Check your private key and allowed IPs. Mullvad requires a specific WireGuard key generated from their account page, not just any key:\n1 2 # Generate a WireGuard key pair (does NOT work with Mullvad — use their tool) # Mullvad requires keys generated via their API or account page Routing to local services from VPN containers: The firewall blocks non-VPN subnets by default. Use FIREWALL_OUTBOUND_SUBNETS to explicitly allow specific subnets. This does not weaken the kill switch — it only opens routes to IPs you specify.\nSummary Gluetun solves the fundamental tension between privacy and accessibility in homelab Docker deployments. By routing only specific containers through a VPN gateway with a built-in kill switch, you keep Plex, Jellyfin, Traefik, and your management tools on the local network while your download stack exits through an encrypted tunnel.\nThe WireGuard setup provides excellent performance — near line-speed throughput with minimal CPU overhead compared to OpenVPN. Combined with depends_on: condition: service_healthy for ordered startup and FIREWALL_OUTBOUND_SUBNETS for LAN access, this pattern handles the full breadth of homelab VPN routing needs without the operational headache of running a VPN on the host.\n","permalink":"https://blog.gntech.me/posts/2026-06-05-gluetun-vpn-gateway-docker-wireguard/","summary":"Route specific Docker containers through a WireGuard VPN while keeping the rest of your stack on the local network. Full Gluetun Docker Compose setup with kill switch, qBittorrent integration, and Traefik compatibility.","title":"Gluetun VPN Gateway — Route Docker Containers Through WireGuard"},{"content":"Why Docker Image Security Matters in Your Homelab Your homelab runs Docker containers — maybe a lot of them. MiniO, Grafana, Nginx, PostgreSQL, Gitea, Uptime Kuma. Most of these come from public images on Docker Hub. And most of them ship with build artifacts, unused libraries, package manager caches, and shells you never need at runtime.\nThe security problem is straightforward: every unnecessary package and binary in your container image is an attack surface. When FROM ubuntu:22.04 pulls a 77 MB base image and your Dockerfile adds build dependencies that inflate it past 1 GB, you\u0026rsquo;re not just wasting disk — you\u0026rsquo;re shipping every CVE in every one of those packages to your homelab.\nCompare exact image digests with the same scanner and vulnerability database. No dated scan evidence is supplied here for a fixed percentage reduction. See the hardened-image migration guide for compatibility and evidence requirements.\nThis guide covers three layers of Docker image security you can apply in your homelab right now:\nBuild-time hardening — multi-stage Dockerfiles that separate build and runtime Image choice — distroless, slim, and hardened base images with minimal attack surface Runtime security — drop capabilities, enforce read-only filesystems, run as non-root You don\u0026rsquo;t need to rebuild every image. Runtime hardening alone reduces risk on existing containers significantly.\nMulti-Stage Builds for Smaller, Cleaner Images Multi-stage builds let you use one Dockerfile with multiple FROM statements. Build tools and dependencies live in an early stage, while the final runtime image copies only what\u0026rsquo;s needed.\nGo Application Example 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # Stage 1: build FROM golang:1.23-alpine AS builder RUN apk add --no-cache ca-certificates git WORKDIR /src COPY go.mod go.sum ./ RUN go mod download COPY . . RUN CGO_ENABLED=0 go build -ldflags=\u0026#34;-s -w\u0026#34; -o /app/my-service . # Stage 2: runtime FROM scratch COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ COPY --from=builder /app/my-service /app/my-service USER 1001:1001 ENTRYPOINT [\u0026#34;/app/my-service\u0026#34;] The scratch base is completely empty — no shell, no package manager, no utilities. The final image is exactly one binary plus CA certificates, typically 10–20 MB instead of 500 MB+.\nPython Application Example Python is trickier because you need a Python interpreter at runtime. The multi-stage approach still cuts the image drastically:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # Stage 1: install dependencies FROM python:3.12-alpine AS builder RUN python -m venv /opt/venv RUN --mount=type=cache,target=/root/.cache/pip \\ /opt/venv/bin/pip install fastapi uvicorn # Stage 2: runtime FROM python:3.12-alpine RUN addgroup -S appgroup \u0026amp;\u0026amp; adduser -S appuser -G appgroup COPY --from=builder /opt/venv /opt/venv WORKDIR /app COPY app.py . ENV PATH=/opt/venv/bin:$PATH USER appuser CMD [\u0026#34;uvicorn\u0026#34;, \u0026#34;app:app\u0026#34;, \u0026#34;--host\u0026#34;, \u0026#34;0.0.0.0\u0026#34;, \u0026#34;--port\u0026#34;, \u0026#34;8000\u0026#34;] Key improvements over a single-stage Dockerfile:\nPackage archives in /root/.cache/pip don\u0026rsquo;t persist into the final image because the RUN --mount=type=cache mount disappears after that layer The final image is based on -alpine (only ~50 MB) instead of python:3.12 full (~335 MB) The non-root user is created explicitly in the RUN layer The virtual environment is outside /root, so the non-root runtime user can traverse its path. FastAPI is an ASGI application and this example launches it with Uvicorn. Pin dependencies for a reproducible build and test the actual app.py; this audit did not execute the image.\nCache Mounts for Faster Builds When developing custom homelab Dockerfiles (custom Nginx with modules, home automation Python scripts, data processing pipelines), use cache mounts to avoid re-downloading packages:\n1 2 3 4 RUN --mount=type=cache,target=/var/cache/apt \\ apt-get update \u0026amp;\u0026amp; apt-get install -y --no-install-recommends \\ libnginx-mod-http-brotli \\ nginx-extras Choosing Secure Base Images — Distroless and Slim Variants Not every image you consume is one you build. For off-the-shelf containers, your security stance is determined by the upstream maintainer\u0026rsquo;s base image choice. Here\u0026rsquo;s how the options compare:\nAspect Full Distro Slim Distroless Docker Hardened Size 150–800 MB 30–80 MB 5–50 MB Variable Shell Yes Yes No Varies Package manager Yes Limited No No apt upgrade Manual Manual Not needed Auto-patched CVEs Scan exact digest Scan exact digest Scan exact digest Scan exact digest Debugging Easy Moderate Hard (no shell) Moderate Google\u0026rsquo;s distroless images (gcr.io/distroless/base, gcr.io/distroless/python3-debian12, gcr.io/distroless/java17-debian12) contain only your application and its runtime dependencies. There is no shell, no apt, no curl, no ls — which removes those particular utilities. Arbitrary code execution can still read accessible secrets, use network access, or call operating-system interfaces; distroless does not neutralize a compromised application.\nBuild an application image using a compatible distroless runtime; changing an application service directly to a bare language runtime does not copy its application code or dependencies. Match the language ABI, architecture, shared libraries, entrypoint, and non-root filesystem permissions. Use the distroless project documentation.\nFor Docker Hardened Images, select an actual catalog reference and review its migration guide. Do not assume nginx:hardened or python:hardened exists. The migration guide replaces the earlier guessed tags and unmeasured pull/CVE claims.\nVulnerability Scanning — Trivy in Your Docker Pipeline You can\u0026rsquo;t fix what you don\u0026rsquo;t measure. Scan your images regularly for CVEs. Trivy from Aqua Security is the go-to open-source scanner for homelabs.\nBasic Scan 1 2 3 4 5 6 7 8 # Scan a local image trivy image myapp:latest # Scan with severity filter and ignore unfixed packages trivy image --severity CRITICAL,HIGH --ignore-unfixed nginx:latest # Scan a Docker Hub image directly trivy image docker.io/library/nginx:1.27-alpine Scanning in a CI Pipeline If you use Gitea Actions with your self-hosted runner, add a scan step to catch issues before deployment:\n1 2 3 4 5 6 7 - name: Scan image for vulnerabilities run: | trivy image \\ --severity CRITICAL,HIGH \\ --ignore-unfixed \\ --exit-code 1 \\ myapp:${GITEA_COMMIT_SHA} With --ignore-unfixed, this gate fails only for matching findings with known fixes. Remove that filter or add a separate report if policy also covers unfixed findings. --exit-code 1 makes the pipeline fail on the selected findings. In a homelab context, you may want --exit-code 0 with a human review instead — few homelabs have dedicated security teams.\nDocker Scout Alternative If you prefer Docker\u0026rsquo;s native tooling:\n1 2 3 docker scout cves myapp:latest docker scout recommendations myapp:latest docker scout quickview myapp:latest Docker Scout connects with Docker Hub\u0026rsquo;s vulnerability database and can suggest base image upgrades automatically. It is included with Docker Desktop. Docker Engine users install the CLI plugin separately using Docker\u0026rsquo;s installation guidance.\nInterpreting Results Report measurements for exact digests and a dated scanner database. This guide supplies no scan evidence for fixed CVE counts by image family.\nFor each finding:\nFixed version available → rebuild with the patched base image Not fixed yet → monitor the CVE, consider temporarily switching to a different base False positive → suppress with .trivyignore: # .trivyignore CVE-2024-XXXXX # Not exploitable in our configuration Runtime Security Hardening for Docker Containers Image hardening reduces risk at build time. Runtime security controls reduce risk when the container is running. These apply to any container, including ones you pull off-the-shelf without modifications.\nRun as Non-Root Inspect the selected image\u0026rsquo;s configured user and runtime behavior; do not assume all official images run as non-root. In your own Dockerfiles:\n1 2 RUN groupadd -r appgroup \u0026amp;\u0026amp; useradd -r -g appgroup appuser USER appuser:appgroup Drop Capabilities Linux capabilities control what a process can do — even as root inside the container. Drop everything, then add back only what\u0026rsquo;s needed:\n1 docker run --cap-drop ALL --cap-add NET_BIND_SERVICE nginx:alpine Required capabilities depend on the image entrypoint, configured user, ports, kernel settings, and mounts. Initialization may need permissions that steady-state operation does not. Test dropped capabilities against the actual application; the example is not a universal profile.\nRead-Only Root Filesystem Prevent the container from writing to its own filesystem:\n1 2 docker run --read-only --tmpfs /run:noexec,nosuid,size=64M \\ --tmpfs /tmp:noexec,nosuid,size=32M nginx:alpine Write-capable directories are mounted explicitly as tmpfs volumes. Any exploit that tries to write a file to /usr, /etc, or /var will fail because those paths are read-only.\nFull Docker Compose Runtime Example Here\u0026rsquo;s how all three options look in a docker-compose.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 version: \u0026#34;3.8\u0026#34; services: web: image: nginx:1.27-alpine ports: - \u0026#34;127.0.0.1:8080:80\u0026#34; security_opt: - no-new-privileges:true cap_drop: - ALL cap_add: - NET_BIND_SERVICE - NET_RAW read_only: true tmpfs: - /tmp:noexec,nosuid,size=32M - /var/cache/nginx:noexec,nosuid,size=128M - /run:noexec,nosuid,size=64M user: \u0026#34;101:101\u0026#34; # nginx user within the container restart: unless-stopped This configuration:\nPrevents privilege escalation (no-new-privileges) Strips all capabilities, then adds only NET_BIND_SERVICE (bind port 80) and NET_RAW (health check pings) Makes the entire root filesystem read-only Mounts tmpfs volumes for Nginx\u0026rsquo;s cache, PID file, and temp directories Runs as the nginx user (UID 101) Putting It All Together — A Hardened Docker Compose Stack The following stack applies both image-level and runtime security to a static site served by Nginx:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 version: \u0026#34;3.8\u0026#34; services: static-site: # Use a slim, hardened base image image: nginx:1.27-alpine ports: - \u0026#34;127.0.0.1:8080:80\u0026#34; volumes: - ./html:/usr/share/nginx/html:ro - ./nginx.conf:/etc/nginx/conf.d/default.conf:ro security_opt: - no-new-privileges:true cap_drop: - ALL cap_add: - NET_BIND_SERVICE read_only: true tmpfs: - /tmp:noexec,nosuid,size=16M - /var/cache/nginx:noexec,nosuid,size=64M - /var/run:noexec,nosuid,size=16M user: \u0026#34;101:101\u0026#34; logging: driver: \u0026#34;json-file\u0026#34; options: max-size: \u0026#34;10m\u0026#34; max-file: \u0026#34;3\u0026#34; restart: unless-stopped All volumes are mounted :ro to prevent writes to config and content directories. Logging is capped at 30 MB total. The container runs with no extra capabilities beyond binding its network port.\nApply the same pattern to PostgreSQL, Redis, MinIO, and any other stateful service by adding :ro bind mounts for config files, tmpfs for writable runtime directories, and capability drops for everything except what the process genuinely needs.\nConclusion Docker image security in your homelab doesn\u0026rsquo;t require an enterprise security team. Three practical steps make a measurable difference:\nBuild-time — use multi-stage Dockerfiles to exclude build artifacts and unnecessary packages from your final images Image choice — prefer -slim, distroless, or Docker Hardened Images. Each shaves hundreds of packages and their associated CVEs Runtime — apply no-new-privileges, drop all capabilities, and set read_only: true on every compose service Start with layer 3 (runtime hardening) — it requires zero build changes and works on any existing Docker Compose stack. Then move up to layer 2 (slimmer base images) and layer 1 (custom multi-stage Dockerfiles) as you maintain and rebuild your own images.\nRun trivy image nginx:latest on your homelab right now. The results will tell you exactly where to start.\n","permalink":"https://blog.gntech.me/posts/2026-06-05-docker-image-security-hardening-distroless/","summary":"Harden your Docker images with multi-stage builds, distroless base images, and runtime security options. A practical guide to reducing image size and CVEs in your homelab.","title":"Docker Image Security Hardening — Multi-Stage Builds and Distroless"},{"content":"Why Your Homelab Needs a Documentation Wiki Every homelab accumulates complexity. VLAN assignments, firewall rules, Docker compose stacks, ZFS pool layouts, backup schedules, Proxmox bridge configurations — you remember them today, but will you remember them six months from now after a RouterOS update? Your homelab documentation should live somewhere searchable, editable, and organized, not scattered across sticky notes and SSH history.\nBookStack is a free and open-source wiki platform built on PHP and Laravel. It organizes content in a clean hierarchy: Shelf → Book → Chapter → Page. That maps naturally to how homelab knowledge is structured — group related documentation on shelves, create a book per project or subsystem, write detailed pages per configuration.\nThis guide covers everything you need to run BookStack in Docker Compose on your homelab: MariaDB backend, Traefik reverse proxy with automatic HTTPS, LDAP authentication for team access, and a practical backup strategy.\nWhy BookStack Over Other Documentation Tools BookStack is not the only documentation tool in the self-hosted space. Here is how it compares:\nBookStack: Hierarchical out of the box, WYSIWYG + Markdown editor, built-in search, role-based permissions, LDAP/OAuth/SAML authentication, PDF export, no build step Wiki.js: Modular and extensible, Git-backed storage, but requires a Node.js runtime and client-side rendering that can feel sluggish on lower-end hardware Docusaurus: Static site generator — excellent for developer docs but requires a build step and Git-based workflow; covered in a previous post Outline: Modern UI and real-time editing, but requires MinIO/S3 object storage and a Redis instance, adding operational overhead For a homelab, BookStack wins because it has no external dependencies beyond a database. The linuxserver.io Docker image keeps maintenance near zero, the WYSIWYG editor means you do not need Markdown fluency to contribute, and the hierarchical shelf/book/chapter/page model mirrors real infrastructure organization.\nPrerequisites Docker and Docker Compose v2 installed on a Linux host A working reverse proxy (Traefik or Nginx) for HTTPS termination Port 80 and 443 or internal reverse proxy network access Deploying BookStack with Docker Compose The standard setup uses two containers: MariaDB for database storage and the linuxserver.io BookStack image.\nCreate a directory and the compose file:\n1 2 3 mkdir -p /opt/bookstack cd /opt/bookstack nano docker-compose.yml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 services: mariadb: image: mariadb:11 container_name: bookstack-db restart: unless-stopped environment: - MYSQL_ROOT_PASSWORD=change-this-root-password - MYSQL_DATABASE=bookstack - MYSQL_USER=bookstack - MYSQL_PASSWORD=change-this-db-password volumes: - mariadb_data:/var/lib/mysql networks: - bookstack healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;healthcheck.sh\u0026#34;, \u0026#34;--connect\u0026#34;, \u0026#34;--innodb_initialized\u0026#34;] interval: 15s timeout: 5s retries: 5 bookstack: image: lscr.io/linuxserver/bookstack:latest container_name: bookstack restart: unless-stopped depends_on: mariadb: condition: service_healthy environment: - APP_URL=https://docs.gntech.me - DB_HOST=mariadb - DB_PORT=3306 - DB_DATABASE=bookstack - DB_USERNAME=bookstack - DB_PASSWORD=change-this-db-password - APP_LANG=en volumes: - bookstack_data:/config networks: - bookstack - traefik labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.bookstack.rule=Host(`docs.gntech.me`)\u0026#34; - \u0026#34;traefik.http.routers.bookstack.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.bookstack.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.services.bookstack.loadbalancer.server.port=80\u0026#34; volumes: mariadb_data: bookstack_data: networks: bookstack: name: bookstack driver: bridge traefik: external: true Replace change-this-root-password and change-this-db-password with strong random values:\n1 openssl rand -base64 32 | tee /dev/stderr # generate and show a password For production, consider using Docker secrets or an .env file excluded from version control instead of hardcoding passwords in the compose file.\nStart the stack:\n1 2 docker compose up -d docker compose logs -f bookstack Wait for the health check to pass and the BookStack container to initialize the database schema. The first startup takes 30\u0026ndash;60 seconds.\nTraefik Reverse Proxy Configuration The compose file above assumes Traefik runs on an external traefik network. If you use a different reverse proxy, adjust the network and labels accordingly.\nFor Nginx Proxy Manager or plain Nginx, add an upstream block:\n1 2 3 4 5 6 7 8 9 10 11 12 server { listen 443 ssl; server_name docs.gntech.me; location / { proxy_pass http://127.0.0.1:6875; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; } } Note that BookStack serves on port 80 inside the container. If you need to bind to a host port instead of using a reverse proxy network, add ports: - \u0026quot;6875:80\u0026quot; to the bookstack service.\nInitial Setup and Configuration Access your BookStack instance at the configured domain. Navigate to /register to create the first admin account.\nOnce logged in, go to Settings → Features to configure:\nSite name: \u0026ldquo;GnTech Homelab Docs\u0026rdquo; or whatever fits your lab Allow registration: Disable after creating accounts for team members Default language: Set per user or globally Configuring LDAP Authentication If you run a directory service (Active Directory, OpenLDAP, or Authentik), BookStack supports LDAP SSO out of the box. Add these environment variables to the bookstack service:\n1 2 3 4 5 6 7 8 9 10 11 12 13 environment: - APP_URL=https://docs.gntech.me - LDAP_ENABLED=true - LDAP_SERVER=ldap://10.0.20.10:389 - LDAP_BASE_DN=dc=gntech,dc=me - LDAP_DN=cn=binduser,dc=gntech,dc=me - LDAP_PASS=your-bind-password - LDAP_USER_FILTER=(\u0026amp;(objectClass=user)(sAMAccountName=${user})) - LDAP_VERSION=3 - LDAP_FOLLOW_REFERRALS=false - LDAP_ID_ATTRIBUTE=sAMAccountName - LDAP_DISPLAY_NAME_ATTRIBUTE=displayName - LDAP_EMAIL_ATTRIBUTE=mail Restart the stack after adding LDAP config:\n1 docker compose up -d Users authenticate with their domain credentials. BookStack maps their LDAP group memberships to roles automatically.\nConfiguring SMTP for Email Notifications BookStack can send password reset emails and page notifications. Configure SMTP under Settings → Mail, or inject via environment variables:\n1 2 3 4 5 6 environment: - MAIL_DRIVER=smtp - MAIL_HOST=10.0.20.30 - MAIL_PORT=25 - MAIL_FROM=docs@gntech.me - MAIL_FROM_NAME=GnTech Docs Organizing Your Homelab Documentation A well-structured wiki is easier to maintain. Here is a suggested hierarchy for homelab documentation:\n📚 Networking (Shelf) ├── VLAN Layout (Book) │ ├── VLAN IDs and Purposes (Page) │ ├── Firewall Rules (Page) │ └── DHCP Scopes (Page) ├── MikroTik Config (Book) │ ├── RouterOS 7 Setup (Page) │ ├── BGP Peering (Page) │ └── WireGuard VPN (Page) └── DNS Records (Book) ├── Internal Zones (Page) └── Pi-hole / Unbound (Page) 📚 Proxmox (Shelf) ├── Host Configuration (Book) ├── VM Templates (Book) └── Backup Strategy (Book) ├── PBS Setup (Page) └── Offsite Replication (Page) 📚 Docker (Shelf) ├── Compose Stacks (Book) ├── Networking (Book) └── Security (Book) BookStack supports code blocks with syntax highlighting for configs, inline image uploads for network diagrams, and file attachments for PDF documentation. The search bar indexes all page content including code blocks, so you can find that one firewall rule or compose override file in seconds.\nBackup and Maintenance BookStack stores content in MariaDB and file uploads in the /config volume. A complete backup requires both.\nAutomated Database Backup Create a systemd timer or cron job to dump the database daily:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 #!/bin/bash # /opt/bookstack/backup.sh TIMESTAMP=$(date +%Y%m%d-%H%M%S) BACKUP_DIR=/tank/backups/bookstack mkdir -p \u0026#34;$BACKUP_DIR\u0026#34; docker exec bookstack-db mariadb-dump \\ --single-transaction \\ --user=bookstack \\ --password=\u0026#34;change-this-db-password\u0026#34; \\ bookstack | gzip \u0026gt; \u0026#34;$BACKUP_DIR/bookstack-db-$TIMESTAMP.sql.gz\u0026#34; # Backup volume data docker run --rm -v bookstack_data:/source -v \u0026#34;$BACKUP_DIR\u0026#34;:/backup \\ alpine tar czf \u0026#34;/backup/bookstack-files-$TIMESTAMP.tar.gz\u0026#34; -C /source . Make it executable and set up a cron entry:\n1 2 chmod +x /opt/bookstack/backup.sh echo \u0026#34;0 3 * * * /opt/bookstack/backup.sh\u0026#34; | crontab - Restore Procedure 1 2 3 4 5 6 7 # Restore database gunzip -c bookstack-db-20260605-030000.sql.gz | \\ docker exec -i bookstack-db mariadb -u bookstack -p\u0026#34;change-this-db-password\u0026#34; bookstack # Restore files docker run --rm -v bookstack_data:/target -v \u0026#34;$PWD\u0026#34;:/restore \\ alpine tar xzf \u0026#34;/restore/bookstack-files-20260605-030000.tar.gz\u0026#34; -C /target Updating BookStack Since we use the linuxserver.io image, updating is straightforward:\n1 2 docker compose pull docker compose up -d The image auto-runs database migrations on startup, so the schema stays current. Check the BookStack release notes before major version bumps.\nConclusion BookStack turns the chore of documentation into a habit. With Docker Compose, you go from zero to a fully functional homelab wiki in under ten minutes. Traefik handles TLS, LDAP handles authentication, and a five-line backup script protects your data.\nStart by documenting your VLAN layout and firewall rules — the things you most frequently reference. Then expand into Proxmox configs, Docker compose stacks, and recovery procedures. Your future self will thank you.\nRelated reading:\nBookStack Installation Docs linuxserver/bookstack Docker Image Docusaurus Documentation Site for Homelab ","permalink":"https://blog.gntech.me/posts/2026-06-05-bookstack-docker-homelab-documentation/","summary":"Deploy BookStack with Docker Compose for a self-hosted homelab documentation wiki. Real configs for MariaDB, Traefik, LDAP, and automated backups.","title":"BookStack Docker Deployment — Self-Hosted Wiki for Homelab Documentation"},{"content":"Introduction to MikroTik RouterOS 7 Routing Filters If you moved from RouterOS 6 to RouterOS 7, the first thing you noticed about routing is that everything changed. The old firewall-style routing filter rules are gone. In their place is a declarative filter chain — a script-like language that operates on route attributes rather than firewall chains. This change affects BGP, OSPF, RIP, and even your static route selection.\nPolicy-based routing (PBR) is separate but complementary. Where routing filters control which routes exist in the routing table, PBR controls which routing table traffic uses. Combined, they give you complete traffic engineering in your MikroTik network.\nThis guide covers:\nRouterOS 7 routing filter architecture and syntax BGP route filtering with AS path, prefix, and community filters Policy-based routing with mangle rules and routing marks Dual WAN traffic steering and failover Troubleshooting common filter and PBR issues Understanding RouterOS 7 Routing Filter Architecture RouterOS 7 introduces a unified routing filter engine used by all routing protocols. Filters are defined as independent chain rules and then attached to a protocol or VRF.\nFilter Chains Each routing protocol has up to four filter chains:\nChain Direction Purpose in Inbound Filter routes before they enter the routing table out Outbound Filter routes before advertising to peers default-in Inbound Applied when no in chain exists default-out Outbound Applied when no out chain exists Filters are configured under /routing filter rule and referenced by name in the protocol configuration.\nRoute Attributes Available in Filters RouterOS 7 routing filters operate on per-route attributes. The most commonly used ones:\ndst-address — destination prefix gateway — next-hop address distance — administrative distance routing-mark — associated routing table bgp-as-path — BGP AS_PATH attribute bgp-communities — BGP community list bgp-local-pref — BGP local preference pref-src — preferred source address scope — route scope for next-hop resolution Filter Flow — Match and Action Every filter rule has a condition (match) and an action. Rules are evaluated in order. The first matching rule with a terminal action (accept/reject) terminates the chain.\n/routing filter rule add chain=bgp-in \\ if-dst-length=32 \\ action=reject add chain=bgp-in \\ if-gateway=\u0026#34;10.0.0.1\u0026#34; \\ action=accept If no rule matches, the implicit reject applies — the route is denied.\nBasic Routing Filter Syntax Filter rules use a consistent syntax. Here are the core building blocks:\nMatching on Prefix /routing filter rule add chain=bgp-in if-dst-address=\u0026#34;10.0.0.0/8\u0026#34; action=accept # Reject /32 routes (host routes from BGP) add chain=bgp-in if-dst-length=32 action=reject # Accept only prefixes /24 or smaller add chain=bgp-in if-dst-length=24-32 action=accept Matching on BGP AS Path # Accept only routes with origin AS 65001 add chain=bgp-in if-bgp-as-path=\u0026#34;^65001$\u0026#34; action=accept # Reject routes from AS 65099 (transit ISP) add chain=bgp-in if-bgp-as-path=\u0026#34;.*65099.*\u0026#34; action=reject # Accept routes with exactly 2 AS hops add chain=bgp-in if-bgp-as-path-length=2 action=accept BGP Route Filtering with Routing Filters Let\u0026rsquo;s put together a complete BGP filter setup. This example assumes a BGP session with an upstream ISP (AS 64500) and an internal peer (AS 65001).\nInbound Filter — Control What Enters Your Table /routing filter rule # Chain: bgp-in — applied to inbound BGP updates # 1. Reject default route from internal peer add chain=bgp-in \\ if-dst-address=\u0026#34;0.0.0.0/0\u0026#34; \\ if-bgp-as-path=\u0026#34;^65001$\u0026#34; \\ action=reject # 2. Accept specific prefixes from upstream (AS 64500) add chain=bgp-in \\ if-bgp-as-path=\u0026#34;^64500$\u0026#34; \\ if-dst-address=\u0026#34;203.0.113.0/24\u0026#34; \\ action=accept # 3. Accept default route only from upstream add chain=bgp-in \\ if-dst-address=\u0026#34;0.0.0.0/0\u0026#34; \\ if-bgp-as-path=\u0026#34;^64500$\u0026#34; \\ action=accept # 4. Accept internal routes from AS 65001 add chain=bgp-in \\ if-bgp-as-path=\u0026#34;^65001$\u0026#34; \\ action=accept # 5. Reject everything else (security) add chain=bgp-in \\ action=reject Outbound Filter — Control What You Advertise /routing filter rule # Chain: bgp-out — applied to outbound BGP updates # 1. Only advertise your own prefixes add chain=bgp-out \\ if-dst-address=\u0026#34;198.51.100.0/24\u0026#34; \\ action=accept # 2. Don\u0026#39;t advertise private ranges add chain=bgp-out \\ if-dst-address=\u0026#34;10.0.0.0/8\u0026#34; \\ action=reject add chain=bgp-out \\ if-dst-address=\u0026#34;192.168.0.0/16\u0026#34; \\ action=reject # 3. Reject everything else add chain=bgp-out \\ action=reject Attaching Filters to a BGP Connection /routing bgp connection add name=isp-upstream \\ remote.address=203.0.113.1 \\ remote.as=64500 \\ local.role=ebgp \\ routing-table=main \\ .in.filter=bgp-in \\ .out.filter=bgp-out Note the syntax: .in.filter and .out.filter reference the filter chain names you created.\nPolicy-Based Routing with Routing Marks and Mangle Policy-based routing (PBR) lets you route traffic based on source address, protocol, port, packet size, or any combination — rather than just the destination.\nArchitecture The PBR pipeline in RouterOS 7 is:\nFirewall mangle marks packets or connections Routing marks identify the routing table to use Separate routing tables contain the custom routes Routing rules direct marked traffic to the correct table Step 1 — Mark Traffic with Mangle /ip firewall mangle # Mark all traffic from VLAN 10 (office) to use the office routing table add chain=prerouting \\ src-address=10.0.10.0/24 \\ action=mark-routing \\ new-routing-mark=office \\ passthrough=yes # Mark all HTTP/HTTPS traffic from VLAN 20 (guest) to use a filtered WAN add chain=prerouting \\ src-address=10.0.20.0/24 \\ protocol=tcp \\ dst-port=80,443 \\ action=mark-routing \\ new-routing-mark=guest-web \\ passthrough=yes # Mark all traffic to specific destination (VPN subnet) via dedicated table add chain=prerouting \\ dst-address=10.200.0.0/16 \\ action=mark-routing \\ new-routing-mark=vpn \\ passthrough=yes The passthrough=yes parameter makes the packet continue to subsequent mangle rules. Use passthrough=no if you want to stop processing after the match.\nConnection Marking for Stateful Routing For stateful PBR (recommended), mark connections instead of individual packets:\n# Mark connections from the office VLAN add chain=prerouting \\ src-address=10.0.10.0/24 \\ action=mark-connection \\ new-connection-mark=office-conn \\ passthrough=yes # Route all packets in those connections add chain=prerouting \\ connection-mark=office-conn \\ action=mark-routing \\ new-routing-mark=office \\ passthrough=no Connection marking ensures that return traffic follows the same path.\nStep 2 — Create Routing Tables /routing table add name=office fib add name=guest-web fib add name=vpn fib # The \u0026#39;fib\u0026#39; flag means the table participates in FIB resolution # — required for the routes to be usable Step 3 — Add Routes to Each Table /ip route # Office traffic goes via ISP1 add dst-address=0.0.0.0/0 \\ gateway=172.16.1.1 \\ routing-mark=office \\ distance=1 # Guest web traffic goes via ISP2 add dst-address=0.0.0.0/0 \\ gateway=172.16.2.1 \\ routing-mark=guest-web \\ distance=1 # VPN traffic via Wireguard interface add dst-address=0.0.0.0/0 \\ gateway=wg0 \\ routing-mark=vpn \\ distance=1 Step 4 — Routing Rules (Optional) Routing rules direct traffic before the main routing table lookup. They are evaluated before standard destination-based routing:\n/routing rule # Send traffic from specific source to custom table add src-address=10.0.10.0/24 table=office # Send traffic to specific destination via VPN table add dst-address=10.200.0.0/16 table=vpn # Everything else uses the main table add table=main Use routing rules as an alternative to mangle-based marking when you only need source or destination-based PBR.\nDual WAN with Policy-Based Routing A common homelab scenario: two ISPs where one handles general traffic and the other handles specific services (or backup).\nBasic Dual WAN Setup Assume WAN1 on ether1 (gateway 172.16.1.1) and WAN2 on ether2 (gateway 172.16.2.1).\n# Default route via WAN1 (lower distance = preferred) /ip route add dst-address=0.0.0.0/0 \\ gateway=172.16.1.1 \\ distance=1 # Backup route via WAN2 add dst-address=0.0.0.0/0 \\ gateway=172.16.2.1 \\ distance=2 PBR for Specific Traffic via WAN2 Force certain traffic to use WAN2 even when WAN1 is up:\n/ip firewall mangle # Mark DNS and NTP traffic to use WAN2 add chain=prerouting \\ protocol=udp \\ dst-port=53,123 \\ action=mark-routing \\ new-routing-mark=wan2 \\ passthrough=no # Mark torrent traffic to use WAN2 add chain=prerouting \\ protocol=tcp \\ dst-port=6881-6999 \\ action=mark-routing \\ new-routing-mark=wan2 \\ passthrough=no /routing table add name=wan2 fib /ip route add dst-address=0.0.0.0/0 \\ gateway=172.16.2.1 \\ routing-mark=wan2 \\ distance=1 Failover with Route Checks For production failover, add recursive next-hop checks:\n/ip route # WAN1 with recursive check add dst-address=0.0.0.0/0 \\ gateway=172.16.1.1 \\ distance=1 \\ check-gateway=ping # WAN2 with recursive check add dst-address=0.0.0.0/0 \\ gateway=172.16.2.1 \\ distance=2 \\ check-gateway=ping When check-gateway=ping is set, RouterOS periodically pings the gateway. If the ping fails, the route is disabled and traffic falls back to the next available route.\nAdvanced Filtering — BGP Communities and Local Preference Setting Local Preference with Filters Control inbound route preference before it reaches the routing table:\n/routing filter rule # Set high local-pref for routes tagged with community 64512:100 add chain=bgp-in \\ if-bgp-communities=\u0026#34;64512:100\u0026#34; \\ set-bgp-local-pref=200 \\ action=accept # Set medium local-pref for routes from upstream add chain=bgp-in \\ if-bgp-as-path=\u0026#34;^64500$\u0026#34; \\ set-bgp-local-pref=150 \\ action=accept # Default local-pref for everything else add chain=bgp-in \\ action=accept Manipulating BGP Communities # Append a community to outbound routes add chain=bgp-out \\ if-dst-address=\u0026#34;198.51.100.0/24\u0026#34; \\ set-bgp-communities=\u0026#34;64501:200\u0026#34; \\ action=accept # Strip communities from routes before advertising to upstream add chain=bgp-out \\ set-bgp-communities=\u0026#34;\u0026#34; \\ action=accept Route Tagging with Routing Filters Use routing filters as a route-map to tag routes for later PBR:\n/routing filter rule # Tag routes from specific BGP peer with a routing mark add chain=bgp-in \\ if-bgp-as-path=\u0026#34;^65001$\u0026#34; \\ set-routing-mark=internal \\ action=accept Troubleshooting Routing Filters and PBR Verify Active Routes # Show all routes /routing route print # Show routes in a specific routing table /routing route print where routing-mark=office # Show BGP routes specifically /routing route print where protocol=bgp Check What BGP Is Advertising # Show advertised routes for a BGP session /routing bgp session advertisements print \\ session=isp-upstream # Show received routes /routing bgp session routes print \\ session=isp-upstream Debug PBR Traffic Flow # Check which routing table a specific packet would use # First check mangle hits /ip firewall mangle print stats # Then check the routing table lookup with routing mark /routing route print where routing-mark=guest-web # Verify routing rules are in order /routing rule print Common Pitfalls 1. Filter chain ordering matters\nRules are evaluated in the order they were created. An accept rule before a reject rule means the reject never runs. Always put specific match rules before general ones.\n# WRONG: This accepts everything before the reject add chain=bgp-in action=accept add chain=bgp-in if-dst-address=\u0026#34;10.0.0.0/8\u0026#34; action=reject # RIGHT: Specific reject first add chain=bgp-in if-dst-address=\u0026#34;10.0.0.0/8\u0026#34; action=reject add chain=bgp-in action=accept 2. Implicit reject at end of chain\nIf no rule matches, the route is rejected. Always end your chains with an explicit action=accept for the traffic you want to allow, or an explicit action=reject to block everything else.\n3. Mangle rules need the correct chain\nprerouting for traffic coming into the router (both forwarded and locally destined) forward only for forwarded traffic output for traffic generated by the router itself For PBR, always use chain=prerouting so the routing mark is applied before the route lookup.\n4. Routing table missing FIB flag\nIf you create a routing table without the fib flag, routes in that table won\u0026rsquo;t be resolvable:\n# Check if your table has FIB enabled /routing table print # Enable it /routing table set office fib=yes Putting It All Together RouterOS 7 routing filters and policy-based routing give you complete traffic engineering control. The key differences from RouterOS 6:\nFilters are declarative, not firewall-style — define the match condition and action PBR uses mangle + routing-mark + separate routing tables — no more routing-mark in static routes Routing rules provide an alternative to mangle for simple source/destination-based PBR Start simple: create a BGP inbound filter that blocks obvious garbage (host routes, bogons), then layer on policy routing for traffic segmentation. Always test with print commands before applying to production.\nA well-configured routing filter chain keeps your routing table clean. Combined with PBR, you get per-client, per-protocol traffic control that rivals enterprise routers — all on MikroTik hardware.\n","permalink":"https://blog.gntech.me/posts/2026-06-04-mikrotik-routeros7-routing-filters-pbr/","summary":"Configure MikroTik RouterOS 7 routing filters for BGP, OSPF, and policy-based routing. Real CLI examples for route maps, mangle rules, routing marks, and traffic steering.","title":"MikroTik RouterOS 7 Routing Filters — Policy-Based Routing and Route Selection"},{"content":"Btrfs and OpenZFS both offer checksums, snapshots and compression, but that does not make them interchangeable. The host distribution, storage layout, kernel, management tools and recovery procedure matter as much as the filesystem name.\nThere is no universal rule to use ZFS for every hypervisor and Btrfs for every Docker volume. Start with the platform\u0026rsquo;s supported configuration and a tested recovery workflow.\nVerification status Area Evidence and limits Technical review Official references below reviewed on October 31, 2026. Examples Read-only inventory commands; Bash syntax checked. Runtime No filesystem creation, migration, expansion, benchmark, quota or recovery test executed. Recommendation scope Decision framework, not measured performance or a promise of support across releases. 1. Record the installed stack Run applicable commands on the host that owns the storage. A guest\u0026rsquo;s filesystem does not identify the hypervisor\u0026rsquo;s backing pool.\n1 2 3 uname -r findmnt -o TARGET,SOURCE,FSTYPE,OPTIONS lsblk -o NAME,SIZE,TYPE,FSTYPE,MOUNTPOINTS On a PVE host with ZFS:\n1 2 3 4 pveversion -v zfs --version zpool status zfs list -o name,type,used,available,mountpoint On a Btrfs host, inspect the actual mount:\n1 2 3 4 btrfs version read -r -p \u0026#39;Existing Btrfs mount path: \u0026#39; btrfs_mount sudo btrfs filesystem show \u0026#34;$btrfs_mount\u0026#34; sudo btrfs filesystem usage \u0026#34;$btrfs_mount\u0026#34; Btrfs user-space tools and kernel implementation have separate versions. OpenZFS is maintained outside the mainline Linux kernel, but packaging varies: do not assume every PVE kernel upgrade performs a local DKMS rebuild. Use the distribution\u0026rsquo;s supported kernel/module packages.\n2. Compare operational requirements Question OpenZFS Btrfs Storage organization Pools, vdevs, filesystems and zvols Filesystems, devices and subvolumes Snapshot scope Dataset or zvol Subvolume Cache model ARC plus other system memory consumers Linux page-cache-based operation Repair opportunity Requires an available correct copy or recoverable redundancy Requires an available correct copy or recoverable redundancy Capacity changes Depend on topology, features and version Depend on allocation profiles, free space and version Docker suitability Check the actual image store and mounted data paths Check the actual image store and mounted data paths A checksum can detect damage without being able to repair it. Neither a snapshot on the same storage nor a redundant layout is an independent backup.\nBtrfs profiles have different support status. The upstream feature-status table currently marks RAID56 unstable. Do not treat all Btrfs RAID profiles as equivalent or infer your older kernel\u0026rsquo;s support from the newest table.\n3. Check platform support before choosing PVE documents both ZFS and a Btrfs storage backend. The earlier statement that Btrfs has no platform integration was too broad. Read the documentation matching the installed release, including feature limitations and installer support.\nThe PVE Btrfs backend describes how raw guest disks use subvolumes for snapshot support. That is different from placing a Btrfs filesystem inside a guest.\nFor a PVE deployment, evaluate the PVE ZFS guidance, required guest-storage features, maintenance support and available recovery skills. PBS backups do not require choosing ZFS for every source guest or host filesystem.\nFor Docker, choose a deployment boundary you can support. A VM provides its own kernel and filesystem stack; an LXC shares the host kernel and introduces nesting and permission constraints. Do not infer a supported Docker layout merely from the outer host\u0026rsquo;s filesystem.\n4. Separate Docker image storage from application data On the Docker host:\n1 2 docker version docker info Record the active storage backend and backing filesystem. New Docker Engine 29 installations use the containerd image store by default; upgraded installations can retain classic storage drivers. Consult Docker storage selection.\nThe classic native Btrfs driver uses subvolumes and snapshots. It is not the same implementation as overlay2. Neither Btrfs nor ZFS alone proves that a particular overlay or snapshotter combination is supported. See the Btrfs driver documentation.\nNamed volumes and bind mounts are separate from a container\u0026rsquo;s writable layer. Evaluate application data on its actual backing storage, not from an image-layer benchmark. Do not change Docker\u0026rsquo;s storage configuration on a populated host without a migration and rollback plan.\n5. Measure memory and compression ARC\u0026rsquo;s maximum is a cache limit, not a fixed allocation reserved before guests start. Defaults and distribution tuning vary. Observe memory pressure, guest demand and ARC behavior before setting a smaller limit; an arbitrary 2 GiB cap is not a universal improvement.\nBtrfs also uses memory for caches and filesystem work. Neither filesystem has a single RAM threshold that predicts performance for every workload.\nCompression consumes CPU and saves space only where the data is compressible. Check actual properties, files and measured workload behavior. Do not promise a fixed percentage gain or call compression free.\nBtrfs compression options affect file extents; a defragment/recompress operation rewrites existing data and can unshare extents. It is not a harmless way to toggle a subvolume setting. See Btrfs compression and filesystem operations.\n6. Plan expansion and quotas by feature The old claim that ZFS can never add a disk to an existing vdev is incorrect. On supporting versions, zpool attach can add a mirror member or expand a RAID-Z group. RAID-Z expansion does not increase its parity level. Check installed features, topology, capacity behavior and recovery implications first.\nBtrfs device addition and removal are also constrained operations. Removal needs enough space and readable or reconstructable data; it is not a guarantee that arbitrary disks can be removed safely.\nFor quotas, select documented ZFS dataset/user limits or Btrfs quota mechanisms and test snapshot accounting. Do not apply ext4/XFS project-quota mount advice to Btrfs as a substitute for its own quota documentation.\nThis comparison intentionally includes no pool creation, formatting, balancing, device removal or live cache-tuning commands.\n7. Decide from recovery evidence Need Practical next step PVE-managed guest storage Compare release-specific backend features and support first. Docker application persistence Test the actual mounted path and application\u0026rsquo;s consistency requirements. Frequent disk-layout changes Model permitted operations, free-space needs and rebuild exposure. Strict data protection Test checks, alerts, independent copies and recovery on replacement hardware. Performance problem Benchmark a controlled workload before changing filesystem or hardware. Document the selected versions, topology, included data, maintenance procedures and recovery credentials. Test an isolated restore before moving important services. Snapshot send/receive is a transfer mechanism, not proof that a complete application can be recovered.\nContinue reading Storage benchmarking — Record a bounded baseline. ZFS snapshots and recovery — Separate dataset recovery from guest backups. Backup strategy — Plan independent copies and recovery targets. ","permalink":"https://blog.gntech.me/posts/2026-06-04-btrfs-vs-zfs-homelab-storage/","summary":"Choose a supported storage design from workload and recovery requirements rather than blanket filesystem rankings.","title":"Btrfs vs ZFS for Homelab Storage — Proxmox and Docker Guide"},{"content":"Why a Self-Hosted Dashboard Belongs in Every Homelab As your homelab grows, so does the list of services you manage. Traefik routes traffic, Portainer manages containers, Grafana visualizes metrics, Pi-hole filters DNS — each with its own URL, port, and login page. A self-hosted dashboard gives you a single entry point to launch any service, see which containers are healthy, and monitor system resources at a glance.\nHomepage is the most polished option in this space. It is a lightweight, YAML-driven dashboard that integrates directly with Docker containers, exposes real-time system stats, and provides API widgets for services like Traefik, Pi-hole, Prometheus, Proxmox, and Uptime Kuma. Unlike heavier solutions like Homer or Heimdall, Homepage requires no database and no complex backend — just a Docker container and a few YAML files.\nWhat you will end up with:\nA single dashboard at http://homelab:3000 or via your reverse proxy Auto-discovered Docker containers with live status, CPU, and memory System resource widgets (CPU, memory, disk, uptime) Service-specific API widgets for Traefik, Pi-hole, Uptime Kuma, and Proxmox A clean, customizable interface with bookmarks and search Prerequisites Docker and Docker Compose v2 installed on any Linux host A working reverse proxy (Traefik or Nginx) if you want TLS and a clean domain Port 3000 available on the host (or change it in compose) Docker socket access for container discovery Deploying Homepage with Docker Compose Homepage runs as a single container with no external dependencies. The Docker Compose deployment is straightforward:\n1 2 3 4 5 6 7 8 9 10 11 12 13 services: homepage: image: ghcr.io/gethomepage/homepage:latest container_name: homepage restart: unless-stopped ports: - \u0026#34;3000:3000\u0026#34; environment: - PUID=1000 - PGID=1000 volumes: - ./config:/app/config - /var/run/docker.sock:/var/run/docker.sock:ro Create the project directory and start the container:\n1 2 3 mkdir -p ~/docker/homepage/config cd ~/docker/homepage docker compose up -d Open http://your-host:3000. You will see a blank page — Homepage ships with no configuration by default. The config directory is empty until you populate it.\nAdding a Traefik Reverse Proxy If you run Traefik, add labels for automatic TLS and routing:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 services: homepage: image: ghcr.io/gethomepage/homepage:latest container_name: homepage restart: unless-stopped expose: - \u0026#34;3000\u0026#34; environment: - PUID=1000 - PGID=1000 volumes: - ./config:/app/config - /var/run/docker.sock:/var/run/docker.sock:ro labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.homepage.rule=Host(`homepage.lab.example.com`)\u0026#34; - \u0026#34;traefik.http.services.homepage.loadbalancer.server.port=3000\u0026#34; - \u0026#34;traefik.http.routers.homepage.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.homepage.tls.certresolver=letsencrypt\u0026#34; Security note: Mounting the Docker socket (/var/run/docker.sock) gives Homepage access to control Docker. This is required for automatic container discovery. If you prefer to avoid the socket mount, you can manually define services in services.yaml without Docker stats.\nConfiguration Structure Homepage reads its configuration from three YAML files in the config directory. You can start with just services.yaml and bookmarks.yaml and add more as needed:\n~/docker/homepage/config/ ├── services.yaml # Service groups and links ├── bookmarks.yaml # Quick-access bookmarks ├── widgets.yaml # Widget configuration for API integrations ├── docker.yaml # Docker container overrides ├── settings.yaml # Layout, theme, and general settings └── custom.css # Theme overrides After adding or modifying any config file, restart the container:\n1 cd ~/docker/homepage \u0026amp;\u0026amp; docker compose restart Homepage reloads configuration on restart only — there is no hot-reload.\nServices Configuration with Docker Integration The services.yaml file defines how services are grouped and displayed. Combined with the Docker socket mount, Homepage automatically shows the health status and resource usage of each container.\nManual Service Groups Create config/services.yaml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 --- - Infrastructure: - Traefik: href: https://traefik.lab.example.com description: Reverse Proxy Dashboard icon: traefik ping: https://traefik.lab.example.com container: traefik - Portainer: href: https://portainer.lab.example.com description: Container Management icon: portainer container: portainer - Proxmox: href: https://proxmox.lab.example.com:8006 description: Virtualization Platform icon: proxmox - Monitoring: - Grafana: href: https://grafana.lab.example.com description: Metrics Dashboard icon: grafana container: grafana - Uptime Kuma: href: https://uptime.lab.example.com description: Status Monitoring icon: uptime-kuma container: uptime-kuma - Netdata: href: https://netdata.lab.example.com description: System Monitoring icon: netdata container: netdata - Media: - Jellyfin: href: https://media.lab.example.com description: Media Server icon: jellyfin container: jellyfin Key fields:\nicon: Homepage ships with 200+ built-in icons — just use the service name container: Links the service to a Docker container name; Homepage uses the socket to show status, CPU, and memory ping: Homepage periodically checks the URL and shows a green/red dot status indicator description: Shown beneath the service name Docker Auto-Discovery with docker.yaml Without any configuration, Homepage can auto-discover all running containers if the Docker socket is mounted. By default, it creates a group called \u0026ldquo;Docker\u0026rdquo; with every container listed. To control what appears and in which group, create config/docker.yaml:\n1 2 3 4 5 6 7 8 9 10 --- my-docker-host: host: 127.0.0.1 docker: host monitors: - traefik - portainer - grafana - uptime-kuma - jellyfin This creates a server group named \u0026ldquo;my-docker-host\u0026rdquo; with only the listed containers. Each container shows its status, CPU usage, and memory consumption pulled from the Docker API.\nDocker Labels for Customization You can override Homepage display settings by adding labels to your Docker compose files:\n1 2 3 4 5 6 7 8 9 10 11 services: jellyfin: image: jellyfin/jellyfin labels: - \u0026#34;homepage.group=Media\u0026#34; - \u0026#34;homepage.name=Jellyfin\u0026#34; - \u0026#34;homepage.icon=jellyfin\u0026#34; - \u0026#34;homepage.href=https://media.lab.example.com\u0026#34; - \u0026#34;homepage.description=Movies, TV, Music\u0026#34; - \u0026#34;homepage.ping=https://media.lab.example.com\u0026#34; - \u0026#34;homepage.weight=1\u0026#34; This approach keeps everything in the container\u0026rsquo;s own configuration — Homepage picks it up from Docker labels automatically.\nSystem Monitoring Widgets The widgets.yaml file adds panels to the top of the dashboard showing real-time system information. Homepage supports several built-in widgets:\nResources Widget 1 2 3 4 5 6 7 --- - resources: label: SRV1 cpu: true memory: true disk: / uptime: true This widget displays CPU usage, memory usage, root disk usage, and system uptime for the host running Homepage.\nDocker Engine Widget 1 2 3 - docker: label: Docker Engine host: /var/run/docker.sock Shows total container count, running count, and a quick health summary.\nSearch Widget 1 2 3 - search: provider: google target: _blank For a privacy-respecting homelab, point it at your own SearXNG instance:\n1 2 3 4 - search: provider: custom url: https://search.lab.example.com/search?q={query} target: _blank Clock and Greeting 1 2 3 - datetime: text_format: HH:mm locale: en-US Or with the date:\n1 2 3 - datetime: text_format: EEEE, MMMM do yyyy \u0026#39;|\u0026#39; HH:mm locale: en-US All widgets in one widgets.yaml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 --- - resources: label: SRV1 cpu: true memory: true disk: / uptime: true - search: provider: custom url: https://search.lab.example.com/search?q={query} target: _blank - datetime: text_format: EEEE, MMMM do yyyy \u0026#39;|\u0026#39; HH:mm locale: en-US Service-Specific API Widgets Homepage\u0026rsquo;s real power is in API widgets — live data pulled directly from your services\u0026rsquo; APIs and rendered as panel cards.\nTraefik Widget Enable the Traefik API endpoint first:\n1 2 3 4 5 6 7 services: traefik: image: traefik:v3.3 command: - \u0026#34;--api.insecure=false\u0026#34; - \u0026#34;--api.dashboard=true\u0026#34; # ... other config Then in widgets.yaml:\n1 2 3 4 - traefik: label: Traefik url: https://traefik.lab.example.com key: your-api-key-or-basic-auth If Traefik is behind authentication, you can pass the credentials as the key. Check the Traefik widget docs for exact authentication methods.\nPi-hole Widget Requires the Pi-hole API token. Find it in Pi-hole\u0026rsquo;s Settings → API/Web Interface → \u0026ldquo;Show API Token\u0026rdquo;.\n1 2 3 4 - pihole: label: Pi-hole url: http://pihole.lab.local key: your-api-token-here Shows total queries blocked today, percentage blocked, and query count.\nUptime Kuma Widget Uptime Kuma exposes a status page API by default. Enable the status page in Uptime Kuma settings, then:\n1 2 3 4 - uptimekuma: label: Uptime Kuma url: https://uptime.lab.example.com slug: your-status-page-slug The widget shows a green/red indicator for each monitored endpoint.\nProxmox Widget Requires an API token with read permissions. Create one in Proxmox at Datacenter → Permissions → API Tokens:\n1 2 3 4 5 - proxmox: label: Proxmox VE host: 10.0.20.30 username: your-token-id password: your-secret Shows node status, memory usage, CPU load, and running VM/LXC counts.\nBookmark Organization Bookmarks are grouped links for quick access. Create config/bookmarks.yaml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 --- - Tools: - GitHub: - icon: github href: https://github.com - Cloudflare: - icon: cloudflare href: https://dash.cloudflare.com - Docker Hub: - icon: docker href: https://hub.docker.com - Internal: - Router: - icon: mikrotik href: https://10.0.20.1 - Proxmox Host: - icon: proxmox href: https://10.0.20.30:8006 - NAS: - icon: nas href: https://nas.lab.local Bookmarks appear in a collapsible sidebar on the dashboard.\nTheming and Customization Homepage supports full visual customization through settings.yaml and custom.css.\nLayout and Layout Settings Create config/settings.yaml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 --- title: Homelab theme: dark background: image: https://images.unsplash.com/photo-1519681393784-d120267933ba blur: xs saturate: 50 layout: Media: style: row columns: 4 Monitoring: style: column columns: 3 Custom CSS Add config/custom.css with your own style overrides:\n1 2 3 4 body { --color-primary: #7c3aed; --color-primary-light: #a78bfa; } Common customizations include accent colors, card border radius, and header opacity. See the Homepage theming guide for the full variable list.\nAdvanced: Custom API Widgets If a service isn\u0026rsquo;t in Homepage\u0026rsquo;s built-in widget list, you can build a custom widget that fetches from any JSON API.\n1 2 3 4 5 6 7 - customapi: label: Custom Endpoint url: https://your-service/api/status headers: Authorization: Bearer your-token jsonpath: \u0026#34;.status\u0026#34; refreshInterval: 30000 Use jsonpath to extract a specific field from the JSON response. This works with any HTTP endpoint that returns JSON — NUT UPS status, TrueNAS API, or a custom script.\nTroubleshooting Common Issues Empty Dashboard After Deployment Homepage starts with a blank page until you create config files. Create at least services.yaml with a service group and restart the container.\nDocker Socket Permission Denied Permission denied on /var/run/docker.sock The PUID/PGID in the container must match a user that belongs to the docker group on the host. Verify:\n1 2 3 4 5 ls -l /var/run/docker.sock # Output: srw-rw---- 1 root docker 0 ... getent group docker # Output: docker:x:999:youruser usermod -aG docker youruser # if needed Then set PUID=$(id -u youruser) and PGID=$(getent group docker | cut -d: -f3).\nContainers Not Appearing If Docker auto-discovery isn\u0026rsquo;t showing containers:\nVerify the socket is mounted correctly: docker exec homepage ls -l /var/run/docker.sock should show the socket Check docker.yaml is loading the correct host (defaults to 127.0.0.1) Ensure containers are running: docker ps --all Widget Not Showing Data API widgets fail silently if the endpoint is unreachable or authentication is wrong. Check the Homepage container logs:\n1 docker logs homepage | grep -i widget Common causes:\nWrong URL or port (containers behind a Docker network need the internal URL, not the public one) Missing or expired API token Self-signed TLS certificate causing connection errors Putting It All Together A self-hosted dashboard transforms a scattered collection of services into a unified homelab experience. Homepage delivers this with minimal overhead — a single container, three YAML files, and optional API integrations give you a polished, responsive interface that works on desktop and mobile.\nKey takeaways:\nStart with the basic compose file and add service groups one at a time Mount the Docker socket for automatic container health, CPU, and memory display Add API widgets for Traefik, Pi-hole, and Uptime Kuma as high-value integrations Use docker.yaml to filter which containers appear instead of showing everything Customize the theme through settings.yaml and optional custom.css Test changes with docker compose restart — no database or rebuild needed Your dashboard will become the first page you open every time you check on the homelab. Start with the basics, then layer on widgets and integrations as your infrastructure grows.\n","permalink":"https://blog.gntech.me/posts/2026-06-04-homepage-docker-dashboard-homelab/","summary":"Deploy Homepage, the lightweight homelab dashboard, with Docker Compose. Configure service widgets, system monitors, Docker stats, and API integrations in a single YAML-driven interface.","title":"Homepage Docker Deployment — Self-Hosted Dashboard for Homelab Services"},{"content":"What Is MCP and Why Your Homelab Needs It The Model Context Protocol (MCP) is an open protocol developed by Anthropic that standardizes how AI assistants connect to external tools and data sources. Think of it as a USB-C port for AI — one standard connection that lets Claude reach into your infrastructure instead of guessing or hallucinating.\nBefore MCP, asking Claude about your homelab meant pasting terminal output into a chat window. MCP flips that: Claude connects directly to your services via MCP servers and executes real queries against live data. You ask \u0026ldquo;What containers are running on my Docker host?\u0026rdquo; and Claude calls a Docker API endpoint, formats the response, and shows you the result.\nThe Homelab MCP project bundles seven specialized MCP servers into a single Docker container. Deploy it once, configure Claude Desktop, and you have natural language access to Docker containers, Ollama AI models, Pi-hole DNS stats, Unifi network devices, UPS power status, Ansible inventory, and ICMP connectivity checks.\nPrerequisites: Docker Engine, Claude Desktop (free tier works), and at least one homelab service you want to monitor.\nWhat the Homelab MCP Server Does The unified container runs all seven MCP servers in a single process. Each server exposes namespaced tools so Claude knows which service to query:\nServer Tools What You Can Ask Docker/Podman docker_get_containers, docker_inspect \u0026ldquo;List all running containers\u0026rdquo; Ollama ollama_list_models, ollama_ps \u0026ldquo;What models are loaded?\u0026rdquo; Pi-hole pihole_get_stats, pihole_get_query_types \u0026ldquo;Top DNS queries in the last hour\u0026rdquo; Unifi unifi_list_devices, unifi_list_clients \u0026ldquo;Show me all connected clients\u0026rdquo; UPS/NUT nut_get_status, nut_get_battery \u0026ldquo;What is the UPS battery level?\u0026rdquo; Ansible ansible_list_hosts, ansible_get_groups \u0026ldquo;Show my Ansible inventory summary\u0026rdquo; Ping ping_ping_host \u0026ldquo;Ping my Proxmox host\u0026rdquo; The unified mode uses namespaced tool names (e.g., docker_get_containers) so there is no collision between servers. This is the recommended mode and the only one supported for Docker deployments.\nDocker Deployment Pull the image and set up a compose file. The container is available on Docker Hub:\n1 docker pull bjeans/homelab-mcp:latest Docker Compose Configuration Create docker-compose.yml in a dedicated directory:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 services: homelab-mcp: image: bjeans/homelab-mcp:latest container_name: homelab-mcp restart: unless-stopped env_file: - .env volumes: - /var/run/docker.sock:/var/run/docker.sock:ro - ./ansible_hosts.yml:/app/ansible_hosts.yml:ro networks: - homelab-mcp # No ports exposed — Claude Desktop connects via # Docker\u0026#39;s internal networking or host-mode transport networks: homelab-mcp: name: homelab-mcp internal: true The internal: true network ensures the MCP server cannot be reached from outside the host. Claude Desktop communicates with the container through Docker\u0026rsquo;s STDIO transport or a bind-mount socket — not over the network.\nIf your Docker host has a Unix socket for Docker (the default), mount it read-only. The container uses this to query container status without needing elevated permissions on the host.\nStarting the Container 1 2 docker compose up -d docker compose logs -f # Verify it started cleanly A successful start logs:\n[INFO] Homelab MCP Unified Server v3.0.0 starting... [INFO] Registered 7 server modules: docker, ollama, pihole, unifi, nut, ansible, ping [INFO] Using stdio transport [INFO] Server initialized and ready Environment Configuration The .env file is the single configuration point. Copy the template from the repository and fill in your service endpoints:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 # Required — Docker socket path (inside container) DOCKER_SOCKET=/var/run/docker.sock # Optional — Ollama OLLAMA_HOST=http://10.0.20.30:11434 # Optional — Pi-hole PIHOLE_HOST=http://10.0.20.40 PIHOLE_API_KEY=your-pihole-api-key-here # Optional — Unifi controller UNIFI_HOST=https://10.0.20.50:8443 UNIFI_USER=homelab-monitor UNIFI_PASS=readonly-api-user-password # Optional — NUT UPS server NUT_HOST=10.0.20.60 NUT_PORT=3493 NUT_UPS_NAME=ups # Optional — Ansible inventory (mounted volume) ANSIBLE_INVENTORY=/app/ansible_hosts.yml Only DOCKER_SOCKET is required. Each service you leave out simply means Claude won\u0026rsquo;t have that tool available. Start with Docker and Pi-hole, then add the rest.\nGenerate a fresh Pi-hole API key from the Pi-hole admin interface under Settings \u0026gt; API \u0026gt; Show API Token. For Unifi, create a dedicated read-only local user on the controller.\nAnsible Inventory (Optional but Useful) If you use Ansible to manage your homelab, mount your inventory file. The MCP server reads it to provide Claude with host names, IP addresses, groups, and variables:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 # ansible_hosts.yml all: hosts: srv1-proxmox: ansible_host: 10.0.20.30 srv2-docker: ansible_host: 10.0.20.31 mikrotik-gw: ansible_host: 10.0.20.1 children: proxmox: hosts: srv1-proxmox: docker: hosts: srv2-docker: network: hosts: mikrotik-gw: Claude can now answer \u0026ldquo;What hosts are in my Proxmox group?\u0026rdquo; or \u0026ldquo;What is the IP of my Docker host?\u0026rdquo;\nClaude Desktop Integration Configure Claude Desktop to connect to the MCP server. Open Claude Desktop settings and edit the MCP configuration file:\nmacOS: ~/Library/Application Support/Claude/claude_desktop_config.json Windows: %APPDATA%\\Claude\\claude_desktop_config.json Linux: ~/.config/Claude/claude_desktop_config.json\nFor the Docker deployment, Claude connects via Docker exec:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 { \u0026#34;mcpServers\u0026#34;: { \u0026#34;homelab\u0026#34;: { \u0026#34;command\u0026#34;: \u0026#34;docker\u0026#34;, \u0026#34;args\u0026#34;: [ \u0026#34;exec\u0026#34;, \u0026#34;-i\u0026#34;, \u0026#34;homelab-mcp\u0026#34;, \u0026#34;python\u0026#34;, \u0026#34;/app/homelab_unified_mcp.py\u0026#34; ] } } } This tells Claude to start the MCP server inside the running container. The -i flag keeps STDIN open for the MCP protocol transport. No network ports, no firewall rules — the communication happens through the Docker CLI.\nRestart Claude Desktop after adding the configuration. You should see a hammer icon appear in the chat input area, indicating MCP tools are available.\nTesting the Connection Type \u0026ldquo;What tools do you have available?\u0026rdquo; in Claude Desktop. If the connection is working, Claude lists all registered tools:\nI have access to these tools: - docker_get_containers — List Docker containers - docker_inspect — Inspect a specific container - ollama_list_models — List loaded Ollama models - pihole_get_stats — Get Pi-hole statistics - nut_get_status — Get UPS status - ping_ping_host — Ping a host Real-World Usage Examples Once connected, Claude can run live queries against your infrastructure. Here are practical commands to try:\nContainer management:\nShow me all Docker containers, their status, and resource usage. Claude calls docker_get_containers and returns a formatted table with container names, images, status, ports, and uptime.\nDNS health check:\nWhat are the top DNS queries on Pi-hole right now? Any blocked domains spiking? Claude calls pihole_get_stats and pihole_get_query_types to show query volume, blocked percentage, and top domains.\nAI model status:\nCheck if Ollama is running and list which models are loaded. Claude runs ollama_ps to show loaded models and their sizes.\nNetwork diagnostics:\nPing my Proxmox host and my MikroTik gateway. Are they reachable? Claude calls ping_ping_host sequentially for each target and reports latency.\nInfrastructure overview:\nShow me a summary of my Ansible inventory with groups and host IPs. Claude queries the Ansible server and returns your infrastructure map.\nUPS monitoring:\nWhat is the battery level on my UPS? How much runtime is left? Claude calls nut_get_status and nut_get_battery for real-time power data.\nSecurity Considerations The Homelab MCP server has access to your Docker socket, which is effectively root access to the host. Treat this deployment with the same care as any privileged container:\nNever expose the Docker socket to the network. A read-only bind mount does not restrict Docker API methods. Direct socket access can control the host; use a reviewed API authorization boundary and keep it private. Use a read-only API key for Pi-hole and Unifi. Do not use admin credentials. Homelab MCP only queries data — it does not modify configurations. Keep .env out of version control. Add it to .gitignore. It contains API keys and credentials for your infrastructure. Run on a management VLAN. Place the Docker host in a dedicated management VLAN (VLAN 99 or similar) with strict ACLs limiting which services the MCP container can reach. Audit the Docker socket. The container runs docker_get_containers against your entire local Docker API. Consider running a Docker Socket Proxy for granular access control instead of mounting /var/run/docker.sock directly. Monitor access logs. Claude Desktop conversations run locally, but if you use Claude Pro or Teams, your queries transit Anthropic\u0026rsquo;s servers. Do not include secrets in prompts. Troubleshooting Claude Desktop shows no MCP tools available. Verify the container is running: docker ps | grep homelab-mcp. Check logs: docker compose logs. Ensure the claude_desktop_config.json path is correct and the JSON is valid.\n\u0026ldquo;Permission denied\u0026rdquo; on Docker socket. The user running Docker must have access. Run sudo usermod -aG docker $USER and log out/in. Or run Claude Desktop with the user that has Docker access.\nContainer exits immediately. Missing or incorrect .env variables. The container validates required configs on startup. Run docker compose logs — missing DOCKER_SOCKET is the most common cause.\nPi-hole returns empty data. Check that PIHOLE_API_KEY is correct and the Pi-hole API is reachable from the Docker container. The container runs on the internal network — if Pi-hole is on a different Docker network, they may not connect. Use docker compose exec homelab-mcp curl http://pihole-ip/admin/api.php to test.\nClaude says \u0026ldquo;I don\u0026rsquo;t have access to that yet.\u0026rdquo; You asked for a tool from a server not configured. Only tools from services with valid .env entries are registered. Check your .env and restart the container.\nWhat This Changes About Homelab Management Before MCP, homelab monitoring meant dashboards. Grafana panels, Prometheus alerts, Uptime Kuma status pages — all visual, all pull-based. You stare at charts and wait for something red.\nMCP inverts the model. You ask a question in natural language and Claude goes to the source, queries the live data, and returns an answer. No dashboard navigation, no SQL queries, no docker ps | grep chains. It is querying your infrastructure the same way you query a database — but the database is your entire homelab.\nThe Homelab MCP project is actively maintained and the server count grows with each release. Start with Docker and Pi-hole, add Unifi and Ansible as you get comfortable, and before long you will find yourself reaching for Claude instead of SSH to check on your homelab.\nFull stack summary:\nHomelab MCP — unified Docker container with 7 MCP servers Claude Desktop — natural language interface for infrastructure queries Docker — runtime for the MCP container; a socket bind mount is not read-only API authorization Optional: Pi-hole, Ollama, Unifi, NUT, Ansible — each adds more tools Deploy it tonight. Tomorrow morning ask Claude \u0026ldquo;Did anything interesting happen in my homelab while I was asleep?\u0026rdquo; and watch it query Pi-hole, check Docker containers, ping your hosts, and report back. That is the future of homelab management — it asks you less and answers you more.\n","permalink":"https://blog.gntech.me/posts/2026-06-03-homelab-mcp-server-docker-deployment/","summary":"Deploy the Homelab MCP server in Docker to manage your homelab infrastructure through Claude Desktop. Monitor Docker containers, Ollama AI models, Pi-hole DNS, and network devices using natural language commands.","title":"Homelab MCP Server — AI-Powered Infrastructure via Claude Desktop"},{"content":"Why Fail2ban Belongs in Every Homelab Within hours of exposing any SSH server or web application to the internet, you will see failed login attempts in your logs. Automated scanners cycle through credential lists, targeting SSH, web login pages, and API endpoints. Fail2ban is the simplest and most effective first line of defense — it reads log files, detects repeated failures, and temporarily bans the offending IP using the host firewall.\nI run Fail2ban on every homelab host alongside Docker containers managed by Traefik. This guide covers the complete setup: SSH jail, Docker log integration for Traefik and Nginx, custom jails for specific applications, email alerts, and management commands.\nWhat you will end up with:\nSSH brute-force attempts blocked after 5 failures Traefik and Nginx authentication failures banned automatically Custom jails for any service that writes to logs Email notifications when bans occur Prerequisites A Debian or Ubuntu host (works on any Linux distribution) SSH server running on the host Docker with containers whose logs you want to monitor (Traefik, Nginx, etc.) Root or sudo access Installing Fail2ban Installation is straightforward on any Debian-based system:\n1 2 3 sudo apt update sudo apt install fail2ban -y sudo systemctl enable --now fail2ban Verify the service is running:\n1 2 sudo systemctl status fail2ban sudo fail2ban-client ping A successful ping returns pong.\nFail2ban ships with a default configuration file at /etc/fail2ban/jail.conf. Never edit this file directly — package updates will overwrite it. Instead, create a local override:\n1 2 sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local sudo nano /etc/fail2ban/jail.local The jail.local file overrides the defaults. Set global values first, then configure individual jails.\nSSH Jail Configuration The SSH jail is the most important one. It monitors /var/log/auth.log for authentication failures and bans IPs that exceed the threshold.\nConfigure it in /etc/fail2ban/jail.local:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 [DEFAULT] bantime = 1h findtime = 10m maxretry = 5 bantime.increment = true bantime.factor = 2 banaction = nftables-multiport action = %(action_mwl)s [sshd] enabled = true port = ssh logpath = %(sshd_log)s maxretry = 5 Key settings explained:\nbantime = 1h — initial ban duration bantime.increment = true — doubles ban time for repeat offenders (1h → 2h → 4h → \u0026hellip;) banaction = nftables-multiport — uses nftables instead of legacy iptables (default on modern Debian/Ubuntu) action = %(action_mwl)s — sends email with whois and log lines (requires MTA, see below) Activate and check:\n1 2 sudo fail2ban-client reload sudo fail2ban-client status sshd The status output shows total bans, currently banned IPs, and the log file being monitored.\nDocker Log Integration with Traefik Jail Docker containers write logs to journald or to a JSON file under /var/lib/docker/containers/ by default — not where Fail2ban expects them. You have two options:\nOption A: Configure the container\u0026rsquo;s logging driver to use local files.\nAdd a bind mount in your Traefik compose file so the access log is accessible to Fail2ban on the host:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 services: traefik: image: traefik:v3.3 container_name: traefik command: - \u0026#34;--providers.docker\u0026#34; - \u0026#34;--entrypoints.websecure.address=:443\u0026#34; - \u0026#34;--accesslog=true\u0026#34; - \u0026#34;--accesslog.filepath=/var/log/traefik/access.log\u0026#34; ports: - \u0026#34;443:443\u0026#34; - \u0026#34;80:80\u0026#34; volumes: - /var/run/docker.sock:/var/run/docker.sock - /var/log/traefik:/var/log/traefik Create the log directory on the host:\n1 2 sudo mkdir -p /var/log/traefik sudo chown 1000:1000 /var/log/traefik # match traefik UID or use world-readable Option B: Use syslog or journald logging driver.\nIf you prefer not to write log files inside containers, configure journald as the log driver for Traefik:\n1 2 3 4 5 6 services: traefik: image: traefik:v3.3 logging: driver: journald # Preserve other existing service settings. Then use the journald backend in Fail2ban:\n1 2 3 4 5 6 7 [traefik-auth] enabled = true backend = journald journalmatch = CONTAINER_NAME=traefik maxretry = 5 findtime = 10m bantime = 1h I recommend Option A — plain log files are easier to debug and work with fail2ban-regex for testing.\nTraefik Jail Configuration Create a jail configuration for Traefik:\n1 sudo nano /etc/fail2ban/jail.d/traefik.conf 1 2 3 4 5 6 7 [traefik-auth] enabled = true port = http,https logpath = /var/log/traefik/access.log maxretry = 5 findtime = 10m bantime = 1h Now create the filter file. Traefik access logs have a different format from standard Nginx or Apache logs:\n1 sudo nano /etc/fail2ban/filter.d/traefik-auth.conf 1 2 3 4 [Definition] failregex = ^\u0026lt;HOST\u0026gt; - - \\[.*\\] \u0026#34;\\w+ /.*\u0026#34; 401 .*$ ^\u0026lt;HOST\u0026gt; - - \\[.*\\] \u0026#34;\\w+ /.*\u0026#34; 403 .*$ ignoreregex = This regex matches Traefik access log lines with HTTP 401 (Unauthorized) or 403 (Forbidden) status codes. These are the responses Traefik returns when authentication middleware rejects a request.\nTest the regex with your actual logs:\n1 sudo fail2ban-regex /var/log/traefik/access.log /etc/fail2ban/filter.d/traefik-auth.conf Reload and verify:\n1 2 sudo fail2ban-client reload sudo fail2ban-client status traefik-auth Nginx Jail for Docker Containers If you use Nginx as a reverse proxy instead of Traefik, the setup is similar — bind mount the Nginx log directory to the host.\nExample Nginx compose log mount:\n1 2 3 4 5 services: nginx: image: nginx:alpine volumes: - /var/log/nginx:/var/log/nginx Jail configuration (uses the built-in filter):\n1 2 3 4 5 6 7 [nginx-http-auth] enabled = true port = http,https logpath = /var/log/nginx/error.log maxretry = 5 findtime = 10m bantime = 1h Nginx also has a built-in filter for nginx-botsearch — useful for blocking bots that probe for vulnerabilities like wp-admin, phpmyadmin, or .env files:\n1 2 3 4 5 6 7 [nginx-botsearch] enabled = true port = http,https logpath = /var/log/nginx/access.log maxretry = 2 findtime = 5m bantime = 24h The aggressive maxretry = 2 ensures bots get banned after just two suspicious requests.\nCreating Custom Jails Any service that writes failure events to a log file can be protected. The recipe is always the same:\nDetermine the log file path and format Write a regex to match failure lines Create the jail and filter files Test with fail2ban-regex Example: Custom Jail for a Web Application Suppose you run Nextcloud in Docker and want to ban IPs that fail authentication:\n1 sudo nano /etc/fail2ban/filter.d/nextcloud-auth.conf 1 2 3 4 [Definition] failregex = ^.*Login failed: \u0026#39;.*\u0026#39; \\(Remote IP: \u0026#39;\u0026lt;HOST\u0026gt;\u0026#39;\\).*$ ^.*Trusted domain error\\. \\\u0026#34;\u0026lt;HOST\u0026gt;\\\u0026#34; is not trusted.*$ ignoreregex = 1 sudo nano /etc/fail2ban/jail.d/nextcloud.conf 1 2 3 4 5 6 7 [nextcloud-auth] enabled = true port = http,https logpath = /path/to/nextcloud/data/nextcloud.log maxretry = 5 findtime = 10m bantime = 1h Always test before reloading:\n1 sudo fail2ban-regex /path/to/nextcloud/data/nextcloud.log /etc/fail2ban/filter.d/nextcloud-auth.conf The tool shows matched lines, unmatched lines, and a summary — invaluable for debugging regex patterns.\nEmail Alerts and Management The %(action_mwl)s action sends an email with whois information and the most recent log lines from the offending IP. This requires a working MTA on the host.\nInstall msmtp for Lightweight Email For a minimal MTA that works without a full mail server:\n1 sudo apt install msmtp msmtp-mta -y Configure /etc/msmtprc:\ndefaults auth on tls on tls_trust_file /etc/ssl/certs/ca-certificates.crt account default host smtp.gmail.com port 587 from your-homelab@example.com user your-homelab@example.com password your-app-password Test it:\n1 echo \u0026#34;Fail2ban test\u0026#34; | msmtp you@example.com Then in /etc/fail2ban/jail.local, set the sender and recipient:\n1 2 3 4 5 6 [DEFAULT] action = %(action_mwl)s sendername = Fail2ban (homelab) sender = fail2ban@homelab.local mta = msmtp destemail = you@example.com Fail2ban Management Commands Essential fail2ban-client commands for day-to-day management:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 # Check status of all jails sudo fail2ban-client status # Check specific jail sudo fail2ban-client status sshd # Unban an IP from all jails sudo fail2ban-client unban 192.168.1.100 # Unban from a specific jail sudo fail2ban-client set sshd unbanip 192.168.1.100 # Ban an IP manually sudo fail2ban-client set sshd banip 192.168.1.100 # Reload configuration sudo fail2ban-client reload # View the log sudo tail -f /var/log/fail2ban.log Testing Your Setup Test the SSH jail by intentionally failing SSH logins:\n1 2 ssh invalid@localhost # Enter wrong password 6 times Then check:\n1 sudo fail2ban-client status sshd You should see the localhost IP (or your IP) listed in the \u0026ldquo;Banned IP list\u0026rdquo;. Remember to unban it afterward:\n1 sudo fail2ban-client unban 127.0.0.1 Test the Traefik jail by hitting a protected endpoint with an invalid token or no authentication:\n1 2 3 4 curl -k https://your-service.example.com/ # Should return 401 or 403 # Repeat 5+ times sudo fail2ban-client status traefik-auth Monitoring Fail2ban Check per-jail state through the installed client and logs:\n1 2 3 sudo fail2ban-client status sudo fail2ban-client status sshd sudo journalctl -u fail2ban --since \u0026#39;1 hour ago\u0026#39; --no-pager Use your actual jail name in place of sshd. If you add Prometheus integration, select a maintained exporter, verify its image exists, and review the permissions it needs. The previously linked exporter repository and image could not be verified and have been removed; this article does not provide a tested Prometheus exporter deployment.\nPutting It All Together Fail2ban is a essential security layer for any homelab exposed to the internet. It integrates cleanly with Docker containers through bind-mounted log files, uses nftables for efficient firewall rules, and keeps you informed through email alerts.\nKey takeaways:\nInstall and configure the SSH jail first — it is the highest-impact change Use bind mounts or journald for Docker container log access Always test new jails with fail2ban-regex before enabling them Start with conservative thresholds (maxretry = 5, bantime = 1h) and tighten over time Combine with CrowdSec, Traefik rate limiting, and firewall rules for defense in depth Your homelab will thank you the first time you see hundreds of banned IPs piling up in the fail2ban log instead of filling your auth log.\n","permalink":"https://blog.gntech.me/posts/2026-06-03-fail2ban-docker-deployment-homelab/","summary":"Protect your homelab from brute-force attacks with Fail2ban. Deploy on bare-metal for SSH and configure custom jails for Docker containers using Traefik and Nginx logs with email alerting.","title":"Fail2ban Docker Deployment — SSH and Service Protection for Homelab"},{"content":"Why SNMP Monitoring Matters for Homelab Networking You already monitor your Docker hosts and Proxmox server with Prometheus and Grafana. But what about the network devices that make it all work — the MikroTik router, the managed switch, the access points, the UPS? When a switch port starts dropping packets or the router CPU hits 100% under load, you need to know before users notice.\nSNMP (Simple Network Management Protocol) is the universal language network hardware speaks. Every managed switch, router, access point, and UPS exposes metrics via SNMP — interface traffic counters, CPU load, memory usage, temperature sensors, and more.\nThe Prometheus SNMP Exporter translates those SNMP OID walks into Prometheus metrics, which means you can add your entire network to your existing Grafana dashboards without running a separate monitoring stack. A single Docker Compose service bridges the gap between your network hardware and Prometheus.\nWhat you can monitor with this stack:\nInterface bandwidth in/out (bps) and packet rates per port Interface errors, discards, and CRC failures CPU utilization and load averages on routers and switches Memory usage (total, used, free) System uptime and reachability UPS battery charge, runtime, and load percentage Sensor temperature readings Prerequisites Before deploying, make sure you have:\nDocker and Docker Compose on a host in your homelab (Proxmox LXC or dedicated host) A running Prometheus + Grafana stack — either the full Prometheus + Grafana + Loki stack from the monitoring guide or a minimal setup One or more SNMP-enabled devices — MikroTik RouterOS router, managed switch, or any device exposing SNMP SNMP credentials — a community string (v2c) or username/auth phrase (v3) Enabling SNMP on MikroTik RouterOS MikroTik RouterOS supports SNMP v1, v2c, and v3 out of the box. For a homelab on a trusted network, v2c with a non-default community string is acceptable. For production or any device exposed to untrusted networks, use v3 with authentication and encryption.\nSNMP v2c Configuration Connect via SSH or Winbox and run these commands:\n/ip service set snmp disabled=no /snmp community set [find] name=monitoring address=10.0.0.0/24 /snmp set enabled=yes contact=\u0026#34;admin@gntech.me\u0026#34; location=\u0026#34;SRV1 Rack\u0026#34; This does three things:\nEnables the SNMP IP service on the router Creates (or updates) a community string monitoring restricted to your management subnet 10.0.0.0/24 Enables the SNMP agent with contact and location metadata If this is a fresh RouterOS installation and no community exists yet, create one explicitly:\n/snmp community add name=monitoring address=10.0.0.0/24 /snmp set enabled=yes SNMP v3 Configuration (Recommended for Production) SNMP v3 provides authentication (SHA) and encryption (AES). RouterOS supports it natively:\n/snmp set enabled=yes /snmp community set [find] name=monitoring /user add name=snmp-monitor group=read password=\u0026#34;your-secure-password\u0026#34; /snmp security add user=snmp-monitor group=read authentication=sha1 encryption=aes verify=yes The security entry ties the local user snmp-monitor to authentication and encryption methods. Prometheus SNMP Exporter needs matching v3 parameters in its config.\nVerify SNMP Reachability From your Docker host or any Linux machine on the management network, test with snmpwalk:\n1 2 3 4 5 6 7 8 # Install snmp tools if needed sudo apt install snmp -y # Test SNMP v2c — should return system information snmpwalk -v 2c -c monitoring 10.0.20.1 .1.3.6.1.2.1.1 # Test interface counters specifically snmpwalk -v 2c -c monitoring 10.0.20.1 .1.3.6.1.2.1.2.2 A successful walk returns OID trees with values. If it hangs or returns no data, check:\nThe firewall on the MikroTik — UDP port 161 must be reachable from your monitoring host The community string and subnet restriction That SNMP is enabled (/snmp print) Docker Compose Deployment The Prometheus SNMP Exporter runs as a single Docker container. Create a project directory and add this configuration:\n1 2 3 4 5 6 7 8 9 10 11 12 # docker-compose.yml services: snmp-exporter: image: prom/snmp-exporter:v0.27.0 container_name: snmp-exporter restart: unless-stopped ports: - \u0026#34;9116:9116\u0026#34; volumes: - ./snmp.yml:/etc/snmp_exporter/snmp.yml:ro command: - \u0026#34;--config.file=/etc/snmp_exporter/snmp.yml\u0026#34; Deploy with:\n1 docker compose up -d The exporter exposes metrics on port 9116. Verify it is running by hitting the web endpoint:\n1 curl http://localhost:9116/metrics You should see internal Prometheus process metrics — this confirms the exporter is healthy but has no device targets yet. Targets are specified as query parameters in Prometheus scrape configs, not in the exporter itself.\nGenerating the SNMP Exporter Configuration The SNMP Exporter uses a YAML configuration file (snmp.yml) that maps OID trees to Prometheus metric names and labels. Writing this by hand is tedious and error-prone, so the upstream project provides a generator that compiles a human-readable generator.yml into the optimized snmp.yml.\nPull and Run the Generator Use Docker to run the generator against the same image version:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 # Create a generator configuration file cat \u0026gt; generator.yml \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; modules: if_mib: walk: - 1.3.6.1.2.1.2 # interfaces - 1.3.6.1.2.1.31.1.1 # ifXTable (high capacity counters) - 1.3.6.1.2.1.10 # transmission version: 2 auth: community: monitoring lookups: - source_indexes: [ifIndex] lookup: 1.3.6.1.2.1.2.2.1.2 # ifDescr - source_indexes: [ifIndex] lookup: 1.3.6.1.2.1.31.1.1.1.1 # ifName overrides: ifType: type: EnumAsInfo ifAdminStatus: type: EnumAsInfo ifOperStatus: type: EnumAsInfo mikrotik: walk: - 1.3.6.1.4.1.14988.1 # MikroTik MIB (CPU, memory, uptime) - 1.3.6.1.4.1.14988.2 # MikroTik interfaces - 1.3.6.1.2.1.25 # host resources (storage) version: 2 auth: community: monitoring EOF Now generate snmp.yml:\n1 2 3 4 docker run --rm \\ -v \u0026#34;${PWD}:/opt\u0026#34; \\ prom/snmp-generator:v0.27.0 \\ generate -m /opt/generator.yml -o /opt/snmp.yml This produces snmp.yml in the current directory — an optimized metric mapping file ready to mount into the exporter.\nUnderstanding the Modules The generator.yml defines two modules:\nif_mib — The standard interfaces MIB (RFC 1213 / RFC 2863). Walks interface counters, descriptions, and operational status. Works on any SNMP-capable device — switches, routers, APs, NAS devices. The lookups section ensures Prometheus labels like ifDescr and ifName are resolved from index numbers into human-readable interface names.\nmikrotik — The MikroTik private MIB (1.3.6.1.4.1.14988). Provides RouterOS-specific metrics: CPU load, total memory and used memory, storage usage, uptime, and BIOS-level hardware info. This module only works on RouterOS devices.\nPrometheus Scrape Configuration Add the SNMP targets to your prometheus.yml scrape configuration:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 scrape_configs: - job_name: snmp scrape_interval: 60s scrape_timeout: 30s static_configs: - targets: - 10.0.20.1 # MikroTik CCR or hAP router - 10.0.20.2 # MikroTik CRS switch - 10.0.20.5 # APC UPS metrics_path: /snmp params: module: [if_mib] relabel_configs: - source_labels: [__address__] target_label: __param_target - source_labels: [__param_target] target_label: instance - target_label: __address__ replacement: snmp-exporter:9116 # Docker Compose service name - job_name: mikrotik_system scrape_interval: 120s scrape_timeout: 30s static_configs: - targets: - 10.0.20.1 metrics_path: /snmp params: module: [mikrotik] relabel_configs: - source_labels: [__address__] target_label: __param_target - source_labels: [__param_target] target_label: instance - target_label: __address__ replacement: snmp-exporter:9116 This configuration separates interface metrics (every 60 seconds) from RouterOS-specific metrics (every 120 seconds) into two jobs. Each job:\nSets the target IP via __param_target (the device to query) Labels the metric with the target IP for identification Redirects the scrape to the SNMP Exporter container on port 9116 If your SNMP Exporter runs on a different host, replace snmp-exporter:9116 with \u0026lt;host-ip\u0026gt;:9116.\nReload Prometheus after updating the config:\n1 curl -X POST http://localhost:9090/-/reload Or restart the Prometheus container if HTTP reload is not configured. Check the Prometheus targets page (http://prometheus:9090/targets) to confirm both jobs show UP.\nGrafana Dashboard — Visualizing SNMP Metrics With metrics flowing into Prometheus, build Grafana panels to visualize your network.\nImport a Pre-Built Dashboard Go to Dashboards → New → Import in Grafana. The most popular SNMP dashboard for Prometheus is SNMP Interface Detail (ID: 12492). Paste the ID and select your Prometheus data source.\nThis dashboard provides:\nInterface traffic line charts (bits in/out per second) Packet rate panels per interface Error and discard counters Interface status table (up/down/admin) Creating a MikroTik System Panel For RouterOS-specific metrics, create a custom panel:\nClick Dashboards → New Dashboard → Add Visualization Select Prometheus as the data source Use the Prometheus Explore tab (http://prometheus:9090/explore) to discover the exact metric names from the generated snmp.yml. Query with label matchers to find MikroTik-specific metrics: 1 {__name__=~\u0026#34;.*cpu.*\u0026#34;,instance=\u0026#34;10.0.20.1\u0026#34;} Common RouterOS metrics exposed by the MikroTik module include:\nsnmp_mikrotik_cpuLoad — CPU utilization percentage snmp_mikrotik_totalMemory and snmp_mikrotik_usedMemory — memory in bytes snmp_mikrotik_totalHddSpace and snmp_mikrotik_usedHddSpace — storage usage For interface traffic, use the if_mib module metrics:\n1 2 3 4 5 # Interface bits in per second (per interface) rate(snmp_ifHCInOctets{ifDescr=~\u0026#34;ether1|sfp-plus1|bridge\u0026#34;}[5m]) * 8 # Interface bits out per second rate(snmp_ifHCOutOctets{ifDescr=~\u0026#34;ether1|sfp-plus1|bridge\u0026#34;}[5m]) * 8 Add units → bps (bits/sec) in the panel configuration for human-readable axis labels.\nAdvanced Configuration SNMP v3 for Encrypted Monitoring For environments where network traffic traverses untrusted segments, use SNMP v3. Update the generator.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 modules: if_mib_v3: walk: - 1.3.6.1.2.1.2 - 1.3.6.1.2.1.31.1.1 version: 3 auth: username: snmp-monitor security_level: authPriv password: \u0026#34;your-auth-password\u0026#34; auth_protocol: SHA priv_protocol: AES priv_password: \u0026#34;your-priv-password\u0026#34; Regenerate snmp.yml with the generator and redeploy the exporter.\nMonitoring Multiple Device Types You can scrape multiple device types with different modules from the same exporter. Add more entries to static_configs in Prometheus:\n1 2 3 4 5 6 static_configs: - targets: - 10.0.20.1 # MikroTik router - 10.0.20.2 # MikroTik switch - 10.0.20.5 # APC UPS - 10.0.20.10 # UniFi AP For UPS units, you may need additional modules like apcups or ups-mib. Add them to generator.yml and regenerate.\nSecuring the Exporter with Traefik If you expose the exporter endpoint externally, run it behind Traefik with authentication middleware:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 services: snmp-exporter: image: prom/snmp-exporter:v0.27.0 container_name: snmp-exporter restart: unless-stopped expose: - \u0026#34;9116\u0026#34; volumes: - ./snmp.yml:/etc/snmp_exporter/snmp.yml:ro networks: - traefik-public labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.snmp.rule=Host(`snmp.gntech.me`)\u0026#34; - \u0026#34;traefik.http.routers.snmp.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.snmp.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.services.snmp.loadbalancer.server.port=9116\u0026#34; - \u0026#34;traefik.http.routers.snmp.middlewares=secHeaders@file,basicAuth@file\u0026#34; networks: traefik-public: external: true Remove the ports mapping — the exporter only needs to be reachable via the internal Docker network and Traefik.\nVerification and Troubleshooting Test the Exporter Endpoint Hit the exporter\u0026rsquo;s metrics endpoint with a target parameter to confirm SNMP communication:\n1 curl \u0026#39;http://localhost:9116/snmp?target=10.0.20.1\u0026amp;module=if_mib\u0026#39; If successful, this returns SNMP metrics from the router wrapped in Prometheus exposition format. If it returns an empty response or errors, the issue is between the exporter and the device.\nCommon Issues and Fixes Problem Likely Cause Fix Error scraping target: context deadline exceeded SNMP request timed out — firewall blocking UDP 161 Check ip firewall filter on MikroTik, allow UDP 161 from monitoring host Empty metric output OID tree has no data — wrong module or community Run snmpwalk -v 2c -c monitoring 10.0.20.1 manually Unknown module error Module name mismatch in Prometheus config Verify module name in snmp.yml matches params.module snmp: authentication failure Wrong community string or v3 credentials Double-check community string case and subnet restriction Exporter not returning MikroTik specific metrics mikrotik module not in snmp.yml Re-run generator with mikrotik module in generator.yml Prometheus target shows DOWN Exporter unreachable or scrape timeout Check exporter container is running: docker ps | grep snmp Using snmpwalk for Diagnostics Always verify device-side SNMP before debugging Prometheus:\n1 2 3 4 5 6 7 8 # General system info snmpwalk -v 2c -c monitoring 10.0.20.1 system # Interface table snmpwalk -v 2c -c monitoring 10.0.20.1 ifTable # MikroTik private MIB snmpwalk -v 2c -c monitoring 10.0.20.1 .1.3.6.1.4.1.14988.1 If snmpwalk works from your host but the exporter gets empty results, the issue is likely in the snmp.yml module definition.\nSummary You now have a complete SNMP monitoring pipeline for your homelab network devices:\nSNMP enabled on MikroTik RouterOS with v2c (or v3 for production) Prometheus SNMP Exporter deployed as a single Docker Compose service Generated snmp.yml mapping standard IF-MIB and MikroTik-specific MIBs Prometheus scrape config targeting each network device Grafana dashboards visualizing interface traffic, CPU, and memory The best part: this integrates directly into your existing Prometheus and Grafana stack — no separate monitoring platform, no additional databases. Network device metrics coexist alongside container, host, and application metrics in the same dashboards and alerting rules.\nNext steps to expand this stack:\nAdd alerting rules in Prometheus for interface down detection and high CPU Import the UPS MIB module to monitor battery health and runtime Add SNMP monitoring to managed switches, access points, and NAS devices Set up a Grafana alert contact point for Telegram or email notifications on interface failures ","permalink":"https://blog.gntech.me/posts/2026-06-01-prometheus-snmp-exporter-mikrotik-monitoring/","summary":"Deploy Prometheus SNMP Exporter with Docker Compose to collect bandwidth, CPU, memory, and interface metrics from MikroTik routers and network devices. Configure SNMP v2c/v3, generate the exporter config, and visualize everything in Grafana.","title":"Prometheus SNMP Exporter — Monitoring MikroTik and Network Devices"},{"content":"Why MinIO for Homelab Object Storage The S3 API has become the universal interface for object storage. From backup tools like restic and duplicati, to applications like Immich, Mattermost, and Grafana Loki — everything speaks S3. Running a MinIO instance in your homelab gives you that API locally without depending on AWS, Wasabi, or Backblaze.\nMinIO is a high-performance, S3-compatible object store written in Go. It runs as a single binary or Docker container, supports erasure coding for data protection, and ships with a web UI for management. Unlike hosted S3, you control the data, there are no egress fees, and it works perfectly on a single server or a cluster of homelab nodes.\nKey features for homelab use:\nS3 API compatibility (works with all S3 SDKs and CLI tools) Built-in web console for management Erasure coding with configurable parity (N/2 and N/4) Bucket versioning and object locking (WORM) Identity management with IAM policies Prometheus metrics for monitoring Lightweight — ~50 MB RAM at idle Use cases in a homelab include:\nBackup target for restic, borg, duplicati Application storage for Immich photos, Mattermost files, Grafana Loki chunks Static file hosting with public bucket policies S3-compatible test environment before deploying to production AWS Docker Compose Single-Node Deployment MinIO is designed to run on a single machine or a cluster. For most homelabs, a single-node deployment with one or more drives is sufficient.\nBasic Docker Compose Setup Create a directory and add this docker-compose.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 services: minio: image: quay.io/minio/minio:latest container_name: minio restart: unless-stopped command: server /data --console-address \u0026#34;:9001\u0026#34; ports: - \u0026#34;9000:9000\u0026#34; - \u0026#34;9001:9001\u0026#34; volumes: - ./data:/data environment: MINIO_ROOT_USER: minioadmin MINIO_ROOT_PASSWORD: changeme-minio-2026 healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;curl\u0026#34;, \u0026#34;-f\u0026#34;, \u0026#34;http://localhost:9000/minio/health/live\u0026#34;] interval: 30s timeout: 10s retries: 3 The command line is critical: server /data starts MinIO serving on port 9000 (API), and --console-address \u0026quot;:9001\u0026quot; starts the web UI on port 9001. Without --console-address, the console and API share port 9000 with path-based routing, which is less clean for reverse proxy setups.\nReplace MINIO_ROOT_USER and MINIO_ROOT_PASSWORD with strong credentials. Store these in your password manager — they are the admin keys for the entire instance.\nDeploy with:\n1 docker compose up -d Verify the API is healthy:\n1 curl -s http://localhost:9000/minio/health/live A healthy response is an HTTP 200 with body {\u0026quot;status\u0026quot;:\u0026quot;alive\u0026quot;}.\nTraefik Reverse Proxy Setup Exposing MinIO through your Traefik reverse proxy gets you automatic Let\u0026rsquo;s Encrypt TLS, a clean subdomain, and middleware security. MinIO exposes two services — the S3 API on port 9000 and the Console web UI on port 9001 — so you need two router definitions.\nAdd labels to the compose service and switch to an internal-only port mapping:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 services: minio: image: quay.io/minio/minio:latest container_name: minio restart: unless-stopped command: server /data --console-address \u0026#34;:9001\u0026#34; ports: - \u0026#34;127.0.0.1:9000:9000\u0026#34; - \u0026#34;127.0.0.1:9001:9001\u0026#34; volumes: - ./data:/data environment: MINIO_ROOT_USER: minioadmin MINIO_ROOT_PASSWORD: changeme-minio-2026 MINIO_SERVER_URL: https://s3.gntech.me MINIO_BROWSER_REDIRECT_URL: https://minio.gntech.me networks: - traefik-public labels: - \u0026#34;traefik.enable=true\u0026#34; # S3 API router - \u0026#34;traefik.http.routers.minio-s3.rule=Host(`s3.gntech.me`)\u0026#34; - \u0026#34;traefik.http.routers.minio-s3.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.minio-s3.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.services.minio-s3.loadbalancer.server.port=9000\u0026#34; # Console UI router - \u0026#34;traefik.http.routers.minio-ui.rule=Host(`minio.gntech.me`)\u0026#34; - \u0026#34;traefik.http.routers.minio-ui.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.minio-ui.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.services.minio-ui.loadbalancer.server.port=9001\u0026#34; - \u0026#34;traefik.http.routers.minio-ui.middlewares=secHeaders@file\u0026#34; networks: traefik-public: external: true Two environment variables are important when running behind a reverse proxy:\nMINIO_SERVER_URL — tells MinIO the external S3 API URL. S3 clients need this to generate pre-signed URLs correctly. MINIO_BROWSER_REDIRECT_URL — sets the redirect target for Console authentication callbacks. Without this, MinIO redirects to the internal port after login. After deploying, access the Console at https://minio.gntech.me and the S3 API at https://s3.gntech.me.\nInitial MinIO Configuration Creating Access Keys and Buckets Log into the MinIO Console. The left sidebar has menu sections for Buckets, Access Keys, Identity, and Monitoring.\nCreate an access key:\nGo to Access Keys → Create Access Key Give it a description (e.g., \u0026ldquo;restic-backups\u0026rdquo;) Optionally restrict it to specific buckets or paths Save the generated key and secret — MinIO shows them only once Create a bucket:\nGo to Buckets → Create Bucket Name it (e.g., homelab-backups, immich-photos) Enable Versioning to protect against accidental overwrites Set Object Locking if you need WORM compliance Bucket Access Policies MinIO supports both managed policies and raw S3-style bucket policies. From the Console, you can attach built-in policies:\nPolicy Effect consoleAdmin Full admin access to MinIO itself diagnostics Read-only monitoring access readonly Read-only access to all buckets readwrite Read/write to all buckets writeonly Write-only (log sink pattern) For finer control, use a raw bucket policy in JSON. To make a bucket publicly readable (for static file hosting):\n1 2 3 4 5 6 7 8 9 10 11 { \u0026#34;Version\u0026#34;: \u0026#34;2012-10-17\u0026#34;, \u0026#34;Statement\u0026#34;: [ { \u0026#34;Effect\u0026#34;: \u0026#34;Allow\u0026#34;, \u0026#34;Principal\u0026#34;: {\u0026#34;AWS\u0026#34;: [\u0026#34;*\u0026#34;]}, \u0026#34;Action\u0026#34;: [\u0026#34;s3:GetObject\u0026#34;], \u0026#34;Resource\u0026#34;: [\u0026#34;arn:aws:s3:::public-files/*\u0026#34;] } ] } Paste this into Buckets → [bucket name] → Anonymous in the Console UI.\nMulti-Drive Erasure Coding Setup For production homelabs, running MinIO with multiple disks provides data durability through erasure coding. MinIO splits objects into data and parity shards across drives, tolerating up to half the drives failing.\nDocker Compose with Multiple Volumes Mount each physical or logical drive as a separate volume:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 services: minio: image: quay.io/minio/minio:latest container_name: minio restart: unless-stopped command: server /data{1...4} --console-address \u0026#34;:9001\u0026#34; ports: - \u0026#34;127.0.0.1:9000:9000\u0026#34; - \u0026#34;127.0.0.1:9001:9001\u0026#34; volumes: - /mnt/disk1/minio:/data1 - /mnt/disk2/minio:/data2 - /mnt/disk3/minio:/data3 - /mnt/disk4/minio:/data4 environment: MINIO_ROOT_USER: minioadmin MINIO_ROOT_PASSWORD: changeme-minio-2026 MINIO_SERVER_URL: https://s3.gntech.me The command server /data{1...4} tells MinIO to use all four directories as erasure-coded drives. With 4 drives and default parity (N/2), MinIO tolerates 2 drive failures while still serving reads and writes.\nParity levels by drive count:\n4 drives: EC:4, parity 2 (survives 2 failures) 6 drives: EC:6, parity 2-4 8 drives: EC:8, parity 2-4 12 drives: EC:12, parity 2-6 16 drives: EC:16, parity 2-8 MinIO automatically selects optimal parity when not explicitly specified.\nUsing the MinIO Client (mc) The MinIO Client (mc) is a command-line tool that works with any S3-compatible service. The easiest way to use it in a homelab is with a Docker alias.\nConfigure mc 1 2 3 4 5 # Docker alias for mc alias mc=\u0026#34;docker run --rm -it --network host quay.io/minio/mc\u0026#34; # Set up an alias pointing to your MinIO instance mc alias set homelab https://s3.gntech.me YOUR_ACCESS_KEY YOUR_SECRET_KEY Basic Operations 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # List buckets mc ls homelab # List objects in a bucket mc ls homelab/homelab-backups # Copy a file to S3 mc cp /path/to/file.tar.gz homelab/homelab-backups/ # Mirror an entire directory (useful for backups) mc mirror --watch /local/data homelab/homelab-data/ # Set a bucket policy mc anonymous set public homelab/public-files # Admin info (cluster status, disk usage) mc admin info homelab Scripted Bucket Creation For automated deployments, script bucket and user creation:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 #!/bin/bash # setup-minio.sh — run once after MinIO is deployed mc alias set homelab https://s3.gntech.me \u0026#34;$MINIO_ACCESS_KEY\u0026#34; \u0026#34;$MINIO_SECRET_KEY\u0026#34; # Create buckets mc mb homelab/homelab-backups mc mb homelab/immich-photos mc mb homelab/logs # Enable versioning on backup bucket mc version enable homelab/homelab-backups # Set lifecycle policy — delete objects older than 90 days mc ilm import homelab/homelab-backups \u0026lt;\u0026lt;EOF { \u0026#34;Rules\u0026#34;: [ { \u0026#34;ID\u0026#34;: \u0026#34;expire-old-backups\u0026#34;, \u0026#34;Status\u0026#34;: \u0026#34;Enabled\u0026#34;, \u0026#34;Expiration\u0026#34;: {\u0026#34;Days\u0026#34;: 90} } ] } EOF # Create a restricted user (readwrite for one bucket only) mc admin user add homelab immich-user MySecurePassword123 mc admin policy attach homelab readwrite --user immich-user Backup and Migration Strategies MinIO itself stores S3 data on disk, so you have several backup options.\nUsing mc Mirror for Backup Mirror MinIO data to a local directory for offline backup:\n1 mc mirror homelab/homelab-backups /backups/minio-mirror/ Or to another S3 service (S3-to-S3 replication):\n1 2 mc alias set backblaze https://s3.us-west-004.backblazeb2.com B2_KEY B2_SECRET mc mirror homelab/important-data backblaze/offsite-bucket/ Restic with MinIO Backend If MinIO is your backup target, configure restic with the S3 backend:\n1 2 3 4 5 6 export AWS_ACCESS_KEY_ID=your_minio_access_key export AWS_SECRET_ACCESS_KEY=your_minio_secret_key export RESTIC_REPOSITORY=s3:https://s3.gntech.me/restic-repo restic init restic backup /opt/docker-data MinIO Versioning for Self-Service Recovery Enable bucket versioning for automatic recovery from accidental deletions:\n1 mc version enable homelab/homelab-data List and restore object versions:\n1 2 3 mc ls --versions homelab/homelab-data # Copy a specific version back mc cp homelab/homelab-data/backup.tar.gz --version-id \u0026#34;UUID\u0026#34; /restored/ Volume-Level Backup Stop the container and back up the data directory:\n1 2 3 docker compose stop rsync -av ./data/ /backup/minio-$(date +%Y%m%d)/ docker compose start If your MinIO data directory is on a ZFS dataset, use ZFS snapshots for instant, crash-consistent backups:\n1 2 zfs snapshot tank/minio@backup-$(date +%Y%m%d) zfs send tank/minio@backup-20260601 | ssh backup-server \u0026#34;zfs recv tank/backups/minio\u0026#34; Prometheus Monitoring MinIO exposes Prometheus metrics at the /minio/v2/metrics/cluster endpoint. Configure Prometheus to scrape it:\n1 2 3 4 5 6 7 8 # In prometheus.yml scrape_configs: - job_name: minio metrics_path: /minio/v2/metrics/cluster scheme: https static_configs: - targets: - s3.gntech.me Or scrape directly from Docker metrics if you bind the port:\n1 2 3 4 5 scrape_configs: - job_name: minio static_configs: - targets: - \u0026#34;10.0.20.30:9000\u0026#34; Key metrics to alert on:\nminio_disk_storage_bytes / minio_disk_free_bytes — disk space minio_s3_requests_total — request rate minio_s3_errors_total — error rate minio_heal_objects_total — ongoing healing operations Resource Usage MinIO is efficient on modest hardware:\nResource Idle Under Load (10 concurrent clients) RAM 40–60 MB 200–400 MB CPU \u0026lt; 0.5% 2–5% per core Disk Depends on data stored I/O-bound on writes A single MinIO instance handles hundreds of concurrent S3 clients. For a homelab with 5–10 services using MinIO, the overhead is negligible.\nBest Practices Use strong credentials — MINIO_ROOT_USER / MINIO_ROOT_PASSWORD are the root keys. Do not share them with services. Create restricted access keys per application. Always use HTTPS — MinIO access keys are sent in plain text over HTTP. Run behind Traefik or provide your own TLS certificate. Set MINIO_SERVER_URL correctly — without this, pre-signed URLs generated by MinIO will point to the internal container address and break for clients hitting the reverse proxy. Enable versioning on backup buckets — protects against accidental overwrites or ransomware. Use separate access keys per service — if one service is compromised, revoke its key without affecting others. Monitor disk space — MinIO continues accepting writes until the disk is full. Set up Prometheus alerting or configure mc admin watchdog thresholds. Pin your MinIO version — instead of :latest, use a specific tag like :RELEASE.2026-04-01T00-00-00Z for reproducible deployments. Summary MinIO brings S3-compatible object storage to your homelab with a lightweight Docker deployment. The S3 API is the lingua franca for modern application storage — backup tools, media servers, and analytics pipelines all speak it natively.\nIn this guide you deployed MinIO with Docker Compose, configured Traefik reverse proxy for both the S3 API and Console UI, set up erasure coding for data protection, learned the mc client for management and automation, and established backup strategies including versioning and cross-site replication. Deploy it today and give every service in your homelab its own S3 endpoint.\n","permalink":"https://blog.gntech.me/posts/2026-06-01-minio-docker-object-storage/","summary":"Deploy MinIO with Docker Compose for self-hosted S3-compatible object storage. Covers single-node and multi-drive setups, Traefik reverse proxy, bucket policies, and backup strategies.","title":"MinIO Docker Deployment — Self-Hosted S3-Compatible Object Storage"},{"content":"Why Uptime Kuma for Homelab Monitoring You run Proxmox, a dozen Docker Compose stacks, a MikroTik router, and various self-hosted services. When something goes down — the NAS goes offline, the reverse proxy stops responding, or a database container crashes — you want to know immediately, not when a user complains.\nUptime Kuma is a lightweight, self-hosted monitoring tool that checks your services and sends alerts when they fail. It is a single Node.js application with SQLite persistence, runs in under 100 MB of RAM, and supports more monitor types than most alternatives:\nHTTP/HTTPS monitors with keyword validation Ping and TCP port checks Push monitors (receive health pings from scripts or cron jobs) Certificate expiry tracking (SSL/TLS) Docker container health via socket DNS, Steam Game Server, MySQL, PostgreSQL, Redis, MongoDB and more Compared to hosted solutions like Pingdom or Better Uptime, Uptime Kuma is fully self-contained. You own the data, there are no monthly fees, and the status page can be public or internal.\nDocker Compose Deployment The deployment is a single Docker Compose service. Create a directory and drop in this configuration:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 # docker-compose.yml services: uptime-kuma: image: louislam/uptime-kuma:latest container_name: uptime-kuma restart: unless-stopped ports: - \u0026#34;3001:3001\u0026#34; volumes: - ./data:/app/data - /var/run/docker.sock:/var/run/docker.sock:ro environment: - UPTIME_KUMA_PORT=3001 - UPTIME_KUMA_DISABLE_FRAME_SAMEORIGIN=true healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;node\u0026#34;, \u0026#34;extra/healthcheck.js\u0026#34;] interval: 30s timeout: 10s retries: 3 The ./data volume holds the SQLite database and configuration. The Docker socket bind mount is optional — it gives Uptime Kuma the ability to list running containers and monitor their health states directly. If you use a Docker socket proxy for security (recommended), point the volume to the proxy socket instead:\n1 2 3 volumes: - ./data:/app/data - /var/run/docker-proxy.sock:/var/run/docker.sock:ro Deploy with:\n1 docker compose up -d Uptime Kuma will be available at http://\u0026lt;your-host\u0026gt;:3001 on first run.\nTraefik Reverse Proxy Setup Exposing Uptime Kuma through your Traefik reverse proxy enables HTTPS with Let\u0026rsquo;s Encrypt, clean subdomain URLs, and middleware security. Add labels to the compose service:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 services: uptime-kuma: # ... image, restart, volumes as above networks: - traefik-public labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.uptime.rule=Host(`status.gntech.me`)\u0026#34; - \u0026#34;traefik.http.routers.uptime.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.uptime.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.services.uptime.loadbalancer.server.port=3001\u0026#34; - \u0026#34;traefik.http.routers.uptime.middlewares=secHeaders@file,rateLimit@file\u0026#34; networks: traefik-public: external: true Remove the ports mapping if Uptime Kuma only needs to be reachable through Traefik — the container is still reachable on the internal Docker network:\n1 2 ports: - \u0026#34;3001:3001\u0026#34; # remove this line if using Traefik only After redeploying docker compose up -d, visit https://status.gntech.me to create your admin account.\nFirst Configuration Walkthrough 1. Admin Setup On the first visit, Uptime Kuma prompts you to create an admin account with a username and password. Store this in your password manager — there is no password reset flow built in (back up the SQLite database instead).\n2. Adding Monitors Click Add Monitor and configure:\nType: HTTP(s) for web services Friendly Name: e.g., \u0026ldquo;Proxmox Web UI\u0026rdquo; URL: https://10.0.20.30:8006 Interval: 60 seconds (default) Retries: 0 (notify on first failure) Resend Notification: 5 minutes (aggregate alerts) For internal services behind Traefik, monitor the Traefik-exposed HTTPS URL directly. For services on the Docker network only, use the container name or internal IP.\n3. Certificate Expiry Monitoring Create monitors with type Certificate — enter the domain name and port (default 443). Uptime Kuma will warn you when the certificate is within your configured threshold (default 30 days). This catches expiring Let\u0026rsquo;s Encrypt certificates before they cause outages.\n4. Push Monitors The Push monitor type lets external scripts report their health to Uptime Kuma. This is excellent for cron jobs, systemd timers, and backup scripts. Create a push monitor and Uptime Kuma gives you a unique URL:\nhttps://status.gntech.me/api/push/\u0026lt;YOUR_TOKEN\u0026gt;?status=up\u0026amp;msg=OK Use it in a cron job or systemd timer:\n1 2 3 4 5 6 7 8 #!/bin/bash # /usr/local/bin/push-backup-health.sh BACKUP_STATUS=$(/usr/local/bin/run-backups 2\u0026gt;\u0026amp;1) if [ $? -eq 0 ]; then curl -s -o /dev/null \u0026#34;https://status.gntech.me/api/push/abcd1234?status=up\u0026amp;msg=Backup+OK\u0026#34; else curl -s -o /dev/null \u0026#34;https://status.gntech.me/api/push/abcd1234?status=down\u0026amp;msg=Backup+Failed\u0026#34; fi If the push endpoint has not received a ping within the configured grace period (e.g., 5 minutes), Uptime Kuma marks it as down. This is a great way to monitor:\nNightly database dumps Off-site backups ZFS scrub completion systemd timer executions Telegram Notifications Uptime Kuma supports Telegram, Discord, Slack, Email, Webhook, Gotify, and 20+ other notification channels. To configure Telegram:\nCreate a bot via @BotFather — you get a token like 123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11 Get your chat ID: send a message to the bot, then visit https://api.telegram.org/bot\u0026lt;YOUR_TOKEN\u0026gt;/getUpdates In Uptime Kuma, go to Settings → Notifications → Add Notification → Telegram Enter: Bot Token, Chat ID (negative for groups), and a custom message template Sample message template:\n🔴 {{NAME}} is DOWN at {{HOSTNAME}} Time: {{TIME}} Error: {{ERRORMSG}} Uptime Kuma substitutes these template variables automatically per monitor.\nStatus Page Setup Uptime Kuma includes a built-in public status page. Go to Settings → Status Page, create a new page, and assign monitors to it. Features:\nCustom slug — e.g., status.gntech.me/status or a dedicated subdomain Group monitors — group related services (Infrastructure, Storage, Network, Applications) Custom CSS — match your branding colors Incident history — manually log incidents and maintenance windows Embed badge — SVG badge showing overall uptime percentage For a private homelab, you can keep the status page accessible only through your internal network or Traefik IP whitelist middleware.\nBackup and Maintenance Uptime Kuma stores everything in a single SQLite database at ./data/kuma.db. Backup is trivial:\n1 2 # Backup the database cp ./data/kuma.db ./backups/kuma.db.$(date +%Y%m%d) Restore by stopping the container, replacing the database file, and restarting:\n1 2 3 docker compose down cp ./backups/kuma.db.20260601 ./data/kuma.db docker compose up -d Add a cron job or systemd timer for automated backups. Combine with your existing Docker volume backup strategy (restic, borg, or plain rsync):\n1 0 4 * * * cp /opt/uptime-kuma/data/kuma.db /opt/uptime-kuma/backups/kuma-$(date +\\%Y\\%m\\%d).db To upgrade Uptime Kuma:\n1 2 docker compose pull docker compose up -d The SQLite schema is backwards-compatible across minor versions.\nResource Usage and Best Practices Uptime Kuma is remarkably lightweight. On a Docker host with 30 monitors:\nResource Usage RAM 60–90 MB CPU \u0026lt; 0.5% idle, spikes during checks Disk ~10 MB for SQLite database Network Minimal — single HTTP request per check interval This makes it suitable to run alongside other services on a single Docker host or on a low-powered device like a Raspberry Pi.\nBest practices:\nUse push monitors for cron/systemd timer health — they catch silent failures Monitor from outside your network — run a second Uptime Kuma instance on a cheap VPS for external perspective Keep the SQLite database backed up — the only configuration file you cannot recreate Set resend intervals — 5-15 minutes prevents alert fatigue Combine with Netdata for real-time metrics and Uptime Kuma for availability — they complement each other Use a Docker socket proxy instead of mounting /var/run/docker.sock directly Summary Uptime Kuma delivers production-grade uptime monitoring without the complexity of Prometheus and Alertmanager, and without monthly SaaS fees. A single Docker Compose file gets you HTTP monitoring, certificate tracking, push monitors for cron jobs, and a public status page — all backed by SQLite for zero infrastructure overhead.\nDeploy it today, add your critical services as monitors, configure Telegram alerts, and you will know within 60 seconds any time something in your homelab goes down.\n","permalink":"https://blog.gntech.me/posts/2026-06-01-uptime-kuma-docker-monitoring-dashboard/","summary":"Deploy Uptime Kuma with Docker Compose for real-time homelab monitoring. Configure Traefik reverse proxy, Telegram notifications, status pages, and SQLite-backed persistence in minutes.","title":"Uptime Kuma Docker Deployment — Self-Hosted Monitoring Dashboard"},{"content":"A database backup needs a consistent dump, a usable recovery format, and a restore test. This guide uses PostgreSQL custom-format archives and a separate PostgreSQL instance to check recovered content.\nThe old combined PostgreSQL/MySQL script is replaced with one explicit PostgreSQL workflow. MySQL and MariaDB need their own engine-specific procedures.\nVerification status Area Evidence and limits Technical review Official PostgreSQL and Docker references reviewed on October 31, 2026. Static checks Bash syntax and embedded Compose structure checked. Runtime Recovery lab run 41244956545 passed on October 31, 2026: dump, isolated restore, marker assertion, repeated-restore failure, and stopped-source backup failure. That run covered recovery only. Scheduler Scheduler lab run 41245987064 passed on October 31, 2026: manual service execution, accelerated timer activation, lock-contention failure, stopped-source failure, service result/exit status, and journal error details. Tested with systemd 255.4-1ubuntu8.18. The daily calendar was parsed; CI used a two-second timer override. Tested versions GitHub-hosted Ubuntu 24.04.5 amd64; Docker 28.0.4; Compose 2.38.2; PostgreSQL 16.15. Image digest: sha256:0cefaabd882e42bcb2a2f3dc56d3ce4770560ba62fd7f197f1a2e008fd5731ab. Recovery chain Integrated lab run 41247821482 passed on October 31, 2026: PostgreSQL dump → separately encrypted Restic copy → fresh PostgreSQL database. Original dumps, original repository, and original credentials were unavailable during recovery; restored archive bytes matched and partial files were excluded. Both repositories were local to CI. Restic 0.19.1 image digest: sha256:136600b6ff6843d61d355f7f71f460a166429f35de6fd11b568fece3c9a4d510. Scope Disposable PostgreSQL 16 lab, not a production deployment or point-in-time recovery solution. The repository includes an automated replay of this lab in tools/check-postgres-lab.py. It runs the article\u0026rsquo;s Bash examples on a disposable Docker host, checks recovery, rejects a repeated restore, and confirms a failed dump preserves existing archives. Check the PostgreSQL recovery lab Actions run for runtime evidence; merely adding the test does not establish a passing result. It also tests the generated systemd service, an accelerated timer, and the integrated Restic-to-database recovery chain on a disposable runner. Off-site recovery, external alert delivery, missed-run monitoring, downtime catch-up, and production data remain outside this test.\n1. Prepare a separate lab Use Bash, OpenSSL, Docker Engine, the Compose plugin, and host flock. This project publishes no database ports. Its two services have separate data volumes; only the test service is a restore target.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 db_lab_dir=$(mktemp -d -t postgres-backup-lab.XXXXXX) cd \u0026#34;$db_lab_dir\u0026#34; || exit 1 umask 077 mkdir backups openssl rand -base64 32 \u0026gt; db-password cat \u0026gt; compose.yaml \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; services: source: image: postgres:16-alpine environment: POSTGRES_DB: appdb POSTGRES_USER: lab POSTGRES_PASSWORD_FILE: /run/secrets/db-password secrets: - db-password volumes: - source_data:/var/lib/postgresql/data healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;pg_isready\u0026#34;, \u0026#34;-h\u0026#34;, \u0026#34;127.0.0.1\u0026#34;, \u0026#34;-U\u0026#34;, \u0026#34;lab\u0026#34;, \u0026#34;-d\u0026#34;, \u0026#34;appdb\u0026#34;] interval: 5s timeout: 3s retries: 12 start_period: 10s restore_test: image: postgres:16-alpine environment: POSTGRES_DB: restoredb POSTGRES_USER: lab POSTGRES_PASSWORD_FILE: /run/secrets/db-password secrets: - db-password volumes: - restore_data:/var/lib/postgresql/data healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;pg_isready\u0026#34;, \u0026#34;-h\u0026#34;, \u0026#34;127.0.0.1\u0026#34;, \u0026#34;-U\u0026#34;, \u0026#34;lab\u0026#34;, \u0026#34;-d\u0026#34;, \u0026#34;restoredb\u0026#34;] interval: 5s timeout: 3s retries: 12 start_period: 10s secrets: db-password: file: ./db-password volumes: source_data: restore_data: EOF docker compose config --quiet docker compose up --wait --wait-timeout 120 Use a Compose version supporting the wait flags and inspect logs if startup fails. Record image digests: the version-series tag can change.\nThe official image initializes the lab user on an empty volume. These commands use its local socket from inside the container. Production remote authentication and backup privileges need separate review. See the official image documentation.\n2. Create a recognizable source record 1 2 3 4 docker compose exec -T source psql -X -U lab -d appdb -v ON_ERROR_STOP=1 \u0026lt;\u0026lt;\u0026#39;SQL\u0026#39; CREATE TABLE backup_marker (id integer PRIMARY KEY, value text NOT NULL); INSERT INTO backup_marker VALUES (1, \u0026#39;gntech-restore-check\u0026#39;); SQL This intentionally creates a new table in the disposable source. If you rerun it against the same lab, the existing-table error is expected; inspect the lab state rather than suppressing the error.\n3. Write an atomic backup script 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 cat \u0026gt; backup.sh \u0026lt;\u0026lt;\u0026#39;SCRIPT\u0026#39; #!/usr/bin/env bash set -euo pipefail umask 077 cd -- \u0026#34;$(dirname -- \u0026#34;$0\u0026#34;)\u0026#34; # One instance of this workflow at a time. exec 9\u0026gt;backup.lock flock -n 9 || { echo \u0026#34;Backup already running\u0026#34; \u0026gt;\u0026amp;2; exit 1; } test -d backups partial=$(mktemp \u0026#34;$PWD/backups/appdb-$(date -u +%Y%m%dT%H%M%SZ).XXXXXX.partial\u0026#34;) trap \u0026#39;rm -f -- \u0026#34;$partial\u0026#34;\u0026#39; EXIT docker compose exec -T source pg_dump -U lab -d appdb --format=custom \u0026gt; \u0026#34;$partial\u0026#34; test -s \u0026#34;$partial\u0026#34; # A readable archive index is useful but is not a full restore test. docker compose exec -T source pg_restore --list \u0026lt; \u0026#34;$partial\u0026#34; \u0026gt; /dev/null final=\u0026#34;${partial%.partial}.dump\u0026#34; mv -- \u0026#34;$partial\u0026#34; \u0026#34;$final\u0026#34; printf \u0026#39;Published dump: %s\\n\u0026#39; \u0026#34;$final\u0026#34; SCRIPT chmod 700 backup.sh ./backup.sh A dump or archive-list failure exits nonzero and leaves no published .dump file. The temporary file and final file share a filesystem, allowing publication by rename. Existing successful archives are retained.\nThe custom archive is already compressed by default and is read by pg_restore; it is not a gzip-compressed SQL script. Plain SQL requires a different restore command. See pg_dump.\nThe source database remains available during a logical dump. The dump covers one database, not cluster roles, tablespaces, application attachments, or a coordinated snapshot of multiple databases. Plan those separately.\n4. Restore a chosen archive into the empty test instance Run this once against the fresh restore_test database. Select the exact .dump filename printed by backup.sh:\n1 2 3 4 5 read -r -p \u0026#39;Archive path under backups/: \u0026#39; archive test -s \u0026#34;$archive\u0026#34; \u0026amp;\u0026amp; docker compose exec -T restore_test pg_restore \\ --exit-on-error --single-transaction --no-owner --no-privileges \\ -U lab -d restoredb \u0026lt; \u0026#34;$archive\u0026#34; Stop if the restore fails. Repeating it into a populated database is not an independent recovery test. Do not add clean/drop options or redirect it at source to bypass that problem.\nVerify the marker using an assertion that fails on missing or incorrect data:\n1 2 3 4 5 6 7 8 9 10 docker compose exec -T restore_test psql -X -U lab -d restoredb -v ON_ERROR_STOP=1 \u0026lt;\u0026lt;\u0026#39;SQL\u0026#39; DO $$ BEGIN IF (SELECT count(*) FROM backup_marker WHERE id = 1 AND value = \u0026#39;gntech-restore-check\u0026#39;) \u0026lt;\u0026gt; 1 THEN RAISE EXCEPTION \u0026#39;Restore marker verification failed\u0026#39;; END IF; END $$; SQL A successful marker test establishes this lab\u0026rsquo;s expected row, not an entire application\u0026rsquo;s correctness. Add schema, data, permissions, extension, and application checks for production. See pg_restore.\nFor plain SQL imports, use psql with ON_ERROR_STOP so SQL errors propagate. Do not hide stderr or treat “some tables exist” as a passed restore.\n5. Schedule after the manual recovery test Move the validated project to a stable host path before scheduling. Test backup.sh as the scheduler\u0026rsquo;s actual user, using the installed Docker executable and Compose plugin. Confirm its PATH, permissions, mount state, logs, and time zone.\nThe script already takes a nonblocking lock. Use a host timer or cron job to invoke it once per scheduled run; do not wrap it in a tight retry loop or rely on installing client tools every time a container starts.\nConnect failure and missed-run alerts to the actual command result. Send success only after every required stage succeeds, including any required off-site transfer. A notification transport failure should be visible and should not turn a failed dump into success.\nKeep retention separate until recovery is proven. Preview candidate archives, verify off-site availability and required recovery history, and scope deletion to this workflow. Do not delete older backups before confirming the new required copy succeeded.\nInstall a systemd service and daily timer Run this section from a root Bash shell on a systemd host, inside the validated project at its stable path. Keep the same Compose project name when moving an existing deployment; changing it can select different volumes. The example assumes a local, root-managed Docker Engine and Python 3 for reading Compose\u0026rsquo;s resolved project name.\nThe path check keeps generated unit values simple. Use a stable path without spaces, percent signs, or shell metacharacters. Review any existing units with these names before replacing them.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 backup_project_dir=$(pwd -P) backup_project_name=$(docker compose config --format json | python3 -c \u0026#39;import json,sys; print(json.load(sys.stdin)[\u0026#34;name\u0026#34;])\u0026#39;) [[ \u0026#34;$backup_project_dir\u0026#34; =~ ^/[a-zA-Z0-9_./-]+$ ]] || exit 1 [[ \u0026#34;$backup_project_name\u0026#34; =~ ^[a-z0-9][a-z0-9_-]*$ ]] || exit 1 test -x \u0026#34;$backup_project_dir/backup.sh\u0026#34; || exit 1 cat \u0026gt; /etc/systemd/system/gntech-postgres-backup.service \u0026lt;\u0026lt;EOF [Unit] Description=GNTECH PostgreSQL logical backup Requires=docker.service After=docker.service [Service] Type=oneshot User=root WorkingDirectory=$backup_project_dir Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin Environment=COMPOSE_PROJECT_NAME=$backup_project_name ExecStart=$backup_project_dir/backup.sh UMask=0077 TimeoutStartSec=30min StandardOutput=journal StandardError=journal EOF cat \u0026gt; /etc/systemd/system/gntech-postgres-backup.timer \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; [Unit] Description=Daily GNTECH PostgreSQL backup [Timer] OnCalendar=*-*-* 02:00:00 RandomizedDelaySec=15min Persistent=true Unit=gntech-postgres-backup.service [Install] WantedBy=timers.target EOF systemd-analyze verify /etc/systemd/system/gntech-postgres-backup.service \\ /etc/systemd/system/gntech-postgres-backup.timer systemctl daemon-reload The service uses the resolved Compose project explicitly and runs without your interactive shell environment. Root access to Docker is intentional here; these units require review before adapting to a rootless engine or a remote Docker context. All database commands must still target the intended project.\nTest the service before enabling the timer:\n1 2 3 4 5 6 systemctl start gntech-postgres-backup.service systemctl show gntech-postgres-backup.service \\ -p Result -p ExecMainStatus -p ActiveState journalctl -u gntech-postgres-backup.service --no-pager -n 40 systemctl enable --now gntech-postgres-backup.timer systemctl list-timers --all gntech-postgres-backup.timer Require Result=success, ExecMainStatus=0, and a new published archive. A completed oneshot normally becomes inactive; that alone is not a failure. This timer runs at 02:00 in the host\u0026rsquo;s time zone, plus up to 15 minutes of random delay. Check timedatectl. Persistent=true can trigger a catch-up after downtime; it does not replay every missed daily backup.\nA nonzero script exit, including lock contention or a stopped source, marks the service failed and records stderr in the journal. Inspect Result, ExecMainStatus, and the journal before retrying. The timer being active proves only that it is waiting, not that backups are succeeding.\nUse these signals in your monitoring system and separately alert on overdue successful archives. This example sends no email or webhook and cannot detect a powered-off host by itself. Avoid RemainAfterExit=yes: leaving this oneshot active prevents later timer activations from running it again. See systemd timers.\nTo stop this schedule while retaining the units and backups:\n1 systemctl disable --now gntech-postgres-backup.timer 6. Add encrypted off-site recovery Database dumps here are plaintext data in a compressed archive. File permissions restrict local access but are not encryption.\nUse the Restic backup and restore guide to protect completed archives. Select only published .dump files, excluding .partial files. Validate the remote repository and restore an archive from that remote copy into another isolated instance.\nKeep repository credentials, database versions, extension requirements, application secrets, and recovery instructions outside the original host. Measure recovery time; neither a successful upload nor a nonempty archive proves recoverability.\nLogical dumps do not provide arbitrary point-in-time recovery. Review PostgreSQL backup guidance and WAL-based recovery requirements when the recovery target demands more than periodic snapshots.\nRecover a database dump from a copied Restic repository This extension joins the two labs. Both repositories are local to the disposable host; this tests the recovery chain, not off-site durability or a remote backend. Run it after successful dump creation. Stop the timer during this rehearsal so the selected dump directory does not change.\nUse a reviewed official Restic image. The example below matches the Restic lab\u0026rsquo;s tested 0.19.1 release. Keep these passwords and the copied repository outside the original host in a real deployment.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 umask 077 mkdir chain-repository chain-copy chain-restored openssl rand -base64 32 \u0026gt; chain-password openssl rand -base64 32 \u0026gt; chain-copy-password openssl rand -base64 32 \u0026gt; recovery-db-password cat \u0026gt; compose.chain.yaml \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; services: archive: image: restic/restic:0.19.1 environment: RESTIC_REPOSITORY: /repository RESTIC_PASSWORD_FILE: /run/secrets/chain-password volumes: - ./backups:/dumps:ro - ./chain-repository:/repository - ./chain-copy:/copy secrets: - chain-password - chain-copy-password recover_archive: image: restic/restic:0.19.1 environment: RESTIC_REPOSITORY: /repository RESTIC_PASSWORD_FILE: /run/secrets/chain-copy-password volumes: - ./chain-copy:/repository - ./chain-restored:/restore secrets: - chain-copy-password recovered_db: image: postgres:16-alpine environment: POSTGRES_DB: restoredb POSTGRES_USER: lab POSTGRES_PASSWORD_FILE: /run/secrets/recovery-db-password secrets: - recovery-db-password volumes: - chain_recovery_data:/var/lib/postgresql/data healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;pg_isready\u0026#34;, \u0026#34;-h\u0026#34;, \u0026#34;127.0.0.1\u0026#34;, \u0026#34;-U\u0026#34;, \u0026#34;lab\u0026#34;, \u0026#34;-d\u0026#34;, \u0026#34;restoredb\u0026#34;] interval: 5s timeout: 3s retries: 12 start_period: 10s secrets: chain-password: file: ./chain-password chain-copy-password: file: ./chain-copy-password recovery-db-password: file: ./recovery-db-password volumes: chain_recovery_data: EOF docker compose -f compose.yaml -f compose.chain.yaml run --rm --interactive=false archive init docker compose -f compose.yaml -f compose.chain.yaml run --rm --interactive=false archive \\ backup /dumps --exclude \u0026#39;*.partial\u0026#39; --host postgres-recovery-lab --tag pgdump docker compose -f compose.yaml -f compose.chain.yaml run --rm --interactive=false archive \\ --repo /copy --password-file /run/secrets/chain-copy-password init docker compose -f compose.yaml -f compose.chain.yaml run --rm --interactive=false archive \\ --repo /copy --password-file /run/secrets/chain-copy-password \\ copy --from-repo /repository --from-password-file /run/secrets/chain-password Keep this staging directory limited to the workflow\u0026rsquo;s published dumps and temporary partial files. The exclude rule prevents a partially written dump from entering the snapshot. It does not make other arbitrary files safe to publish.\nThe recovery service has no mount for the original dumps, original repository, or its password. Recover from the copy and import a selected dump into the new database:\n1 2 3 4 5 6 7 8 9 10 11 12 13 docker compose -f compose.yaml -f compose.chain.yaml run --rm --interactive=false recover_archive snapshots docker compose -f compose.yaml -f compose.chain.yaml run --rm --interactive=false recover_archive check --read-data read -r -p \u0026#39;Copied snapshot ID: \u0026#39; copied_id test -n \u0026#34;$copied_id\u0026#34; || exit 1 docker compose -f compose.yaml -f compose.chain.yaml run --rm --interactive=false recover_archive \\ restore \u0026#34;$copied_id\u0026#34; --target /restore read -r -p \u0026#39;Recovered dump filename (without directory): \u0026#39; recovered_dump [[ \u0026#34;$recovered_dump\u0026#34; =~ ^[a-zA-Z0-9_.-]+\\.dump$ ]] || exit 1 test -s \u0026#34;chain-restored/dumps/$recovered_dump\u0026#34; || exit 1 docker compose -f compose.yaml -f compose.chain.yaml up --wait --wait-timeout 120 recovered_db docker compose -f compose.yaml -f compose.chain.yaml exec -T recovered_db pg_restore \\ --exit-on-error --single-transaction --no-owner --no-privileges \\ -U lab -d restoredb \u0026lt; \u0026#34;chain-restored/dumps/$recovered_dump\u0026#34; Require the same marker assertion against this new database:\n1 2 3 4 5 6 7 8 9 10 11 docker compose -f compose.yaml -f compose.chain.yaml exec -T recovered_db \\ psql -X -U lab -d restoredb -v ON_ERROR_STOP=1 \u0026lt;\u0026lt;\u0026#39;SQL\u0026#39; DO $$ BEGIN IF (SELECT count(*) FROM backup_marker WHERE id = 1 AND value = \u0026#39;gntech-restore-check\u0026#39;) \u0026lt;\u0026gt; 1 THEN RAISE EXCEPTION \u0026#39;Copied repository database recovery failed\u0026#39;; END IF; END $$; SQL Do not repeat this import into the populated database or point it at the source. The new database uses a fresh volume and its own bootstrap credential; production roles, extensions, attachments, and permissions still need their own recovery plan. Include both Compose files when stopping this extended lab.\nStop the lab For the extended Restic lab, stop all its services with both files; otherwise use the original single-file command.\n1 2 3 4 5 if [ -f compose.chain.yaml ]; then docker compose -f compose.yaml -f compose.chain.yaml down else docker compose down fi This retains the lab volumes and archives for inspection. Removing the disposable volumes is a separate decision after checking recovery evidence. No production database should be attached to this project.\nContinue reading Compose health checks — Separate startup readiness from job success. Backup strategy — Define recovery targets and independent copies. ","permalink":"https://blog.gntech.me/posts/2026-06-01-docker-database-backup-automation-pgdump-cron/","summary":"A focused PostgreSQL lab with atomic dump publication, error propagation, and a separate restore destination.","title":"Docker PostgreSQL Backups — Atomic Dumps and an Isolated Restore Test"},{"content":"Why Automate Docker Dependency Updates in Your Homelab You maintain a dozen Docker Compose stacks — databases, reverse proxies, media servers, monitoring tools. Each uses Docker images with version tags like nginx:1.25-alpine, postgres:16-bookworm, or traefik:v3.1. Over time these go stale. Security patches ship, bugs get fixed, and your compose files still reference the old tag.\nMost homelabbers rely on Watchtower or Diun for runtime updates — pull the image, restart the container, done. That works for keeping running containers current, but it doesn\u0026rsquo;t update your source of truth: the docker-compose.yml in your git repository.\nRenovate Bot fills that gap. It scans your git repos for Dockerfile FROM lines, docker-compose image references, and even Kubernetes manifests. When a newer version exists, Renovate opens a pull request with the updated tag, optionally pins the digest, and includes release notes from the upstream registry.\nThe result is a full GitOps workflow: your compose definitions stay current in git, and you decide when to merge the update.\nHow Renovate Differs from Watchtower Aspect Watchtower Renovate What it updates Running containers Source definitions (git) When it acts On schedule + new image On schedule per cron Workflow Pull + restart Create PR Rollback Re-pull old tag Revert commit in git Auditing Container logs Git history + PR Both tools complement each other. Renovate keeps your git definitions accurate; Watchtower picks up merged changes and restarts containers.\nSelf-Hosted vs Cloud Renovate Renovate has two deployment models:\nCloud bot — GitHub and GitLab each have a Renovate app. Install it, add renovate.json5 to your repos, and it runs on every push. Free for public repos, simple setup.\nSelf-hosted — Run the Renovate Docker image on your own schedule against any git forge: Gitea, Forgejo, GitLab CE, or Bitbucket. Full control over schedule, configuration, and which repos are scanned.\nFor a homelab with a self-hosted Gitea instance, self-hosted Renovate is the natural choice.\nStep-by-Step — Self-Hosted Renovate with Gitea 1. Shared Configuration Repository Create a private repository called renovate-config to hold your default preset. This keeps per-repo configs minimal.\n// renovate-config/default.json5 { $schema: \u0026#34;https://docs.renovatebot.com/renovate-schema.json\u0026#34;, extends: [ \u0026#34;config:recommended\u0026#34;, \u0026#34;docker:pinDigests\u0026#34;, \u0026#34;docker:enableMajor\u0026#34;, \u0026#34;:maintainLockFilesMonthly\u0026#34;, ], labels: [\u0026#34;dependencies\u0026#34;, \u0026#34;docker\u0026#34;], schedule: [\u0026#34;before 6am on Monday\u0026#34;], packageRules: [ { matchManagers: [\u0026#34;docker-compose\u0026#34;, \u0026#34;dockerfile\u0026#34;], groupName: \u0026#34;Docker dependencies\u0026#34;, groupSlug: \u0026#34;docker\u0026#34;, automerge: false, labels: [\u0026#34;docker-update\u0026#34;], }, { matchUpdateTypes: [\u0026#34;pin\u0026#34;, \u0026#34;digest\u0026#34;], automerge: true, labels: [\u0026#34;docker-pin\u0026#34;], }, { matchUpdateTypes: [\u0026#34;major\u0026#34;], labels: [\u0026#34;docker-major\u0026#34;], automerge: false, description: \u0026#34;Major version bumps require manual review\u0026#34;, }, ], docker: { pinDigests: true, }, } This preset:\nExtends recommended rules and Docker digest pinning Groups all Docker updates into a single weekly PR (Monday morning) Auto-merges digest/pin updates (low risk) Flags major version bumps for manual review Applies labels for easy filtering in the PR list 2. Per-Repository renovate.json5 Each docker-compose repository includes a minimal config referencing the shared preset:\n// homelab-infra/renovate.json5 { $schema: \u0026#34;https://docs.renovatebot.com/renovate-schema.json\u0026#34;, extends: [\u0026#34;local\u0026gt;gitea.yourlab.internal/renovate-config:default.json5\u0026#34;], } The local\u0026gt; preset reference tells Renovate to fetch the config from the configured git forge. No need to duplicate rules across a dozen repos.\n3. Deploy Renovate with Docker Compose Create a compose file for the Renovate bot itself:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 services: renovate: image: renovate/renovate:39 container_name: renovate restart: unless-stopped volumes: - ./config.js:/usr/src/app/config.js:ro - ./data:/usr/src/app/data environment: RENOVATE_TOKEN: ${RENOVATE_TOKEN} RENOVATE_ENDPOINT: \u0026#34;${GITEA_URL}/api/v1\u0026#34; RENOVATE_AUTODISCOVER: \u0026#34;true\u0026#34; RENOVATE_AUTODISCOVER_FILTER: \u0026#34;homelab-*,infra-*\u0026#34; RENOVATE_PLATFORM: \u0026#34;gitea\u0026#34; RENOVATE_BASE_DIR: \u0026#34;/usr/src/app/data\u0026#34; RENOVATE_CONFIG_FILE: \u0026#34;/usr/src/app/config.js\u0026#34; LOG_LEVEL: \u0026#34;info\u0026#34; Start it with:\n1 docker compose up -d Renovate runs continuously in daemon mode, scanning on the schedule defined in your config.\nFor single-shot execution (useful for testing), run:\n1 docker compose run --rm renovate 4. Gitea API Access Token Generate a Gitea token with read:repository, write:repository, and read:issue scopes:\n1 2 3 4 # Using Gitea CLI or API gitea admin user generate-access-token \\ --username renovate \\ --scopes \u0026#34;read:repository,write:repository,read:issue\u0026#34; Set the token as RENOVATE_TOKEN in your .env file:\nRENOVATE_TOKEN=gitea_xxxxxxxxxxxx GITEA_URL=https://gitea.yourlab.internal 5. First Run — Verification Run Renovate in dry mode to verify everything works without creating PRs:\n1 docker compose run --rm renovate --dry-run Check the logs for \u0026ldquo;Found Dockerfile package files\u0026rdquo; and \u0026ldquo;PR created\u0026rdquo; entries. If Autodiscover filtered correctly, only the matching repos appear.\nOnce dry-run is clean, run for real:\n1 docker compose run --rm renovate Renovate scans each repo, checks current image tags against registries, and opens PRs for any updates found.\n6. What a Renovate PR Looks Like When Renovate detects an update, the PR title, body, and diff look like this:\nTitle: chore(deps): update docker.io/library/nginx:1.25-alpine to 1.26-alpine Body: - **Schedule**: Before 6 AM on Monday - **Branch**: renovate/docker-nginx-1.x - **Update type**: minor - **Release notes**: https://github.com/nginx/nginx/releases/tag/release-1.26.0 ![nginx-version](https://img.shields.io/badge/nginx-1.25%20%E2%86%92%201.26-blue) Diff: - image: nginx:1.25-alpine + image: nginx:1.26-alpine@sha256:abc123def456... If you enabled digest pinning, the PR includes the SHA256 digest for immutable builds.\nConfiguration Deep Dive Presets Worth Using config:recommended — Sensible defaults for most projects docker:pinDigests — Replace tag-only references with tag+digest docker:enableMajor — Allow major version PRs (disabled by default for Docker) :maintainLockFilesMonthly — Monthly lock file refresh (mostly for npm, but applies generally) :automergeMinor — Auto-merge minor/patch updates (enable with caution — test your config first) Package Rules for Docker Groups Group all Docker updates into one PR to reduce noise:\n{ packageRules: [ { matchManagers: [\u0026#34;docker-compose\u0026#34;, \u0026#34;dockerfile\u0026#34;], groupName: \u0026#34;Docker dependencies\u0026#34;, labels: [\u0026#34;docker\u0026#34;], }, ], } Without grouping, each outdated image gets its own PR — noisy for busy compose files with 10+ images.\nCustom Managers for Non-Standard Files If you reference Docker images in shell scripts, Ansible variables, or Terraform, add a custom manager:\n{ customManagers: [ { customType: \u0026#34;regex\u0026#34;, fileMatch: [\u0026#34;\\\\.env$\u0026#34;, \u0026#34;\\\\.bash$\u0026#34;, \u0026#34;scripts/.*\\\\.sh$\u0026#34;], matchStrings: [ \u0026#34;IMAGE_TAG=(?\u0026lt;currentValue\u0026gt;.*)\\\\n\u0026#34;, ], datasourceTemplate: \u0026#34;docker\u0026#34;, depNameTemplate: \u0026#34;library/nginx\u0026#34;, versioningTemplate: \u0026#34;docker\u0026#34;, }, ], } This catches environment variable-driven image tags — useful when your compose file references an $IMAGE_TAG variable.\nDigest Pinning for Immutable Builds Docker tags are mutable. nginx:1.25-alpine today might not be nginx:1.25-alpine tomorrow if the image is overwritten or a patch release gets re-tagged.\nDigest pinning replaces the tag with an immutable content hash:\n1 2 3 4 5 # Without pinning — mutable image: nginx:1.25-alpine # With pinning — immutable image: nginx:1.25-alpine@sha256:d4f9c3b9f5a7c8b9a1b2c3d4e5f6a7b8c9d0e1f2 Renovate automatically pins digests when docker:pinDigests is enabled. It also opens digest-type PRs when the underlying digest changes for the same tag. This prevents accidental supply-chain issues from retagged images.\nTrade-off: More PRs to review (or auto-merge), but every build is reproducible.\nEnd-to-End Workflow Here is the practical cycle once Renovate is deployed:\nMonday 05:00 — Cron triggers Renovate in daemon mode Renovate scans 8 repos, finds 3 outdated Docker images Opens 1 grouped PR: \u0026#34;chore(deps): update Docker dependencies\u0026#34; PR includes: - nginx:1.25 → nginx:1.26@sha256:abc - postgres:16 → postgres:16@sha256:def - redis:7-alpine → redis:7-alpine@sha256:ghi Monday 09:00 — You review the PR in Gitea Merge it to main branch Monday 09:01 — Gitea webhook triggers your CI pipeline (optional — you can also skip CI and just merge) Monday 09:05 — Watchtower picks up new digest on the host Pulls updated images, restarts containers Or you run: docker compose pull \u0026amp;\u0026amp; docker compose up -d The same cycle runs weekly. No more logging into each host to check image versions.\nAuto-Merge Safe Updates For low-risk updates (digest changes, patch bumps), enable auto-merge in your Renovate config:\n{ packageRules: [ { matchUpdateTypes: [\u0026#34;pin\u0026#34;, \u0026#34;digest\u0026#34;], automerge: true, }, { matchManagers: [\u0026#34;docker-compose\u0026#34;, \u0026#34;dockerfile\u0026#34;], matchUpdateTypes: [\u0026#34;patch\u0026#34;], automerge: true, }, ], } Works best with branch protection rules that require CI checks to pass before merge. Otherwise auto-merge pushes directly to your main branch.\nConclusion Renovate Bot transforms how you manage Docker dependencies in a homelab. Instead of manually checking image versions or relying entirely on runtime updates, Renovate keeps your git-tracked definitions current with automated pull requests. Pair it with Watchtower for a complete pipeline:\nRenovate updates the definition (git → PR → merge) Watchtower applies the update (merge → pull → restart) The self-hosted setup with Gitea runs on minimal hardware — a single Docker container with a cron schedule. Shared presets keep per-repo configs under ten lines. Digest pinning locks every image to an immutable content hash.\nIn my homelab, Renovate manages 12 repositories with 30+ Docker Compose stacks. Weekly grouped PRs replace the tedious \u0026ldquo;check every image tag\u0026rdquo; chore. That\u0026rsquo;s time saved for actually building things.\nSee Also Gitea Docker CI/CD Pipeline for Homelab Docker Watchtower — Automated Container Image Updates Docker Compose Production Patterns GitOps Patterns for Homelab Infrastructure ","permalink":"https://blog.gntech.me/posts/2026-06-01-renovate-bot-automated-docker-updates-homelab/","summary":"Automate Docker dependency updates in your homelab with Renovate Bot. Self-host or use the cloud bot to scan Docker Compose files and Dockerfiles, then open pull requests with version bumps, digest updates, and changelog summaries.","title":"Renovate Bot Homelab — Automated Docker Dependency Updates with Git"},{"content":"Why CPU Frequency Scaling Matters in a Homelab Your homelab server runs 24/7. Every watt it draws adds up over a year — at $0.12/kWh, a 100W difference costs about $105 annually. But power is not the only consideration. A Proxmox host running a DNS resolver, a reverse proxy, and a database needs different CPU behavior than a gaming rig or a CI build runner.\nLinux\u0026rsquo;s CPUFreq subsystem controls the processor\u0026rsquo;s clock speed and voltage dynamically. By tuning frequency scaling governors, p-state drivers, Turbo Boost behavior, and idle states (C-states), you can shift your server\u0026rsquo;s power profile from \u0026ldquo;full blast always\u0026rdquo; to \u0026ldquo;responsive but efficient\u0026rdquo; without touching hardware.\nThis guide covers every layer: checking your current configuration, choosing the right governor, tuning Intel and AMD drivers, controlling Turbo Boost, managing C-states, and applying profiles for common homelab workloads.\nChecking Your Current CPU Governor and Frequency Before making changes, inspect your system\u0026rsquo;s current CPUFreq configuration. The cpupower utility from the linux-cpupower package is the primary tool.\n1 2 3 4 5 # Install cpupower sudo apt update \u0026amp;\u0026amp; sudo apt install linux-cpupower -y # Show full frequency info for all cores cpupower frequency-info The output shows the driver in use (intel_pstate, intel_cpufreq, acpi-cpufreq, or amd_pstate_epp), the available governors, the current policy, and the hardware frequency limits.\n1 2 3 4 5 # Quick check per-CPU governor grep . /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor # Live frequency monitoring watch -n 2 \u0026#34;grep \u0026#39;MHz\u0026#39; /proc/cpuinfo\u0026#34; The driver matters: intel_pstate and amd_pstate_epp handle frequency selection in hardware with software hints, while acpi-cpufreq and intel_cpufreq use software-level ACPI tables. This distinction determines which tuning knobs are available.\nAvailable CPU Governors Explained Linux provides several frequency governors, each with a different strategy for balancing performance and power.\nGovernor Behavior Best For performance Maximum frequency always Latency-sensitive workloads (databases, real-time audio) powersave Minimum frequency always Maximum power savings, very low load servers ondemand Scales up on CPU load, scales down when idle General purpose, older kernels only conservative Like ondemand but smoother ramp Systems sensitive to frequency transitions schedutil Scheduler-driven, uses Per-Entity Load Tracking Modern default, balanced performance/power schedutil is the default on most modern Linux distributions with kernel 5.4+. It integrates directly with the scheduler and provides the best balance across varied workloads. The scheduler knows exactly what each task needs, so frequency decisions are made with full context.\nondemand and conservative are legacy governors. On modern hardware with intel_pstate or amd_pstate_epp active, the p-state driver\u0026rsquo;s hardware control outperforms any software governor. Always prefer the driver\u0026rsquo;s hardware-governed modes when available.\n1 2 3 4 # Test each governor manually sudo cpupower frequency-set -g schedutil sudo cpupower frequency-set -g performance sudo cpupower frequency-set -g powersave Configuring the Governor with cpupower Set the governor persistently with a systemd service or the distro-specific configuration file.\n1 2 3 4 5 # Set for all CPUs immediately sudo cpupower frequency-set -g schedutil # Set for a specific CPU core sudo cpupower -c 0 frequency-set -g performance To make the change persistent across reboots, create a systemd oneshot service:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # /etc/systemd/system/cpupower.service [Unit] Description=Set CPU governor to schedutil After=sysinit.target Before=multi-user.target [Service] Type=oneshot ExecStart=/usr/bin/cpupower frequency-set -g schedutil RemainAfterExit=yes [Install] WantedBy=multi-user.target 1 2 sudo systemctl daemon-reload sudo systemctl enable --now cpupower.service On Debian and Ubuntu, you can instead edit /etc/default/cpupower and set GOVERNOR=\u0026quot;schedutil\u0026quot;, then enable the cpupower service if the package ships one. This approach varies by distribution, so check your local init scripts first.\nIntel P-State and AMD P-State Driver Tuning The p-state drivers expose hardware-level performance controls that go beyond simple governor selection.\nIntel P-State (intel_pstate) intel_pstate operates in two modes:\nActive mode (default on Intel Core i3/i5/i7/i9): The hardware manages P-states internally, and the CPUFreq governor acts as a hint. The governor selection mostly affects Energy Performance Bias (EPB). Passive mode (intel_pstate=passive kernel parameter): The driver behaves like intel_cpufreq, and software governors have full control. Useful for custom frequency policies. 1 2 3 4 5 # Check current mode cat /sys/devices/system/cpu/intel_pstate/status # Switch to passive (until reboot) echo \u0026#34;passive\u0026#34; | sudo tee /sys/devices/system/cpu/intel_pstate/status When Hardware-Managed P-States (HWP) are enabled — common on Skylake and newer — the x86_energy_perf_policy utility provides a higher-level hint:\n1 2 3 4 5 6 7 8 9 10 11 # Show current EPB value x86_energy_perf_policy -r # Set to performance (lowest power savings, best latency) sudo x86_energy_perf_policy --set performance # Set to balance-performance (recommended for most homelab servers) sudo x86_energy_perf_policy --set balance-performance # Set to power (maximum power savings) sudo x86_energy_perf_policy --set power EPB values range from 0 (performance) to 15 (power save). The hardware uses this hint to decide when to enter higher P-states (higher frequency) or lower ones.\nAMD P-State (amd_pstate_epp) AMD\u0026rsquo;s equivalent uses Energy Performance Preference (EPP), exposed through sysfs:\n1 2 3 4 5 6 7 8 9 10 # Check available EPP values cat /sys/devices/system/cpu/cpu0/cpufreq/energy_performance_available_preferences # Set EPP (available values: performance, balance_performance, default, balance_power, power) echo \u0026#34;balance_performance\u0026#34; | sudo tee /sys/devices/system/cpu/cpu0/cpufreq/energy_performance_preference # Apply to all CPUs for cpu in /sys/devices/system/cpu/cpu*/cpufreq/energy_performance_preference; do echo \u0026#34;balance_performance\u0026#34; | sudo tee \u0026#34;$cpu\u0026#34; \u0026gt; /dev/null done On AMD Ryzen and EPYC systems running kernel 6.3+, amd_pstate_epp is the default driver and provides the best balance of responsiveness and power efficiency.\nTurbo Boost Control Turbo Boost (Intel Turbo Boost / AMD Precision Boost) allows cores to exceed their base frequency when thermal and power headroom allows. This delivers snappy burst performance but increases peak power draw and heat.\n1 2 3 4 5 6 7 8 # Check if turbo is currently enabled (0=enabled, 1=disabled) cat /sys/devices/system/cpu/intel_pstate/no_turbo # Disable turbo immediately echo \u0026#34;1\u0026#34; | sudo tee /sys/devices/system/cpu/intel_pstate/no_turbo # Re-enable turbo echo \u0026#34;0\u0026#34; | sudo tee /sys/devices/system/cpu/intel_pstate/no_turbo The no_turbo sysfs knob applies to both Intel and AMD systems using their respective p-state drivers. For acpi-cpufreq users, the control is at cat /sys/devices/system/cpu/cpufreq/boost.\nDisabling Turbo Boost is useful when:\nRunning in a small form factor case with limited cooling Running latency-sensitive workloads where frequency variability hurts predictability Prioritizing power efficiency over peak performance — disabling turbo typically reduces peak power by 15-30% while only reducing peak throughput by 10-20% Make the setting persistent with a sysfs.d drop-in:\n1 2 # /etc/sysfs.d/turbo.conf devices/system/cpu/intel_pstate/no_turbo = 1 Or use a systemd tmpfiles.d rule:\n1 echo \u0026#34;w /sys/devices/system/cpu/intel_pstate/no_turbo - - - - 1\u0026#34; | sudo tee /etc/tmpfiles.d/turbo.conf CPU Idle States (C-States) Configuration Modern processors have multiple idle states, ranging from C0 (active) through C1 (halt), C1E (enhanced halt), C6 (deep sleep with cache flush), up to C10 (package-level deep sleep).\n1 2 3 4 5 # View available idle states with latency cpupower idle-info # Monitor C-state residency cpupower monitor Deeper C-states save more power but add wake latency. C1 wake takes ~1 µs, while C6 wake can take 100+ µs. For a Proxmox host running VMs with real-time workloads, 100 µs wake latency can cause timing issues. For a general purpose Docker host, deeper C-states save meaningful power — often 5-15W on an idle server.\n1 2 3 4 5 6 # Disable deep C-states (stay in C1 only) # Kernel boot parameters in /etc/default/grub # GRUB_CMDLINE_LINUX_DEFAULT=\u0026#34;... processor.max_cstate=1 intel_idle.max_cstate=0\u0026#34; # Apply per-CPU via sysfs (example: disable C2 on CPU 0) echo 1 | sudo tee /sys/devices/system/cpu/cpu0/cpuidle/state1/disable To apply permanently, add the kernel parameters:\n1 2 3 4 5 6 # Edit /etc/default/grub GRUB_CMDLINE_LINUX_DEFAULT=\u0026#34;quiet processor.max_cstate=1 intel_idle.max_cstate=0\u0026#34; # Update grub sudo update-grub sudo reboot Tradeoff: disabling deep C-states costs 5-15W at idle but guarantees sub-10 µs wake latency, essential for real-time containers or Proxmox with latency-sensitive VMs.\nPractical Homelab Scenarios Different workloads need different power profiles. Here are three common homelab configurations.\nScenario 1: Low-Power Proxmox Host (24/7) An always-on server running lightweight containers and a few VMs.\n1 2 3 4 5 6 7 8 9 # Governor: schedutil (balanced) sudo cpupower frequency-set -g schedutil # Turbo: Enabled (handles burst workloads well) echo 0 | sudo tee /sys/devices/system/cpu/intel_pstate/no_turbo # C-states: All allowed (save power at idle) # EPB/EPP: balance_power sudo x86_energy_perf_policy --set balance-power Scenario 2: Database or Latency-Sensitive Server A PostgreSQL or low-latency application host where consistent response times matter.\n1 2 3 4 5 6 7 8 9 10 11 # Governor: performance (fixed max frequency) sudo cpupower frequency-set -g performance # Turbo: Disabled (eliminates frequency variance) echo 1 | sudo tee /sys/devices/system/cpu/intel_pstate/no_turbo # C-states: C1 only (minimal wake latency) # Boot with: processor.max_cstate=1 intel_idle.max_cstate=0 # EPB/EPP: performance sudo x86_energy_perf_policy --set performance Scenario 3: Media or File Server A server running Plex, Jellyfin, or Samba/NFS with bursty workloads and long idle periods.\n1 2 3 4 5 6 7 8 # Governor: schedutil sudo cpupower frequency-set -g schedutil # Turbo: Enabled (handles transcoding bursts) echo 0 | sudo tee /sys/devices/system/cpu/intel_pstate/no_turbo # C-states: C1E allowed, deeper states allowed (batches of I/O benefit less from instant wake) # EPB/EPP: balance-power Quick Profile Switching Script Save this as /usr/local/bin/cpu-profile for on-the-fly switching between power profiles:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 #!/bin/bash case \u0026#34;${1:-balanced}\u0026#34; in powersave|power) echo \u0026#34;Switching to powersave profile\u0026#34; cpupower frequency-set -g powersave echo 1 \u0026gt; /sys/devices/system/cpu/intel_pstate/no_turbo 2\u0026gt;/dev/null || true x86_energy_perf_policy --set power 2\u0026gt;/dev/null || true ;; balanced|default) echo \u0026#34;Switching to balanced profile\u0026#34; cpupower frequency-set -g schedutil echo 0 \u0026gt; /sys/devices/system/cpu/intel_pstate/no_turbo 2\u0026gt;/dev/null || true x86_energy_perf_policy --set balance-performance 2\u0026gt;/dev/null || true ;; performance|latency) echo \u0026#34;Switching to performance profile\u0026#34; cpupower frequency-set -g performance echo 0 \u0026gt; /sys/devices/system/cpu/intel_pstate/no_turbo 2\u0026gt;/dev/null || true cpupower idle-set -d 2 2\u0026gt;/dev/null || true x86_energy_perf_policy --set performance 2\u0026gt;/dev/null || true ;; *) echo \u0026#34;Usage: $0 {powersave|balanced|performance}\u0026#34; exit 1 ;; esac Make it executable and test:\n1 2 3 sudo chmod +x /usr/local/bin/cpu-profile sudo cpu-profile balanced sudo cpu-profile performance Measuring Your Changes Use these tools to verify the actual impact of your tuning:\n1 2 3 4 5 6 7 8 # Monitor frequency in real time watch -n 2 \u0026#34;grep MHz /proc/cpuinfo; echo \u0026#39;---\u0026#39;; cpupower monitor\u0026#34; # Measure actual power consumption (requires turbostat) sudo turbostat --show PkgWatt,CorWatt,GFXWatt,RAMWatt --interval 10 # Measure over time with powerstat sudo powerstat 10 6 turbostat reads the CPU\u0026rsquo;s built-in RAPL (Running Average Power Limit) counters and provides package-level power draw. powerstat runs a longer test and gives you min/max/avg after sampling.\nConclusion CPU frequency scaling and power management are free tuning knobs that every homelab admin should understand. The defaults work, but they favor performance over efficiency or vice versa depending on your distribution. By selecting the right governor, tuning your p-state driver, deciding on Turbo Boost, and setting the appropriate C-state ceiling, you can shave watts off your power bill or microseconds off your latency — or find the balance that suits your workload.\nStart with schedutil and default C-states (most balanced). If you run latency-critical services, move to performance with Turbo and deep C-states disabled. If you prioritize power efficiency, use schedutil with Turbo enabled and let the deeper C-states save you money at idle.\nFor further reading, see the Linux kernel documentation on CPUFreq at Documentation/admin-guide/pm/, or run cpupower help for all available subcommands.\n","permalink":"https://blog.gntech.me/posts/2026-05-31-linux-cpu-frequency-scaling-power-management-homelab/","summary":"Complete guide to Linux CPU frequency scaling and power management for 24/7 homelab servers. Covers cpupower governors, Intel and AMD p-state drivers, Turbo Boost control, C-states, and performance tuning tradeoffs with real commands.","title":"Linux CPU Frequency Scaling and Power Management for Homelab Servers"},{"content":"Why Run BGP in Your Homelab BGP is the routing protocol that runs the internet, but it is not just for ISPs and data centers. Running BGP in your homelab unlocks several practical capabilities that static routes cannot match.\nIf you have multiple routers, hypervisors with virtual routers, or a mix of MikroTik devices and Linux servers, BGP gives you automatic failover, traffic engineering with local-preference and community tags, and a hands-on learning environment for one of the most important protocols in networking.\nFRRouting (FRR) is the modern fork of the Quagga routing suite. It is actively maintained, packaged for Debian and Ubuntu, and features a Cisco-style CLI through vtysh. This guide walks through deploying FRR on a Linux server, peering with MikroTik RouterOS over iBGP and eBGP, filtering routes with prefix-lists, redistributing OSPF into BGP, and applying community-based policy.\nInstalling FRRouting on Debian or Ubuntu FRR provides official packages through their apt repository. Add the repository, install the daemon, and enable the BGP process.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 # Add FRR GPG key and repository curl -s https://deb.frrouting.org/frr/keys.asc | gpg --dearmor | sudo tee /usr/share/keyrings/frr.gpg \u0026gt; /dev/null echo \u0026#34;deb [signed-by=/usr/share/keyrings/frr.gpg] https://deb.frrouting.org/frr $(lsb_release -sc) frr-stable\u0026#34; | sudo tee /etc/apt/sources.list.d/frr.list # Install FRR and utilities sudo apt update \u0026amp;\u0026amp; sudo apt install frr frr-pythontools # Enable the BGP daemon sudo sed -i \u0026#39;s/^bgpd=no/bgpd=yes/\u0026#39; /etc/frr/daemons # Enable IP forwarding echo \u0026#34;net.ipv4.ip_forward=1\u0026#34; | sudo tee -a /etc/sysctl.d/99-frr.conf sudo sysctl -p /etc/sysctl.d/99-frr.conf # Restart FRR sudo systemctl restart frr Verify the vtysh shell works:\n1 vtysh -c \u0026#34;show version\u0026#34; You should see the FRR version, build configuration, and enabled daemons.\nBasic iBGP Configuration Within a Single AS iBGP (Internal BGP) runs between routers in the same autonomous system. In a homelab, use a private AS number from the 64512-65535 range.\nThis example configures two Linux servers in AS 64512. The first server (10.0.20.10) peers with the second (10.0.20.11) and advertises the 10.0.30.0/24 subnet.\n# Enter vtysh configuration mode sudo vtysh # Configure in FRR CLI configure terminal router bgp 64512 bgp router-id 10.0.20.10 neighbor 10.0.20.11 remote-as 64512 neighbor 10.0.20.11 description iBGP-Peer-1 ! address-family ipv4 unicast network 10.0.30.0/24 neighbor 10.0.20.11 next-hop-self exit-address-family end write memory Key points about iBGP:\nEvery iBGP peer must form a full mesh or use route reflectors — FRR supports both next-hop-self ensures the router advertises itself as the next hop for routes learned from eBGP peers The network statement tells BGP which locally reachable prefixes to advertise On the peer, configure the same AS with the router\u0026rsquo;s own ID:\nconfigure terminal router bgp 64512 bgp router-id 10.0.20.11 neighbor 10.0.20.10 remote-as 64512 neighbor 10.0.20.10 description iBGP-Peer-0 ! address-family ipv4 unicast neighbor 10.0.20.10 next-hop-self exit-address-family end write memory Verify the BGP session:\n1 vtysh -c \u0026#34;show ip bgp summary\u0026#34; You should see the peer state as Established with prefixes exchanged.\neBGP Peering with MikroTik RouterOS eBGP connects routers in different autonomous systems. This is the same protocol that ISPs use to exchange routes, and it gives you full control over route preference through local-preference, AS-path prepending, and community tagging.\nThis setup peers a Linux FRR router (AS 64512) with a MikroTik router (AS 65000) over a shared transit network.\nFRR side (Linux at 10.0.10.1):\nconfigure terminal router bgp 64512 bgp router-id 10.0.10.1 neighbor 10.0.10.2 remote-as 65000 neighbor 10.0.10.2 description eBGP-MikroTik neighbor 10.0.10.2 ebgp-multihop 2 neighbor 10.0.10.2 password encrypted-secret-here ! address-family ipv4 unicast network 10.0.20.0/24 neighbor 10.0.10.2 route-map OUTBOUND-FILTER out neighbor 10.0.10.2 route-map INBOUND-FILTER in exit-address-family end write memory MikroTik RouterOS side (7.x):\n/routing/bgp/connection add name=eBGP-FRR \\ remote.address=10.0.10.1/32 \\ remote.as=64512 \\ local.role=ebgp \\ local.address=10.0.10.2 \\ multihop=yes \\ tcp.md5.key=encrypted-secret-here \\ routing-table=main \\ address-families=ipv4 \\ templates=default /routing/bgp/connection/eBGP-FRR/network add network=10.0.30.0/24 add network=192.168.1.0/24 Verify both sides:\n1 vtysh -c \u0026#34;show ip bgp neighbors 10.0.10.2\u0026#34; On MikroTik:\n/routing/bgp/connection/print detail /routing/route/print where bgp The session should show established with routes exchanged in both directions.\nRoute Filtering with Prefix-Lists and Route Maps Filtering BGP routes prevents unwanted prefixes from entering your routing table and controls what you advertise to peers. In a homelab this is especially important if you are experimenting with route redistribution or connecting multiple routing domains.\nFRR prefix-list to allow specific subnets inbound:\nconfigure terminal ip prefix-list ALLOW-LAN seq 5 permit 10.0.0.0/8 ge 16 le 24 ip prefix-list ALLOW-LAN seq 10 permit 192.168.0.0/16 ge 24 le 24 ip prefix-list ALLOW-LAN seq 15 deny 0.0.0.0/0 le 32 route-map INBOUND-FILTER permit 10 match ip address prefix-list ALLOW-LAN set local-preference 150 route-map INBOUND-FILTER deny 100 end write memory Outbound filter to advertise only specific subnets to MikroTik:\nconfigure terminal ip prefix-list ADVERTISE-LAN seq 5 permit 10.0.20.0/24 ip prefix-list ADVERTISE-LAN seq 10 permit 10.0.30.0/24 ip prefix-list ADVERTISE-LAN seq 15 deny 0.0.0.0/0 le 32 route-map OUTBOUND-FILTER permit 10 match ip address prefix-list ADVERTISE-LAN route-map OUTBOUND-FILTER deny 100 end write memory Apply to the neighbor and verify:\n1 2 vtysh -c \u0026#34;show ip bgp prefix-list ALLOW-LAN\u0026#34; vtysh -c \u0026#34;show ip bgp neighbors 10.0.10.2 advertised-routes\u0026#34; OSPF-to-BGP Route Redistribution Many homelabs run OSPF for internal dynamic routing between hypervisors and storage networks. Redistributing OSPF routes into BGP lets you propagate internal prefixes to eBGP peers without duplicating network statements.\nFirst, configure OSPF on FRR:\nconfigure terminal router ospf router-id 10.0.20.10 network 10.0.20.0/24 area 0 network 10.0.30.0/24 area 0 passive-interface default no passive-interface eth0 exit Redistribute OSPF routes into BGP with a route-map filter:\nconfigure terminal route-map OSPF-TO-BGP permit 10 match ip address prefix-list OSPF-INTERNAL set community 64512:1000 router bgp 64512 address-family ipv4 unicast redistribute ospf route-map OSPF-TO-BGP exit-address-family end write memory Create the prefix-list for redistribution:\nconfigure terminal ip prefix-list OSPF-INTERNAL seq 5 permit 10.0.20.0/24 ip prefix-list OSPF-INTERNAL seq 10 permit 10.0.30.0/24 ip prefix-list OSPF-INTERNAL seq 15 deny 0.0.0.0/0 le 32 end write memory Verify redistributed routes:\n1 vtysh -c \u0026#34;show ip bgp\u0026#34; Routes from OSPF should appear with the \u0026gt; marker and a community of 64512:1000.\nBGP Community Tagging for Advanced Policy Communities are BGP attributes that tag routes with metadata. Other routers can match these tags and apply policy — change local-preference, filter, or prepend AS-path.\nFRR supports setting and matching communities in route-maps.\nSet a no-export community to keep routes within your AS:\nconfigure terminal route-map SET-NO-EXPORT permit 10 set community 64512:999 no-export set local-preference 200 router bgp 64512 address-family ipv4 unicast neighbor 10.0.10.2 route-map SET-NO-EXPORT out exit-address-family end write memory Match a custom community and apply policy:\nconfigure terminal ip community-list standard DMZ permit 64512:500 route-map DMZ-POLICY permit 10 match community DMZ set local-preference 50 router bgp 64512 neighbor 10.0.10.2 route-map DMZ-POLICY in end write memory View community information in the BGP table:\n1 vtysh -c \u0026#34;show ip bgp 10.0.30.0/24\u0026#34; Monitoring and Troubleshooting BGP Sessions FRR provides the vtysh shell with familiar Cisco-style show and debug commands.\nCheck BGP neighbor state:\n1 vtysh -c \u0026#34;show ip bgp summary\u0026#34; Display the full BGP table:\n1 vtysh -c \u0026#34;show ip bgp\u0026#34; View the kernel routing table for BGP-learned routes:\n1 ip route show proto bgp Enable BGP debug logging (use carefully in production):\nconfigure terminal debug bgp updates debug bgp keepalives debug bgp events log file /var/log/frr/bgpd.log end write memory Tail the log:\n1 sudo tail -f /var/log/frr/bgpd.log Confirm configuration persistence:\n1 vtysh -c \u0026#34;show running-config\u0026#34; FRR saves to /etc/frr/frr.conf with the write memory command.\nConclusion FRRouting turns any Linux server into a full-featured BGP router. With iBGP for communication within a single AS, eBGP for peering with MikroTik or other vendors, prefix-list filtering for route hygiene, redistribution from OSPF, and community tagging for advanced policy, you have the same toolkit that ISPs and data centers use every day.\nFrom here you can explore more advanced topologies: anycast services using BGP (such as DNS anycast with Unbound), VXLAN EVPN for multi-tenant L2 stretching across hosts, or BGP-based load balancing with BGP PIC (Prefix Independent Convergence).\nThe FRR documentation at docs.frrouting.org covers every feature in detail. For homelab-specific routing topologies, check the earlier posts on MikroTik OSPF configuration and WireGuard performance tuning.\n","permalink":"https://blog.gntech.me/posts/2026-05-31-frrouting-bgp-linux-dynamic-routing-homelab/","summary":"Deploy FRRouting on Linux for BGP dynamic routing in your homelab. Covers FRR installation, iBGP/eBGP peering with MikroTik, route filtering, prefix-lists, and OSPF-to-BGP redistribution with production-ready configs.","title":"FRRouting BGP on Linux — Dynamic Routing for Your Homelab Network"},{"content":"MikroTik released RouterOS 7.23 to the stable channel on May 25, 2026, bringing several important fixes and security improvements. This release focuses on upgrade security, bridge reliability for MLAG deployments, and WiFi VLAN handling. RouterOS 7.24beta1 followed on May 26 with additional app platform and container improvements, confirming that RouterOS development continues at a steady pace.\nThis post covers every notable change in 7.23, the step-by-step upgrade process, and post-upgrade verification steps to ensure your network stays stable.\nRouterOS 7.23 HTTPS Upgrade Security Hardening The headline change in 7.23 is marked with a breaking-change indicator:\n!) upgrade — use HTTPS by default when connecting to MikroTik upgrade servers\nBefore 7.23, RouterOS devices contacted upgrade.mikrotik.com over plain HTTP to check for and download firmware updates. An attacker on the local network or upstream path could potentially inject a malicious firmware image via a man-in-the-middle attack. Starting with 7.23, HTTPS is the default transport for all upgrade-server communication.\nMikroTik also added the option to configure the HTTP/HTTPS mode:\n/system upgrade set http=no To enforce HTTPS only (recommended):\n/system upgrade set http=no /system upgrade check-update To revert to HTTP if you have compatibility issues with specific network configurations (not recommended):\n/system upgrade set http=yes Check your current upgrade channel and settings:\n/system upgrade print This change alone makes 7.23 a must-update for any MikroTik deployment. The security improvement eliminates a long-standing attack surface that affected every auto-upgrade across all RouterOS versions prior to 7.23.\nBridge MLAG MAC Synchronization Fix Multi-chassis Link Aggregation (MLAG) lets you connect a single downstream device to two switches for redundancy. The switches present themselves as a single logical LAG endpoint, which requires continuous MAC address synchronization between the peers.\nRouterOS 7.23 addresses a significant bug in MLAG deployments:\nbridge — improved MAC synchronization for MLAG\nIn earlier releases, MAC entries learned on one MLAG peer could take too long to propagate to the other peer during failover events. This caused temporary blackholing of traffic — the downstream switch sent frames to switch A, but switch A had already aged out the MAC or hadn\u0026rsquo;t received the update from switch B. Environments using LACP + MLAG on CRS3xx or CRS5xx series switches should notice improved convergence times during link or node failures.\nWiFi VLAN Dynamic Update Fixes A practical issue for wireless deployments is fixed:\nbridge — fixed dynamic VLAN update for WiFi interfaces\nWhen using CAPsMAN or local WiFi interfaces with VLAN tagging, bridge VLAN configuration changes sometimes failed to propagate to WiFi interfaces dynamically. Clients would lose connectivity until a manual interface restart or CAP reboot. RouterOS 7.23 ensures that VLAN membership updates for wireless interfaces apply in real time without service disruption.\nIf you manage APs with per-VLAN SSIDs or dynamic VLAN assignment via RADIUS, this fix is directly relevant to your setup.\nDHCP Snooping Enhancement DHCP snooping received a meaningful improvement:\nbridge — recognize more DHCP message types when dhcp-snooping is enabled\nRouterOS now processes DHCPDECLINE, DHCPRELEASE, and DHCPINFORM messages through snooping. Previously the snooping engine only tracked DHCPACK and DHCPOFFER messages for building the DHCP snooping binding table. This enhancement provides more complete visibility into DHCP activity on your network:\nDHCPDECLINE — detects when a client rejects an offered address (potential IP conflict) DHCPRELEASE — tracks when clients voluntarily release addresses DHCPINFORM — monitors clients requesting only configuration parameters without lease Enable DHCP snooping on your bridges if not already done:\n/interface bridge set [find] dhcp-snooping=yes Timezone Data and Other Smaller Changes timezone — updated timezone information from the tzdata2026b release. This affects systems using automatic timezone detection or DST transition rules. If you schedule maintenance windows or log analysis around DST boundaries, verify your timezone configuration after upgrading.\nupgrade — added the option to configure HTTP/HTTPS modes when connecting (mentioned above).\nPreparing Your MikroTik Device for Upgrade Before upgrading, run through this checklist:\nBack up everything /export file=backup-pre-723 /system backup save name=backup-pre-723 The /export gives you a human-readable script. The .backup file is binary but preserves everything including passwords and certificates. Keep both on a separate machine.\nCheck available storage /system resource print Verify you have at least 20 MB of free space. The 7.23 package is ~18 MB for most architectures.\nIdentify your architecture /system resource print Confirm your device model is supported by 7.23 (all current ARM, MIPS, MIPSBE, PPC, and x86 devices are covered).\nCheck current version /system resource print Note your current version. If you are on 7.21 or earlier, read the 7.22 changelog for any intermediate changes that might affect your configuration.\nUpgrading to RouterOS 7.23 — Step by Step Method 1: WinBox GUI Open WinBox and connect to your MikroTik device Click System \u0026gt; RouterBOARD or System \u0026gt; Upgrade The upgrade dialog shows available versions — select 7.23 (stable) Click Download \u0026amp; Upgrade The device downloads the package, reboots, and comes back online Method 2: CLI Upgrade This is more scriptable and works over SSH:\n/system upgrade check-update /system upgrade download /system reboot The first command checks the latest available version for your configured channel. The second downloads the packages. After reboot, the new version activates automatically.\nMethod 3: Manual Package Upload If your device is isolated from the internet:\nDownload the 7.23 NPK package from mikrotik.com/download Upload via WinBox (Files) or SCP: scp routeros-7.23-arm.npk admin@10.0.20.1: Reboot the device: /system reboot The package is installed during the boot process.\nVerifying a Successful Upgrade After the device reboots, confirm everything is working:\n/system resource print Look for version: 7.23 in the output.\nCheck upgrade logs:\n/log print where topics=upgrade Walk through these functional checks:\nBridge connectivity — ping between VLAN-tagged and untagged interfaces MLAG — if applicable, trigger a link failure on one peer and measure convergence WiFi clients — verify wireless clients on VLAN SSIDs connect and maintain DHCP leases DHCP snooping — check the binding table with /ip dhcp-snooping print Routing protocols — confirm BGP/OSPF neighbors are established Firewall rules — run your standard connectivity tests in both directions /ping 10.0.20.1 count=5 /ip dhcp-snooping print /ip route print Should You Upgrade to RouterOS 7.23? Upgrade immediately if:\nYour devices have direct internet access (HTTPS upgrade fix applies) You run MLAG between CRS3xx/5xx switches You use CAPsMAN with VLAN-tagged SSIDs You rely on DHCP snooping for network security Test in staging first if:\nYou run production containers on RouterOS (consider beta testing with 7.24beta1) You have custom firewall scripts or BGP policies (safe in 7.23, but always test) Your devices are in remote locations without console access Looking Ahead to RouterOS 7.24beta RouterOS 7.24beta1 was released the day after 7.23 stable, on May 26, 2026. Early changes include:\nadlist — improved service stability when adjusting adlist configuration app — added network-outgoing permission for app platform apps If you run the RouterOS container feature or the adlist/AdBlock functionality, the 7.24 development branch is worth watching. The network-outgoing permission for apps is particularly interesting for deploying custom monitoring or integration tools directly on your MikroTik device.\nConclusion RouterOS 7.23 is a focused stable release that rounds out the bridge and upgrade security improvements introduced in the 7.22 cycle. The HTTPS-default upgrade change is the most impactful — it closes a security gap that affected every auto-upgrade scenario since RouterOS gained internet-based upgrade capabilities. Combined with MLAG MAC synchronization fixes and WiFi VLAN dynamic updates, 7.23 is a solid, recommended upgrade for all production MikroTik devices.\nDownload RouterOS 7.23 at mikrotik.com/download or check for it via your device\u0026rsquo;s built-in upgrade system. For a searchable changelog across all RouterOS versions, check out MikroTik Changelog Tracker by Hextet Systems.\n","permalink":"https://blog.gntech.me/posts/2026-05-31-mikrotik-routeros-7-23-stable-upgrade-guide/","summary":"RouterOS 7.23 stable is out with HTTPS-default upgrades, improved MLAG bridge MAC synchronization, WiFi VLAN fixes, and DHCP snooping message-type recognition. This guide covers the changelog, key fixes, and step-by-step upgrade steps for your MikroTik devices.","title":"MikroTik RouterOS 7.23: HTTPS, MLAG Fixes, and Upgrade Guide"},{"content":"Your Proxmox host has NVMe SSDs, a ZFS pool tuned to the gills, and plenty of RAM. But your VMs still feel sluggish under disk-heavy workloads — database commits take too long, file transfers stall, and iowait climbs uncomfortably high.\nThe default Proxmox VM configuration is safe but leaves significant performance on the table. A few targeted tweaks to the disk controller, iothread configuration, cache settings, and async IO engine can double or triple your VM\u0026rsquo;s disk throughput without spending a cent on hardware.\nThis guide walks through each tuning knob in order of impact: controller selection → iothreads → cache mode → discard/trim → async IO → benchmarking. Apply the ones that match your storage layout, measure results, and keep what works.\nChoosing the Right Disk Controller — virtio-blk vs virtio-scsi Proxmox offers two virtio-based disk controller families for Linux guests. The choice directly affects queue depth, parallelism, and overall throughput.\nvirtio-blk is the simpler option. It is baked into the Linux kernel since 2.6.24, requires zero extra drivers, and introduces minimal overhead. By default it exposes a single I/O queue. That single queue limits parallelism when multiple processes or threads hit the disk simultaneously. Starting with QEMU 6.0 and Proxmox 8, virtio-blk supports multi-queue via the num-queues property, which matches the number of vCPUs assigned to the VM.\nvirtio-scsi is the more modern choice. It supports multiple I/O queues out of the box, a default queue depth of 256 (versus 128 for virtio-blk), and better integration with SCSI-3 persistent reservations. The virtio-scsi-single controller model creates one IOThread per disk — ideal for workloads with concurrent access patterns.\nWhen to use each:\nCriterion virtio-blk virtio-scsi Overhead Lowest Very low Default queue depth 128 256 Multi-queue Manual (num-queues=N) Built-in Best for Single-process, low-concurrency workloads Databases, file servers, multi-threaded apps Pass-through features Limited Full SCSI support, TRIM pass-through For most homelab VMs running Docker, databases, or file storage, virtio-scsi-single is the better default. For minimal-overhead single-user VMs (e.g., a build runner), virtio-blk with multi-queue is fine.\nFor an existing VM, first record its full configuration and create a recoverable backup. Changing disk buses can make the guest unbootable if its drivers or boot settings are not prepared. A command such as --scsi0 STORAGE:32 allocates a new disk; it does not migrate your existing boot disk.\nUse the Proxmox hardware editor to plan any bus/controller change with the VM stopped. Preserve the original disk volume and all its options. Do not append raw args: -device ... lines to an existing VM to add multi-queue: that can duplicate a device already generated by Proxmox.\nCheck your current controller with:\n1 qm config \u0026#34;\u0026lt;VMID\u0026gt;\u0026#34; | grep -E \u0026#39;(scsihw|virtio|scsi)\u0026#39; IOThreads — Dedicated I/O Processing Pins QEMU can handle disk I/O on its main event loop or on dedicated I/O threads, depending on the configured device and backend. When a heavy write operation blocks, it steals cycles from the guest\u0026rsquo;s compute. IOThreads carve out dedicated threads for I/O processing, isolating storage latency from CPU-bound workloads.\nFor a supported disk/controller combination, enable IO thread in that disk\u0026rsquo;s Advanced options in the Proxmox hardware editor. Preserve its existing volume, cache, discard, and other settings. qm set VMID --virtio0 iothread=1 alone is incomplete because the disk specification must include its volume.\nThen verify the iothread is running on the host:\n1 ps aux | grep iothread For maximum benefit, pin iothreads to physical cores that are not shared with vCPUs. On a 6-core Proxmox host, a sensible layout reserves cores 4-5 for iothreads:\nEdit /etc/pve/nodes/\u0026lt;node\u0026gt;/qemu-server/\u0026lt;VMID\u0026gt;.conf and add after the iothread lines:\ncpuunits: 1024 cpulimit: 4 affinity: 0-3 Then pin iothread processes via taskset:\n1 2 3 4 5 # Find iothread PIDs ps -eo pid,comm | grep iothread # Pin to cores 4-5 taskset -cp 4-5 \u0026#34;\u0026lt;PID\u0026gt;\u0026#34; A manual thread affinity change is temporary; thread IDs can change when the VM restarts. Do not treat a copied PID or an assumed GUI feature as a persistent pinning configuration. Measure scheduling and I/O behavior before adding affinity rules.\nDisk Cache Mode Comparison The disk cache mode controls how QEMU interacts with the host page cache and how it reports write completion to the guest. Proxmox offers five modes — in a homelab with ZFS, only two matter:\ncache=none — the Proxmox default since version 2.x. QEMU opens the disk with O_DIRECT, bypassing the host page cache entirely. The guest sees a writeback cache (so it sends flushes when needed), and ZFS handles all caching through its ARC. This avoids double-caching and is the recommended mode when the backing storage is ZFS.\ncache=writeback — data lands in the host page cache before being written to disk. Writes complete faster from the guest\u0026rsquo;s perspective, but the host page cache adds another layer. On hardware RAID with battery-backed write cache (BBU), this is fine. On ZFS without a UPS, a power loss can lose data sitting in the host page cache.\nZFS recommendation: Use cache=none. ZFS ARC already caches frequently accessed data in RAM. Adding the host page cache on top wastes memory that ARC could put to better use.\n1 2 3 4 5 # Set cache mode per disk qm set \u0026#34;\u0026lt;VMID\u0026gt;\u0026#34; --virtio0 cache=none # For NVMe without ZFS: qm set \u0026#34;\u0026lt;VMID\u0026gt;\u0026#34; --virtio0 cache=writeback Enabling Discard / TRIM for Thin-Provisioned Storage If you use thin provisioning on ZFS datasets or qcow2 images, discarded blocks in the guest should be forwarded to the host so ZFS can reclaim space. Without discard, deleted files inside the VM leave allocated blocks on the host forever.\nEnable discard on the Proxmox disk:\n1 qm set \u0026#34;\u0026lt;VMID\u0026gt;\u0026#34; --virtio0 discard=on Inside the guest (Linux), run an immediate trim and enable the periodic timer:\n1 2 fstrim -av systemctl enable fstrim.timer --now For Debian/Ubuntu guests, the fstrim.timer runs weekly by default. On Proxmox hosts using ZFS, verify that discard is freeing space:\n1 zpool get freeing rpool A non-zero freeing value means blocks are actively being reclaimed.\nAsync I/O Engine Selection — threads vs native QEMU can dispatch disk I/O using two engines:\nthreads (default) — spawns a POSIX thread per I/O request. Simple and compatible with all storage backends, but context-switching overhead shows at high IOPS. native — uses Linux kernel AIO (io_submit/io_getevents). Lower overhead per I/O, better for high-queue-depth workloads. Switch to native AIO on ZFS or raw block devices:\n1 qm set \u0026#34;\u0026lt;VMID\u0026gt;\u0026#34; --virtio0 aio=native Caveat: Some older Linux kernels (pre-5.4) and Ceph RBD backends have issues with aio=native. If your host kernel is 5.15+ (standard on Proxmox 8.x and 9.x), native AIO is safe and recommended.\nBenchmarking Disk Performance — fio and qemu-img Never apply tuning blindly. Measure before and after with real workloads.\nInside the guest — fio is the standard benchmark tool:\n1 2 3 4 5 6 7 8 9 10 11 12 # Install fio on Debian/Ubuntu guest apt install fio # 4K random read/write — simulates database IO fio --name=rand --ioengine=libaio --iodepth=32 --rw=randrw \\ --bs=4k --direct=1 --size=1G --numjobs=4 \\ --group_reporting --runtime=60 --time_based # Sequential throughput test fio --name=seq --ioengine=libaio --iodepth=64 --rw=read \\ --bs=1M --direct=1 --size=4G --numjobs=2 \\ --group_reporting --runtime=30 --time_based On the host — benchmark the raw block device through QEMU\u0026rsquo;s block layer:\n1 2 # QEMU built-in benchmark qemu-img bench -c 1024 -d 64 -p -f raw /dev/zvol/rpool/data/vm-\u0026#34;\u0026lt;VMID\u0026gt;\u0026#34;-disk-0 Key metrics to track:\n4K random IOPS — the single best indicator of real-world VM performance 99th percentile latency — matters more than average for databases Sequential throughput — important for file transfers and backups A tuned VM on consumer NVMe (e.g., Samsung 990 Pro) through Proxmox with ZFS should hit 150k-200k+ random read IOPS with sub-millisecond p99 latency.\nPutting It All Together For a production homelab VM on ZFS-backed storage, here is the complete tuning command:\n1 2 3 qm set \u0026#34;\u0026lt;VMID\u0026gt;\u0026#34; \\ --scsihw virtio-scsi-single \\ --scsi0 local-zfs:32,iothread=1,cache=none,discard=on,aio=native Setting ZFS (recommended) Raw NVMe Ceph/RBD Controller virtio-scsi-single virtio-blk (multi-queue) virtio-scsi-single Cache mode none writeback none IOThread yes yes yes Discard on on off aio native native threads Test each change independently on your actual workloads. Storage topology, CPU count, and RAM all interact with these settings. There is no universal \u0026ldquo;fastest\u0026rdquo; config — only the best config for your specific workload on your specific hardware.\nFor deeper reading, see the Proxmox VE Performance Tweaks Wiki and the QEMU block layer documentation.\n","permalink":"https://blog.gntech.me/posts/2026-05-31-proxmox-vm-disk-performance-optimization/","summary":"A practical guide to tuning Proxmox VM disk performance — covering virtio disk controller selection, iothread pinning, cache modes, discard/trim, and async IO settings for maximum throughput on ZFS and NVMe storage.","title":"Proxmox VM Disk Performance — virtio-blk, iothreads, cache modes, and async IO"},{"content":"Why Docker Compose Override Files Matter in a Homelab You have a Docker Compose stack that works great in development — bind volumes for live code reload, relaxed resource limits, ports exposed for debugging. Then you want to run the same stack in \u0026ldquo;production\u0026rdquo; behind Traefik, with resource constraints, restart policies, and read-only filesystems. Do you maintain two separate docker-compose.yml files? Branch the repo? Use profiles?\nNone of those scale well. Override files are the clean solution Docker Compose provides for exactly this scenario.\nDocker Compose lets you merge multiple compose files together. A base file defines the shared structure — services, networks, volumes. Override files layer on environment-specific changes: different port mappings, volumes, resource limits, and security settings. The same config drives dev, staging, and prod without duplication.\nIn my homelab, I run a web application stack across three environments: an LXC container for development on my Proxmox host, a staging VM for integration testing, and a production VM behind Traefik. Override files keep all three in one directory.\nHow Docker Compose File Merging Works Docker Compose merges multiple YAML files into a single configuration before applying it. The merge follows these rules:\nMaps (dictionaries) are merged recursively — if the base file defines environment keys and the override defines environment keys, both sets are combined. Arrays (sequences) are fully replaced — if the base file has a ports array and the override has a ports array, the override wins entirely. Scalar values from the override replace the base. This behaviour matters. A common mistake is putting override ports alongside base ports and wondering why both appear.\nDefault Override File When you run docker compose up -d without the -f flag, Docker Compose automatically loads docker-compose.override.yml if it exists alongside docker-compose.yml. This makes docker-compose.override.yml ideal for local development overrides — checked-into-git structures, secrets in .env.\nExplicit Named Override Files For staging, production, or other environments, use the -f flag to specify exactly which files to merge:\n1 2 docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d docker compose -f docker-compose.yml -f docker-compose.staging.yml up -d The later file wins in conflicts. The project name defaults to the directory name. Override it with -p:\n1 docker compose -p myapp -f docker-compose.yml -f docker-compose.prod.yml up -d Base Configuration — Shared docker-compose.yml Start with a base file that defines the common structure for all environments. Use YAML extension fields (x- prefixed keys) to keep it DRY:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 x-app-defaults: \u0026amp;app-defaults restart: unless-stopped networks: - backend environment: - TZ=America/Santo_Domingo - APP_ENV=${APP_ENV:-development} x-healthcheck: \u0026amp;healthcheck healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;curl\u0026#34;, \u0026#34;-f\u0026#34;, \u0026#34;http://localhost:80/health\u0026#34;] interval: 30s timeout: 5s retries: 3 start_period: 15s services: app: image: myapp:latest container_name: myapp \u0026lt;\u0026lt;: [*app-defaults, *healthcheck] expose: - \u0026#34;9000\u0026#34; db: image: mariadb:11 container_name: myapp-db \u0026lt;\u0026lt;: *app-defaults volumes: - db-data:/var/lib/mysql environment: MYSQL_ROOT_PASSWORD: ${DB_ROOT_PASSWORD} MYSQL_DATABASE: ${DB_NAME:-myapp} expose: - \u0026#34;3306\u0026#34; web: image: nginx:alpine container_name: myapp-web \u0026lt;\u0026lt;: *app-defaults volumes: - ./nginx.conf:/etc/nginx/conf.d/default.conf:ro depends_on: - app expose: - \u0026#34;80\u0026#34; volumes: db-data: networks: backend: driver: bridge Key points: services use expose (internal only, no host ports), extension fields keep configs consistent, and APP_ENV defaults to development.\nDevelopment Override — docker-compose.override.yml This file loads automatically when you run docker compose up -d in the project directory:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 services: app: ports: - \u0026#34;127.0.0.1:9000:9000\u0026#34; volumes: - ./src:/var/www/html:cached environment: APP_DEBUG: \u0026#34;1\u0026#34; XDEBUG_MODE: debug db: ports: - \u0026#34;127.0.0.1:3306:3306\u0026#34; volumes: - db-data-dev:/var/lib/mysql environment: MYSQL_ROOT_PASSWORD: devpassword MYSQL_DATABASE: ${DB_NAME:-myapp} web: ports: - \u0026#34;127.0.0.1:8080:80\u0026#34; volumes: - ./src:/var/www/html:cached - ./nginx.dev.conf:/etc/nginx/conf.d/default.conf:ro volumes: db-data-dev: What the dev override does:\nBinds service ports to 127.0.0.1 only — accessible from the Docker host but not the network. Bind-mounts source code into app and web for live reload. Enables debug mode and Xdebug for the PHP/Node.js app service. Sets a simple dev password in the environment (never in the base file). Uses a separate named volume for dev database data so it doesn\u0026rsquo;t pollute production volumes. Production Override — docker-compose.prod.yml For production, add resource limits, healthchecks, read-only filesystems, and reverse proxy integration:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 services: app: ports: [] restart: always read_only: true security_opt: - no-new-privileges:true cap_drop: - ALL cap_add: - NET_BIND_SERVICE deploy: resources: limits: cpus: \u0026#34;2\u0026#34; memory: \u0026#34;512M\u0026#34; reservations: cpus: \u0026#34;0.5\u0026#34; memory: \u0026#34;256M\u0026#34; logging: driver: journald options: tag: myapp labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.myapp.rule=Host(`app.yourdomain.com`)\u0026#34; - \u0026#34;traefik.http.routers.myapp.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.services.myapp.loadbalancer.server.port=80\u0026#34; db: restart: always deploy: resources: limits: memory: \u0026#34;1G\u0026#34; reservations: memory: \u0026#34;512M\u0026#34; volumes: - db-data-prod:/var/lib/mysql logging: driver: journald healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;healthcheck.sh\u0026#34;, \u0026#34;--connect\u0026#34;, \u0026#34;--innodb_initialized\u0026#34;] interval: 15s timeout: 10s retries: 5 start_period: 30s web: restart: always read_only: true deploy: resources: limits: memory: \u0026#34;128M\u0026#34; logging: driver: journald volumes: db-data-prod: Key production hardening:\nread_only: true prevents any container filesystem writes. security_opt: no-new-privileges:true prevents privilege escalation inside the container. cap_drop: ALL followed by minimal cap_add implements least-privilege. deploy.resources.limits caps CPU and memory — essential on a shared homelab host. deploy.resources.reservations guarantees minimum resources. logging.driver: journald sends logs to the host\u0026rsquo;s systemd journal, centralised with Loki or your log aggregator. Traefik labels route traffic to the web service. The ports: [] in the app service ensures no direct host port exposure. Separate named volumes for production data prevent accidental data overwrite. Staging Override — docker-compose.staging.yml Staging sits between dev and prod — similar structure to prod but relaxed limits and a different DNS name:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 services: app: ports: - \u0026#34;127.0.0.1:9001:9000\u0026#34; restart: always deploy: resources: limits: cpus: \u0026#34;1\u0026#34; memory: \u0026#34;256M\u0026#34; environment: APP_DEBUG: \u0026#34;1\u0026#34; labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.myapp-staging.rule=Host(`staging.yourdomain.com`)\u0026#34; - \u0026#34;traefik.http.routers.myapp-staging.entrypoints=websecure\u0026#34; db: restart: always deploy: resources: limits: memory: \u0026#34;512M\u0026#34; volumes: - db-data-staging:/var/lib/mysql web: restart: always deploy: resources: limits: memory: \u0026#34;96M\u0026#34; volumes: db-data-staging: Running Each Environment With all files in place, the workflow is clean:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # Development — auto-loads docker-compose.override.yml cd /opt/myapp docker compose up -d # Staging — explicit merge with staging override docker compose -f docker-compose.yml -f docker-compose.staging.yml -p myapp-staging up -d # Production — explicit merge with prod override docker compose -f docker-compose.yml -f docker-compose.prod.yml -p myapp-prod up -d # Check what\u0026#39;s running docker compose -p myapp-staging ps docker compose -p myapp-prod ps Shell Aliases for Convenience Add these to your .bashrc or .zshrc:\n1 2 3 4 5 6 alias dc=\u0026#39;docker compose\u0026#39; alias dcdev=\u0026#39;docker compose up -d\u0026#39; alias dcstag=\u0026#39;docker compose -f docker-compose.yml -f docker-compose.staging.yml -p myapp-staging up -d\u0026#39; alias dcprod=\u0026#39;docker compose -f docker-compose.yml -f docker-compose.prod.yml -p myapp-prod up -d\u0026#39; alias dcstag-down=\u0026#39;docker compose -p myapp-staging down\u0026#39; alias dcprod-down=\u0026#39;docker compose -p myapp-prod down\u0026#39; Gitignore Strategy Keep the override structure clean in version control:\n# .gitignore .env # Ignore the auto-loaded dev override — it\u0026#39;s personal docker-compose.override.yml # Track named overrides — they define environment configs # docker-compose.prod.yml # docker-compose.staging.yml The auto-loaded docker-compose.override.yml is ignored because it contains personal dev preferences and possibly secrets. Named overrides (prod.yml, staging.yml) are tracked in git — they are infrastructure definitions, not secrets.\nAdvanced Patterns: Extension Fields with YAML Anchors Extension fields (x-*) let you reuse YAML blocks across services and override files:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 # docker-compose.yml x-logging: \u0026amp;default-logging driver: json-file options: max-size: \u0026#34;10m\u0026#34; max-file: \u0026#34;3\u0026#34; x-healthcheck-web: \u0026amp;healthcheck-web test: [\u0026#34;CMD\u0026#34;, \u0026#34;wget\u0026#34;, \u0026#34;-qO-\u0026#34;, \u0026#34;http://localhost:80/health\u0026#34;] interval: 30s timeout: 10s retries: 3 start_period: 10s services: app: image: myapp:latest logging: *default-logging web: image: nginx:alpine logging: *default-logging healthcheck: *healthcheck-web In production override, swap the logging driver:\n1 2 3 4 5 6 7 8 9 # docker-compose.prod.yml x-logging: \u0026amp;default-logging driver: journald services: app: logging: *default-logging web: logging: *default-logging Override files merge YAML anchors just like any other map key.\nSecurity Considerations for Override Deployments Never put secrets in override files tracked in git. Use .env files with .gitignore for passwords, API keys, and tokens. Override files are for structure and config — not secrets.\nDev overrides should bind to 127.0.0.1, not 0.0.0.0, to prevent accidental network exposure on your homelab.\nProduction overrides should strip all dev ports. Use ports: [] or expose only. If you need host access, route through your reverse proxy exclusively.\nResource limits prevent noisy-neighbour problems. Without deploy.resources.limits, one container can starve others on the same Docker host.\nRead-only root filesystem + no-new-privileges should be default for production containers. Override files make this easy — define it once in the override, not in every base service.\nConclusion Docker Compose override files solve the multi-environment problem without branching, template engines, or duplicate compose files. A single docker-compose.yml holds the shared structure. docker-compose.override.yml provides dev defaults automatically. Named override files (docker-compose.prod.yml, docker-compose.staging.yml) layer on environment-specific hardening, resources, and network config.\nIn my homelab, this pattern eliminated config drift between environments. Port mapping, volume mounts, resource limits, and security settings each live in the right layer. Using extension fields and YAML anchors keeps everything DRY. One directory, three environments, zero duplication.\nSee Also Docker Compose Profiles — Selective Service Management for Homelab Docker Compose Environment Variables — .env, Substitution, and Secrets Docker Compose Production Patterns ","permalink":"https://blog.gntech.me/posts/2026-05-31-docker-compose-override-files-multi-environment/","summary":"Learn how Docker Compose override files let you manage dev, staging, and production deployments from a single base docker-compose.yml — with port mapping overrides, resource limits, volume mounts, and security hardening for your homelab.","title":"Docker Compose Override Files — Multi-Environment Homelab Deployments"},{"content":"Why Self-Host a Meta Search Engine Every search query you type into Google, Bing, or DuckDuckGo sends your IP address, browser fingerprint, and search terms to servers you don\u0026rsquo;t control. SearXNG solves this by acting as a privacy proxy between you and the search engines — your browser talks to your SearXNG instance, and SearXNG queries the upstream engines on your behalf without leaking your IP.\nRunning SearXNG in your homelab gives you:\nFull visibility into which engines handle each query Engine selection per category (general, files, images, maps, social media) No rate limits from shared public instances Custom themes and dark mode out of the box Self-contained stack — Docker Compose, no external dependencies I run SearXNG at my homelab behind Traefik, and it replaced browser-based search entirely. On mobile, I set the instance as the default search engine in Firefox. One config file controls which engines are enabled, how queries are routed, and what data is stored.\nPrerequisites for SearXNG Docker Deployment Before starting, make sure you have:\nDocker and Docker Compose installed on your Docker host (a Proxmox LXC or low-resource VM works fine — SearXNG runs on 512 MB RAM) A domain name pointing to your homelab (optional but recommended for Traefik) Traefik or another reverse proxy already running (this guide uses Traefik) The SearXNG stack uses two containers: the SearXNG application itself and Valkey (the Redis-compatible key-value store) for caching and rate limiting.\nDirectory Structure Create a clean directory for the stack:\n1 2 mkdir -p /opt/searxng cd /opt/searxng SearXNG Docker Compose Setup The official SearXNG compose template uses a docker-compose.yml and a .env file. Here is a version configured for homelab use with Traefik:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 # /opt/searxng/docker-compose.yml services: core: image: searxng/searxng:latest container_name: searxng-core restart: unless-stopped volumes: - ./core-config:/etc/searxng:rw - ./data:/var/cache/searxng:rw environment: - SEARXNG_BASE_URL=https://search.yourdomain.com - SEARXNG_SECRET_KEY=${SEARXNG_SECRET_KEY:?err} cap_drop: - ALL cap_add: - CHOWN - SETGID - SETUID networks: - proxy - internal labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.searxng.rule=Host(`search.yourdomain.com`)\u0026#34; - \u0026#34;traefik.http.routers.searxng.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.services.searxng.loadbalancer.server.port=8080\u0026#34; valkey: image: valkey/valkey:8-alpine container_name: searxng-valkey restart: unless-stopped volumes: - valkey-data:/data cap_drop: - ALL cap_add: - SETGID - SETUID - SETRLIMIT networks: - internal command: valkey-server --save 3600 1 --loglevel warning volumes: valkey-data: networks: proxy: external: true name: traefik-net internal: driver: bridge Generate a strong secret key for the .env file:\n1 openssl rand -hex 48 Then create /opt/searxng/.env:\n1 SEARXNG_SECRET_KEY=\u0026#34;\u0026lt;paste the 96-character hex string\u0026gt;\u0026#34; What the Compose File Does core — The SearXNG application listening on port 8080 inside the container. The SEARXNG_BASE_URL tells SearXNG what external URL to use for link generation. The SEARXNG_SECRET_KEY encrypts session data and rate limiter state. valkey — In-memory cache used by SearXNG for rate limiting and temporary data. Because it replaces Redis, all SearXNG Redis configuration works against it with no changes. The --save 3600 1 flag writes a snapshot every hour if at least one change occurred. networks — The proxy network is external (shared with Traefik). The internal network isolates Valkey from the outside. cap_drop / cap_add — Minimal Linux capabilities for container hardening. Configuring SearXNG Search Engines and Features SearXNG generates a default settings.yml on first start. You can override it by placing a file in the mounted ./core-config directory.\nCreate /opt/searxng/core-config/settings.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 # /opt/searxng/core-config/settings.yml use_default_settings: true search: # Safe search: 0 = off, 1 = moderate, 2 = strict safe_search: 0 # Language set in search results default_lang: \u0026#34;en\u0026#34; # Format autocomplete suggestions autocomplete: \u0026#34;google\u0026#34; # Output formats formats: - html - csv - json - rss server: # Do not register this instance in the public instance list public_instance: false # Show the instance name in the UI instance_name: \u0026#34;GnTech Search\u0026#34; # Enable image proxy to hide Referer headers image_proxy: true # Enable method to limit requests limiter: true # Redis/Valkey URL for the limiter redis_url: \u0026#34;redis://valkey:6379/0\u0026#34; ui: # Default theme: simple, oscar default_theme: simple # Dark mode: auto, dark, light default_theme_style: dark # Custom CSS (optional) # static_path: \u0026#34;/etc/searxng/static/themes/simple/css/custom.css\u0026#34; # Engines — enable the ones you use, disable the rest engines: # Google Web - name: google disabled: false # DuckDuckGo (no JS required) - name: duckduckgo disabled: false # Bing - name: bing disabled: false # Brave Search - name: brave disabled: false # Wikipedia - name: wikipedia disabled: false # GitHub - name: github disabled: false # StackOverflow - name: stackoverflow disabled: false # Reddit - name: reddit disabled: false # Docker Hub - name: docker_hub disabled: false # YouTube (Invidious instance) - name: invidious disabled: false instance_url: \u0026#34;https://inv.nadeko.net\u0026#34; # Pixabay images - name: pixabay disabled: false # DuckDuckGo definitions - name: duckduckgo_definitions disabled: false # Disable less useful defaults - name: 1x disabled: true - name: apk_mirror disabled: true - name: google_images disabled: true - name: google_news disabled: true - name: google_videos disabled: true - name: media.ccc.de disabled: true - name: mixcloud disabled: true - name: rumble disabled: true - name: soundcloud disabled: true - name: twitcher disabled: true Key Configuration Points use_default_settings: true — Loads the built-in defaults first, then your overrides apply on top. This keeps the file small and maintainable. image_proxy: true — Images served through SearXNG, not directly from the source, preventing referrer leakage. limiter: true — Enables rate limiting using Valkey. Without this, any bot can hammer your upstream search engines and get your IP rate-limited. redis_url: \u0026quot;redis://valkey:6379/0\u0026quot; — Points to the Valkey container in the internal Docker network. The redis:// protocol prefix works because Valkey is Redis-protocol compatible. public_instance: false — Prevents your instance from appearing in the public SearXNG instance list. Keep your instance private. default_theme_style: dark — Dark mode by default, with the simple theme. Exposing SearXNG with Traefik Reverse Proxy If you already run Traefik as a Docker reverse proxy, the compose file labels above handle the routing. Make sure you have:\nA DNS A record for search.yourdomain.com pointing to your homelab The traefik-net Docker network available (adjust the name to match your setup) Traefik\u0026rsquo;s ACME (Let\u0026rsquo;s Encrypt) configuration active for TLS Create or verify the Docker network:\n1 docker network create traefik-net The Traefik labels in the compose file:\ntraefik.http.routers.searxng.rule=Host(\\search.yourdomain.com`)` — Routes traffic for that hostname to SearXNG traefik.http.routers.searxng.entrypoints=websecure — Uses the HTTPS entrypoint traefik.http.services.searxng.loadbalancer.server.port=8080 — Points to the container\u0026rsquo;s internal port 8080 If you use Nginx Proxy Manager or Caddy instead, the setup is similar — just proxy pass to http://searxng-core:8080 from the reverse proxy container.\nStarting SearXNG and Verifying the Deployment Start the stack:\n1 2 cd /opt/searxng docker compose up -d Check the logs to confirm startup:\n1 docker compose logs -f core Look for the line:\n[INFO] starting server on 0.0.0.0:8080 Visit your instance at https://search.yourdomain.com. You should see the SearXNG search interface with your configured instance_name.\nTesting Searches Type a query and verify:\nResults load from multiple engines The \u0026ldquo;Categories\u0026rdquo; dropdown shows enabled categories The preferences page (/preferences) lists your configured settings Dark mode renders correctly Common Issues \u0026ldquo;Invalid base_url\u0026rdquo; error in logs — The SEARXNG_BASE_URL in docker-compose.yml must match the actual URL you are accessing. If you are testing locally, use http://localhost:8080.\nSearch returns no results — Some engines require JavaScript and may not work with all themes. Try the simple theme if results fail with oscar. Check docker compose logs core for engine-specific error messages.\nRate limiter errors — If you see 429 Too Many Requests for upstream engines, SearXNG respects upstream rate limits. Reduce query frequency or add a delay in settings.yml:\n1 2 3 4 search: safe_search: 0 # Add delay in ms between queries to avoid rate limiting request_delay: 1.0 Maintaining Your SearXNG Instance Updating Containers Pull the latest images and restart:\n1 2 3 4 cd /opt/searxng docker compose down docker compose pull docker compose up -d Always check the SearXNG release notes before updating, as configuration keys occasionally change.\nBackup Back up two directories — everything else is disposable:\n1 tar czf searxng-backup-$(date +%F).tar.gz ./core-config/ ./data/ Store the backup off-server (Proxmox Backup Server, remote storage).\nMonitoring Add a health check to the SearXNG service in docker-compose.yml:\n1 2 3 4 5 6 7 8 9 services: core: # ... existing config healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;wget -qO- http://localhost:8080/health || exit 1\u0026#34;] interval: 30s timeout: 5s retries: 3 start_period: 15s SearXNG exposes a /health endpoint that returns a 200 status when the application is running and engines are responsive.\nConclusion SearXNG replaces public search engines with a self-hosted, privacy-respecting alternative that gives you full control over which engines process your queries, what data is cached, and how rate limiting works. The Docker Compose stack runs on minimal hardware, integrates with any reverse proxy, and costs nothing beyond your homelab\u0026rsquo;s electricity.\nSet up search categories — images through Pixabay, code through GitHub and StackOverflow, general queries through DuckDuckGo and Brave. Disable the engines you never use. Drop the default search engine into your browser settings and never expose your IP to a search engine again.\nSee Also SearXNG Official Documentation Docker Compose Production Patterns Traefik Middleware Security Hardening ","permalink":"https://blog.gntech.me/posts/2026-05-30-searxng-self-hosted-meta-search-engine-docker/","summary":"Deploy SearXNG with Docker Compose for a privacy-focused self-hosted meta search engine. Covers reverse proxy setup with Traefik, search engine configuration, rate limiting, and theme customization.","title":"SearXNG — Self-Hosted Private Meta Search Engine with Docker"},{"content":"Portainer is the most widely deployed container management UI in the self-hosted world, and for good reason. It gives you a clean web interface for everything you normally type into a terminal — starting and stopping containers, inspecting logs, pulling images, managing volumes and networks, and deploying full Docker Compose stacks.\nWhen your homelab runs five containers, docker ps and a handful of aliases are fine. When it runs thirty containers across three Docker hosts plus a handful of Proxmox LXCs, the CLI overhead adds up fast. Portainer absorbs that overhead without requiring you to abandon the terminal entirely — it complements the CLI rather than replacing it.\nThis guide covers deploying Portainer Community Edition with Docker Compose, setting up agents for multi-host management, deploying stacks through the UI, hardening the deployment with a socket proxy, and backing up your Portainer database so you never lose your configuration.\nPortainer CE vs Business Edition — Which One for Your Homelab Portainer comes in two editions:\nCommunity Edition (CE) — Free, open source, and fully capable for single-user homelab environments. CE gives you the full dashboard, container management, image management, volumes, networks, stacks, and basic access controls.\nBusiness Edition (BE) — Adds RBAC with teams, registry management, Kubernetes support, and audit logging. These features matter in corporate environments with multiple teams sharing infrastructure, but are overkill for the typical homelab. BE requires a paid license after the free trial.\nFor this guide we deploy Portainer CE. If you need multi-user RBAC or Kubernetes later, migrating to BE is straightforward — the data directory is compatible between editions.\nDeploying Portainer CE with Docker Compose Portainer ships as a single container that mounts the Docker socket to manage the local host. The deployment is lightweight — Portainer itself consumes about 100 MB of RAM and near-zero CPU at idle.\nCreate a directory for Portainer and a docker-compose.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 services: portainer: image: portainer/portainer-ce:latest container_name: portainer restart: unless-stopped ports: - \u0026#34;9000:9000\u0026#34; volumes: - /var/run/docker.sock:/var/run/docker.sock:ro - portainer_data:/data environment: - TZ=America/Santo_Domingo networks: - proxy networks: proxy: external: true volumes: portainer_data: The socket mount (:ro for read-only) gives Portainer the API access it needs. The data volume stores the internal SQLite database, TLS certificates, user settings, and stack definitions. The proxy network assumes you already run Traefik or another reverse proxy — if you do not, remove the network section and access Portainer directly on port 9000.\nBring it up:\n1 docker compose up -d Access http://\u0026lt;host-ip\u0026gt;:9000 and create the admin user on first launch. Set a strong password — Portainer uses bcrypt hashing, and this account controls every Docker host you attach.\nTraefik Reverse Proxy Configuration If you already run Traefik, front Portainer with a proper domain and automatic TLS. Add Traefik labels to the compose file:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 services: portainer: image: portainer/portainer-ce:latest container_name: portainer restart: unless-stopped ports: - \u0026#34;9000:9000\u0026#34; volumes: - /var/run/docker.sock:/var/run/docker.sock:ro - portainer_data:/data environment: - TZ=America/Santo_Domingo networks: - proxy labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.portainer.rule=Host(`portainer.lab.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.portainer.entrypoints=https\u0026#34; - \u0026#34;traefik.http.routers.portainer.tls=true\u0026#34; - \u0026#34;traefik.http.routers.portainer.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.services.portainer.loadbalancer.server.port=9000\u0026#34; networks: proxy: external: true volumes: portainer_data: Replace portainer.lab.example.com with your actual domain. The entrypoint and cert resolver names must match your Traefik configuration.\nMulti-Host Management with Portainer Agents Portainer\u0026rsquo;s real value in a multi-host homelab is the agent architecture. An agent container runs on each additional Docker host and communicates back to the Portainer server over an encrypted tunnel on port 9001.\nAgent Deployment On each secondary Docker host, create a docker-compose.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 services: portainer_agent: image: portainer/agent:latest container_name: portainer_agent restart: unless-stopped ports: - \u0026#34;9001:9001\u0026#34; volumes: - /var/run/docker.sock:/var/run/docker.sock:ro - /var/lib/docker/volumes:/var/lib/docker/volumes:ro environment: - AGENT_CLUSTER_ADDR=tasks.portainer_agent networks: - agent_net networks: agent_net: name: portainer_agent The AGENT_CLUSTER_ADDR variable is required for Docker Swarm mode; for standalone hosts it is ignored but does no harm.\nConnecting an Agent to the Portainer Server Log into the Portainer UI at https://portainer.lab.example.com Go to Home → Add Endpoint Select Docker as the environment type Choose Agent as the method Enter the agent URL: http://\u0026lt;agent-host-ip\u0026gt;:9001 Click Connect The agent host immediately appears in the Portainer dashboard alongside your local endpoint. You can now manage containers, images, volumes, and networks on that host as if it were local to the server.\nFirewall Rules On Proxmox hosts or physical servers, restrict agent access to the Portainer server IP only:\n1 2 iptables -A INPUT -p tcp --dport 9001 -s 10.0.20.30 -j ACCEPT iptables -A INPUT -p tcp --dport 9001 -j DROP Replace 10.0.20.30 with your Portainer server IP. On MikroTik routers, create a corresponding firewall filter rule:\n/ip firewall filter add chain=input protocol=tcp dst-port=9001 \\ src-address=10.0.20.30 action=accept /ip firewall filter add chain=input protocol=tcp dst-port=9001 \\ action=drop Managing Containers Through the Portainer UI The Portainer dashboard presents a real-time overview of your environment. From the left sidebar you access every management function:\nContainers — Start, stop, restart, pause, kill, or remove containers. Click any container to view live logs, inspect configuration, exec into a shell, see resource usage charts (CPU, memory, network I/O), and attach or detach networks without typing a single docker command.\nImages — Pull images from Docker Hub or your private registry, tag them, push to registries, build from a Dockerfile (uploaded or pasted), and prune unused images.\nVolumes — Create, inspect, and remove volumes. The volume browser shows mounted containers, mount points, and size — useful for identifying orphaned volumes after stack removal.\nNetworks — Create custom bridge, macvlan, or ipvlan networks. Portainer shows which containers are attached to each network with their assigned IPs, which is invaluable when debugging connectivity between services.\nStack Deployment — Visual Docker Compose Portainer Stacks implement Docker Compose through the browser. This is the feature that makes Portainer indispensable in a multi-service homelab: you can version and deploy entire application stacks through a UI with git-backed sources.\nCreating a Stack from YAML Go to Stacks → Add Stack Name the stack (e.g., monitoring-stack) Choose Web editor and paste your compose file, or select Repository and provide a git URL and branch Optionally upload an .env file for environment variables Click Deploy the stack Portainer parses the compose file, creates the networks and volumes, pulls images if missing, and starts all services. The stack appears as a single deployable unit — redeploy to update, or remove the stack to tear down every associated resource.\nGit-Backed Stacks The repository option is the preferred workflow for production homelabs. Point Portainer at a GitHub, Gitea, or Forgejo repository containing your compose files. Portainer clones the repo, deploys the stack, and provides a Pull and redeploy button to apply updates:\nRepository URL: https://git.gntech.dev/infra/docker-compose.git Repository ref: refs/heads/main Compose path: monitoring/prometheus-grafana/docker-compose.yml This git-backed pattern means your infrastructure-as-code lives in a git repository, visible to team members, CI pipelines, and disaster recovery procedures, while Portainer handles the deployment mechanics.\nSecurity Hardening with Docker Socket Proxy The single biggest security concern with Portainer — or any tool that mounts the Docker socket — is that a compromise of the web UI gives an attacker root on the host. The Docker socket is root-equivalent.\nThe mitigation is a reverse proxy that sits between Portainer and the socket, exposing only the API endpoints Portainer actually needs. The community-standard solution is Tecnativa\u0026rsquo;s docker-socket-proxy.\nAdd it to your compose file:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 services: docker-socket-proxy: image: tecnativa/docker-socket-proxy:latest container_name: docker-socket-proxy restart: unless-stopped volumes: - /var/run/docker.sock:/var/run/docker.sock:ro environment: - CONTAINERS=1 - IMAGES=1 - NETWORKS=1 - VOLUMES=1 - TASKS=1 - SERVICES=1 - NODES=1 - INFO=1 - EVENTS=1 - AUTH=1 - POST=1 networks: - internal portainer: image: portainer/portainer-ce:latest container_name: portainer restart: unless-stopped ports: - \u0026#34;9000:9000\u0026#34; volumes: - portainer_data:/data environment: - TZ=America/Santo_Domingo - DOCKER_HOST=tcp://docker-socket-proxy:2375 networks: - proxy - internal labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.portainer.rule=Host(`portainer.lab.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.portainer.entrypoints=https\u0026#34; - \u0026#34;traefik.http.routers.portainer.tls=true\u0026#34; - \u0026#34;traefik.http.routers.portainer.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.services.portainer.loadbalancer.server.port=9000\u0026#34; networks: proxy: external: true internal: internal: true volumes: portainer_data: Key points:\nThe socket proxy runs with :ro (read-only) socket mount Environment flags grant only the Docker API endpoints Portainer needs The internal network is internal: true — no external connectivity Portainer connects to the proxy via DOCKER_HOST=tcp://docker-socket-proxy:2375 Portainer no longer mounts the socket at all This three-container pattern (proxy → portainer → UI) reduces the blast radius of a Portainer compromise to what the proxy allows, which is container management operations without direct host access.\nBackup and Restore Portainer Portainer stores everything — user accounts, endpoint definitions, stack configurations, and settings — in a SQLite database inside the portainer_data volume. Backing up this data is all you need.\nManual Backup 1 2 3 4 5 docker run --rm \\ --volumes-from portainer \\ -v \u0026#34;$(pwd):/backup\u0026#34; \\ alpine tar czf /backup/portainer-backup-$(date +%F).tar.gz \\ -C /data . This runs a temporary Alpine container that mounts the Portainer data volume and creates a compressed archive in the current directory.\nAutomated Backup with Systemd Timer Create a backup script at /usr/local/bin/backup-portainer.sh:\n1 2 3 4 5 6 7 8 9 #!/bin/bash BACKUP_DIR=\u0026#34;/opt/backups/portainer\u0026#34; mkdir -p \u0026#34;$BACKUP_DIR\u0026#34; docker run --rm \\ --volumes-from portainer \\ -v \u0026#34;$BACKUP_DIR:/backup\u0026#34; \\ alpine tar czf \u0026#34;/backup/portainer-$(date +%F).tar.gz\u0026#34; \\ -C /data . find \u0026#34;$BACKUP_DIR\u0026#34; -name \u0026#34;portainer-*.tar.gz\u0026#34; -mtime +30 -delete Make it executable and wire up a systemd timer:\n1 chmod +x /usr/local/bin/backup-portainer.sh # /etc/systemd/system/portainer-backup.service [Unit] Description=Portainer data backup [Service] Type=oneshot ExecStart=/usr/local/bin/backup-portainer.sh # /etc/systemd/system/portainer-backup.timer [Unit] Description=Daily Portainer backup [Timer] OnCalendar=daily Persistent=true [Install] WantedBy=timers.target Enable and start:\n1 2 systemctl daemon-reload systemctl enable --now portainer-backup.timer Restoring from Backup 1 2 3 4 5 6 7 docker compose down docker run --rm \\ -v portainer_data:/data \\ -v \u0026#34;$(pwd):/backup\u0026#34; \\ alpine tar xzf \u0026#34;/backup/portainer-2026-05-30.tar.gz\u0026#34; \\ -C /data docker compose up -d Stop Portainer, extract the backup into the data volume, then restart.\nReal-World Usage Tips Use container labels to organize your endpoints. In the Portainer UI, you can apply labels to containers, images, and volumes. Set labels like project=monitoring, project=media, or environment=production and filter the dashboard by label. This makes a cluttered dashboard navigable.\nWebhooks for auto-deploy. Portainer stacks with git sources support webhook triggers. Configure your Gitea or GitHub repository to send a POST request to Portainer\u0026rsquo;s webhook URL on push, and your stack redeploys automatically. Find the webhook URL in Stack details → Webhooks.\nSet resource limits through the UI. Under Container → Duplicate/Edit → Resource limits, set CPU and memory quotas. Portainer translates these to --cpus, --memory, and --memory-reservation in the container configuration. This prevents a runaway container from starving the rest of your services.\nUse maintenance mode. Under Endpoints → Maintenance, you can enable maintenance mode which stops all containers on the endpoint while preserving their configurations. Useful before host upgrades or Docker engine restarts.\nWrapping Up Portainer solves a real problem in the growing homelab: the gap between \u0026ldquo;a few containers you can manage from memory\u0026rdquo; and \u0026ldquo;enough containers that you need an inventory.\u0026rdquo; It does not replace docker compose or make you a worse operator — it gives you a dashboard for the operations you perform dozens of times a day and frees mental energy for the work that actually matters.\nDeploy CE on your main Docker host, add agents as you grow to multiple hosts, and always use the socket proxy if Portainer is exposed beyond your LAN. Your data volume is tiny — back it up daily with the systemd timer and you will never lose your configuration.\nReferences:\nPortainer Documentation Portainer CE GitHub Tecnativa Docker Socket Proxy Docker Compose Documentation ","permalink":"https://blog.gntech.me/posts/2026-05-30-portainer-docker-deployment-homelab/","summary":"Deploy Portainer with Docker Compose for visual container management across your homelab. Covers CE vs Business Edition, multi-host agent setup, stack deployment, security hardening, and backup strategies.","title":"Portainer Docker Deployment — Homelab Container Management Guide"},{"content":"CVE-2026-31431 is a Linux kernel vulnerability involving the AF_ALG AEAD cryptographic interface. It matters to Docker operators because ordinary containers share their host\u0026rsquo;s kernel.\nCorrection — October 30, 2026: The original article claimed that every unprivileged container could escape to host root, and treated a simple kernel version or module check as a complete exposure test. Those statements were too broad. Exposure depends on the host kernel\u0026rsquo;s patch status, available kernel functionality, and the container\u0026rsquo;s runtime restrictions. This revision uses the Linux CNA record and Docker\u0026rsquo;s security documentation as its baseline.\nWhat the Official Record Establishes The Linux CNA record describes the fix as reverting algif_aead to out-of-place operation. Its CVSS 3.1 score is 7.8 (High), with local access and low privileges required.\nThe record lists several fixed upstream branches, including 5.10.254, 5.15.204, 6.1.170, 6.6.137, 6.12.85, 6.18.22, 6.19.12, and 7.0. Distribution and Proxmox packages may backport fixes. Use your kernel vendor\u0026rsquo;s advisory and exact installed package revision, not an upstream version comparison alone.\n1. Inventory the Host and Runtime Run on the actual Docker host:\n1 2 3 4 uname -r cat /etc/os-release docker version docker info --format \u0026#39;{{json .SecurityOptions}}\u0026#39; On Debian-based systems, identify the running kernel\u0026rsquo;s package:\n1 dpkg-query -S \u0026#34;/boot/vmlinuz-$(uname -r)\u0026#34; If that path is unavailable, use your platform\u0026rsquo;s package inventory. Docker inside a Proxmox LXC still uses the Proxmox host kernel; updating packages only inside the LXC does not patch that kernel. Docker inside a VM uses the VM\u0026rsquo;s guest kernel.\nRecord which advisory confirms the fix for your vendor package. An installed update does not protect the running kernel until that kernel is activated, usually by rebooting.\n2. Review Container Restrictions 1 2 docker ps --format \u0026#39;table {{.Names}}\\t{{.Image}}\u0026#39; docker inspect YOUR_CONTAINER --format \u0026#39;Privileged={{.HostConfig.Privileged}} SecurityOpt={{json .HostConfig.SecurityOpt}} CapAdd={{json .HostConfig.CapAdd}}\u0026#39; Replace YOUR_CONTAINER with a real container name. Investigate privileged containers, seccomp=unconfined, and unnecessary added capabilities. An empty per-container SecurityOpt value does not mean seccomp is disabled; check the daemon\u0026rsquo;s security options too.\nDocker\u0026rsquo;s default seccomp profile blocks many potentially dangerous operations. Its behavior depends on the Engine/profile version. Do not assume a universal escape path, and do not replace the default profile with a tiny custom allowlist copied from a blog: an incomplete profile may stop applications from working or remove protections you intended to keep.\n3. Apply the Vendor Kernel Fix On Debian or Ubuntu, refresh package information and inspect available upgrades:\n1 2 sudo apt update apt list --upgradable Install the kernel update specified by your vendor through your normal maintenance procedure. On Proxmox, follow Proxmox\u0026rsquo;s supported update process for its kernel packages. Schedule the reboot, account for VM/container downtime or migration, and retain a known-good boot entry.\nAfter booting the patched kernel:\n1 2 3 uname -r systemctl --failed docker ps Confirm the running package revision against the advisory, then test application health. Upgrading the Docker image or Docker Engine alone does not patch the host kernel.\n4. Use Temporary Mitigations Only When Justified If patching must wait, consult your vendor\u0026rsquo;s specific mitigation guidance. Restrict untrusted local code and container workloads, retain seccomp confinement, and remove unnecessary privileges.\nModule listings are diagnostic information, not proof of safety:\n1 2 lsmod | grep -E \u0026#39;(^algif_aead|^af_alg)\u0026#39; modinfo algif_aead No lsmod entry can mean the module is currently unloaded; it may still autoload. Kernel functionality can also be built in. A blacklist file alone does not unload already active code, and disabling crypto functionality can affect legitimate applications. Avoid presenting either check as a complete vulnerability verdict.\nVerification Checklist Identify the kernel that actually isolates the containers. Match its package revision to the vendor\u0026rsquo;s CVE status. Boot the fixed kernel and confirm the running version. Review privileged and unconfined containers. Verify services after maintenance. Record the advisory and package revision used to establish remediation. This is an operational verification checklist, not an exploit test. No exploit needs to be run to establish that your vendor\u0026rsquo;s fixed kernel package is active.\nReferences CVE record: CVE-2026-31431 Linux CNA record with affected and fixed branches Docker: seccomp security profiles Docker: Engine security ","permalink":"https://blog.gntech.me/posts/2026-05-30-docker-cve-2026-31431-copy-fail-mitigation/","summary":"Verify host-kernel patch status for CVE-2026-31431, review Docker runtime restrictions, and validate remediation without assuming every container is exploitable.","title":"Linux Kernel CVE-2026-31431 — Verification and Remediation for Docker Hosts"},{"content":"If your homelab has five containers running on a single host, you can get away with occasional glances at docker stats. But when you\u0026rsquo;re running twenty-plus services across multiple hosts — databases, reverse proxies, media servers, automation pipelines — you need observability that keeps up.\nYou probably already have Prometheus scraping metrics every 15 seconds and Grafana dashboards for historical analysis. That covers the what happened question. But it misses the what is happening right now question — the spikes, the blips, the containers that briefly peg the CPU and settle down before Prometheus comes back for its next scrape.\nThat\u0026rsquo;s where Netdata comes in.\nNetdata is a real-time monitoring agent that collects thousands of metrics per-second with zero configuration. Spin it up in a container, point your browser at port 19999, and you get per-second dashboards for CPU, memory, disk, network, processes, and every running container. No setup, no queries to write, no dashboard building.\nThis guide covers deploying Netdata with Docker Compose, understanding the dashboard, configuring health alarms and notifications, setting up parent-child streaming for multi-host environments, and integrating with Prometheus and Grafana for a complete observability stack.\nDeploying Netdata with Docker Compose Netdata needs access to host-level system files to collect metrics. The cleanest way to provide this in Docker is to bind-mount /proc, /sys, and /var/run/docker.sock into the container and use host networking mode.\nBasic Docker Compose Configuration Create a docker-compose.yml for Netdata:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 services: netdata: image: netdata/netdata:latest container_name: netdata hostname: srv1 network_mode: host pid: host cap_add: - SYS_PTRACE - SYS_ADMIN security_opt: - apparmor:unconfined environment: - NETDATA_CLAIM_TOKEN= - NETDATA_CLAIM_URL= - NETDATA_CLAIM_ROOMS= volumes: - /etc/passwd:/host/etc/passwd:ro - /etc/group:/host/etc/group:ro - /proc:/host/proc:ro - /sys:/host/sys:ro - /var/run/docker.sock:/var/run/docker.sock:ro - netdata_config:/etc/netdata - netdata_lib:/var/lib/netdata - netdata_cache:/var/cache/netdata restart: unless-stopped volumes: netdata_config: netdata_lib: netdata_cache: Key points about this configuration:\nnetwork_mode: host — Gives Netdata direct access to host network interfaces so it can report per-interface bandwidth and per-connection details. Without it, Netdata only sees the container\u0026rsquo;s virtual interface. pid: host — Allows Netdata to see all host processes, including container processes running outside its PID namespace. cap_add: SYS_PTRACE — Required for process-level monitoring. Without it, Netdata cannot inspect the cgroups of other containers. Volume mounts — The /host/* bind mounts let Netdata read system files through a consistent path. The Docker socket gives it container-level metrics via cgroups. Quick Start with Docker Run If you want to test Netdata before committing to compose:\n1 2 3 4 5 6 7 8 9 10 11 12 docker run -d --name=netdata \\ --pid=host \\ --network=host \\ -v /etc/passwd:/host/etc/passwd:ro \\ -v /etc/group:/host/etc/group:ro \\ -v /proc:/host/proc:ro \\ -v /sys:/host/sys:ro \\ -v /var/run/docker.sock:/var/run/docker.sock:ro \\ --cap-add SYS_PTRACE \\ --cap-add SYS_ADMIN \\ --security-opt apparmor=unconfined \\ netdata/netdata Access the dashboard at http://YOUR_HOST_IP:19999.\nUnderstanding the Netdata Dashboard Open the dashboard and you\u0026rsquo;ll see a scrollable page of charts grouped by subsystem. The layout is intentionally flat — no drill-downs, no navigation tree. Every chart is visible on one page.\nSections You Get Automatically System Overview — CPU usage (per-core), load average, uptime, context switches, interrupts, softirqs CPU — Per-core frequency, temperature, c-states, throttling Memory — RAM usage, swap, page faults, memory available, committed memory Disk — Per-disk and per-partition I/O (read/write ops, bandwidth, latency, backlog, utilization) Network — Per-interface bandwidth, packets, errors, drops, retransmits, TCP states Processes — Running, blocked, zombie, forks, threads Containers — Per-container CPU, memory, disk I/O, network traffic (auto-detected from cgroups) The container section is where Netdata shines for Docker users. Every running container appears automatically with per-second CPU and memory charts. Click any container to see its dedicated view with network and disk metrics scoped to that container.\nPer-Second Resolution Every chart updates every second. When Prometheus scrapes every 15 seconds, it captures 4 data points per minute. Netdata captures 60. This granularity catches short-lived spikes — a cron job that pegs CPU for three seconds, a database checkpoint that bursts I/O, a container that OOM-kills and restarts between Prometheus scrape intervals.\nChart Interactions Hover — See the exact value at any point Click and drag — Zoom into a time range Double-click — Reset zoom Pause — Freeze the live view to inspect a specific moment Volume (heatmap) mode — Toggle charts to show distribution instead of line plots Configuring Health Alarms and Notifications Netdata ships with 200+ pre-configured health alarms covering CPU, memory, disk, network, and container metrics. They work immediately and send alerts to the dashboard\u0026rsquo;s \u0026ldquo;Alarms\u0026rdquo; tab.\nBuilt-In Alarm Examples Alarm Warning Threshold Critical Threshold CPU usage 80% for 2 minutes 90% for 1 minute RAM usage 85% 95% Disk space 80% 95% Disk I/O time 90% for 2 minutes 95% for 1 minute Network interface dropped packets 0.1% of total 0.5% of total Outbound OOM kills — 1 event These values are defined in /etc/netdata/health.d/ and are customizable. To override an alarm, create a file in /var/lib/netdata/health.d/ (mounted as netdata_lib):\n1 2 3 4 5 6 7 8 9 10 11 12 # Create custom directory on the host mkdir -p /opt/netdata/health.d # Override CPU alarm thresholds cat \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; \u0026gt; /opt/netdata/health.d/cpu.conf alarm: cpu_user on: system.cpu lookup: average -5m of user units: % every: 10s warn: $this \u0026gt; 70 crit: $this \u0026gt; 85 Setting Up Notifications Netdata supports multiple notification channels. The most practical for homelabs are Discord, Telegram, and email.\nTelegram notifications:\nAdd these environment variables to your Compose service:\n1 2 3 4 5 environment: - NETDATA_NOTIFICATION_TELEGRAM_BOT_TOKEN=your_bot_token - NETDATA_NOTIFICATION_TELEGRAM_CHAT_ID=your_chat_id - NETDATA_NOTIFICATION_TELEGRAM_SEND_BUTTON=YES - NETDATA_NOTIFICATION_TELEGRAM_TRIGGER_ROLE=silent Discord notifications:\n1 2 3 environment: - NETDATA_NOTIFICATION_DISCORD_WEBHOOK_URL=https://discord.com/api/webhooks/your_webhook - NETDATA_NOTIFICATION_DISCORD_SEND_BUTTON=YES The SEND_BUTTON=YES option adds a link back to the relevant chart in the notification message, so you can jump directly to the metric that triggered the alarm.\nParent-Child Streaming for Multi-Host Monitoring When you have multiple Proxmox hosts, LXCs, or VMs, running a separate Netdata dashboard per host is impractical. Parent-child streaming lets you designate one host as the \u0026ldquo;parent\u0026rdquo; that receives and displays metrics from all \u0026ldquo;child\u0026rdquo; nodes.\nHow Streaming Works The child node collects metrics locally and streams them to the parent over TCP port 19999 The parent stores the metrics and serves a unified dashboard Each child\u0026rsquo;s section appears grouped under its hostname Communication uses API key authentication Configure the Parent Node Add a streaming configuration file:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 mkdir -p /opt/netdata/stream cat \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; \u0026gt; /opt/netdata/stream/stream.conf [stream] enabled = yes default memory mode = dbengine default history = 3600 [API_KEY] enabled = yes type = api user = stream_user guid = your-random-api-key-here default history = 3600 default memory mode = dbengine health enabled by default = auto allow from = * EOF Mount this into the parent\u0026rsquo;s container:\n1 2 volumes: - /opt/netdata/stream/stream.conf:/etc/netdata/stream.conf:ro Configure a Child Node On each child host, add these environment variables:\n1 2 3 4 environment: - NETDATA_STREAM_DESTINATION=PARENT_HOST_IP:19999 - NETDATA_STREAM_API_KEY=your-random-api-key-here - NETDATA_STREAM_SEND_ALARMS=YES Replace PARENT_HOST_IP with the IP of your parent Netdata instance. All child nodes use the same API key for authentication.\nNetdata as a Prometheus Scrape Target Netdata exposes a Prometheus-compatible metrics endpoint at /api/v1/allmetrics?format=prometheus. This means you can keep your existing Prometheus + Grafana stack and add Netdata as an additional data source for real-time overlay.\nAdd Netdata to Prometheus Scrape Config In your Prometheus prometheus.yml:\n1 2 3 4 5 6 7 8 9 10 scrape_configs: - job_name: \u0026#39;netdata\u0026#39; scrape_interval: 15s metrics_path: \u0026#39;/api/v1/allmetrics\u0026#39; params: format: [\u0026#39;prometheus\u0026#39;] static_configs: - targets: [\u0026#39;10.0.20.30:19999\u0026#39;] labels: host: srv1 Build Grafana panels from the metrics your Netdata instance actually exports. The previously linked dashboard JSON is no longer available; do not assume an old dashboard matches your metric names or labels.\nWhy Use Both? Aspect Prometheus/Grafana Netdata Time resolution 15-60s scrape intervals Per-second Data retention Weeks/months (TSDB) Configurable (default 1h RAM) Configuration Query language, dashboard building Zero-config, auto-detection Alerting PromQL alert rules Pre-built health alarms Best for Historical trends, long-term analysis, custom dashboards Real-time troubleshooting, per-second visibility, hands-off monitoring Resource Usage and Performance Netdata is designed to be lightweight. A typical homelab Netdata instance uses:\nMemory: 150-250 MB RAM for hundreds of metrics with default retention (1 hour of per-second data in memory) CPU: 0.5-2% of a single core on modern x86 hardware Disk I/O: Near zero with memory or alloc DB mode; ~50 MB/day with dbengine mode (journaled persistent storage) The key is the in-memory ring buffer design. Metrics cycle through RAM and are discarded after retention expires, so disk writes are minimal unless you enable the dbengine mode for longer persistence.\nDB Engine Modes Set via the NETDATA_DB_ENGINE environment variable or netdata.conf:\nMode Behavior Use Case ram All in memory. Fastest, no disk writes. Ephemeral monitoring alloc Memory-mapped files. Balanced. Default dbengine Journaled persistent storage. Historical queries, streaming parent nodes Production Deployment Tips 1. Persistent Storage Always mount persistent volumes for config and lib data. Without them, updating the container loses alarm customizations and the alarms log.\n1 2 3 4 volumes: - netdata_config:/etc/netdata - netdata_lib:/var/lib/netdata - netdata_cache:/var/cache/netdata 2. Reverse Proxy with Traefik If you prefer bridge networking or want TLS, expose Netdata behind your reverse proxy. With Traefik:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 services: netdata: image: netdata/netdata:latest container_name: netdata network_mode: host # ... rest of config netdata_proxy: image: nginx:alpine container_name: netdata-proxy volumes: - ./nginx.conf:/etc/nginx/nginx.conf:ro networks: - proxy labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.netdata.rule=Host(`netdata.yourdomain.com`)\u0026#34; - \u0026#34;traefik.http.services.netdata.loadbalancer.server.port=19999\u0026#34; networks: proxy: external: true 3. Update with Watchtower Netdata updates frequently with new collectors and improvements. Pair with Watchtower for automatic updates:\n1 2 3 4 docker run -d --name watchtower \\ -v /var/run/docker.sock:/var/run/docker.sock \\ containrrr/watchtower \\ --cleanup --interval 86400 netdata 4. Access Control By default, the Netdata dashboard has no authentication. For homelab use behind a VPN or Tailscale this is fine, but if you expose it through Cloudflare Tunnel or a public reverse proxy, add HTTP basic auth via your reverse proxy\u0026rsquo;s middleware.\nNetdata Cloud (Optional) Netdata Cloud is a free SaaS layer that aggregates dashboards from multiple agents without setting up parent-child streaming. Agents connect via the NETDATA_CLAIM_TOKEN environment variable. It\u0026rsquo;s useful if you don\u0026rsquo;t want to manage a parent node yourself, but for a homelab the parent-child approach gives you full control and zero data leaving your network.\nVerifying the Installation After deploying, confirm Netdata is working:\n1 2 3 4 5 6 7 8 9 10 11 # Check the container is running docker ps --filter name=netdata # Check the dashboard responds curl -s -o /dev/null -w \u0026#34;%{http_code}\u0026#34; http://localhost:19999/api/v1/info # View container metrics directly curl -s http://localhost:19999/api/v1/allmetrics?format=prometheus | head -20 # Check alarms status curl -s http://localhost:19999/api/v1/alarms | python3 -m json.tool | head -30 Expected output for the health check: 200.\nConclusion Netdata fills a gap that every homelab operator eventually hits: real-time, per-second observability with zero configuration. While Prometheus and Grafana handle your long-term retention and custom dashboards, Netdata gives you the live view — the spikes, the blips, the containers that misbehave for five seconds and settle down before your next Grafana refresh.\nDeploying it takes five minutes. The Docker Compose above gives you a complete monitoring agent that auto-discovers every container on your host, ships with 200+ pre-configured alarms, and can stream to a parent node for centralized dashboards across multiple hosts.\nPair it with Prometheus and Grafana for a comprehensive observability stack that covers both real-time and historical perspectives. Your future self — chasing a container that\u0026rsquo;s pegging CPU at 3 AM — will thank you.\n","permalink":"https://blog.gntech.me/posts/2026-05-30-netdata-docker-monitoring-homelab/","summary":"Complete guide to deploying Netdata with Docker Compose for real-time container and infrastructure monitoring in your homelab. Covers auto-detection, health alarms, notifications, parent-child streaming, and Prometheus integration.","title":"Netdata Docker Monitoring — Real-Time Container Monitoring Guide"},{"content":"HAProxy is the de facto standard for high-performance TCP and HTTP load balancing. It handles millions of requests per second in production at companies like GitHub, Reddit, and Twitter. In a homelab, it brings the same reliability to your internal services.\nWhen you have multiple Docker containers offering the same service — API replicas, redundant database read replicas, or parallel media transcoders — HAProxy distributes traffic across them, detects failures, and routes around dead backends automatically. It also terminates TLS, provides a real-time stats dashboard, and supports advanced routing patterns like session stickiness and path-based ACLs.\nThis guide walks through deploying HAProxy with Docker Compose in your homelab. Validate the examples against the installed HAProxy, Docker and Compose versions; source parsing does not establish a deployment test.\nPrerequisites for HAProxy Docker Load Balancer Before starting, verify your host has these basics in place:\nDocker Engine 24+ and Docker Compose v2 installed Ports 80, 443, and 8404 available on the host At least two backend containers to demonstrate load balancing (we\u0026rsquo;ll use whoami for HTTP and two PostgreSQL containers for TCP) A domain or subdomain that resolves to your Proxmox host IP (for SSL termination with real certificates) If Docker isn\u0026rsquo;t installed yet, run:\n1 2 curl -fsSL https://get.docker.com | bash sudo apt install docker-compose-plugin Docker Compose HAProxy Deployment Create a project directory and a compose file:\n1 mkdir -p /opt/haproxy/config \u0026amp;\u0026amp; cd /opt/haproxy 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 # compose.yaml services: haproxy: image: haproxy:3.0-alpine container_name: haproxy restart: unless-stopped ports: - \u0026#34;80:80\u0026#34; - \u0026#34;443:443\u0026#34; - \u0026#34;8404:8404\u0026#34; volumes: - ./config/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro - ./certs:/etc/ssl/certs:ro - ./config/errors:/etc/haproxy/errors:ro cap_add: - NET_BIND_SERVICE networks: - proxy logging: driver: \u0026#34;json-file\u0026#34; options: max-size: \u0026#34;10m\u0026#34; max-file: \u0026#34;3\u0026#34; # Backend examples for load balancing whoami-1: image: traefik/whoami container_name: whoami-1 restart: unless-stopped networks: - proxy whoami-2: image: traefik/whoami container_name: whoami-2 restart: unless-stopped networks: - proxy networks: proxy: driver: bridge The NET_BIND_SERVICE capability lets HAProxy bind to privileged ports inside the container without running as root. The config, certificates, and error pages are mounted as read-only volumes for security.\nHAProxy Configuration for Docker Load Balancing The configuration file is the heart of your HAProxy deployment. Save this as config/haproxy.cfg:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 global maxconn 4096 ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256 ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384 ssl-default-bind-options no-sslv3 no-tlsv10 no-tlsv11 no-tlsv12 ssl-default-server-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256 ssl-default-server-options no-sslv3 no-tlsv10 no-tlsv11 no-tlsv12 tune.ssl.default-dh-param 2048 log stdout format raw local0 daemon defaults mode http log global option httplog option dontlognull option http-server-close option forwardfor except 127.0.0.0/8 option redispatch retries 3 timeout http-request 10s timeout queue 1m timeout connect 5s timeout client 30s timeout server 30s timeout http-keep-alive 10s timeout check 10s maxconn 2048 frontend http-in bind *:80 mode http # Redirect all HTTP to HTTPS redirect scheme https code 301 if !{ ssl_fc } frontend https-in bind *:443 ssl crt /etc/ssl/certs/homelab.pem mode http option httplog # Security headers http-response set-header Strict-Transport-Security \u0026#34;max-age=31536000; includeSubDomains; preload\u0026#34; http-response set-header X-Frame-Options \u0026#34;SAMEORIGIN\u0026#34; http-response set-header X-Content-Type-Options \u0026#34;nosniff\u0026#34; http-response set-header X-XSS-Protection \u0026#34;1; mode=block\u0026#34; # ACLs for path-based routing acl whoami-api path_beg /whoami acl app-dashboard path_beg /app # Route to backends use_backend whoami-backend if whoami-api use_backend app-backend if app-dashboard # Default backend default_backend whoami-backend # HTTP backend with round-robin load balancing backend whoami-backend mode http balance roundrobin option httpchk GET / HTTP/1.1\\r\\nHost:\\ whoami http-check expect status 200 server whoami-1 whoami-1:80 check inter 5s fall 3 rise 2 server whoami-2 whoami-2:80 check inter 5s fall 3 rise 2 backend app-backend mode http balance roundrobin option httpchk GET /health HTTP/1.1\\r\\nHost:\\ app http-check expect status 200 server app-1 app:80 check inter 5s fall 3 rise 2 # TCP backend for database read replicas (basic TCP healthcheck) backend pgsql-read-backend mode tcp balance source server pgsql-replica-1 postgres-1:5432 check inter 10s fall 3 rise 2 server pgsql-replica-2 postgres-2:5432 check inter 10s fall 3 rise 2 # Stats dashboard at port 8404 frontend stats bind *:8404 mode http option httplog stats enable stats uri / stats refresh 5s stats show-legends stats auth admin:CHANGEME-STRONG-PASSWORD Key configuration elements:\nSSL termination — Homelab uses a single combined PEM file at /etc/ssl/certs/homelab.pem containing both the certificate chain and private key. Health checks — HAProxy probes backends every 5 seconds. After 3 failed checks a server is marked down; 2 successful checks bring it back up. TCP mode — The pgsql-read-backend uses balance source for session stickiness based on client IP and TCP port checks for basic health verification, essential for database read replicas. SSL Certificate Setup for HAProxy HAProxy expects certificates in a single PEM file. Concatenate your full chain and private key:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # Let\u0026#39;s Encrypt — assuming certbot has issued a certificate cat /etc/letsencrypt/live/homelab.example.com/fullchain.pem \\ /etc/letsencrypt/live/homelab.example.com/privkey.pem \\ \u0026gt; /opt/haproxy/certs/homelab.pem chmod 600 /opt/haproxy/certs/homelab.pem # Auto-renew hook — add to certbot renew hook directory # /etc/letsencrypt/renewal-hooks/post/haproxy-reload.sh #!/bin/bash cat /etc/letsencrypt/live/homelab.example.com/fullchain.pem \\ /etc/letsencrypt/live/homelab.example.com/privkey.pem \\ \u0026gt; /opt/haproxy/certs/homelab.pem chmod 600 /opt/haproxy/certs/homelab.pem docker exec haproxy haproxy -c -f /usr/local/etc/haproxy/haproxy.cfg docker exec haproxy kill -USR2 1 Verifying HAProxy Load Balancing Start the stack and verify everything works:\n1 cd /opt/haproxy \u0026amp;\u0026amp; docker compose up -d Test HTTP requests cycle through both whoami backends:\n1 2 3 # Each request should alternate between container hostnames curl -s http://localhost/whoami | grep Hostname curl -s http://localhost/whoami | grep Hostname Output should show the hostnames alternating:\nHostname: whoami-1 Hostname: whoami-2 Validate the HAProxy configuration file syntax:\n1 docker exec haproxy haproxy -c -f /usr/local/etc/haproxy/haproxy.cfg Access the stats dashboard at http://your-host-ip:8404 with the credentials set in the stats section.\nMonitoring HAProxy with Prometheus For persistent metrics collection, add the HAProxy Prometheus exporter to your compose stack:\n1 2 3 4 5 6 7 8 haproxy-exporter: image: prom/haproxy-exporter:latest container_name: haproxy-exporter restart: unless-stopped command: - \u0026#34;--haproxy.scrape-uri=http://admin:CHANGEME-STRONG-PASSWORD@haproxy:8404/;csv\u0026#34; networks: - proxy Then configure Prometheus to scrape haproxy-exporter:9101 and wire it into a Grafana dashboard using the HAProxy (prometheus) dashboard template from the Grafana marketplace.\nHAProxy Security Hardening Before exposing HAProxy externally, apply these hardening measures:\nRun as non-root user — Use the distroless haproxy:3.0-alpine image which runs as haproxy user by default, but confirm: 1 2 docker exec haproxy whoami # Should return \u0026#34;haproxy\u0026#34;, not \u0026#34;root\u0026#34; Restrict stats access — Change from bind *:8404 to bind 127.0.0.1:8404 and tunnel through ssh or Traefik: frontend stats bind 127.0.0.1:8404 Firewall — Allow only ports 80 and 443 from external networks: 1 2 3 sudo ufw allow 80/tcp sudo ufw allow 443/tcp sudo ufw deny 8404 Rate limiting — Protect backends from abuse with stick-tables: frontend https-in # ... stick-table type ip size 100k expire 30s store http_req_rate(10s) http-request track-sc0 src http-request deny deny_status 429 if { sc_http_req_rate(0) gt 100 } Conclusion HAProxy running in Docker Compose gives your homelab production-grade load balancing with a single container. SSL termination, health checks, backend failover, the real-time stats dashboard, and Prometheus metrics integrate cleanly with any existing monitoring stack.\nThe configuration patterns from this guide — HTTP mode with round-robin balancing, TCP mode with source-based stickiness, and path-based ACLs — cover the overwhelming majority of homelab load balancing needs. Add rate limiting, custom error pages, or multiple frontends as your infrastructure grows.\nNext steps: Wire HAProxy in front of a Docker Swarm cluster, add HTTP/2 support in the bind line with alpn h2,http/1.1, or pair it with Keepalived for active-passive high availability across two VMs.\n","permalink":"https://blog.gntech.me/posts/2026-05-29-haproxy-docker-load-balancer/","summary":"Step-by-step guide to deploying HAProxy with Docker Compose for TCP/HTTP load balancing in your homelab — SSL termination, health checks, stats monitoring, and backend failover with real configuration files.","title":"HAProxy Docker Load Balancer — Homelab Reverse Proxy Guide"},{"content":"BGP (Border Gateway Protocol) is the backbone routing protocol of the internet. Every ISP, every data center, every CDN uses BGP to exchange routes between autonomous systems. Adding BGP to your homelab brings carrier-grade routing capabilities to your lab network — and it\u0026rsquo;s a skill that directly transfers to production environments.\nMikroTik RouterOS 7 overhauled the routing stack compared to v6. The new routing engine introduces filter chains (replacing the old route filters), proper MP-BGP support for IPv6, BGP large communities (RFC 8092), and a cleaner configuration model under /routing/bgp/connection. RouterOS 7.23 and the recently released 7.24beta1 continue to refine this implementation.\nThis guide covers eBGP peering, iBGP with route reflectors, prefix advertisement with filter chains, BGP communities for traffic engineering, and IPv6 MP-BGP. RouterOS routing syntax and behavior vary by release; this source review does not establish runtime evidence across 7.1 through 7.23. You will need two or more MikroTik routers (or CHR instances) with IP connectivity between loopback or physical interfaces.\nUnderstanding BGP vs OSPF in a Homelab OSPF is an Interior Gateway Protocol (IGP) — it runs inside your autonomous system and discovers the complete network topology using link-state advertisements. BGP is an Exterior Gateway Protocol (EGP) designed to exchange reachability information between different autonomous systems, and it uses path-vector logic with no topology database.\nIn a homelab, you would use OSPF when you need fast convergence within a single routed domain (your campus/DC fabric). You would use BGP when:\nSimulating a multi-homed internet connection with multiple upstreams Participating in DN42 (a large BGP-based VPN network for learning) Learning BGP concepts for certifications (JNCIP, CCNP, etc.) Implementing traffic engineering across multiple WAN links Running iBGP alongside an IGP as a control-plane separation exercise RouterOS 7 handles both eBGP (different AS numbers) and iBGP (same AS number between internal routers). The configuration blocks are the same; only the ASN rules differ.\nBasic eBGP Peering Setup on RouterOS 7 RouterOS 7 organises BGP configuration under /routing/bgp/connection. Each connection block represents a single BGP peering session. Create one per peer.\nHere is a minimal eBGP peering between two routers with different ASNs:\nRouter A (AS 64500) — 10.0.0.1/30\n/routing/bgp/connection add name=peer-to-RB \\ remote.address=10.0.0.2/32 \\ remote.as=64501 \\ local.address=10.0.0.1 \\ local.role=ebgp \\ address-families=ipv4 \\ connect=yes \\ router-id=10.0.0.1 Router B — RB (AS 64501) — 10.0.0.2/30\n/routing/bgp/connection add name=peer-to-router-a \\ remote.address=10.0.0.1/32 \\ remote.as=64500 \\ local.address=10.0.0.2 \\ local.role=ebgp \\ address-families=ipv4 \\ connect=yes \\ router-id=10.0.0.2 Key parameters explained:\nremote.address — The peer\u0026rsquo;s IP with a /32 mask. RouterOS 7 expects a /32 prefix here. remote.as — The peer\u0026rsquo;s AS number. eBGP requires different ASNs. local.address — The source IP for the BGP TCP session. local.role — Set to ebgp for external peering. connect=yes — Initiate the session from this router (active mode). router-id — A unique 32-bit identifier per router, typically the highest loopback IP or a manually assigned value. Verify the session comes up:\n/routing/bgp/session/print detail where connection=peer-to-router-a Expected output:\nFlags: E - established, I - idle, A - active, O - open_sent, C - open_confirm 0 E name=\u0026#34;peer-to-router-a\u0026#34; . established=1m23s ... The E flag means the session is established and exchanging routes.\nPrefix Advertisement and Route Injection A BGP session can exchange routes only when they are injected into the BGP routing process. RouterOS 7 uses two methods:\nNetwork statements — Explicitly list prefixes to advertise Redistribution via filter chains — Pull routes from other sources (connected, static, OSPF) into BGP Network Statements /routing/bgp/network add network=192.168.1.0/24 connection=peer-to-router-b add network=10.10.0.0/24 connection=peer-to-router-b Each network entry belongs to a specific connection block. The prefix must exist in the routing table (as connected, static, or from another IGP) before BGP can advertise it.\nFilter Chain for Selective Advertisement RouterOS 7 replaces the old v6 route filters with a chain-based filter system that is more flexible and consistent with other routing protocols.\nTo advertise only your inside networks and discard everything else, create an outbound filter chain on the eBGP connection:\n/routing/filter/rule add chain=bgp-out-ebgp \\ rule=\u0026#34;if (dst 192.168.0.0/16) {accept}\u0026#34; \\ disabled=no add chain=bgp-out-ebgp \\ rule=\u0026#34;if (dst 10.0.0.0/8) {accept}\u0026#34; \\ disabled=no add chain=bgp-out-ebgp \\ rule=\u0026#34;reject\u0026#34; \\ disabled=no Then apply it to the connection:\n/routing/bgp/connection/set [find name=peer-to-router-b] \\ output.filter=bgp-out-ebgp This chain accepts only RFC 1918 addresses and rejects everything else — preventing accidental prefix leakage of internet routes back to your upstream.\nRoute Filtering with Filter Chains Filtering inbound routes is just as important. Without it, your router accepts every prefix the peer sends, which can fill your routing table with unwanted entries.\nInbound Prefix Filter /routing/filter/rule add chain=bgp-in-ebgp \\ rule=\u0026#34;if (dst 10.100.0.0/24) {set bgp-local-pref 200; accept}\u0026#34; \\ disabled=no add chain=bgp-in-ebgp \\ rule=\u0026#34;reject\u0026#34; \\ disabled=no /routing/bgp/connection/set [find name=peer-to-router-b] \\ input.filter=bgp-in-ebgp This accepts only 10.100.0.0/24 from the peer and tags it with a local preference of 200 (higher is more preferred). All other prefixes are rejected.\nAS-Path Filtering Filter routes based on which ASes they traversed:\n/routing/filter/rule add chain=bgp-in-filter \\ rule=\u0026#34;if (bgp-as-path-length \u0026gt; 10) {reject}\u0026#34; \\ disabled=no add chain=bgp-in-filter \\ rule=\u0026#34;if (bgp-as-path IN 64501,64502) {accept}\u0026#34; \\ disabled=no add chain=bgp-in-filter \\ rule=\u0026#34;reject\u0026#34; \\ disabled=no This prevents routes with long AS paths (possible loop or unstable paths) and only accepts routes originating from or traversing AS 64501 or 64502.\niBGP and Route Reflectors iBGP has a hard rule: all iBGP speakers must be fully meshed (every router peers with every other router in the same AS). In a lab with 3+ routers, full mesh becomes tedious. A route reflector (RR) solves this by acting as a central hub.\nRoute Reflector Topology:\nRouter A — Route Reflector (AS 64500) Router B — Client (AS 64500) Router C — Client (AS 64500) Clients peer only with the RR. The RR reflects routes between them.\nRouter A (Route Reflector) — iBGP to B and C:\n/routing/bgp/connection add name=ibgp-to-b \\ remote.address=10.0.1.2/32 \\ remote.as=64500 \\ local.address=10.0.1.1 \\ local.role=ibgp \\ address-families=ipv4 \\ connect=yes \\ router-id=10.0.1.1 \\ rr.client=yes \\ output.filter=bgp-out-ibgp \\ input.filter=bgp-in-ibgp The .rr.client=yes flag on the RR marks the session as a route reflector client session. The RR will reflect routes learned from B to C and vice versa.\nRouter B (Client) — iBGP to RR only:\n/routing/bgp/connection add name=ibgp-to-rr \\ remote.address=10.0.1.1/32 \\ remote.as=64500 \\ local.address=10.0.1.2 \\ local.role=ibgp \\ address-families=ipv4 \\ connect=yes \\ router-id=10.0.1.2 Router C uses an identical configuration on a different local IP. Three connection blocks instead of six (full mesh would need six).\nNext-Hop Resolution in iBGP iBGP does not change the next-hop attribute when advertising routes between iBGP peers. If Router B learns 192.168.1.0/24 via eBGP with next-hop 10.0.0.1, it advertises that same next-hop to the RR and to Router C. If C has no route to 10.0.0.1 (because iBGP does not carry IGP routes), the prefix is unreachable.\nThe fix: either run an IGP (OSPF) inside your AS to carry the underlying interfaces/IPs, or use set-bgp-nexthop in the filter chain to override the next-hop:\n/routing/filter/rule add chain=bgp-out-ibgp \\ rule=\u0026#34;set-bgp-nexthop 10.0.1.1; accept\u0026#34; \\ disabled=no This forces the RR to rewrite the next-hop to its own iBGP peering address, which is reachable by all iBGP clients.\nBGP Communities for Traffic Engineering BGP communities are tags carried with a route that influence routing decisions on the receiving router. RouterOS 7 supports standard (32-bit), extended (64-bit), and large (RFC 8092) communities.\nSetting a Community on Advertised Routes /routing/filter/rule add chain=bgp-out-ebgp \\ rule=\u0026#34;if (dst 192.168.10.0/24) {set bgp-community 64500:100; accept}\u0026#34; \\ disabled=no This tags 192.168.10.0/24 with community 64500:100 before sending it to the eBGP peer. The peer can match on this community to apply local preference changes.\nMatching Communities on Inbound Routes /routing/filter/rule add chain=bgp-in-ebgp \\ rule=\u0026#34;if (bgp-community 64501:200) {set bgp-local-pref 150; accept}\u0026#34; \\ disabled=no Routes from AS 64501 tagged with community 64501:200 get a local preference of 150, making them more preferred than the default (100).\nLarge Communities (RFC 8092) For more granular tagging, use large communities with three 32-bit values instead of two:\n/routing/filter/rule add chain=bgp-out-ebgp \\ rule=\u0026#34;set bgp-large-community 64500:1:10; accept\u0026#34; \\ disabled=no Large communities are the modern standard and supported by all major network vendors in recent code.\nIPv6 with MP-BGP RouterOS 7 supports MP-BGP for IPv6 (also known as BGP multiprotocol extensions). Enable it with address-families=ipv6 or both address families:\n/routing/bgp/connection add name=ebgp-ipv6 \\ remote.address=2001:db8:1::2/128 \\ remote.as=64501 \\ local.address=2001:db8:1::1 \\ local.role=ebgp \\ address-families=ipv6 \\ connect=yes \\ router-id=10.0.0.1 Filter chains for IPv6 work identically — the dst field matches IPv6 prefixes:\n/routing/filter/rule add chain=bgp-out-ebgp6 \\ rule=\u0026#34;if (dst 2001:db8:100::/48) {accept}\u0026#34; \\ disabled=no add chain=bgp-out-ebgp6 \\ rule=\u0026#34;reject\u0026#34; \\ disabled=no For dual-stack, set address-families=ipv4,ipv6 in a single connection block instead of creating two separate sessions.\nVerification and Troubleshooting Check BGP Session State /routing/bgp/session/print detail where connection=peer-name Look for the E (established) flag. Common non-established states:\nI (Idle) — No TCP connection. Check IP reachability (ping), firewall rules blocking port 179 (TCP). A (Active) — TCP established, waiting for OPEN message. Usually an ASN mismatch or hold timer mismatch. O (OpenSent/OpenConfirm) — BGP OPEN exchanged, waiting for keepalive. Certificate/TTL issues. Inspect the Routing Table for BGP Routes /routing/route/print where bgp Or filter by a specific prefix:\n/routing/route/print where dst-address=192.168.1.0/24 Common Issues Next-hop unreachable — The router has a BGP route but the next-hop IP is not in the routing table. Either add a static route to the next-hop subnet or enable an IGP to carry those reachability prefixes.\nTTL mismatch (eBGP multihop) — eBGP uses TTL=1 by default. If peers are not directly connected, add multihop=yes and set multihop-ttl:\n/routing/bgp/connection/set multihop=yes multihop-ttl=2 [find name=peer-name] ASN loop prevention — BGP will not accept a route that already contains its own ASN in the AS_PATH. This is intentional loop prevention. If testing in a lab with the same ASN appearing in the path, disable loop detection for learning purposes:\n/routing/bgp/connection/set as-override=yes [find name=peer-name] Use as-override only in isolated lab environments.\nEnd-to-End Route Verification 1 2 3 # From a Linux host behind Router B ping -c 3 192.168.1.1 traceroute 192.168.1.1 BGP routes should appear in the traceroute as the ASes are crossed (if ICMP extensions are enabled on the routers).\nConclusion BGP in MikroTik RouterOS 7 turns your homelab from a simple static-routed network into a realistic multi-AS topology. The filter chain system gives you granular control over route advertisement and acceptance, while communities and route reflectors introduce concepts that scale to production ISP and data center networks.\nAfter mastering these fundamentals, take it further:\nJoin DN42 — a large BGP-based VPN network designed for learning Deploy route reflectors with multiple RRs for redundancy Use BGP communities with your upstream if you have a public ASN and prefix Combine with OSPF for the IGP and redistribute selectively into BGP for a complete control-plane design The MikroTik RouterOS 7 BGP implementation is production-grade, free to lab with CHR instances, and documented thoroughly in the official MikroTik BGP manual.\n","permalink":"https://blog.gntech.me/posts/2026-05-29-mikrotik-routeros7-bgp-configuration/","summary":"Configure BGP on MikroTik RouterOS 7 for homelab dynamic routing. Covers eBGP/iBGP peering, prefix advertisement, route filtering with filter chains, BGP communities, and multi-router topology with real RouterOS CLI commands.","title":"MikroTik RouterOS 7 BGP — Homelab Dynamic Routing with BGP"},{"content":"If you manage more than one Proxmox VE cluster, you know the pain of jumping between separate web interfaces, checking cluster health in five different browser tabs, and manually syncing configuration changes across sites. Proxmox Datacenter Manager (PDM) removes all of that friction.\nPDM is the centralized management plane for the Proxmox ecosystem. Think of it as vCenter for Proxmox — a single pane of glass that aggregates multiple Proxmox VE clusters and Proxmox Backup Server instances into one unified dashboard. Released in version 1.0 earlier this year, PDM hit 1.1 in late May 2026 with major additions: automated installation workflows, centralized Ceph monitoring, subscription key management, a world map visualization, and a refreshed platform built on Debian 13.5 \u0026ldquo;Trixie\u0026rdquo;, Linux Kernel 7.0, and OpenZFS 2.4.\nThis guide covers everything you need to deploy PDM 1.1 in your homelab, add remote clusters, use the new features, and upgrade from PDM 1.0.\nWhat Is Proxmox Datacenter Manager? Proxmox Datacenter Manager provides a unified web interface at https://\u0026lt;pdm-ip\u0026gt;:8006 that connects to multiple independent PVE clusters. You get:\nAggregate resource overview across all linked clusters Centralized VM/CT migration between clusters Unified update management and subscription oversight A remote CLI via proxmox-datacenter-manager-client Role-based access control spanning all remotes For homelab environments, PDM really shines once you cross the threshold from a single-node setup to a multi-node or multi-site layout. Whether you have two nodes in a living-room rack and one at a colo, or you run a three-node cluster at home and a separate PBS for backups at a remote site, PDM ties everything together without adding complexity.\nWhat\u0026rsquo;s New in Proxmox Datacenter Manager 1.1 Version 1.1 is a significant feature drop. Here is what changed.\n1. Automated Installation Workflows The headline feature in PDM 1.1 is the Automated Installations tab under the Remotes section. You can now define unattended installation profiles for Proxmox VE nodes and Proxmox Backup Server systems directly from the PDM web UI. The system tracks installation progress and reports results back into the dashboard.\nThis is huge for provisioning new nodes at scale. Define an answer file once, boot the target hardware from the Proxmox ISO with the automated install flag, and PDM handles the rest.\n2. Centralized Subscription Key Management Managing enterprise subscription keys across a dozen nodes was a chore. PDM 1.1 adds a centralized subscription management page where you can upload, assign, and track subscription keys across all managed remotes. Compliance validation runs automatically, so you always know which nodes are covered and which are about to expire.\n3. Centralized Ceph Monitoring If you run Proxmox with hyper-converged Ceph storage, PDM 1.1 gives you a single Ceph monitoring panel covering all linked clusters. Health status, OSD utilization, PG states, and performance metrics are aggregated in one view. No more SSH-ing into each monitor node to check cluster health.\n4. World Map Visualization New in 1.1, the world map provides a geographic overview of all your remotes. Assign coordinates to each site and PDM renders them on an interactive map. Click a pin to jump into that cluster. It is a quality- of-life feature that makes multi-site management feel less abstract.\n5. Local Metric Collection Improvements PDM 1.1 improves the metric collection pipeline with finer-grained host-level data. CPU, memory, disk IO, and network metrics are now collected locally on each remote node and forwarded to PDM at configurable intervals. This feeds better capacity planning charts and trend analysis.\n6. Updated Platform Components PDM 1.1 moves to Debian 13.5 \u0026ldquo;Trixie\u0026rdquo; as its base, with Linux Kernel 7.0 and OpenZFS 2.4. The newer kernel brings improved hardware support and performance improvements, while ZFS 2.4 adds faster snapshot operations and compatibility with newer SSD and NVMe hardware.\nDeploying Proxmox Datacenter Manager 1.1 PDM runs as a standalone service on a dedicated Debian VM or physical host. It does not require a Proxmox VE license for basic operation, though some advanced features require an active subscription.\nMinimum Requirements 2 vCPUs (4 recommended for more than 5 remotes) 4 GB RAM (8 GB for heavy monitoring loads) 32 GB disk (SSD preferred) Debian 13 \u0026ldquo;Trixie\u0026rdquo; or the official PDM ISO Network reachability to all managed PVE clusters (port 8006) Installation from Proxmox Repository Set up the repository and install the package:\n1 2 3 4 5 6 7 8 9 10 11 # Add the Proxmox Datacenter Manager repository echo \u0026#34;deb http://download.proxmox.com/debian/pdm trixie pdm-no-subscription\u0026#34; | \\ sudo tee /etc/apt/sources.list.d/pdm.list # Import the Proxmox GPG key wget -qO- https://enterprise.proxmox.com/debian/proxmox-release-bookworm.gpg | \\ sudo gpg --dearmor -o /etc/apt/trusted.gpg.d/proxmox.gpg # Install sudo apt update sudo apt install proxmox-datacenter-manager The installer configures the web interface, sets up the local database, and starts the PDM service automatically.\nFirst Login Open a browser to https://\u0026lt;pdm-ip\u0026gt;:8006. The first login prompts you to create the admin account and upload a TLS certificate. For a homelab, a self-signed certificate is fine — just accept the browser warning.\nAfter the initial setup, you land on the Dashboard. It is empty until you add remote clusters.\nAdding and Managing Remote Clusters Via the Web UI Navigate to Datacenter → Remotes → Add. Enter the cluster name, the Proxmox VE node IP or hostname, and authentication credentials. You can use either root@pam with a password or an API token (preferred).\n1 2 3 4 # Create an API token on the PVE node for PDM pveum user add pdm-mon@pve pveum acl modify / --user pdm-mon@pve --role PVEAuditor pveum token add pdm-mon@pve --tokenid pdm-token --privsep 0 Via the Command Line 1 2 3 4 proxmox-datacenter-manager-client remote add homelab \\ --host 10.0.20.30 \\ --user root@pam \\ --password PDM polls each remote every 30 seconds by default. The status panel shows connection health, resource usage, and pending updates for every linked cluster in real time.\nUsing Automated Installation Workflows The Automated Installations feature requires a Proxmox VE or PBS ISO configured with an answer file. From the Remotes → Automated Installations tab in PDM:\nUpload or paste your answer file (YAML format) Assign a profile name and description Boot the target hardware from the ISO with the auto-install kernel parameter pointing at the PDM server URL PDM tracks each installation in the dashboard. You see provisioning progress, any errors, and the final deployment status. This is invaluable when rolling out multiple nodes to a freshly built rack.\nMonitoring Ceph Across Clusters If any of your remotes run Ceph, the Ceph Monitoring panel under the storage section aggregates everything:\nOverall cluster health (HEALTH_OK / HEALTH_WARN / HEALTH_ERR) OSD count, status, and utilization per cluster Pool statistics and PG distribution Performance dashboards (IOPS, throughput, latency) All data is read from the remote Ceph monitors via the PVE API — no extra agents or daemons needed on the Ceph nodes.\nUpgrading from PDM 1.0 to 1.1 Upgrading is a straightforward in-place operation. Run these steps on the PDM host:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # Backup the PDM configuration sudo cp -r /etc/proxmox-datacenter-manager \\ /etc/proxmox-datacenter-manager.backup # Update repository info and upgrade sudo apt update sudo apt full-upgrade # Reboot to load the new kernel sudo reboot # Verify the installed version proxmox-datacenter-manager-client version If you use the enterprise repository, make sure your subscription key is valid before upgrading:\n1 2 # Enterprise repository # deb https://enterprise.proxmox.com/debian/pdm trixie pdm-enterprise Pre-Upgrade Checklist Back up PDM configuration files Verify all managed PVE clusters are healthy Confirm sufficient disk space on the PDM host (2 GB free minimum) Check that no remote upgrades are in progress Review the official PDM 1.1 release notes Proxmox Datacenter Manager Configuration Tips A few things I have learned running PDM in production:\nUse API tokens, not passwords. Passwordless tokens with scoped permissions (PVEAuditor or PVEAdmin) are more secure and survive password rotations. Run PDM in its own VM. It does not need to be on a PVE cluster node, and isolating it keeps the management plane independent of your workload infrastructure. Configure alerting early. PDM can forward cluster alerts via email or webhook. Set this up at deploy time rather than after something goes wrong. Lock the remote poll interval. The default 30-second poll is fine for most homelabs. If you manage many clusters and see CPU load, bump it to 60 seconds under Datacenter → Options. Conclusion Proxmox Datacenter Manager 1.1 transforms multi-cluster management from a series of SSH sessions and browser tabs into a unified, well-organized dashboard. Automated installations, centralized Ceph monitoring, and subscription management make it genuinely useful for anyone running more than a single Proxmox node — whether in a homelab or at enterprise scale.\nThe best part? PDM is free to use with up to three managed nodes without a subscription. That covers a huge range of homelab setups. Give it a try on a lightweight Debian VM and see how much smoother your multi-cluster workflow becomes.\nFor more details, check the Proxmox Datacenter Manager documentation.\n","permalink":"https://blog.gntech.me/posts/2026-05-29-proxmox-datacenter-manager-1-1/","summary":"Step-by-step guide to deploying Proxmox Datacenter Manager 1.1 for centralized multi-cluster management. Covers automated installation workflows, Ceph monitoring, subscription key management, world map visualization, and upgrade procedures.","title":"Proxmox Datacenter Manager 1.1 — Centralized Multi-Cluster Management Guide"},{"content":"Your homelab\u0026rsquo;s docker-compose.yml starts small. Traefik, PostgreSQL, maybe a Redis cache. Then you add Grafana for dashboards. Prometheus and Node Exporter for metrics. Loki collects container logs. Adminer makes database queries convenient. Portainer gives you a web UI for containers. Watchtower auto-updates everything.\nSuddenly you have twenty services in one file, and docker compose up starts every single one. Your 8 GB Proxmox LXC hits swap before you can disable the monitoring stack you only need for debugging. The dev tools you forgot to stop eat RAM all night.\nDocker Compose profiles fix this. They let you tag services into groups and start only the groups you need. Same compose file, zero duplication, full control over what runs when.\nHow Docker Compose Profiles Work The profiles attribute on a service defines which profile names activate it. A service with no profiles always starts with a plain docker compose up. A service with at least one profile only starts when that profile is explicitly requested.\n1 2 3 4 5 6 7 8 9 10 11 12 services: app: image: nginx:alpine profiles: [\u0026#34;frontend\u0026#34;] ports: - \u0026#34;8080:80\u0026#34; db: image: postgres:16-alpine # no profiles — always starts with `docker compose up` environment: POSTGRES_PASSWORD: changeme Run docker compose up -d and only db starts. Run docker compose --profile frontend up -d and both services start.\nThis small mechanism unlocks a clean organization pattern for homelabs running a dozen or more containers on a single Docker host.\nHomelab Infrastructure Stack with Profiles The most practical setup defines service tiers as profiles. Core infrastructure runs automatically. Monitoring, dev tools, and batch jobs start on demand.\nHere is the complete structure I run on a Proxmox LXC at home:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 # /opt/homelab/docker-compose.yml services: # === Always-on core services === traefik: image: traefik:v3.3 container_name: homelab-traefik restart: unless-stopped volumes: - /var/run/docker.sock:/var/run/docker.sock:ro - ./traefik:/etc/traefik:ro ports: - \u0026#34;80:80\u0026#34; - \u0026#34;443:443\u0026#34; postgres: image: postgres:16-alpine container_name: homelab-postgres restart: unless-stopped volumes: - pgdata:/var/lib/postgresql/data environment: POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?err} healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;pg_isready\u0026#34;] interval: 10s timeout: 5s retries: 5 redis: image: redis:7-alpine container_name: homelab-redis restart: unless-stopped volumes: - redis-data:/data # === Infrastructure profile — dashboards and observability === prometheus: image: prom/prometheus container_name: homelab-prometheus profiles: [\u0026#34;infra\u0026#34;, \u0026#34;monitoring\u0026#34;] restart: unless-stopped volumes: - ./prometheus:/etc/prometheus:ro - prometheus-data:/prometheus grafana: image: grafana/grafana container_name: homelab-grafana profiles: [\u0026#34;infra\u0026#34;] restart: unless-stopped volumes: - grafana-data:/var/lib/grafana - ./grafana/dashboards:/etc/grafana/provisioning/dashboards:ro loki: image: grafana/loki:3.4 container_name: homelab-loki profiles: [\u0026#34;infra\u0026#34;] restart: unless-stopped volumes: - ./loki:/etc/loki:ro - loki-data:/loki # === Monitoring agent profile — per-host exporters === node-exporter: image: prom/node-exporter container_name: homelab-node-exporter profiles: [\u0026#34;monitoring\u0026#34;] restart: unless-stopped volumes: - /proc:/host/proc:ro - /sys:/host/sys:ro - /:/rootfs:ro command: - \u0026#34;--path.procfs=/host/proc\u0026#34; - \u0026#34;--path.rootfs=/rootfs\u0026#34; - \u0026#34;--path.sysfs=/host/sys\u0026#34; cadvisor: image: gcr.io/cadvisor/cadvisor:latest container_name: homelab-cadvisor profiles: [\u0026#34;monitoring\u0026#34;] restart: unless-stopped volumes: - /:/rootfs:ro - /var/run:/var/run:ro - /sys:/sys:ro - /var/lib/docker/:/var/lib/docker:ro # === Dev tools profile — admin interfaces === adminer: image: adminer container_name: homelab-adminer profiles: [\u0026#34;dev\u0026#34;] restart: unless-stopped ports: - \u0026#34;127.0.0.1:8081:8080\u0026#34; portainer: image: portainer/portainer-ce:2-lts container_name: homelab-portainer profiles: [\u0026#34;dev\u0026#34;] restart: unless-stopped volumes: - /var/run/docker.sock:/var/run/docker.sock:ro - portainer-data:/data ports: - \u0026#34;127.0.0.1:9000:9000\u0026#34; # === Batch profile — periodic housekeeping === watchtower: image: containrrr/watchtower container_name: homelab-watchtower profiles: [\u0026#34;batch\u0026#34;] restart: unless-stopped volumes: - /var/run/docker.sock:/var/run/docker.sock:ro environment: WATCHTOWER_CLEANUP: \u0026#34;true\u0026#34; WATCHTOWER_SCHEDULE: \u0026#34;0 0 4 * * *\u0026#34; WATCHTOWER_INCLUDE_STOPPED: \u0026#34;false\u0026#34; volumes: pgdata: redis-data: prometheus-data: grafana-data: loki-data: portainer-data: What This Setup Achieves Profile Services Start Command When to Use (none) Traefik, PostgreSQL, Redis docker compose up -d Always on — boot with the host infra Prometheus, Grafana, Loki --profile infra Daily monitoring monitoring Node Exporter, cAdvisor --profile monitoring Debug or capacity planning dev Adminer, Portainer --profile dev Active development sessions batch Watchtower --profile batch Nightly updates Startup Commands for Day-to-Day Usage Start the core stack at boot:\n1 docker compose up -d Add the infrastructure dashboard stack:\n1 docker compose --profile infra up -d Fire up everything for a maintenance session:\n1 docker compose --profile infra --profile monitoring --profile dev up -d The --profile flag accepts multiple values by repeating it. You can combine any set of profiles depending on what you need right now.\nStart every profiled and unprofiled service at once:\n1 docker compose --profile \u0026#39;*\u0026#39; up -d Stop all monitoring services without touching the core stack:\n1 docker compose --profile monitoring down Dev vs Production Profiles Profiles pair naturally with environment-specific overrides. A typical dev setup exposes database ports and includes query browsers that should never appear in production.\n1 2 3 4 5 6 7 8 9 10 11 # docker-compose.override.yml — loaded automatically by Compose services: postgres: profiles: [\u0026#34;dev\u0026#34;] ports: - \u0026#34;127.0.0.1:5432:5432\u0026#34; redis: profiles: [\u0026#34;dev\u0026#34;] ports: - \u0026#34;127.0.0.1:6379:6379\u0026#34; Now docker compose --profile dev up -d starts the core stack and exposes database ports for external tools like DBeaver or psql. Without the --profile dev flag, the databases run but remain inaccessible from the Docker host — exactly what you want in production.\nThis pattern also works for service variants. Use different images or tags per profile:\n1 2 3 4 5 6 7 8 9 10 11 12 services: app: image: myapp:${APP_TAG:-latest} profiles: [\u0026#34;prod\u0026#34;] app-dev: image: myapp:develop profiles: [\u0026#34;dev\u0026#34;] ports: - \u0026#34;3000:3000\u0026#34; volumes: - ./src:/app/src:ro Splitting Large Compose Files with Includes When a single compose file grows past 300 lines, split it by profile domain using Docker Compose includes (Compose v2.30+):\n1 2 3 4 5 6 7 8 9 # /opt/homelab/docker-compose.yml services: traefik: # core services here include: - ./infra/monitoring.yml - ./infra/devtools.yml - ./batch/watchtower.yml Each included file defines its own services with profiles:\n1 2 3 4 5 6 7 8 9 # /opt/homelab/infra/monitoring.yml services: prometheus: image: prom/prometheus profiles: [\u0026#34;infra\u0026#34;] grafana: image: grafana/grafana profiles: [\u0026#34;infra\u0026#34;] Profiles work identically across includes — --profile infra starts all services tagged with infra regardless of which file they live in.\nBest Practices for Docker Compose Profiles in Homelab Default services stay unprofiled. If a service must run for the homelab to function (reverse proxy, auth, database), do not assign a profile. It starts automatically.\nName profiles by purpose, not by service. Use monitoring, dev, batch, experimental — not grafana, prometheus, adminer. This keeps the profile count manageable and the intent clear.\nStart everything at boot with --profile '*' in your systemd service or init script. The host reboot starts all containers regardless of profile:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 # /etc/systemd/system/docker-homelab.service [Unit] Description=Docker Compose Homelab Requires=docker.service After=docker.service [Service] Type=oneshot RemainAfterExit=yes WorkingDirectory=/opt/homelab ExecStart=/usr/bin/docker compose --profile \u0026#39;*\u0026#39; up -d ExecStop=/usr/bin/docker compose --profile \u0026#39;*\u0026#39; down User=root [Install] WantedBy=multi-user.target Document the profile assignment in your README or as a comment at the top of each compose file. When you return to a project after six months, you want to know --profile monitoring starts which services.\nUse environment variables for profile-level configuration. Set MONITORING_ENABLED=true in your .env and use it to conditionally include files or configure service parameters.\nTroubleshooting Common Profile Issues Container won\u0026rsquo;t start with docker compose up\nCheck that the service has a profile. Services with a profiles list are excluded from the default up. Start them with --profile \u0026lt;name\u0026gt;.\nService with depends_on fails because its dependency has a profile\nCompose does not automatically activate profiles for dependencies. If service A (profiled) depends on service B (profiled), you must pass both profiles:\n1 docker compose --profile monitoring --profile infra up -d Listing which services belong to each profile\n1 docker compose config --services Services without profiles appear in the default output. Add --profile to see profiled services:\n1 docker compose --profile monitoring config --services Check if a profile is active\n1 docker compose --profile dev ps Conclusion Docker Compose profiles solve a real problem for growing homelabs. Instead of maintaining multiple compose files, environment-specific overrides, or commenting services in and out, you tag each service with a profile name and start only what you need.\nThe monitoring stack starts when you want to debug. Dev tools run only during development sessions. Batch jobs execute on a schedule. Core infrastructure stays up without interference.\nOne compose file, one directory, multiple service groups. Your RAM budget — and your future self — will thank you.\nSee Also Docker Compose Production Patterns Docker Compose Networking Patterns Docker Container Resource Limits ","permalink":"https://blog.gntech.me/posts/2026-05-29-docker-compose-profiles-homelab/","summary":"Learn how Docker Compose profiles let you start only the services you need — enable monitoring, disable dev tools in production, and keep a single docker-compose.yml for your entire homelab stack.","title":"Docker Compose Profiles — Selective Service Management for Homelab"},{"content":"You set up Pi-hole on your homelab network. Ad blocking works on every device — phones, laptops, even the IoT lightbulbs. But your Docker containers? They\u0026rsquo;re hitting Google\u0026rsquo;s 8.8.8.8 directly, bypassing all your carefully configured blocklists and internal hostnames.\nThis is the default Docker DNS behavior, and it breaks two things: external ad-blocking and internal service discovery. Containers can\u0026rsquo;t resolve pihole.internal or router.homelab because they\u0026rsquo;re not talking to your DNS infrastructure. This guide covers every level of Docker DNS configuration — from a quick --dns flag to global daemon.json settings, Compose service discovery, and troubleshooting when things go wrong.\nUnderstanding Docker DNS Resolution Inside Containers Every Docker container gets a /etc/resolv.conf file. By default, Docker copies the host\u0026rsquo;s DNS configuration. If your Proxmox LXC or VM uses 8.8.8.8 via systemd-resolved, that\u0026rsquo;s what your containers use.\nWhen you connect a container to a user-defined bridge network (the kind you create with docker network create or define in Compose), Docker injects its own embedded DNS resolver at 127.0.0.11. This resolver handles container-to-container name resolution — service names in Compose, container names, and network aliases. It does not handle external domain resolution. External queries still go to whatever DNS servers are configured in the container\u0026rsquo;s /etc/resolv.conf.\nContainer /etc/resolv.conf ├── 127.0.0.11 → Docker embedded resolver │ └── Resolves: service-name, container-name, network-alias └── 10.0.20.5 → Custom DNS (Pi-hole) └── Resolves: google.com, pihole.internal, *.homelab The embedded resolver listens on 127.0.0.11 inside every container on a user-defined network. For external queries, it acts as a forwarder to whatever upstream DNS servers are configured.\nConfiguring DNS Per Container and Per Service The simplest approach is setting DNS per container. This works for individual services or testing.\nDocker CLI 1 docker run --dns 10.0.20.5 --dns 10.0.20.6 nginx Docker Compose 1 2 3 4 5 6 7 services: web-app: image: nginx dns: - 10.0.20.5 - 10.0.20.6 dns_search: homelab.internal The dns_search directive adds a search domain. When you ping router, Docker appends .homelab.internal and tries router.homelab.internal automatically.\nSetting Global DNS via daemon.json Per-container DNS is tedious when you have twenty services. Set the global default for all containers by editing /etc/docker/daemon.json:\n1 2 3 4 5 { \u0026#34;dns\u0026#34;: [\u0026#34;10.0.20.5\u0026#34;, \u0026#34;10.0.20.6\u0026#34;], \u0026#34;dns-opts\u0026#34;: [\u0026#34;timeout:2\u0026#34;, \u0026#34;attempts:3\u0026#34;], \u0026#34;dns-search\u0026#34;: [\u0026#34;homelab.internal\u0026#34;] } Apply the change:\n1 sudo systemctl restart docker Every new container now uses your homelab DNS servers by default. The dns-opts settings control query timeout (2 seconds) and retries (3 attempts) before falling back — critical when a DNS server is temporarily down.\nImportant: daemon.json DNS settings apply only to containers that don\u0026rsquo;t specify their own dns: in Compose or --dns on the command line. Per-container overrides always win.\nChecking the Current daemon.json 1 2 3 cat /etc/docker/daemon.json # Check if DNS is set: docker info | grep -i dns If daemon.json doesn\u0026rsquo;t exist, create it. Docker reads this file on startup.\nDocker Compose Service Discovery with Network Aliases When containers need to talk to each other by name inside a Compose stack, the embedded DNS resolver at 127.0.0.11 handles it automatically — as long as all services share the same network.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 services: db: image: postgres:16-alpine networks: - backend volumes: - pgdata:/var/lib/postgresql/data app: image: my-app:latest networks: - backend depends_on: - db dns: - 10.0.20.5 networks: backend: driver: bridge In this example, the app container resolves db to the PostgreSQL container\u0026rsquo;s IP on the backend network. External lookups (like api.example.com) go through 10.0.20.5 (your Pi-hole).\nNetwork Aliases Need to resolve a service by multiple names? Use aliases:\n1 2 3 4 5 6 7 8 9 services: redis: image: redis:7-alpine networks: backend: aliases: - cache - session-store - redis Now any container on the backend network can resolve cache, session-store, or redis — all pointing to the same Redis instance.\nThe Legacy Links Approach Docker links (links:) also provide name resolution but are deprecated. User-defined bridge networks with aliases are the modern replacement:\n1 2 3 4 5 6 7 8 9 10 11 # Legacy — avoid services: app: links: - db:database # Modern services: app: networks: - backend Using Extra Hosts for Static DNS Entries For hostnames that don\u0026rsquo;t change, extra_hosts adds static entries to the container\u0026rsquo;s /etc/hosts:\n1 2 3 4 5 6 services: app: extra_hosts: - \u0026#34;pihole.internal:10.0.20.5\u0026#34; - \u0026#34;router.homelab:10.0.20.1\u0026#34; - \u0026#34;proxmox.srv1:10.0.20.30\u0026#34; This is equivalent to --add-host on docker run and is useful for:\nHostnames not served by your DNS server Overriding DNS for testing Referencing hosts on different subnets Use extra_hosts when you need static resolution and custom DNS via dns: when you want dynamic DNS resolution through your infrastructure.\nTroubleshooting Container DNS Issues DNS problems in Docker are frustrating because the failure looks like a network issue. Here\u0026rsquo;s a systematic approach.\nStep 1: Check Resolv Conf 1 docker exec -it container-name cat /etc/resolv.conf Expected output shows 127.0.0.11 (embedded resolver) and your custom DNS servers. If you see only 8.8.8.8, your daemon.json or per-service DNS setting isn\u0026rsquo;t applied.\nStep 2: Test Resolution 1 2 docker exec -it container-name nslookup google.com docker exec -it container-name dig +short service-name If nslookup google.com fails but ping 8.8.8.8 works, it\u0026rsquo;s a DNS issue, not a network issue.\nStep 3: Check Embedded Resolver 1 docker exec -it container-name nslookup db 127.0.0.11 If service names via the embedded resolver (127.0.0.11) fail but external resolution works, check that all services are on the same user-defined network. The default bridge network (docker0) does not support DNS-based service discovery.\nCommon Issues and Fixes systemd-resolved interference On Ubuntu/Debian hosts, systemd-resolved creates a resolv.conf symlink. Docker may follow the symlink instead of the actual DNS configuration:\n1 2 3 4 5 ls -la /etc/resolv.conf # If symlinked to /run/systemd/resolve/stub-resolv.conf: # Point Docker daemon to the resolved configuration directly: echo \u0026#34;{\\\u0026#34;dns\\\u0026#34;:[\\\u0026#34;10.0.20.5\\\u0026#34;]}\u0026#34; | sudo tee /etc/docker/daemon.json sudo systemctl restart docker DNS breaks after network change When you disconnect and reconnect a network, the embedded resolver\u0026rsquo;s entry in resolv.conf can disappear:\n1 2 3 docker network disconnect backend container-name docker network connect backend container-name # Container may lose 127.0.0.11 entry Fix: restart the container or use docker compose down \u0026amp;\u0026amp; docker compose up.\nContainer can\u0026rsquo;t reach Pi-hole Verify network connectivity between the container and your DNS server:\n1 docker exec -it container-name ping 10.0.20.5 If ping fails, the container is on an isolated network and can\u0026rsquo;t reach your homelab DNS server. Ensure the container\u0026rsquo;s network has a route to the DNS server, or use the host\u0026rsquo;s Docker bridge.\nDebugging with Tcpdump 1 2 3 4 # On the Docker host: sudo tcpdump -i docker0 port 53 # On the Proxmox host: sudo tcpdump -i vmbr0 port 53 and host 10.0.20.5 Watch for DNS queries leaving the Docker bridge toward your Pi-hole. If queries hit the host but not Pi-hole, a firewall rule may be blocking UDP 53 between Docker networks.\nFull Working Example: Web Stack with Homelab DNS Here\u0026rsquo;s a complete docker-compose.yml that ties everything together:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 services: postgres: image: postgres:16-alpine restart: unless-stopped volumes: - pgdata:/var/lib/postgresql/data networks: - internal environment: POSTGRES_DB: appdb POSTGRES_PASSWORD_FILE: /run/secrets/db_password secrets: - db_password redis: image: redis:7-alpine restart: unless-stopped networks: internal: aliases: - cache - session-store app: image: nginx:alpine restart: unless-stopped networks: - internal - proxy depends_on: - postgres - redis dns: - 10.0.20.5 dns_search: homelab.internal extra_hosts: - \u0026#34;pihole.internal:10.0.20.5\u0026#34; - \u0026#34;api.example.com:203.0.113.50\u0026#34; traefik: image: traefik:v3.3 restart: unless-stopped networks: - proxy ports: - \u0026#34;80:80\u0026#34; - \u0026#34;443:443\u0026#34; volumes: - /var/run/docker.sock:/var/run/docker.sock:ro networks: internal: driver: bridge internal: true proxy: driver: bridge secrets: db_password: file: ./secrets/db_password.txt Key points:\npostgres and redis are on the internal network — no external DNS needed, the embedded resolver handles service discovery. redis has aliases (cache, session-store) for flexible references. app uses 10.0.20.5 (Pi-hole) for external DNS, plus extra_hosts for static entries that bypass DNS entirely. traefik is on the proxy network only — no custom DNS needed, default daemon.json settings apply. Setting internal: true on the internal network isolates it from external access but still supports embedded DNS resolution between containers. Best Practices Summary Set global DNS in daemon.json — point all containers to your Pi-hole or Unbound resolver by default. Override per-service when specific containers need different DNS.\nUse user-defined networks for service discovery — the default bridge (docker0) does not support DNS-based name resolution. Always define named networks in Compose.\nDocker embedded resolver for service names, custom DNS for external domains — the 127.0.0.11 resolver handles container-to-container discovery. Configure upstream DNS for everything else.\nUse network aliases instead of links — aliases are more flexible and support multiple names per service without deprecated syntax.\nAdd extra_hosts for critical infrastructure — hostnames like your Pi-hole, router, or Proxmox host benefit from static entries that work even if DNS is temporarily down.\nTest DNS isolation — when debugging, always check whether failure is in the embedded resolver (internal) or the upstream resolver (external). Use nslookup with explicit server IPs to narrow it down.\nDocker DNS configuration is one of those things you set once and forget about — until a container silently bypasses your ad-blocking DNS server. Start with a sensible daemon.json default, use user-defined networks with aliases for service discovery, and keep extra_hosts in your back pocket for static entries that must survive any DNS outage.\n","permalink":"https://blog.gntech.me/posts/2026-05-29-docker-container-dns-homelab-guide/","summary":"Master Docker container DNS for your homelab — configure custom resolvers like Pi-hole, enable internal service discovery with Docker Compose network aliases, set global DNS via daemon.json, and troubleshoot container DNS failures step by step.","title":"Docker Container DNS Guide — Custom DNS and Service Discovery for Homelab"},{"content":"PostgreSQL needs persistent storage and a recovery procedure before applications depend on it. A shared instance can save resources, but also couples maintenance, extensions, and failures across applications. Follow each application\u0026rsquo;s supported database version and extension requirements; a shared server is not always suitable.\nVerification status Area Evidence and limits Source review Compose and SQL examples reviewed against the references below on October 31, 2026. Runtime This deployment example has not been executed in this audit. The separate backup lab has linked CI recovery evidence. Scope A new, isolated PostgreSQL 16 instance. No claim of production certification, HA, pgvector installation, or universal tuning. Start a separate instance Use Bash, OpenSSL, Docker Engine, and a Compose plugin supporting up --wait. These commands create a new private project directory. Keep its path and Compose project name stable for subsequent runs.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 postgres_dir=$(mktemp -d -t postgres-deployment.XXXXXX) cd \u0026#34;$postgres_dir\u0026#34; || exit 1 umask 077 openssl rand -base64 32 \u0026gt; postgres-password cat \u0026gt; compose.yaml \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; services: postgres: image: postgres:16-alpine restart: unless-stopped environment: POSTGRES_USER: labadmin POSTGRES_DB: appdb POSTGRES_PASSWORD_FILE: /run/secrets/postgres-password secrets: - postgres-password volumes: - pgdata:/var/lib/postgresql/data healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;pg_isready\u0026#34;, \u0026#34;-h\u0026#34;, \u0026#34;127.0.0.1\u0026#34;, \u0026#34;-U\u0026#34;, \u0026#34;labadmin\u0026#34;, \u0026#34;-d\u0026#34;, \u0026#34;appdb\u0026#34;] interval: 5s timeout: 3s retries: 12 start_period: 10s secrets: postgres-password: file: ./postgres-password volumes: pgdata: EOF docker compose config --quiet docker compose up --wait --wait-timeout 120 docker compose exec -T postgres psql -X -U labadmin -d appdb -v ON_ERROR_STOP=1 \\ -c \u0026#39;SELECT current_database(), version();\u0026#39; There is no published database port. Host administration uses docker compose exec; applications in another project need an explicitly planned private network. Do not publish port 5432 just to solve a missing network attachment.\nThe official image initializes its bootstrap user as a superuser on an empty data volume. Its local socket authentication permits the administrative command above. Changing the password file later does not rotate an initialized database password. See the official image documentation.\nGive applications their own roles Do not give an application the bootstrap superuser credential. Open an interactive administrative session:\n1 docker compose exec postgres psql -X -U labadmin -d appdb -v ON_ERROR_STOP=1 For one new example application, run the following inside psql. \\password prompts for its password without embedding it in shell history:\n1 2 3 CREATE ROLE example_app LOGIN; \\password example_app CREATE DATABASE example_app OWNER example_app; Database ownership alone does not establish full tenant isolation. Review CONNECT, schema privileges, role memberships, and pg_hba.conf, and test access that should be denied. Use separate instances when extension, version, or trust boundaries require them.\nExtensions and tuning The stock postgres:16-alpine image does not provide pgvector\u0026rsquo;s extension files. An initialization SQL statement cannot install missing extension binaries. Follow the pgvector project\u0026rsquo;s installation instructions or the application\u0026rsquo;s supported database image. The quoted extension identifier \u0026quot;uuid-ossp\u0026quot; is distinct from an unquoted identifier containing a hyphen.\n1 2 docker compose exec -T postgres psql -X -U labadmin -d appdb -v ON_ERROR_STOP=1 \\ -c \u0026#34;SELECT name, default_version, installed_version FROM pg_available_extensions ORDER BY name;\u0026#34; Size memory settings for the database\u0026rsquo;s actual memory limit and concurrent workload. work_mem can be used by multiple operations per query and multiple sessions. Do not copy a host-RAM percentage into every database container. Measure before changing planner costs or WAL settings. Physical streaming replication normally needs wal_level=replica or higher; selecting logical is not a complete replica setup. Replication requires authentication, a base backup, standby configuration, WAL retention monitoring, and a tested failover procedure. It does not replace backups.\nBack up and recover before upgrading Use the atomic dump, systemd scheduling, and isolated database recovery lab. It records actual recovery evidence and includes a separately encrypted Restic copy exercise.\nA pg_dump archive covers one database. Cluster roles and tablespaces require separate planning with pg_dumpall. Restore into a new volume and verify application data before any cutover. Keep the original instance and backup materials intact; deleting the original volume is not a recovery prerequisite.\nFor minor updates, review release notes, back up, test the selected image, and recreate in a maintenance window. Major upgrades require an appropriate dump/restore or pg_upgrade plan and compatible extensions; changing the image\u0026rsquo;s major tag against an old data directory is not that plan. See PostgreSQL upgrades.\nTo stop this example while retaining its data:\n1 docker compose down Before moving the project, record its resolved name, volume identity, image digest, credentials, and recovery instructions. A volume name is a reference to data on a particular Docker daemon; copying that name to another machine does not copy the database.\n","permalink":"https://blog.gntech.me/posts/2026-05-28-postgresql-docker-homelab-deployment/","summary":"Start a private PostgreSQL 16 instance, verify persistence, separate application roles, and follow the tested backup and restore lab.","title":"PostgreSQL with Docker Compose — Deployment and Recovery Boundaries"},{"content":"You\u0026rsquo;ve outgrown your first Docker host, you\u0026rsquo;re rebuilding Proxmox nodes, or you just want to move a stack to a beefier server without starting from scratch. Migrating Docker containers between hosts is a task every homelab operator eventually faces, and doing it wrong means losing persistent data, breaking DNS, or forgetting critical .env files.\nThis guide covers three migration methods — volume-level, NFS-based zero-copy, and full-environment rsync — with real commands and a complete workflow example. By the end you\u0026rsquo;ll know exactly how to move any container or Compose stack between hosts cleanly.\nAll commands target Debian 12 / Ubuntu 24.04 with Docker Engine 27+ and Docker Compose v2.\n1. When and Why You Migrate Migration scenarios in a homelab typically fall into one of three buckets:\nServer upgrade — replacing an older LXC or VM with a newer Proxmox host Hardware failure — a disk is failing, ZFS pool is degrading, move services off fast Rebalancing — spreading workloads across multiple hosts to avoid resource contention The core challenge is always the same: containers are ephemeral, but the data they write — databases, uploaded files, configuration — is not. A well-planned migration treats the container as disposable and the data as precious.\n2. Strategy Overview — Three Migration Methods Method Downtime Complexity Best For Volume backup/restore Medium (minutes) Low Single containers, small stacks NFS zero-copy Low (seconds) Medium Services with shared storage Full /var/lib/docker rsync High (10-30 min) Low Full environment rebuilds Picking the right method depends on how much downtime you can tolerate and how much data you\u0026rsquo;re moving.\n3. Method A — Container-Level Migration with Volume Backup This is the most portable approach. You back up each named volume as a compressed tarball, save the container image, and re-create the container on the new host.\nBackup Volumes 1 2 3 4 5 6 7 8 # List your named volumes docker volume ls # Backup a single volume (e.g., grafana_data) docker run --rm \\ -v grafana_data:/data \\ -v /tmp/backup:/backup \\ alpine tar czf /backup/grafana_data.tar.gz -C /data . For bind mounts, you can tar the host path directly:\n1 tar czf /tmp/backup/prometheus_data.tar.gz -C /tank/monitoring/prometheus . Save and Compress Images 1 2 docker save grafana/grafana:latest | gzip \u0026gt; /tmp/backup/grafana-image.tar.gz docker save prom/prometheus:latest | gzip \u0026gt; /tmp/backup/prometheus-image.tar.gz If you have a local registry, push images instead for a faster transfer:\n1 2 docker tag grafana/grafana:latest registry.gntech.dev/grafana:latest docker push registry.gntech.dev/grafana:latest Transfer and Restore on the New Host 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # Transfer backups rsync -avz /tmp/backup/ root@10.0.20.31:/tmp/backup/ # On the new host — load images gunzip -c /tmp/backup/grafana-image.tar.gz | docker load gunzip -c /tmp/backup/prometheus-image.tar.gz | docker load # Restore volumes docker run --rm \\ -v grafana_data:/data \\ -v /tmp/backup:/backup \\ alpine tar xzf /backup/grafana_data.tar.gz -C /data docker run --rm \\ -v prometheus_data:/data \\ -v /tmp/backup:/backup \\ alpine tar xzf /backup/prometheus_data.tar.gz -C /data Deploy the Container Keep your docker-compose.yml in version control so it\u0026rsquo;s already on the new host:\n1 2 3 4 5 6 7 8 9 10 11 12 13 services: grafana: image: grafana/grafana:latest container_name: grafana volumes: - grafana_data:/var/lib/grafana ports: - \u0026#34;3000:3000\u0026#34; restart: unless-stopped volumes: grafana_data: external: true 1 docker compose up -d 4. Method B — NFS-Based Zero-Copy Migration If your homelab already uses NFS (see our NFS storage homelab guide), you can migrate without copying a single byte of application data. The idea is simple: mount the same NFS export on both hosts, and all you move is the container runtime.\nNFS Volume in a Compose File 1 2 3 4 5 6 7 8 9 10 11 12 services: prometheus: image: prom/prometheus:latest volumes: - prometheus_data:/prometheus volumes: prometheus_data: driver_opts: type: nfs o: addr=10.0.20.10,nfsvers=4,hard,rw device: :/tank/monitoring/prometheus Migration Sequence 1 2 3 4 5 6 7 8 # On the source host — stop the stack cd /opt/stacks/monitoring docker compose down # On the target host — deploy the exact same compose file cd /opt/stacks/monitoring git pull # or rsync the compose file docker compose up -d The data stays mounted on the NFS share the entire time. Downtime is only the time it takes for docker compose down and docker compose up -d — typically under 10 seconds. This works best for stateless or NFS-friendly services like Prometheus, Loki, Grafana, and Pi-hole.\nCaveat: Databases (PostgreSQL, MariaDB) should not use NFS as their primary data store without proper sync/fsync tuning. For those, use Method A with a proper dump.\n5. Method C — Full Docker Environment Migration When you\u0026rsquo;re replacing a host entirely or the old host is failing, you can rsync the entire Docker data directory. This is brute-force but works.\n1 2 3 4 5 6 7 8 9 10 # On the source host systemctl stop docker # On the target host — rsync the entire data directory rsync -avz --progress \\ root@10.0.20.30:/var/lib/docker/ \\ /var/lib/docker/ # Start Docker on the target systemctl start docker Optimisation tip: Exclude the overlay2 directory (container layers) if you plan to pull fresh images on the target. This saves gigabytes of transfer time:\n1 2 3 rsync -avz --progress --exclude=\u0026#39;overlay2/\u0026#39; \\ root@10.0.20.30:/var/lib/docker/ \\ /var/lib/docker/ After rsync, run docker image ls and docker volume ls on the target to verify everything arrived. Start containers with docker start $(docker ps -aq) or your Compose stack.\n6. Database Migration Considerations Databases are the most sensitive part of any migration. Use application-aware dumps:\nPostgreSQL:\n1 2 3 4 5 docker exec postgres pg_dump -U appdb -d myapp \\ --clean --if-exists \u0026gt; /backup/myapp.sql # Restore cat /backup/myapp.sql | docker exec -i postgres psql -U appdb -d myapp MariaDB / MySQL:\n1 2 3 4 5 docker exec mariadb mysqldump --all-databases \\ --single-transaction \u0026gt; /backup/mariadb.sql # Restore cat /backup/mariadb.sql | docker exec -i mariadb mysql SQLite:\n1 2 3 # Container must be stopped docker stop vaultwarden cp /opt/vaultwarden/data/db.sqlite3 /backup/ Redis:\n1 2 docker exec redis redis-cli SAVE # Then copy /data/dump.rdb from the volume Never raw-copy a database volume without stopping the container first — you risk silent corruption from in-flight writes.\n7. Network Identity After Migration Containers that rely on static IPs (macvlan, ipvlan) lose their network identity when they move.\nMacvlan / Static IPs If your container uses a macvlan network for LAN access, the IP must be available on the target host\u0026rsquo;s network. Update your DHCP reservation or adjust the Compose config:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 networks: my_macvlan: driver: macvlan driver_opts: parent: eth0 ipam: config: - subnet: \u0026#34;10.0.30.0/24\u0026#34; gateway: \u0026#34;10.0.30.1\u0026#34; ip_range: \u0026#34;10.0.30.200/28\u0026#34; services: nginx: networks: my_macvlan: ipv4_address: \u0026#34;10.0.30.201\u0026#34; DNS and Traefik Most homelab stacks front services through Traefik or Nginx. Since Traefik routers are configured in Compose labels, they travel with the stack. The only DNS change is updating your domain\u0026rsquo;s A record or internal DNS to point to the new host IP.\n8. Migration Checklist Before you start pulling the plug, run through this checklist:\nBackup all named volumes to compressed tar files Export container images or push to a private registry Copy .env files and any secrets referenced in Compose Verify volume mount paths exist on the target host Update DNS records or reverse proxy rules for the new IP Test a single non-critical service first Keep the old host running for a 24-hour rollback window 9. Real-World Example: Migrate Traefik + Grafana + Prometheus Here\u0026rsquo;s a complete migration of a monitoring stack from a source LXC (10.0.20.30) to a new host (10.0.20.31):\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 # ==== On source host (10.0.20.30) ==== cd /opt/stacks/monitoring # 1. Backup data volumes docker run --rm -v grafana_data:/data -v $(pwd)/backup:/backup \\ alpine tar czf /backup/grafana_data.tar.gz -C /data . docker run --rm -v prometheus_data:/data -v $(pwd)/backup:/backup \\ alpine tar czf /backup/prometheus_data.tar.gz -C /data . # 2. Save images docker save grafana/grafana:latest | gzip \u0026gt; backup/grafana-image.tar.gz docker save prom/prometheus:latest | gzip \u0026gt; backup/prometheus-image.tar.gz docker save traefik:v3.3 | gzip \u0026gt; backup/traefik-image.tar.gz docker save grafana/loki:latest | gzip \u0026gt; backup/loki-image.tar.gz # 3. Transfer everything to the new host rsync -avz backup/ root@10.0.20.31:/tmp/migration/ rsync -avz docker-compose.yml root@10.0.20.31:/opt/stacks/monitoring/ rsync -avz .env root@10.0.20.31:/opt/stacks/monitoring/ # 4. Stop the stack docker compose down # ==== On target host (10.0.20.31) ==== cd /opt/stacks/monitoring # 5. Load images for f in /tmp/migration/*.tar.gz; do gunzip -c \u0026#34;$f\u0026#34; | docker load done # 6. Create and restore volumes docker volume create grafana_data docker volume create prometheus_data docker run --rm -v grafana_data:/data -v /tmp/migration:/backup \\ alpine tar xzf /backup/grafana_data.tar.gz -C /data docker run --rm -v prometheus_data:/data -v /tmp/migration:/backup \\ alpine tar xzf /backup/prometheus_data.tar.gz -C /data # 7. Bring it up docker compose up -d Total downtime: approximately 3-5 minutes. Rollback is just reversing the process on the old host.\n10. Summary Docker container migration doesn\u0026rsquo;t have to be stressful. Pick the method that fits your stack:\nVolume backup/restore for most single-container and small-stack moves NFS zero-copy for large data sets and minimal downtime Full rsync for whole-environment migrations The most important rule: test your restore process on a throwaway host before doing it for real. Practise the migration with a non-critical service first, then scale up. Keep old hosts around for at least 24 hours post-migration, and version-control your Compose files so you\u0026rsquo;re never hunting for configs after the move.\nFor automated volume backups, check out our restic Docker backup guide — it pairs perfectly with the migration and restore workflows covered here.\nNFS outage behavior The application-storage example uses a hard mount. Soft mounts can return I/O errors and risk corruption; disabling locking is not a general migration workaround. Confirm the selected NFS version, locking semantics, client permissions, and outage/recovery behavior before moving persistent data. This source revision does not establish a live migration or HA failover test. See the NFS mount documentation.\n","permalink":"https://blog.gntech.me/posts/2026-05-28-docker-container-migration-homelab/","summary":"Migrate Docker containers between homelab hosts with minimal downtime. Covers volume backup and restore, image export/import, Compose stack portability, and rsync-based data transfer strategies.","title":"Docker Container Migration — Move Containers Between Hosts in Your Homelab"},{"content":"When your homelab grows past a single router and a handful of VLANs, static routes become a maintenance burden. Every new subnet, every link change, every failover scenario requires manual updates across every router. OSPF (Open Shortest Path First) solves this by dynamically discovering the network topology and converging on the best paths automatically.\nMikroTik RouterOS 7 ships a mature OSPF implementation that supports both OSPFv2 (IPv4) and OSPFv3 (IPv6), routing filter chains for redistribution policy, and full support for stub, NSSA, and area authentication. RouterOS 7.23 and the recent 7.24beta1 continue to refine the routing stack, making this an excellent time to add dynamic routing to your lab.\nThis guide covers everything from a basic two-router setup to multi-area designs, authentication, and cross-platform interop with Linux FRR. All commands target RouterOS 7.1+.\n1. OSPF Concepts for the Homelab OSPF is a link-state IGP (Interior Gateway Protocol). Every router builds a complete map of the network (the LSDB), then runs the SPF (Shortest Path First) algorithm independently. This gives you:\nLoop-free routing by construction Fast convergence when links change Load balancing across equal-cost paths (ECMP) Hierarchical design through areas Key concepts to understand before the CLI:\nConcept Purpose Router ID 32-bit identifier, usually the loopback or highest IP Area Logical grouping of routers — area 0 is the backbone Neighbor Adjacent router exchanging link-state updates DR/BDR Designated Router on broadcast segments (not needed on P2P) Cost Metric based on interface bandwidth (cost = reference / bandwidth) LSA Link-State Advertisement — carries routing information Route types you\u0026rsquo;ll see in the routing table: O (intra-area), O IA (inter-area), O E1/E2 (external, via redistribution).\nIf your homelab has three or more routers, or you run separate VLANs behind different routers, OSPF will save you real time. If you only have one router, static routes are fine — skip ahead or bookmark this for later.\n2. Basic OSPF Configuration — Two-Router Setup Start with the simplest topology: two routers connected by a /30 transit link, each with a LAN subnet behind it.\nTopology:\nRouterA: LAN 10.0.10.0/24, transit .1/30 on ether1 RouterB: LAN 10.0.20.0/24, transit .2/30 on ether1 Transit: 10.99.99.0/30 RouterA Configuration /routing ospf instance set [ find ] name=default-vrf-instance router-id=10.0.10.1 /routing ospf area add name=backbone area-id=0.0.0.0 instance=default-vrf-instance /routing ospf interface-template add networks=10.99.99.0/30 area=backbone add networks=10.0.10.0/24 area=backbone passive=yes RouterB Configuration /routing ospf instance set [ find ] name=default-vrf-instance router-id=10.0.20.1 /routing ospf area add name=backbone area-id=0.0.0.0 instance=default-vrf-instance /routing ospf interface-template add networks=10.99.99.0/30 area=backbone add networks=10.0.20.0/24 area=backbone passive=yes The passive=yes on the LAN-facing network tells OSPF to advertise that subnet but never form a neighbor on it — exactly what you want for a network segment with no other routers.\nVerify OSPF Neighbors /routing ospf neighbor print detail Expected output:\n0 instance=default-vrf-instance router-id=10.0.20.1 address=10.99.99.2 interface=ether1 state=Full priority=1 dead-time=36s State Full means the adjacency is established and databases are synchronized.\nVerify OSPF Routes /routing route print where protocol=ospf You should see:\n0 DAc protocol=ospf dst-address=10.0.20.0/24 gateway=10.99.99.2 distance=110 RouterA now knows about 10.0.20.0/24 via OSPF — no static route needed.\n3. OSPF Network Types and Interface Tuning The default OSPF network type on Ethernet is broadcast, which elects a Designated Router (DR) and Backup DR. On point-to-point links like the transit segment above, this adds unnecessary overhead and delays convergence.\nSwitch the transit network type to ptp (point-to-point):\n/routing ospf interface-template add networks=10.99.99.0/30 area=backbone network-type=ptp Benefits of P2P on transit links:\nNo DR/BDR election (faster convergence) Simpler LSAs (Type 1 only, no Type 2) Smaller OSPF database No multicast timer sensitivity OSPF Interface Costs By default, RouterOS calculates cost as 100 Gbps / interface-speed. For a 1 Gbps link, the cost is 100. For 10 Gbps, cost is 10.\nOverride the cost explicitly if your links have asymmetric speeds:\n/routing ospf interface-template add networks=10.99.99.0/30 area=backbone network-type=ptp cost=50 Lower cost = more preferred path.\nHello/Dead Interval Tuning The default hello interval is 10s with a 40s dead interval. For faster convergence on homelab links:\n/routing ospf interface-template add networks=10.99.99.0/30 area=backbone hello-interval=3s dead-interval=12s Both routers must match hello and dead intervals, or they will reject the adjacency.\n4. Multi-Area OSPF — Three-Router Topology Once you add a third router, multi-area design becomes practical. The backbone area (0.0.0.0) must be contiguous. All other areas connect to it.\nTopology:\nRouterA (10.0.10.1, area 0) — backbone + LAN RouterB (10.0.20.1, area 1) — transit to backbone + LAN RouterC (10.0.30.1, area 2) — transit to backbone + LAN Transit links: 10.99.99.0/30 (A-B), 10.99.99.4/30 (A-C) RouterA (Backbone, Area 0) /routing ospf instance set [ find ] name=default-vrf-instance router-id=10.0.10.1 /routing ospf area add name=backbone area-id=0.0.0.0 instance=default-vrf-instance /routing ospf interface-template add networks=10.99.99.0/30 area=backbone network-type=ptp add networks=10.99.99.4/30 area=backbone network-type=ptp add networks=10.0.10.0/24 area=backbone passive=yes RouterB (Area 1) /routing ospf instance set [ find ] name=default-vrf-instance router-id=10.0.20.1 /routing ospf area add name=area1 area-id=0.0.0.1 instance=default-vrf-instance /routing ospf interface-template add networks=10.99.99.0/30 area=backbone network-type=ptp add networks=10.0.20.0/24 area=area1 passive=yes RouterC (Area 2) /routing ospf instance set [ find ] name=default-vrf-instance router-id=10.0.30.1 /routing ospf area add name=area2 area-id=0.0.0.2 instance=default-vrf-instance /routing ospf interface-template add networks=10.99.99.4/30 area=backbone network-type=ptp add networks=10.0.30.0/24 area=area2 passive=yes Verify inter-area routes on RouterB:\n/routing route print where protocol=ospf You\u0026rsquo;ll see both O (intra-area) and O IA (inter-area) routes. The router learns about 10.0.30.0/24 as an inter-area route via RouterA.\nStub Areas An area can be configured as stub if it has only one ABR (Area Border Router) — a very common homelab pattern. Stub areas block Type 5 LSAs (external routes) and replace them with a default route:\n/routing ospf area add name=stub-area area-id=0.0.0.3 instance=default-vrf-instance type=stub Stub areas reduce the LSDB size and memory usage on lower-end hardware like an RB750Gr3 or hAP ac².\n5. Route Redistribution Redistribution injects routes from other sources (connected, static, BGP) into OSPF.\nRedistribute Connected Routes Advertise all directly connected networks, including management IPs and loopbacks:\n/routing ospf instance set [ find ] redistribute=connected Redistribute Default Route In many homelabs, one router has the default gateway to the ISP. Advertise it into OSPF so all routers know where to send internet-bound traffic:\n/routing ospf instance set [ find ] redistribute=default This injects 0.0.0.0/0 as an O E2 route (external type 2, default metric stays unchanged across the OSPF domain).\nRedistribution with Routing Filters (RouterOS 7) RouterOS 7\u0026rsquo;s routing filter chain gives you fine-grained control over what gets redistributed:\n/routing filter rule add chain=ospf-out rule=\u0026#34;if (dst == 10.0.0.0/8) { accept }\u0026#34; add chain=ospf-out rule=\u0026#34;reject\u0026#34; /routing ospf instance set [ find ] out-filter=ospf-out redistribute=connected This only redistributes connected networks in the 10.0.0.0/8 range, rejecting anything else.\nRoute Tagging for Redistribution Policy Tag redistributed routes so you can match them in filters on other routers:\n/routing filter rule add chain=ospf-out rule=\u0026#34;set tag 100; accept\u0026#34; /routing ospf instance set [ find ] out-filter=ospf-out redistribute=connected Then on another router, use the tag to influence route selection:\n/routing filter rule add chain=ospf-in rule=\u0026#34;if (tag == 100) { set distance 150; accept }\u0026#34; add chain=ospf-in rule=\u0026#34;accept\u0026#34; /routing ospf instance set [ find ] in-filter=ospf-in 6. OSPF Authentication OSPF authentication prevents rogue routers from injecting false routes. RouterOS supports both simple password and MD5 cryptographic authentication.\nSimple Password Authentication /routing ospf interface-template add networks=10.99.99.0/30 area=backbone network-type=ptp authentication=simple auth-key=\u0026#34;homelab-secret\u0026#34; Simple passwords are sent in cleartext — fine for isolated lab networks but not production.\nMD5 Cryptographic Authentication /routing ospf interface-template add networks=10.99.99.0/30 area=backbone network-type=ptp authentication=md5 auth-key-id=1 auth-key=\u0026#34;LabRoute#2026!\u0026#34; Both routers must have matching auth-key-id and auth-key. The key ID rotates keys without breaking the adjacency.\nArea-Level Authentication Apply authentication to all interfaces in an area at once:\n/routing ospf area add name=backbone area-id=0.0.0.0 instance=default-vrf-instance authentication=md5 auth-key-id=1 auth-key=\u0026#34;LabRoute#2026!\u0026#34; Area-level authentication avoids configuring every interface template individually.\n7. OSPF Passive Interfaces Passive interfaces advertise the subnet but never send or process OSPF hellos. Every LAN-facing interface should be passive to prevent OSPF from forming adjacencies on client-facing segments and to reduce OSPF traffic.\nIn the interface-template, you already saw the pattern:\n/routing ospf interface-template add networks=10.0.10.0/24 area=backbone passive=yes Verify that no neighbors exist on the passive subnet:\n/routing ospf neighbor print where interface=bridge_lan An empty output confirms the interface is properly passive.\n8. OSPFv3 (IPv6) Configuration RouterOS 7 supports OSPFv3 for IPv6 routing alongside OSPFv2 for IPv4. They operate independently under the same OSPF instance.\nConfigure OSPFv3 on the same routers:\n/routing ospf instance set [ find ] version=v3 router-id=10.0.10.1 /routing ospf area add name=backbone-v3 area-id=0.0.0.0 instance=default-vrf-instance /routing ospf interface-template add networks=2001:db8:99::/64 area=backbone-v3 network-type=ptp add networks=2001:db8:10::/64 area=backbone-v3 passive=yes Key differences from OSPFv2:\nRouter ID remains an IPv4 address, even when routing only IPv6 Interfaces are identified by link-local IPv6 addresses in neighbor output LSAs carry IPv6 prefixes and next-hops instead of IPv4 ones Authentication uses IPsec AH/ESP instead of built-in MD5 Verify OSPFv3 neighbors:\n/routing ospf neighbor print where version=v3 9. OSPF with Linux FRR (Optional Interop) You can peer a MikroTik router with a Linux host running FRR (Free Range Routing) — useful for testing, education, or routing to a Docker host. This is covered in the Docker networking patterns guide.\nInstall FRR on Debian/Ubuntu 1 apt update \u0026amp;\u0026amp; apt install -y frr frr-pythontools Enable OSPFd 1 2 sed -i \u0026#39;s/ospfd=no/ospfd=yes/\u0026#39; /etc/frr/daemons systemctl restart frr Configure FRR 1 2 3 4 5 6 7 8 9 10 vtysh \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; configure terminal router ospf router-id 10.0.99.1 network 10.99.99.0/30 area 0.0.0.0 network 10.0.100.0/24 area 0.0.0.0 passive-interface eth1 end write EOF Verify Adjacency with MikroTik On FRR:\n1 vtysh -c \u0026#34;show ip ospf neighbor\u0026#34; On the MikroTik side, the peer appears if both sides agree on the network type, area, and authentication. Start with P2P network type on both ends for the simplest interop:\n/routing ospf interface-template add networks=10.99.99.0/30 area=backbone network-type=ptp 10. Monitoring and Troubleshooting These commands will become your daily drivers once OSPF is running:\n# Show all OSPF neighbors and their state /routing ospf neighbor print detail # Dump the OSPF link-state database /routing ospf lsa print # Show OSPF-learned routes in the routing table /routing route print where protocol=ospf # Real-time OSPF logs /log print where topics~\u0026#34;ospf\u0026#34; # Show OSPF instance summary /routing ospf instance print detail Common Issues and Fixes Symptom Likely Cause Fix Neighbor stuck in ExStart/Exchange MTU mismatch Set mtu=1500 on both transit interfaces Neighbor stuck in Init Network type mismatch Set both sides to broadcast or ptp No routes received Area mismatch Verify both sides use the same area ID Authentication fails Mismatched key or key-id Double-check auth-key and auth-key-id Routes flapping Cost oscillation Set explicit cost= on interface templates MTU tip: RouterOS defaults to 1500 on Ethernet. If you use VLAN tagging or bridge VXLAN (common with Proxmox SDN), the effective MTU drops. Check with:\n/interface ethernet print mtu /interface vlan print mtu If the transit MTU is lower than 1500, lower it on the OSPF interface template too, or MTU mismatch will block the adjacency.\nSummary OSPF on MikroTik RouterOS 7 is production-class dynamic routing accessible to any homelab owner with more than one router. Start with a two-router single-area setup, verify the neighbor states, and expand from there.\nThe RouterOS 7 interface-template approach makes configuration clean — no more per-interface OSPF setup like RouterOS 6 required. With routing filters, stub areas, and authentication, you have everything needed for a robust dynamic routing environment.\nNext steps: Once you\u0026rsquo;re comfortable with OSPF, explore BGP for WAN-facing routing, VRF for multi-tenant network segmentation, or pair OSPF with the MikroTik firewall rules for inter-VLAN filtering. For a broader look at the RouterOS 7 routing stack, check the RouterOS 7 best practices guide.\n","permalink":"https://blog.gntech.me/posts/2026-05-28-mikrotik-routeros7-ospf-dynamic-routing/","summary":"Configure OSPF dynamic routing on MikroTik RouterOS 7 for your homelab. Step-by-step guide with real RouterOS CLI commands, area design patterns, route redistribution, and multi-router topology examples.","title":"MikroTik RouterOS 7 OSPF — Dynamic Routing for Your Homelab"},{"content":"If you manage more than a handful of Docker containers, you already know the pain. SSH into a node, scroll through docker logs -f, grep for error strings, repeat. No central search. No retention policy. No correlation with metrics.\nGrafana Loki solves this. It is a log aggregation system designed around the same philosophy as Prometheus — cheap, scalable, and tightly integrated with Grafana. Unlike Elasticsearch, Loki indexes only metadata labels and compresses the log content itself, so it runs comfortably on a homelab server with 2 GB of RAM.\nThis post walks through a complete Loki stack deployment with Grafana Alloy (the modern replacement for Promtail) as the log collector, all in Docker Compose.\nArchitecture: How the Stack Fits Together The stack has three components:\nGrafana Alloy — watches Docker containers, scrapes stdout/stderr, attaches metadata labels, ships logs to Loki Loki — stores and indexes log streams, exposes LogQL query API on port 3100 Grafana — queries Loki, builds dashboards, fires alerts Alloy replaces Promtail. It supports the same pipelines but adds native OpenTelemetry, Prometheus scraping, and a more flexible configuration language.\nDeploying Loki with Docker Compose Create a project directory:\n1 mkdir -p /opt/loki-stack \u0026amp;\u0026amp; cd /opt/loki-stack Start with docker-compose.yml. Loki runs as a single binary and in single-process mode for homelab use — no microservices needed:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 services: loki: image: grafana/loki:3.4 container_name: loki ports: - \u0026#34;3100:3100\u0026#34; volumes: - ./loki-config.yml:/etc/loki/config.yml:ro - ./loki-data:/loki command: -config.file=/etc/loki/config.yml restart: unless-stopped networks: - loki-net networks: loki-net: driver: bridge The config file loki-config.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 auth_enabled: false server: http_listen_port: 3100 grpc_listen_port: 9095 common: instance_addr: 127.0.0.1 path_prefix: /loki storage: filesystem: chunks_directory: /loki/chunks rules_directory: /loki/rules replication_factor: 1 ring: kvstore: store: inmemory schema_config: configs: - from: 2024-01-01 store: tsdb object_store: filesystem schema: v13 index: prefix: index_ period: 24h limits_config: reject_old_samples: true reject_old_samples_max_age: 168h max_query_series: 500 ingestion_rate_mb: 10 per_stream_rate_limit: 5MB compactor: working_directory: /loki/compactor compaction_interval: 10m Key settings:\nschema: v13 with store: tsdb is the current recommended schema — fast queries, low memory reject_old_samples_max_age: 168h discards logs older than 7 days at ingestion ingestion_rate_mb: 10 is fine for homelab; bump to 50 if you run 50+ containers Deploying Grafana Alloy for Log Collection Grafana Alloy watches the Docker socket and streams container logs to Loki. Add it to docker-compose.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 alloy: image: grafana/alloy:v1.8 container_name: alloy ports: - \u0026#34;12345:12345\u0026#34; volumes: - ./alloy-config.alloy:/etc/alloy/config.alloy:ro - /var/run/docker.sock:/var/run/docker.sock:ro command: run --server.http.listen-addr=0.0.0.0:12345 /etc/alloy/config.alloy restart: unless-stopped networks: - loki-net The Alloy config file alloy-config.alloy uses the River configuration language:\n// Discover all running Docker containers discovery.docker \u0026#34;all_containers\u0026#34; { host = \u0026#34;unix:///var/run/docker.sock\u0026#34; } // Scrape logs from discovered containers loki.source.docker \u0026#34;container_logs\u0026#34; { host = \u0026#34;unix:///var/run/docker.sock\u0026#34; targets = discovery.docker.all_containers.targets forward_to = [loki.process.add_compose_label] } // Add useful labels from Docker metadata loki.process \u0026#34;add_compose_label\u0026#34; { forward_to = [loki.write.loki_endpoint] stage.static_labels { values = { source = \u0026#34;docker\u0026#34;, } } stage.labels { values = { container_name = \u0026#34;\u0026#34;, compose_project = \u0026#34;\u0026#34;, container_image = \u0026#34;\u0026#34;, } } } // Ship to Loki loki.write \u0026#34;loki_endpoint\u0026#34; { endpoint { url = \u0026#34;http://loki:3100/loki/api/v1/push\u0026#34; } } What this does:\ndiscovery.docker discovers running containers via the Docker socket loki.source.docker tails stdout/stderr from each container loki.process attaches labels — container_name, compose_project, container_image loki.write pushes everything to Loki No sidecar containers, no logging driver changes in other Compose stacks. Alloy handles it transparently.\nDeploying Grafana with Loki Datasource Add Grafana to docker-compose.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 grafana: image: grafana/grafana:11.5 container_name: grafana ports: - \u0026#34;3000:3000\u0026#34; volumes: - ./grafana-data:/var/lib/grafana - ./grafana-provisioning:/etc/grafana/provisioning:ro environment: - GF_SECURITY_ADMIN_USER=admin - GF_SECURITY_ADMIN_PASSWORD=admin - GF_INSTALL_PLUGINS= restart: unless-stopped networks: - loki-net Provision the Loki datasource automatically so you don\u0026rsquo;t need to click through the UI each time:\nCreate grafana-provisioning/datasources/loki.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 apiVersion: 1 datasources: - name: Loki type: loki access: proxy url: http://loki:3100 isDefault: false editable: true jsonData: maxLines: 1000 derivedFields: - datasourceUid: prometheus matcherRegex: \u0026#34;trace_id=(\\\\w+)\u0026#34; name: TraceID url: \u0026#34;$${__value.raw}\u0026#34; Visualizing Logs with LogQL LogQL is Loki\u0026rsquo;s query language — similar to PromQL but for log streams. Basic patterns:\nSee all logs from a specific container:\n{container_name=\u0026#34;traefik\u0026#34;} Find errors across all containers:\n{container_name=~\u0026#34;.+\u0026#34;} |= \u0026#34;error\u0026#34; Filter by compose project:\n{compose_project=\u0026#34;monitoring\u0026#34;} |= \u0026#34;timeout\u0026#34; | json Count errors per container over 5-minute windows:\nsum by (container_name) (count_over_time({container_name=~\u0026#34;.+\u0026#34;} |= \u0026#34;error\u0026#34; [5m])) Parse JSON logs from Traefik and extract status codes:\n{container_name=\u0026#34;traefik\u0026#34;} | json | status_code \u0026gt;= 500 | line_format \u0026#34;{{.container_name}}: {{.status_code}} {{.request_uri}}\u0026#34; To build a dashboard in Grafana:\nAdd a new dashboard → \u0026ldquo;Logs\u0026rdquo; panel Query: {container_name=~\u0026quot;.+\u0026quot;} |= \u0026quot;error\u0026quot; | json Set visualization to \u0026ldquo;Logs\u0026rdquo; (built-in log viewer with highlighting) Add a \u0026ldquo;Time series\u0026rdquo; panel with rate({container_name=~\u0026quot;.+\u0026quot;}[5m]) for log volume per container Log Retention and S3 Storage For a homelab with 10-20 containers, filesystem storage works fine for about 7-14 days. Beyond that, configure Loki to use MinIO for scalable S3-compatible storage:\nAdd MinIO to the stack:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 minio: image: minio/minio:latest container_name: minio command: server /data --console-address \u0026#34;:9001\u0026#34; ports: - \u0026#34;9000:9000\u0026#34; - \u0026#34;9001:9001\u0026#34; volumes: - ./minio-data:/data environment: MINIO_ROOT_USER: loki MINIO_ROOT_PASSWORD: changeme123 restart: unless-stopped networks: - loki-net Update loki-config.yml storage block:\n1 2 3 4 5 6 7 8 storage: s3: endpoint: minio:9000 access_key_id: loki secret_access_key: changeme123 insecure: true bucketnames: loki-logs s3forcepathstyle: true Retention configuration (28-day retention for chunks, 7-day for index):\n1 2 3 table_manager: retention_deletes_enabled: true retention_period: 672h Alerting on Log Patterns Grafana can alert on log patterns using Loki as a data source.\nAlert: Error rate exceeds threshold\nCreate a Grafana alert rule with the query: sum(rate({container_name=~\u0026#34;.+\u0026#34;} |= \u0026#34;error\u0026#34; [5m])) by (container_name) \u0026gt; 0.1 Set condition: WHEN last() OF query (A, 5m, now) IS ABOVE 0.1 Add notification channel (email, Telegram, Slack) Alert: Container crash loop detected\ncount_over_time({container_name=~\u0026#34;.+\u0026#34;} != \u0026#34;alive\u0026#34; != \u0026#34;ready\u0026#34; [2m]) \u0026gt; 10 Troubleshooting Check Alloy logs:\n1 docker compose logs alloy Verify Loki is running:\n1 curl http://localhost:3100/ready List available labels in Loki:\n1 curl -s http://localhost:3100/loki/api/v1/label | jq . Test log push from curl:\n1 2 3 curl -s -X POST http://localhost:3100/loki/api/v1/push \\ -H \u0026#34;Content-Type: application/json\u0026#34; \\ -d \u0026#39;{\u0026#34;streams\u0026#34;:[{\u0026#34;stream\u0026#34;:{\u0026#34;test\u0026#34;:\u0026#34;log\u0026#34;},\u0026#34;values\u0026#34;:[[\u0026#34;\u0026#39;$(date +%s%N)\u0026#39;\u0026#34;,\u0026#34;this is a test log line\u0026#34;]]}]}\u0026#39; Common issues\nNo logs appearing in Grafana — Alloy needs access to the Docker socket (/var/run/docker.sock). Check the volume mount. Labels missing — The loki.source.docker block in Alloy must have labels forwarded. Ensure the pipeline chain is complete. High memory usage — Reduce max_query_series in limits_config and lower ingestion_rate_mb. Loki refuses old logs — Check reject_old_samples_max_age in the config. Default is 168h (7 days). Next Steps This Loki stack gives you centralized log search, retention policies, and alerting across all Docker containers in your homelab. From here you can:\nAdd Prometheus to the same stack and correlate logs with metrics (CPU, memory, disk) in Grafana Set up Grafana OnCall for incident management Deploy a second Alloy instance on another Docker host and ship to the same Loki Replace the filesystem backend with S3 for production-grade retention Use the Compose and configuration snippets above as a starting point and validate them against your Loki version. This article does not provide a separate downloadable repository.\n","permalink":"https://blog.gntech.me/posts/2026-05-27-grafana-loki-centralized-docker-logging/","summary":"Collect, analyze, and alert on Docker container logs with Grafana Loki and Grafana Alloy. A complete centralized logging stack for your homelab with real Compose configs and dashboards.","title":"Grafana Loki — Centralized Docker Logging for Your Homelab"},{"content":"Manually creating LXC containers and VMs through the Proxmox web UI works for a handful of servers, but once you cross twenty containers across multiple VLANs, you need something repeatable. That is where GitOps comes in: your infrastructure definition lives in Git, and every push triggers an automated pipeline that applies the changes.\nThis guide covers the full stack: Proxmox API token creation, Terraform/OpenTofu provider setup, LXC and VM resource definitions, remote state management, and a CI/CD pipeline that turns git push into deployed infrastructure.\nAll examples target Proxmox VE 8.x with Terraform 1.6+ or OpenTofu 1.6+. I recommend OpenTofu for homelabs — it is the fully open-source fork with no licence changes to worry about.\n1. Proxmox API Token for Terraform Terraform needs API access to your Proxmox host or cluster. Create a dedicated user and token so you can revoke access without touching your root credentials.\n1 2 3 pveum user add terraform@pve --comment \u0026#34;Terraform IaC\u0026#34; pveum acl modify / -user terraform@pve -role Administrator pveum user token add terraform@pve terraform-token --privsep false The last command prints a token ID and secret. Save the secret immediately — Proxmox will not show it again. Store it in your password manager or encrypted vault.\nFor a more restricted setup, create a custom role with only the permissions Terraform needs (VM.Allocate, Datastore.AllocateSpace, Sys.Audit) instead of full Administrator.\n2. Provider and Backend Configuration The community provider bpg/proxmox is the most actively maintained option and supports both LXC and QEMU resources.\nCreate providers.tf:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 terraform { required_version = \u0026#34;\u0026gt;= 1.6\u0026#34; required_providers { proxmox = { source = \u0026#34;bpg/proxmox\u0026#34; version = \u0026#34;~\u0026gt; 0.70\u0026#34; } } } provider \u0026#34;proxmox\u0026#34; { endpoint = var.pm_api_url api_token = var.pm_api_token ssh { agent = true node { name = var.pm_node address = var.pm_node_ip } } } Create terraform.tfvars with your actual values (add to .gitignore):\n1 2 3 4 pm_api_url = \u0026#34;https://10.0.20.30:8006/api2/json\u0026#34; pm_api_token = \u0026#34;terraform@pve!terraform-token=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx\u0026#34; pm_node = \u0026#34;srv1\u0026#34; pm_node_ip = \u0026#34;10.0.20.30\u0026#34; 3. LXC Container with Declarative Config Define an LXC container in containers/web01/main.tf:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 resource \u0026#34;proxmox_virtual_environment_container\u0026#34; \u0026#34;web01\u0026#34; { node_name = var.pm_node vm_id = 210 template { file_id = \u0026#34;local:vztmpl/debian-12-standard_12.7-1_amd64.tar.zst\u0026#34; } startup = true start_on_boot = true unprivileged = true cpu { cores = 2 } memory { dedicated = 1024 swap = 512 } disk { datastore_id = \u0026#34;local-zfs\u0026#34; size = 8 } network { name = \u0026#34;eth0\u0026#34; bridge = \u0026#34;vmbr0\u0026#34; ip = \u0026#34;dhcp\u0026#34; } features { nesting = true } } For multiple similar containers, use a module:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 module \u0026#34;app_containers\u0026#34; { source = \u0026#34;./modules/lxc/\u0026#34; for_each = var.app_containers vm_id = each.value.vm_id hostname = each.key cores = each.value.cores memory = each.value.memory disk_size = each.value.disk_size bridge = each.value.bridge ip_address = each.value.ip_address gateway = each.value.gateway } 4. VM Deployment from Template For full VMs, first build a template with Packer, then clone it with Terraform. This separates the image build (Packer) from the deployment (Terraform).\nPacker template snippet proxmox-debian.pkr.hcl:\n1 2 3 4 5 6 7 8 9 10 11 source \u0026#34;proxmox-iso\u0026#34; \u0026#34;debian\u0026#34; { proxmox_url = var.pm_api_url username = var.pm_api_token_id token = var.pm_api_token_secret node = var.pm_node vm_name = \u0026#34;debian-13-template\u0026#34; template_description = \u0026#34;Debian 13, generated {{ isotime \\\u0026#34;2006-01-02\\\u0026#34; }}\u0026#34; memory = 2048 cores = 2 cloud_init = true } Terraform VM resource cloning the template:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 resource \u0026#34;proxmox_virtual_environment_vm\u0026#34; \u0026#34;app01\u0026#34; { node_name = var.pm_node vm_id = 301 name = \u0026#34;app01\u0026#34; template = \u0026#34;debian-13-template\u0026#34; cpu { cores = 4 type = \u0026#34;host\u0026#34; } memory { dedicated = 4096 } disk { datastore_id = \u0026#34;local-zfs\u0026#34; size = 40 file_format = \u0026#34;raw\u0026#34; } network { bridge = \u0026#34;vmbr0\u0026#34; model = \u0026#34;virtio\u0026#34; vlan = 20 } agent { enabled = true } } 5. Remote State — Required for GitOps Local terraform.tfstate does not work in a GitOps pipeline because every CI runner starts fresh. You need a remote backend.\nOption A — S3-compatible (MinIO, garage):\n1 2 3 4 5 6 7 8 9 10 11 12 13 terraform { backend \u0026#34;s3\u0026#34; { bucket = \u0026#34;terraform-state\u0026#34; key = \u0026#34;proxmox/terraform.tfstate\u0026#34; region = \u0026#34;us-east-1\u0026#34; endpoint = \u0026#34;https://minio.gntech.dev\u0026#34; skip_credentials_validation = true skip_region_validation = true skip_requesting_account_id = true skip_metadata_api_check = true use_path_style = true } } Option B — HTTP backend (Gitea or GitLab):\n1 2 3 4 5 6 7 terraform { backend \u0026#34;http\u0026#34; { address = \u0026#34;https://git.gntech.dev/api/packages/iac/terraform-state/proxmox\u0026#34; lock_address = \u0026#34;https://git.gntech.dev/api/packages/iac/terraform-state/proxmox/lock\u0026#34; unlock_address = \u0026#34;https://git.gntech.dev/api/packages/iac/terraform-state/proxmox/lock\u0026#34; } } 6. Repository Layout iac/ ├── environments/ │ ├── prod/ │ │ ├── main.tf │ │ ├── variables.tf │ │ ├── terraform.tfvars # gitignored │ │ └── terraform.tfvars.example │ └── staging/ │ ├── main.tf │ ├── variables.tf │ └── terraform.tfvars ├── modules/ │ ├── lxc/ │ │ ├── main.tf │ │ ├── variables.tf │ │ └── outputs.tf │ └── vm/ │ ├── main.tf │ └── variables.tf ├── .github/workflows/ (or .gitea/workflows/) ├── providers.tf └── README.md Each environment has its own .tfvars with VLAN IDs, IP ranges, and datastore paths specific to that environment.\n7. Gitea Actions CI/CD Pipeline A Gitea Actions workflow that validates, plans, and applies on push to the main branch:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 name: Terraform Proxmox on: push: branches: [main] pull_request: branches: [main] jobs: validate: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: hashicorp/setup-terraform@v3 with: terraform_version: 1.10.0 - name: Init and Validate run: | cd environments/prod terraform init -backend-config=\u0026#34;address=${{ secrets.TF_HTTP_ADDRESS }}\u0026#34; terraform fmt --check terraform validate plan: needs: validate runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: hashicorp/setup-terraform@v3 - name: Terraform Plan run: | cd environments/prod terraform init -backend-config=\u0026#34;address=${{ secrets.TF_HTTP_ADDRESS }}\u0026#34; terraform plan -no-color env: TF_VAR_pm_api_url: ${{ secrets.PM_API_URL }} TF_VAR_pm_api_token: ${{ secrets.PM_API_TOKEN }} apply: if: github.ref == \u0026#39;refs/heads/main\u0026#39; needs: plan runs-on: ubuntu-latest environment: production steps: - uses: actions/checkout@v4 - uses: hashicorp/setup-terraform@v3 - name: Terraform Apply run: | cd environments/prod terraform init -backend-config=\u0026#34;address=${{ secrets.TF_HTTP_ADDRESS }}\u0026#34; terraform apply -auto-approve env: TF_VAR_pm_api_url: ${{ secrets.PM_API_URL }} TF_VAR_pm_api_token: ${{ secrets.PM_API_TOKEN }} The environment: production step adds a manual approval gate in Gitea/GitHub before Terraform applies changes. Pull requests run only validate and plan — you see exactly what will change before merging.\n8. Full Workflow in Action Here is what happens when you update a container\u0026rsquo;s CPU allocation:\nEdit environments/prod/main.tf — change cores = 2 to cores = 4 Commit and push to a feature branch The CI pipeline runs terraform fmt --check, validate, and plan The plan shows: \u0026ldquo;proxmox_virtual_environment_container.web01 will be updated\u0026rdquo; Open a pull request, review the plan output Merge to main The apply job runs with manual approval Terraform updates the container — CPU limit changed, zero downtime Adding a new LXC is just adding a new resource block to the configuration:\n1 2 3 4 5 6 7 8 9 10 resource \u0026#34;proxmox_virtual_environment_container\u0026#34; \u0026#34;cache\u0026#34; { node_name = var.pm_node vm_id = 211 template { file_id = \u0026#34;local:vztmpl/debian-12-standard_12.7-1_amd64.tar.zst\u0026#34; } cpu { cores = 1 } memory { dedicated = 512 } disk { datastore_id = \u0026#34;local-zfs\u0026#34; size = 4 } network { name = \u0026#34;eth0\u0026#34; bridge = \u0026#34;vmbr0\u0026#34; ip = \u0026#34;dhcp\u0026#34; } } Commit, push, approve — the cache container appears on your Proxmox host. The same workflow works for VM resource changes, network updates, or datastore reconfigurations.\n9. Proxmox GitOps Benefits Manual workflow GitOps workflow Click through web UI Edit YAML/HCL in Git No change history Full git log of every change Easy to skip steps CI enforces validation Human error risk Idempotent applies No rollback plan git revert restores state Conclusion A Proxmox GitOps workflow with Terraform or OpenTofu eliminates the guesswork from infrastructure management. Every LXC and VM is defined declaratively, tracked in Git, and deployed through a repeatable CI/CD pipeline that validates before applying.\nStart small: create an API token, define one LXC container with the bpg/proxmox provider, and set up remote state in MinIO. Once that works, add the CI pipeline and expand to VMs. The infrastructure drift that plagues manual setups disappears when commits drive changes.\nFor the next step, pair this with Ansible for post-deployment configuration management — Terraform provisions the hardware, Ansible configures the software inside it.\n","permalink":"https://blog.gntech.me/posts/2026-05-27-proxmox-terraform-gitops-iac/","summary":"Automate Proxmox LXC and VM provisioning using Terraform or OpenTofu with a GitOps pipeline — declarative infrastructure, remote state, and CI/CD from Git pushes.","title":"Proxmox GitOps — Terraform IaC for LXC and VM Automation"},{"content":"If you\u0026rsquo;ve used Zapier, Make (formerly Integromat), or IFTTT, you know how powerful no-code automation can be. You also know the pain: tiered pricing, execution caps, and your data flowing through third-party servers. n8n solves all three problems by giving you a self-hosted, fair-code workflow automation platform with 400+ integrations, visual node-based editing, and full code support when you need it.\nIn this guide, I\u0026rsquo;ll deploy n8n with Docker Compose using a PostgreSQL backend for production reliability, wire it up behind Traefik with automatic SSL, apply security hardening, and walk through practical homelab automation workflows. Validate the examples on the installed image and tool versions; this revision does not establish runtime evidence for every command or configuration.\nPrerequisites for n8n Docker Deployment Before starting, make sure you have the following:\nDocker Engine 24+ and Docker Compose v2 installed on your Linux host A domain (or subdomain) pointed at your server — we\u0026rsquo;ll use n8n.gntech.dev in examples Traefik running as your reverse proxy (or adapt the labels for Caddy / nginx) At least 1 GB RAM, 1 vCPU, and 10 GB disk for n8n + PostgreSQL data Basic familiarity with docker-compose.yml and environment variables If you\u0026rsquo;re deploying for local-only access without a public domain, you can skip the Traefik section and access n8n directly on port 5678 via your LAN IP.\nProduction-Grade Docker Compose Setup Here\u0026rsquo;s the full docker-compose.yml configured for production — PostgreSQL backend, persistent volumes, Traefik integration, and security hardening from the start.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 # docker-compose.yml — n8n with PostgreSQL and Traefik services: postgres: image: postgres:16-alpine container_name: n8n-postgres restart: unless-stopped volumes: - postgres_data:/var/lib/postgresql/data environment: - POSTGRES_USER=n8n - POSTGRES_PASSWORD=${POSTGRES_PASSWORD:?err} - POSTGRES_DB=n8n healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;pg_isready -U n8n\u0026#34;] interval: 10s timeout: 5s retries: 5 networks: - n8n-internal n8n: image: n8nio/n8n:latest container_name: n8n restart: unless-stopped depends_on: postgres: condition: service_healthy volumes: - n8n_data:/home/node/.n8n - ./local-files:/files environment: # Core - N8N_PORT=5678 - N8N_HOST=${N8N_HOST:?err} - N8N_PROTOCOL=https - N8N_ENCRYPTION_KEY=${N8N_ENCRYPTION_KEY:?err} - N8N_DISABLE_PRODUCTION_MAIN_PROCESS=false - EXECUTIONS_DATA_PRUNE=true - EXECUTIONS_DATA_MAX_AGE=168 # PostgreSQL - DB_TYPE=postgresdb - DB_POSTGRESDB_HOST=postgres - DB_POSTGRESDB_PORT=5432 - DB_POSTGRESDB_DATABASE=n8n - DB_POSTGRESDB_USER=n8n - DB_POSTGRESDB_PASSWORD=${POSTGRES_PASSWORD:?err} # Security - N8N_BLOCK_FILE_ACCESS_TO_N8N_FOLDER_FILES=true - N8N_METRICS=false - N8N_PERSONALIZATION_ENABLED=false - N8N_AI_ENABLED=false networks: - n8n-internal - traefik-public labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.n8n.rule=Host(`${N8N_HOST}`)\u0026#34; - \u0026#34;traefik.http.routers.n8n.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.n8n.tls=true\u0026#34; - \u0026#34;traefik.http.routers.n8n.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.services.n8n.loadbalancer.server.port=5678\u0026#34; - \u0026#34;traefik.http.middlewares.n8n-headers.headers.framedeny=true\u0026#34; - \u0026#34;traefik.http.middlewares.n8n-headers.headers.sslredirect=true\u0026#34; - \u0026#34;traefik.http.middlewares.n8n-headers.headers.stsincludesubdomains=true\u0026#34; - \u0026#34;traefik.http.middlewares.n8n-headers.headers.stspreload=true\u0026#34; - \u0026#34;traefik.http.middlewares.n8n-headers.headers.contenttypenosniff=true\u0026#34; - \u0026#34;traefik.http.routers.n8n.middlewares=n8n-headers\u0026#34; volumes: postgres_data: n8n_data: networks: n8n-internal: internal: true traefik-public: external: true Create a .env file in the same directory with your secrets:\n1 2 3 4 # .env — n8n secrets, never commit this file POSTGRES_PASSWORD=your-strong-password-here N8N_HOST=n8n.gntech.dev N8N_ENCRYPTION_KEY=your-32-char-plus-encryption-key Generate a strong encryption key:\n1 openssl rand -hex 32 Traefik Dynamic Configuration Notes The labels in the compose file assume your Traefik instance is running on a Docker network called traefik-public with a certificate resolver named letsencrypt. If you\u0026rsquo;re using a different proxy, adjust accordingly.\nFor Caddy, replace the labels section with a Caddyfile entry:\n1 2 3 n8n.gntech.dev { reverse_proxy n8n:5678 } For nginx, use a simple location block pointing at the n8n container on port 5678 with standard SSL termination.\nDeploy and Create Your Admin Account Start the stack:\n1 docker compose up -d Check the logs to confirm n8n starts successfully:\n1 docker compose logs -f n8n Wait for the log line indicating the web server is listening, then open https://n8n.gntech.dev in your browser.\nOn the first visit, n8n prompts you to:\nCreate an owner account (email + password) Optionally configure usage data collection (disable for privacy) Set your timezone under Settings → Workflow → Timezone Executions Data Retention The environment variables EXECUTIONS_DATA_PRUNE=true and EXECUTIONS_DATA_MAX_AGE=168 tell n8n to automatically delete workflow execution history older than 7 days (168 hours). This prevents the PostgreSQL database from growing unbounded.\nSecurity Hardening for n8n n8n runs workflow code that may touch files, APIs, and databases. Hardening your deployment is critical.\nDisable File Access to the n8n Directory The variable N8N_BLOCK_FILE_ACCESS_TO_N8N_FOLDER_FILES=true prevents workflows from reading or writing files inside the .n8n data directory. If your workflows need access to specific files, mount a separate volume (like local-files in the compose file) and use the Read/Write File From Disk node with paths outside .n8n.\nRun as Non-Root User The official n8nio/n8n image already runs as the node user (UID 1000). Verify by checking the container:\n1 2 docker exec n8n whoami # → node Avoid overriding the user unless you have a specific reason.\nUse an Environment File Never hardcode secrets in docker-compose.yml. The .env file should be mode 600 and included in .gitignore:\n1 2 chmod 600 .env echo \u0026#34;.env\u0026#34; \u0026gt;\u0026gt; .gitignore Rate Limiting Add Traefik rate-limiting middleware to protect the login endpoint:\n1 2 3 4 5 6 7 # In your Traefik dynamic config http: middlewares: n8n-ratelimit: rateLimit: average: 10 burst: 20 Practical Homelab Automation Workflows Once n8n is running, here are three workflows I use daily in my homelab.\n1. Telegram Bot → Docker Container Notifications Use n8n\u0026rsquo;s Telegram Trigger node to listen for commands, then execute a Execute Command node to restart containers or fetch their status.\n[Telegram Trigger: /status] → [Execute Command: docker ps --format] → [Telegram: Send Message with output] 2. RSS Monitor → Proxmox Backup Status Poll an RSS feed (or a Proxmox API endpoint) on a schedule, format the results, and send notifications via Telegram or email.\n[RSS Feed Read: hourly] → [Code: filter new items] → [Telegram: notification with title + link] 3. Ollama AI Integration Connect n8n to a local Ollama instance for AI-powered workflows — content summarization, tag generation, or intelligent routing:\n[Manual/Webhook trigger] → [HTTP Request: POST to Ollama /api/generate] → [Code: extract response text] → [Telegram: send AI-generated summary] Set the Ollama endpoint in your n8n credentials as http://ollama:11434 if it\u0026rsquo;s on the same Docker network, or http://10.0.20.30:11434 if on the LAN.\nBackup and Recovery Strategy Backing Up n8n Two things need backup: the PostgreSQL database and the .n8n folder containing workflow definitions and credentials.\n1 2 3 4 5 6 7 8 9 # Backup PostgreSQL database docker exec n8n-postgres pg_dump -U n8n n8n \u0026gt; n8n-db-$(date +%F).sql # Backup n8n data folder docker run --rm -v n8n_data:/data -v $(pwd):/backup alpine \\ tar czf /backup/n8n-data-$(date +%F).tar.gz -C /data . # Export all workflows as JSON files (optional) docker exec n8n n8n export:workflow --all --output=/home/node/.n8n/export Restoring To restore on a fresh host:\n1 2 3 4 5 6 # Restore data folder docker run --rm -v n8n_data:/data -v $(pwd):/backup alpine \\ tar xzf /backup/n8n-data-YYYY-MM-DD.tar.gz -C /data # Restore database docker exec -i n8n-postgres psql -U n8n n8n \u0026lt; n8n-db-YYYY-MM-DD.sql Then start the stack with docker compose up -d.\nConclusion n8n brings enterprise-grade workflow automation to your homelab without the subscription costs or data privacy trade-offs of SaaS alternatives. With Docker Compose, PostgreSQL, and Traefik you get a production-ready deployment that survives reboots, handles SSL automatically, and keeps your sensitive data under your control.\nThe 400+ built-in integrations mean you can connect just about anything — Docker, Proxmox, Telegram, email, databases, HTTP APIs, AI models — into automated pipelines that save you hours each week. Start with one simple workflow: a Telegram bot that checks your Proxmox host status. You\u0026rsquo;ll be building complex multi-step automations before you know it.\nFor the full n8n documentation, visit n8n.io and the Docker install docs.\n","permalink":"https://blog.gntech.me/posts/2026-05-27-n8n-docker-deployment-homelab-automation/","summary":"Complete guide to deploying n8n with Docker Compose in your homelab — PostgreSQL backend, persistent workflows, Traefik SSL, security hardening, and practical workflow automation examples.","title":"n8n Docker Deployment — Self-Hosted Workflow Automation Guide"},{"content":"If you already run Docker Compose on a single Proxmox LXC or bare-metal Linux host, you have a solid container setup. But Compose leaves gaps: containers run until they crash, there is no built-in health-based reconciliation, and rolling out a new version means manual docker compose up -d with a brief window of downtime.\nDocker Swarm fills those gaps with zero additional dependencies. It is built into Docker Engine and turns your single host into a one-node cluster with declarative reconciliation, rolling updates, automatic restart on failure, and built-in service discovery. Best of all, you keep the same compose file format — just add a deploy section and run docker stack deploy instead of docker compose up.\nThis guide walks through initialising a single-node Swarm, converting Compose files to stacks, deploying Traefik and Portainer as Swarm services, managing persistent storage, and the day 2 operations you need for a production homelab.\nWhy Docker Swarm for a Single-Node Homelab The common objection: \u0026ldquo;Swarm is for multi-node clusters. I have one machine.\u0026rdquo; The answer: Swarm\u0026rsquo;s orchestration engine gives you infrastructure-level reliability on that one machine that Compose simply does not provide.\nCapability Docker Compose Docker Swarm Desired-state reconciliation Manual up/restart Automatic Failure recovery Stays stopped Restarts with backoff Rolling updates Manual Declarative, configurable Rollback on failure Manual Automatic on failure detection Service scaling Scale with up deploy.replicas Built-in DNS/service discovery No Per-service A records For a homelab running critical services — DNS, reverse proxy, Git server, monitoring — Swarm\u0026rsquo;s automatic reconciliation is the difference between \u0026ldquo;woke up and found Immich was down since 3 AM\u0026rdquo; and waking up to a notification that the service restarted and is healthy again.\nPrerequisites for Docker Swarm Docker Engine 24+ — Swarm mode is shipped with the engine, no extra package Docker Compose v2 (optional, for local testing before deploying as stack) NFS server or adequate local disk for persistent volumes Port 2377/tcp open (Swarm manager communication) — only needed if you ever add nodes Ports 7946/tcp+udp and 4789/udp for overlay networking (safe to leave closed on single-node, but opened for future nodes) Verify your Docker Engine supports swarm mode:\n1 docker info | grep -i swarm Output on a Swarm-enabled host (even if not in Swarm mode):\nSwarm: inactive Initialise the Swarm on Your Homelab Host Pull the trigger:\n1 docker swarm init --advertise-addr 10.0.20.30 Replace 10.0.20.30 with your homelab server\u0026rsquo;s LAN IP. On a single-node setup, this IP only matters for the manager election — no other nodes need to reach it unless you expand later.\nOutput:\nSwarm initialized: current node (abc123def456) is now a manager. To add a worker to this swarm, run the following command: docker swarm join --token SWMTKN-1-xxxxx 10.0.20.30:2377 To add a manager, run \u0026#39;docker swarm join-token manager\u0026#39; and follow the instructions. Verify the cluster state:\n1 docker node ls ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION abc123def456 * srv1 Ready Active Reachable 27.4.1 On a single-node homelab, the manager also runs workloads. No need to drain the manager — you are not adding worker nodes.\nDeploy Your First Swarm Service A quick smoke test to confirm Swarm is working:\n1 2 3 4 docker service create \\ --name whoami \\ --publish 8080:80 \\ traefik/whoami Then scale it to two replicas:\n1 2 3 docker service scale whoami=2 docker service ls docker service ps whoami Access http://10.0.20.30:8080 and refresh — each request hits a different replica container. Swarm\u0026rsquo;s internal load balancer round-robins across them.\nClean up:\n1 docker service rm whoami Convert a Docker Compose File to a Swarm Stack This is the core migration: turning your existing docker-compose.yml into a docker-stack.yml that docker stack deploy can consume.\nKey Differences Between Compose and Stack 1. Version field is optional. Modern Docker (24+) accepts stack files without version:. Omit it.\n2. Use deploy: instead of direct resource fields.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 services: app: image: nginx:alpine deploy: replicas: 2 update_config: order: start-first parallelism: 1 delay: 10s failure_action: rollback restart_policy: condition: any delay: 5s max_attempts: 3 resources: limits: cpus: \u0026#34;0.5\u0026#34; memory: 256M reservations: cpus: \u0026#34;0.25\u0026#34; memory: 128M 3. Networks must exist or be declared as overlay.\n1 2 3 4 networks: traefik-public: driver: overlay attachable: true attachable: true allows standalone containers (e.g., debugging tools) to connect to the overlay network.\n4. Volumes work the same way for single-node.\nNamed volumes like postgres_data: resolve to the local Docker volume store. For multi-node, you would use NFS, but on a single node, local bind mounts and named volumes are fine.\nFull Example: Static Site with Traefik Here is a complete docker-stack.yml that deploys an nginx static site behind Traefik:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 services: nginx: image: nginx:alpine deploy: replicas: 2 labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.nginx.rule=Host(`static.gntech.dev`)\u0026#34; - \u0026#34;traefik.http.routers.nginx.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.services.nginx.loadbalancer.server.port=80\u0026#34; volumes: - ./static-site:/usr/share/nginx/html:ro networks: - traefik-public networks: traefik-public: name: traefik-public driver: overlay attachable: true Deploy with:\n1 docker stack deploy -c docker-stack.yml static-site Note the traefik.http.services.\u0026lt;name\u0026gt;.loadbalancer.server.port label. In Swarm mode, Traefik needs an explicit port declaration — it cannot inspect container ports the same way it does on Compose.\nDeploy Traefik as a Swarm Service Traefik must run in Swarm mode to work with stack-deployed services. Here is the Traefik stack definition:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 services: traefik: image: traefik:v3.3 command: - \u0026#34;--providers.docker.swarmMode=true\u0026#34; - \u0026#34;--providers.docker.exposedbydefault=false\u0026#34; - \u0026#34;--entrypoints.web.address=:80\u0026#34; - \u0026#34;--entrypoints.websecure.address=:443\u0026#34; - \u0026#34;--certificatesresolvers.letsencrypt.acme.tlschallenge=true\u0026#34; - \u0026#34;--certificatesresolvers.letsencrypt.acme.email=admin@gntech.dev\u0026#34; - \u0026#34;--certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json\u0026#34; ports: - target: 80 published: 80 mode: host - target: 443 published: 443 mode: host volumes: - /var/run/docker.sock:/var/run/docker.sock:ro - traefik-le:/letsencrypt networks: - traefik-public deploy: placement: constraints: - node.role == manager labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.dashboard.rule=Host(`traefik.gntech.dev`)\u0026#34; - \u0026#34;traefik.http.routers.dashboard.service=api@internal\u0026#34; - \u0026#34;traefik.http.routers.dashboard.middlewares=auth\u0026#34; - \u0026#34;traefik.http.middlewares.auth.basicauth.users=admin:$$apr1$$xxxxxxxxx\u0026#34; volumes: traefik-le: networks: traefik-public: external: true name: traefik-public Key points:\nmode: host for ports 80 and 443 — bypasses Swarm\u0026rsquo;s routing mesh for direct host-level port binding (required for Let\u0026rsquo;s Encrypt HTTP-01 challenges). node.role == manager constraint ensures Traefik stays on the manager node. traefik.http.services.dashboard.service=api@internal exposes the Traefik dashboard at the /api endpoint. The network traefik-public must be external: true because it was created before the stack. Create the network and deploy:\n1 2 docker network create --driver=overlay --attachable traefik-public docker stack deploy -c traefik-stack.yml traefik Deploy Portainer for Visual Swarm Management Portainer gives you a web UI for managing your Swarm services without reaching for the terminal every time. On a single-node setup, deploy both the Portainer Agent and Server:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 services: agent: image: portainer/agent:latest volumes: - /var/run/docker.sock:/var/run/docker.sock - /var/lib/docker/volumes:/var/lib/docker/volumes networks: - agent-net deploy: mode: global placement: constraints: - node.platform.os == linux portainer: image: portainer/portainer-ce:latest command: -H tcp://tasks.agent:9001 --tlsskipverify volumes: - portainer-data:/data networks: - agent-net - traefik-public deploy: placement: constraints: - node.role == manager labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.portainer.rule=Host(`portainer.gntech.dev`)\u0026#34; - \u0026#34;traefik.http.routers.portainer.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.services.portainer.loadbalancer.server.port=9000\u0026#34; volumes: portainer-data: networks: agent-net: driver: overlay attachable: true traefik-public: external: true Deploy:\n1 docker stack deploy -c portainer-stack.yml portainer Access Portainer at https://portainer.gntech.dev — you can inspect running services, view logs, scale replicas, and deploy new stacks from the web UI.\nPersistent Storage in Single-Node Swarm Named volumes and bind mounts work identically to Docker Compose on a single host. If you ever plan to expand to multiple nodes, move persistent data to an NFS share.\nCreate an NFS-backed volume:\n1 2 3 4 5 6 docker volume create \\ --driver local \\ --opt type=nfs \\ --opt o=addr=10.0.20.50,rw,nfsvers=4 \\ --opt device=:/export/docker/postgres \\ postgres_data Use it in a stack:\n1 2 3 4 5 6 7 8 9 10 11 12 13 services: postgres: image: postgres:17-alpine environment: POSTGRES_PASSWORD_FILE: /run/secrets/db_password volumes: - postgres_data:/var/lib/postgresql/data deploy: replicas: 1 volumes: postgres_data: external: true The volume is created externally (with docker volume create) so it survives stack removal. The stack references it with external: true.\nMonitoring the Swarm Deploy a monitoring stack to keep visibility into your Swarm services:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 services: cadvisor: image: gcr.io/cadvisor/cadvisor:latest command: - \u0026#34;--docker_only=true\u0026#34; volumes: - /:/rootfs:ro - /var/run:/var/run:ro - /sys:/sys:ro - /var/lib/docker/:/var/lib/docker:ro networks: - monitoring deploy: mode: global prometheus: image: prom/prometheus:latest volumes: - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro - prometheus-data:/prometheus networks: - monitoring deploy: replicas: 1 placement: constraints: - node.role == manager grafana: image: grafana/grafana:latest environment: GF_SECURITY_ADMIN_PASSWORD: changeme volumes: - grafana-data:/var/lib/grafana networks: - monitoring - traefik-public deploy: replicas: 1 placement: constraints: - node.role == manager labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.grafana.rule=Host(`grafana.gntech.dev`)\u0026#34; - \u0026#34;traefik.http.routers.grafana.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.services.grafana.loadbalancer.server.port=3000\u0026#34; volumes: prometheus-data: grafana-data: networks: monitoring: driver: overlay attachable: true traefik-public: external: true Day 2 Operations for Your Swarm Homelab Rolling update:\n1 docker service update --image nginx:1.27 nginx-stack_nginx Rollback if something goes wrong:\n1 docker service update --rollback nginx-stack_nginx View logs across all replicas:\n1 docker service logs --tail 50 nginx-stack_nginx Inspect service state:\n1 docker service ps nginx-stack_nginx Scale a service:\n1 docker service scale nginx-stack_nginx=3 Clean up a stack entirely:\n1 docker stack rm static-site Drain the node for maintenance (only relevant in multi-node, but noted for completeness):\n1 docker node update --availability drain srv1 Why This Beats Multi-Node Overkill A single-node Docker Swarm gives you 90% of Kubernetes-style orchestration benefits with zero networking complexity:\nDeclarative service definitions that self-heal Rolling updates with automatic rollback Built-in service discovery via DNS Traefik integration with zero extra service mesh Portainer for visual management If you ever outgrow one host, adding a second is as simple as running docker swarm join on it. Your stacks and networks stay the same — just add more replicas.\nFor every Compose service you care about, try the Swarm stack alternative. The file diff is small, the operational improvement is immediate, and you keep the same Docker skills you already have.\nRelated reading: Docker Compose patterns for homelab, Docker Trivy vulnerability scanning, Traefik middleware security hardening.\n","permalink":"https://blog.gntech.me/posts/2026-05-27-docker-swarm-homelab-single-node-orchestration/","summary":"Deploy Docker Swarm in single-node mode for orchestration in your homelab — initialize the cluster, convert Compose files to stacks, deploy Portainer and Traefik, and manage persistent storage with NFS.","title":"Docker Swarm Homelab — Single-Node Orchestration with Compose to Swarm"},{"content":"Google Photos is convenient until you hit the 15 GB cap or realise your entire photo library is being used to train someone else\u0026rsquo;s models. Immich solves both problems: it gives you Google Photos-grade functionality — automatic backup, facial recognition, smart search, album sharing — entirely self-hosted on your own hardware.\nThis guide walks through a production-grade Immich deployment with Docker Compose: PostgreSQL for metadata, Redis for job queues, hardware-accelerated transcoding, ML model GPU acceleration, Traefik reverse proxy with automatic SSL, and a restic-based backup strategy for both photos and the database.\nAll commands target Debian 12 / Ubuntu 24.04 with Docker Engine 27+ and Docker Compose v2.\n1. Infrastructure Layout Immich has four runtime components that need to coordinate:\nService Role immich-server Main API server and web frontend immich-microservices Background tasks — transcoding, ML postgres Metadata database — albums, users, tags redis Job queue and cache immich-machine-learning Facial recognition, smart search Two optional but highly recommended additions are hardware transcoding (for HEVC/AV1 video thumbnails) and GPU ML acceleration (for fast face detection and CLIP-based search).\nFor storage, plan for: a fast SSD volume for Postgres data and Redis, and a large HDD or NAS share for library storage (photos, videos, thumbnails).\n2. Directory Layout and Environment Create the project tree:\n1 2 mkdir -p /opt/immich/{db,redis,library,profile,backups} cd /opt/immich Create .env with your initial configuration:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 cat \u0026gt; .env \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; # ── Immich Environment ────────────────────────────────── IMMICH_VERSION=release TZ=America/Santo_Domingo COMPOSE_PROJECT_NAME=immich # ── PostgreSQL ─────────────────────────────────────────── DB_HOSTNAME=immich-postgres DB_USERNAME=immich DB_PASSWORD=$(openssl rand -base64 32) DB_DATABASE_NAME=immich # ── Redis ──────────────────────────────────────────────── REDIS_HOSTNAME=immich-redis # ── Upload ─────────────────────────────────────────────── UPLOAD_LOCATION=./library # ── Internal URLs ──────────────────────────────────────── IMMICH_MICROSERVICES_URL=http://immich-microservices:3001 IMMICH_MACHINE_LEARNING_URL=http://immich-machine-learning:3003 # ── Traefik / Domain (set your domain) ────────────────── IMMICH_DOMAIN=photos.yourdomain.com EOF Store the DB password somewhere safe — you will need it for backups.\n3. Docker Compose Configuration 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 # /opt/immich/docker-compose.yml services: immich-server: image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION} container_name: immich-server command: [\u0026#34;start.sh\u0026#34;, \u0026#34;immich\u0026#34;] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload:z - /etc/localtime:/etc/localtime:ro environment: - NODE_ENV=production env_file: - .env depends_on: - immich-redis - immich-postgres restart: unless-stopped networks: - immich-net labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.immich.rule=Host(`${IMMICH_DOMAIN}`)\u0026#34; - \u0026#34;traefik.http.routers.immich.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.immich.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.services.immich.loadbalancer.server.port=2283\u0026#34; immich-microservices: image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION} container_name: immich-microservices command: [\u0026#34;start.sh\u0026#34;, \u0026#34;microservices\u0026#34;] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload:z - /etc/localtime:/etc/localtime:ro environment: - NODE_ENV=production env_file: - .env depends_on: - immich-redis - immich-postgres restart: unless-stopped networks: - immich-net immich-machine-learning: image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION} container_name: immich-ml volumes: - ./model-cache:/cache:z env_file: - .env restart: unless-stopped networks: - immich-net # ── GPU acceleration (optional) ────────────── # deploy: # resources: # reservations: # devices: # - driver: nvidia # count: 1 # capabilities: [gpu] immich-postgres: image: tensorchord/pgvecto-rs:pg14-v0.3.0 container_name: immich-postgres environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} volumes: - ./db:/var/lib/postgresql/data:z restart: unless-stopped networks: - immich-net healthcheck: test: pg_isready --dbname=\u0026#39;${DB_DATABASE_NAME}\u0026#39; --username=\u0026#39;${DB_USERNAME}\u0026#39; || exit 1 interval: 10s timeout: 5s retries: 5 immich-redis: image: redis:7-alpine container_name: immich-redis volumes: - ./redis:/data:z restart: unless-stopped networks: - immich-net healthcheck: test: redis-cli ping || exit 1 interval: 10s timeout: 3s retries: 5 networks: immich-net: name: immich-net external: true Create the external Docker network if it does not already exist:\n1 docker network create immich-net 4. Traefik Reverse Proxy Integration If you already run Traefik as your edge router, add the Traefik network to the Immich Compose file and let Traefik handle SSL.\nAdd this to the bottom of the services section in docker-compose.yml:\n1 2 3 4 networks: immich-net: name: immich-net external: true Then connect Immich to your Traefik network:\n1 docker network connect traefik-net immich-server The labels on the immich-server service (shown above) assume Traefik\u0026rsquo;s letsencrypt certificate resolver is configured. If you use Caddy or Nginx Proxy Manager instead, configure a reverse proxy pass to http://immich-server:2283.\n5. First Boot and Initial Setup Pull images and start the stack:\n1 2 3 cd /opt/immich docker compose pull docker compose up -d Wait 30-60 seconds for Postgres to initialise, then check logs:\n1 docker compose logs -f immich-server You should see Immich Server is running on port 3001 once it is ready.\nOpen https://photos.yourdomain.com (or http://localhost:2283 locally) and create the initial admin account. This account is the system administrator — store the credentials in your password manager immediately.\n5.1 Postgres Tuning for Immich Immich generates a lot of writes during upload and ML processing. Tune Postgres inside the container:\n1 2 3 4 5 6 7 docker exec -it immich-postgres \\ sh -c \u0026#34;echo \u0026#39;shared_buffers=512MB\u0026#39; \u0026gt;\u0026gt; /var/lib/postgresql/data/postgresql.conf\u0026#34; docker exec -it immich-postgres \\ sh -c \u0026#34;echo \u0026#39;effective_cache_size=2GB\u0026#39; \u0026gt;\u0026gt; /var/lib/postgresql/data/postgresql.conf\u0026#34; docker exec -it immich-postgres \\ sh -c \u0026#34;echo \u0026#39;maintenance_work_mem=256MB\u0026#39; \u0026gt;\u0026gt; /var/lib/postgresql/data/postgresql.conf\u0026#34; docker compose restart immich-postgres Adjust shared_buffers to 25% of available RAM, and effective_cache_size to 75% of available RAM.\n6. Hardware Transcoding Immich transcodes video thumbnails on the fly. Without hardware acceleration, a 4K HEVC to WebP thumbnail conversion drains CPU. Enable GPU passthrough in the immich-microservices service.\n6.1 Intel QuickSync (QSV) Add to the immich-microservices service:\n1 2 3 4 5 6 immich-microservices: # ... existing config ... devices: - /dev/dri:/dev/dri group_add: - \u0026#34;44\u0026#34; # video group Then in the Immich admin panel: Administration → Video Transcoding Settings → Hardware Acceleration → Intel QSV\n6.2 NVIDIA GPU Add to immich-microservices:\n1 2 3 4 5 6 7 8 9 immich-microservices: # ... existing config ... deploy: resources: reservations: devices: - driver: nvidia count: 1 capabilities: [gpu, video] The video capability enables NVENC/NVDEC for hardware video encoding and decoding.\nIn the admin panel: Hardware Acceleration → NVIDIA NVENC/NVDEC\n6.3 Verify Transcoding Upload a short HEVC video through the Immich web UI, then check that the microservice used the GPU:\n1 2 3 4 5 # Intel docker exec immich-microservices ls /dev/dri # NVIDIA docker exec immich-microservices nvidia-smi 7. Machine Learning Acceleration The immich-machine-learning container handles facial recognition, object detection (CLIP), and smart search embeddings. These are computationally expensive — expect 1-3 seconds per image on CPU, or 50-100 ms on GPU.\nEnable GPU for the ML container by uncommenting the deploy block in the immich-machine-learning service section of the compose file, then restart:\n1 docker compose up -d --force-recreate immich-machine-learning After enabling, go to Administration → Machine Learning Settings and set:\nFacial Recognition → Enabled CLIP Search → Enabled Scene Detection → Enabled Trigger a full re-extraction on existing photos:\nAdministration → Jobs → All → Click the gear icon → Run All\nMonitor progress from the command line:\n1 docker compose logs -f immich-microservices 8. Mobile App Configuration Immich has native apps for Android and iOS. Configure them to connect to your Traefik domain:\nInstall the Immich app from the app store Connection → Server Endpoint → https://photos.yourdomain.com Log in with your admin credentials Enable Background Backup (Android) or Backup over Cellular (both platforms) The app syncs new photos within minutes of being taken. You can check sync status from the app\u0026rsquo;s Backup tab.\n9. Backup Strategy Immich stores two types of data: the PostgreSQL database (metadata) and uploaded files (raw photos and videos on disk). A proper backup strategy covers both.\n9.1 Database Backup with pg_dump Create a backup script:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 #!/bin/bash # /opt/immich/scripts/backup-db.sh set -euo pipefail BACKUP_DIR=\u0026#34;/opt/immich/backups\u0026#34; TIMESTAMP=$(date +%Y%m%d_%H%M%S) RETENTION_DAYS=14 source /opt/immich/.env mkdir -p \u0026#34;$BACKUP_DIR\u0026#34; docker exec immich-postgres \\ pg_dump -U \u0026#34;$DB_USERNAME\u0026#34; -d \u0026#34;$DB_DATABASE_NAME\u0026#34; \\ --format=custom --compress=9 \\ -f \u0026#34;/tmp/immich-${TIMESTAMP}.dump\u0026#34; docker cp \u0026#34;immich-postgres:/tmp/immich-${TIMESTAMP}.dump\u0026#34; \\ \u0026#34;${BACKUP_DIR}/immich-${TIMESTAMP}.dump\u0026#34; docker exec immich-postgres \\ rm \u0026#34;/tmp/immich-${TIMESTAMP}.dump\u0026#34; find \u0026#34;$BACKUP_DIR\u0026#34; -name \u0026#34;immich-*.dump\u0026#34; -mtime +$RETENTION_DAYS -delete echo \u0026#34;DB backup: immich-${TIMESTAMP}.dump\u0026#34; Make it executable and schedule it:\n1 chmod +x /opt/immich/scripts/backup-db.sh 9.2 Systemd Timer for Daily Backups 1 2 3 4 5 6 7 8 9 # /etc/systemd/system/immich-backup.service [Unit] Description=Immich Database Backup After=docker.service [Service] Type=oneshot ExecStart=/opt/immich/scripts/backup-db.sh User=root 1 2 3 4 5 6 7 8 9 10 11 # /etc/systemd/system/immich-backup.timer [Unit] Description=Daily Immich DB backup [Timer] OnCalendar=03:00 RandomizedDelaySec=3600 Persistent=true [Install] WantedBy=timers.target 1 2 systemctl daemon-reload systemctl enable --now immich-backup.timer 9.3 Restic Backup for Library Files Use restic to back up the library directory to an S3-compatible storage endpoint (MinIO, Backblaze B2, or Wasabi):\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 # /opt/immich/scripts/backup-library.sh #!/bin/bash set -euo pipefail export RESTIC_REPOSITORY=\u0026#34;s3:https://s3.yourdomain.com/immich-library\u0026#34; export RESTIC_PASSWORD=\u0026#34;\u0026lt;your-restic-password\u0026gt;\u0026#34; export AWS_ACCESS_KEY_ID=\u0026#34;...\u0026#34; export AWS_SECRET_ACCESS_KEY=\u0026#34;...\u0026#34; restic backup /opt/immich/library \\ --tag immich-library \\ --exclude \u0026#34;*.tmp\u0026#34; \\ --verbose restic forget \\ --tag immich-library \\ --keep-daily 7 \\ --keep-weekly 4 \\ --keep-monthly 6 \\ --prune Schedule with a second systemd timer, or combine both jobs into a single service file for simplicity.\n10. Performance Tuning and Resource Limits Immich can use significant resources during initial ML processing and first-time transcoding. Set container-level resource limits to prevent it from starving other services:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 immich-server: # ... existing config ... deploy: resources: limits: memory: 2G immich-microservices: # ... existing config ... deploy: resources: limits: memory: 4G immich-machine-learning: # ... existing config ... deploy: resources: limits: memory: 4G cpus: \u0026#34;4\u0026#34; For the initial library import (first upload of thousands of photos), raise the microservices limit temporarily, then dial it back.\n10.1 Docker System Pruning Immich generates many intermediate objects during transcoding and thumbnail generation. Schedule a weekly prune:\n1 2 3 # /etc/cron.weekly/immich-cleanup #!/bin/bash docker system prune -f --filter \u0026#34;until=72h\u0026#34; 11. Updating Immich Immich releases frequently — typically monthly. Update with zero downtime using rolling restarts:\n1 2 3 4 cd /opt/immich docker compose pull docker compose up -d --remove-orphans docker image prune -f Check the Immich release notes before updating — major releases sometimes require manual database migration steps or version-specific upgrade paths.\n12. Final Checks After deployment, verify the full stack is running:\n1 2 docker compose ps docker compose logs --tail=20 immich-server Open the web UI, upload a test photo, trigger a smart search, and confirm facial recognition detects a known face. If everything works, enable the mobile app backups and point your family to the Immich URL.\nImmich replaces Google Photos without the privacy trade-off. Your photos stay on your drives, your face data stays on your server, and the features — timeline view, album sharing, partner sharing, smart search — match or exceed what the cloud offers. Set up backups early and the rest takes care of itself.\n","permalink":"https://blog.gntech.me/posts/2026-05-27-immich-docker-homelab-deployment/","summary":"Deploy Immich as a self-hosted Google Photos alternative with Docker Compose — PostgreSQL, Redis, hardware-accelerated transcoding, ML-based search, and Traefik reverse proxy for secure remote access.","title":"Immich Docker Deployment — Self-Hosted Google Photos Alternative"},{"content":"nftables can filter IPv4 and IPv6 traffic in one ruleset. The practical challenge is ownership: your host firewall, Docker, a VPN, and a virtualization platform may all create rules. A blanket flush ruleset can remove protections and networking rules belonging to those services.\nThis guide adds a small host-input policy in its own table. It does not replace a router firewall or control published Docker ports.\n1. Check Who Owns the Firewall Use console access or a tested recovery path before changing remote access rules. This example assumes a static-address Linux server with an IPv4 management subnet of 10.0.99.0/24. Adapt it for your network, including DHCP or IPv6 management requirements.\n1 2 3 4 sudo nft list ruleset sudo iptables --version sudo systemctl is-active nftables ufw firewalld docker info 2\u0026gt;/dev/null A command reporting an inactive or absent service is diagnostic information. If UFW, firewalld, Proxmox, or another platform already manages this host, configure its policy through that manager rather than stacking an unexplained additional firewall on top.\nSave the existing ruleset for review:\n1 sudo sh -c \u0026#39;nft list ruleset \u0026gt; /root/nftables-before-host-policy.rules\u0026#39; An iptables version containing nf_tables means its rules use the compatibility backend. It does not mean Docker is using its native nftables backend.\n2. Write a Separate Host-Input Table This policy permits established traffic, loopback, ICMP/ICMPv6, SSH from the management subnet, and public HTTP/HTTPS. It drops other new traffic addressed to the host. Remove HTTP/HTTPS if this server does not need them. Add explicit rules for any required DHCP, monitoring, VPN, or IPv6 management traffic before loading it.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 sudo mkdir -p /etc/nftables.d sudo tee /etc/nftables.d/gntech-host.nft \u0026gt;/dev/null \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; table inet gntech_host { chain input { type filter hook input priority 0; policy drop; ct state invalid drop ct state established,related accept iifname \u0026#34;lo\u0026#34; accept ip protocol icmp accept meta l4proto ipv6-icmp accept ip saddr 10.0.99.0/24 tcp dport 22 ct state new accept tcp dport { 80, 443 } ct state new accept limit rate 5/second burst 10 packets log prefix \u0026#34;GNTECH-INPUT-DROP: \u0026#34; counter drop } } EOF sudo nft -c -f /etc/nftables.d/gntech-host.nft The -c check validates the proposed rules without applying them. It does not prove that your SSH client is inside the permitted subnet or that all required services are allowed.\nICMPv6 supports essential functions such as neighbor discovery and path-MTU discovery. Blocking it indiscriminately can break IPv6 even when TCP port rules look correct.\n3. Load Once and Verify Only load this file if there is no existing table named gntech_host:\n1 sudo nft list table inet gntech_host If the table does not exist and the syntax check passed:\n1 2 sudo nft -f /etc/nftables.d/gntech-host.nft sudo nft -a list table inet gntech_host Keep the original administration session open. Start a new SSH connection from the management subnet, test required application ports, and test a blocked port from another client. An existing SSH session can survive through the established-connection rule while new logins fail.\nInspect counters and logs:\n1 2 sudo nft list table inet gntech_host sudo journalctl -k -g GNTECH-INPUT-DROP -n 50 --no-pager The logging limit is global to this rule. It is not a per-source SSH rate limiter or a dynamic ban list.\n4. Integrate Persistence Deliberately The example is a runtime policy until your existing firewall startup process loads it. Integrate the file with that process and test a reboot from a console-accessible maintenance window. Repeatedly loading an additive rules file may duplicate rules, so review how your manager performs replacement.\nDo not overwrite /etc/nftables.conf or enable an unfamiliar default file that contains flush ruleset on a host where Docker or another service owns tables. nft -f does not inherently protect other tables: the commands inside the file determine what is replaced or flushed.\nDocker Is a Separate Forwarding Path Published ports on bridge-network containers generally traverse forwarding/NAT, not the host-input chain above. This policy therefore does not provide a container-port allowlist.\nKeep Docker\u0026rsquo;s firewall-rule creation enabled unless you deliberately implement all required NAT, isolation, and forwarding behavior yourself. A few docker0 accept rules do not replace that behavior, and Compose networks normally use additional bridges.\nUse the backend-specific procedure:\nDocker with iptables and UFW Docker\u0026rsquo;s native nftables backend For native nftables, place custom restrictions in your own table and account for base-chain priorities. An accept verdict in one base chain does not bypass a drop in another.\nRollback Remove only this guide\u0026rsquo;s table:\n1 sudo nft delete table inet gntech_host Also remove its startup inclusion if you configured one. Other firewall tables remain intact. Recheck the host\u0026rsquo;s resulting exposure; removing this table removes its restrictions.\nReferences nftables: configuring chains and base-chain priorities Docker: packet filtering and firewalls Docker: native nftables backend ","permalink":"https://blog.gntech.me/posts/2026-05-27-nftables-linux-firewall-homelab/","summary":"Create and verify a separate nftables host-input policy, preserve service-owned tables, and understand why Docker published ports need forwarding rules.","title":"nftables Host Firewall for a Homelab — Add Rules Without Flushing Docker"},{"content":"Ansible makes repeated configuration changes easier to describe and review. A useful first project manages one harmless file, rather than changing SSH, firewall rules, package repositories and Docker storage in a single run.\nThis guide consolidates the two earlier Ansible introductions. It replaces their overlapping bootstrap examples with a complete small exercise. It does not claim to restore an entire lab from Git or provide a tested production role collection.\nVerification status Scope Evidence and limits Environment Linux control node with a supported Ansible installation; a disposable Linux target for the optional remote step. Preparation Bash syntax and generated YAML structure can be checked independently. Runtime The playbook has not been run against a host during this update. Before relying on it Validate with the installed Ansible version, run on a test target, inspect the result and repeat the run. Install and record your tools Use the official installation guide for your control node. Distribution packages and isolated Python environments have different release policies; an apt install is not automatically the latest upstream release or a PPA installation.\n1 2 ansible --version ansible-playbook --version The example uses built-in modules. Remote Linux targets normally need SSH access and a compatible Python interpreter. RouterOS devices require their own supported connection and collection setup; do not put them in a Linux bootstrap play without separating those requirements.\nCreate a small project Use a new directory and run these commands as your normal user, not root:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 mkdir -p \u0026#34;$HOME/ansible-first-lab/playbooks\u0026#34; cd \u0026#34;$HOME/ansible-first-lab\u0026#34; cat \u0026gt; ansible.cfg \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; [defaults] inventory = ./inventory.yml host_key_checking = True EOF cat \u0026gt; inventory.yml \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; all: children: control_local: hosts: localhost: ansible_connection: local linux_lab: hosts: {} EOF cat \u0026gt; playbooks/demo.yml \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; --- - name: Manage a demonstration file hosts: \u0026#34;{{ target_group | default(\u0026#39;control_local\u0026#39;) }}\u0026#34; gather_facts: true become: false vars: demo_directory: \u0026#34;{{ ansible_env.HOME }}/ansible-first-lab-output\u0026#34; tasks: - name: Create demonstration directory ansible.builtin.file: path: \u0026#34;{{ demo_directory }}\u0026#34; state: directory mode: \u0026#39;0750\u0026#39; - name: Write demonstration file ansible.builtin.copy: dest: \u0026#34;{{ demo_directory }}/managed.txt\u0026#34; content: \u0026#34;Managed by Ansible for {{ inventory_hostname }}\\n\u0026#34; mode: \u0026#39;0640\u0026#39; EOF This writes a dedicated demonstration directory under the execution account\u0026rsquo;s home. It does not create users, change authentication, enable a firewall or restart services. Check that the output path is unused before applying it.\nInspect the targets and validate syntax 1 2 3 4 ansible-inventory --graph ansible-playbook playbooks/demo.yml --list-hosts ansible-playbook playbooks/demo.yml --syntax-check ansible control_local -m ansible.builtin.ping The default play should select only localhost. Ansible\u0026rsquo;s ping module tests module execution; it is not an ICMP network ping.\nPreview, apply, then repeat 1 2 3 4 ansible-playbook playbooks/demo.yml --check --diff ansible-playbook playbooks/demo.yml cat \u0026#34;$HOME/ansible-first-lab-output/managed.txt\u0026#34; ansible-playbook playbooks/demo.yml Check mode is a simulation, not proof that the apply will succeed. In this example the copy task may fail in the initial preview because the directory task only simulates creation. Inspect that dependency rather than treating it as permission to skip validation.\nSome modules do not support check mode, and tasks explicitly configured with check_mode: false can still run normally. Diff output can reveal file contents, including secrets. Review the check and diff documentation for the modules you use.\nAfter a real successful apply, an unchanged second run of this example should report no changes. Inspect the recap and file; do not assume every playbook is idempotent simply because it uses Ansible. A later preview should also be easier to interpret now that the directory exists.\nMove to one disposable remote host Establish working SSH access first and verify the host\u0026rsquo;s fingerprint through a trusted path. Keep host-key checking enabled. Do not treat an unverified key scan as proof of identity.\nReplace the empty lab group with one test host. This example address is documentation-only; replace it and the username before connecting:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 cp -a inventory.yml inventory.yml.before-remote cat \u0026gt; inventory.yml \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; all: children: control_local: hosts: localhost: ansible_connection: local linux_lab: hosts: lab01: ansible_host: 192.0.2.10 ansible_user: YOUR_LINUX_USER EOF ansible-playbook playbooks/demo.yml -e target_group=linux_lab --limit lab01 --list-hosts ansible linux_lab --limit lab01 -m ansible.builtin.ping ansible-playbook playbooks/demo.yml -e target_group=linux_lab --limit lab01 --check --diff Only after reviewing those results, apply to that host and repeat:\n1 2 ansible-playbook playbooks/demo.yml -e target_group=linux_lab --limit lab01 ansible-playbook playbooks/demo.yml -e target_group=linux_lab --limit lab01 Inspect the output file on the remote account, not on the control node. Keep the --limit until you deliberately expand the inventory. Privilege escalation and its credentials are unnecessary for this user-owned-file exercise.\nExpand with roles and collections Once the small workflow works, split configuration into reusable roles and keep a play\u0026rsquo;s hosts scoped to the appropriate group. An imported playbook must exist; referencing an unspecified monitoring role does not create it.\nRecord collection versions in a requirements file and check their current module documentation. Docker deployment requires an installed compatible engine and the collection\u0026rsquo;s prerequisites. Proxmox API provisioning is separate from running Linux modules over SSH on a guest. RouterOS command modules can use a different transport from API modules.\nIntroduce one service at a time. Validate templates before restarting services, keep handlers scoped, and back up existing configuration. Avoid combining daemon data-root changes, package removal and a service restart into an untested generic Docker role.\nHandle secrets deliberately Ansible Vault encrypts stored data, not every place it might appear during execution. Prefer an interactive vault password or an approved password-manager integration, rather than putting a literal password in shell history or committing a password file.\nLoad encrypted variables through an explicit file or a correctly placed inventory variable directory. A filename containing vault does not alone guarantee that Ansible loads it. Tasks handling credentials may need no_log: true and diff: false; avoid debugging those values into logs.\nCreate ignore rules before the first Git commit. Review staged changes for credentials and generated output. Git preserves configuration history; it does not replace backups of databases, volumes, keys or other application data.\nRoll back the demonstration The local exercise creates only the dedicated directory and file. Inspect them before removal:\n1 2 3 ls -la \u0026#34;$HOME/ansible-first-lab-output\u0026#34; rm -i \u0026#34;$HOME/ansible-first-lab-output/managed.txt\u0026#34; rmdir \u0026#34;$HOME/ansible-first-lab-output\u0026#34; Run those commands on the test target if you applied the remote exercise. rmdir refuses to remove a nonempty directory; investigate additional files instead of switching to recursive deletion. Restore the saved inventory if you want to return to the local-only setup.\nContinue reading Docker Compose patterns — understand the service definitions before automating deployments. Monitoring foundations — verify a small application stack before turning it into a role. Ansible Vault documentation — review encryption and secret-handling behavior. This consolidation removes the older disabled-host-verification recommendation, missing-role bootstrap examples and fixed completion-time claims. No remote execution or application-level syntax validation is claimed for this update; use the validation steps above with the versions you install.\n","permalink":"https://blog.gntech.me/posts/2026-05-27-ansible-homelab-automation/","summary":"A staged Ansible introduction with a complete file-management example, explicit target selection, check-mode limits, secrets guidance and rollback.","title":"Ansible for a Homelab — Inventory, First Playbook, and Safe Expansion"},{"content":"You have five services running in Docker. Some are critical — your DNS resolver, your reverse proxy, your database. If one of them crashes or a disk fills up, how do you know?\nSSHing into every host and running htop does not scale past one machine. You need a monitoring stack that collects metrics from every host and service, visualizes them in dashboards, and alerts you when something needs attention.\nPrometheus scrapes metrics on a schedule. Grafana turns those metrics into dashboards. Node Exporter exposes Linux kernel and hardware counters. cAdvisor exposes Docker container resource usage. Alertmanager routes notifications to email, Slack, or Telegram. Together they form the standard observability stack for Linux infrastructure.\nThis guide deploys the entire stack with a single Docker Compose file, configures it for a multi-host homelab, and walks through the first dashboards and alerts.\nArchitecture Overview The stack has four components, all deployed via Docker Compose on one \u0026ldquo;monitoring\u0026rdquo; host:\n┌─────────────────────────────────────┐ │ Docker Network │ │ monitoring-net │ │ │ │ ┌──────────┐ ┌───────────┐ │ │ │ Grafana │──│ Prometheus │ │ │ │ :3000 │ │ :9090 │ │ │ └────┬─────┘ └─────┬─────┘ │ │ │ │ │ │ │ ┌──────────┴──────────┐ │ │ └───┤ Alertmanager :9093 │ │ │ └──────────┬──────────┘ │ │ │ │ │ ┌──────────┴──────────┐ │ │ │ Notification │ │ │ │ (Email/Slack/TG) │ │ │ └─────────────────────┘ │ └─────────────────────────────────────┘ ▲ ▲ │ scrape │ scrape │ :9100 │ :8080 ┌────┴─────┐ ┌────┴─────┐ │Node │ │ cAdvisor │ │Exporter │ │ :8080 │ │(each host)│ │(per host)│ └──────────┘ └──────────┘ Prometheus is the central collector. It pulls metrics from each Node Exporter and cAdvisor instance on a configurable interval (default 15 seconds). Grafana queries Prometheus to render dashboards. Alertmanager evaluates alert rules and fires notifications.\nStep 1 — Create the Directory Structure 1 2 3 mkdir -p /opt/monitoring/{prometheus,alertmanager,grafana} mkdir -p /opt/monitoring/prometheus/data mkdir -p /opt/monitoring/grafana/data Set ownership so Grafana and Prometheus can write their data:\n1 2 chown 472:472 /opt/monitoring/grafana/data chown 65534:65534 /opt/monitoring/prometheus/data The UIDs are the default user inside each container (grafana=472, prometheus=nobody=65534). Mismatch these and the containers will crash on startup with permission errors.\nStep 2 — Prometheus Configuration Write /opt/monitoring/prometheus/prometheus.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 global: scrape_interval: 15s evaluation_interval: 15s scrape_timeout: 10s rule_files: - \u0026#34;alerts.yml\u0026#34; alerting: alertmanagers: - static_configs: - targets: - alertmanager:9093 scrape_configs: # Prometheus self-metrics - job_name: prometheus static_configs: - targets: [\u0026#34;localhost:9090\u0026#34;] # Node Exporter — one target per host - job_name: node static_configs: - targets: - \u0026#34;10.0.20.30:9100\u0026#34; # SRV1 — Proxmox host - \u0026#34;10.0.20.31:9100\u0026#34; # SRV2 — backup node - \u0026#34;10.0.20.10:9100\u0026#34; # GATEWAY — OpenClaw gateway relabel_configs: - source_labels: [\u0026#34;__address__\u0026#34;] regex: \u0026#34;([^:]+):.+\u0026#34; target_label: \u0026#34;instance\u0026#34; # cAdvisor — one target per host - job_name: cadvisor static_configs: - targets: - \u0026#34;10.0.20.30:8080\u0026#34; # SRV1 - \u0026#34;10.0.20.31:8080\u0026#34; # SRV2 relabel_configs: - source_labels: [\u0026#34;__address__\u0026#34;] regex: \u0026#34;([^:]+):.+\u0026#34; target_label: \u0026#34;instance\u0026#34; Alert Rules Write /opt/monitoring/prometheus/alerts.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 groups: - name: homelab-alerts rules: - alert: NodeDown expr: up{job=\u0026#34;node\u0026#34;} == 0 for: 2m labels: severity: critical annotations: summary: \u0026#34;Node {{ $labels.instance }} is down\u0026#34; description: \u0026#34;{{ $labels.instance }} has been unreachable for more than 2 minutes.\u0026#34; - alert: DiskSpaceCritical expr: (1 - (node_filesystem_free_bytes{fstype!=\u0026#34;tmpfs\u0026#34;,mountpoint!=\u0026#34;/boot\u0026#34;} / node_filesystem_size_bytes{fstype!=\u0026#34;tmpfs\u0026#34;,mountpoint!=\u0026#34;/boot\u0026#34;})) * 100 \u0026gt; 90 for: 5m labels: severity: critical annotations: summary: \u0026#34;Disk space \u0026gt;90% on {{ $labels.instance }} - {{ $labels.mountpoint }}\u0026#34; description: \u0026#34;Disk at {{ $labels.mountpoint }} on {{ $labels.instance }} is {{ $value | humanizePercentage }} full.\u0026#34; - alert: DiskSpaceWarning expr: (1 - (node_filesystem_free_bytes{fstype!=\u0026#34;tmpfs\u0026#34;,mountpoint!=\u0026#34;/boot\u0026#34;} / node_filesystem_size_bytes{fstype!=\u0026#34;tmpfs\u0026#34;,mountpoint!=\u0026#34;/boot\u0026#34;})) * 100 \u0026gt; 80 for: 10m labels: severity: warning annotations: summary: \u0026#34;Disk space \u0026gt;80% on {{ $labels.instance }} - {{ $labels.mountpoint }}\u0026#34; - alert: HighCpuLoad expr: 100 - (avg by(instance) (rate(node_cpu_seconds_total{mode=\u0026#34;idle\u0026#34;}[2m])) * 100) \u0026gt; 85 for: 10m labels: severity: warning annotations: summary: \u0026#34;CPU \u0026gt;85% on {{ $labels.instance }}\u0026#34; description: \u0026#34;CPU usage on {{ $labels.instance }} is at {{ $value | humanizePercentage }}.\u0026#34; - alert: HighMemoryUsage expr: (1 - (node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes)) * 100 \u0026gt; 90 for: 5m labels: severity: warning annotations: summary: \u0026#34;Memory \u0026gt;90% on {{ $labels.instance }}\u0026#34; description: \u0026#34;Memory usage on {{ $labels.instance }} is at {{ $value | humanizePercentage }}.\u0026#34; - alert: DockerContainerDown expr: time() - container_last_seen \u0026gt; 60 for: 1m labels: severity: critical annotations: summary: \u0026#34;Container {{ $labels.name }} is down\u0026#34; description: \u0026#34;Container {{ $labels.name }} has not reported metrics for over 60 seconds.\u0026#34; Step 3 — Alertmanager Configuration Write /opt/monitoring/alertmanager/alertmanager.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 route: group_by: [\u0026#34;alertname\u0026#34;, \u0026#34;instance\u0026#34;] group_wait: 30s group_interval: 5m repeat_interval: 4h receiver: \u0026#34;default\u0026#34; receivers: - name: \u0026#34;default\u0026#34; email_configs: - to: \u0026#34;admin@gntech.dev\u0026#34; from: \u0026#34;alerts@gntech.dev\u0026#34; smarthost: \u0026#34;smtp.gmail.com:587\u0026#34; auth_username: \u0026#34;alerts@gntech.dev\u0026#34; auth_password: $SMTP_PASSWORD require_tls: true webhook_configs: - url: \u0026#34;https://hooks.slack.com/services/T00/B00/xxxxxxxxx\u0026#34; send_resolved: true For Telegram notifications, add a webhook receiver:\n1 2 3 webhook_configs: - url: \u0026#34;http://telegram-webhook:8080/alert\u0026#34; send_resolved: true Use environment variables for secrets. The Compose file passes $SMTP_PASSWORD into Alertmanager. Never hardcode tokens in config files committed to git.\nStep 4 — Docker Compose Configuration Write /opt/monitoring/docker-compose.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 services: prometheus: image: prom/prometheus:latest container_name: prometheus hostname: prometheus restart: unless-stopped volumes: - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro - ./prometheus/alerts.yml:/etc/prometheus/alerts.yml:ro - ./prometheus/data:/prometheus command: - \u0026#34;--config.file=/etc/prometheus/prometheus.yml\u0026#34; - \u0026#34;--storage.tsdb.path=/prometheus\u0026#34; - \u0026#34;--storage.tsdb.retention.time=30d\u0026#34; - \u0026#34;--web.console.libraries=/usr/share/prometheus/console_libraries\u0026#34; - \u0026#34;--web.console.templates=/usr/share/prometheus/consoles\u0026#34; - \u0026#34;--web.enable-lifecycle\u0026#34; ports: - \u0026#34;9090:9090\u0026#34; networks: - monitoring-net alertmanager: image: prom/alertmanager:latest container_name: alertmanager hostname: alertmanager restart: unless-stopped volumes: - ./alertmanager/alertmanager.yml:/etc/alertmanager/alertmanager.yml:ro command: - \u0026#34;--config.file=/etc/alertmanager/alertmanager.yml\u0026#34; - \u0026#34;--storage.path=/alertmanager\u0026#34; - \u0026#34;--web.listen-address=:9093\u0026#34; ports: - \u0026#34;9093:9093\u0026#34; environment: - SMTP_PASSWORD=${SMTP_PASSWORD} networks: - monitoring-net grafana: image: grafana/grafana:latest container_name: grafana hostname: grafana restart: unless-stopped volumes: - ./grafana/data:/var/lib/grafana - ./grafana/provisioning:/etc/grafana/provisioning environment: - GF_SECURITY_ADMIN_USER=${GRAFANA_USER:-admin} - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD:-admin} - GF_USERS_ALLOW_SIGN_UP=false - GF_INSTALL_PLUGINS=grafana-piechart-panel - GF_SERVER_ROOT_URL=https://grafana.gntech.dev ports: - \u0026#34;3000:3000\u0026#34; depends_on: - prometheus networks: - monitoring-net networks: monitoring-net: name: monitoring-net driver: bridge Data Retention The --storage.tsdb.retention.time=30d flag keeps 30 days of metrics. For a homelab collecting ~1000 series per host, this uses roughly 500 MB–1 GB of disk per month. Adjust to 7d or 90d based on your storage budget.\nStep 5 — Deploy Node Exporter and cAdvisor on Each Host You need Node Exporter and cAdvisor running on every machine you want to monitor. Node Exporter provides host-level metrics (CPU, RAM, disk, network). cAdvisor provides per-container resource usage.\nNode Exporter (systemd service) Create /etc/systemd/system/node-exporter.service:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 [Unit] Description=Prometheus Node Exporter After=network.target [Service] Type=simple User=nobody Group=nogroup ExecStart=/usr/local/bin/node_exporter \\ --web.listen-address=:9100 \\ --path.procfs=/proc \\ --path.sysfs=/sys \\ --collector.filesystem.mount-points-exclude=\u0026#34;^/(sys|proc|dev|host|etc)($|/)\u0026#34; Restart=always [Install] WantedBy=multi-user.target Install the binary:\n1 2 3 4 5 6 7 wget -O /tmp/node_exporter.tar.gz \\ https://github.com/prometheus/node_exporter/releases/download/v1.10.0/node_exporter-1.10.0.linux-amd64.tar.gz tar xzf /tmp/node_exporter.tar.gz -C /tmp/ sudo mv /tmp/node_exporter-1.10.0.linux-amd64/node_exporter /usr/local/bin/ rm -rf /tmp/node_exporter* sudo systemctl daemon-reload sudo systemctl enable --now node-exporter Verify it is collecting metrics:\n1 curl -s http://localhost:9100/metrics | head -10 cAdvisor (Docker container) 1 2 3 4 5 6 7 8 9 docker run -d --name=cadvisor --restart unless-stopped \\ --privileged \\ --device=/dev/kmsg \\ -p 8080:8080 \\ -v /:/rootfs:ro \\ -v /var/run:/var/run:ro \\ -v /sys:/sys:ro \\ -v /var/lib/docker/:/var/lib/docker:ro \\ gcr.io/cadvisor/cadvisor:latest cAdvisor requires --privileged to access kernel cgroup metrics. On Proxmox LXC containers, you may need lxc.cgroup2: \u0026quot;\u0026quot; in the container config for cAdvisor to work properly.\nFirewall Rules If you use UFW or nftables, open ports 9100 and 8080 to your monitoring host only:\n1 2 3 # UFW — allow from monitoring host only ufw allow from 10.0.20.10 to any port 9100 proto tcp ufw allow from 10.0.20.10 to any port 8080 proto tcp Never expose Node Exporter or cAdvisor ports to the internet. They serve unauthenticated metrics endpoints that leak system information.\nStep 6 — Start the Stack 1 2 cd /opt/monitoring docker compose up -d Check that everything is running:\n1 2 3 4 5 docker compose ps # NAME IMAGE STATUS # alertmanager prom/alertmanager:latest Up 2 minutes # grafana grafana/grafana:latest Up 2 minutes # prometheus prom/prometheus:latest Up 2 minutes Check Prometheus targets:\n1 2 3 # List all configured scrape targets curl -s http://localhost:9090/api/v1/targets | \\ jq \u0026#39;.data.activeTargets[] | {job: .labels.job, instance: .labels.instance, health: .health}\u0026#39; All targets should show \u0026quot;health\u0026quot;: \u0026quot;up\u0026quot;. If any show \u0026quot;down\u0026quot;, check firewall rules and verify that Node Exporter / cAdvisor are running on the target host.\nStep 7 — Import Grafana Dashboards Log into Grafana at http://your-host:3000 (default credentials: admin / admin — change immediately on first login).\nAdd Prometheus as a Data Source Configuration → Data Sources → Add data source Select Prometheus Set URL to http://prometheus:9090 (Docker internal DNS) Click Save \u0026amp; Test Import the Node Exporter Full Dashboard The best dashboard for host metrics is Node Exporter Full (dashboard ID: 1860):\nDashboards → Import Enter dashboard ID 1860 Select the Prometheus data source Click Import Import the Docker Monitoring Dashboard For container metrics, use the Docker Monitoring dashboard (ID: 193):\nDashboards → Import Enter dashboard ID 193 Select the Prometheus data source Click Import Create a Simple CPU Dashboard (Manual) If you prefer a minimal dashboard over importing large ones:\nDashboards → New Dashboard → Add visualization Select Prometheus as data source Enter this PromQL query for CPU: 1 100 - (avg by(instance) (rate(node_cpu_seconds_total{mode=\u0026#34;idle\u0026#34;}[5m])) * 100) Set title to \u0026ldquo;CPU Usage by Host\u0026rdquo; Add another panel with this memory query: 1 (1 - (node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes)) * 100 Add a disk panel: 1 (1 - (node_filesystem_free_bytes{mountpoint=\u0026#34;/\u0026#34;} / node_filesystem_size_bytes{mountpoint=\u0026#34;/\u0026#34;})) * 100 Save the dashboard Key PromQL Queries for Homelab Monitoring What to measure PromQL query CPU per host (%) 100 - (avg by(instance) (rate(node_cpu_seconds_total{mode=\u0026quot;idle\u0026quot;}[5m])) * 100) Memory used (%) (1 - (node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes)) * 100 Disk used per mount (%) (1 - (node_filesystem_free_bytes / node_filesystem_size_bytes)) * 100 Disk IOPS (reads) rate(node_disk_reads_completed_total[5m]) Disk IOPS (writes) rate(node_disk_writes_completed_total[5m]) Network throughput (rx) rate(node_network_receive_bytes_total{device!=\u0026quot;lo\u0026quot;}[5m]) Network throughput (tx) rate(node_network_transmit_bytes_total{device!=\u0026quot;lo\u0026quot;}[5m]) Container CPU per name sum by(name) (rate(container_cpu_usage_seconds_total{name!=\u0026quot;\u0026quot;}[5m])) Container memory per name container_memory_usage_bytes{name!=\u0026quot;\u0026quot;} Uptime (days) (time() - node_boot_time_seconds) / 86400 Load average (1m) node_load1 Step 8 — Test an Alert Trigger a test alert to verify Alertmanager is working:\n1 2 3 4 5 6 7 # Send a test message to Alertmanager curl -X POST -H \u0026#34;Content-Type: application/json\u0026#34; \\ -d \u0026#39;{ \u0026#34;labels\u0026#34;: {\u0026#34;alertname\u0026#34;: \u0026#34;TestAlert\u0026#34;, \u0026#34;severity\u0026#34;: \u0026#34;critical\u0026#34;, \u0026#34;instance\u0026#34;: \u0026#34;test-host\u0026#34;}, \u0026#34;annotations\u0026#34;: {\u0026#34;summary\u0026#34;: \u0026#34;This is a test alert\u0026#34;, \u0026#34;description\u0026#34;: \u0026#34;Homelab monitoring stack is configured correctly.\u0026#34;} }\u0026#39; \\ http://localhost:9093/api/v1/alerts Check Alertmanager status:\n1 curl -s http://localhost:9093/api/v2/alerts | jq \u0026#39;.[].labels\u0026#39; If email or Slack webhook is configured, you should receive the test notification within a few seconds.\nStep 9 — Long-Term Data and Backup Prometheus stores metrics as TSDB blocks on disk. After 30 days (default from this guide), old blocks are deleted. If you want longer retention, either increase the retention period or add remote write to VictoriaMetrics or Thanos.\nBack up Grafana Grafana dashboards and data sources live in the SQLite database at /opt/monitoring/grafana/data/grafana.db. Back it up daily:\n1 2 3 # Simple backup script cp /opt/monitoring/grafana/data/grafana.db \\ /opt/monitoring/grafana/data/grafana.db.$(date +%Y%m%d) Better approach: use Grafana\u0026rsquo;s Provisioning API to define dashboards as YAML files in ./grafana/provisioning/dashboards/. This way, dashboards are version-controlled and survive container recreations.\nGrafana Provisioning Example Create /opt/monitoring/grafana/provisioning/datasources/prometheus.yml:\n1 2 3 4 5 6 7 8 apiVersion: 1 datasources: - name: Prometheus type: prometheus access: proxy url: http://prometheus:9090 isDefault: true Create /opt/monitoring/grafana/provisioning/dashboards/dashboard.yml:\n1 2 3 4 5 6 7 8 9 10 11 apiVersion: 1 providers: - name: \u0026#34;default\u0026#34; orgId: 1 folder: \u0026#34;\u0026#34; type: file disableDeletion: false editable: true options: path: /var/lib/grafana/dashboards Drop exported JSON dashboard files into ./grafana/dashboards/ and they will load automatically on container restart.\nStep 10 — Adding More Hosts Adding a new host to the monitoring stack takes two minutes:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 # On the new host: # 1. Install and start Node Exporter sudo systemctl enable --now node-exporter # 2. Start cAdvisor (run as docker container) docker run -d --name=cadvisor --restart unless-stopped \\ --privileged -p 8080:8080 \\ -v /:/rootfs:ro -v /var/run:/var/run:ro \\ -v /sys:/sys:ro -v /var/lib/docker/:/var/lib/docker:ro \\ gcr.io/cadvisor/cadvisor:latest # 3. On the monitoring host, add to prometheus.yml # under node job and cadvisor job targets: # - \u0026#34;10.0.20.50:9100\u0026#34; # - \u0026#34;10.0.20.50:8080\u0026#34; # 4. Reload Prometheus without restart curl -X POST http://localhost:9090/-/reload The --web.enable-lifecycle flag on Prometheus enables the /-/reload endpoint without a full restart. Use it whenever you add or remove scrape targets.\nSecurity Considerations Never expose ports 9090 (Prometheus), 9093 (Alertmanager), or 3000 (Grafana) to the internet. These are management interfaces. Route Grafana through your reverse proxy with HTTPS and authentication. Node Exporter and cAdvisor serve unauthenticated metrics. Firewall them to your monitoring subnet only. A reverse proxy is not enough — these should not be accessible from outside your LAN at all. Change Grafana admin password on first login. In the Compose file, use $GRAFANA_PASSWORD from an .env file instead of the default. Alertmanager SMTP password should be in an .env file, not committed to git. Create /opt/monitoring/.env:\n1 2 3 GRAFANA_USER=admin GRAFANA_PASSWORD=your-strong-password-here SMTP_PASSWORD=your-gmail-app-password Add .env to .gitignore if you version-control this directory.\nTroubleshooting Prometheus targets show \u0026ldquo;down\u0026rdquo; 1 2 3 4 5 6 7 8 # Check if Node Exporter is listening nc -zv 10.0.20.30 9100 # Check if cAdvisor responds curl -s http://10.0.20.30:8080/metrics | head # Check Prometheus logs docker logs prometheus --tail 50 Grafana says \u0026ldquo;No data\u0026rdquo; 1 2 3 4 5 # Verify Prometheus has metrics for the time range curl -s \u0026#39;http://localhost:9090/api/v1/query?query=up\u0026#39; # Check Grafana data source is pointing to http://prometheus:9090 # (Docker internal DNS, not localhost) Container metrics missing in cAdvisor On Proxmox LXC containers, cAdvisor needs cgroup v2 access:\n1 2 # In LXC config (/etc/pve/lxc/\u0026#34;\u0026lt;CT_ID\u0026gt;\u0026#34;.conf), add: lxc.cgroup2: \u0026#34;\u0026#34; Then restart the container.\nPrometheus storage growing too fast Reduce retention or limit scraped metrics:\n1 2 3 4 5 6 7 8 9 # In prometheus.yml, add metric_relabel_configs to drop # high-cardinality labels you don\u0026#39;t need: scrape_configs: - job_name: node metric_relabel_configs: - source_labels: [__name__] regex: \u0026#34;(node_cpu_.*|node_memory_.*|node_filesystem_.*)\u0026#34; action: keep This keeps only CPU, memory, and filesystem metrics from Node Exporter, dropping disk scheduler stats, network hardware info, and other rarely-used metrics.\nSummary The Prometheus + Grafana stack gives you full observability into every host and container in your homelab. With today\u0026rsquo;s setup you get:\nHost metrics — CPU, RAM, disk, network, and load from every machine via Node Exporter Container metrics — per-container CPU, memory, network, and filesystem via cAdvisor Dashboards — pre-built Node Exporter and Docker monitoring dashboards in Grafana Alerts — disk space, CPU, memory, and container-down alerts routed to email or Slack via Alertmanager Easy expansion — add hosts by deploying Node Exporter + cAdvisor and adding one line to prometheus.yml Start with the single-host Compose stack, add Node Exporters to your other machines one at a time, and within an afternoon you will have full visibility into your entire lab. When a disk fills up at 3 AM, your phone will buzz instead of a service crashing silently.\n","permalink":"https://blog.gntech.me/posts/2026-05-26-prometheus-grafana-homelab-monitoring/","summary":"A practical guide to deploying a full Prometheus and Grafana monitoring stack in your homelab with Docker Compose — collect CPU, RAM, disk, network, and Docker container metrics from every host, visualize with pre-built dashboards, and get alerts when things go wrong.","title":"Prometheus and Grafana — Homelab Monitoring Stack with Docker Compose"},{"content":"MikroTik RouterOS 7 is the most capable routing platform at its price point, but it\u0026rsquo;s also one of the most complex to configure correctly. A few wrong firewall rules can lock you out. A missing FastTrack exception can tank your throughput. An exposed admin interface on the WAN side is an invitation for scans and exploits.\nThis guide is a distilled set of best practices gathered from running RouterOS 7 in production homelab environments. It covers security hardening, firewall architecture, VLAN design with bridge filtering, FastTrack performance tuning, DNS over HTTPS setup, container management, and backup automation — all with concrete commands that work on RouterOS 7.15 and later.\nThe examples require validation on the actual RouterOS release and hardware. This source review does not establish deployment evidence for CCR, RB5009, hAP ax or CHR. Adjust interface names and IP ranges to match your network.\n1. Initial Security Hardening Before configuring anything else, lock down the router itself. These steps disable the default attack surface that makes unconfigured MikroTiks a target.\nChange the Default Admin User The first thing every automated scanner tries: admin with a blank or weak password.\n1 2 3 4 5 6 7 8 9 10 # Create a new admin user with a strong name /user add name=netadmin group=full \\ password=\u0026#34;!={zQ8mN4fL2xPv!}\u0026#34; # Disable the default admin user (don\u0026#39;t delete it — keep for # emergency recovery via reset, but disable it) /user disable admin # Verify /user print where disabled=no Why disable instead of delete: RouterOS keeps the admin user in the system user database. If you ever need to factory reset and restore from backup, an existing admin user in the config can interact with the default admin account in unexpected ways. Disabling it avoids that edge case.\nRestrict Management Access By default, RouterOS listens on all interfaces for WinBox, SSH, API, and web access. Restrict these to your management VLAN or a specific trusted IP range.\n# Create a management address list /ip firewall address-list add list=mgmt_trusted address=10.0.20.0/24 comment=\u0026#34;LAN mgmt subnet\u0026#34; add list=mgmt_trusted address=10.0.30.0/24 comment=\u0026#34;VPN pool\u0026#34; # Set allowed interface lists for all services /ip service set winbox address=10.0.20.0/24 /ip service set ssh address=10.0.20.0/24 /ip service set www address=10.0.20.0/24 /ip service set www-ssl address=10.0.20.0/24 /ip service set api address=none /ip service set api-ssl address=none /ip service set telnet address=none /ip service set ftp address=none # Enable strong SSH crypto /ip ssh set strong-crypto=yes forwarding-enabled=no # Disable MAC-telnet, MAC-WinBox, and neighbor discovery /tool mac-server set allowed-interface-list=none /tool mac-server mac-winbox set allowed-interface-list=none /tool mac-server ping set enabled=no /ip neighbor discovery-settings set discover-interface-list=none # Disable bandwidth server, proxy, socks, UPnP /tool bandwidth-server set enabled=no /ip proxy set enabled=no /ip socks set enabled=no /ip upnp set enabled=no # Disable MikroTik cloud (DDNS and remote access) /ip cloud set ddns-enabled=no update-time=no Update Firmware RouterOS releases frequently patch security vulnerabilities. Pin a stable release channel and check for updates monthly:\n# Set update channel to stable (not release-candidate) /system package update set channel=stable # Check for updates /system package update check-for-updates # Install and reboot /system package update install /system reboot Make this a scheduled task:\n/system scheduler add name=check-updates interval=7d start-time=03:00 \\ on-event=\u0026#34;/system package update check-for-updates\u0026#34; \\ comment=\u0026#34;Weekly update check\u0026#34; 2. Firewall Architecture RouterOS 7\u0026rsquo;s firewall is stateful with connection tracking. The classic three-chain architecture — input, forward, output — handles every packet path. Good rule ordering is critical because RouterOS evaluates rules top-down and stops on the first match (for filter rules with action=accept or action=drop).\nThe Default Drop Pattern A production firewall starts with a single default-drop rule at the bottom of each chain, then opens specific holes above it.\n# Set default policies /ip firewall filter set [find where chain=input] action=drop set [find where chain=forward] action=drop set [find where chain=output] action=accept Then build rules above the default drop. The order matters:\n/ip firewall filter # Layer 1: Connection state — accept established/related first add chain=input connection-state=established,related action=accept \\ comment=\u0026#34;Accept established/related input\u0026#34; add chain=forward connection-state=established,related action=accept \\ comment=\u0026#34;Accept established/related forward\u0026#34; # Layer 2: ICMP — rate-limited ping, accept everywhere add chain=input protocol=icmp action=accept \\ comment=\u0026#34;Allow ICMP input\u0026#34; add chain=forward protocol=icmp action=accept \\ comment=\u0026#34;Allow ICMP forward\u0026#34; # Layer 3: Management access — restricted by address list add chain=input protocol=tcp dst-port=22 src-address-list=mgmt_trusted \\ action=accept comment=\u0026#34;SSH from mgmt subnets\u0026#34; add chain=input protocol=tcp dst-port=8291 src-address-list=mgmt_trusted \\ action=accept comment=\u0026#34;WinBox from mgmt subnets\u0026#34; add chain=input protocol=tcp dst-port=80,443 \\ src-address-list=mgmt_trusted action=accept \\ comment=\u0026#34;WebFig from mgmt subnets\u0026#34; # Layer 4: Forwarded traffic — permit inter-VLAN routing add chain=forward in-interface=bridge action=accept \\ comment=\u0026#34;Allow all bridge forwarding\u0026#34; # Layer 5: Drop invalid — catch packets outside connection state add chain=input connection-state=invalid action=drop \\ comment=\u0026#34;Drop invalid input\u0026#34; add chain=forward connection-state=invalid action=drop \\ comment=\u0026#34;Drop invalid forward\u0026#34; # Layer 6: Default drop is already set as chain default Connection Tracking Limits RouterOS 7 defaults to 655k connection tracking entries. For homelabs with many clients and P2P traffic, that can fill up. Monitor and tune:\n# View current tracking stats /ip firewall connection tracking print # Show connection count /ip firewall connection print count-only # Inspect connection tracking settings and resource use /ip firewall connection tracking print /system resource print Forward configured firewall log entries to a remote collector (this is not a connection-count threshold alarm):\n/system logging action add name=conn_warn target=remote remote=10.0.20.50 remote-port=514 syslog-facility=local0 /system logging add topics=firewall action=conn_warn FastTrack and Connection Tracking Exception FastTrack bypasses connection tracking for established UDP and TCP connections, dramatically improving throughput. On a CCR or RB5009, FastTrack can process 5-10 Gbps without breaking a sweat. But it has a critical limitation: FastTrack connections are invisible to the firewall, so NAT rules, address list matching, and layer-7 inspection do not apply.\nPlace FastTrack rules before your general forward rules:\n/ip firewall filter # Must come before other forward rules add chain=forward action=fasttrack-connection \\ connection-state=established,related \\ comment=\u0026#34;FastTrack established connections\u0026#34; add chain=forward action=accept \\ connection-state=established,related \\ comment=\u0026#34;Accept established/related (non-fast-tracked)\u0026#34; When to skip FastTrack:\nYou use queue tree or simple queues for bandwidth shaping You need per-connection logging or accounting You apply layer-7 protocol matching on forwarded traffic You use transparent proxy features In those cases, leave FastTrack disabled and rely on connection tracking. The throughput difference at \u0026lt;500 Mbps is negligible.\n3. Bridge VLAN Filtering RouterOS 7\u0026rsquo;s bridge VLAN filtering is the recommended way to segment your network. It works at the bridge level rather than with switch chip ACLs, giving consistent behavior across all hardware (x86, ARM, MIPS).\nVLAN Layout Design your VLANs around security zones:\nVLAN Network Purpose 10 10.0.10.0/24 Management (router/L2) 20 10.0.20.0/24 Homelab servers 30 10.0.30.0/24 IoT / untrusted devices 40 10.0.40.0/24 Guest WiFi (internet only) 50 10.0.50.0/24 Voice / SIP 99 10.0.99.0/24 Native/untrusted uplink Bridge VLAN Configuration # Create the bridge with VLAN filtering /interface bridge add name=bridge1 protocol-mode=rstp vlan-filtering=yes \\ igmp-snooping=yes fast-forward=yes # Add VLAN interfaces (one per VLAN, parent=bridge1) /interface vlan add name=vlan10 vlan-id=10 interface=bridge1 add name=vlan20 vlan-id=20 interface=bridge1 add name=vlan30 vlan-id=30 interface=bridge1 add name=vlan40 vlan-id=40 interface=bridge1 add name=vlan50 vlan-id=50 interface=bridge1 # Add physical ports to the bridge with PVID /interface bridge port add bridge=bridge1 interface=ether1 pvid=99 comment=\u0026#34;WAN uplink\u0026#34; add bridge=bridge1 interface=ether2 pvid=20 comment=\u0026#34;Homelab server\u0026#34; add bridge=bridge1 interface=ether3 pvid=30 comment=\u0026#34;IoT devices\u0026#34; add bridge=bridge1 interface=ether4 pvid=40 comment=\u0026#34;Guest WiFi AP\u0026#34; # Configure trunk port for APs or switches add bridge=bridge1 interface=ether5 pvid=20 \\ frame-types=admit-only-vlan-tagged comment=\u0026#34;Trunk to AP\u0026#34; # Define allowed VLANs on the bridge /interface bridge vlan add bridge=bridge1 tagged=bridge1,ether5 vlan-ids=10,20,30,40,50 add bridge=bridge1 tagged=bridge1 untagged=ether2 vlan-ids=20 add bridge=bridge1 tagged=bridge1 untagged=ether3 vlan-ids=30 add bridge=bridge1 tagged=bridge1 untagged=ether4 vlan-ids=40 Inter-VLAN Routing Rules If you need selective inter-VLAN routing, the cleanest approach is an address list filter on the forward chain. Block all inter-VLAN traffic by default, then open specific paths:\n# Block inter-VLAN by default (before the general accept rule) /ip firewall filter add chain=forward in-interface=vlan30 out-interface=!bridge1 \\ action=drop comment=\u0026#34;Block IoT -\u0026gt; other VLANs\u0026#34; add chain=forward in-interface=vlan40 out-interface=!bridge1 \\ action=drop comment=\u0026#34;Block Guest -\u0026gt; other VLANs\u0026#34; # Allow IoT -\u0026gt; homelab for specific services (e.g., MQTT) add chain=forward in-interface=vlan30 out-interface=vlan20 \\ protocol=tcp dst-port=1883 action=accept \\ comment=\u0026#34;Allow IoT MQTT to homelab\u0026#34; 4. DNS and Caching RouterOS 7 supports DNS over HTTPS (DoH) natively, which encrypts your DNS queries and prevents ISP-level snooping or manipulation. Combined with the local DNS cache and optional ad-filtering via address lists, you can replace Pi-hole for basic network-wide blocking.\nDNS over HTTPS Setup # Set upstream DoH servers /ip dns set servers=1.1.1.1,8.8.8.8 \\ use-doh-server=https://cloudflare-dns.com/dns-query \\ verify-doh-cert=yes \\ allow-remote-requests=yes \\ cache-size=4096KiB \\ max-udp-packet-size=4096 Server choices:\nhttps://cloudflare-dns.com/dns-query — Cloudflare (low latency, ECS support) https://dns.quad9.net/dns-query — Quad9 (threat blocking) https://dns.google/dns-query — Google (high reliability) Note: The router MUST have a valid system clock for TLS certificate verification. Ensure NTP is configured:\n/system ntp client set enabled=yes /system ntp client servers add address=pool.ntp.org /system ntp client print DNS Adlists On RouterOS versions supporting /ip dns adlist, use the built-in DNS adlist facility rather than parsing a hosts file into firewall address lists. An IP-level block can affect unrelated domains sharing the same address and is not equivalent to DNS filtering.\n/ip dns adlist add url=\u0026#34;https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts\u0026#34; ssl-verify=yes /ip dns adlist print Run this once, check available memory and certificate validation, and direct intended clients to the router\u0026rsquo;s DNS resolver. Restrict TCP/UDP port 53 to authorized clients in the input firewall before enabling remote DNS requests. Clients using another resolver can bypass this policy. Follow the RouterOS DNS documentation for supported formats, limits, and updates.\n5. Container Management RouterOS 7 on ARM64 and x86 hardware supports running Linux containers directly on the router. This is useful for lightweight services like a Tailscale exit node, Cloudflare Tunnel connector, or simple HTTP health checkers — but it comes with resource and security constraints.\nContainer Setup 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 # Enable container mode and create a mount /container config set ram-high=256M ram-low=128M /disk/file-system add slot=usb1 type=ext4 label=containers # Create mount point for persistent storage /disk/directory add name=containers /disk/mount-point add directory=containers slot=usb1 # Pull and run an image (example: cloudflared) /container add remote-image=cloudflare/cloudflared:latest \\ envlist=cloudflare_env \\ mount=[{\u0026#34;src=/containers/cloudflared\u0026#34; \u0026#34;dst=/home/nonroot\u0026#34;}] \\ interface=vlan20 \\ root-dir=/containers/cloudflared/root \\ logging=yes # Set environment variables /container env add name=cloudflare_env key=TUNNEL_TOKEN value=\u0026#34;your-token-here\u0026#34; Container Best Practices # Limit CPU and memory per container /container set cloudflared cpu-limit=20% memory-limit=64M # Isolate with a dedicated VLAN interface — don\u0026#39;t bridge # containers directly to the management VLAN /interface vlan add name=container_vlan vlan-id=99 interface=bridge1 # Use container_vlan for all containers # Monitor resource usage /container/print stats detail When to use containers on the router (vs. on a server):\nGood for router Better on a server Tailscale/Headscale CPU-heavy apps Cloudflare Tunnel Databases Simple health checks Media transcoding Net-data light monitoring Git servers The router has limited RAM (typically 512MB-2GB) and a modest CPU. Running anything more than 2-3 lightweight containers will impact routing performance.\n6. Performance Tuning These settings extract maximum throughput from RouterOS 7 on homelab hardware.\nEthernet and Interface Settings # Disable auto-negotiation on SFP+ and set fixed rate /interface ethernet set sfp-sfpplus1 advertise=10G-full-duplex \\ comment=\u0026#34;Disable autoneg, lock to 10G\u0026#34; # Enable flow control on high-throughput links set sfp-sfpplus2 flow-control=on # Disable interfaces you\u0026#39;re not using /interface print where running=no # Then: set [find where name~\u0026#34;ether[5-9]\u0026#34;] disabled=yes Queuing and FastTrack The default queue type on RouterOS 7 is only-pending for egress, which can cause bufferbloat. Switch to a CoDel-based queue for latency-sensitive traffic:\n1 2 3 4 5 6 7 8 9 # Apply fq_codel on WAN interface for bufferbloat control /queue type add name=fq-codel kind=fq-codel /queue simple add name=wan-shaping target=10.0.99.0/24 \\ queue=fq-codel/fq-codel \\ max-limit=950M/950M \\ comment=\u0026#34;Shaped WAN with fq_codel\u0026#34; Important: queue simple and queue tree are mutually exclusive with FastTrack. If you use per-connection queuing, remove the FastTrack rule:\n# Find and disable the FastTrack rule /ip firewall filter print where action=fasttrack-connection # Then disable or remove it SFP+ DAC Cable Compatibility SFP+ DAC cables (passive twinax) sometimes fail to link on MikroTik unless the port is configured correctly:\n# Common fix: set the SFP+ interface to advertise only # supported rates and disable auto-negotiation fallback /interface ethernet set sfp-sfpplus1 advertise=10G-full-duplex \\ sfp-rate-select=high \\ comment=\u0026#34;10G DAC cable\u0026#34; Disable Unnecessary Hardware Features 1 2 3 4 5 6 # Disable IPv6 if not using it (saves CPU cycles) /ipv6 settings set disable-ipv6=yes # Disable MPLS and routing filter if not needed /mpls interface print # If no interfaces listed, keep it disabled 7. Backup Automation A lost MikroTik config is painful to rebuild. Automate backups to an external location.\nCreate Local Backups, Then Transfer Them Off the Router These commands create a local text export and an encrypted binary backup. Replace the password before running them, and store it separately from the backup:\n/export file=router-config /system backup save name=router-binary password=\u0026#34;REPLACE_WITH_A_UNIQUE_BACKUP_PASSWORD\u0026#34; /file print where name~\u0026#34;router-\u0026#34; Download both files through WinBox or a supported file-transfer method, then verify that your external backup system actually received them. A local file listing does not upload anything. Binary backups are intended for restoring a compatible device; inspect and adapt text exports when moving between models or RouterOS versions.\nFor recurring backups, automate the transfer and monitor its success before adding rotation. The previous examples in this section claimed SMB/cloud upload but did not implement a valid transfer; they have been removed. See RouterOS backup documentation.\n8. Monitoring and Logging Remote Syslog Send all firewall and system logs to your homelab logging stack (Loki, Grafana, or a simple syslog server):\n/system logging action add name=remote_syslog target=remote remote=10.0.20.50 remote-port=514 syslog-facility=local0 # Send firewall drops to remote /system logging add topics=firewall,!info action=remote_syslog # Send critical system events add topics=critical action=remote_syslog # Keep local buffer reasonable (default is large) /system logging action set memory memory-lines=1000 SNMP for Grafana 1 2 3 4 5 6 # Enable SNMP with read-only community (change the community!) /snmp community add name=homelab-mon address=10.0.20.0/24 read-access=yes /snmp set enabled=yes contact=\u0026#34;admin@gntech.dev\u0026#34; location=\u0026#34;Homelab Rack\u0026#34; Pair with a Grafana dashboard using the official MikroTik SNMP MIBs for beautiful interface traffic graphs and system health.\nInterface Traffic Graphs For a live readout, use a monitor command:\n/interface monitor-traffic ether1 once For stored WebFig graphs, add the actual interfaces to /tool graphing. This example restricts graph access to the management subnet:\n/tool graphing interface add interface=ether1 allow-address=10.0.99.0/24 /tool graphing interface add interface=ether2 allow-address=10.0.99.0/24 Adapt interface names and permitted clients, and keep the router\u0026rsquo;s web service restricted to management access. monitor-traffic displays measurements; it does not create graphing objects.\n9. Upgrade Strategy RouterOS stable releases come every 4-8 weeks. A conservative upgrade strategy prevents breaking changes from hitting your production network:\nBefore upgrading, create and download an export and encrypted backup using the backup procedure above. Read the release notes for your chosen channel and check hardware/package compatibility.\n/system package update check-for-updates /system package update print During a maintenance window, after reviewing the proposed version:\n/system package update install The router can reboot during installation. After reconnecting:\n/system resource print /system package print /ip firewall filter print count-only /interface print /ip route print Check actual connectivity and required services, not only rule counts. Keep console/Netinstall recovery available. A text export is not a binary backup, and /system backup load requires a real compatible .backup file; do not attempt to load an invented filename as an automatic rollback step.\nKeep spare hardware. A $50 hAP lite as a cold spare can save your weekend when an upgrade bricks a primary router. With a backup export, you can have a replacement online in 10 minutes.\n10. Quick Audit Checklist Run this script periodically to verify your security posture:\n# One-liner audit of key security settings :put \u0026#34;=== RouterOS 7 Security Audit ===\u0026#34; :put \u0026#34;Default admin disabled: $[/user get [find where name=admin] disabled]\u0026#34; :put \u0026#34;SSH strong crypto: $[/ip ssh get strong-crypto]\u0026#34; :put \u0026#34;Neighbor discovery: $[/ip neighbor discovery-settings get discover-interface-list]\u0026#34; :put \u0026#34;Bandwidth server: $[/tool bandwidth-server get enabled]\u0026#34; :put \u0026#34;UPnP: $[/ip upnp get enabled]\u0026#34; :put \u0026#34;MAC WinBox: $[/tool mac-server mac-winbox get allowed-interface-list]\u0026#34; :put \u0026#34;Socks: $[/ip socks get enabled]\u0026#34; :put \u0026#34;Proxy: $[/ip proxy get enabled]\u0026#34; :put \u0026#34;RouterOS version: $[/system package update get installed-version]\u0026#34; :put \u0026#34;Uptime: $[/system resource get uptime]\u0026#34; :put \u0026#34;CPU load: $[/system resource get cpu-load]%\u0026#34; :put \u0026#34;Free RAM: $[/system resource get free-memory]\u0026#34; :put \u0026#34;===================================\u0026#34; Save it as a boot-on-failure event:\n/system script add name=audit source={...} /system scheduler add name=audit-run interval=7d start-time=06:00 \\ on-event=\u0026#34;/system script run audit\u0026#34; Summary RouterOS 7 is a remarkable platform for homelab and small-biz networking, but its flexibility demands disciplined configuration. These best practices cover the critical bases:\nHardening — Change the admin user, restrict management access, disable unused services, enable SSH strong crypto, update RouterOS regularly. Firewall — Use the default-drop pattern with proper rule ordering: stateful tracking first, then ICMP, then management access, then forwarding. Place FastTrack rules before general accept rules. VLAN segmentation — Bridge VLAN filtering with vlan-filtering=yes and explicit tagged/untagged port configuration. Block inter-VLAN traffic by default, open specific paths. DNS — DoH via Cloudflare or Quad9 with local caching. Optional DNS-based ad-blocking using address lists. Containers — Lightweight services only, with CPU/RAM limits and dedicated VLAN isolation. Performance — FastTrack for throughput, fq_codel for latency control, proper SFP+ configuration, disable unused features. Backup — Daily automated exports with remote upload. Test restores quarterly. Apply these configurations incrementally. Change one section, test it for a few days, then move to the next. RouterOS does not abstract complexity away — it rewards those who understand every rule they write.\n","permalink":"https://blog.gntech.me/posts/2026-05-26-mikrotik-routeros7-best-practices/","summary":"Practical RouterOS 7 best practices for homelab and small-business deployments — security hardening, firewall design, VLAN segmentation, FastTrack performance tuning, DoH configuration, and backup automation with real commands.","title":"MikroTik RouterOS 7 — Hardening and Best Practices Guide"},{"content":"Docker Compose starts all services in a stack simultaneously by default. If your application depends on PostgreSQL or Redis being fully ready, you get connection errors, retry loops, and startup race conditions.\nThe fix is healthchecks. Not the Dockerfile-level HEALTHCHECK instruction — the full Compose healthcheck block with depends_on + condition: service_healthy. This post covers everything from basic setup to real-world examples for common homelab services, custom scripts, debugging, and the common pitfalls.\nHow Docker Healthchecks Actually Work A healthcheck is a command that runs inside the container every interval seconds. Exit code 0 means healthy. Exit code 1 means unhealthy. If the check fails retries times in a row, Docker marks the container unhealthy.\n1 2 3 4 5 6 7 8 9 services: app: image: myapp:latest healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;curl\u0026#34;, \u0026#34;-f\u0026#34;, \u0026#34;http://localhost:8080/health\u0026#34;] interval: 30s timeout: 10s retries: 3 start_period: 40s Each field has a specific purpose:\nField Default What it controls test NONE Command to run (array form preferred) interval 30s How often the check runs timeout 30s Max time for one check before it\u0026rsquo;s a failure retries 3 Consecutive failures before marking unhealthy start_period 0s Grace period — no health checks run during this time The start_period is critical for databases and apps with long initialization. During this period, Docker treats any exit code as success so the service has time to boot before health tracking begins.\ndepends_on with condition: service_healthy Compose\u0026rsquo;s depends_on has three conditions:\n1 2 3 4 5 depends_on: db: condition: service_started # default — just started the container redis: condition: service_healthy # wait until healthcheck passes service_started is the default and is practically useless for ordering — it only means Docker started pulling the image and creating the container, not that the process inside is listening.\nFor real dependency ordering, always use service_healthy with a properly configured healthcheck on the dependency.\nFull Example — Web App with PostgreSQL 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 services: db: image: postgres:16-alpine environment: POSTGRES_USER: app POSTGRES_PASSWORD: changeme POSTGRES_DB: myapp healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;pg_isready -U app -d myapp\u0026#34;] interval: 10s timeout: 5s retries: 5 start_period: 30s app: build: . depends_on: db: condition: service_healthy environment: DATABASE_URL: postgres://app:changeme@db:5432/myapp The web app will not start until PostgreSQL is accepting connections. No retry loops, no connection refused errors.\nHealthcheck Examples for Common Homelab Services PostgreSQL 1 2 3 4 5 6 healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;pg_isready -U ${POSTGRES_USER:-postgres}\u0026#34;] interval: 10s timeout: 5s retries: 5 start_period: 30s pg_isready ships with PostgreSQL and returns 0 when the server is accepting connections. It hits the unix socket or TCP port directly — no extra tools needed.\nRedis 1 2 3 4 5 6 healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;redis-cli\u0026#34;, \u0026#34;ping\u0026#34;] interval: 10s timeout: 3s retries: 5 start_period: 10s redis-cli ping returns PONG when the server is ready. Works with or without a password (add -a $REDIS_PASSWORD if needed).\nNginx / Caddy 1 2 3 4 5 6 healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;curl\u0026#34;, \u0026#34;-f\u0026#34;, \u0026#34;http://localhost/\u0026#34;] interval: 15s timeout: 5s retries: 3 start_period: 5s Requires curl in the image. For Alpine-based images, install it in your Dockerfile:\n1 RUN apk add --no-cache curl MariaDB / MySQL 1 2 3 4 5 6 healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;mysqladmin ping -h localhost -u root -p${MYSQL_ROOT_PASSWORD}\u0026#34;] interval: 10s timeout: 5s retries: 5 start_period: 30s mysqladmin ping returns 0 when the server is alive. On MariaDB official images, healthcheck.sh is provided:\n1 2 3 4 5 6 healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;healthcheck.sh\u0026#34;, \u0026#34;--connect\u0026#34;, \u0026#34;--innodb_initialized\u0026#34;] interval: 10s timeout: 5s retries: 5 start_period: 30s RabbitMQ 1 2 3 4 5 6 healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;rabbitmq-diagnostics -q ping\u0026#34;] interval: 15s timeout: 10s retries: 5 start_period: 60s RabbitMQ ships rabbitmq-diagnostics which does a full cluster health check — not just port availability.\nMinIO 1 2 3 4 5 6 healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;mc\u0026#34;, \u0026#34;ready\u0026#34;, \u0026#34;local\u0026#34;] interval: 15s timeout: 5s retries: 5 start_period: 15s Requires the MinIO client (mc) inside the container. The MinIO official image includes it.\nVaultwarden (Bitwarden) 1 2 3 4 5 6 healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;curl\u0026#34;, \u0026#34;-f\u0026#34;, \u0026#34;http://localhost:80/alive\u0026#34;] interval: 30s timeout: 10s retries: 3 start_period: 30s Custom Healthcheck Scripts When a single command is not enough, mount a script or use the CMD-SHELL form to chain commands.\nExample — Multi-Condition Check for a Web App 1 2 3 4 5 6 7 healthcheck: test: | [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;curl -f http://localhost:8080/api/health \u0026amp;\u0026amp; curl -f http://localhost:8080/static/index.html\u0026#34;] interval: 30s timeout: 15s retries: 3 start_period: 60s Mounted Script Pattern For complex checks that need branching logic:\n1 2 3 4 5 6 7 8 9 10 11 12 13 #!/bin/sh # healthcheck.sh — check app health via multiple endpoints # Check main health endpoint curl -sf http://localhost:8080/health || exit 1 # Check database connectivity via app endpoint curl -sf http://localhost:8080/health/db || exit 1 # Check cache curl -sf http://localhost:8080/health/cache || exit 1 exit 0 Mount it into the container and reference it:\n1 2 3 4 5 6 7 8 9 10 11 services: app: image: myapp:latest volumes: - ./healthcheck.sh:/usr/local/bin/healthcheck.sh healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;healthcheck.sh\u0026#34;] interval: 30s timeout: 15s retries: 3 start_period: 60s Dockerfile-Level vs Compose-Level Healthchecks You can define healthchecks in two places, and they work differently.\nDockerfile HEALTHCHECK 1 2 HEALTHCHECK --interval=30s --timeout=5s --start-period=30s --retries=3 \\ CMD curl -f http://localhost:8080/health || exit 1 Pros:\nBundled with the image — every container gets it automatically Works without Compose (raw docker run) Cons:\nCan\u0026rsquo;t use depends_on with condition: service_healthy on the Dockerfile-level check unless Compose is also involved Less flexible per-instance Compose-Level healthcheck 1 2 3 4 5 6 healthcheck: test: ... interval: 30s timeout: 5s retries: 3 start_period: 30s When to use which:\nDockerfile HEALTHCHECK — base images that always need the same check (e.g., your custom application base image) Compose healthcheck — environment-specific overrides, services you don\u0026rsquo;t control the image for, stacks where you need depends_on: condition: service_healthy Compose healthcheck overrides the Dockerfile one when both are defined. This is useful when a database config difference changes the healthcheck endpoint.\nThe curl Problem — Healthchecks Without curl Many minimal images (Alpine, distroless, scratch) do not ship curl or wget. Installing them defeats the purpose of a small image. Alternatives:\nUse wget Instead 1 2 healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;wget\u0026#34;, \u0026#34;-qO-\u0026#34;, \u0026#34;http://localhost:8080/health\u0026#34;] wget is smaller than curl and available on more base images.\nUse the Application\u0026rsquo;s Built-in Check PostgreSQL has pg_isready. Redis has redis-cli ping. MySQL has mysqladmin ping. Prefer the service\u0026rsquo;s native tool when it exists.\nUse /dev/tcp (Bash Built-in) For pure-shell environments without curl or wget:\n1 2 3 4 5 6 healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;exec 3\u0026lt;\u0026gt;/dev/tcp/localhost/5432 \u0026amp;\u0026amp; echo 1 \u0026gt;\u0026amp;3 \u0026amp;\u0026amp; read -u 3 \u0026amp;\u0026amp; exec 3\u0026gt;\u0026amp;-\u0026#34;] interval: 15s timeout: 5s retries: 5 start_period: 20s This opens a TCP connection using bash\u0026rsquo;s /dev/tcp virtual filesystem. Works on any image with bash (not Alpine\u0026rsquo;s busybox sh by default — needs bash installed).\nUse a Dedicated Health HTTP Server For custom apps, embed a tiny health server:\n1 2 healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;/healthz\u0026#34;] Where /healthz is a compiled binary or script that checks TCP ports, file descriptors, and returns 0 or 1. The Go standard library\u0026rsquo;s net/http/pprof serves this pattern well.\nDebugging Healthcheck Failures Healthchecks fail silently unless you know where to look. Here is the diagnostic workflow.\n1. Check Container Health State 1 2 3 4 docker ps --filter \u0026#34;health=unhealthy\u0026#34; # See all containers and their health docker ps --format \u0026#34;table {{.Names}}\\t{{.Status}}\u0026#34; Output includes the health status in the STATUS column: (healthy), (unhealthy), or (starting) during start_period.\n2. Inspect Health Log 1 docker inspect --format=\u0026#39;{{json .State.Health}}\u0026#39; container_name | jq . This shows the full health history — every check, its exit code, and output:\n1 2 3 4 5 6 7 8 9 10 11 12 { \u0026#34;Status\u0026#34;: \u0026#34;unhealthy\u0026#34;, \u0026#34;FailingStreak\u0026#34;: 5, \u0026#34;Log\u0026#34;: [ { \u0026#34;Start\u0026#34;: \u0026#34;2026-05-26T15:30:00Z\u0026#34;, \u0026#34;End\u0026#34;: \u0026#34;2026-05-26T15:30:05Z\u0026#34;, \u0026#34;ExitCode\u0026#34;: 1, \u0026#34;Output\u0026#34;: \u0026#34;curl: (7) Failed to connect to localhost port 8080: Connection refused\u0026#34; } ] } The Output field is gold — it tells you exactly why the check failed.\n3. Run the Check Command Manually 1 docker exec -it container_name curl -f http://localhost:8080/health This reproduces the exact same environment the healthcheck runs in. If it fails here, the check is correctly detecting a real problem. If it succeeds but the healthcheck still fails, check:\nDid start_period expire? Is the command in $PATH inside the container? Does the command use a shell feature not available in sh? 4. Common Failures and Fixes Symptom Likely cause Fix curl: not found No curl in minimal image Use native tool or install curl Connection refused Service not listening on localhost Check bind 127.0.0.1 config pg_isready: command not found Wrong user or PATH Use /usr/bin/pg_isready (starting) never becomes healthy start_period too short Increase start_period unhealthy after booting fine interval too fast for GC pauses Increase interval to 30-60s Advanced Pattern — Intra-Stack Dependency Chains In complex stacks, you need to chain dependencies through multiple levels.\nLayered Services Example 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 services: postgres: image: postgres:16-alpine healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;pg_isready -U app\u0026#34;] interval: 10s timeout: 5s retries: 5 start_period: 30s redis: image: redis:7-alpine healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;redis-cli\u0026#34;, \u0026#34;ping\u0026#34;] interval: 10s timeout: 3s retries: 5 start_period: 10s api: build: ./api depends_on: postgres: condition: service_healthy redis: condition: service_healthy healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;curl\u0026#34;, \u0026#34;-f\u0026#34;, \u0026#34;http://localhost:3000/health\u0026#34;] interval: 15s timeout: 5s retries: 3 start_period: 30s nginx: build: ./nginx ports: - \u0026#34;80:80\u0026#34; depends_on: api: condition: service_healthy monitoring: image: grafana/grafana:latest ports: - \u0026#34;3001:3000\u0026#34; depends_on: postgres: condition: service_healthy Compose resolves this dependency graph at startup. PostgreSQL and Redis start first. API waits for both. Nginx waits for API. Monitoring waits only for PostgreSQL.\nHealthcheck Propagation in Compose Watch Compose Watch (v2.27+) can wait for healthchecks before restarting dependent services during development:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 services: api: build: ./api healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;curl\u0026#34;, \u0026#34;-f\u0026#34;, \u0026#34;http://localhost:3000/health\u0026#34;] interval: 5s timeout: 3s retries: 3 start_period: 10s develop: watch: - action: sync path: ./api/src target: /app/src ignore: - node_modules/ - .env nginx: image: nginx:alpine depends_on: api: condition: service_healthy When api source files change, Compose Watch syncs them and waits for the API healthcheck to pass before signaling nginx that the dependency is ready.\nAvoiding the Retry Spiral — Graceful Degradation Even with perfect healthchecks, services crash. The container runtime will restart them per the restart policy, and the healthcheck will detect when they recover. But transient failures during the restart window can cascade.\nSet restart: unless-stopped on all services and configure your application code to retry connections with exponential backoff:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # Python example — reconnect with exponential backoff import time import psycopg2 from psycopg2 import OperationalError def wait_for_db(max_retries=10, base_delay=1): for attempt in range(max_retries): try: conn = psycopg2.connect( host=\u0026#34;db\u0026#34;, dbname=\u0026#34;myapp\u0026#34;, user=\u0026#34;app\u0026#34;, password=\u0026#34;pass\u0026#34; ) return conn except OperationalError: delay = base_delay * (2 ** attempt) time.sleep(delay) raise RuntimeError(\u0026#34;Database not available\u0026#34;) The combination of Docker healthchecks on the infrastructure side + application-level retry logic on the app side gives you reliable recovery from any failure scenario.\nSummary Docker healthchecks are the difference between a stack that seems to work sometimes and one that boots reliably every time.\nThe four rules:\nAlways use depends_on with condition: service_healthy for databases and services your app requires at startup Set start_period to match your service\u0026rsquo;s boot time — underestimate this and the check fails before it should Prefer native tools (pg_isready, redis-cli ping) over curl for database healthchecks Debug with docker inspect — the health log output tells you exactly why the check failed Copy the examples for PostgreSQL, Redis, Nginx, MariaDB, and others into your compose files and your applications will start in the right order, every time.\n","permalink":"https://blog.gntech.me/posts/2026-05-26-docker-healthchecks-compose-service-readiness/","summary":"A practical guide to Docker healthchecks in Compose — define checks with test, interval, timeout, and start_period, use depends_on condition: service_healthy for ordered startup, and debug health state with docker inspect.","title":"Docker Healthchecks — Service Readiness and Dependencies in Compose"},{"content":"Your homelab runs dozens of Docker containers. Each image pulls in layers from upstream registries — Alpine, Debian, nginx, Python, PostgreSQL. And every layer can ship with known vulnerabilities that get discovered months after you pulled it.\nManual audits don\u0026rsquo;t scale. You need an automated scanner that checks every running image against the latest CVE databases, flags misconfigurations, detects embedded secrets, and generates reports you can actually act on.\nTrivy from Aqua Security does all of this. It is fast, dependency-free (one binary or a container), covers OS packages and language-specific libraries, and integrates cleanly into a homelab automation workflow. This guide walks through installation, scanning workflows, output formats, and a complete systemd timer setup for weekly unattended scans.\nInstalling Trivy Trivy ships as a single static binary, a Docker image, and packages for every major distro. For a Proxmox or bare-metal Linux host, install the binary directly — it avoids the overhead of scanning images from inside a container.\nDebian/Ubuntu Repository Install 1 2 3 4 5 6 7 # Add Aqua Security repository wget -qO- https://aquasecurity.github.io/trivy-repo/deb/public.key | \\ gpg --dearmor | sudo tee /usr/share/keyrings/trivy.gpg \u0026gt; /dev/null echo \u0026#34;deb [signed-by=/usr/share/keyrings/trivy.gpg] \\ https://aquasecurity.github.io/trivy-repo/deb generic main\u0026#34; | \\ sudo tee /etc/apt/sources.list.d/trivy.list sudo apt update \u0026amp;\u0026amp; sudo apt install trivy -y Docker-based Usage 1 2 docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \\ aquasec/trivy image \u0026#34;\u0026lt;image-name\u0026gt;\u0026#34; Verify Installation 1 2 3 4 5 6 trivy --version # Should output something like: # Version: 0.61.0 # Vulnerability DB: # Type: Light # Next Update: 2026-05-27 12:00:00 (UTC) Trivy downloads a vulnerability database on first run. The database is cached under ~/.cache/trivy and updated automatically when you scan with a DB older than 12 hours. You can force an update with:\n1 trivy image --download-db-only Scanning Running Docker Images The most useful scan for a homelab: check every image currently running for known CVEs.\nScan a Single Image 1 trivy image nginx:1.27-alpine This produces a severity-sorted table of every CVE found in the OS packages inside that image. Critical findings are highlighted in red.\nScan All Running Containers Combine docker ps with Trivy to scan your entire runtime:\n1 2 3 4 docker ps --format \u0026#39;{{.Image}}\u0026#39; | sort -u | while read img; do echo \u0026#34;=== Scanning: $img ===\u0026#34; trivy image --severity HIGH,CRITICAL --quiet \u0026#34;$img\u0026#34; done Add --ignore-unfixed to skip CVEs that have no available fix — this reduces noise significantly in production-style scans.\n1 trivy image --severity HIGH,CRITICAL --ignore-unfixed nginx:1.27-alpine Example Output nginx:1.27-alpine (alpine 3.21.3) =============================== Total: 3 (HIGH: 2, CRITICAL: 1) ┌──────────┬────────────────┬──────────┬──────────────┬───────────────┐ │ Library │ Vulnerability │ Severity │ Installed Ver│ Fixed Version │ ├──────────┼────────────────┼──────────┼──────────────┼───────────────┤ │ libcrypto3│ CVE-2026-1234 │ CRITICAL │ 3.3.0-r0 │ 3.3.1-r0 │ │ libssl3 │ CVE-2026-1235 │ HIGH │ 3.3.0-r0 │ 3.3.1-r0 │ │ zlib │ CVE-2026-1236 │ HIGH │ 1.3.1-r0 │ 1.3.2-r0 │ └──────────┴────────────────┴──────────┴──────────────┴───────────────┘ Scanning for Misconfigurations and Secrets Trivy is not just for CVEs. It also scans Dockerfiles and docker-compose files for misconfigurations and hardcoded secrets.\nScan Dockerfile Misconfigurations 1 trivy config ./docker-compose.yml This catches issues like:\nContainers running as root when they don\u0026rsquo;t need to Missing health checks Exposed ports without documented purpose Privileged mode without justification Environment variables that look like secrets Scan for Secrets in Images 1 trivy image --scanners secret nginx:1.27-alpine The secrets scanner checks for embedded API keys, tokens, passwords, and private keys in image layers. Useful before pushing custom images to a registry.\nFull Multi-Scanner Run 1 trivy image --scanners vuln,secret,config my-custom-image:latest Generating Actionable Reports Plain terminal output is fine for interactive use, but for automated scans you want formats you can archive, email, or visualize.\nHTML Report 1 2 trivy image --format template --template \u0026#34;@contrib/html.tpl\u0026#34; \\ -o report.html nginx:1.27-alpine Install the template first:\n1 2 3 mkdir -p ~/.trivy/templates wget -O ~/.trivy/templates/html.tpl \\ https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/html.tpl JSON Report for Scripting 1 trivy image --format json -o scan-results.json nginx:1.27-alpine Use jq to extract only critical findings:\n1 2 3 4 jq \u0026#39;.Results[]?.Vulnerabilities[]? | select(.Severity == \u0026#34;CRITICAL\u0026#34;) | {PkgName, VulnerabilityID, InstalledVersion, FixedVersion, Title}\u0026#39; \\ scan-results.json SARIF Output for IDE Integration 1 trivy image --format sarif -o results.sarif nginx:1.27-alpine SARIF files can be opened in VS Code with the SARIF viewer extension.\nAutomating Weekly Scans with Systemd Timers The most valuable setup for a homelab: unattended weekly scans that email or log results. Systemd timers are the cleanest way on Linux.\nCreate the Scan Script 1 sudo mkdir -p /opt/trivy-scanner Write /opt/trivy-scanner/scan-all.sh:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 #!/bin/bash # Trivy weekly scan — all running images, HTML report REPORT_DIR=\u0026#34;/var/log/trivy-reports\u0026#34; mkdir -p \u0026#34;$REPORT_DIR\u0026#34; DATE=$(date +%Y-%m-%d-%H%M) REPORT=\u0026#34;$REPORT_DIR/trivy-scan-$DATE.html\u0026#34; # Update vulnerability database trivy image --download-db-only --quiet # Scan each unique running image docker ps --format \u0026#39;{{.Image}}\u0026#39; | sort -u | while read img; do echo \u0026#34;Scanning: $img\u0026#34; trivy image --severity HIGH,CRITICAL --ignore-unfixed \\ --format template --template \u0026#34;@/home/gntech/.trivy/templates/html.tpl\u0026#34; \\ --output \u0026#34;/tmp/trivy-partial-$DATE.html\u0026#34; \\ \u0026#34;$img\u0026#34; 2\u0026gt;/dev/null if [ -f \u0026#34;/tmp/trivy-partial-$DATE.html\u0026#34; ]; then sed -i \u0026#34;s|\u0026#34;\u0026lt;title\u0026gt;\u0026#34;|\u0026#34;\u0026lt;title\u0026gt;\u0026#34;$img - |\u0026#34; \u0026#34;/tmp/trivy-partial-$DATE.html\u0026#34; cat \u0026#34;/tmp/trivy-partial-$DATE.html\u0026#34; \u0026gt;\u0026gt; \u0026#34;$REPORT\u0026#34; rm \u0026#34;/tmp/trivy-partial-$DATE.html\u0026#34; fi done # Also check configs trivy config --severity HIGH,CRITICAL /opt/projects/gntech-blog/docker/ \\ --format template --template \u0026#34;@/home/gntech/.trivy/templates/html.tpl\u0026#34; \\ --output \u0026#34;/tmp/trivy-config-$DATE.html\u0026#34; 2\u0026gt;/dev/null if [ -f \u0026#34;/tmp/trivy-config-$DATE.html\u0026#34; ]; then cat \u0026#34;/tmp/trivy-config-$DATE.html\u0026#34; \u0026gt;\u0026gt; \u0026#34;$REPORT\u0026#34; rm \u0026#34;/tmp/trivy-config-$DATE.html\u0026#34; fi echo \u0026#34;Report written to: $REPORT\u0026#34; Make it executable:\n1 sudo chmod +x /opt/trivy-scanner/scan-all.sh Create the Systemd Service /etc/systemd/system/trivy-scan.service:\n1 2 3 4 5 6 7 8 9 10 11 12 13 [Unit] Description=Trivy weekly vulnerability scan of running Docker images After=docker.service Requires=docker.service [Service] Type=oneshot ExecStart=/opt/trivy-scanner/scan-all.sh User=gntech Group=docker Nice=19 IOSchedulingClass=idle Environment=HOME=/home/gntech The Nice=19 and IOSchedulingClass=idle lines ensure the scan doesn\u0026rsquo;t compete with production workloads.\nCreate the Systemd Timer /etc/systemd/system/trivy-scan.timer:\n1 2 3 4 5 6 7 8 9 10 11 [Unit] Description=Weekly Trivy vulnerability scan Requires=trivy-scan.service [Timer] OnCalendar=weekly Persistent=true RandomizedDelaySec=30min [Install] WantedBy=timers.target Enable and start:\n1 2 3 sudo systemctl daemon-reload sudo systemctl enable trivy-scan.timer sudo systemctl start trivy-scan.timer Check Timer Status 1 2 3 4 5 6 7 8 systemctl status trivy-scan.timer # ● trivy-scan.timer - Weekly Trivy vulnerability scan # Trigger: Wed 2026-05-27 00:00:00 UTC; 10h left # Triggers: trivy-scan.service # Force a manual run: sudo systemctl start trivy-scan.service journalctl -u trivy-scan.service --since \u0026#34;5 min ago\u0026#34; Cron Alternative for Simpler Setups If systemd timers feel heavy, a cron job works just as well:\n1 2 # Run every Sunday at 3 AM 0 3 * * 0 /opt/trivy-scanner/scan-all.sh \u0026gt;\u0026gt; /var/log/trivy-cron.log 2\u0026gt;\u0026amp;1 Add with crontab -e or drop a file in /etc/cron.weekly/.\nIntegrating with a Monitoring Dashboard Trivy JSON output can be visualized in Grafana or Grafana Alloy. A lightweight alternative: serve the HTML reports with a one-liner HTTP server and catch them in your monitoring stack.\nServe Reports via Docker 1 2 3 docker run -d --name trivy-reports --restart unless-stopped \\ -v /var/log/trivy-reports:/usr/share/nginx/html:ro \\ -p 8088:80 nginx:alpine Then check http://homelab-host:8088 for the latest scan reports.\nExtract Key Metrics with jq 1 2 3 4 5 6 # Count critical/high per report for f in /var/log/trivy-reports/*.json; do echo \u0026#34;$(basename $f):\u0026#34; echo \u0026#34; CRITICAL: $(jq \u0026#39;[.Results[]?.Vulnerabilities[]? | select(.Severity==\u0026#34;CRITICAL\u0026#34;)] | length\u0026#39; $f)\u0026#34; echo \u0026#34; HIGH: $(jq \u0026#39;[.Results[]?.Vulnerabilities[]? | select(.Severity==\u0026#34;HIGH\u0026#34;)] | length\u0026#39; $f)\u0026#34; done Keeping Images Up to Date Scanning is only half the battle. When Trivy flags a critical CVE, the fix is almost always a newer version of the base image. A practical workflow:\nScan runs weekly, report lands in /var/log/trivy-reports/ Review report for CRITICAL findings where a fix exists Pull the fixed image: docker pull nginx:1.27-alpine Recreate containers: docker compose up -d Re-run scan to confirm all CVEs are resolved For images you build yourself, rebuild with updated base images and redeploy. If you run Watchtower (covered in a previous post), it can handle the pulling step automatically — but Trivy gives you the visibility to decide which updates matter.\nKey Takeaways Trivy scans images, filesystems, configs, and secrets — it is the single tool you need for container security in a homelab. Use --severity HIGH,CRITICAL --ignore-unfixed to focus on actionable findings and ignore noise. Systemd timers with idle scheduling run scans without disrupting workloads. Nice=19 keeps CPU impact minimal. HTML or JSON reports make results reviewable at a glance and storable for trend analysis. Scan first, then update — Trivy tells you what to fix. Your image update strategy (Watchtower, manual pull, or rebuild) closes the loop. Security is a process, not a one-time task. An automated weekly scan with Trivy keeps your homelab safe with minimal effort — and gives you the confidence that your containers aren\u0026rsquo;t running known-critical vulnerabilities between updates.\n","permalink":"https://blog.gntech.me/posts/2026-05-26-docker-trivy-vulnerability-scanning/","summary":"A practical guide to running automated vulnerability scanning for Docker containers in your homelab — install Trivy, scan images for CVEs and secrets, set up systemd timers for weekly scans, and integrate with dashboards for actionable security reports.","title":"Docker Trivy Scanning — Automated Vulnerability Detection for Homelab Containers"},{"content":"ZFS on Proxmox is not set-and-forget. The defaults work, but they optimize for nobody. A pool you built with zpool create tank /dev/sdb /dev/sdc is likely running in a suboptimal configuration — wrong ashift, default recordsize, uncompressed data, and an ARC sized for a desktop, not a hypervisor.\nThis post covers every layer of ZFS tuning that matters for a Proxmox homelab: pool topology, creation-time parameters, dataset properties, ARC sizing, and the SLOG/L2ARC decision. Every command works on Proxmox VE 9.x / Debian 12 with OpenZFS 2.3+.\nStep 1 — Pool Topology: Mirrors vs RAID-Z This is the single most impactful decision you will make. It determines your IOPS, your capacity, your rebuild speed, and your resilience profile. There is no universally correct choice.\nMirror VDEVs — IOPS King A mirror vdev group (two or more disks, each storing a full copy) gives you the best random IO performance. Reads can come from any member. Writes go to all members simultaneously and complete when the slowest finishes.\n1 2 3 4 5 # 6-drive pool — 3 mirror vdevs, usable capacity = 3 × disk size zpool create tank \\ mirror /dev/sdb /dev/sdc \\ mirror /dev/sdd /dev/sde \\ mirror /dev/sdf /dev/sdg Good for:\nVM disk storage (random 4K IOPS matter here) Database workloads (PostgreSQL, MySQL) Anything latency-sensitive Bad for:\nCapacity efficiency (you lose 50% of raw space) Very large pools on a budget Rebuild speed: Fast. ZFS copies data from the surviving mirror member. A 4 TB drive resilvers in 2–4 hours on a loaded host.\nRAID-Z VDEVs — Capacity Efficient RAID-Z distributes data and parity across all disks in a single vdev. RAID-Z2 (double parity) is the sweet spot for homelabs.\n1 2 3 # 6-drive pool — single RAID-Z2 vdev, usable ≈ 4 × disk size zpool create tank \\ raidz2 /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdg Good for:\nBulk media storage, ISO libraries, backups Sequential write workloads Maximizing usable space per dollar Bad for:\nRandom IOPS (bottlenecked by a single parity group) Mixed VM workloads (latency spikes under contention) Rebuild speed: Slow. Every block must be reconstructed from parity across all surviving disks. A 4 TB drive in a 6-wide RAID-Z2 can take 12–24+ hours.\nThe Hybrid Approach Many Proxmox homelabs run two pools:\n1 2 3 4 5 6 7 8 # Fast pool — mirrors for VMs and databases zpool create fastpool \\ mirror nvme0n1 nvme1n1 \\ mirror nvme2n1 nvme3n1 # Bulk pool — RAID-Z2 for media and backups zpool create bulkpool \\ raidz2 sda sdb sdc sdd sde sdf Register each as a separate storage in Proxmox (Datacenter → Storage → Add → ZFS). This lets you pin each VM and CT to the right tier.\nStep 2 — ashift: The Most Common Mistake ashift controls ZFS\u0026rsquo;s logical sector size. The default is 0 (auto-detect), which reads the physical sector size reported by the drive. The problem: almost every SSD and many modern HDDs lie and report 512 bytes when their real sector size is 4 KB or 8 KB.\nRunning ashift=9 (512 bytes) on a 4 KB-native drive causes ZFS to issue 8× more IOs than needed, thrashing the drive with partial sector writes. Performance loss is 20–50% on random writes.\nWhat ashift Value to Use Drive type Recommended ashift Notes Modern NVMe (Samsung PM9A3, Kioxia CD8, etc.) 13 (8 KB) Most enterprise NVMes use 8 KB pages Consumer NVMe (Samsung 990 Pro, WD SN850X) 12 (4 KB) or 13 Test both — check with nvme id-ns SATA SSD (870 Evo, MX500) 12 (4 KB) Universally correct SATA HDD, Advanced Format (≥2011) 12 (4 KB) All modern HDDs use 4 KB sectors Legacy HDD (pre-2010, 512 byte native) 12 (4 KB) still safe Minor overhead, enables future drive swaps Set ashift at pool creation time. It is immutable afterward.\n1 2 3 4 5 # Correct: create pool with ashift=12 on SATA SSDs zpool create -o ashift=12 tank mirror sda sdb # Correct: create pool with ashift=13 on modern NVMe zpool create -o ashift=13 tank mirror nvme0n1 nvme1n1 Verify After Creation 1 2 zdb -C tank | grep ashift # Output: ashift: 12 (means 2^12 = 4096 bytes) If you built a pool without setting ashift and suspect it is wrong, unfortunately the only fix is to destroy and recreate. Backup your data, destroy the pool, recreate with the right ashift, and restore.\nStep 3 — Compression: Always Enable lz4 (or zstd) Compression in ZFS is essentially free on modern CPUs. LZ4 can compress at 1–2 GB/s per core. Enabling it reduces storage usage, lowers write amplification on SSDs, and frequently improves read latency because less data is fetched from disk.\n1 2 3 4 5 # Enable globally on the pool zfs set compression=lz4 tank # Verify zfs get compression tank lz4 vs zstd Algorithm Speed Ratio When to use lz4 3–4 GB/s per core 2–3× on text/logs Default everywhere zstd-1..3 1–2 GB/s per core 2–4× Media metadata, container images zstd-6..9 200–500 MB/s per core 3–5× Cold storage, backups (save space, accept CPU cost) zstd-10..19 \u0026lt;100 MB/s per core Up to 6× Archival only gzip-9 Slow Comparable to zstd-3 Legacy — use zstd instead For a homelab running VMs, databases, and media:\n1 2 3 4 5 6 # Default for everything zfs set compression=lz4 fastpool zfs set compression=lz4 bulkpool # For backup datasets on the bulk pool — zstd for better ratio zfs set compression=zstd-3 bulkpool/backups Real-World Compression Ratios 1 2 3 4 5 6 # Check compression on a dataset zfs get compressratio bulkpool/media # Output: compressratio 1.48x # See per-file estimates zfs list -o name,compressratio,used,logicalused Typical numbers:\nVM disk images (qemu raw/img): 1.1–1.3× ISO backups: 1.0× (already compressed) Docker overlay2 data: 1.8–2.5× PostgreSQL databases: 1.5–2.0× Logs and config files: 3–5× Step 4 — recordsize and volblocksize ZFS issues IOs in chunks called records (for datasets) and blocks (for zvols). The size of these chunks has a massive impact on performance.\nDataset recordsize (for file storage) Use this for SMB/NFS shares, backup directories, and media stores.\n1 2 3 4 5 6 7 8 # Default is 128K — fine for most file workloads zfs set recordsize=128K bulkpool/media # Small files (git repos, code): 64K zfs set recordsize=64K bulkpool/projects # Dump/backup datasets: 1M for sequential throughput zfs set recordsize=1M bulkpool/backups zvol volblocksize (for VM block storage) This is the size ZFS uses when the VM issues a write. Proxmox VM disks are zvols by default. Mismatching this to the guest workload is the #2 ZFS performance mistake (after ashift).\n1 2 3 4 5 6 7 8 9 10 # Default is 8K — reasonable for mixed VM workloads # For database VMs (PostgreSQL, MySQL): 4K or 8K # The guest DB engine issues 8K random writes # For media/file-server VMs: 16K or 32K # Sequential streaming benefits from larger blocks # For backup-target VMs: 64K # Pure sequential write pattern Set it at zvol creation. Unlike dataset recordsize, you cannot efficiently change volblocksize after data exists.\nTo check the volblocksize of an existing VM disk:\n1 2 zdb -dddd fastpool/vm-100-disk-0 2\u0026gt;/dev/null | grep volblocksize # Output: volblocksize = 8192 If you want to change it, the practical approach in Proxmox is:\nShut down the VM Create a new zvol with the desired volblocksize dd the old zvol to the new one (or use qemu-img convert) Detach the old disk, attach the new one Step 5 — ARC Sizing ZFS uses Adaptive Replacement Cache (ARC) as an in-memory read cache. It competes with your VMs and containers for RAM.\nHow the Default Works By default, ZFS will consume up to 50% of system RAM for the ARC. On a 64 GB host, that is 32 GB of RAM locked by ZFS, leaving only 32 GB for VMs and the OS.\nTuning ARC max 1 2 3 4 5 6 7 8 9 10 11 # Set ARC max to 8 GB echo 8589934592 \u0026gt; /sys/module/zfs/parameters/zfs_arc_max # Make it permanent cat \u0026gt; /etc/modprobe.d/zfs.conf \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; # Limit ZFS ARC to 8 GB on a 64 GB host options zfs zfs_arc_max=8589934592 EOF # Regenerate initramfs update-initramfs -u How Much ARC Is Enough? General guidance for Proxmox:\nSystem RAM Suggested ARC max Notes 8 GB 1 GB Bare minimum — monitor arcstats 16 GB 2–4 GB Good for 2–3 light VMs 32 GB 4–8 GB Good for 4–6 VM/CT workloads 64 GB 8–16 GB Typical Proxmox single node 128 GB 16–32 GB Heavy VM density 256 GB+ 32–64 GB ARC will grow but VMs matter more Monitor ARC Effectiveness Install and run arcstat:\n1 2 3 4 5 6 7 8 9 apt install sysstat # Provides sar, also arcstat on Proxmox # Watch ARC in real time, every 2 seconds arcstat -f time,read,hits,miss,hit%,dhit%,l2hits,l2miss,l2asize 2 # Key metrics: # hit% — ARC read hit rate (target \u0026gt;90%) # miss — reads that hit disk (want these low) # dhit% — demand-data hit rate (important — excludes prefetch) If hit% is below 85% and you have free RAM, increase ARC. If dhit% is above 95%, your ARC is doing its job. Decreasing it may free RAM for VMs without a significant performance penalty.\nStep 6 — SLOG and L2ARC: When and Whether SLOG (Separate ZFS Intent Log) A SLOG is a dedicated NVMe device for synchronous write operations. It does not cache reads. It only absorbs the ZIL (ZFS Intent Log) to accelerate fsync() and O_SYNC writes from databases and journaling filesystems.\nYou need a SLOG if:\nYour main pool is HDD-based and you run databases inside VMs You see high zil_commit latency in zpool iostat -l 1 Your pool has NVMe but you want to isolate ZIL traffic You do NOT need a SLOG if:\nYour pool is all-NVMe (native NVMe latency for ZIL is already \u0026lt;100 µs) You run no sync-heavy workloads (media servers, file shares) Hardware requirements for SLOG:\nPower-loss protection (PLP) — mandatory. Consumer NVMe drives lie about flushing writes. If the SLOG lies about a write and power fails, your pool is corrupt. Use enterprise NVMe: Intel Optane (best — sold as \u0026ldquo;Intel Optane Memory H10/H20\u0026rdquo; on eBay), Samsung PM9A3, Kioxia CD6, or any NVMe with a supercapacitor. Mirror it. A single SLOG is a single point of failure. 1 2 3 4 5 6 # Add a mirrored SLOG to an existing pool zpool add tank log mirror /dev/nvme0n1 /dev/nvme1n1 # Verify zpool status tank # Look for: logs: mirror-2 (or similar) SLOG sizing: The ZIL is a small ring buffer. 10–20 GB per SLOG device is more than enough for any homelab.\nL2ARC (Level 2 ARC) L2ARC is a read cache on a secondary device (usually cheaper SSD/NVMe). Data evicted from ARC is written to L2ARC.\nWhen it helps:\nYou have a large HDD pool with a small ARC (e.g., 8 GB ARC for 40 TB of HDD storage) Your working data set is larger than ARC but fits on a single SSD When it does NOT help:\nYour ARC hit rate is already \u0026gt;90% Your pool is all-NVMe (ARC on NVMe is already fast) Your workload is write-heavy (L2ARC is read-only for data) 1 2 3 4 5 # Add an L2ARC device zpool add tank cache /dev/sdc # Monitor L2ARC effectiveness arcstat -f time,l2hits,l2miss,l2hit%,l2asize 2 Opinion: Most homelabs do not benefit from L2ARC. The ARC hit rate on a 16 GB ARC serving a few VMs is usually above 95%. Add more RAM instead. L2ARC only makes sense when you have a large pool with insufficient ARC and cannot add RAM.\nStep 7 — Dataset Properties for Common Homelab Workloads This table summarizes the recommended zfs set properties for different use cases:\nWorkload recordsize compression atime special notes VM disks (zvol) 8K (volblocksize) lz4 off zfs set primarycache=metadata for DB VMs Docker overlay2 64K lz4 off xattr=sa for extended attr perf Samba/NFS media 128K lz4 off aclmode=passthrough for NFSv4 ACLs Backup target 1M zstd-3 off Sequential streaming benefits from large records ISOs/Templates 128K lz4 off Read mostly, cache trivial PostgreSQL data dir 8K (zvol) lz4 off logbias=throughput to bypass ZIL on async Proxmox container root 64K lz4 off Container filesystems use small blocks Samba Time Machine 128K lz4 off dnodesize=legacy for macOS compat Set these immediately after creating the dataset or zvol:\n1 2 3 4 5 zfs create -o recordsize=64K -o compression=lz4 \\ -o atime=off tank/docker-overlay2 zfs create -V 100G -o volblocksize=8K -o compression=lz4 \\ -o atime=off fastpool/vm-200-disk-0 Step 8 — Monitoring ZFS Health and Performance ZFS ships excellent tools. Use them weekly.\nPool Health 1 2 3 4 5 6 7 8 9 10 11 12 # Quick health check zpool status -v tank # Look for: # state: ONLINE ← good # state: DEGRADED ← failed or removed disk # errors: No known data errors # scrub: ... with 0 errors ← last scrub passed # Detailed error counters zpool status -v -s tank # Shows read/write/checksum errors per drive IO Statistics 1 2 3 4 5 6 7 8 9 # Continuous IO throughput and latency zpool iostat -v tank 2 # Full latency breakdown zpool iostat -v -l tank 2 # Adds: sync_wait, async_wait, scrub, trim latency # With queue depth per vdev zpool iostat -v -q tank 2 Scrubs — Schedule Them A scrub reads every block and verifies checksums. It is the only way to detect and repair silent data corruption.\n1 2 3 4 5 6 # Manual scrub zpool scrub tank # Check scrub status zpool status tank # Shows: scan: scrub in progress since ... Proxmox schedules a monthly scrub by default. Verify:\n1 2 cat /etc/cron.d/zfsutils-linux # Should contain: 0 4 * * 0 root /usr/sbin/zpool scrub -a If not present, add a weekly systemd timer:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 # /etc/systemd/system/zfs-scrub@.service [Unit] Description=ZFS scrub on %I [Service] Type=oneshot ExecStart=/usr/sbin/zpool scrub %I ExecStartPost=/usr/sbin/zpool status %I # /etc/systemd/system/zfs-scrub@.timer [Unit] Description=Weekly ZFS scrub on %I [Timer] OnCalendar=Sun 03:00 Persistent=true [Install] WantedBy=timers.target arcstat Quick Reference 1 2 3 4 5 # Watch ARC in real time arcstat 2 # One-shot summary arcstat -o -s 2 1 Key columns:\nread — total ARC reads per second hits — reads satisfied from ARC miss — reads that went to disk hit% — overall hit ratio dhit% — demand data hit ratio (ignore prefetch) l2hits / l2miss — L2ARC hits and misses (useful only if you have L2ARC) Putting It All Together: A Complete Example Here is a real configuration for a 64 GB Proxmox host with 4 × 1 TB NVMe (VM pool) and 6 × 4 TB HDD (bulk storage):\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 # --- VM pool: 4x NVMe, 2x mirror vdevs, ashift=13 --- zpool create -o ashift=13 fastpool \\ mirror nvme0n1 nvme1n1 \\ mirror nvme2n1 nvme3n1 zfs set compression=lz4 fastpool zfs set atime=off fastpool # --- Bulk pool: 6x HDD, RAID-Z2, ashift=12 --- zpool create -o ashift=12 bulkpool \\ raidz2 sda sdb sdc sdd sde sdf zfs set compression=zstd-3 bulkpool zfs set atime=off bulkpool # --- Limit ARC to 16 GB --- echo 17179869184 \u0026gt; /sys/module/zfs/parameters/zfs_arc_max cat \u0026gt; /etc/modprobe.d/zfs.conf \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; options zfs zfs_arc_max=17179869184 EOF update-initramfs -u # --- Create datasets --- zfs create -o recordsize=64K bulkpool/docker zfs create -o recordsize=1M -o compression=zstd-6 bulkpool/backups zfs create -o recordsize=128K bulkpool/media zfs create -o recordsize=128K fastpool/templates Summary ZFS is the best storage filesystem for Proxmox, but only if you configure it intentionally. The checklist from this post:\nChoose topology — mirrors for IOPS, RAID-Z for capacity Set ashift at pool creation — 12 for SATA, 13 for NVMe Enable compression — lz4 everywhere, zstd on backups Match recordsize to workload — 8K for VMs, 64K for Docker, 128K for media, 1M for backups Size the ARC — 1 GB per 8 TB of pool, or whatever your VM RAM budget allows Skip SLOG on all-NVMe pools — add one only for HDD pools with sync-heavy VMs Monitor weekly — zpool status, arcstat, zpool iostat A properly tuned ZFS pool will outlast the hardware it runs on, migrate seamlessly between Proxmox nodes, and never silently corrupt a byte of data. That is the point of running ZFS.\n","permalink":"https://blog.gntech.me/posts/2026-05-26-zfs-pool-design-proxmox-homelab/","summary":"A practical guide to ZFS pool design and performance tuning for Proxmox homelabs — topology selection, ashift, recordsize, compression, ARC sizing, and real-world monitoring with arcstat and zpool iostat.","title":"ZFS Pool Design — Mirrors, Recordsize, ARC Tuning for Proxmox"},{"content":"Docker networking is the number one source of \u0026ldquo;it works on my machine\u0026rdquo; failures in homelabs. A container can\u0026rsquo;t reach the database. DNS resolves on the host but not inside the container. Port 8080 is already in use. The error messages are cryptic, and docker inspect spews JSON that\u0026rsquo;s hard to parse under pressure.\nThis guide covers the five most common Docker networking failure modes, how to diagnose each one with concrete commands, and the permanent fix. You\u0026rsquo;ll learn to use netshoot like a network engineer, decode Docker\u0026rsquo;s internal DNS resolution, debug bridge and macvlan interfaces, and build robust multi-service compose files that survive host reboots and IP reassignments.\nThe Diagnostic Toolkit — netshoot Before fixing anything, you need a containerized network toolkit that runs in the same network namespace as the broken service. The nicolaka/netshoot image bundles everything: tcpdump, dig, nslookup, curl, ping, mtr, iftop, iperf3, nmap and more — zero installation on the host.\nRun netshoot in a Container\u0026rsquo;s Network Namespace 1 2 # Launch netshoot sharing the target container\u0026#39;s network stack docker run -it --rm --net container:\u0026#34;\u0026lt;container-name\u0026gt;\u0026#34; nicolaka/netshoot Once inside, every tool sees the same network interfaces, routing table, and DNS configuration as the container you\u0026rsquo;re debugging.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 # Inside netshoot — check interfaces ip addr show # Check routing ip route # Check DNS config cat /etc/resolv.conf # Check DNS resolution dig +short service-name.local nslookup db-service # Hit the target port curl -v http://other-service:8080/metrics # Trace the path mtr -r database-host Common CLI Installation Alternatives If you can\u0026rsquo;t pull netshoot (air-gapped or constrained homelab), drop these into any container:\n1 2 3 4 5 # On Debian/Ubuntu-based containers apt update \u0026amp;\u0026amp; apt install -y dnsutils iproute2 curl netcat-openbsd mtr-tiny # On Alpine-based containers apk add bind-tools iproute2 curl netcat-openbsd mtr 1. DNS Resolution Failures Inside Containers Symptom: curl http://service-name works on the host but fails inside a container with Name or service not known. Pinging by IP works fine.\nRoot cause: Docker\u0026rsquo;s embedded DNS resolver (127.0.0.11) isn\u0026rsquo;t reaching the configured upstream DNS servers, or the container isn\u0026rsquo;t on a user-defined bridge network.\nDiagnosis 1 2 3 4 5 6 7 # Check which DNS servers the container sees docker run --rm alpine cat /etc/resolv.conf # Expected: \u0026#34;nameserver 127.0.0.11\u0026#34; (Docker\u0026#39;s internal resolver) # Check if the container is on a user-defined network docker inspect \u0026#34;\u0026lt;container\u0026gt;\u0026#34; --format \u0026#39;{{json .NetworkSettings.Networks}}\u0026#39; # If only \u0026#34;bridge\u0026#34; appears, you\u0026#39;re on the default bridge — no DNS. The Problem with the Default Bridge Containers attached to Docker\u0026rsquo;s bridge network (the default) use the host\u0026rsquo;s /etc/resolv.conf directly through the host\u0026rsquo;s loopback, inheriting whatever DNS servers the host uses. User-defined bridge networks, on the other hand, get Docker\u0026rsquo;s embedded DNS resolver at 127.0.0.11, which enables container-name resolution.\nFix — always use a user-defined network:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # docker-compose.yml services: app: image: my-app:latest networks: - app-net db: image: postgres:16-alpine networks: - app-net networks: app-net: driver: bridge If you\u0026rsquo;re using docker run instead:\n1 2 3 docker network create app-net docker run -d --name db --network app-net postgres:16-alpine docker run -it --rm --network app-net alpine ping db # Works! DNS Timeout on User-Defined Networks If DNS still fails on user-defined networks:\n1 2 3 4 5 6 7 8 9 10 11 12 # Check Docker daemon DNS settings docker info | grep -i dns # Override DNS per-container docker run -d --dns 10.0.20.1 --dns 1.1.1.1 --network app-net my-app # Or per-compose-service services: app: dns: - 10.0.20.1 # Your homelab DNS - 1.1.1.1 # Cloudflare fallback System-wide fix — configure Docker daemon DNS:\n1 2 3 { \u0026#34;dns\u0026#34;: [\u0026#34;10.0.20.1\u0026#34;, \u0026#34;1.1.1.1\u0026#34;] } Save this to /etc/docker/daemon.json and restart:\n1 systemctl restart docker 2. Port Already in Use Symptom: docker: Error response from daemon: driver failed programming external connectivity on endpoint \u0026lt;name\u0026gt;: Bind for 0.0.0.0:8080 failed: port is already allocated.\nDiagnosis 1 2 3 4 5 6 7 8 9 10 # Find what\u0026#39;s using the port ss -tlnp | grep 8080 # or lsof -i :8080 # Check if another Docker container has the port docker ps --format \u0026#34;table {{.Names}}\\t{{.Ports}}\u0026#34; | grep 8080 # Check if a systemd service is listening systemctl list-sockets | grep 8080 Common Causes in Homelabs Another container on the same host claims the port. Two services can\u0026rsquo;t both bind to host port 8080. Move one to a different port: 1 2 3 4 5 6 7 services: app1: ports: - \u0026#34;8081:8080\u0026#34; # Map host 8081 to container 8080 app2: ports: - \u0026#34;8080:8080\u0026#34; # Keep this one A previous container didn\u0026rsquo;t release the port. Docker does release ports on container stop, but a stopped container still reserves its IP. Restart or remove it: 1 docker rm -f \u0026#34;\u0026lt;old-container\u0026gt;\u0026#34; # Force remove Traefik/NGINX reverse proxy is listening there. Your reverse proxy binds to 80/443/8080 on the host. Your service shouldn\u0026rsquo;t also expose those ports — expose an unprivileged port and let the proxy route traffic: 1 2 3 4 5 6 7 services: my-service: expose: - \u0026#34;3000\u0026#34; # Internal only — no host port binding labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.my-service.rule=Host(`app.gntech.dev`)\u0026#34; systemd-resolved or dnsmasq on port 53. Docker maps container DNS to 127.0.0.11, but if you\u0026rsquo;re running a local DNS service on the host (Pi-hole, AdGuard Home, dnsmasq), they compete for port 53: 1 2 3 4 5 6 7 8 9 # Check what\u0026#39;s on port 53 ss -tulpn | grep \u0026#39;:53\u0026#39; # Solutions: # - Move Pi-hole/AdGuard to another port (e.g., 5353) # - Run DNS services in Docker with network_mode: host # - Disable systemd-resolved on the host systemctl stop systemd-resolved systemctl disable systemd-resolved 3. Cross-Container and Cross-Host Connectivity Symptom: Containers on the same Docker network can\u0026rsquo;t communicate, or containers on different hosts can\u0026rsquo;t reach each other.\nSame Host, Same User-Defined Network Containers on the same user-defined bridge can reach each other by container name, but not by localhost. The container\u0026rsquo;s idea of localhost is its own network namespace — not the host\u0026rsquo;s.\n1 2 3 4 5 # This works (same network, by name): docker exec app curl http://db:5432 # This does NOT work from inside a container: docker exec app curl http://localhost:5432 Fix — use service names, not localhost:\n1 2 3 4 5 # Wrong: container tries to reach another service via localhost # Wrong: container tries to reach host services via 127.0.0.1 # Right: use the service name (Docker DNS resolves it) # Right: use host.docker.internal for host services To reach services on the host from inside a container:\n1 2 3 4 services: my-app: extra_hosts: - \u0026#34;host.docker.internal:host-gateway\u0026#34; This adds a /etc/hosts entry that resolves to the Docker gateway IP (172.x.x.1), which routes to the host\u0026rsquo;s network stack. Then inside the container:\n1 curl http://host.docker.internal:9090/api # Hits host\u0026#39;s port 9090 Cross-Host Networking (Multi-Node Docker) Docker\u0026rsquo;s default bridge network does not span hosts. Containers on different machines cannot reach each other by container name. Three solutions:\nOption A — Expose ports and use host IPs:\n1 2 3 4 services: my-app: ports: - \u0026#34;3000:3000\u0026#34; # Expose on the host\u0026#39;s IP Then from another host: curl http://10.0.20.30:3000.\nOption B — Docker Swarm overlay network:\n1 2 3 4 5 6 7 8 # Initialize swarm on first node docker swarm init --advertise-addr 10.0.20.30 # On other nodes docker swarm join --token \u0026#34;\u0026lt;token\u0026gt;\u0026#34; 10.0.20.30:2377 # Create overlay network that spans all nodes docker network create --driver overlay --attachable homelab-overlay Then any container attached to homelab-overlay can reach any other container on that network across hosts — by container name.\nOption C — Tailscale or WireGuard mesh:\nFor a homelab without Docker Swarm, route over Tailscale/WireGuard:\n1 2 3 4 5 services: app: networks: app-net: ipv4_address: \u0026#34;100.100.1.10\u0026#34; # Tailscale IP This requires configuring Tailscale on each Docker host and attaching containers to the Tailscale interface.\n4. Bridge and Macvlan Networking Issues Bridge Network — Container Cannot Reach External Network Symptom: Containers can talk to each other but cannot reach the internet. ping 1.1.1.1 hangs.\nDiagnosis:\n1 2 3 4 5 6 7 8 # From inside the container (via netshoot) ip route show default # Expected: default via 172.x.x.1 dev eth0 # If missing, the container has no default gateway # Check iptables NAT rules on the host iptables -t nat -L -n | grep MASQUERADE # Docker should have a MASQUERADE rule for the bridge subnet Fix — enable IP forwarding and check iptables:\n1 2 3 4 5 6 7 8 9 # On the Docker host sysctl net.ipv4.ip_forward=1 # Make permanent echo \u0026#34;net.ipv4.ip_forward=1\u0026#34; \u0026gt;\u0026gt; /etc/sysctl.conf # Verify Docker\u0026#39;s iptables rules exist docker network inspect bridge --format \u0026#39;{{json .IPAM.Config}}\u0026#39; # Ensure iptables is not fully blocking Docker bridge traffic iptables -I DOCKER-USER -i br-\u0026#34;\u0026lt;id\u0026gt;\u0026#34; -j ACCEPT If you use UFW, it overrides Docker\u0026rsquo;s iptables rules. See the Docker UFW fix for the permanent solution:\n1 2 3 4 5 6 7 8 # /etc/ufw/after.rules — add before the final COMMIT *filter :DOCKER-USER - [0:0] -A DOCKER-USER -j RETURN -s 10.0.0.0/8 -A DOCKER-USER -j RETURN -s 172.16.0.0/12 -A DOCKER-USER -j DROP -p tcp --dport 2375:2376 -A DOCKER-USER -j DROP -p tcp --dport 3375 COMMIT Macvlan — Container Receives DHCP but Can\u0026rsquo;t Reach Host Symptom: Macvlan containers get IPs from your DHCP server and can reach external hosts, but cannot reach services running on the Docker host itself.\nRoot cause: Macvlan bypasses the host\u0026rsquo;s network stack. The host cannot communicate with macvlan containers unless you create a subinterface.\nDiagnosis:\n1 2 3 4 5 6 # From the container ip addr show # Should show the macvlan (not bridge) interface # From the host — try reaching the container\u0026#39;s IP ping 10.0.20.50 # The macvlan container\u0026#39;s IP Fixes:\nOption 1 — Use IPvlan L2 instead (lighter, no MAC address per container):\n1 2 3 4 5 6 docker network create -d ipvlan \\ --subnet=10.0.20.0/24 \\ --gateway=10.0.20.1 \\ -o parent=eth0 \\ -o ipvlan_mode=l2 \\ ipvlan-net Option 2 — Add a macvlan subinterface on the host so it can communicate with macvlan containers:\n1 2 3 4 5 # On the Docker host ip link add macvlan-host link eth0 type macvlan mode bridge ip addr add 10.0.20.49/32 dev macvlan-host ip link set macvlan-host up ip route add 10.0.20.50/32 dev macvlan-host Option 3 — Route through an external gateway (MikroTik/OPNsense) if you need macvlan isolation:\n# On the MikroTik router, add a static route # /ip route add dst-address=10.0.20.50/32 gateway=10.0.20.1 5. Docker Compose — Networking Configuration Gotchas Containers Created Before the Network Exists Symptom: docker compose up fails with network not found.\nCompose creates networks declared as managed resources in its project. It does not create a network marked external: true. Check the rendered configuration and the actual Docker daemon/context before changing network definitions:\n1 2 3 docker compose config docker network ls docker compose up -d If an external network is intended, create and manage it separately with the required driver, subnet, and ownership. Do not delete application volumes to resolve a missing network. A network with an unexpected name or project label needs investigation, not an assumption that any manually created network conflicts.\nIf you must share a network across multiple compose files, create it once with external: true:\n1 2 3 4 5 6 7 8 9 # compose-file-a.yml networks: shared-net: external: true # compose-file-b.yml networks: shared-net: external: true Then create it once:\n1 docker network create shared-net Dependency Start Order — Not Network, But Timing Symptom: Service A (database) takes 30 seconds to initialize. Service B starts immediately and fails because the database isn\u0026rsquo;t ready — even though the network works.\nFix — add healthchecks and depends_on with condition:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 services: db: image: postgres:16-alpine healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;pg_isready -U postgres\u0026#34;] interval: 5s timeout: 5s retries: 5 networks: - app-net app: image: my-app:latest depends_on: db: condition: service_healthy networks: - app-net Compose can wait for service_healthy dependencies before starting consumers; this condition does not require Compose 2.20 specifically. This is a timing problem, not a networking problem — but the error looks exactly like one.\nDepends_on Without Condition Is Not Enough 1 2 3 4 5 6 7 8 # This is NOT sufficient — app starts before db is ready depends_on: - db # This is correct — app waits for db healthcheck depends_on: db: condition: service_healthy 6. Diagnostic Reference — Quick Command Reference 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 # ----- Container-Level Diagnostics ----- # Check all network interfaces inside a container docker exec \u0026#34;\u0026lt;container\u0026gt;\u0026#34; ip addr show # Check routing table docker exec \u0026#34;\u0026lt;container\u0026gt;\u0026#34; ip route # Check DNS resolution docker exec \u0026#34;\u0026lt;container\u0026gt;\u0026#34; nslookup google.com # Check firewall rules inside container (if any) docker exec \u0026#34;\u0026lt;container\u0026gt;\u0026#34; iptables -L -n 2\u0026gt;/dev/null || echo \u0026#34;No iptables inside\u0026#34; # Check open ports inside container docker exec \u0026#34;\u0026lt;container\u0026gt;\u0026#34; ss -tlnp # ----- Docker-Level Diagnostics ----- # List all Docker networks docker network ls # Inspect a network in detail docker network inspect app-net # Show containers attached to a network (with IPs) docker network inspect app-net --format \u0026#39;{{range .Containers}}{{.Name}} {{.IPv4Address}}{{\u0026#34;\\n\u0026#34;}}{{end}}\u0026#39; # Check container\u0026#39;s network settings docker inspect \u0026#34;\u0026lt;container\u0026gt;\u0026#34; --format \u0026#39;{{json .NetworkSettings.Networks}}\u0026#39; | jq # Test DNS from Docker\u0026#39;s perspective docker run --rm --network app-net alpine nslookup db # ----- Host-Level Diagnostics ----- # Check IP forwarding sysctl net.ipv4.ip_forward # Check iptables NAT rules for Docker iptables -t nat -L -n | grep -E \u0026#34;DOCKER|MASQUERADE\u0026#34; # Check port usage ss -tlnp # Trace traffic to a container tcpdump -i docker0 -n port 8080 # ----- Full Network Audit for a Down Container ----- # Step 1: Is the container running? docker ps -a --filter name=\u0026#34;\u0026lt;name\u0026gt;\u0026#34; # Step 2: What network is it on? docker inspect \u0026#34;\u0026lt;name\u0026gt;\u0026#34; --format \u0026#39;{{range $k,$v := .NetworkSettings.Networks}}{{printf \u0026#34;%s: %s\\n\u0026#34; $k $v.IPAddress}}{{end}}\u0026#39; # Step 3: Can it reach the gateway? docker exec \u0026#34;\u0026lt;name\u0026gt;\u0026#34; ping -c 2 $(docker inspect \u0026#34;\u0026lt;name\u0026gt;\u0026#34; --format \u0026#39;{{range .NetworkSettings.Networks}}{{.Gateway}}{{end}}\u0026#39;) # Step 4: Can it resolve DNS? docker exec \u0026#34;\u0026lt;name\u0026gt;\u0026#34; nslookup google.com # Step 5: Can it reach the specific target? docker exec \u0026#34;\u0026lt;name\u0026gt;\u0026#34; nc -zv target-service 5432 # Step 6: Restart DNS resolver inside container (without restarting it) docker exec \u0026#34;\u0026lt;name\u0026gt;\u0026#34; sh -c \u0026#39;echo \u0026#34;nameserver 1.1.1.1\u0026#34; \u0026gt; /etc/resolv.conf\u0026#39; Troubleshooting Flowchart ┌──────────────────────────┐ │ Container can\u0026#39;t connect │ └──────────┬───────────────┘ ▼ ┌──────────────────────────┐ │ Is the container running?│ └──────────┬───────────────┘ ├── No → docker start \u0026lt;name\u0026gt; or docker logs \u0026lt;name\u0026gt; │ ▼ Yes ┌──────────────────────────┐ │ Can it reach the gateway?│ └──────────┬───────────────┘ ├── No → Check iptables, IP forwarding, bridge interface │ └─ sysctl net.ipv4.ip_forward │ └─ iptables -t nat -L | grep MASQUERADE │ ▼ Yes ┌──────────────────────────┐ │ Can it resolve DNS? │ └──────────┬───────────────┘ ├── No → Is it on a user-defined network? │ ├── No → Create one, move container │ └── Yes → Check Docker DNS (127.0.0.11) │ └── Override with --dns │ ▼ Yes ┌──────────────────────────┐ │ Can it reach the target │ │ service by IP? │ └──────────┬───────────────┘ ├── No → Firewall between containers? │ └── Check docker-proxy, iptables DOCKER-USER │ └── Check container\u0026#39;s exposed ports │ ▼ Yes ┌──────────────────────────┐ │ Can it reach by name? │ └──────────┬───────────────┘ ├── No → Same user-defined network? │ ├── No → Attach to same network │ └── Yes → Check docker DNS resolution │ └── docker compose restart │ ▼ Yes The network works. Check the application logs. Summary Docker networking failures are frustrating but follow predictable patterns. When a container can\u0026rsquo;t connect:\nRun netshoot in the same namespace — docker run -it --rm --net container:\u0026lt;name\u0026gt; nicolaka/netshoot Check the network type — user-defined bridge or default? Use docker inspect \u0026lt;name\u0026gt; to confirm. Verify DNS resolution — nslookup inside the container. 127.0.0.11 means Docker\u0026rsquo;s resolver is active. Anything else means it inherited the host\u0026rsquo;s network. Check port conflicts — ss -tlnp | grep \u0026lt;port\u0026gt; on the host. One port, one process. Test by IP first — if IP works but name doesn\u0026rsquo;t, it\u0026rsquo;s DNS. If neither works, it\u0026rsquo;s routing, firewalls, or the network driver. Use user-defined networks for all compose files — default bridge lacks DNS-based service discovery. Add healthchecks with depends_on conditions — many \u0026ldquo;networking\u0026rdquo; errors are actually race conditions at startup. Separate reverse proxy ports — let Traefik/NGINX bind to 80/443. Your services expose unprivileged ports internally. The diagnostic skill that separates experienced homelab operators from beginners isn\u0026rsquo;t knowing all the answers — it\u0026rsquo;s knowing the right three commands to run next.\n","permalink":"https://blog.gntech.me/posts/2026-05-25-docker-network-troubleshooting-guide/","summary":"A practical guide to diagnosing and fixing Docker container networking problems — DNS timeouts, port conflicts, bridge network isolation, cross-host communication, and the netshoot toolkit for homelab deployments.","title":"Docker Network Troubleshooting — Diagnostics and Fixes for Container Connectivity"},{"content":"Disk I/O is the most common bottleneck in a homelab. A misconfigured IO scheduler can add 5–10 ms of latency to every write. Wrong mount options silently double metadata writes. And one noisy container can starve every other service on the box.\nMost Linux distributions ship conservative defaults that work on anything from a Raspberry Pi to a 48-bay storage server. That means they are tuned for nobody. Your homelab deserves better.\nThis guide covers the three layers of disk I/O tuning that matter:\nIO schedulers — which one for NVMe, SATA SSD, and spinning rust Mount options — noatime, commit, discard, and nobarrier cgroup IO throttling — taming container I/O with cgroup v2 Every command and config here works on Debian 12, Ubuntu 24.04, Proxmox VE 9.x, and the current Linux 7.x kernel series.\nStep 1 — Understanding Your IO Scheduler The IO scheduler sits between the block layer and the storage driver. Its job: reorder, merge, and dispatch I/O requests to maximize performance. The right choice depends entirely on your hardware.\nCheck What You Are Running 1 2 3 4 5 6 7 8 # Show the current scheduler for every block device cat /sys/block/*/queue/scheduler # Typical output for NVMe (kernel 6.8+ defaults to none): # nvme0n1 — [none] mq-deadline kyber bfq # # Typical for SATA SSD/HDD: # sda — none [mq-deadline] kyber bfq The bracket shows the active scheduler. Modern Linux on NVMe defaults to none (noop), which is correct. SATA drives often default to mq-deadline. Both are sane, but not always optimal.\nWhich Scheduler When Hardware Recommended Scheduler Why NVMe SSD (datacenter grade) none NVMe controllers have native command queuing. Adding a scheduler just burns CPU. Leave it alone. NVMe SSD (consumer / DRAM-less) mq-deadline These benefit from write-merging. Test both. SATA SSD (Samsung 870, MX500, etc.) mq-deadline or kyber Deadline gives latency guarantees. Kyber is leaner. SATA HDD (spinning disks, bulk storage) bfq BFQ provides fair I/O bandwidth sharing. Crucial for multi-VM hosts sharing one HDD. RAID controller (HBA, LSI, etc.) none The controller handles ordering. Schedulers add overhead. Changing the Scheduler Temporary change (live, no reboot):\n1 2 3 4 5 6 # Set NVMe drive to none echo none \u0026gt; /sys/block/nvme0n1/queue/scheduler # Set spinning HDD to bfq echo bfq \u0026gt; /proc/sys/kernel/iosched_bfq # only if module loaded echo bfq \u0026gt; /sys/block/sda/queue/scheduler Permanent change via udev rule:\n1 2 3 4 5 6 7 8 9 10 cat \u0026gt; /etc/udev/rules.d/60-ioscheduler.rules \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; # NVMe SSDs — no scheduler needed ACTION==\u0026#34;add|change\u0026#34;, KERNEL==\u0026#34;nvme[0-9]*n[0-9]\u0026#34;, ATTR{queue/scheduler}=\u0026#34;none\u0026#34; # SATA SSDs — deadline for balanced latency ACTION==\u0026#34;add|change\u0026#34;, KERNEL==\u0026#34;sd*\u0026#34;, ATTR{queue/rotational}==\u0026#34;0\u0026#34;, ATTR{queue/scheduler}=\u0026#34;mq-deadline\u0026#34; # Spinning disks — bfq for fair multi-tenant IO ACTION==\u0026#34;add|change\u0026#34;, KERNEL==\u0026#34;sd*\u0026#34;, ATTR{queue/rotational}==\u0026#34;1\u0026#34;, ATTR{queue/scheduler}=\u0026#34;bfq\u0026#34; EOF The queue/rotational attribute is the kernel\u0026rsquo;s way to distinguish HDDs (rotational=1) from SSDs (rotational=0). The udev rule applies on boot and on hotplug.\nStep 2 — Tuning Block Device Queue Parameters Beyond the scheduler, every device exposes tunables under /sys/block/\u0026lt;dev\u0026gt;/queue/. These four make the biggest difference:\nnr_requests — Queue Depth Controls how many I/O requests the block layer queues before throttling the application. Higher values improve throughput under load but increase per-request latency.\n1 2 3 4 5 # NVMe: keep it low for latency-sensitive workloads echo 128 \u0026gt; /sys/block/nvme0n1/queue/nr_requests # HDD: higher for sequential throughput echo 512 \u0026gt; /sys/block/sda/queue/nr_requests read_ahead_kb — Prefetch Window How many kilobytes the kernel reads ahead on sequential access. Default 128 KB is conservative. For media storage and backup targets, increase it.\n1 2 3 4 5 # Media storage: 4 MB read-ahead echo 4096 \u0026gt; /sys/block/sdb/queue/read_ahead_kb # Boot/OS drives: keep default (128 KB) # Database workloads: 512 KB–1 MB max_sectors_kb — Maximum I/O Size Largest single I/O request the block layer will issue. Modern NVMe drives can handle 2 MB+ IOs. The default is often 1280 sectors (~640 KB). Raise it for sequential workloads.\n1 2 # NVMe backup drives: allow large IOs echo 4096 \u0026gt; /sys/block/nvme1n1/queue/max_sectors_kb nomerges — Disable I/O Merging For NVMe drives where the native controller does merging better than the kernel, you can skip block-layer merging to save CPU.\n1 2 # 0 = merging enabled (default), 2 = completely disabled echo 2 \u0026gt; /sys/block/nvme0n1/queue/nomerges Persist these through the same udev rule, or use a systemd tmpfiles d entry:\n1 2 3 4 w /sys/block/nvme0n1/queue/nr_requests - - - - 128 w /sys/block/nvme0n1/queue/read_ahead_kb - - - - 256 w /sys/block/sda/queue/nr_requests - - - - 512 w /sys/block/sda/queue/read_ahead_kb - - - - 4096 Step 3 — Filesystem Mount Options That Matter Mount options are the highest-impact, lowest-effort tuning you can do. These apply to ext4, xfs, btrfs, and zfs (in different ways).\nnoatime — Skip Access Time Updates Every file read used to write an atime update — a metadata write for every read. noatime eliminates this. Safe to use everywhere unless you rely on atime for mail spools or backup tools.\n1 2 # /etc/fstab — add noatime to every filesystem UUID=xxx-xxx / ext4 defaults,noatime,nodiratime 0 1 For databases or container storage, combine with nobarrier only if you have battery-backed RAID or ZFS with a SLOG.\ncommit= — Dirty Writeback Interval Controls how often the kernel flushes dirty pages to disk. Default is 5 seconds (ext4) or 30 seconds (XFS). Lower = better crash safety. Higher = better sequential throughput.\n1 2 3 4 5 # Database server: flush every 5s defaults,noatime,commit=5 # Backup/media store: flush every 30s for write coalescing defaults,noatime,commit=30 discard — Online TRIM Enable for SSD/NVMe filesystems. Modern kernels use discard=async (ext4) or discard (XFS), which queues TRIM commands without blocking writes.\n1 2 3 4 5 # ext4 on SSD defaults,noatime,discard=async # XFS on SSD defaults,noatime,discard relatime — The Compromise If you need POSIX-compliant access times for some reason, use relatime. It updates atime only when the file was accessed more recently than the last modification — vastly fewer writes than default atime.\nEvery current Linux distribution defaults to relatime. If your fstab uses defaults, check with mount | grep relatime — you might already have it.\nPutting It Together 1 2 3 4 5 6 # /etc/fstab for a Proxmox ZFS dataset (zfs handles its own mount opts) # For ext4 data volume on SSD: UUID=abcd-1234 /mnt/data ext4 defaults,noatime,nodiratime,discard=async,commit=15 0 2 # For XFS backup volume on HDD: UUID=efgh-5678 /mnt/backup xfs defaults,noatime,commit=30 0 2 Apply without reboot:\n1 2 mount -o remount /mnt/data mount -o remount /mnt/backup Step 4 — Monitoring I/O in Real Time You cannot tune what you do not measure. These four commands will diagnose 90% of homelab I/O problems.\niostat — Aggregate Device Stats 1 2 3 4 5 6 7 8 # Every 2 seconds, show device utilization and latency iostat -x 2 # Key columns: # %util — fraction of time device was busy (warning at \u0026gt;90%) # await — average I/O latency in ms (target \u0026lt;5 ms for SSD) # r_await / w_await — read vs write latency separately # avgqu-sz — average queue length (\u0026gt;1 suggests saturation) ioping — Direct Latency Probing ioping bypasses caches and filesystems to test raw device latency.\n1 2 3 4 5 6 7 8 9 10 11 # Install apt install ioping # Test raw device latency (NVMe should be \u0026lt;100 µs) ioping -c 20 /dev/nvme0n1 # Test with 1 MB requests for throughput approximation ioping -c 10 -s 1M /mnt/data/test.img # Watch latency over time (every 1 second) ioping -i 1 -c 60 /dev/sda Target latencies:\nNVMe direct: 50–150 µs SATA SSD direct: 100–400 µs SATA HDD direct: 4–12 ms NFS over 1GbE: 200–600 µs iotop — Per-Process I/O 1 2 3 4 5 6 # Show top I/O consumers, accumulated iotop -oPa # Output example: # TID PRIO USER DISK READ DISK WRITE SWAPIN IO\u0026gt; COMMAND # 2345 be/4 postgres 12.45 M/s 3.21 M/s 0.00 % 15.23 % postgres: writer blktrace — Deep Block-Level Tracing For when you need to trace a specific I/O request through the block layer. Heavyweight, but irreplaceable for debugging.\n1 2 3 4 5 # Trace sda for 5 seconds blktrace -d /dev/sda -o - | blkparse -i - # Or use the friendlier btt to analyze seek patterns btt \u0026lt; trace output Step 5 — Container I/O Throttling with cgroup v2 In a homelab running Docker, your databases, media servers, download clients, and monitoring containers all share the same disks. Without throttling, a runaway transcoder can tank database queries.\nChecking cgroup Version 1 2 3 # cgroup v2 is required for Docker\u0026#39;s --device-*-bps/--device-*-iops stat -fc %T /sys/fs/cgroup/ # Should print \u0026#34;cgroup2fs\u0026#34; Docker Engine 25+ defaults to cgroup v2 on modern kernels. Proxmox 9.x also uses cgroup v2 by default.\nThrottle by Bandwidth (Bytes Per Second) 1 2 3 4 5 6 7 8 9 10 11 12 13 # Limit a Jellyfin or Plex container to 100 MB/s reads, 50 MB/s writes docker run -d \\ --name jellyfin \\ --device-read-bps /dev/sda:100mb \\ --device-write-bps /dev/sda:50mb \\ jellyfin/jellyfin # In Docker Compose v3.8+ services: jellyfin: image: jellyfin/jellyfin device_read_bps: /dev/sda:100mb device_write_bps: /dev/sda:50mb Throttle by IOPS (Operations Per Second) For SSDs where latency matters more than sequential throughput, cap by IOPS instead:\n1 2 3 4 5 6 7 8 9 10 11 12 docker run -d \\ --name postgres \\ --device-read-iops /dev/nvme0n1:5000 \\ --device-write-iops /dev/nvme0n1:3000 \\ postgres:17 # Compose equivalent services: postgres: image: postgres:17 device_read_iops: /dev/nvme0n1:5000 device_write_iops: /dev/nvme0n1:3000 Verify Throttling Is Active 1 2 3 # Check the cgroup IO limits directly cat /sys/fs/cgroup/system.slice/docker-\u0026#34;\u0026lt;container-id\u0026gt;\u0026#34;.scope/io.max # Output: 8:0 rbps=104857600 wbps=52428800 riops=5000 wiops=3000 Use docker stats to confirm actual IO stays below the limit:\n1 2 3 docker stats jellyfin # Block IO column shows MB/s in/out A Practical Throttling Strategy For a typical homelab with one NVMe root drive and one HDD for storage:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 services: # Database — priority access, low latency critical postgres: image: postgres:17 device_write_bps: /dev/nvme0n1:200mb device_read_bps: /dev/nvme0n1:200mb volumes: - pgdata:/var/lib/postgresql/data # Media server — no need for high priority jellyfin: image: jellyfin/jellyfin device_read_bps: /dev/sda:80mb device_write_bps: /dev/sda:40mb volumes: - media:/media # Download client — cap hard to avoid saturating disk transmission: image: ghcr.io/linuxserver/transmission device_read_bps: /dev/sda:50mb device_write_bps: /dev/sda:30mb volumes: - downloads:/downloads Step 6 — Proxmox-Specific I/O Tuning If you run Proxmox, the host handles I/O for every VM and container. Three settings matter most:\nZFS Recordsize For VM storage on ZFS, set recordsize=64K (not the default 128K). VM disk images use 4K–64K blocks. Larger recordsizes waste ARC memory and amplify write latency.\n1 zfs set recordsize=64K rpool/data/vm-disks KVM IO Threads Pin IO threads to dedicated CPU cores to prevent vCPU scheduling from blocking disk operations:\n1 2 # In VM config /etc/pve/qemu-server/\u0026#34;\u0026lt;VMID\u0026gt;\u0026#34;.conf args: -object iothread,id=iothread1 -device virtio-blk-pci,drive=drive0,iothread=iothread1 LXC IO Limits LXC containers support disk I/O throttling natively. Set it in the container resource tab or via CLI:\n1 2 3 4 # Limit LXC container 101 to 50 MB/s writes pct set 101 --dev0 /dev/sda,mp=/storage,acl=1 --dev1 /dev/sdb,mp=/backups,acl=1 # Then apply cgroup limits manually: echo \u0026#34;8:0 rbps=100000000 wbps=50000000\u0026#34; \u0026gt; /sys/fs/cgroup/lxc/101/io.max Verification Checklist Run these benchmarks before and after tuning to confirm improvements:\n1 2 3 4 5 6 7 8 9 10 # Sequential read speed ioping -c 10 -s 1M /mnt/data/testfile # Random 4K IOPS (what matters for databases) fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 \\ --name=test --bs=4k --iodepth=32 --size=1G \\ --readwrite=randwrite --filename=/mnt/data/fio-test # Latency percentile distribution ioping -c 100 -i 0.1 /dev/nvme0n1 Target numbers for a well-tuned NVMe on a modern Linux kernel:\nSequential read: 3000–7000 MB/s (depends on drive) Random 4K write: 50K–200K IOPS Direct latency (ioping): \u0026lt;150 µs P99 Summary Disk I/O tuning in a homelab is a three-act play:\nPick the right IO scheduler for each device — none for NVMe, mq-deadline for SATA SSD, bfq for HDDs, and persist it with a udev rule. Set mount options — noatime everywhere, commit= to control writeback cadence, discard=async for SSDs. Throttle containers with cgroup v2 --device-*-bps and --device-*-iops to keep one noisy service from tanking the whole host. The monitoring commands — iostat -x, ioping, iotop -oPa — will tell you when something is wrong. The knobs in this post will let you fix it.\n","permalink":"https://blog.gntech.me/posts/2026-05-25-linux-disk-io-performance-tuning/","summary":"A practical guide to Linux disk I/O performance tuning for homelab servers — IO scheduler selection for NVMe, SSD, and HDD, mount option optimization, real-time monitoring with iostat and ioping, and container IO throttling with cgroup v2.","title":"Linux Disk I/O Tuning — IO Schedulers, Mount Options, and cgroup Throttling"},{"content":"A homelab with three Proxmox nodes, a dozen LXC containers, and fifty Docker services quickly outgrows local storage. Backups scatter across machines. Media libraries duplicate. Container volumes pin data to specific hosts, making migration painful.\nNFS (Network File System) solves this. A single shared storage pool accessible from every Proxmox host, every LXC container, and every Docker container means your data lives in one place — and you only need to back up one place.\nThis guide walks through deploying NFS shared storage from scratch: server setup on Debian/Ubuntu, Proxmox storage integration, Docker volume mounts, LXC bind mounts, and performance tuning for realistic homelab networks.\nWhy NFS Instead of Samba or iSCSI? For a homelab storage backend, three protocols dominate:\nProtocol Best For Overhead Docker Support Proxmox Integration NFSv4 Linux-to-Linux, concurrent reads Low Native volume driver First-class storage type SMB/CIFS Mixed OS (Windows clients) Moderate Requires CSI driver Additional mount config iSCSI Block-level VMs, databases Low (block) Not supported LVM/block storage only In a Linux-only homelab running Proxmox and Docker, NFSv4 is the natural choice — native kernel client, sub-millisecond latency on modern networks, no additional software, and first-class support in both Proxmox (pvesm) and Docker (volume --driver local --type nfs).\nStep 1: NFS Server Setup on Debian/Ubuntu A minimal VM or LXC container with a large attached disk makes the best NFS server in a Proxmox homelab. Dedicated hardware is better for performance but unnecessary for most home labs.\nInstall and Configure 1 2 3 4 5 6 7 8 9 10 11 # Install NFS server apt update \u0026amp;\u0026amp; apt install -y nfs-kernel-server # Create export directory on your storage pool mkdir -p /srv/nfs/backups mkdir -p /srv/nfs/media mkdir -p /srv/nfs/containers # Set ownership for NFS user mapping chown -R nobody:nogroup /srv/nfs chmod 755 /srv/nfs Configure Exports Edit /etc/exports:\n1 2 3 4 5 6 7 8 9 10 # /etc/exports — NFS exports file # Backups: restricted to Proxmox subnet, synchronous writes /srv/nfs/backups 10.0.20.0/24(rw,sync,no_subtree_check,no_root_squash) # Media: read-mostly, allow cross-subnet /srv/nfs/media 10.0.0.0/16(rw,sync,no_subtree_check,no_root_squash) # Containers: Docker/LXC data, synchronous writes /srv/nfs/containers 10.0.20.0/24(rw,sync,no_subtree_check,no_root_squash) Export options explained:\nrw — read-write access sync — writes acknowledged after disk commit (safer, slightly slower) async — writes acknowledged before disk commit (faster, minor crash risk) no_subtree_check — disables subtree checking, improves reliability no_root_squash — allows root on client to keep root privileges (required for Proxmox backups. For Docker/LXC, use root_squash for security and map UIDs instead) Apply and Verify 1 2 3 4 5 6 7 8 exportfs -ra showmount -e localhost # Output: # Export list for localhost: # /srv/nfs/backups 10.0.20.0/24 # /srv/nfs/media 10.0.0.0/16 # /srv/nfs/containers 10.0.20.0/24 Firewall rule if using UFW:\n1 2 ufw allow from 10.0.0.0/16 to any port nfs ufw reload Secure with NFSv4 Only Modern NFSv4.2 is more secure than v3 — single protocol port, strong auth, and Kerberos support. Disable v3 on the server:\n1 2 3 # /etc/default/nfs-kernel-server RPCNFSDCOUNT=8 RPCMOUNTDOPTS=\u0026#34;--no-nfs-version 3\u0026#34; Restart:\n1 systemctl restart nfs-kernel-server NFSv4 uses TCP port 2049. No more rpcbind portmap surprises.\nStep 2: Add NFS Storage to Proxmox Proxmox supports NFS as a first-class storage backend. Adding an NFS share makes it available for VM/CT backups, ISO templates, container storage, and even live VM disks.\nVia Web UI Navigate to Datacenter → Storage → Add → NFS Fill in: ID: nfs-backups Server: 10.0.20.10 (your NFS server IP) Export: /srv/nfs/backups Content: VZDump backup, ISO image, Container template Click Add Via CLI 1 2 3 4 5 pvesm add nfs nfs-backups \\ --server 10.0.20.10 \\ --export /srv/nfs/backups \\ --content backup,iso,vztmpl \\ --options vers=4.2,hard,timeo=600,retrans=3 Verify 1 2 3 4 5 6 7 pvesm status # Output: # Name Type Status Total Used Available % # nfs-backups nfs active 1904G 342G 1562G 18% # Test a backup to NFS store vzdump 100 --storage nfs-backups --mode snapshot Mount Options for Proxmox NFS Add these to /etc/pve/storage.cfg under the NFS section:\n1 2 3 4 5 6 7 nfs: nfs-backups path /mnt/pve/nfs-backups server 10.0.20.10 export /srv/nfs/backups options vers=4.2,hard,intr,timeo=600,retrans=3,noatime content backup,iso,vztmpl maxfiles 7 hard — retry indefinitely on server outage (preferred for backups; avoid soft for application data because timeouts can surface as I/O errors) intr — allow interrupt of hung operations (Linux 5.x+, signals work with hard mounts) timeo=600 — 60-second timeout before retry retrans=3 — retry 3 times before major timeout noatime — skip access time updates, reduces NFS traffic Step 3: Mount NFS Shares in LXC Containers LXC containers benefit from NFS for shared data that multiple containers access — media libraries, download directories, shared configurations.\nBind Mount Inside a Privileged Container On the Proxmox host, edit the container\u0026rsquo;s config:\n1 2 3 # /etc/pve/lxc/101.conf — add at the end mp0: /srv/nfs/media,mp=/media mp1: /srv/nfs/containers,mp=/data/containers Then inside the container:\n1 2 3 4 # Inside the LXC mount -a # or restart the container df -h /media # 10.0.20.10:/srv/nfs/media 2.0T 342G 1.6T 18% /media For Unprivileged Containers (Best Practice) Unprivileged containers require UID mapping so the container\u0026rsquo;s root maps to a non-root user on the host. Mount the NFS share on the Proxmox host first, then bind-mount it into the container:\n1 2 3 4 5 6 7 # On Proxmox host — add fstab entry echo \u0026#34;10.0.20.10:/srv/nfs/containers /mnt/nfs-container nfs4 noauto,vers=4.2,hard,intr,noatime 0 0\u0026#34; \u0026gt;\u0026gt; /etc/fstab mkdir -p /mnt/nfs-container mount /mnt/nfs-container # Bind mount into unprivileged LXC — /etc/pve/lxc/102.conf mp0: /mnt/nfs-container,mp=/data This avoids UID mapping issues because the mount happens at the kernel level on the host, then is bind-mounted into the container\u0026rsquo;s namespace.\nStep 4: Docker NFS Volume Mounts Docker supports NFS natively through the local volume driver with type: nfs. This is the cleanest way to share storage across Docker hosts without any CSI plugins.\nIn Docker Compose 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 # docker-compose.yml services: jellyfin: image: jellyfin/jellyfin:latest volumes: - media-data:/media - jellyfin-config:/config ports: - \u0026#34;8096:8096\u0026#34; volumes: media-data: driver: local driver_opts: type: nfs4 o: \u0026#34;addr=10.0.20.10,hard,rw,vers=4.2,noatime\u0026#34; device: \u0026#34;:/srv/nfs/media\u0026#34; jellyfin-config: driver: local driver_opts: type: nfs4 o: \u0026#34;addr=10.0.20.10,hard,rw,vers=4.2\u0026#34; device: \u0026#34;:/srv/nfs/containers/jellyfin\u0026#34; With Docker Run 1 2 3 4 5 6 7 8 9 10 11 docker volume create \\ --driver local \\ --opt type=nfs4 \\ --opt o=addr=10.0.20.10,hard,rw,vers=4.2,noatime \\ --opt device=:/srv/nfs/media \\ media-shared docker run -d \\ --name jellyfin \\ -v media-shared:/media \\ jellyfin/jellyfin:latest NFS Volume Options for Docker Use hard mounts for application data. Soft mounts can return I/O errors during outages and can cause data corruption; they are not a general Docker performance recommendation. See the Linux NFS manual.\nOption Purpose Recommended NFSv4 locking Integrated into NFSv4; separate NLM is not used Do not add nolock as a universal Docker requirement hard Retry requests during server outages Application data; plan for blocked I/O during outages vers=4.2 Request NFSv4.2 Only when both endpoints support it noatime Skip access time updates Media/read-only rsize=1048576 Read buffer size (1 MB) 1GbE+ networks wsize=1048576 Write buffer size (1 MB) 1GbE+ networks Full Example: Jellyfin with NFS Media 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 # /opt/jellyfin/docker-compose.yml services: jellyfin: image: jellyfin/jellyfin:latest container_name: jellyfin environment: - PUID=1000 - PGID=1000 - TZ=America/Santo_Domingo volumes: - type: volume source: media target: /media volume: nocopy: true - type: volume source: config target: /config volume: nocopy: true ports: - \u0026#34;8096:8096\u0026#34; - \u0026#34;8920:8920\u0026#34; restart: unless-stopped volumes: media: driver: local driver_opts: type: nfs4 o: \u0026#34;addr=10.0.20.10,hard,rw,vers=4.2,noatime\u0026#34; device: \u0026#34;:/srv/nfs/media\u0026#34; config: driver: local driver_opts: type: nfs4 o: \u0026#34;addr=10.0.20.10,hard,rw,vers=4.2\u0026#34; device: \u0026#34;:/srv/nfs/containers/jellyfin\u0026#34; Step 5: NFS Performance Tuning NFS performance in a homelab depends on three knobs: network, server storage, and mount options.\nrsize/wsize — The Single Biggest Lever NFS read and write buffer sizes default to 1 MB in modern kernels, but older clients may negotiate smaller. Set them explicitly:\n1 2 # /etc/fstab mount 10.0.20.10:/srv/nfs/media /mnt/media nfs4 rsize=1048576,wsize=1048576,noatime,vers=4.2 0 0 On 1GbE networks (125 MB/s theoretical), 1 MB buffers are optimal. On 10GbE, increase to 2 MB:\n1 rsize=2097152,wsize=2097152 Server-Side Tuning 1 2 3 4 5 6 7 8 9 10 11 12 # /etc/sysctl.d/90-nfs.conf # Increase NFS server thread count # Default: 8 threads per CPU core — increase for many concurrent clients # Formula: min(128, max(8, clients * 2)) for most workloads # Network buffer tuning for NFS net.core.rmem_default = 262144 net.core.rmem_max = 16777216 net.core.wmem_default = 262144 net.core.wmem_max = 16777216 # Increase max NFS server threads (RPCNFSDCOUNT in defaults file) Apply:\n1 sysctl -p /etc/sysctl.d/90-nfs.conf NFS Server Thread Count On the NFS server, each nfsd kernel thread handles one concurrent request. For a homelab with fewer than 10 clients, 8 threads is plenty. For 20+ concurrent services hitting the same server:\n1 2 # /etc/default/nfs-kernel-server RPCNFSDCOUNT=32 Storage Backend Matters The NFS server\u0026rsquo;s disk subsystem is the actual bottleneck. ZFS with ARC and L2ARC provides excellent NFS performance:\n1 2 3 4 5 # ZFS tuning for NFS server zfs set recordsize=1M pool/nfs-data zfs set atime=off pool/nfs-data zfs set compression=lz4 pool/nfs-data zfs set xattr=sa pool/nfs-data recordsize=1M — matches NFS rsize, reduces fragmentation atime=off — eliminates access-time writes on every read compression=lz4 — near-zero overhead compression Benchmarking NFS Performance Before and after tuning, benchmark from a client:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 # Sequential write test dd if=/dev/zero of=/mnt/nfs/test bs=1M count=1000 conv=fdatasync # Sequential read test dd if=/mnt/nfs/test of=/dev/null bs=1M count=1000 # Fio random I/O test apt install -y fio fio --name=nfs-test --directory=/mnt/nfs --size=1G \\ --rw=randrw --bs=4K --iodepth=16 --numjobs=4 \\ --runtime=30 --time_based --group_reporting # Quick network throughput test (between server and client) # On server: iperf3 -s # On client: iperf3 -c 10.0.20.10 -t 30 Expected results on 1GbE:\nSequential read: 110–118 MB/s (near wire speed) Sequential write: 100–112 MB/s (sync=slow, async=faster) 4K random read: 8,000–15,000 IOPS (depends on server storage) If sequential throughput is below 80 MB/s on 1GbE, check:\nSwitch or cable negotiation (ethtool eth0) NFS version negotiation (mount | grep nfs — confirm v4.2) rsize/wsize values Server disk utilization (iostat -x 1) Step 6: Backup Strategy for NFS Storage Centralizing storage means you only need to back up one server. NFS volumes are not a backup themselves — the server\u0026rsquo;s disks can fail too.\nAutomated Snapshot with ZFS If the NFS server runs on ZFS:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 #!/bin/bash # /usr/local/bin/nfs-snapshot.sh # Run daily via systemd timer DATASET=\u0026#34;pool/nfs-data\u0026#34; KEEP_DAILY=7 KEEP_WEEKLY=4 zfs snapshot \u0026#34;${DATASET}@nfs-$(date +%Y%m%d-%H%M)\u0026#34; # Prune daily snapshots older than 7 days zfs list -H -o name -t snapshot -r \u0026#34;${DATASET}\u0026#34; | \\ grep \u0026#34;nfs-\u0026#34; | \\ while read snap; do ts=$(echo \u0026#34;$snap\u0026#34; | grep -oP \u0026#39;\\d{8}-\\d{4}\u0026#39;) if [[ $ts \u0026lt; $(date -d \u0026#34;7 days ago\u0026#34; +%Y%m%d-%H00) ]]; then zfs destroy \u0026#34;$snap\u0026#34; fi done Rsync to Off-Site For critical data (Docker configs, databases), rsync from the NFS server to an off-site destination:\n1 2 3 4 5 6 7 #!/bin/bash # /usr/local/bin/nfs-offsite-backup.sh rsync -az --delete \\ --exclude \u0026#39;downloads/\u0026#39; \\ --exclude \u0026#39;cache/\u0026#39; \\ /srv/nfs/containers/ \\ user@offsite-backup.example.com:/backup/nfs-containers/ Common Pitfalls and Solutions Permission Denied on Docker NFS Volumes Problem: Container writes to NFS volume fail with \u0026ldquo;Permission denied\u0026rdquo;.\nFix: Ensure UID/GID in the container matches the NFS server\u0026rsquo;s file ownership. NFS maps root (UID 0) to nobody by default unless you enable no_root_squash on the export.\n1 2 # On NFS server, set ownership for the service\u0026#39;s UID chown -R 1000:1000 /srv/nfs/containers/jellyfin Then in the Docker compose service, set PUID=1000 PGID=1000 matching the server.\nLXC Container Can\u0026rsquo;t Mount NFS Problem: mount.nfs: access denied inside an unprivileged container.\nFix: Mount NFS on the Proxmox host, then bind-mount it into the container via mp0: in the CT config. Unprivileged containers lack the CAP_SYS_ADMIN needed for direct NFS mounts.\nNFS Volume Not Available at Docker Start Problem: Container starts before the NFS server is reachable, volume mount fails.\nFix: Add restart: unless-stopped to the compose service and configure the Docker host to mount NFS volumes early:\n1 2 # /etc/fstab — mount on boot so Docker always sees it 10.0.20.10:/srv/nfs/containers /mnt/nfs-containers nfs4 _netdev,noatime,vers=4.2 0 0 The _netdev option tells systemd to wait for networking before mounting.\nPoor Write Performance with sync Problem: Writes are slow (20 MB/s on 1GbE).\nFix: The export uses sync by default. Switch to async for workloads where a tiny data loss window on crash is acceptable (e.g., media libraries, download directories):\n1 2 # /etc/exports /srv/nfs/media 10.0.0.0/16(rw,async,no_subtree_check,no_root_squash) For databases and critical data, keep sync and add a ZFS SLOG device to absorb the synchronous write overhead.\nFull Reference: NFS Server Checklist 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 # ----- Server Setup ----- apt install nfs-kernel-server mkdir -p /srv/nfs/{backups,media,containers} chown nobody:nogroup /srv/nfs # ----- Exports (review paths and permitted clients first) ----- mkdir -p /etc/exports.d cat \u0026gt; /etc/exports.d/homelab.exports \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; /srv/nfs/backups 10.0.20.0/24(rw,sync,no_subtree_check,no_root_squash) /srv/nfs/media 10.0.0.0/16(rw,async,no_subtree_check,no_root_squash) /srv/nfs/containers 10.0.20.0/24(rw,sync,no_subtree_check,no_root_squash) EOF # ----- Apply ----- exportfs -ra systemctl enable --now nfs-server ufw allow from 10.0.0.0/16 to any port nfs # ----- Client Mount Test ----- mount -t nfs4 -o vers=4.2,noatime 10.0.20.10:/srv/nfs/media /mnt/media df -h /mnt/media # ----- Docker Verify ----- docker volume create --driver local \\ --opt type=nfs4 \\ --opt o=addr=10.0.20.10,hard,rw,vers=4.2 \\ --opt device=:/srv/nfs/containers test-nfs docker run --rm -v test-nfs:/data alpine ls -la /data # ----- Performance Check ----- dd if=/dev/zero of=/mnt/media/test bs=1M count=1000 conv=fdatasync 2\u0026gt;\u0026amp;1 | tail -1 # Expect: \u0026gt;100 MB/s on 1GbE, \u0026gt;500 MB/s on 10GbE Summary Centralized NFS shared storage transforms a multi-node homelab from a collection of isolated machines into a unified cluster where data flows freely between Proxmox, LXC containers, and Docker services:\nStart with a dedicated NFS server — a simple Debian VM or LXC with a ZFS dataset provides all the performance most homelabs need. Use NFSv4.2 exclusively — single port (TCP 2049), better performance, and simpler firewall rules than NFSv3. Add NFS storage to Proxmox for unified VM/CT backup targets and ISO/CT template storage accessible from all nodes. Mount NFS on the Proxmox host then bind-mount into unprivileged LXC containers to avoid UID mapping complexity. Use Docker\u0026rsquo;s native NFS volume driver with driver_opts for per-service NFS mounts — no CSI plugins required. Tune rsize/wsize for your network — 1 MB for 1GbE, 2 MB for 10GbE. Benchmark before and after tuning with dd and fio — if throughput is below 80% of wire speed, something is misconfigured. Back up the NFS server — centralization concentrates risk. ZFS snapshots plus off-site rsync cover both local corruption and physical disaster. A properly configured NFS setup is the backbone of a scalable homelab. It takes an hour to stand up and saves you weeks of scattered storage management over the life of your lab.\n","permalink":"https://blog.gntech.me/posts/2026-05-25-nfs-storage-homelab-proxmox-docker/","summary":"Set up centralized NFS shared storage for your Proxmox homelab with Docker volume mounts, LXC bind mounts, and performance tuning — practical configs for backup storage, media libraries, and container data.","title":"NFS Shared Storage — Centralized Homelab with Proxmox and Docker"},{"content":"Cron has been Linux\u0026rsquo;s go-to scheduler for fifty years. It works. But it also has blind spots: no logging beyond mail spool, no dependency chaining, no catch-up for missed runs, and no integration with the rest of the system\u0026rsquo;s init machinery.\nSystemd timers solve all of this natively. If your homelab runs systemd — and any modern Linux distribution does — you already have a better scheduler installed. This guide covers when and why to replace cron with systemd timers, the exact unit file patterns you need, and real-world homelab examples you can copy today.\nWhy Systemd Timers Beat Cron in a Homelab Cron is simple. A one-liner in a crontab and you are done. But in a homelab with containers, ZFS datasets, Proxmox nodes, and MikroTik backups, your tasks accumulate fast. The cracks start to show:\nCapability cron systemd timer Logging Mail spool or syslog journalctl with structured metadata Missed-run catch-up Task is skipped Persistent=true fires on next boot Random delay Manual sleep hack RandomizedDelaySec= built-in Dependencies None After=, Requires=, BindsTo= Sandboxing None ProtectHome=, PrivateTmp=, etc. Failed-run notification Mail on stderr OnFailure= unit chaining Per-user scheduling crontab -e per user --user systemd instance Timezone-aware Requires CRON_TZ= Respects system timezone Verification Parse visually systemd-analyze calendar For a homelab running 10-20 recurring tasks — container restarts, zfs snapshot rotation, log cleanup, health pings, backup syncs — systemd timers provide observability and reliability that cron simply cannot match.\nAnatomy of a Systemd Timer A timer always comes in a pair: a .timer unit defines when to run, and a .service unit defines what to run. The timer activates the service, the service does the work, and systemd tracks both.\nService Unit 1 2 3 4 5 6 7 # /etc/systemd/system/docker-cleanup.service [Unit] Description=Clean up Docker resources according to the reviewed cleanup script [Service] Type=oneshot ExecStart=/usr/local/bin/docker-cleanup.sh Timer Unit 1 2 3 4 5 6 7 8 9 10 11 # /etc/systemd/system/docker-cleanup.timer [Unit] Description=Run Docker cleanup daily at 3 AM [Timer] OnCalendar=*-*-* 03:00:00 Persistent=true RandomizedDelaySec=30m [Install] WantedBy=timers.target Enable and Start 1 2 3 4 5 systemctl daemon-reload systemctl enable --now docker-cleanup.timer # Verify systemctl list-timers --all Output:\nNEXT LEFT LAST PASSED UNIT ACTIVATES Tue 2026-05-26 03:00:00 EDT 13h left Mon 2026-05-25 03:17:42 EDT 9h ago docker-cleanup.timer docker-cleanup.service Compare that to crontab -l showing 0 3 * * * /usr/local/bin/docker-cleanup.sh. The timer output tells you the next run, last run, and how long ago it ran — all without grepping logs.\nOnCalendar Syntax — The Replacement for Cron Expressions Systemd\u0026rsquo;s OnCalendar syntax is more readable than cron but follows its own pattern: DOW YYYY-MM-DD HH:MM:SS. Missing components act as wildcards.\nCommon Patterns Schedule OnCalendar value Equivalent cron Every day at 2:30 AM *-*-* 02:30:00 30 2 * * * Every 15 minutes *:0/15 */15 * * * * Every hour hourly or *:0 0 * * * * Daily daily or *-*-* 00:00:00 0 0 * * * Weekly (Monday midnight) Mon *-*-* 00:00:00 0 0 * * 1 First day of month 6 AM *-*-1 06:00:00 0 6 1 * * Every 30 minutes *:0/30 */30 * * * * Weekdays at 8 PM Mon..Fri 20:00:00 0 20 * * 1-5 Validate Before Deploying 1 systemd-analyze calendar \u0026#34;Mon..Fri 20:00:00\u0026#34; Output:\nOriginal form: Mon..Fri 20:00:00 Normalized form: Mon..Fri 20:00:00 Next elapse: Mon 2026-05-25 20:00:00 EDT (in UTC): Mon 2026-05-25 20:00:00 EDT From now: 6h left Always run systemd-analyze calendar before deploying a new timer. A typo in the DOW portion does not error — it silently produces an empty schedule and the job never runs.\nMonotonic Timers — Run Relative to Boot or Last Run Not every task needs a wall-clock schedule. Some tasks should run relative to system state:\n1 2 3 4 5 6 7 # /etc/systemd/system/healthcheck.timer [Timer] # Run 5 minutes after boot OnBootSec=5min # Then every 30 minutes after the last run OnUnitActiveSec=30min This is perfect for:\nHealth pings to a monitoring service after the machine boots Temp file cleanup that should run periodically but has no specific preferred wall time Docker container restart policies that need retry logic outside Docker\u0026rsquo;s built-in restart Monotonic timers do not accept OnCalendar — the two families are mutually exclusive in a single timer unit. Use monotonic when the exact wall clock is irrelevant; use OnCalendar when it matters.\nPersistent Timers — Catch Up on Missed Runs The Persistent=true flag is the biggest practical advantage over cron for homelab machines that may not run 24/7.\n1 2 3 [Timer] OnCalendar=daily Persistent=true With cron, if your machine was off at 3 AM, that day\u0026rsquo;s backup simply never runs. With a persistent systemd timer, the service fires immediately on the next boot — even if the machine was off for three days, it catches up on the first boot after the outage.\nPair Persistent=true with RandomizedDelaySec to avoid hammering resources when multiple persistent timers trigger simultaneously after a reboot:\n1 2 3 4 [Timer] OnCalendar=daily Persistent=true RandomizedDelaySec=15m Real-World Homelab Examples 1. ZFS Snapshot Rotation 1 2 3 4 5 6 7 # /etc/systemd/system/zfs-snapshots.service [Unit] Description=Rotate ZFS snapshots [Service] Type=oneshot ExecStart=/usr/local/bin/zfs-snapshot-rotate.sh 1 2 3 4 5 6 7 8 9 10 11 # /etc/systemd/system/zfs-snapshots.timer [Unit] Description=Run ZFS snapshot rotation every 4 hours [Timer] OnCalendar=*-*-* 00/4:00:00 Persistent=true RandomizedDelaySec=5m [Install] WantedBy=timers.target The script (/usr/local/bin/zfs-snapshot-rotate.sh):\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 #!/bin/bash # Keep periodic snapshots for 24 hours and midnight snapshots for 7 days. set -euo pipefail POOL=\u0026#34;tank\u0026#34; now=$(date +%s) zfs snapshot -r \u0026#34;${POOL}@auto-$(date +%Y%m%d-%H%M)\u0026#34; while IFS=$\u0026#39;\\t\u0026#39; read -r snap created; do # Only prune snapshots belonging to this naming scheme. [[ \u0026#34;$snap\u0026#34; =~ @auto-[0-9]{8}-[0-9]{4}$ ]] || continue age=$((now - created)) keep_seconds=86400 if [[ \u0026#34;$snap\u0026#34; =~ -0000$ ]]; then keep_seconds=604800 fi if (( age \u0026gt; keep_seconds )); then zfs destroy \u0026#34;$snap\u0026#34; fi done \u0026lt; \u0026lt;(zfs list -Hp -o name,creation -t snapshot -r \u0026#34;$POOL\u0026#34;) This deletes matching snapshots after their retention period. Review the generated names and retention requirements on a test dataset first. It does not implement weekly retention. Randomized timer delays can prevent an exact 0000 snapshot name; if you require a guaranteed daily tier, schedule it separately instead of relying on this naming convention.\n2. Docker Prune with Health Notification 1 2 3 4 5 6 7 8 9 10 11 12 13 14 # /etc/systemd/system/docker-prune.service [Unit] Description=Prune unused Docker resources Wants=docker.service [Service] Type=oneshot ExecStart=docker image prune -f ExecStartPost=/usr/local/bin/notify-health.sh \u0026#34;Docker prune completed\u0026#34; # Sandboxing — Docker needs the socket but nothing else ProtectSystem=strict PrivateTmp=true NoNewPrivileges=true 1 2 3 4 5 6 7 8 # /etc/systemd/system/docker-prune.timer [Timer] OnCalendar=Sun 03:00:00 Persistent=true RandomizedDelaySec=15m [Install] WantedBy=timers.target The Wants=docker.service dependency ensures the Docker daemon is running before the prune fires. Without this, the exec would fail silently.\n3. Proxmox PBS Garbage Collection For Proxmox Backup Server, garbage collection reclaims space from pruned backup chunks. A weekly GC keeps your datastore healthy:\n1 2 3 4 5 6 7 8 9 # /etc/systemd/system/pbs-gc.service [Unit] Description=Proxmox Backup Server garbage collection After=proxmox-backup.service [Service] Type=oneshot ExecStart=proxmox-backup-manager garbage-collection --output-format text StandardOutput=journal 1 2 3 4 5 6 7 8 # /etc/systemd/system/pbs-gc.timer [Timer] OnCalendar=Sun 04:00:00 Persistent=true RandomizedDelaySec=10m [Install] WantedBy=timers.target 4. Homelab Health Ping 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 # /etc/systemd/system/homelab-healthcheck.service [Unit] Description=Ping Uptime Kuma or similar health endpoint [Service] Type=oneshot ExecStart=/usr/bin/curl -sf -o /dev/null \\ https://health.gntech.dev/ping/homelab-srv1 # Sandbox the curl call ProtectSystem=strict PrivateTmp=true ProtectHome=true NoNewPrivileges=true CapabilityBoundingSet= SystemCallFilter=@basic-io 1 2 3 4 5 6 7 # /etc/systemd/system/homelab-healthcheck.timer [Timer] OnBootSec=2min OnUnitActiveSec=5min [Install] WantedBy=timers.target This runs 2 minutes after boot, then every 5 minutes. Uptime Kuma (or your monitoring tool of choice) alerts you if the ping stops — dead simple, zero agent overhead.\n5. Off-Site Backup Sync 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 # /etc/systemd/system/offsite-backup.service [Unit] Description=Sync backups to off-site storage After=network-online.target Wants=network-online.target [Service] Type=oneshot ExecStart=/usr/local/bin/restic-backup.sh # Security: read-only filesystem except where needed ProtectSystem=full ReadWritePaths=/mnt/backup-cache PrivateTmp=true ProtectHome=true NoNewPrivileges=true 1 2 3 4 5 6 7 8 # /etc/systemd/system/offsite-backup.timer [Timer] OnCalendar=*-*-* 02:00:00 Persistent=true RandomizedDelaySec=20m [Install] WantedBy=timers.target The After=network-online.target ensures the network is actually up before the backup starts — something cron users typically hack with a sleep 30 in the script.\nHardening Timer Service Units Since every timer pair includes a service unit, you get systemd\u0026rsquo;s full sandboxing arsenal for free. These are the most useful options for homelab timer services:\nOption Effect ProtectSystem=strict Read-only /usr, /etc (except whitelisted) ProtectHome=true /root, /home are inaccessible PrivateTmp=true Isolated /tmp per invocation NoNewPrivileges=true Prevents privilege escalation CapabilityBoundingSet= Drops all kernel capabilities SystemCallFilter=@system-service Restricts allowed syscalls MemoryMax=256M Limits memory usage TasksMax=32 Limits number of tasks/processes ReadWritePaths= Explicit writable paths, everything else read-only Example for a script that only needs to write to one backup directory:\n1 2 3 4 5 6 7 8 9 10 11 [Service] Type=oneshot ExecStart=/usr/local/bin/backup.sh ProtectSystem=strict ReadWritePaths=/mnt/backup ProtectHome=true PrivateTmp=true NoNewPrivileges=true CapabilityBoundingSet= MemoryMax=512M TasksMax=16 User Timers — Per-User Scheduling Not all homelab automation runs as root. Systemd supports per-user timers that run under your user\u0026rsquo;s systemd instance:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 # Create user unit directory mkdir -p ~/.config/systemd/user/ # Create the service cat \u0026gt; ~/.config/systemd/user/update-hosts.service \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; [Unit] Description=Update local hosts file from Pi-hole [Service] Type=oneshot ExecStart=/home/gntech/bin/update-hosts.sh EOF # Create the timer cat \u0026gt; ~/.config/systemd/user/update-hosts.timer \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; [Unit] Description=Update hosts file every 6 hours [Timer] OnCalendar=*-*-* 00/6:00:00 Persistent=true [Install] WantedBy=timers.target EOF # Enable for the user session systemctl --user daemon-reload systemctl --user enable --now update-hosts.timer # Verify systemctl --user list-timers Enable lingering so user timers survive logout:\n1 loginctl enable-linger $(whoami) User timers are ideal for:\nUser-level backups (restic repos, git repos) Updating personal dotfiles or hosts files Emailing reports or syncing calendars Any automation that should not run as root Monitoring and Debugging List All Active Timers 1 2 3 4 systemctl list-timers --all # Filter for a specific timer systemctl list-timers --all | grep backup Check Timer Status 1 2 systemctl status docker-cleanup.timer systemctl status docker-cleanup.service # Shows last run result Journal Logs for the Last Run 1 2 3 4 5 6 7 8 # Last 50 lines from the service\u0026#39;s most recent run journalctl -u docker-cleanup.service -n 50 --no-pager # Follow logs in real time journalctl -u docker-cleanup.service -f # Show all runs from today journalctl -u docker-cleanup.service --since today Force a Run (Manual Trigger) 1 systemctl start docker-cleanup.service Running the service unit directly bypasses the timer schedule but still logs through journald — useful for testing.\nTest the Timer Schedule 1 2 systemd-analyze verify /etc/systemd/system/docker-cleanup.timer systemd-analyze calendar \u0026#34;*-*-* 02:30:00\u0026#34; Check for Failures 1 2 3 4 5 # Show failed units from the last 24 hours systemctl list-units --state=failed # Show timer units that missed their schedule journalctl -u docker-cleanup.timer --since \u0026#34;24 hours ago\u0026#34; | grep -i \u0026#34;missed\\|skipped\\|failed\u0026#34; Converting Existing Cron Jobs Migration script to convert crontab entries to systemd timer units is straightforward for simple cases:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 #!/bin/bash # /usr/local/bin/cron2systemd.sh # Usage: cron2systemd.sh \u0026#34;30 2 * * * /usr/local/bin/backup.sh\u0026#34; \u0026#34;backup\u0026#34; EXPR=\u0026#34;$1\u0026#34; NAME=\u0026#34;$2\u0026#34; CRON_REGEX=\u0026#39;([0-9*,/-]+) ([0-9*,/-]+) ([0-9*,/-]+) ([0-9*,/-]+) ([0-9*,/-]+) (.*)\u0026#39; if [[ \u0026#34;$EXPR\u0026#34; =~ $CRON_REGEX ]]; then MINUTE=\u0026#34;${BASH_REMATCH[1]}\u0026#34; HOUR=\u0026#34;${BASH_REMATCH[2]}\u0026#34; DAY=\u0026#34;${BASH_REMATCH[3]}\u0026#34; MONTH=\u0026#34;${BASH_REMATCH[4]}\u0026#34; DOW=\u0026#34;${BASH_REMATCH[5]}\u0026#34; COMMAND=\u0026#34;${BASH_REMATCH[6]}\u0026#34; cat \u0026gt; \u0026#34;/etc/systemd/system/${NAME}.service\u0026#34; \u0026lt;\u0026lt; EOF [Unit] Description=${NAME} (converted from cron) [Service] Type=oneshot ExecStart=${COMMAND} EOF cat \u0026gt; \u0026#34;/etc/systemd/system/${NAME}.timer\u0026#34; \u0026lt;\u0026lt; EOF [Unit] Description=${NAME} timer (converted from cron) [Timer] OnCalendar=${DOW} *-*-${DAY} ${HOUR}:${MINUTE}:00 Persistent=true [Install] WantedBy=timers.target EOF echo \u0026#34;Created ${NAME}.service and ${NAME}.timer\u0026#34; echo \u0026#34;Run: systemctl daemon-reload \u0026amp;\u0026amp; systemctl enable --now ${NAME}.timer\u0026#34; fi This handles the common case. For complex schedules with step values (*/15), you will need to adjust the OnCalendar expression manually.\nWhen to Still Use Cron Systemd timers are not a universal replacement. Keep cron for:\nMinimal containers — Alpine-based Docker images do not run systemd. Use the container\u0026rsquo;s own cron or supervisord.\nLegacy systems — Older distributions with sysvinit or systemd \u0026lt; v236 lack timer features like RandomizedDelaySec.\nSimple one-off scripts — If you need literally one line and will never need logging, cron\u0026rsquo;s @reboot and one-liner syntax is faster to type.\nNon-systemd environments — WSL1, some embedded Linux, or systems with systemd masked.\nFor everything else in a modern homelab — backups, pruning, healthchecks, sync jobs, snapshot rotation — systemd timers provide better observability, reliability, and security with no external dependencies.\nSummary Systemd timers replace cron with a first-class scheduling system that integrates with the rest of your Linux stack:\nEach timer needs two files: a .timer for the schedule and a .service for the command — enable both with systemctl enable --now. Use OnCalendar for wall-clock schedules and OnBootSec / OnUnitActiveSec for monotonic intervals. Always add Persistent=true for tasks that should catch up after downtime (backups, pruning). Add RandomizedDelaySec to prevent multiple timers from hammering resources simultaneously after boot. Sandbox service units with ProtectSystem, PrivateTmp, and NoNewPrivileges — these apply to any systemd unit, not just timers. Use journalctl -u servicename.service for structured logging — no more grepping mail spool or syslog files. Validate schedules with systemd-analyze calendar before deploying — catch typos before they cause silent failures. A homelab running a dozen timer-managed tasks — ZFS snapshots, Docker pruning, PBS garbage collection, health pings, and off-site backups — runs more reliably and observably than the same tasks managed by cron, with no additional software to install.\n","permalink":"https://blog.gntech.me/posts/2026-05-25-systemd-timer-automation-homelab/","summary":"Replace cron with systemd timer units for reliable, observable homelab task scheduling — persistent schedules, real journald logging, monotonic intervals, and hardened service units with practical backup, maintenance, and healthcheck examples.","title":"Systemd Timer Units — Replacing Cron for Homelab Automation"},{"content":"You SSH into your homelab from a coffee shop, manage Proxmox from your phone on cellular data, or need access to a dashboard while traveling. A WireGuard VPN is the simplest, fastest, and most secure way to reach your internal services without exposing them to the public internet.\nWireGuard lives inside the Linux kernel. On Proxmox, you can run it in a lightweight LXC container that consumes ~64 MB of RAM and practically zero CPU when idle. No VM overhead. No heavyweight OpenSSL configuration. One UDP port, three config files, and you\u0026rsquo;re connected.\nThis guide covers the full deployment: creating a Debian 12 LXC, installing WireGuard, configuring the server with NAT for homelab routing, generating client configs, setting up persistence across reboots, and securing the stack with firewall rules and a MikroTik port forward.\nWhy WireGuard in an LXC Instead of a VM WireGuard runs natively in the Linux kernel. An LXC container shares the host kernel, which means WireGuard kernel modules are available without any extra work. The resource profile is dramatically smaller than a VM:\nApproach RAM Disk Boot time Complexity WireGuard in LXC 64-128MB 2 GB ~3 sec Low WireGuard in VM 512+ MB 8+ GB ~20 sec Medium WireGuard on host 0 0 Instant High risk Running WireGuard directly on the Proxmox host is possible but frowned upon — it mixes host networking with VPN routing, and a configuration mistake can lock you out of the host entirely. The LXC approach isolates the VPN into its own network namespace, so a misconfigured WireGuard tunnel never touches the Proxmox host.\nThe only caveat: LXC containers need a privileged container with the tun device and the wireguard kernel module available. We configure this during the container creation step.\nStep 1: Create the Debian 12 LXC Container Create a new container in Proxmox or from the CLI. Privileged mode is required because WireGuard needs access to create the tun interface and set iptables rules.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 # Create the container (adjust storage ID and bridge as needed) pct create 200 \\ local:vztmpl/debian-12-standard_12.7-1_amd64.tar.zst \\ --storage local-zfs \\ --net0 name=eth0,bridge=vmbr0,ip=dhcp,firewall=1 \\ --ostype debian \\ --arch amd64 \\ --cores 1 \\ --memory 128 \\ --swap 0 \\ --hostname wireguard \\ --unprivileged 0 \\ --features keyctl=1,nesting=1,fuse=1 # Start the container pct start 200 # Enter the shell pct enter 200 What each flag does:\n--unprivileged 0 — Creates a privileged container, required for WireGuard to create kernel interfaces --features keyctl=1,nesting=1 — Enables kernel key retention (used by WireGuard internally) and container nesting --memory 128 --swap 0 — WireGuard needs almost no RAM; 128 MB is generous firewall=1 on the network interface enables Proxmox\u0026rsquo;s built-in firewall for this container After entering the container, update the system:\n1 2 apt update \u0026amp;\u0026amp; apt upgrade -y apt install -y curl gnupg wireguard-tools iptables resolvconf Verify the WireGuard kernel module is available:\n1 2 3 4 5 6 7 modprobe wireguard lsmod | grep wireguard # Output should show the module loaded # wireguard 126976 0 # ip6_udp_tunnel 16384 1 wireguard # udp_tunnel 32768 1 wireguard If modprobe wireguard fails, the module may need to be loaded on the Proxmox host first. Exit the container (exit) and run:\n1 2 3 4 # On the Proxmox host modprobe wireguard echo \u0026#34;wireguard\u0026#34; \u0026gt;\u0026gt; /etc/modules pct push 200 /etc/modules /etc/modules Then re-enter and verify again.\nStep 2: Server Configuration Generate the server key pair inside the container:\n1 2 3 4 cd /etc/wireguard umask 077 wg genkey | tee server.key | wg pubkey \u0026gt; server.pub chmod 600 server.key server.pub Create the server config at /etc/wireguard/wg0.conf:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 [Interface] PrivateKey = $(cat /etc/wireguard/server.key) Address = 10.10.10.1/24 ListenPort = 51820 # NAT traffic to the homelab network PostUp = iptables -A FORWARD -i wg0 -j ACCEPT PostUp = iptables -A FORWARD -o wg0 -j ACCEPT PostUp = iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE PostDown = iptables -D FORWARD -i wg0 -j ACCEPT PostDown = iptables -D FORWARD -o wg0 -j ACCEPT PostDown = iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE # Optional: DNS pushed to clients # (use your internal DNS resolver IP here) DNS = 10.0.20.10 # Enable routing IPForward = yes Enable IP forwarding on the container:\n1 2 3 echo \u0026#34;net.ipv4.ip_forward = 1\u0026#34; \u0026gt;\u0026gt; /etc/sysctl.conf echo \u0026#34;net.ipv6.conf.all.forwarding = 1\u0026#34; \u0026gt;\u0026gt; /etc/sysctl.conf sysctl -p Explanation of the routing:\nAddress = 10.10.10.1/24 — This is the VPN subnet. Clients get IPs in this range. The PostUp iptables rules forward VPN traffic to eth0 (your homelab network) and apply NAT masquerade so responses route back through the container. IPForward = yes — Required for the kernel to route traffic between wg0 and eth0. Without the NAT masquerade, clients could reach the container but not other hosts on your homelab (10.0.20.0/24 in this example). The masquerade rewrites the source IP to the container\u0026rsquo;s eth0 address, making replies reachable.\nStep 3: Add a Client Client configs follow the same key generation pattern. Inside the container:\n1 2 3 4 # Generate client keys cd /etc/wireguard wg genkey | tee laptop.key | wg pubkey \u0026gt; laptop.pub chmod 600 laptop.key laptop.pub Append the client peer to the server config:\n1 2 3 4 5 6 7 cat \u0026gt;\u0026gt; /etc/wireguard/wg0.conf \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; [Peer] # laptop PublicKey = $(cat /etc/wireguard/laptop.pub) AllowedIPs = 10.10.10.2/32 EOF Now substitute the actual public key by writing it directly:\n1 2 3 4 5 6 7 cat \u0026gt;\u0026gt; /etc/wireguard/wg0.conf \u0026lt;\u0026lt; EOF [Peer] # laptop PublicKey = $(cat /etc/wireguard/laptop.pub) AllowedIPs = 10.10.10.2/32 EOF Repeat for each client. Use a new /32 address per client. Keep a mapping file:\n1 2 3 4 5 6 7 cat \u0026gt; /etc/wireguard/clients.txt \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; # Client IP Public key # laptop 10.10.10.2 \u0026#34;\u0026lt;key\u0026gt;\u0026#34; # phone 10.10.10.3 \u0026#34;\u0026lt;key\u0026gt;\u0026#34; # tablet 10.10.10.4 \u0026#34;\u0026lt;key\u0026gt;\u0026#34; # server-remote 10.10.10.10 \u0026#34;\u0026lt;key\u0026gt;\u0026#34; EOF Generate the client config file to import on the client device:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # Generate client wg config cat \u0026gt; /etc/wireguard/laptop.conf \u0026lt;\u0026lt; EOF [Interface] PrivateKey = $(cat /etc/wireguard/laptop.key) Address = 10.10.10.2/32 DNS = 10.0.20.10 [Peer] PublicKey = $(cat /etc/wireguard/server.pub) Endpoint = vpn.gntech.dev:51820 AllowedIPs = 10.0.20.0/24, 10.10.10.0/24 PersistentKeepalive = 25 EOF Key details in the client config:\nEndpoint — Your public-facing hostname or IP, plus the UDP port 51820 AllowedIPs — Which networks route through the VPN. Include your homelab subnet (10.0.20.0/24) and the VPN subnet itself (10.10.10.0/24). Add 0.0.0.0/0 for full-tunnel (all traffic routes through home) PersistentKeepalive = 25 — Sends a keepalive every 25 seconds, essential for NAT traversal if your client is behind a carrier-grade NAT For quick client deployment, generate a QR code:\n1 2 apt install -y qrencode qrencode -t ansiutf8 \u0026lt; /etc/wireguard/laptop.conf Scan this directly with the WireGuard mobile app — no file transfer needed.\nStep 4: Enable and Start the Tunnel 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 systemctl enable wg-quick@wg0 systemctl start wg-quick@wg0 systemctl status wg-quick@wg0 # Verify the interface is up wg show # Output should look like: # interface: wg0 # public key: \u0026#34;\u0026lt;server public key\u0026gt;\u0026#34; # private key: (hidden) # listening port: 51820 # # peer: \u0026#34;\u0026lt;laptop public key\u0026gt;\u0026#34; # endpoint: \u0026#34;\u0026lt;client public IP\u0026gt;\u0026#34;:\u0026#34;\u0026lt;ephemeral port\u0026gt;\u0026#34; # allowed ips: 10.10.10.2/32 # transfer: 1.23 KiB received, 4.56 KiB sent Verify routing works:\n1 2 3 4 5 # From the container, ping a client (not connected yet) # From a connected client, ping the homelab # Internal homelab host ping 10.0.20.1 ping 10.0.20.30 # Proxmox host Step 5: Firewall and Port Forwarding WireGuard listens on UDP 51820. You need to forward this port from your edge router to the LXC container\u0026rsquo;s IP address.\nMikroTik Port Forward If your edge router is MikroTik (like the GnTech homelab):\n/ip firewall nat add chain=dstnat \\ protocol=udp \\ dst-port=51820 \\ action=dst-nat \\ to-addresses=10.0.20.200 \\ to-ports=51820 \\ comment=\u0026#34;WireGuard VPN\u0026#34; /ip firewall filter add chain=input \\ protocol=udp \\ dst-port=51820 \\ action=accept \\ comment=\u0026#34;Allow WireGuard\u0026#34; UFW Inside the Container If you use UFW inside the LXC:\n1 2 ufw allow 51820/udp comment \u0026#34;WireGuard VPN\u0026#34; ufw reload Proxmox Firewall (if enabled) 1 2 3 4 5 6 7 8 9 10 11 12 13 # On Proxmox host pct set 200 --net0 name=eth0,bridge=vmbr0,ip=dhcp,firewall=1 # Create/append firewall rules for CT 200 cat \u0026gt; /etc/pve/firewall/200.fw \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; [IN] DROP LOG_INVALID [INET] ACCEPT udp dport 51820 ACCEPT icmp EOF Step 6: iptables Persistence WireGuard\u0026rsquo;s wg-quick runs the PostUp/PostDown rules when the interface comes up, but if the container reboots and iptables rules get flushed before wg-quick starts, there\u0026rsquo;s a brief gap. Install iptables-persistent to lock in the forwarding rules:\n1 apt install -y iptables-persistent During installation it will prompt to save current rules. Answer yes, or save manually after:\n1 2 iptables-save \u0026gt; /etc/iptables/rules.v4 ip6tables-save \u0026gt; /etc/iptables/rules.v6 This ensures the container can still forward packets even if wg-quick hasn\u0026rsquo;t started yet on boot.\nStep 7: Managing Clients at Scale For more than 3-4 clients, managing individual key files gets tedious. Two options:\nOption A: wg-gen-web (Web UI) Deploy wg-gen-web in a separate container or via Docker. It provides a clean web UI for adding/removing clients and generates QR codes.\n1 2 3 4 5 6 7 # Docker compose for wg-gen-web docker run -d \\ --name wg-gen-web \\ -p 8080:8080 \\ -v /etc/wireguard:/data \\ -e WG_INTERFACE=wg0 \\ vx3r/wg-gen-web:latest Security note: Do not expose this to the internet. Access it through your WireGuard VPN or a Tailscale/ZeroTier connection.\nOption B: Scripted Client Creation A simple bash wrapper for client management:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 #!/bin/bash # /usr/local/bin/wg-add-client.sh # Usage: wg-add-client.sh \u0026#34;\u0026lt;client-name\u0026gt;\u0026#34; \u0026#34;\u0026lt;vpn-ip\u0026gt;\u0026#34; CLIENT=$1 IP=$2 cd /etc/wireguard || exit 1 # Generate keys wg genkey | tee \u0026#34;${CLIENT}.key\u0026#34; | wg pubkey \u0026gt; \u0026#34;${CLIENT}.pub\u0026#34; chmod 600 \u0026#34;${CLIENT}.key\u0026#34; \u0026#34;${CLIENT}.pub\u0026#34; # Add to server config cat \u0026gt;\u0026gt; wg0.conf \u0026lt;\u0026lt; EOF [Peer] # ${CLIENT} PublicKey = $(cat \u0026#34;${CLIENT}.pub\u0026#34;) AllowedIPs = ${IP}/32 EOF # Generate client config cat \u0026gt; \u0026#34;${CLIENT}.conf\u0026#34; \u0026lt;\u0026lt; EOF [Interface] PrivateKey = $(cat \u0026#34;${CLIENT}.key\u0026#34;) Address = ${IP}/32 DNS = 10.0.20.10 [Peer] PublicKey = $(cat server.pub) Endpoint = vpn.gntech.dev:51820 AllowedIPs = 10.0.20.0/24, 10.10.10.0/24 PersistentKeepalive = 25 EOF # Reload WireGuard wg syncconf wg0 \u0026lt;(wg-quick strip wg0) echo \u0026#34;Client ${CLIENT} created at ${IP}\u0026#34; echo \u0026#34;Config: /etc/wireguard/${CLIENT}.conf\u0026#34; qrencode -t ansiutf8 \u0026lt; \u0026#34;${CLIENT}.conf\u0026#34; Usage:\n1 2 chmod +x /usr/local/bin/wg-add-client.sh /usr/local/bin/wg-add-client.sh phone 10.10.10.3 Step 8: Testing and Verification From a connected client, run these checks:\n1 2 3 4 5 6 7 8 9 10 11 # Check the tunnel is up ping 10.10.10.1 # Verify LAN access (your internal Proxmox host or a service) curl -I https://10.0.20.30:8006 # Proxmox web UI # Verify DNS resolves through your internal server nslookup pihole.lan # Check transfer stats wg show Back on the server, monitor connections:\n1 2 3 4 5 6 7 # Watch live WireGuard peers watch -n 2 wg show # Check handshake timing — a recent handshake (\u0026lt; 2 min) means # the client is actively connected # Check iptables NAT counters iptables -t nat -L POSTROUTING -v -n If a client can reach the container but not other LAN hosts:\n1 2 3 4 5 6 7 8 9 10 # Verify IP forwarding on the container sysctl net.ipv4.ip_forward # Should return 1 # Check iptables FORWARD chain — ensure it\u0026#39;s not set to DROP iptables -L FORWARD -v -n # If FORWARD policy is DROP, add explicit ACCEPT rules iptables -I FORWARD 1 -i wg0 -j ACCEPT iptables -I FORWARD 1 -o wg0 -j ACCEPT Performance Tuning WireGuard is already fast — kernel-mode encryption with ChaCha20 and Poly1305 is hardware-accelerated on modern x86_64 and ARM CPUs. But a few tweaks apply in the LXC context:\nMTU The default WireGuard MTU of 1420 is conservative. If your homelab has jumbo frame support (MTU 9000 on internal links), you can increase the tunnel MTU:\n1 2 3 4 5 [Interface] PrivateKey = ... Address = 10.10.10.1/24 MTU = 1500 # Up from 1420 default ListenPort = 51820 Calculate MTU as physical MTU minus 80 bytes (60 IP/UDP header\n20 WireGuard overhead). For 9000 MTU internal networks, 8920 works. CPU Pinning If you have many simultaneous VPN clients or high-throughput transfers, pin the container to a specific core:\n1 2 # On Proxmox host echo \u0026#34;0\u0026#34; \u0026gt; /sys/fs/cgroup/cpuset/lxc/200/cpuset.cpus Or in the container config (/etc/pve/lxc/200.conf):\n1 2 lxc.cgroup2.cpuset.cpus: 0 lxc.cgroup2.cpuset.mems: 0 Kernel Module Verification When WireGuard appears slow, verify you\u0026rsquo;re using the kernel module, not the userspace Go implementation:\n1 2 3 4 5 6 7 # Check which implementation is loaded lsmod | grep wireguard # If the module isn\u0026#39;t loaded, wg-quick falls back to # boringtun (a userspace implementation) which is 2-3x slower # Force kernel module modprobe wireguard Backing Up the WireGuard Config The entire WireGuard state lives in /etc/wireguard/. On a Proxmox host, back this up from the host side:\n1 2 3 4 5 6 7 8 9 10 # Copy config out of the container for backup pct pull 200 /etc/wireguard /backup/wireguard-$(date +%F) # Or use a cron job inside the container cat \u0026gt; /etc/cron.daily/wireguard-backup \u0026lt;\u0026lt; \u0026#39;SCRIPT\u0026#39; #!/bin/bash tar czf \u0026#34;/root/wireguard-backup-$(date +%F).tar.gz\u0026#34; /etc/wireguard/ find /root/ -name \u0026#34;wireguard-*.tar.gz\u0026#34; -mtime +30 -delete SCRIPT chmod +x /etc/cron.daily/wireguard-backup If you ever need to restore, just copy the configs back and restart:\n1 systemctl restart wg-quick@wg0 Summary WireGuard in a Proxmox LXC is the ideal VPN solution for homelab remote access:\nCreate a privileged Debian 12 LXC with 128 MB RAM — smaller than any VM alternative Configure WireGuard with key-based authentication, NAT masquerade, and the VPN subnet 10.10.10.0/24 Set up clients with minimal config files or QR codes — no client software needed beyond the official WireGuard apps Forward UDP port 51820 from your edge router (MikroTik, OPNsense, or consumer router) to the LXC IP Enable IP forwarding and iptables persistence for reliable routing across reboots The result: secure, wire-speed access to your entire homelab from any device, anywhere, with under 30 minutes of setup time. No public-facing dashboards, no VPN appliance VMs, no subscription fees — just a container, a config, and a fast UDP tunnel.\n","permalink":"https://blog.gntech.me/posts/2026-05-25-wireguard-vpn-proxmox-lxc/","summary":"Deploy a lightweight WireGuard VPN server in a Proxmox LXC container — configure persistent tunnels, NAT routing, client generation, and firewall rules for secure remote access to your entire homelab network.","title":"WireGuard VPN on Proxmox — LXC Setup Guide for Homelab Remote Access"},{"content":"Many homelab administrators focus on network firewalls, container security, and kernel parameters — but overlook the sandboxing features built into systemd itself. Every service that runs on your Linux host can be locked down with zero code changes, using systemd\u0026rsquo;s security directives.\nThe systemd-analyze security command exposes the current exposure level of every unit on your system, scoring each one on a scale from 0 (safe) to 10 (exposed). By default, most services score 9+. With the directives in this guide, you can push them to 3 or lower.\nThis guide covers the security directives that matter, how to audit your current services, how to apply hardening through drop-in overrides without touching vendor unit files, and real hardened configurations for common homelab daemons like Nginx, Prometheus, and Docker containers.\nEvery command and config works on systemd 250+ (Ubuntu 22.04+, Debian 12+, Fedora 38+) and most will work on systemd 240+ (Ubuntu 20.04+).\nsystemd Security Directives — The Essential Set systemd provide around 40 security-related directives for [Service] sections. Most fall into four categories: filesystem isolation, process isolation, capability reduction, and resource limits. These are the ones that give you the most security per line of config.\nFilesystem Sandboxing 1 2 3 4 5 6 7 8 [Service] ProtectSystem=strict ProtectHome=true PrivateTmp=true PrivateDevices=true ProtectKernelTunables=true ProtectKernelModules=true ProtectControlGroups=true Directive What it does Effect ProtectSystem=strict Makes /usr and /etc read-only. The service can only write to explicitly allowed paths. Blocks malware from modifying system binaries or config. ProtectSystem=full Makes /usr read-only, /etc read-only, /var writable. Good balance for services that need to write logs to /var/log. ProtectHome=true Makes /home, /root, /run/user inaccessible. The process sees empty directories. Prevents reading SSH keys, user documents, and dotfiles. PrivateTmp=true Mounts a private /tmp and /var/tmp namespace. Prevents /tmp race-condition exploits and temp file snooping. PrivateDevices=true Provides a minimal /dev with only null, zero, random, and urandom. Blocks direct hardware access. ProtectKernelTunables=true Makes /sys and /proc/sys read-only. Blocks kernel parameter changes from the service. ProtectKernelModules=true Blocks loading or listing kernel modules. Prevents kernel module escalation. ProtectControlGroups=true Makes cgroup filesystems read-only. Prevents escaping cgroup isolation. Process Isolation 1 2 3 4 5 [Service] NoNewPrivileges=true MemoryDenyWriteExecute=true RestrictSUIDSGID=true LockPersonality=true Directive What it does NoNewPrivileges=true Blocks the process and its children from gaining new privileges via setuid, setcap, or capability-granting syscalls. This is the single most impactful hardening directive. It is essentially non-negotiable. MemoryDenyWriteExecute=true Blocks mmap(PROT_EXEC) on writable memory. Prevents JIT spray and shellcode injection. Breaks some JIT compilers (Python with JIT, V8). RestrictSUIDSGID=true Blocks chmod and chown on setuid/setgid bits. LockPersonality=true Prevents changing the execution domain via personality(). Capability Reduction 1 2 3 [Service] CapabilityBoundingSet= AmbientCapabilities= CapabilityBoundingSet is a whitelist of Linux capabilities the service can ever use. Start with an empty set and add only what the service needs:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # Common minimal sets for different service types # Static file server (Nginx serving static content) CapabilityBoundingSet=CAP_NET_BIND_SERVICE # Reverse proxy that needs privilege ports but nothing else CapabilityBoundingSet=CAP_NET_BIND_SERVICE CAP_NET_RAW # Backup agent that binds sockets and forks CapabilityBoundingSet=CAP_NET_BIND_SERVICE CAP_SYS_PTRACE # Docker daemon (broad capabilities required) CapabilityBoundingSet=CAP_NET_BIND_SERVICE CAP_NET_ADMIN CAP_SYS_ADMIN CAP_SYS_PTRACE To find out what capabilities a specific service needs, run it without CapabilityBoundingSet, then check the journal:\n1 journalctl -u myservice.service -p err | grep -i capability If the service logs something like \u0026ldquo;Operation not permitted\u0026rdquo; at startup, add the missing capability and retry.\nNetwork and Syscall Filtering 1 2 3 4 5 [Service] RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX SystemCallFilter=@system-service SystemCallArchitectures=native IPAddressDeny=any Directive What it does RestrictAddressFamilies Whitelist of socket address families. A service that only connects to localhost doesn\u0026rsquo;t need AF_INET or AF_INET6. SystemCallFilter Whitelist or blacklist of syscalls. @system-service is a systemd-maintained set appropriate for most daemons. SystemCallArchitectures=native Only allow native syscalls. Blocks 32-bit syscalls on 64-bit systems, preventing some compatibility-layer exploits. IPAddressDeny=any Blocks all IPv4 and IPv6 traffic. Pair with IPAddressAllow= to whitelist specific addresses. Resource Limits 1 2 3 4 [Service] MemoryMax=512M TasksMax=128 LimitNOFILE=4096 These prevent a single compromised service from DoSing the host by exhausting memory, processes, or file descriptors.\nAuditing with systemd-analyze security Before hardening anything, establish a baseline. The systemd-analyze security command evaluates every active unit and assigns a numeric exposure level.\n1 2 # Score every service, sorted by exposure (most exposed first) systemd-analyze security --vulnerable | head -20 Example output for a default-service state:\nSERVICE NAME EXPOSURE ~ sshd.service 9.6 SAFE ~ systemd-journald.service 4.6 SAFE ~ prometheus-node-exporter.service 9.3 SAFE ~ nginx.service 9.0 SAFE A score of 9.6 means the service has almost no sandboxing. With hardening, you can bring these down to 2–3.\nTo inspect a single unit in detail:\n1 systemd-analyze security nginx.service This shows exactly which security directives are set (or missing) and why the score is what it is:\nNAME DESCRIPTION EXPOSURE ✗ RootDirectoryOrRootImage=/… Root directory is not changed 0.1 ✗ User=/DynamicUser=… Service runs as root user 0.2 ✗ CapabilityBoundingSet=~CAP_SETUID CAP_SETGID … Service has no capability whitelist 0.1 … Use this output to identify the low-hanging fruit on each service.\nApplying Hardening with Drop-In Overrides Never edit vendor unit files in /lib/systemd/system/. Package updates overwrite them. Instead, use drop-ins:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 # Create a drop-in directory for any service sudo mkdir -p /etc/systemd/system/nginx.service.d/ # Create a hardened override sudo tee /etc/systemd/system/nginx.service.d/hardening.conf \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; [Service] # Filesystem sandboxing ProtectSystem=strict ProtectHome=true PrivateTmp=true PrivateDevices=true ProtectKernelTunables=true ProtectKernelModules=true ProtectControlGroups=true # Process isolation NoNewPrivileges=true MemoryDenyWriteExecute=true RestrictSUIDSGID=true LockPersonality=true # Capabilities — Nginx only needs port binding CapabilityBoundingSet=CAP_NET_BIND_SERVICE CAP_NET_RAW # Syscall filtering SystemCallFilter=@system-service SystemCallArchitectures=native # Network—Nginx needs full network access # (omit RestrictAddressFamilies for reverse proxies) # Resource limits MemoryMax=256M TasksMax=256 LimitNOFILE=16384 EOF Then apply and verify:\n1 2 3 sudo systemctl daemon-reload sudo systemctl restart nginx.service sudo systemd-analyze security nginx.service The score should drop from ~9.0 to ~2.5 on a clean configuration.\nCommon breakage points and fixes:\nIf Nginx fails to start with socket() errors: add CAP_NET_RAW If Nginx can\u0026rsquo;t read SSL certificates from a custom path: add ReadWritePaths=/etc/ssl/private If the service can\u0026rsquo;t write logs: change ProtectSystem=full (not strict) or add ReadWritePaths=/var/log/nginx If MemoryDenyWriteExecute=true breaks a JIT-based tool (Python, Lua, some Node.js addons): remove that line or set it to false ReadWritePaths for selective writes When ProtectSystem=strict is too restrictive, use ReadWritePaths to punch holes in the read-only filesystem:\n1 2 3 4 5 [Service] ProtectSystem=strict ReadWritePaths=/var/lib/prometheus ReadWritePaths=/var/log/prometheus ReadWritePaths=/etc/prometheus This keeps everything else read-only while allowing the service to write to exactly the directories it needs.\nReal-World Hardened Configurations Prometheus Node Exporter (Minimal Score Target: 1.5) Node exporter needs to read /proc, /sys, and /host filesystem paths. It does not need network access to arbitrary hosts, kernel modules, or user home directories.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 # /etc/systemd/system/prometheus-node-exporter.service.d/hardening.conf [Service] ProtectSystem=full ProtectHome=true PrivateTmp=true PrivateDevices=true ProtectKernelTunables=true ProtectKernelModules=true ProtectControlGroups=true NoNewPrivileges=true MemoryDenyWriteExecute=true RestrictSUIDSGID=true CapabilityBoundingSet= RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX SystemCallFilter=@system-service SystemCallArchitectures=native MemoryMax=128M TasksMax=32 Node exporter needs zero capabilities and only the standard syscall set. Its exposure drops to ~1.5.\nDocker Daemon (As Locked As Possible) Docker requires broad capabilities by nature — it manages namespaces, mounts, networking, and cgroups. But you can still clamp down:\n1 2 3 4 5 6 7 8 9 10 11 12 # /etc/systemd/system/docker.service.d/hardening.conf [Service] ProtectSystem=full ProtectHome=true PrivateTmp=true PrivateDevices=true ProtectKernelTunables=true ProtectControlGroups=true NoNewPrivileges=true CapabilityBoundingSet=CAP_NET_BIND_SERVICE CAP_NET_ADMIN CAP_NET_RAW CAP_SYS_ADMIN CAP_SYS_PTRACE CAP_SYS_RESOURCE CAP_DAC_OVERRIDE CAP_CHOWN CAP_FOWNER CAP_SETUID CAP_SETGIP Note: ProtectKernelModules=true and MemoryDenyWriteExecute=true may break Docker\u0026rsquo;s overlay2 storage driver. Test thoroughly.\nPostgreSQL 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 # /etc/systemd/system/postgresql.service.d/hardening.conf [Service] ProtectSystem=full ProtectHome=true PrivateTmp=true PrivateDevices=true ProtectKernelTunables=true ProtectKernelModules=true ProtectControlGroups=true NoNewPrivileges=true MemoryDenyWriteExecute=true RestrictSUIDSGID=true CapabilityBoundingSet= RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX SystemCallFilter=@system-service SystemCallArchitectures=native MemoryMax=4G TasksMax=500 LimitNOFILE=65536 First Steps for Every Homelab If you want the most impact with the least effort, apply these three directives to every service:\n1 2 3 4 [Service] NoNewPrivileges=true PrivateTmp=true ProtectHome=true These three alone drop most services from ~9.5 to ~3.8. They are safe for essentially every daemon — I have yet to find a common service that breaks with only these three.\nFrom there, audit each service with systemd-analyze security and add directives incrementally:\nRun systemd-analyze security \u0026lt;service\u0026gt; and note the issues. Add ProtectSystem=full → test → if it works, move to strict. Add CapabilityBoundingSet= (empty) → test → if it fails, add back capabilities one at a time from the journal errors. Add SystemCallFilter=@system-service → test. Add MemoryDenyWriteExecute=true → test (most likely to break JIT). Scripted Hardening For homelabs with tens of services, hardening each one manually is tedious. This one-liner generates drop-in files for every service on the system with the minimum safe directives:\n1 2 3 4 5 6 7 8 9 10 11 for unit in $(systemctl list-units --type=service --no-legend | awk \u0026#39;{print $1}\u0026#39;); do dir=\u0026#34;/etc/systemd/system/${unit}.d\u0026#34; sudo mkdir -p \u0026#34;$dir\u0026#34; sudo tee \u0026#34;$dir/hardening.conf\u0026#34; \u0026gt; /dev/null \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; [Service] NoNewPrivileges=true PrivateTmp=true ProtectHome=true EOF done sudo systemctl daemon-reload This is a starting point. After applying it, identify the services that break (and they will — some daemons legitimately need root access across the filesystem) and remove the offending directives from their individual drop-ins.\nVerification After hardening, confirm everything is working:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # Check the new exposure levels systemd-analyze security --vulnerable | head -20 # Verify specific services are still running systemctl is-active nginx.service prometheus-node-exporter.service # Check for capability-related errors journalctl -p err -g \u0026#34;capability\\|permission\\|denied\u0026#34; --since \u0026#34;5 minutes ago\u0026#34; # Confirm NoNewPrivileges is effective (should print nothing) for pid in $(systemctl show -p MainPID nginx.service | cut -d= -f2); do grep -l NoNewPrivs /proc/$pid/status done With proper hardening, your systemd services run with the minimum filesystem access, the minimum capabilities, and the minimum syscall surface needed to do their jobs. No root access to your home directory, no writable system binaries, no privilege escalation vectors. And nothing depends on custom tooling — it is all built into the init system that already manages every service on your Linux host.\nApply these changes incrementally, test each one, and within an afternoon you can drop your average service exposure from 9 to 3 without touching a single line of application code.\n","permalink":"https://blog.gntech.me/posts/2026-05-24-systemd-service-hardening-linux/","summary":"Harden Linux systemd services with built-in sandboxing directives — ProtectHome, ProtectSystem, CapabilityBoundingSet, NoNewPrivileges, PrivateTmp, and systemd-analyze security scoring. Includes real hardened service files for common homelab daemons.","title":"Linux systemd Service Hardening — Sandboxing, Capabilities, and Security Auditing"},{"content":"Caddy is the only reverse proxy that gives you automatic HTTPS without plugins, sidecars, or 50 lines of YAML. Point it at a domain, and it provisions a Let\u0026rsquo;s Encrypt certificate, redirects HTTP to HTTPS, sets up OCSP stapling, and renews before expiry — all on first request. No certbot cron, no Traefik middlewares, no Nginx SSL blocks.\nIf you are running a homelab with a dozen Docker services and a public domain, Caddy is the simplest path from http://localhost:8080 to https://app.yourlab.com with an A+ SSL Labs rating.\nThis guide covers a production-grade Caddy 2.11+ deployment in Docker Compose: the compose file, the Caddyfile structure, reusable security headers and compression snippets, zero-downtime reloads, Cloudflare DNS-01 wildcard certificates, HTTP/3, admin API hardening, and a pattern for proxying any number of homelab services with virtually zero configuration per service.\nEvery config works on Docker Engine 24+ with Compose V2 on any Linux host. No Go, no xcaddy, no npm install.\nWhy Caddy Instead of Nginx or Traefik Each reverse proxy has a sweet spot. Here is where Caddy wins:\nFeature Caddy Nginx Traefik Automatic HTTPS Built-in, enabled by default Requires certbot/nginx Requires ACME block + certresolver Config syntax Caddyfile (concise, readable) nginx.conf (verbose) Labels or dynamic YAML Service discovery Manual Caddyfile or Docker labels Manual upstreams Docker provider auto-discovers Memory footprint ~15 MB idle ~3 MB idle ~25 MB idle HTTP/3 Built-in (quic-go) Requires nginx-quic build Built-in Plugin model xcaddy build Third-party modules Provider system For a homelab where you want to add a service with exactly one line and get HTTPS immediately, Caddy is the fastest path. If you need Kubernetes-style auto-discovery, Traefik is stronger. If you need raw throughput on static content, Nginx still edges out both.\nCaddy\u0026rsquo;s killer feature is the Caddyfile: you define a snippet once (security headers, compression, logging), then apply it to every site with import. Adding a new service is a four-line block.\nDocker Compose Stack Create a project directory and three files:\n1 mkdir -p caddy-stack/caddy \u0026amp;\u0026amp; cd caddy-stack compose.yaml 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 services: caddy: image: caddy:2.11-alpine restart: unless-stopped cap_add: - NET_ADMIN ports: - \u0026#34;80:80\u0026#34; - \u0026#34;443:443\u0026#34; - \u0026#34;443:443/udp\u0026#34; volumes: - ./caddy:/etc/caddy - caddy_data:/data - caddy_config:/config environment: - DOMAIN=${DOMAIN:-example.com} - ACME_EMAIL=${ACME_EMAIL:-admin@example.com} networks: - edge # Example services behind Caddy whoami: image: traefik/whoami:latest restart: unless-stopped networks: - edge volumes: caddy_data: caddy_config: networks: edge: driver: bridge Key details in this file:\nNo version: key — The Compose Spec deprecated the top-level version: field in 2023. Docker Compose V2 prints a deprecation warning if you include it. cap_add: NET_ADMIN — Allows quic-go to raise UDP buffer sizes for HTTP/3. Without it, HTTP/3 works but throughput suffers. caddy_data volume — Persists certificates, private keys, and OCSP staples. Treat this as non-ephemeral. If you delete it, Caddy re-issues every certificate from scratch, which triggers Let\u0026rsquo;s Encrypt rate limits. caddy_config volume — Persists the API config (autosaves). Less critical than data, but nice to have. ports: \u0026quot;443:443/udp\u0026quot; — Needed for HTTP/3 (QUIC) over UDP. .env 1 2 DOMAIN=gntech.dev ACME_EMAIL=admin@gntech.dev The Caddyfile Save this as caddy/Caddyfile:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 # Global options block { email {$ACME_EMAIL} admin 127.0.0.1:2019 servers { trusted_proxies 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16 } } # Reusable security headers snippet (security) { header { X-Content-Type-Options \u0026#34;nosniff\u0026#34; X-Frame-Options \u0026#34;DENY\u0026#34; X-XSS-Protection \u0026#34;0\u0026#34; Referrer-Policy \u0026#34;strict-origin-when-cross-origin\u0026#34; Permissions-Policy \u0026#34;camera=(), geolocation=(), microphone=()\u0026#34; -Server } } # Reusable compression snippet (compress) { encode gzip zstd } # Catch-all redirect — any unknown hostname → HTTPS with the base domain {$DOMAIN} { redir https://www.{$DOMAIN}{uri} } www.{$DOMAIN} { redir https://{$DOMAIN}{uri} } # Primary site — reverse proxy to whoami container {$DOMAIN} { import security import compress log { output file /data/access.log { roll_size 100mb roll_keep 5 roll_keep_for 720h } } reverse_proxy whoami:80 } # Subdomain example — any service you add gets its own block api.{$DOMAIN} { import security import compress reverse_proxy whoami:80 } Everything in this file is self-documenting:\nThe (security) snippet is applied to every site block via import security. Add or remove headers in one place, and every site inherits the change. The admin directive binds the admin API to localhost only. More on this below. trusted_proxies tells Caddy which subnets are internal. Set this to your homelab IP ranges so Caddy respects X-Forwarded-For headers from your Docker bridge and any upstream load balancer. The log block writes structured logs to the data volume. Adjust roll_size to your homelab\u0026rsquo;s traffic volume. Zero-Downtime Reloads The Caddyfile is loaded on startup. When you edit the file on the host, the running container does not see the changes until you reload. Caddy supports a graceful, connection-draining reload:\n1 docker compose exec -w /etc/caddy caddy caddy reload This tells the running Caddy process to re-read /etc/caddy/Caddyfile, apply changes atomically, and keep existing connections alive until they complete. There is zero downtime — ongoing requests are not interrupted.\nImportant: The reload only works when /etc/caddy is mounted as a directory (not a single file bind). If you mount ./Caddyfile:/etc/caddy/Caddyfile, the reload command fails because Caddy needs to write temporary files in the config directory.\nIf you script deploys via CI/CD (Gitea Actions, GitHub Actions), add this as the final step:\n1 docker compose exec caddy caddy reload \u0026amp; The \u0026amp; backgrounding avoids timeout issues when the previous connections take more than a few seconds to drain.\nFor fully automated reloads, use inotifywait on the host to watch the Caddyfile and trigger the reload automatically:\n1 2 3 4 # Install inotify-tools once, then run: while inotifywait -e modify /opt/caddy-stack/caddy/Caddyfile; do docker compose exec -w /etc/caddy caddy caddy reload done Admin API Hardening By default, the Caddy admin API listens on localhost:2019. This is the endpoint that caddy reload and the autosave engine talk to. In Docker, localhost inside the container is the container itself, so the default is safe — external hosts cannot reach it.\nHowever, if you ever bind port 2019 in your compose file, you expose the admin API to your Docker network and potentially the LAN. Caddy\u0026rsquo;s admin API has no built-in authentication — anyone who can reach it can:\nRead the entire config (including domain names and upstream addresses) Modify routes dynamically Trigger certificate re-issuance Never expose port 2019 externally. If you need remote access to caddy reload, SSH into the host and run the command locally.\nIf you need to reverse-proxy the admin endpoint (for monitoring purposes), add authentication and IP whitelisting:\n1 2 3 4 5 6 admin.caddy.{$DOMAIN} { basicauth { admin $2a$14$HASHED_PASSWORD } reverse_proxy 127.0.0.1:2019 } Cloudflare DNS-01 Wildcard Certificates The standard Caddy image does HTTP-01 challenges on port 80. This works for most homelab setups, but it has two limitations:\nNo wildcard certificates — HTTP-01 only validates specific hostnames. You need a separate certificate for *.gntech.dev. Port 80 must be open — Not always possible behind NAT or CGNAT. Cloudflare supports DNS-01 challenges, which verify domain ownership through TXT records instead of HTTP. Caddy handles DNS-01 via the tls directive, but you need a custom image with the Cloudflare DNS module.\nCustom Dockerfile 1 2 3 4 5 6 7 8 FROM caddy:2.11-alpine AS builder RUN xcaddy build \\ --with github.com/caddy-dns/cloudflare FROM caddy:2.11-alpine COPY --from=builder /usr/bin/caddy /usr/bin/caddy Build and tag it:\n1 2 docker build -t caddy-cloudflare:2.11 . docker compose build caddy # if you reference it in compose Updated compose.yaml (service override) 1 2 3 4 5 6 services: caddy: build: context: . dockerfile: Dockerfile # ... rest unchanged Caddyfile wildcard block 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 *.{$DOMAIN}, {$DOMAIN} { tls { dns cloudflare {env.CLOUDFLARE_API_TOKEN} } @api host api.{$DOMAIN} handle @api { reverse_proxy api-container:3000 } @app host app.{$DOMAIN} handle @app { reverse_proxy app-container:8080 } # Fallback handle { respond 404 } } Add the Cloudflare API token to .env:\n1 CLOUDFLARE_API_TOKEN=your_api_token_here With this setup, you create one TXT record in Cloudflare, Caddy provisions a single wildcard certificate for *.gntech.dev, and every subdomain automatically gets valid HTTPS without individual challenges. DNS propagation takes 30-60 seconds — Caddy retries automatically.\nAdding New Services To add a new homelab service behind Caddy, three steps:\nAdd a service block in the Caddyfile: 1 2 3 4 5 6 jellyfin.{$DOMAIN} { import security import compress reverse_proxy jellyfin:8096 } Define the container in compose.yaml: 1 2 3 4 5 6 7 8 9 services: jellyfin: image: jellyfin/jellyfin:latest restart: unless-stopped networks: - edge volumes: - jellyfin_config:/config - /path/to/media:/media:ro Reload Caddy: 1 docker compose exec -w /etc/caddy caddy caddy reload That is it. Caddy provisions a certificate for jellyfin.gntech.dev on the first HTTPS request, and from that point on all traffic is encrypted. No certbot, no nginx sites-available symlinks, no Traefik labels on the container.\nProduction Checklist Before putting this stack in front of production-adjacent homelab services, run through these hardening steps:\nDisable HTTP/3 on container-to-container traffic — QUIC over UDP has overhead inside Docker networks. Leave it on for external traffic, but use HTTP/1.1 or H2C for internal reverse_proxy destinations.\nPin the Caddy image tag — Never use :latest in production. Pin to a specific minor: caddy:2.11-alpine.\nUse Docker secrets for the Cloudflare token — The .env file leaks the token into Compose V2\u0026rsquo;s docker compose config output. Use a Docker secret or --env-file with restricted permissions:\n1 chmod 600 .env Rate-limit the Caddy admin API — If you must expose it to a monitoring tool, add rate_limit middleware: 1 2 3 4 5 6 7 8 9 10 @admin_api { path /config/* } rate_limit @admin_api { zone admin_zone { key {remote_host} events 5 window 30s } } Monitor certificate expiry — Caddy auto-renews, but verify with: 1 2 # List all certificates and their expiry dates docker compose exec caddy caddy cert-info Set up log shipping — Caddy writes structured JSON logs. Collect them with Loki or any log aggregator: 1 2 3 # In compose.yaml volumes: - caddy_data:/data Then configure Promtail or Alloy to tail /data/access.log.\nVerification Once deployed, verify everything is working:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 # Check Caddy is running docker compose ps # Check the config is loaded docker compose exec caddy caddy adapt # Test HTTPS curl -vI https://gntech.dev # Check HTTP/3 (requires curl with quiche support) curl --http3 -I https://gntech.dev # Run SSL Labs test (web browser) # https://www.ssllabs.com/ssltest/analyze.html?d=gntech.dev Expected results: A+ rating, TLS 1.3 only, HSTS enabled, HTTP/3 available, no Server header leak.\nCaddy gives you production-grade HTTPS in Docker with dramatically less boilerplate than the alternatives. For a homelab where services come and go and you want every domain to just work with TLS, it is the pragmatic choice. The Caddyfile pattern — snippets, import, one reverse_proxy line per service — scales from a single dashboard to thirty containers without becoming unmanageable.\n","permalink":"https://blog.gntech.me/posts/2026-05-24-caddy-reverse-proxy-docker/","summary":"Set up Caddy as a Docker reverse proxy for your homelab — automatic Let\u0026rsquo;s Encrypt HTTPS with zero config, reusable Caddyfile snippets, zero-downtime reloads, Cloudflare DNS-01 wildcards, HTTP/3 support, and production security hardening.","title":"Caddy Reverse Proxy Docker — Automatic HTTPS and Zero-Downtime Reloads"},{"content":"The MikroTik firewall is one of the most flexible packet filtering engines available at any price point. Built on top of Linux Netfilter, RouterOS gives you four independent firewall tables (filter, NAT, mangle, raw) with connection tracking, stateful inspection, Layer 7 matching, and FastTrack hardware-accelerated forwarding. It\u0026rsquo;s also where most homelab configurations have subtle gaps that leave the router vulnerable.\nThis guide covers every aspect of the MikroTik firewall as it exists in RouterOS 7.x: how packets move through each table and chain, how connection tracking assigns states, when to use FastTrack and when to avoid it, how to build effective address lists for brute force protection, and a complete, practical firewall ruleset you can apply to any RouterOS device.\nEvery command shown here works in RouterOS 7.16+. All examples use the CLI (terminal), but the same rules can be configured through Winbox or WebFig.\nFirewall Tables and Packet Flow RouterOS has four firewall tables, processed in a strict order:\nRaw — Pre-connection tracking filtering. Use this to bypass connection tracking for traffic you don\u0026rsquo;t need to track (bulk transfers, trusted LAN-to-LAN flows). Mangle — Packet marking for routing policy, QoS, and routing table selection. Runs after connection tracking. Filter — Accept or drop packets. The main security rules live here. NAT — Source (srcnat) and destination (dstnat) address translation. Evaluated after filter rules. Each table has up to three built-in chains:\nInput — Packets destined for the router itself (SSH, Winbox, API) Forward — Packets passing through the router between interfaces (LAN → WAN, VLAN → VLAN) Output — Packets originating from the router (DNS queries, NTP syncs) The order matters. Raw processes first, then mangle, then filter, then NAT. A packet dropped in the filter chain never reaches NAT — which is why you place security rules in filter and don\u0026rsquo;t duplicate them in NAT.\n1 2 3 4 5 6 7 8 9 10 11 12 13 Packet arrives ↓ Raw (pre-connection tracking) ↓ Connection tracking (assigns state) ↓ Mangle (mark routing, QoS) ↓ Filter (accept or drop) ↓ NAT (srcnat/dstnat) ↓ Route decision Connection Tracking and Packet States Connection tracking is what makes RouterOS a stateful firewall. Every packet is classified into one of five states:\nState Meaning new First packet of a connection (TCP SYN, or first UDP packet) established Part of an existing, tracked connection related Belongs to a related connection (FTP data, ICMP errors) invalid Not part of any tracked connection — usually malformed or spoofed untracked Explicitly excluded from connection tracking via raw table The standard stateful rule pattern places accept rules for established/related/untracked and a drop rule for invalid at the top of each chain, so only new packets need full rule evaluation:\n/ip firewall filter add chain=input connection-state=invalid action=drop comment=\u0026#34;Drop Invalid Input\u0026#34; add chain=input connection-state=established,related,untracked action=accept comment=\u0026#34;Allow Established/Related/Untracked Input\u0026#34; add chain=forward connection-state=invalid action=drop comment=\u0026#34;Drop Invalid Forward\u0026#34; add chain=forward connection-state=established,related,untracked action=accept comment=\u0026#34;Allow Established/Related/Untracked Forward\u0026#34; This pattern drastically reduces CPU load. Established connections match at rule 1 and skip the remaining rules.\nConnection Tracking Table You can inspect active connections with:\n/ip firewall connection print count-only /ip firewall connection print where protocol=tcp /ip firewall connection print where dst-port=443 If your tracking table grows large (50k+ entries), tune the timeouts:\n/ip firewall connection tracking set tcp-established-timeout=1d /ip firewall connection tracking set udp-timeout=30s /ip firewall connection tracking set generic-timeout=5m For routers with limited RAM, reduce max-connections:\n/ip firewall connection tracking set max-connections=32768 FastTrack — Wire-Speed Forwarding FastTrack is a special handler that bypasses the Linux network stack entirely for established TCP and UDP connections. It enables wire-speed forwarding on capable hardware (ARM64, x86) even at 1 Gbps+ line rates.\nWhen to Use FastTrack FastTrack is safe when you don\u0026rsquo;t need per-connection accounting, queue trees, or IPSec on the same traffic. It\u0026rsquo;s ideal for:\nHome internet routing (LAN → WAN bulk traffic) Media streaming traffic General web browsing When to Avoid FastTrack FastTrack bypasses firewall, connection tracking, simple queues, queue tree with parent=global, IP accounting, and IPSec. Don\u0026rsquo;t FastTrack traffic that needs:\nVLAN-to-VLAN firewall rules on the same router Bandwidth limiting with queue trees Traffic marked for VPN routing Per-connection accounting FastTrack Configuration Place these two rules at the top of the forward filter chain:\n/ip firewall filter add chain=forward action=fasttrack-connection connection-state=established,related comment=\u0026#34;FastTrack Established/Related\u0026#34; add chain=forward action=accept connection-state=established,related comment=\u0026#34;Accept Established/Related\u0026#34; The second rule is required because not all packets in a FastTracked connection go through the fast path — some still hit the slow path and need an explicit accept.\nTo verify FastTrack is active:\n/ip firewall filter print where action=fasttrack-connection /ip firewall connection print where fasttrack=yes count-only Raw Table — Connection Tracking Bypass For traffic that doesn\u0026rsquo;t need stateful inspection at all (trusted LAN backups, rsync, NFS), bypass connection tracking entirely:\n/ip firewall raw add chain=forward src-address=10.0.20.0/24 dst-address=10.0.30.0/24 action=notrack comment=\u0026#34;Bypass CT for LAN inter-vlan\u0026#34; This saves RAM and CPU. The packets show as connection-state=untracked in the filter chain and must be allowed with a matching accept rule.\nSecure Default Filter Ruleset The following ruleset provides a solid foundation. Adjust interface names (ether1, bridge) and trusted subnets to match your setup.\n# =========================================== # Input Chain — Protect the Router Itself # =========================================== /ip firewall filter # 1. Invalid packets → drop immediately add chain=input connection-state=invalid action=drop comment=\u0026#34;Drop Invalid Input\u0026#34; # 2. Established/related → accept add chain=input connection-state=established,related,untracked action=accept comment=\u0026#34;Allow Established/Related/Untracked Input\u0026#34; # 3. Allow ICMP (ping) from anywhere — but rate-limit add chain=input protocol=icmp action=accept comment=\u0026#34;Allow ICMP\u0026#34; \\ icmp-options=8:0 limit=5,5:packet # 4. Allow management from trusted LAN only add chain=input protocol=tcp dst-port=22,8291 src-address=10.0.0.0/16 \\ action=accept comment=\u0026#34;Allow SSH/Winbox from LAN\u0026#34; # 5. Allow API/API-SSL from trusted sources if needed add chain=input protocol=tcp dst-port=8728,8729 src-address=10.0.0.0/16 \\ action=accept comment=\u0026#34;Allow API from LAN\u0026#34; # 6. Allow essential services (DHCP, NTP, DNS) add chain=input protocol=udp dst-port=67,68 action=accept comment=\u0026#34;Allow DHCP\u0026#34; add chain=input protocol=udp dst-port=123 action=accept comment=\u0026#34;Allow NTP\u0026#34; add chain=input protocol=udp dst-port=53 action=accept comment=\u0026#34;Allow DNS\u0026#34; # 7. Drop everything else to the router add chain=input action=drop comment=\u0026#34;Drop All Other Input\u0026#34; # =========================================== # Forward Chain — Traffic Through Router # =========================================== # 1. FastTrack at the top add chain=forward action=fasttrack-connection connection-state=established,related \\ comment=\u0026#34;FastTrack Established/Related\u0026#34; add chain=forward action=accept connection-state=established,related \\ comment=\u0026#34;Accept Established/Related Forward\u0026#34; # 2. Drop invalid add chain=forward connection-state=invalid action=drop comment=\u0026#34;Drop Invalid Forward\u0026#34; # 3. Allow LAN to WAN (srcnat handles masquerade) add chain=forward src-address=10.0.0.0/16 out-interface=ether1 action=accept \\ comment=\u0026#34;Allow LAN to WAN\u0026#34; # 4. Allow related responses back add chain=forward dst-address=10.0.0.0/16 in-interface=ether1 connection-state=established,related \\ action=accept comment=\u0026#34;Allow WAN Established to LAN\u0026#34; # 5. Optional: block inter-VLAN by default, allow selectively # add chain=forward src-address=10.0.10.0/24 dst-address=10.0.20.0/24 action=accept \\ # comment=\u0026#34;Allow IoT to Services VLAN\u0026#34; # add chain=forward action=drop comment=\u0026#34;Drop Inter-VLAN by Default\u0026#34; # 6. Drop everything else forward add chain=forward action=drop comment=\u0026#34;Drop All Other Forward\u0026#34; NAT — Source Masquerade /ip firewall nat add chain=srcnat out-interface=ether1 action=masquerade comment=\u0026#34;Masquerade LAN to WAN\u0026#34; For static IP or multiple WAN IPs, use action=src-nat to-addresses=WAN_IP instead of masquerade.\nAddress Lists for Dynamic Blocking Address lists are the RouterOS equivalent of IP set or groups — named collections of IPs you can reference in firewall rules. They can be static or populated dynamically.\nDynamic Brute Force Protection This is the single most impactful security rule you can add. It tracks connection attempts from each IP and blocks those exceeding a threshold:\n/ip firewall filter # Track new SSH connections from WAN add chain=input protocol=tcp dst-port=22 connection-state=new \\ src-address-list=ssh_blacklist action=drop \\ comment=\u0026#34;Drop SSH from Blacklisted IPs\u0026#34; add chain=input protocol=tcp dst-port=22 connection-state=new \\ action=add-src-to-address-list address-list=ssh_blacklist \\ address-list-timeout=1h \\ comment=\u0026#34;Add SSH Attempt to Blacklist\u0026#34; # Track new Winbox connections add chain=input protocol=tcp dst-port=8291 connection-state=new \\ src-address-list=winbox_blacklist action=drop \\ comment=\u0026#34;Drop Winbox from Blacklisted IPs\u0026#34; add chain=input protocol=tcp dst-port=8291 connection-state=new \\ action=add-src-to-address-list address-list=winbox_blacklist \\ address-list-timeout=1h \\ comment=\u0026#34;Add Winbox Attempt to Blacklist\u0026#34; This only works if SSH/Winbox are already allowed from LAN only — it\u0026rsquo;s defense-in-depth. If you must expose management to WAN, add a connection-rate check:\nadd chain=input protocol=tcp dst-port=22 src-address-list=!allowed_admin \\ connection-state=new connection-rate=10,3 action=add-src-to-address-list \\ address-list=ssh_blacklist address-list-timeout=1h \\ comment=\u0026#34;Rate-limit SSH, moving violators to blacklist\u0026#34; Static Address Lists for Whitelisting Define trusted IP ranges or VPN peers:\n/ip firewall address-list add list=allowed_admin address=10.0.0.10 comment=\u0026#34;Admin Desktop\u0026#34; add list=allowed_admin address=10.0.1.0/24 comment=\u0026#34;Management VLAN\u0026#34; add list=vpn_peers address=172.16.0.0/24 comment=\u0026#34;Wireguard Peers\u0026#34; add list=known_crawlers address=1.1.1.0/24 comment=\u0026#34;Cloudflare DNS\u0026#34; Then reference them in rules:\nadd chain=input protocol=tcp dst-port=22 src-address-list=allowed_admin \\ action=accept comment=\u0026#34;Allow SSH from Admin Only\u0026#34; Layer 7 Filtering RouterOS supports basic Layer 7 (application-layer) regex matching. Use it sparingly — it consumes significant CPU and works only on the first packets.\nBlocking Streaming or P2P /ip firewall layer7-protocol add name=p2p regexp=\u0026#34;^(torrent|.*\\.torrent)\u0026#34; /ip firewall filter add chain=forward layer7-protocol=p2p action=drop \\ comment=\u0026#34;Drop P2P traffic\u0026#34; Layer 7 patterns are matched against the first 2 KB of application data. They won\u0026rsquo;t match encrypted traffic (HTTPS, modern QUIC).\nLogging and Monitoring Enable logging on drop rules during initial deployment to verify rules behave as expected:\n/ip firewall filter add chain=input action=drop comment=\u0026#34;Drop Input (Logged)\u0026#34; \\ log=yes log-prefix=\u0026#34;FW_DROP_INPUT \u0026#34; Check logs:\n/log print where message~\u0026#34;FW_DROP_INPUT\u0026#34; Inspect hit counts per rule:\n/ip firewall filter print stats The stats column shows packet and byte counts for every rule. Use this to identify rules that never match (dead rules) or rules matching too aggressively.\nRule Ordering Checklist Firewall rules in RouterOS are evaluated from top to bottom. First match wins. Follow these ordering rules:\nFastTrack rules first (forward chain only) Connection state accept rules (established/related/untracked) Invalid drop rules Targeted allow rules (specific ports, IPs, services) Address list dynamic rules (rate limiting, blacklisting) Catch-all drop rules Never place blanket accept rules before specific drop rules unless the accept is for established/related traffic only.\nComplete Quick-Start One-Liner If you\u0026rsquo;re starting from a factory reset and want a reasonably secure baseline in one session:\n/ip firewall filter add chain=input connection-state=invalid action=drop add chain=input connection-state=established,related,untracked action=accept add chain=input protocol=icmp action=accept icmp-options=8:0 limit=5,5:packet add chain=input protocol=tcp dst-port=22,8291 src-address=10.0.0.0/16 action=accept add chain=input protocol=udp dst-port=67,68,123,53 action=accept add chain=input action=drop add chain=forward action=fasttrack-connection connection-state=established,related add chain=forward action=accept connection-state=established,related add chain=forward connection-state=invalid action=drop add chain=forward src-address=10.0.0.0/16 out-interface=ether1 action=accept add chain=forward action=drop /ip firewall nat add chain=srcnat out-interface=ether1 action=masquerade Replace 10.0.0.0/16 with your LAN subnet and ether1 with your WAN interface name.\nSummary The MikroTik RouterOS 7 firewall is powerful but demands correct rule ordering and a firm understanding of connection tracking. Start with the stateful accept/drop pattern, add FastTrack for forwarding performance, protect management services with address-list-based dynamic blocking, and always monitor with print stats and logs during initial deployment. The raw table gives you fine-grained control over what gets tracked, and NAT runs after filtering so security rules aren\u0026rsquo;t duplicated.\nA properly configured MikroTik firewall handles 1 Gbps WAN links at under 5% CPU utilization while blocking brute force attempts autonomously — that\u0026rsquo;s the sweet spot for any homelab router.\n","permalink":"https://blog.gntech.me/posts/2026-05-24-mikrotik-firewall-rules/","summary":"Configure a production-ready MikroTik RouterOS 7 firewall — understand filter, NAT, mangle, and raw tables, enable FastTrack for wire-speed forwarding, implement connection tracking state rules, and build brute force protection with dynamic address lists.","title":"MikroTik RouterOS 7 Firewall — Rules, FastTrack, and Connection Tracking"},{"content":"Every Docker Compose user hits the wall eventually. You define POSTGRES_PASSWORD=mydbpass in an .env file, but the container reads an empty string. You try ${VAR:-default} syntax and it doesn\u0026rsquo;t behave the way the documentation suggested. You switch between environments and miss a variable, causing cryptic startup failures.\nThe .env file in Docker Compose looks simple on the surface, but the scoping rules, substitution behavior, and interaction with environment, env_file, and Docker secrets have subtle traps that trip up everyone — including experienced operators.\nThis guide covers everything you need to manage environment variables in Docker Compose: how .env files work, when variables are substituted and when they aren\u0026rsquo;t, the difference between environment and env_file, and the security boundaries between env vars and Docker secrets. Every example uses real docker-compose.yml snippets you can adapt to your homelab.\nHow .env Files Work in Docker Compose Docker Compose reads the .env file from the project directory (the directory containing your docker-compose.yml). Variables defined there are interpolated into the compose file before services are built or started.\n1 2 3 4 5 # Project structure ~/homelab/nginx/ ├── docker-compose.yml ├── .env # ← Compose reads this automatically └── nginx.conf 1 2 3 4 5 6 7 8 # docker-compose.yml services: nginx: image: nginx:${NGINX_VERSION:-alpine} ports: - \u0026#34;${HTTP_PORT:-80}:80\u0026#34; environment: - NGINX_HOST=${DOMAIN:-localhost} 1 2 3 4 # .env NGINX_VERSION=1.27-alpine HTTP_PORT=8080 DOMAIN=web.gntech.dev When you run docker compose up -d, Compose reads .env, substitutes ${NGINX_VERSION}, ${HTTP_PORT}, and ${DOMAIN}, then creates the container with nginx:1.27-alpine, port 8080, and NGINX_HOST=web.gntech.dev.\nWhere Docker Compose Looks for .env The .env file must be in the same directory as the compose file or in the project directory (the parent of the compose file if you use -f). Compose does not search parent directories recursively.\n1 2 3 # These all look for .env in /opt/homelab cd /opt/homelab \u0026amp;\u0026amp; docker compose up -d docker compose -f /opt/homelab/docker-compose.yml up -d 1 2 3 # This does NOT load /opt/homelab/.env docker compose -f /opt/homelab/services/web/compose.yml up -d # .env must be at /opt/homelab/services/web/.env Shell Variables vs .env Precedence When a variable exists in both the shell environment and the .env file, the shell environment wins. This is by design — it lets you override .env values without editing the file:\n1 2 3 # .env has DB_NAME=mydb # Override it for one-off commands DB_NAME=testdb docker compose up -d Variable resolution order (highest to lowest priority):\nShell environment variables .env file variables Compose file hardcoded values Variable default (:-) or error (:?) fallback This means you can chain overrides safely. The compose file defines the structure, .env stores the defaults for your environment, and shell vars handle temporary overrides.\nVariable Substitution Syntax Docker Compose supports the full set of shell-like variable substitution expressions. These work in any string value in docker-compose.yml — image tags, port numbers, volume paths, environment values, labels, and extension fields.\nBasic Substitution 1 2 3 services: app: image: myapp:${TAG} If TAG is undefined, Compose raises an error and refuses to start. Always provide a default or explicitly handle missing variables.\nDefault Values with :- 1 2 3 4 5 services: app: image: myapp:${TAG:-latest} ports: - \u0026#34;${PORT:-3000}:3000\u0026#34; If TAG is not set, Compose substitutes latest. If PORT is not set, it substitutes 3000. This is the safest pattern for optional configuration.\nMandatory Variables with :? 1 2 3 4 5 6 services: postgres: image: postgres:${PG_VERSION:-16-alpine} environment: POSTGRES_PASSWORD: ${DB_PASSWORD:?Database password is required} POSTGRES_DB: ${DB_NAME:?Database name is required} When :? is used and the variable is unset or empty, Compose prints the error message and exits before creating any containers. This catches missing configuration at the parsing stage instead of at container runtime.\nAlternate Value with :+ 1 2 3 4 services: app: environment: - LOG_LEVEL=${VERBOSE:+debug} If VERBOSE is set (to any non-empty value), LOG_LEVEL becomes debug. If VERBOSE is unset, LOG_LEVEL is an empty string. Use this for toggle-style configuration.\nEscape $ for Literal Dollar Signs Docker Compose interprets $ as the start of a variable. To use a literal $ in a value, double it:\n1 2 3 4 5 services: app: environment: - SHELL_CHAR=$$ - CURRENCY=\\$50 The $$ produces a single $ at runtime. The backslash-escape \\$ is recognized by Docker Compose 2.x as an alternative.\nenvironment vs env_file — When to Use Each Docker Compose provides two ways to pass variables into a container. They look similar but behave differently.\nThe environment Block Variables in environment are always passed to the container, regardless of how they\u0026rsquo;re defined:\n1 2 3 4 5 6 7 8 9 services: app: environment: # Hardcoded value MODE: production # From compose interpolation (reads .env or shell) DB_HOST: ${DB_HOST:-postgres} # Pairs syntax (equivalent to above) DB_PORT: ${DB_PORT:-5432} The environment block is interpolated at compose parsing time — ${DB_HOST} is resolved before the container starts. If DB_HOST is not set anywhere, the default postgres is used.\nThe env_file Directive 1 2 3 4 5 services: app: env_file: - ./app.env - ./app.override.env # later files override earlier ones 1 2 3 4 # app.env DB_HOST=postgres DB_PORT=5432 DB_NAME=myapp Variables from env_file are not interpolated by Compose. They are passed verbatim to the container. If app.env contains DB_HOST=${DB_HOST:-postgres}, the container sees the literal string ${DB_HOST:-postgres}, not the resolved value.\nThis is the most common source of confusion. Use env_file for static key-value pairs that don\u0026rsquo;t need interpolation. Use environment when you need variable substitution.\nWhich One to Use Situation Use Static config that never changes environment (hardcoded) Values that differ per environment .env + environment (interpolation) Long lists of variables (10+) env_file Secrets or credentials Docker secrets (see below) Third-party config (Terraform output) env_file generated by script A practical hybrid approach for a homelab stack:\n1 2 3 4 5 6 7 8 # docker-compose.yml services: app: env_file: - ./app.env # static config environment: - DB_HOST=${DB_HOST} # from .env or shell - DB_PORT=${DB_PORT} 1 2 3 # app.env (static, never changes per environment) TZ=America/Santo_Domingo LANG=en_US.UTF-8 1 2 3 # .env (per-environment config) DB_HOST=pg1.gntech.dev DB_PORT=5432 Multiple .env Files and Override Chains Compose 2.x supports loading multiple .env files with the --env-file flag:\n1 2 # Load base.env first, then override with production.env docker compose --env-file ./base.env --env-file ./production.env up -d Later files take precedence. Variables from the shell environment always win over any .env file.\nEnvironment-Specific Compose Files For more complex setups, combine .env with multiple compose files and the -f flag:\n1 2 3 4 5 docker compose \\ -f docker-compose.yml \\ -f docker-compose.prod.yml \\ --env-file .env.prod \\ up -d 1 2 3 4 5 6 7 8 # docker-compose.prod.yml — overrides development defaults services: app: deploy: resources: limits: memory: 512M restart: always The production override file adds restart policies and resource limits without modifying the base compose file. The .env.prod file supplies production-specific variables (database URLs, API keys, domain names).\nProject-Level .env with Multiple Stacks If you run multiple compose stacks in subdirectories, create a .env in each project directory:\n1 2 3 4 5 6 7 8 9 10 11 /opt/homelab/ ├── .env # Top-level (NOT automatically loaded) ├── web/ │ ├── docker-compose.yml │ └── .env # Auto-loaded for web stack ├── monitoring/ │ ├── docker-compose.yml │ └── .env # Auto-loaded for monitoring stack └── database/ ├── docker-compose.yml └── .env # Auto-loaded for database stack If you want a shared .env at the top level, load it explicitly:\n1 2 cd /opt/homelab/web docker compose --env-file ../.env up -d Extension Fields for Dry Variable Reuse Compose extension fields (x-*) let you define reusable variable blocks and interpolate them across services:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 # docker-compose.yml x-logging: \u0026amp;logging driver: \u0026#34;json-file\u0026#34; options: max-size: \u0026#34;${LOG_MAX_SIZE:-10m}\u0026#34; max-file: \u0026#34;${LOG_MAX_FILES:-3}\u0026#34; x-deploy: \u0026amp;deploy restart: unless-stopped deploy: resources: limits: memory: ${MEM_LIMIT:-256M} services: app: image: myapp:${TAG:-latest} ports: - \u0026#34;${APP_PORT:-3000}:3000\u0026#34; environment: - NODE_ENV=${NODE_ENV:-production} \u0026lt;\u0026lt;: [*logging, *deploy] worker: image: myapp:${TAG:-latest} command: worker environment: - NODE_ENV=${NODE_ENV:-production} \u0026lt;\u0026lt;: [*logging, *deploy] All services inherit the same logging config and restart policy from the extension fields. Change LOG_MAX_SIZE or MEM_LIMIT in .env and all services pick it up.\nEnvironment Variables vs Docker Secrets Environment variables are not secrets. Any process that can run docker inspect on a container can read its environment variables. Any process inside the container can read /proc/1/environ. Compose files (and their .env files) are often committed to version control.\nDocker secrets are the correct mechanism for sensitive data:\n1 2 3 4 5 6 7 8 9 10 11 12 # docker-compose.yml services: postgres: image: postgres:16-alpine secrets: - db_password environment: POSTGRES_PASSWORD_FILE: /run/secrets/db_password secrets: db_password: file: ./secrets/db_password.txt Secrets are mounted as tmpfs files at /run/secrets/\u0026lt;name\u0026gt; inside the container. They never appear in docker inspect output, command lines, or environment dumps.\nWhen to Use Each Use environment variables for:\nService names and ports Runtime mode (development vs production) Feature flags and toggles Log levels and debug settings Database names, usernames (but not passwords) Use Docker secrets for:\nDatabase passwords and API tokens TLS private keys and certificates Any value that would cause damage if leaked Credentials for external services (S3, SMTP, Cloudflare) Hybrid Pattern for Legacy Images Some Docker images only accept credentials through environment variables (e.g., POSTGRES_PASSWORD). For these, use the _FILE convention:\n1 2 3 4 5 6 7 services: postgres: image: postgres:16-alpine secrets: - db_password environment: POSTGRES_PASSWORD_FILE: /run/secrets/db_password Postgres detects POSTGRES_PASSWORD_FILE and reads the secret file instead of the environment variable. Many official images support this pattern (*_FILE). Check the image documentation.\nPutting It All Together — Real Homelab Compose Here is a complete example combining every pattern from this post:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 # .env TAG=2026.05 LOG_MAX_SIZE=10m LOG_MAX_FILES=3 MEM_LIMIT=512M APP_DOMAIN=app.gntech.dev APP_PORT=3000 DB_HOST=postgres DB_PORT=5432 DB_NAME=myapp POSTGRES_DB=myapp 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 # docker-compose.yml x-logging: \u0026amp;logging driver: \u0026#34;json-file\u0026#34; options: max-size: \u0026#34;${LOG_MAX_SIZE}\u0026#34; max-file: \u0026#34;${LOG_MAX_FILES}\u0026#34; x-healthcheck: \u0026amp;healthcheck healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;curl\u0026#34;, \u0026#34;-f\u0026#34;, \u0026#34;http://localhost/health\u0026#34;] interval: 30s timeout: 10s retries: 3 start_period: 40s services: traefik: image: traefik:${TAG:-latest} secrets: - cf_api_token environment: - CF_DNS_API_TOKEN_FILE=/run/secrets/cf_api_token ports: - \u0026#34;80:80\u0026#34; - \u0026#34;443:443\u0026#34; volumes: - /var/run/docker.sock:/var/run/docker.sock:ro \u0026lt;\u0026lt;: [*logging, *healthcheck] restart: unless-stopped app: image: myapp:${TAG:-latest} env_file: - ./app.env environment: DB_HOST: ${DB_HOST:?Database host is required} DB_PORT: ${DB_PORT:-5432} DB_NAME: ${DB_NAME:?Database name is required} NODE_ENV: ${NODE_ENV:-production} secrets: - db_password \u0026lt;\u0026lt;: [*logging, *healthcheck] restart: unless-stopped postgres: image: postgres:16-alpine secrets: - db_password environment: POSTGRES_PASSWORD_FILE: /run/secrets/db_password POSTGRES_DB: ${POSTGRES_DB:?Database name required} POSTGRES_USER: ${POSTGRES_USER:-app} volumes: - pgdata:/var/lib/postgresql/data \u0026lt;\u0026lt;: [*logging, *healthcheck] restart: unless-stopped volumes: pgdata: secrets: cf_api_token: file: ./secrets/cf_api_token.txt db_password: file: ./secrets/db_password.txt 1 2 3 4 # secrets/db_password.txt # Not in version control! Add to .gitignore # chmod 600 secrets/db_password.txt V3ryS3cur3P4ssw0rd! This stack uses every pattern:\n.env for environment-specific values Extension fields for shared logging and healthcheck config :? mandatory variable validation for database config :- defaults for ports, log sizes, and tags Docker secrets for database password and Cloudflare API token *_FILE convention for images that only accept env vars env_file for large static config blocks Debugging Variable Substitution When variables don\u0026rsquo;t resolve the way you expect:\n1 2 3 4 5 # Print the final compose config after interpolation docker compose config # This shows every service definition with all variables # resolved to their final values 1 2 3 4 5 6 # Check which variables are set in the shell echo ${TAG:-unset} echo ${DB_HOST:-unset} # View .env file content (excluding secrets) grep -v -f .gitignore .env Use docker compose config as your primary debugging tool. If it shows the wrong value, trace the resolution chain: shell env → .env → compose file defaults.\nSummary Docker Compose variable management has clear rules once you know them:\n.env is auto-loaded from the compose file directory; shell vars override it :- provides defaults, :? enforces required variables, :+ enables toggle patterns environment is interpolated at parse time; env_file is passed verbatim to the container Use --env-file for loading .env from custom paths or chaining multiple files Extension fields (x-*) let you define reusable variable-driven config blocks Docker secrets for credentials; _FILE convention for images that only support env vars docker compose config shows the fully resolved config for debugging Start every new compose stack with a .env file for environment-specific values, :? for mandatory variables, Docker secrets for credentials, and extension fields for shared config. Your future self — and anyone else deploying your stack — will thank you.\n","permalink":"https://blog.gntech.me/posts/2026-05-24-docker-compose-env-variables/","summary":"Master Docker Compose environment variables — understand .env file scoping, variable substitution syntax with defaults and errors, env_file vs environment blocks, and when to use Docker secrets over env vars for sensitive data in homelab deployments.","title":"Docker Compose Environment Variables — .env, Substitution, and Secrets"},{"content":"Your Proxmox host runs ten LXCs — a Pi-hole, a database container, a reverse proxy, Authentik, Gitea, and six more. Everything works until one container decides to consume all available RAM or peg every CPU core. Suddenly your DNS starts timing out, the reverse proxy returns 502s, and the database container gets OOM-killed before the culprit even notices.\nThis is the noisy-neighbor problem in virtualization. Proxmox provides per-container resource limits out of the box, but most homelab setups leave them at defaults — which effectively means no limits at all.\nThis guide covers every LXC resource control available in Proxmox VE 8.x+: CPU pinning and cgroup CPU quotas, memory and swap caps, disk I/O throttling, and ZFS storage quotas. Every config includes the Proxmox CLI command, the UI path, and runtime verification.\nWhy Default LXC Resource Limits Aren\u0026rsquo;t Enough When you create an LXC in Proxmox, the defaults are:\n1 2 3 4 5 CPU units: 1024 CPU limit: 0 (unlimited) Memory: unlimited Swap: unlimited Disk: size of the rootfs volume The only real constraint is the rootfs disk size. CPU and memory are shared freely. On a busy host with 16 GB RAM and eight LXCs, a single misbehaving container can exhaust memory and trigger the OOM killer. The kernel picks a victim — and it\u0026rsquo;s rarely the one you wanted killed.\nProxmox exposes LXC resource controls through two layers:\ncgroups (v1 and v2) — kernel-level CPU, memory, and I/O limits enforced by the container runtime Storage quotas — ZFS, LVM, or directory-level size restrictions on the root filesystem The commands in this guide use pct set, which maps directly to cgroup parameters. All configs persist across container restarts and host reboots.\nCPU Resource Controls Proxmox LXCs support three CPU controls: CPU units (weight), CPU limits (ceiling), and CPU pinning (affinity).\nCPU Units — Relative Weight CPU units set proportional weight, not a hard cap. A container with 2048 units gets roughly twice the CPU time of a container with 1024 units when both are busy. When only one container is active, it can burst to 100%.\n1 2 3 4 5 # Set CPU weight to 2048 (double the default of 1024) pct set 100 --cpulimit 2048 # Set CPU weight to 512 (half the default) pct set 101 --cpulimit 512 The default of 1024 is reasonable for general-purpose containers. Reserve higher weights for latency-sensitive services (DNS, reverse proxy, databases). Assign lower weights to batch or testing containers.\nUse case CPU units Latency-critical (DNS, proxy, auth) 2048 Standard workload (Gitea, dashboards) 1024 Background/batch (backup tools, sync) 512 Testing/experimental 256 CPU Limit — Hard Cap CPU limit (cpulimit) enforces a hard ceiling as a percentage of a single core. A limit of 50 means the container cannot exceed 50% of one core — or 400 for four full cores.\n1 2 3 4 5 # Limit container 100 to 2 CPU cores pct set 100 --cpulimit 2 # Limit to 50% of one core pct set 101 --cpulimit 0.5 CPUs across all containers on the host should sum to roughly the total available cores × 1.0-1.5 overhead. On an 8-core host with ten containers, keep each under 6-8 cores to allow headroom for Proxmox itself.\nCPU Pinning — Affinity CPU pinning locks a container to specific physical cores. This prevents cache thrashing when containers migrate between cores and ensures consistent performance for real-time workloads.\n1 2 3 4 # Pin container 100 to CPU cores 0 and 1 pct set 100 --cpulimit 2 --cpuunits 2048 # Pin via cgroup directly on the host echo \u0026#34;0-1\u0026#34; \u0026gt; /sys/fs/cgroup/cpuset/lxc/100/cpuset.cpus Alternatively, set CPU affinity in the container\u0026rsquo;s config file:\n1 2 3 # /etc/pve/lxc/100.conf lxc.cgroup2.cpuset.cpus: 0-1 lxc.cgroup2.cpuset.mems: 0 When to pin:\nDatabase containers that benefit from dedicated L2/L3 cache Real-time audio or video processing Containers with hard real-time requirements When not to pin: Most general workloads. Pinning reduces scheduler flexibility and can leave cores idle while other containers wait.\nVerify CPU assignments:\n1 2 3 4 5 # Check per-container CPU usage from the host pct exec 100 -- cat /proc/stat | grep cpu # View cgroup CPU stats cat /sys/fs/cgroup/cpu,lxc/100/cpu.stat Memory and Swap Limits Memory limits prevent a single container from starving the host. Proxmox enforces memory caps through cgroup memory controllers.\nHard Memory Limit Set a hard cap that the container cannot exceed. When reached, the container\u0026rsquo;s processes start getting OOM-killed rather than the host deciding victims.\n1 2 3 4 5 # Limit container 100 to 2 GB RAM pct set 100 --memory 2048 # Limit to 512 MB pct set 101 --memory 512 If you set --memory 0 (default), the container can use all host memory. On a 16 GB host with eight containers, that means any single one can eat 14+ GB before the OOM killer fires.\nSwap Limit Swap works differently in LXCs than in VMs. The swap limit controls the amount of swap the container can use on the host.\n1 2 3 4 5 # Set 512 MB swap maximum for container 100 pct set 100 --swap 512 # Disable swap entirely for a container pct set 100 --swap 0 Most homelab containers don\u0026rsquo;t need swap. Databases should never swap. Disable swap for critical LXCs:\n1 2 3 # Disable swap inside the container too pct exec 100 -- swapoff -a echo \u0026#39;swapoff -a\u0026#39; \u0026gt;\u0026gt; /etc/cron.d/disable-swap Practical Memory Budget for a Homelab Host On an 8-core, 32 GB RAM Proxmox host with ten LXCs:\n1 2 3 4 5 6 7 8 9 10 11 12 13 Container | RAM | Swap | Notes DNS (Pi-hole) | 256M | 0 | Minimal footprint Reverse proxy | 256M | 0 | Traefik or Nginx Auth (Authentik) | 2G | 512M | Python-heavy, needs room Database (Postgres)| 4G | 0 | Never swap databases Gitea | 1G | 256M | Monitoring stack | 2G | 512M | Prometheus + Grafana File syncing | 512M | 256M | Syncthing or Nextcloud Backup tools | 512M | 256M | Restic, rclone Dev/test LXC | 1G | 512M | Media (Jellyfin) | 4G | 0 | ffmpeg needs RAM | | | Host reserve | 4G+ | | Proxmox + ZFS ARC Sum allocated: ~16 GB. The host reserve of 4+ GB covers ZFS ARC, Proxmox services, and the kernel. Leave at least 20% headroom.\nMemory Hot-Plug for LXCs Proxmox supports live memory changes on running containers without restart:\n1 2 3 4 5 # Increase memory to 4 GB on a running container pct set 100 --memory 4096 # Reduce back to 2 GB (if current usage is under the limit) pct set 100 --memory 2048 This works because cgroup limits are dynamic. The container kernel sees available memory shrink but handles it gracefully unless it was already using more than the new limit.\nDisk I/O Throttling CPU and memory limits prevent compute contention, but disk I/O is often the real bottleneck. Ten containers fighting over a single NVMe drive can cause latency spikes even if CPU is idle.\nIOPS Limits Set read and write IOPS limits per container:\n1 2 3 4 5 # Limit container 100 to 500 read IOPS, 200 write IOPS pct set 100 --iolimit \u0026#34;read=500,write=200\u0026#34; # Remove I/O limits pct set 100 --iolimit 0 This uses cgroup I/O controllers (io.max v2 or blkio.throttle v1). The limit applies to the container\u0026rsquo;s rootfs device.\nBandwidth Limits Cap read and write throughput by bytes per second:\n1 2 3 4 5 # Limit container 100 to 50 MB/s read, 20 MB/s write pct set 100 --iobwlimit \u0026#34;read=52428800,write=20971520\u0026#34; # Remove bandwidth limits pct set 100 --iobwlimit 0 Always round to reasonable values for your storage hardware:\nDrive type Sequential read Sequential write Random IOPS NVMe (Gen 4) 5000 MB/s 4000 MB/s 500K+ SATA SSD 500 MB/s 450 MB/s 80K HDD (7200 RPM) 180 MB/s 150 MB/s 150 ZFS mirror (NVMe) 3000 MB/s 2000 MB/s 300K+ For ZFS pools, leave at least 20% I/O headroom for the ARC flush, ZIL writes, and scrub operations.\nI/O Priority by Container Type A practical priority scheme for a 7-container homelab on NVMe:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # Database — high priority, needs consistent I/O pct set 101 --iolimit \u0026#34;read=200000,write=100000\u0026#34; pct set 101 --iobwlimit \u0026#34;read=2147483648,write=1073741824\u0026#34; # Media server — high reads, moderate writes pct set 102 --iolimit \u0026#34;read=100000,write=50000\u0026#34; pct set 102 --iobwlimit \u0026#34;read=4294967296,write=536870912\u0026#34; # General services — moderate everything pct set 103 --iolimit \u0026#34;read=50000,write=25000\u0026#34; pct set 103 --iobwlimit \u0026#34;read=1073741824,write=268435456\u0026#34; # Backup containers — dampen impact on other services pct set 104 --iolimit \u0026#34;read=10000,write=5000\u0026#34; pct set 104 --iobwlimit \u0026#34;read=268435456,write=134217728\u0026#34; The backup container is the key one to restrict. During nightly backups, restic and rclone can saturate the disk and degrade everything else. A 10K read / 5K write IOPS limit and ~250 MB/s read / 125 MB/s write bandwidth cap keeps backups slightly slower but prevents disruption.\nZFS Quotas and Refquota For containers on ZFS storage — the default Proxmox configuration — two quota types apply: quota limits the total space including snapshots, while refquota limits only the container\u0026rsquo;s active data (excluding snapshots).\n1 2 3 4 5 # Set a 50 GB quota (including snapshots) pct set 100 --quota 50 # Set a 40 GB refquota (snapshots don\u0026#39;t count) pct set 100 --refquota 40 Use refquota for containers where snapshot churn could fill the volume. A Gitea container with 10 GB of git data and 50 GB of weekly snapshots should have at least a 20 GB refquota plus enough quota headroom for snapshots:\n1 pct set 103 --refquota 20 --quota 60 This container can use 20 GB of active data. Snapshots can accumulate up to 40 GB before hitting the 60 GB quota wall — which gives you time to prune old snapshots.\nView ZFS Quota Usage 1 2 3 4 5 6 7 8 # Check ZFS usage for a container dataset zfs get all rpool/data/subvol-100-disk-0 | grep -E \u0026#34;quota|refquota|used|available\u0026#34; # Output: # rpool/data/subvol-100-disk-0 quota 50G local # rpool/data/subvol-100-disk-0 refquota 20G local # rpool/data/subvol-100-disk-0 used 12G - # rpool/data/subvol-100-disk-0 available 38G - Resize ZFS Disk on the Fly Need to grow a container\u0026rsquo;s disk? Proxmox resizes the ZFS subvolume online:\n1 2 3 4 5 # Grow container 100 rootfs to 80 GB pct resize 100 rootfs 80G # Shrink (dangerous — requires unmount) pct resize 100 rootfs 40G --shrink Shrinking ZFS volumes requires the filesystem inside to be smaller than the target. Always pct exec 100 -- df -h first to verify actual usage.\nComprehensive Resource Profile Combine everything into a single profile for a container:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 # Container 101 — Postgres database pct set 101 \\ --cpulimit 4 \\ --cpuunits 2048 \\ --memory 4096 \\ --swap 0 \\ --iolimit \u0026#34;read=200000,write=100000\u0026#34; \\ --iobwlimit \u0026#34;read=2147483648,write=1073741824\u0026#34; \\ --refquota 30 \\ --quota 80 # Container 102 — Jellyfin media server pct set 102 \\ --cpulimit 6 \\ --cpuunits 1024 \\ --memory 4096 \\ --swap 0 \\ --iolimit \u0026#34;read=100000,write=50000\u0026#34; \\ --iobwlimit \u0026#34;read=4294967296,write=536870912\u0026#34; \\ --refquota 50 \\ --quota 100 # Container 105 — Backup worker (restic/rclone) pct set 105 \\ --cpulimit 2 \\ --cpuunits 256 \\ --memory 512 \\ --swap 256 \\ --iolimit \u0026#34;read=10000,write=5000\u0026#34; \\ --iobwlimit \u0026#34;read=268435456,write=134217728\u0026#34; \\ --refquota 10 \\ --quota 30 The backup container profile is the most restrained: low CPU weight so it yields to everything else, minimal IOPS to avoid disrupting database and media reads, and small RAM since backup operations are streaming, not memory-bound.\nMonitoring Resource Pressure After setting limits, verify they work under load:\n1 2 3 4 5 6 7 8 9 10 11 12 # Watch container CPU and memory in real time pct top # Check container I/O stats from the host cat /sys/fs/cgroup/io.stat/lxc/101 # View cgroup memory pressure cat /sys/fs/cgroup/memory/lxc/101/memory.pressure # Inside the container: check if limits are binding pct exec 101 -- free -h pct exec 101 -- cat /proc/meminfo | grep -i swap The memory.pressure file shows PSI (Pressure Stall Information). A some avg10 value over 50 means the container is frequently waiting on memory. Raise the limit or investigate the workload.\nFor CPU pressure:\n1 cat /sys/fs/cgroup/cpu/lxc/101/cpu.pressure A full avg10 over 10 indicates the container is CPU-bound and hitting its cgroup ceiling. Consider raising --cpulimit or pinning the container to dedicated cores.\nProxmoz UI vs CLI — When to Use Each Most resource limits can be set from the Proxmox web UI:\nContainer → Resources → Add (or double-click an existing row):\nMemory, Swap, CPU units, CPU limit — all available in the UI I/O limits — available in the UI under \u0026ldquo;IO Limit\u0026rdquo; and \u0026ldquo;IO Bandwidth\u0026rdquo; CPU pinning — not available in the UI, must use CLI or edit lxc.cgroup2 in /etc/pve/lxc/\u0026lt;CTID\u0026gt;.conf For bulk operations across containers, use the CLI:\n1 2 3 4 # Apply standard limits to multiple containers for ctid in 101 102 103; do pct set $ctid --memory 2048 --swap 256 --cpulimit 2 done Or better, script it with a config file:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 #!/bin/bash # /usr/local/bin/apply-lxc-limits.sh declare -A LIMITS LIMITS[100]=\u0026#34;--memory 512 --swap 0 --cpulimit 1\u0026#34; LIMITS[101]=\u0026#34;--memory 4096 --swap 0 --cpulimit 4\u0026#34; LIMITS[102]=\u0026#34;--memory 4096 --swap 0 --cpulimit 6\u0026#34; LIMITS[105]=\u0026#34;--memory 512 --swap 256 --cpulimit 2\u0026#34; for ctid in \u0026#34;${!LIMITS[@]}\u0026#34;; do if pct status $ctid \u0026amp;\u0026gt;/dev/null; then echo \u0026#34;Setting limits for CT $ctid...\u0026#34; pct set $ctid ${LIMITS[$ctid]} fi done Run this from a cron job or Ansible to enforce consistency.\nSummary A methodical approach to LXC resource management prevents the noisy-neighbor problem in dense Proxmox homelabs:\nCPU — Set cpuunits for proportional weight; use cpulimit for hard caps on noisy containers; pin only for databases and real-time workloads Memory — Always set a hard --memory limit on every container; disable swap for databases and latency-sensitive services I/O — Cap backup workers to 10K IOPS to prevent nightly contention; leave database and media containers with higher limits Disk — Use refquota for active data limits and quota for total-with-snapshots; resize from CLI with pct resize Start with conservative limits: 2 GB RAM, 2 CPU cores, 50 GB disk for most containers. Monitor pressure indicators with pct top and cgroup PSI files. Adjust upward only when the container shows genuine need.\nEvery container gets a hard limit. No container gets to steal resources from the rest of your lab.\n","permalink":"https://blog.gntech.me/posts/2026-05-24-proxmox-lxc-resource-management/","summary":"Master Proxmox LXC resource management — configure CPU pinning with cgroups, set memory and swap limits, throttle disk I/O, and apply ZFS quotas to prevent noisy-neighbor contention in your homelab containers.","title":"Proxmox LXC Resource Management — CPU, Memory, and Disk I/O Limits"},{"content":"A backup strategy starts with the failure you want to survive. A second directory on the same disk helps little when that disk fails. An off-site copy is useful only if its credentials, metadata, and encryption keys are available during recovery.\nThis article helps you choose copies and recovery goals. For installation and datastore configuration, use the PBS setup and first restore guide.\nVerification status Area Evidence and limits Technical review Reviewed against the official documentation linked below on October 31, 2026. Policies Schedules and retention counts are planning examples, not measured results. Runtime No backup, sync, disk formatting, or guest restore was executed for this revision. Required evidence Test a restore from each copy you intend to depend on, including saved encryption keys. 1. Set two recovery targets Recovery point objective (RPO) is how much recent work you can lose. Recovery time objective (RTO) is how long the service can remain unavailable.\nWrite these down per service before choosing schedules:\nService Example RPO Example RTO Recovery dependency Lab application 24 hours One day Guest backup and application secrets Important documents Four hours Four hours Data backup, permissions, and encryption key DNS or identity service Set from your outage tolerance Set from your outage tolerance Network access and offline recovery credentials These are proposed targets. Time a real restore to see whether your hardware and network meet them. Application data may need database-native backups in addition to guest backups.\n2. Count copies and failure boundaries Use the 3-2-1 rule as a planning baseline: three copies including the working data, two media types, and one copy off-site. Also inspect shared hosts, power, storage controllers, and administrator credentials.\nCopy Example location What to check Working data PVE guest disks This is the source, not an extra backup. Local backup PBS on separate hardware and storage Can it survive loss of the PVE host? Off-site backup Second PBS at another site Can you restore if the first site and its credentials are unavailable? Two datastores are not automatically two independent copies. Creating a second empty datastore does not duplicate the first. A PBS VM on its protected PVE host can support a lab exercise, but shares that host\u0026rsquo;s failure risks.\nAn offline removable copy can reduce exposure to compromised online credentials. Plan secure storage, rotation, and a recovery test for it.\n3. Choose archive storage or PBS PVE\u0026rsquo;s backup tooling can target file storage or PBS. PBS integration is a storage choice within PVE\u0026rsquo;s backup workflow.\nDestination Useful when Planning tradeoff File storage such as an NFS backup target You want conventional backup archives Budget space and transfer time for full archives. PBS datastore You want deduplication, incremental transfer, and scheduled integrity checks Operate PBS, its maintenance jobs, and recovery credentials. A PBS snapshot is a complete restore point even when only changed chunks transfer. Measure savings with your own guests; there is no fixed deduplication ratio.\nUse the PVE backup documentation for backup modes. VM and container stop, suspend, and snapshot modes behave differently. Live backup and guest-agent filesystem freezing do not by themselves prove database consistency. Check which disks and container mount points are included.\nIn PVE Datacenter → Backup, first create a job for one disposable guest and an explicit target storage. Run it manually and inspect the log. Then choose a calendar schedule using the interface\u0026rsquo;s schedule selector and preview. Do not paste a Unix cron expression into a field expecting Proxmox calendar syntax.\n4. Keep retention separate from cleanup An example starting policy is 7 daily, 4 weekly, and 3 monthly restore points. Adjust it for change rate, recovery needs, and measured capacity.\nConfigure retention in the appropriate PVE backup job or PBS prune job. Avoid conflicting policies that remove restore points another policy was meant to keep.\nFor PBS, pruning removes snapshot metadata; garbage collection later reclaims eligible unreferenced chunks after its grace period. Verification checks integrity. These are separate tasks, and none replaces booting a restored guest. See PBS maintenance.\nKeep capacity headroom and review job failures. A nightly schedule is only a plan until task results show successful backups.\n5. Make the off-site copy recoverable For a second PBS, use a supported sync job. A straightforward design is a pull job on the destination PBS:\nRegister the source PBS as a remote on the destination, verifying its TLS identity. Select the source datastore and destination datastore, with appropriate access permissions. Run the job manually on a small test backup and inspect the destination contents. Decide the destination\u0026rsquo;s own retention policy. Leave removal of vanished source snapshots disabled initially so source deletion is not automatically mirrored. Restore from the destination before enabling its regular schedule. Follow PBS remotes and sync for your installed version. Restrict source access to the permissions required by the sync job. Keep destination administration credentials separate from the PVE backup account.\nCopying only .chunks/ omits the metadata needed to identify and restore snapshots. Treat ad hoc copying of a changing datastore as an unproven recovery method. Prefer a documented PBS workflow, and test the resulting copy.\nFor removable storage, use the installed release\u0026rsquo;s datastore management guidance. Identify the device before creating a filesystem, verify its mount state, and complete tasks before unmounting. This strategy guide intentionally provides no disk-formatting command.\n6. Test recovery without touching the original Select a known snapshot in PVE\u0026rsquo;s backup list and restore it to an unused guest ID with automatic startup disabled. Before booting, disconnect its network interfaces or attach them to an isolated test network.\nCheck the expected files and applications. Record the source snapshot, destination storage, installed versions, elapsed time, and result. Include a restore from the off-site copy and, if encrypted, recovery using the separately saved key.\nOnly remove the disposable restored guest after confirming its identity. Automate this exercise after a manual run works and the automation explicitly checks target IDs, isolation, failure handling, and cleanup. A boot or guest-agent response alone does not verify application data.\n7. Keep a recovery worksheet outside the cluster Record:\nHow to reach backup storage when normal DNS, identity, or VPN services are down. PBS configuration recovery, storage attachment, and account recreation steps. Encryption-key locations and required recovery secrets. Guest restore order based on dependencies. The last successful backup, off-site sync, and restore-test dates. For a host loss, recover access to the backup copy first, prepare replacement PVE capacity, and restore services in dependency order. For site loss, repeat the exercise using only the off-site materials. Update RTO estimates from the time this actually takes.\nContinue reading PBS setup and first restore — Configure the backup service. Intel AMT Serial over LAN — Prepare another route to a host console. Proxmox bridges and VLANs — Document the network needed during recovery. Application backups alongside guest backups For PostgreSQL application data, follow the atomic dump, scheduler, and Restic-to-database recovery lab. For ordinary prepared files, use the Restic file recovery guide. Their disposable Docker results do not validate a PBS sync, VM restore, or production application; keep those recovery tests distinct.\n","permalink":"https://blog.gntech.me/posts/2026-05-23-proxmox-backup-strategies/","summary":"Plan backup copies around host and site failures, then prove recovery with an isolated restore exercise.","title":"Proxmox Backup Strategies — Vzdump, PBS, and 3-2-1 for Homelabs"},{"content":"Storage benchmarks answer a specific workload question. A cached read result, a queue-depth-32 random test, and a database commit latency are different measurements.\nStart with a modest file test on the intended filesystem. Record conditions before changing hardware or tuning properties.\nVerification status Area Evidence and limits Technical review fio, Docker and OpenZFS references reviewed on October 31, 2026. Static checks Bash syntax and result-path structure checked. Runtime No fio workload, storage modification, NFS test or container benchmark executed for this revision. Scope Proposed buffered, single-job filesystem baseline; not a raw-device or durable-transaction benchmark. 1. Identify the path and workload On Debian or Ubuntu:\n1 2 3 4 sudo apt update sudo apt install fio sysstat python3 fio --version lsblk -o NAME,SIZE,TYPE,MOUNTPOINTS,FSTYPE,MODEL Confirm this is the Flexible I/O Tester, not another executable with the same name. Choose an existing writable test directory on the desired mount, with free space and no production data that the test should touch.\nAvoid a default /tmp target: it might be memory-backed or on a different disk. For external mounts, verify the expected source is actually mounted before creating test files. Run during an agreed maintenance window; even a file-based workload competes with other users.\nWrite down the question: sequential file transfer, random reads, application commits, or concurrent clients. The small demo below measures only the first two under buffered conditions.\n2. Create a bounded baseline script The script requires an explicit directory. It writes a 256 MiB preparation file in a newly created subdirectory, then reads that file. It never accepts a raw block-device target.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 cat \u0026gt; filesystem-baseline.sh \u0026lt;\u0026lt;\u0026#39;SCRIPT\u0026#39; #!/usr/bin/env bash set -euo pipefail target=${1:?Usage: filesystem-baseline.sh EXISTING_TEST_DIRECTORY} test -d \u0026#34;$target\u0026#34; test -w \u0026#34;$target\u0026#34; target=$(realpath -- \u0026#34;$target\u0026#34;) findmnt -T \u0026#34;$target\u0026#34; df -h \u0026#34;$target\u0026#34; run_dir=$(mktemp -d \u0026#34;$target/fio-baseline.XXXXXX\u0026#34;) printf \u0026#39;Run directory: %s\\n\u0026#39; \u0026#34;$run_dir\u0026#34; { date -u uname -r fio --version findmnt -T \u0026#34;$target\u0026#34; df -h \u0026#34;$target\u0026#34; } \u0026gt; \u0026#34;$run_dir/context.txt\u0026#34; fio --name=prepare --ioengine=psync --direct=0 --iodepth=1 \\ --rw=write --bs=1M --size=256M --numjobs=1 \\ --refill_buffers=1 --end_fsync=1 \\ --filename=\u0026#34;$run_dir/test.bin\u0026#34; \\ --output-format=json --output=\u0026#34;$run_dir/prepare.json\u0026#34; fio --name=sequential-read --ioengine=psync --direct=0 --iodepth=1 \\ --rw=read --bs=128k --size=256M --numjobs=1 \\ --readonly --allow_file_create=0 --invalidate=0 \\ --filename=\u0026#34;$run_dir/test.bin\u0026#34; \\ --output-format=json --output=\u0026#34;$run_dir/sequential-read.json\u0026#34; fio --name=random-read --ioengine=psync --direct=0 --iodepth=1 \\ --rw=randread --bs=4k --size=256M --numjobs=1 \\ --runtime=15 --time_based --readonly --allow_file_create=0 --invalidate=0 \\ --filename=\u0026#34;$run_dir/test.bin\u0026#34; \\ --output-format=json --output=\u0026#34;$run_dir/random-read.json\u0026#34; python3 - \u0026#34;$run_dir\u0026#34; \u0026lt;\u0026lt;\u0026#39;PY\u0026#39; import json, pathlib, sys root = pathlib.Path(sys.argv[1]) for name in (\u0026#34;prepare\u0026#34;, \u0026#34;sequential-read\u0026#34;, \u0026#34;random-read\u0026#34;): data = json.loads((root / (name + \u0026#34;.json\u0026#34;)).read_text()) jobs = data[\u0026#34;jobs\u0026#34;] if not jobs or any(job[\u0026#34;error\u0026#34;] != 0 for job in jobs): raise SystemExit(\u0026#34;fio job error: \u0026#34; + name) print(\u0026#34;All result files parsed and report zero job errors.\u0026#34;) PY printf \u0026#39;Retained test file and results in %s\\n\u0026#39; \u0026#34;$run_dir\u0026#34; SCRIPT chmod 700 filesystem-baseline.sh read -r -p \u0026#39;Existing test directory on the intended filesystem: \u0026#39; benchmark_dir ./filesystem-baseline.sh \u0026#34;$benchmark_dir\u0026#34; Review free space before running; the script does not enforce a capacity threshold. The 15-second read phase is bounded in time, while preparation and flush duration depend on storage. Stop and investigate errors rather than disabling durability or changing production properties to obtain a number.\nThe preparation write uses fresh buffers rather than a zero-filled dd stream. The read jobs cannot create missing test files. Each result is one JSON document; context text stays separate.\nThe synchronous engine runs one request at a time here. Raising iodepth with this engine would not create the same concurrency as an asynchronous engine. See the fio reference for option semantics.\n3. Read results without inventing expectations This is deliberately buffered and cache-aware: the preparation may warm caches, and the read working set is small. Read throughput may therefore describe memory or filesystem cache behavior rather than physical disk speed. It is not a cold-cache hardware comparison.\nInspect the saved JSON and verify units. Bandwidth, operation rate, and completion latency describe different aspects of the test. Percentiles are nested in the latency structure produced by the installed fio version; do not assume invented p50/p99 top-level keys.\nRecord errors and variability alongside headline throughput. Compare repeated runs with the same file size, cache policy, engine, block size, concurrency and background load. Keep preparation results separate from steady-state claims.\nDo not convert one random-I/O rate directly into application INSERT latency or promise a universal throughput target for media streams. Validate representative application behavior too.\n4. Adapt by storage type Path Additional evidence to record Local filesystem Device, filesystem, mount options, free space and competing workloads ZFS Pool layout, properties, ARC behavior, dataset versus zvol, and installed OpenZFS version NFS Actual mount source/options, client and server load, network path and server cache Docker bind mount or volume Backing filesystem, mount type, cgroup limits, image and tools used Direct I/O support and behavior depend on the filesystem and version. Do not assume direct=1 bypasses every cache, especially across ZFS or network storage. Keep buffered and supported direct-I/O tests as separately labelled experiments.\nFor ZFS, inspect properties before tuning. Do not change sync, recordsize or compression on a live production dataset as a benchmark step. The sync property defaults to standard, not always. See OpenZFS properties.\nDocker named volumes and bind mounts are distinct from the container writable layer; they do not inherently traverse overlay2. Use a new disposable volume for a container comparison, not a running database\u0026rsquo;s storage. Compare equivalent backing storage and conditions. See Docker volumes.\n5. Diagnose before buying or tuning Watch host and storage activity during a separate run, for example with iostat, and correlate it with application latency and errors. Investigate mount selection, thermal behavior, queueing and competing jobs before blaming the disk model.\nBenchmarking is not disk-health validation or a backup test. Review the disk-health guide and protect recoverable data before stressing suspect storage.\nRetain JSON results and context for comparison. When finished, remove only the exact disposable run directory printed by the script after inspecting its contents. No raw-device write, global cache drop, pool-property change or automatic deletion is included.\nContinue reading ZFS snapshots and recovery — Understand dataset and guest-storage boundaries. Docker Compose patterns — Document mounts and resource constraints. ","permalink":"https://blog.gntech.me/posts/2026-05-23-linux-storage-benchmarking/","summary":"A small file-based baseline with explicit target selection and machine-readable results.","title":"Linux Storage Benchmarking — A Bounded fio Filesystem Baseline"},{"content":"You have mastered Docker Compose. Your homelab runs a dozen stacks behind Traefik, your backups are automated, and monitoring covers every container. But managing twenty individual Compose files across multiple hosts is starting to hurt. Rolling out an update means SSH into the right VM, pulling the right stack, and praying nothing conflicts.\nK3s — Rancher\u0026rsquo;s CNCF-certified lightweight Kubernetes distribution — solves this. It strips Kubernetes down to a single binary under 100 MB, replaces etcd with SQLite (or embedded etcd for HA), and runs comfortably on 1 GB RAM VMs. It is Kubernetes, not a toy: every standard kubectl command, Helm chart, and Kubernetes resource works as expected.\nThis guide walks through deploying a production-grade K3s cluster on Proxmox VMs, adding longhorn for persistent storage, MetalLB for bare-metal load balancing, Traefik for ingress, and migrating your Docker Compose workloads into Kubernetes.\nWhy K3s Over Full Kubernetes or Docker Swarm Full Kubernetes (kubeadm, kube-spawn) assumes three control plane nodes with etcd, which wastes resources in a homelab. Docker Swarm is simpler but lacks the ecosystem — no Helm, no CRDs, limited CNI choices.\nK3s hits the sweet spot: it speaks real Kubernetes API, supports every standard workload, and runs on a single VM if that\u0026rsquo;s all you have. Rancher reports over 200,000 production K3s deployments in edge and IoT environments. If it runs factory robots and oil rig sensors, it handles your homelab.\nResource comparison for a three-node cluster:\nPlatform RAM per node Disk Binary size kubeadm (etcd) 2–4 GB 20 GB+ 500+ MB K3s (SQLite) 512 MB–1 GB 10 GB ~90 MB K3s (embedded etcd, HA) 1–2 GB 15 GB ~90 MB For a homelab, a three-node cluster with embedded etcd (HA) or a single server with SQLite plus two agents (agents only) is the sweet spot.\nPrerequisites and VM Provisioning on Proxmox Start with three Proxmox VMs running Ubuntu Server 24.04 LTS (minimal install, no Docker — K3s bundles containerd):\nNode Role vCPU RAM Disk k3s-srv1 Server (control plane) 2 2 GB 20 GB k3s-agent1 Agent (worker) 2 4 GB 40 GB k3s-agent2 Agent (worker) 2 4 GB 40 GB Create them with cloud-init or manually. The minimum viable setup is a single server and one agent, but three nodes give you tolerance for Longhorn replication.\nNetwork Setup Assign static IPs on your management VLAN. These examples use the 10.0.30.x/24 subnet:\nk3s-srv1: 10.0.30.10 k3s-agent1: 10.0.30.11 k3s-agent2: 10.0.30.12 Enable promiscuous mode on the Proxmox vNIC if you plan to use MetalLB in L2 mode — K3s nodes will respond to ARP for service IPs on that interface.\n1 2 3 4 # On each Proxmox host, for each K3s VM network interface: # Hardware → select NIC → enable \u0026#34;VLAN aware\u0026#34; and \u0026#34;Promiscuous\u0026#34; # Or via CLI: qm set \u0026#34;\u0026lt;VMID\u0026gt;\u0026#34; --net0 virtio,bridge=vmbr0,firewall=0 Installing the K3s Server Node SSH into k3s-srv1 and run the one-line install:\n1 2 3 4 5 6 curl -sfL https://get.k3s.io | sh -s - server \\ --cluster-init \\ --disable=traefik \\ --disable=servicelb \\ --node-ip=10.0.30.10 \\ --flannel-iface=eth0 Flags explained:\n--cluster-init — Start the cluster with embedded etcd (HA mode). Omit for single-node SQLite mode. --disable=traefik — We install Traefik ourselves via Helm for full control. K3s bundles a minimal Traefik by default. --disable=servicelb — Disable K3s\u0026rsquo;s built-in LoadBalancer. We use MetalLB instead. --node-ip — Pin the node IP so cluster traffic uses the correct interface. --flannel-iface — Tell Flannel (the CNI) which interface to use for pod networking. After the install completes, copy the node token — you need it for agent nodes:\n1 sudo cat /var/lib/rancher/k3s/server/node-token Save this token. Also copy the kubeconfig for local access:\n1 2 3 4 5 mkdir -p ~/.kube sudo cp /etc/rancher/k3s/k3s.yaml ~/.kube/config sudo chown $USER:$USER ~/.kube/config sed -i \u0026#39;s/127.0.0.1/10.0.30.10/\u0026#39; ~/.kube/config kubectl get nodes You should see k3s-srv1 in Ready status.\nJoining Agent Nodes On k3s-agent1 and k3s-agent2, run the agent install with the server\u0026rsquo;s IP and token:\n1 2 3 4 curl -sfL https://get.k3s.io | K3S_URL=https://10.0.30.10:6443 \\ K3S_TOKEN=\u0026#34;\u0026lt;YOUR_TOKEN\u0026gt;\u0026#34; sh -s - agent \\ --node-ip=10.0.30.11 \\ --flannel-iface=eth0 Repeat for agent2 with --node-ip=10.0.30.12.\nBack on the server, verify all nodes joined:\n1 kubectl get nodes -o wide Expect output like:\nNAME STATUS ROLES AGE VERSION k3s-srv1 Ready control-plane,master 5m v1.32.3+k3s1 k3s-agent1 Ready \u0026lt;none\u0026gt; 2m v1.32.3+k3s1 k3s-agent2 Ready \u0026lt;none\u0026gt; 2m v1.32.3+k3s1 Installing Helm Helm is the Kubernetes package manager. Everything from Longhorn to Traefik installs via a single Helm command:\n1 2 3 4 5 curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash helm repo add longhorn https://charts.longhorn.io helm repo add traefik https://traefik.github.io/charts helm repo add metallb https://metallb.github.io/metallb helm repo update MetalLB — Bare-Metal Load Balancer Kubernetes LoadBalancer services only work on cloud providers (AWS, GCP) where a cloud controller provisions real load balancers. In a homelab, MetalLB assigns IPs from your local subnet and announces them via ARP (L2 mode) or BGP.\nInstall MetalLB 1 2 3 4 helm upgrade --install metallb metallb/metallb \\ --namespace metallb-system \\ --create-namespace \\ --set speaker.frr.enabled=false Wait for the pods to start:\n1 2 kubectl -n metallb-system wait --for=condition=ready pod \\ --selector=app.kubernetes.io/instance=metallb --timeout=60s Configure an IP Address Pool Create metallb-pool.yaml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 apiVersion: metallb.io/v1beta1 kind: IPAddressPool metadata: name: homelab-pool namespace: metallb-system spec: addresses: - 10.0.30.200-10.0.30.220 --- apiVersion: metallb.io/v1beta1 kind: L2Advertisement metadata: name: l2-advert namespace: metallb-system spec: ipAddressPools: - homelab-pool Apply it:\n1 kubectl apply -f metallb-pool.yaml Any service with type: LoadBalancer now gets an IP from the 10.0.30.200-10.0.30.220 range without needing a real load balancer.\nLonghorn — Distributed Persistent Storage Docker Compose volumes map to host directories. Kubernetes requires CSI (Container Storage Interface) drivers for persistent storage. Longhorn creates replicated block storage across your K3s nodes using the available disk space on each node\u0026rsquo;s /var/lib/longhorn/ directory.\nInstall Longhorn 1 2 3 4 5 helm upgrade --install longhorn longhorn/longhorn \\ --namespace longhorn-system \\ --create-namespace \\ --set defaultSettings.defaultReplicaCount=2 \\ --set persistence.defaultFsType=ext4 With defaultReplicaCount=2, Longhorn stores two copies of every volume — enough redundancy for a three-node homelab without wasting the disk space of three replicas.\nWait for all pods:\n1 kubectl -n longhorn-system get pods -w Access the Longhorn UI at http://10.0.30.10:30000 (it spawns as a NodePort service by default). Every node\u0026rsquo;s available disk is automatically detected and added as storage.\nCreate a StorageClass for Workloads Longhorn creates a default longhorn StorageClass. Verify it:\n1 kubectl get sc Output:\nNAME PROVISIONER RECLAIMPOLICY longhorn (default) driver.longhorn.io Delete longhorn-static driver.longhorn.io Delete Your PVCs (PersistentVolumeClaims) now automatically provision replicated block storage across the cluster.\nTraefik Ingress Controller Traefik is the standard ingress for K3s homelabs. It receives external HTTP/HTTPS traffic, routes it to the correct service based on hostname, and handles TLS termination.\nInstall Traefik with Helm:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 cat \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; \u0026gt; traefik-values.yaml deployment: replicas: 2 kind: Deployment ingressClass: enabled: true isDefaultClass: true service: type: LoadBalancer annotations: {} ports: web: port: 80 expose: true exposedPort: 80 websecure: port: 443 expose: true exposedPort: 443 providers: kubernetesCRD: enabled: true kubernetesIngress: enabled: true publishedService: enabled: true EOF helm upgrade --install traefik traefik/traefik \\ --namespace traefik \\ --create-namespace \\ --values traefik-values.yaml Check that Traefik received a MetalLB IP:\n1 kubectl -n traefik get svc traefik Example output:\nNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) traefik LoadBalancer 10.43.231.45 10.0.30.200 80:31234,443:32567 Point *.apps.yourdomain.com DNS to 10.0.30.200 and Traefik routes all subdomain traffic to the matching Ingress resources.\nDeploying Your First Workload Let\u0026rsquo;s deploy Whoami — a simple HTTP echo service — to verify everything works end-to-end.\nCreate a Deployment and Service 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 # whoami.yaml apiVersion: apps/v1 kind: Deployment metadata: name: whoami spec: replicas: 2 selector: matchLabels: app: whoami template: metadata: labels: app: whoami spec: containers: - name: whoami image: traefik/whoami:v1.10 ports: - containerPort: 80 resources: requests: cpu: 50m memory: 64Mi limits: cpu: 200m memory: 128Mi --- apiVersion: v1 kind: Service metadata: name: whoami spec: selector: app: whoami ports: - port: 80 --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: whoami spec: ingressClassName: traefik rules: - host: whoami.apps.yourdomain.com http: paths: - path: / pathType: Prefix backend: service: name: whoami port: number: 80 Apply it:\n1 kubectl apply -f whoami.yaml Wait a moment, then curl the ingress:\n1 curl -H \u0026#34;Host: whoami.apps.yourdomain.com\u0026#34; http://10.0.30.200 You should see the container\u0026rsquo;s hostname, IP, and request headers.\nAdding TLS with Let\u0026rsquo;s Encrypt Create a ClusterIssuer for automatic certificates:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # cluster-issuer.yaml apiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: letsencrypt-prod spec: acme: email: admin@yourdomain.com server: https://acme-v02.api.letsencrypt.org/directory privateKeySecretRef: name: letsencrypt-prod-key solvers: - http01: ingress: class: traefik Install cert-manager first:\n1 2 3 4 5 6 7 helm repo add jetstack https://charts.jetstack.io helm upgrade --install cert-manager jetstack/cert-manager \\ --namespace cert-manager \\ --create-namespace \\ --set crds.enabled=true kubectl apply -f cluster-issuer.yaml Then annotate any Ingress to get automatic TLS:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: whoami annotations: cert-manager.io/cluster-issuer: \u0026#34;letsencrypt-prod\u0026#34; spec: ingressClassName: traefik tls: - hosts: - whoami.apps.yourdomain.com secretName: whoami-tls rules: - host: whoami.apps.yourdomain.com http: paths: - path: / pathType: Prefix backend: service: name: whoami port: number: 80 cert-manager handles the HTTP-01 challenge automatically, stores the certificate in whoami-tls, and Traefik serves it.\nMigrating from Docker Compose to Kubernetes Moving existing Compose workloads to K3s follows a pattern:\nImages — Your existing Docker images work as-is. No rebuild needed. Volumes — Replace bind mounts with PVCs. Networks — Kubernetes Services replace Compose network names. DNS resolves as \u0026lt;service\u0026gt;.\u0026lt;namespace\u0026gt;.svc.cluster.local. Environment — Use ConfigMaps (non-secret) and Secrets. Reverse proxy — Replace Traefik Compose labels with Ingress resources + annotations. Example: PostgreSQL from Compose to K3s Docker Compose volume:\n1 2 3 volumes: pgdata: driver: local Kubernetes equivalent:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 apiVersion: v1 kind: PersistentVolumeClaim metadata: name: postgres-data spec: accessModes: - ReadWriteOnce resources: requests: storage: 10Gi storageClassName: longhorn --- apiVersion: apps/v1 kind: StatefulSet metadata: name: postgres spec: serviceName: postgres replicas: 1 selector: matchLabels: app: postgres template: metadata: labels: app: postgres spec: containers: - name: postgres image: postgres:17 env: - name: POSTGRES_PASSWORD valueFrom: secretKeyRef: name: postgres-secret key: password ports: - containerPort: 5432 volumeMounts: - name: data mountPath: /var/lib/postgresql/data volumes: - name: data persistentVolumeClaim: claimName: postgres-data A tool like kompose (https://kompose.io/) can auto-translate Compose files into Kubernetes manifests as a starting point, but reviewing and hand-tuning the output is recommended for production workloads.\nMonitoring the Cluster Install the kube-prometheus-stack for real-time cluster visibility:\n1 2 3 4 helm repo add prometheus-community https://prometheus-community.github.io/helm-charts helm upgrade --install kube-prometheus prometheus-community/kube-prometheus-stack \\ --namespace monitoring \\ --create-namespace Access Grafana:\n1 kubectl -n monitoring port-forward svc/kube-prometheus-grafana 3000:80 Open http://localhost:3000 — default credentials are admin/prom-operator. The \u0026ldquo;Kubernetes / Views / Global\u0026rdquo; and \u0026ldquo;Nodes\u0026rdquo; dashboards give immediate insight into pod resource usage, node health, and cluster events.\nLonghorn also exposes Prometheus metrics at http://\u0026lt;any-node-ip\u0026gt;:9500/metrics if you want to scrape storage metrics into the same Prometheus instance.\nWhat\u0026rsquo;s Next With a working K3s cluster, your homelab gains real Kubernetes capabilities:\nNamespace isolation — Separate environments (staging, prod, monitoring) with resource quotas. GitOps — Connect a GitHub/Gitea repo with ArgoCD or Flux for declarative deployments. Horizontal pod autoscaling — Let Kubernetes scale web services based on CPU or custom metrics. Cluster upgrades — follow the manual upgrade procedure, back up the datastore, preserve installation configuration, and respect Kubernetes version skew. Upgrade server nodes one at a time before agents. The earlier --update-server shortcut was not a valid upgrade procedure. K3s is Kubernetes without the baggage — small enough for a Raspberry Pi, capable enough for production edge workloads. Your homelab deserves the upgrade.\n","permalink":"https://blog.gntech.me/posts/2026-05-23-k3s-kubernetes-homelab-deployment/","summary":"Deploy a production-grade K3s Kubernetes cluster in your homelab on Proxmox VMs — multi-node setup, Longhorn persistent storage, MetalLB load balancing, Traefik ingress, and migrating existing Docker Compose workloads.","title":"K3s Kubernetes Cluster — Lightweight Deployment for Your Homelab"},{"content":"Your homelab server has an NVIDIA GPU sitting idle in the PCIe slot. Docker containers run fine without it, but every AI query runs on CPU at a crawl, every Jellyfin stream pegs the processor for transcoding, and Frigate object detection chugs through frames at single-digit FPS.\nThe NVIDIA Container Toolkit fixes this. It exposes the GPU to Docker containers as a natively accessible resource — no privileged mode, no raw device passthrough, no manual file descriptor forwarding. Just install the runtime, set a single Compose option, and your containers see the GPU as if it were running on bare metal.\nThis guide covers the complete setup from a fresh install through production workloads on Ubuntu/Debian and Proxmox LXC hosts.\nWhy GPU Accelerate Docker Containers? A GPU in a Docker workflow is not about gaming or rendering. In a homelab, the three workloads that benefit most are:\nAI inference — Ollama runs LLMs 5–10× faster on GPU than CPU. A $200 Tesla P4 or RTX 3060 handles 7B–13B parameter models comfortably. Media transcoding — Jellyfin (or Plex) uses NVIDIA NVENC for hardware-accelerated video encoding. A single transcode stream on CPU uses 60–80% of a modern processor; NVENC does it for under 5%. Computer vision — Frigate uses TensorRT or OpenCV with CUDA for real-time object detection on camera feeds. GPU detection slashes inference latency from 500ms to under 30ms per frame. All three share the same underlying setup: the NVIDIA Container Toolkit. Once it is running, any Docker container can request GPU access with a single line in its Compose file.\nPrerequisites Linux host (Debian 12, Ubuntu 24.04, or Proxmox LXC) NVIDIA GPU (Turing or newer for NVENC — GTX 1650, RTX 2060+, Tesla P4/P40, A-series) NVIDIA proprietary driver installed (nvidia-smi works) Docker Engine 24+ with containerd Verify the Driver 1 nvidia-smi You should see the driver version, CUDA version, and your GPU:\n+-----------------------------------------------------------------------------+ | NVIDIA-SMI 550.120 Driver Version: 550.120 CUDA Version: 12.4 | |-------------------------------+----------------------+----------------------+ | GPU Name TCC/WDDM | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | |===============================|======================|======================| | 0 Tesla P4 Off | 00000000:02:00.0 Off | Off | | N/A 48C P0 23W / 75W | 10MiB / 7680MiB | 0% Default | +-----------------------------------------------------------------------------+ If nvidia-smi returns a command-not-found or driver error, install the driver first:\n1 2 3 4 # Ubuntu / Debian sudo apt update sudo apt install -y nvidia-driver-550-server sudo reboot Install the NVIDIA Container Toolkit The toolkit provides the nvidia-container-runtime that Docker uses to inject GPU devices into containers.\nAdd the Repository 1 2 3 4 5 6 curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \\ sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \\ sed \u0026#39;s#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g\u0026#39; | \\ sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list Install and Configure Docker 1 2 3 4 5 6 sudo apt update sudo apt install -y nvidia-container-toolkit # Configure Docker daemon to use nvidia runtime sudo nvidia-ctk runtime configure --runtime=docker sudo systemctl restart docker The configure step adds the NVIDIA runtime to /etc/docker/daemon.json:\n1 2 3 4 5 6 7 8 { \u0026#34;runtimes\u0026#34;: { \u0026#34;nvidia\u0026#34;: { \u0026#34;args\u0026#34;: [], \u0026#34;path\u0026#34;: \u0026#34;nvidia-container-runtime\u0026#34; } } } Verify the Installation 1 2 docker run --rm --runtime=nvidia \\ nvidia/cuda:12.4-base-ubuntu22.04 nvidia-smi You should see the same nvidia-smi output inside the container. This confirms the GPU is accessible within the container namespace with no manual device mapping.\nDocker Compose GPU Configuration The modern way to expose GPUs in Docker Compose is the deploy.resources.reservations.devices block. This works with Compose v2.23+ and Docker Engine 24+.\n1 2 3 4 5 6 7 8 9 10 services: gpu-workload: image: my-workload:latest deploy: resources: reservations: devices: - driver: nvidia count: 1 capabilities: [gpu] For a specific GPU in a multi-GPU system:\n1 2 3 4 5 6 7 deploy: resources: reservations: devices: - driver: nvidia device_ids: [\u0026#34;GPU-12a3b4c5-def6-7890-abcd-ef1234567890\u0026#34;] capabilities: [gpu] To request specific compute capabilities:\n1 2 3 4 5 6 7 deploy: resources: reservations: devices: - driver: nvidia count: all capabilities: [gpu, utility, compute] The capabilities list controls what is exposed:\ngpu — basic GPU device access utility — nvidia-smi monitoring counters compute — CUDA compute API video — NVENC/NVDEC video encode/decode display — display output For Jellyfin and Frigate, always include video:\n1 capabilities: [gpu, video] Real Workload: Ollama with GPU Acceleration Ollama is the most straightforward GPU-accelerated workload for a homelab. It detects the GPU automatically when the NVIDIA runtime is configured.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 services: ollama: image: ollama/ollama:latest container_name: ollama restart: unless-stopped volumes: - ./ollama_data:/root/.ollama environment: - OLLAMA_KEEP_ALIVE=24h deploy: resources: reservations: devices: - driver: nvidia count: 1 capabilities: [gpu, utility] ports: - \u0026#34;11434:11434\u0026#34; Pull and run a model to validate GPU usage:\n1 2 docker compose exec ollama ollama pull mistral:7b docker compose exec ollama ollama run mistral:7b \u0026#34;Why is GPU inference faster than CPU?\u0026#34; Check GPU utilization during inference:\n1 watch -n1 nvidia-smi You should see 60–95% GPU compute utilization during token generation. Compare this to the same model running on CPU — typical speedup is 5–10× depending on the model size.\nReal Workload: Jellyfin with NVENC Transcoding Jellyfin uses NVIDIA NVENC for hardware-accelerated video transcoding. This requires the video capability in addition to gpu.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 services: jellyfin: image: jellyfin/jellyfin:latest container_name: jellyfin restart: unless-stopped volumes: - ./jellyfin_config:/config - ./media:/media group_add: - \u0026#34;44\u0026#34; # video group on host environment: - NVIDIA_DRIVER_CAPABILITIES=all deploy: resources: reservations: devices: - driver: nvidia count: 1 capabilities: [gpu, video] ports: - \u0026#34;8096:8096\u0026#34; After deployment:\nOpen Jellyfin admin dashboard → Playback → Transcoding Set Hardware acceleration to NVIDIA NVENC Select the GPU device from the dropdown Enable Hardware encoding and Hardware decoding Save and test by playing a 4K HEVC file through the web client — check the playback info for \u0026ldquo;Hardware: yes\u0026rdquo; To verify transcoding is GPU-accelerated:\n1 2 3 4 5 # Monitor encoder utilization nvidia-smi -q -d ENCODER # Watch decoder utilization nvidia-smi -q -d DECODER A single Tesla P4 handles 4–6 simultaneous 1080p transcodes or 2–3 4K → 1080p transcodes without breaking a sweat.\nReal Workload: Frigate with TensorRT Detection Frigate 0.15+ supports NVIDIA TensorRT for hardware-accelerated object detection. This dramatically reduces inference latency compared to the default CPU-based OpenVINO or Coral TPU modes.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 services: frigate: image: ghcr.io/blakeblackshear/frigate:stable container_name: frigate restart: unless-stopped privileged: false volumes: - ./frigate_config:/config - ./frigate_storage:/media/frigate environment: - FRIGATE_RTSP_PASSWORD=${RTSP_PASSWORD} deploy: resources: reservations: devices: - driver: nvidia count: 1 capabilities: [gpu, utility, compute] tmpfs: - /tmp/cache:size=1G ports: - \u0026#34;5000:5000\u0026#34; Frigate config snippet for TensorRT:\n1 2 3 4 5 6 7 8 9 10 11 12 # /config/config.yml detectors: tensorrt: type: tensorrt device: 0 model: path: /config/modelcache/tensorrt/yolov7-320.trt width: 320 height: 320 input_tensor: nchw input_pixel_format: rgb With GPU detection, frame processing goes from 2–3 FPS on CPU to 25–30 FPS on a Tesla P4 — the difference between missing motion events and catching everything.\nProxmox LXC Considerations Running Docker inside a Proxmox LXC requires an extra step to pass the GPU into the container. Add these lines to the LXC config file (/etc/pve/lxc/\u0026lt;ID\u0026gt;.conf):\n1 2 3 4 5 6 7 8 lxc.cgroup2.devices.allow: c 195:* rwm lxc.cgroup2.devices.allow: c 509:* rwm lxc.cgroup2.devices.allow: c 226:* rwm lxc.mount.entry: /dev/nvidia0 dev/nvidia0 none bind,optional,create=file lxc.mount.entry: /dev/nvidiactl dev/nvidiactl none bind,optional,create=file lxc.mount.entry: /dev/nvidia-modeset dev/nvidia-modeset none bind,optional,create=file lxc.mount.entry: /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir Then run pct reboot \u0026lt;ID\u0026gt; to apply. After reboot, install the NVIDIA driver and Container Toolkit inside the LXC as described above.\nTroubleshooting \u0026ldquo;No available runtime\u0026rdquo; or \u0026ldquo;Unknown runtime nvidia\u0026rdquo; The Docker daemon does not see the NVIDIA runtime:\n1 2 3 4 5 6 # Check daemon.json cat /etc/docker/daemon.json # Re-run the toolkit config sudo nvidia-ctk runtime configure --runtime=docker sudo systemctl restart docker \u0026ldquo;could not select device driver\u0026rdquo; error The Compose file references driver: nvidia but the runtime is not registered. This is the same fix as above — run nvidia-ctk runtime configure and restart Docker.\nContainer sees no GPU despite runtime 1 2 3 4 5 6 7 8 9 # Verify from the host nvidia-smi # Verify inside the container docker run --rm --gpus all ubuntu:22.04 nvidia-smi # If the container still fails, check the nvidia-fabricmanager sudo systemctl status nvidia-fabricmanager sudo systemctl restart nvidia-fabricmanager Fabric Manager is required for multi-GPU Tesla cards (P40, A2, T4) in certain configurations.\nNVENC not available in Jellyfin The video capability must be in the capabilities list, and the GPU must support NVENC. Check your GPU\u0026rsquo;s encode/decode support:\n1 2 3 # List supported codecs for your GPU nvidia-smi -q -d ENCODER nvidia-smi -q -d DECODER GPUs without NVENC (GTX 10-series non-Ti, older Kepler cards) cannot accelerate encoding. If your card lacks NVENC, use the Intel QuickSync path instead.\nSummary The NVIDIA Container Toolkit transforms a homelab GPU from an idle PCIe occupant into a shared accelerator for every container on the host. The setup is four commands:\nInstall nvidia-container-toolkit Configure the Docker runtime Add deploy.resources.reservations.devices to your Compose files Deploy GPU-accelerated Ollama, Jellyfin, or Frigate No Kubernetes, no custom images, no privileged containers. Just a GPU that actually earns its slot in the case.\n","permalink":"https://blog.gntech.me/posts/2026-05-23-docker-gpu-acceleration-homelab/","summary":"Set up GPU acceleration in Docker containers with the NVIDIA Container Toolkit — from installation to real workloads like Ollama AI inference, Jellyfin hardware transcoding, and Frigate object detection.","title":"Docker GPU Acceleration — NVIDIA Container Toolkit for Homelab"},{"content":"Public TLS certificates from Let\u0026rsquo;s Encrypt work great for internet-facing services, but they fall short for internal-only hosts. If you run a homelab with services on .internal, .local, or RFC 1918 IP addresses, you cannot get publicly trusted certificates for them. The typical workaround — self-signed certificates — means browsers throw security warnings, your monitoring tools fail TLS checks, and every new service requires manual certificate generation and distribution.\nYou need an internal Certificate Authority (CA) that your devices trust, combined with ACME automation so you never touch a certificate by hand.\nstep-ca from Smallstep is the best tool for this job. It runs in a single Docker container, speaks ACME (the same protocol Let\u0026rsquo;s Encrypt uses), supports DNS-01 and HTTP-01 challenges, and integrates directly with Traefik, Caddy, and certbot.\nThis guide walks through deploying step-ca in Docker, bootstrapping clients, setting up ACME, integrating with Traefik for automatic certificate provisioning, and distributing the root CA to every device in your homelab.\nWhy an Internal CA Beats Self-Signed Certificates Self-signed certificates work for testing, but they have real problems in a homelab:\nBrowser warnings: Every self-signed cert triggers a full-page \u0026ldquo;Your connection is not private\u0026rdquo; warning with no easy way to trust it permanently. No automation: You regenerate and redistribute certs manually every time they expire. No revocation: If a key is compromised, you cannot revoke the certificate. An internal CA solves all three. You install the root CA once on each device, and every certificate signed by that CA is automatically trusted. ACME automates issuance and renewal. And step-ca supports certificate revocation via CRL and OCSP if needed.\nDeploying step-ca with Docker Compose Create a project directory and a docker-compose.yml for step-ca:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # docker-compose.yml services: step-ca: image: smallstep/step-ca:latest container_name: step-ca restart: unless-stopped hostname: ca.internal ports: - \u0026#34;10.0.20.30:9000:9000\u0026#34; volumes: - step-config:/home/step environment: - DOCKER_STEPCA_INIT_NAME=GnTech Internal CA - DOCKER_STEPCA_INIT_DNS_NAMES=ca.internal,10.0.20.30,localhost - DOCKER_STEPCA_INIT_PROVISIONER_NAME=admin Bind the port to a specific IP (your Docker host\u0026rsquo;s LAN address) instead of 0.0.0.0 to avoid exposing the CA to VLANs that should not talk to it directly.\nFire it up:\n1 docker compose up -d On first run, the container initializes the CA. It generates a root certificate, an intermediate certificate, a password for the encrypted CA keys, and the initial JWK provisioner. The CA password is saved inside the Docker volume.\nRetrieve the fingerprint needed for client bootstrapping:\n1 2 docker run --rm -v step-config:/home/step smallstep/step-ca \\ step certificate fingerprint /home/step/certs/root_ca.crt Save this fingerprint — you need it for every client that bootstraps with the CA.\nRetrieve the automatically generated CA password:\n1 2 docker run --rm -v step-config:/home/step smallstep/step-ca \\ cat /home/step/secrets/password You need this password when issuing certificates with the admin provisioner.\nInstalling the step CLI on Clients Every machine that needs to interact with the CA — your Docker host, workstations, or other servers — needs the step CLI.\nOn Debian/Ubuntu 1 2 3 wget -O /tmp/step-cli.deb \\ https://dl.smallstep.com/gh-release/cli/gh-release-header/v0.28.10/step-cli_0.28.10_amd64.deb sudo dpkg -i /tmp/step-cli.deb On macOS 1 brew install step On Alpine Linux 1 apk add step-cli Bootstrapping Clients With the CLI installed and the CA running, bootstrap a client:\n1 2 3 4 5 6 CA_URL=https://10.0.20.30:9000 CA_FINGERPRINT=86a278f34e58c7ab04313aff0e8e5114f1d1da955ecb20412b3d32cc2267ddcd step ca bootstrap \\ --ca-url $CA_URL \\ --fingerprint $CA_FINGERPRINT This downloads the root CA certificate, saves it to ~/.step/certs/root_ca.crt, and writes the defaults configuration. After bootstrapping, you can issue certificates directly from the client:\n1 2 3 4 step ca certificate \\ \u0026#34;nginx.internal\u0026#34; \\ nginx.internal.crt \\ nginx.internal.key You are prompted for the admin provisioner password (from the secrets file). The result is a TLS certificate signed by your internal CA, valid for 24 hours by default.\nEnabling the ACME Provisioner The JWK provisioner works fine for manual operations, but ACME is where step-ca really shines. Add an ACME provisioner to the CA configuration.\nRead the current CA config from the volume:\n1 2 docker run --rm -v step-config:/home/step smallstep/step-ca \\ cat /home/step/config/ca.json Look for the \u0026quot;provisioners\u0026quot; array. Add an ACME provisioner entry: create a file called ca.json on the volume path by copying the existing config and adding:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 { \u0026#34;type\u0026#34;: \u0026#34;ACME\u0026#34;, \u0026#34;name\u0026#34;: \u0026#34;acme\u0026#34;, \u0026#34;forceCN\u0026#34;: true, \u0026#34;options\u0026#34;: { \u0026#34;x509\u0026#34;: { \u0026#34;template\u0026#34;: \u0026#34;/home/step/templates/acme.tpl\u0026#34; }, \u0026#34;claims\u0026#34;: { \u0026#34;enableX509CA\u0026#34;: false, \u0026#34;minTLSDur\u0026#34;: \u0026#34;1h\u0026#34;, \u0026#34;maxTLSDur\u0026#34;: \u0026#34;2160h\u0026#34; } } } The easier approach — edit the config file stored in the Docker volume. On the Docker host:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 # Copy the config out docker cp step-ca:/home/step/config/ca.json /tmp/ca.json # Edit — add the ACME provisioner (use jq for precision) jq \u0026#39;.provisioners += [{ \u0026#34;type\u0026#34;: \u0026#34;ACME\u0026#34;, \u0026#34;name\u0026#34;: \u0026#34;acme\u0026#34;, \u0026#34;forceCN\u0026#34;: true, \u0026#34;options\u0026#34;: { \u0026#34;claims\u0026#34;: { \u0026#34;enableX509CA\u0026#34;: false, \u0026#34;minTLSDur\u0026#34;: \u0026#34;1h\u0026#34;, \u0026#34;maxTLSDur\u0026#34;: \u0026#34;2160h\u0026#34; } } }]\u0026#39; /tmp/ca.json \u0026gt; /tmp/ca-new.json # Copy it back docker cp /tmp/ca-new.json step-ca:/home/step/config/ca.json # Restart step-ca to pick up the change docker compose restart Now any ACME client — Traefik, Caddy, or certbot — can request certificates from your internal CA.\nIntegrating with Traefik If you run Traefik as your reverse proxy (covered in our Traefik setup guide), adding the internal CA as an ACME certificate provider is straightforward.\nFirst, make Traefik trust your internal CA. Mount the root certificate and update the Traefik static configuration:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 # traefik.yml certificatesResolvers: letsencrypt: acme: # ...existing LE DNS-01 config for public services... internal: acme: caServer: https://10.0.20.30:9000/acme/acme/directory email: admin@gntech.dev storage: /etc/traefik/acme-internal.json httpChallenge: entryPoint: web # keyType: EC256 # Optional — use ECDSA keys And mount the root CA into the Traefik container:\n1 2 3 4 5 6 services: traefik: image: traefik:v3.3 volumes: - /etc/ssl/certs/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt:ro - ~/.step/certs/root_ca.crt:/usr/local/share/ca-certificates/step-ca.crt:ro Run update-ca-certificates inside the container or rebuild the image with the CA bundled. For the easiest path, bind-mount the CA file and configure Traefik to use the system trust store by adding to the entrypoint.\nNow annotate services that need internal certificates with the internal resolver in their Docker labels:\n1 2 3 4 5 6 7 services: grafana: labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.grafana.rule=Host(`grafana.internal`)\u0026#34; - \u0026#34;traefik.http.routers.grafana.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.grafana.tls.certresolver=internal\u0026#34; Traefik automatically requests a certificate from step-ca via ACME for grafana.internal, renews it before expiry, and terminates TLS.\nDNS-01 Challenges for Wildcard Certificates For wildcard certificates like *.internal, use DNS-01 instead of HTTP-01. step-ca supports DNS-01 out of the box, and you can pair it with any DNS provider that has an API.\nIf you run your own DNS server (like Pi-hole or Unbound), the simplest approach is a manual DNS challenge script. Create a script that the ACME client calls to set and clear TXT records:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 #!/bin/bash # /usr/local/bin/dns-step-hook.sh # Called by ACME client with: # $1 = action (present/cleanup) # $2 = FQDN # $3 = token value set -euo pipefail ACTION=$1 FQDN=$2 TOKEN=$3 # If your DNS server has an API (e.g., PowerDNS, Bind with nsupdate) # Update the TXT record for _acme-challenge.$FQDN case $ACTION in present) # nsupdate -k /etc/bind/rndc.key \u0026lt;\u0026lt;EOF # server ns.internal 53 # zone internal # update add _acme-challenge.$FQDN 60 TXT \u0026#34;$TOKEN\u0026#34; # send # EOF echo \u0026#34;DNS challenge: $FQDN -\u0026gt; $TOKEN\u0026#34; ;; cleanup) # Remove the TXT record echo \u0026#34;DNS cleanup: $FQDN\u0026#34; ;; esac Or use certbot with step-ca\u0026rsquo;s ACME endpoint and the DNS plugin for your provider:\n1 2 3 4 5 6 7 8 certbot certonly \\ --manual \\ --preferred-challenges dns \\ --manual-auth-hook /usr/local/bin/dns-step-hook.sh \\ --manual-cleanup-hook /usr/local/bin/dns-step-hook.sh \\ --server https://10.0.20.30:9000/acme/acme/directory \\ -d \u0026#34;*.internal\u0026#34; \\ --non-interactive Managing Certificate Lifetime step-ca issues short-lived certificates by default — 24 hours — which means renewal is frequent but revocation risk is minimal. For homelab use, 24 hours is short enough to be annoying if something breaks. Extend the default via the CA configuration:\n1 2 3 4 5 6 7 { \u0026#34;claims\u0026#34;: { \u0026#34;minTLSDur\u0026#34;: \u0026#34;24h\u0026#34;, \u0026#34;maxTLSDur\u0026#34;: \u0026#34;2160h\u0026#34;, \u0026#34;defaultTLSDur\u0026#34;: \u0026#34;720h\u0026#34; } } This sets the default to 30 days (720h), with a maximum of 90 days (2160h). ACME clients can request a specific duration within this range.\nDistributing the Root CA to Clients Every device that needs to trust your internal certificates must have the root CA installed. Here is how to distribute it to common platforms.\nLinux (Debian/Ubuntu) 1 2 3 # On the Docker host where step-ca runs sudo cp ~/.step/certs/root_ca.crt /usr/local/share/ca-certificates/step-ca.crt sudo update-ca-certificates For other machines, copy the root CA file and run the same commands.\nWindows (via Group Policy or script) 1 2 # Run as Administrator Import-Certificate -FilePath \u0026#34;\\\\server\\share\\step-ca.crt\u0026#34; -CertStoreLocation \u0026#34;Cert:\\LocalMachine\\Root\u0026#34; macOS 1 2 sudo security add-trusted-cert -d -r trustRoot \\ -k /Library/Keychains/System.keychain ~/.step/certs/root_ca.crt Android Copy the .crt file to the device, then go to Settings → Security → Encryption \u0026amp; credentials → Install a certificate → CA certificate and select the file.\niOS/iPadOS Send the .crt file via email or AirDrop, tap to install, then go to Settings → General → About → Certificate Trust Settings and enable the root CA.\nFirefox (separate trust store) Firefox uses its own certificate store, not the system one:\n1 2 3 4 5 # Import root CA into Firefox profile certutil -A -n \u0026#34;GnTech Internal CA\u0026#34; \\ -t \u0026#34;TCu,Cu,Tu\u0026#34; \\ -i ~/.step/certs/root_ca.crt \\ -d sql:/home/user/.mozilla/firefox/*.default-release Health Checks and Monitoring Verify the CA is running and healthy:\n1 2 curl -k https://10.0.20.30:9000/health # {\u0026#34;status\u0026#34;:\u0026#34;ok\u0026#34;} Add step-ca to your monitoring stack:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # prometheus scrape config via blackbox exporter - job_name: step-ca-health metrics_path: /probe params: module: [http_2xx] static_configs: - targets: - https://10.0.20.30:9000/health relabel_configs: - source_labels: [__address__] target_label: __param_target - source_labels: [__param_target] target_label: instance - target_label: __address__ replacement: blackbox:9115 Monitor certificate expiry for your critical services using Uptime Kuma or Prometheus Blackbox Exporter with the ssl_certificate_expiry probe. Set alerts at 7, 3, and 1 day before expiry.\nSecurity Considerations Running an internal CA is powerful, but treat it with care:\nKeep the CA password safe: The encrypted root key is in the Docker volume\u0026rsquo;s secrets/ directory. Backup this directory, but never share the password. Restrict network access: Bind step-ca to a management IP or use firewall rules to limit who can reach port 9000. Use a dedicated volume: Step-ca stores private keys on the Docker volume. Keep regular backups. Rotate provisioner keys: The admin JWK provisioner is powerful. Create separate ACME provisioners with shorter lifespans for different use cases. Audit certificate issuance: step-ca logs every certificate request. Monitor logs for unexpected issuance. Backup the step-ca volume regularly:\n1 2 docker run --rm -v step-config:/home/step -v /backup:/backup \\ alpine tar czf /backup/step-ca-$(date +%F).tar.gz -C /home/step . To restore, extract the archive into a new volume of the same name.\nSummary An internal CA with ACME support is the missing piece in most homelab TLS setups. With step-ca in Docker you get:\nAutomatic certificates for any internal hostname or IP No browser warnings once the root CA is trusted Traefik integration for seamless reverse proxy TLS Short-lived certificates that minimize revocation risk Open source with no licensing costs One deploy and root CA distribution session eliminates self-signed certificate pain forever. Every new service you add — whether it is Grafana, Home Assistant, or a custom API — gets valid TLS automatically.\n","permalink":"https://blog.gntech.me/posts/2026-05-22-step-ca-acme-internal-certificates/","summary":"Run your own internal Certificate Authority with step-ca in Docker, issue TLS certificates via ACME, integrate with Traefik for automatic renewal, and distribute the root CA to all your devices — no more self-signed certificate warnings.","title":"Step CA + ACME — Internal TLS Certificates for Your Homelab"},{"content":"Docker\u0026rsquo;s HEALTHCHECK instruction tells you when a container is broken, but it does nothing about it. A PostgreSQL container that loses its WAL directory, a web app stuck in a deadlock, or a Redis instance that silently drops connections — all stay running in an unhealthy state, serving errors until you notice and restart them manually.\nThis is the single biggest reliability gap in standalone Docker deployments. Kubernetes and Docker Swarm handle this natively, but if you run plain Docker Compose stacks in your homelab, the responsibility falls on you.\nThis guide covers every practical approach to container auto-healing: from the simplest one-line solution to custom watchdog scripts for edge cases. You will leave with a self-healing setup that requires zero manual intervention for common failure modes.\nDocker HEALTHCHECK: The Foundation Auto-healing starts with good health checks. Without them, no tool can distinguish a working container from a broken one.\nHEALTHCHECK in a Dockerfile Embed health checks directly in images you build:\n1 2 3 4 5 6 7 8 9 FROM node:20-alpine WORKDIR /app COPY . . HEALTHCHECK --interval=15s --timeout=5s --retries=3 --start-period=30s \\ CMD wget --no-verbose --tries=1 --spider http://localhost:3000/health || exit 1 CMD [\u0026#34;node\u0026#34;, \u0026#34;server.js\u0026#34;] For databases, use protocol-native checks:\n1 2 3 4 5 6 7 # PostgreSQL HEALTHCHECK --interval=10s --timeout=5s --retries=5 \\ CMD pg_isready -U postgres || exit 1 # Redis HEALTHCHECK --interval=10s --timeout=5s --retries=5 \\ CMD redis-cli ping | grep -q PONG || exit 1 HEALTHCHECK at Runtime For third-party images you cannot modify, pass health check parameters when running the container:\n1 2 3 4 5 6 7 8 9 docker run -d \\ --name my-app \\ --restart unless-stopped \\ --health-cmd=\u0026#34;curl -f http://localhost:8080/health || exit 1\u0026#34; \\ --health-interval=15s \\ --health-timeout=5s \\ --health-retries=3 \\ --health-start-period=30s \\ my-app:latest HEALTHCHECK in Docker Compose The same pattern in Compose format:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 services: app: image: my-app:latest restart: unless-stopped healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;curl\u0026#34;, \u0026#34;-f\u0026#34;, \u0026#34;http://localhost:3000/health\u0026#34;] interval: 15s timeout: 5s retries: 3 start_period: 30s postgres: image: postgres:17-alpine restart: unless-stopped healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;pg_isready -U $${POSTGRES_USER} || exit 1\u0026#34;] interval: 10s timeout: 5s retries: 5 Checking Health Status 1 2 3 4 5 6 7 8 # Quick status docker inspect --format=\u0026#39;{{.State.Health.Status}}\u0026#39; my-app # Full health log docker inspect --format=\u0026#39;{{json .State.Health}}\u0026#39; my-app | jq # Watch health transitions docker events --filter \u0026#39;event=health_status\u0026#39; --filter \u0026#39;type=container\u0026#39; Once these health checks are in place, you can detect failures. The next step is acting on them.\ndocker-autoheal: The Simple Solution willfarrell/docker-autoheal is a standalone container that watches Docker events for health status changes and restarts unhealthy containers automatically. It is the closest thing to native auto-healing for standalone Docker.\nDeploy with Compose 1 2 3 4 5 6 7 8 9 10 11 services: autoheal: image: willfarrell/autoheal:latest container_name: autoheal restart: unless-stopped environment: - AUTOHEAL_CONTAINER_LABEL=all - AUTOHEAL_INTERVAL=10 - AUTOHEAL_START_PERIOD=30 volumes: - /var/run/docker.sock:/var/run/docker.sock This restarts every container that becomes unhealthy. For finer control, label specific containers:\n1 2 3 4 5 6 7 8 9 10 services: app: image: my-app:latest labels: - autoheal=true healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;curl\u0026#34;, \u0026#34;-f\u0026#34;, \u0026#34;http://localhost:3000/health\u0026#34;] interval: 15s retries: 3 start_period: 30s Then set AUTOHEAL_CONTAINER_LABEL=autoheal instead of all.\nConfiguration Options Variable Default Description AUTOHEAL_CONTAINER_LABEL autoheal Label to filter containers, or all AUTOHEAL_INTERVAL 5 Seconds between health check polls AUTOHEAL_START_PERIOD 0 Seconds to wait before monitoring AUTOHEAL_DOCKER_SOCK unset Path to Docker socket inside container Pros and Cons Pros:\nOne Docker Compose service, zero config Works with any container that has HEALTHCHECK Respects restart policies after restart Active project, updated regularly Cons:\nRequires mounting the Docker socket (security consideration) Race condition if the container restarts itself during heal No notification when auto-heal fires Custom Watchdog Script with Docker API If you want more control or no-socket alternatives, write a watchdog script using the Docker SDK. This approach also lets you add notifications, rate limiting, and selective restart logic.\nPython Watchdog with Slack Alerts 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 #!/usr/bin/env python3 \u0026#34;\u0026#34;\u0026#34;Watchdog: restart unhealthy containers and notify.\u0026#34;\u0026#34;\u0026#34; import os import time import subprocess import json import requests SLACK_WEBHOOK = os.getenv(\u0026#34;SLACK_WEBHOOK_URL\u0026#34;, \u0026#34;\u0026#34;) CHECK_INTERVAL = int(os.getenv(\u0026#34;CHECK_INTERVAL\u0026#34;, \u0026#34;15\u0026#34;)) def notify(container_name, status): if not SLACK_WEBHOOK: return message = f\u0026#34;🚨 *{container_name}* is *{status}* — restarting...\u0026#34; requests.post(SLACK_WEBHOOK, json={\u0026#34;text\u0026#34;: message}) def get_health_status(): result = subprocess.run( [\u0026#34;docker\u0026#34;, \u0026#34;ps\u0026#34;, \u0026#34;--format\u0026#34;, \u0026#34;{{.ID}} {{.Names}} {{.Status}}\u0026#34;], capture_output=True, text=True, timeout=10 ) unhealthy = [] for line in result.stdout.strip().split(\u0026#34;\\n\u0026#34;): if not line: continue parts = line.split() if len(parts) \u0026gt;= 3 and \u0026#34;(unhealthy)\u0026#34; in line: unhealthy.append({\u0026#34;id\u0026#34;: parts[0], \u0026#34;name\u0026#34;: parts[1]}) return unhealthy def restart_container(name): subprocess.run( [\u0026#34;docker\u0026#34;, \u0026#34;restart\u0026#34;, name], capture_output=True, timeout=30 ) if __name__ == \u0026#34;__main__\u0026#34;: while True: for c in get_health_status(): print(f\u0026#34;[{time.ctime()}] Restarting unhealthy: {c[\u0026#39;name\u0026#39;]}\u0026#34;) notify(c[\u0026#34;name\u0026#34;], \u0026#34;unhealthy\u0026#34;) restart_container(c[\u0026#34;name\u0026#34;]) # Cooldown: avoid restart loops time.sleep(30) time.sleep(CHECK_INTERVAL) Run it as a systemd service:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 [Unit] Description=Docker auto-heal watchdog After=docker.service Requires=docker.service [Service] Type=simple ExecStart=/usr/local/bin/docker-watchdog.py Restart=always RestartSec=10 Environment=CHECK_INTERVAL=15 [Install] WantedBy=multi-user.target Bash Watchdog (Minimal) For a lighter alternative without Python dependencies:\n1 2 3 4 5 6 7 8 #!/bin/bash while true; do for container in $(docker ps --filter \u0026#34;health=unhealthy\u0026#34; --format \u0026#34;{{.Names}}\u0026#34;); do echo \u0026#34;$(date) - Restarting unhealthy container: $container\u0026#34; docker restart \u0026#34;$container\u0026#34; done sleep 15 done systemd Docker Service Watchdog For critical infrastructure containers (DNS, reverse proxy, VPN), bypass Docker\u0026rsquo;s health check entirely and use systemd to manage the container as a service.\nCreate a systemd Service for a Container 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 [Unit] Description=Traefik reverse proxy (managed via systemd) After=docker.service Requires=docker.service StartLimitIntervalSec=60 StartLimitBurst=5 [Service] Type=simple Restart=always RestartSec=10 ExecStartPre=-/usr/bin/docker rm -f traefik ExecStart=/usr/bin/docker run --rm \\ --name traefik \\ --network=proxy \\ -p 80:80 -p 443:443 \\ -v /opt/traefik/traefik.yml:/etc/traefik/traefik.yml:ro \\ -v /var/run/docker.sock:/var/run/docker.sock:ro \\ traefik:v3.2 ExecStop=/usr/bin/docker stop -t 10 traefik ExecStopPost=/usr/bin/docker rm -f traefik [Install] WantedBy=multi-user.target The key advantages:\nsystemd restarts: if the Docker daemon fails or restarts, systemd respawns the container automatically. Rate limiting: StartLimitIntervalSec and StartLimitBurst prevent restart loops. Logging: container logs go to the systemd journal. Health depends on process: if the container process exits, systemd restarts it regardless of Docker restart policies. systemd Health Check Extension You can combine systemd with a timer unit that checks the container health and restarts the service if unhealthy:\n1 2 3 4 5 6 7 [Unit] Description=Health check for traefik [Timer] OnCalendar=*-*-* *:0/5:00 Persistent=true [Install] WantedBy=timers.target Service file:\n1 2 3 4 5 [Unit] Description=Verify Traefik health every 5 minutes [Service] Type=oneshot ExecStart=/bin/sh -c \u0026#39;docker inspect --format=\u0026#34;{{.State.Health.Status}}\u0026#34; traefik | grep -q healthy || systemctl restart docker-traefik.service\u0026#39; Uptime Kuma Webhook Restart Pattern Uptime Kuma monitors HTTP endpoints and can trigger webhooks on failure. Pair it with a lightweight webhook receiver that restarts the failing container.\nDeploy the Webhook Receiver Create a simple webhook handler with a shell script:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 #!/bin/bash # /usr/local/bin/kuma-webhook.sh PORT=${PORT:-8080} echo \u0026#34;Listening on port $PORT for Kuma webhooks...\u0026#34; while true; do request=$(nc -l -p \u0026#34;$PORT\u0026#34; -q 1 2\u0026gt;/dev/null) container=$(echo \u0026#34;$request\u0026#34; | grep -oP \u0026#39;container=\\K[a-zA-Z0-9_-]+\u0026#39;) if [ -n \u0026#34;$container\u0026#34; ]; then echo \u0026#34;$(date) - Kuma alert: restarting $container\u0026#34; docker restart \u0026#34;$container\u0026#34; fi done Or use a dedicated webhook service like webhook with a minimal config:\n1 2 3 4 5 # /etc/webhook/hooks.yaml - id: restart-container execute-command: /usr/local/bin/restart-container.sh command-working-directory: /tmp response-message: \u0026#34;Restart triggered\u0026#34; In Uptime Kuma, configure the monitor\u0026rsquo;s Notification to send a POST webhook to http://your-watchdog:8080/hooks/restart-container with the container name in the payload.\nSelf-Healing Docker Compose Stack: Complete Example Here is a real-world auto-healing stack combining all the techniques above:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 services: # Core database with native health check postgres: image: postgres:17-alpine restart: unless-stopped volumes: - pgdata:/var/lib/postgresql/data environment: - POSTGRES_PASSWORD_FILE=/run/secrets/db_password secrets: - db_password healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;pg_isready -U postgres || exit 1\u0026#34;] interval: 10s timeout: 5s retries: 5 start_period: 15s labels: - autoheal=true # Application with HTTP health endpoint app: image: my-app:latest restart: unless-stopped depends_on: postgres: condition: service_healthy ports: - \u0026#34;8080:3000\u0026#34; healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;curl\u0026#34;, \u0026#34;-f\u0026#34;, \u0026#34;http://localhost:3000/api/health\u0026#34;] interval: 15s timeout: 5s retries: 3 start_period: 30s labels: - autoheal=true # Auto-heal daemon autoheal: image: willfarrell/autoheal:latest restart: unless-stopped environment: - AUTOHEAL_CONTAINER_LABEL=autoheal - AUTOHEAL_INTERVAL=5 - AUTOHEAL_START_PERIOD=30 volumes: - /var/run/docker.sock:/var/run/docker.sock secrets: db_password: file: ./secrets/db_password.txt volumes: pgdata: Restart Policy Deep Dive Docker provides four restart policies. Understanding their interaction with auto-healing is critical:\nPolicy Behavior Auto-Heal Required no Never restart Yes on-failure[:max-retries] Restart on non-zero exit Yes (unhealthy ≠ exit) unless-stopped Restart unless manually stopped Yes always Always restart Yes Key insight: none of these trigger on health status. A container exiting with code 0 then immediately entering an idle loop will not restart — unless you add a HEALTHCHECK that fails and an auto-heal mechanism that acts on it.\nFor containers that crash frequently, combine restart policies with auto-heal:\n1 2 3 4 5 6 7 8 9 10 services: flaky-service: image: flaky:latest restart: on-failure:5 healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;curl\u0026#34;, \u0026#34;-f\u0026#34;, \u0026#34;http://localhost:80/health\u0026#34;] interval: 10s retries: 3 labels: - autoheal=true This handles both crash loops (Docker restart on exit) and stuck processes (autoheal restart on unhealthy).\nHandling System Restarts and Daemon Failures Auto-healing containers do nothing if the Docker daemon itself crashes or the host reboots. For a fully self-healing setup:\n1. Enable Docker Restart on Boot 1 systemctl enable --now docker 2. Use restart: unless-stopped on All Services This ensures all containers restart after a daemon restart.\n3. Use systemd for Critical Infrastructure As described above, systemd services survive Docker daemon failures better than pure Docker restart policies:\n1 systemctl enable docker-traefik.service 4. Monitor the Auto-Heal Daemon Itself The auto-heal container has no one to heal it. Use a simple cron job or systemd timer:\n1 2 # /etc/cron.d/autoheal-health * * * * * root docker inspect --format=\u0026#39;{{.State.Health.Status}}\u0026#39; autoheal | grep -q healthy \u0026amp;\u0026amp; exit 0 || docker restart autoheal Or run the watchdog script as a systemd service so systemd restarts it if it crashes.\nNotification and Observability When auto-heal fires, you want to know about it. Wire up notifications to catch patterns before they become chronic:\ndocker-autoheal with Webhook docker-autoheal can call a webhook after each restart. Set the AUTOHEAL_WEBHOOK_URL environment variable:\n1 2 3 4 5 6 7 8 services: autoheal: image: willfarrell/autoheal:latest environment: - AUTOHEAL_CONTAINER_LABEL=all - AUTOHEAL_WEBHOOK_URL=https://hooks.slack.com/services/xxx volumes: - /var/run/docker.sock:/var/run/docker.sock Log Aggregation Forward Docker events to your logging stack:\n1 2 3 4 docker events --filter \u0026#39;event=health_status\u0026#39; --format \u0026#39;{{json .}}\u0026#39; | \\ while read event; do echo \u0026#34;$event\u0026#34; | logger -t docker-health done Prometheus Metrics For node_exporter deployments with the textfile collector configured for /var/lib/node_exporter/textfile, this example exports container health. Run it with permission to query Docker and write to that directory:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 #!/bin/bash # Simple health metrics exporter for Prometheus node_exporter textfile mkdir -p /var/lib/node_exporter/textfile while true; do { echo \u0026#34;# HELP docker_container_health Docker container health status\u0026#34; echo \u0026#34;# TYPE docker_container_health gauge\u0026#34; docker ps --format \u0026#39;{{.Names}}\\t{{.Status}}\u0026#39; | while IFS=$\u0026#39;\\t\u0026#39; read name status; do case \u0026#34;$status\u0026#34; in *unhealthy*) health=0 ;; *healthy*) health=1 ;; *) health=-1 ;; esac echo \u0026#34;docker_container_health{container=\\\u0026#34;$name\\\u0026#34;} $health\u0026#34; done } \u0026gt; /var/lib/node_exporter/textfile/docker_health.prom.$$ mv /var/lib/node_exporter/textfile/docker_health.prom.$$ \\ /var/lib/node_exporter/textfile/docker_health.prom sleep 15 done When Not to Auto-Heal Auto-healing is not always the right answer. Consider the following before enabling it on every container:\nStateful databases: restarting a corrupted database container may make things worse. Prefer manual investigation. One-shot jobs: containers that run to completion should not be auto-healed. Rate-limited services: if a container is unhealthy due to API rate limiting, restarting resets the timer but does not solve the root cause. Config errors: a container that fails at startup due to a bad config file will fail again after restart. Fix the config. Use labels selectively. Apply autoheal=true only to containers where automatic recovery is safe and desirable.\nSummary Docker provides excellent health detection but no built-in recovery. For a truly self-healing homelab:\nAdd HEALTHCHECK to every service — in Dockerfiles, Compose files, or runtime flags. Deploy docker-autoheal for zero-config auto-restart on healthy labeled containers. Use systemd services for critical infrastructure that must survive Docker daemon failures. Add notifications — Slack, webhook, or Prometheus — so you know when auto-heal fires. Label selectively — not every container should auto-heal. The five minutes it takes to add health checks and auto-heal to your Compose file pays for itself the first time a container fails at 3 AM.\n","permalink":"https://blog.gntech.me/posts/2026-05-22-docker-container-auto-healing/","summary":"Docker marks unhealthy containers but does not restart them. Build a self-healing homelab with docker-autoheal, systemd watchdog services, custom health scripts, and Uptime Kuma webhooks — no Kubernetes needed.","title":"Docker Container Auto-Healing for Homelab Deployments"},{"content":" Correction — October 30, 2026: The original version of this post described an alleged “nginx-poolslip” zero-day as confirmed and unpatched. That claim was not supported by an identified vendor advisory. The current official NGINX advisory index does not list that name. This does not prove that no private report exists, but the original article did not justify its certainty. Its claimed exploit details, affected-version table, and workaround claims have been withdrawn.\nUse the vendor\u0026rsquo;s published advisories to decide whether your actual NGINX package needs an update. Do not treat this post\u0026rsquo;s historical title or publication date as a current vulnerability notification.\n1. Identify the Running Build For a host installation:\n1 2 nginx -v nginx -V For a container, replace nginx with its actual name:\n1 2 3 docker exec nginx nginx -v docker exec nginx nginx -V docker inspect nginx --format \u0026#39;{{.Config.Image}}\u0026#39; Record the package source, image digest where applicable, configured modules, and deployed version. Distribution packages may include backported fixes without adopting the upstream version number.\n2. Match a Published Advisory Read the official NGINX security advisories and the relevant distribution or image publisher\u0026rsquo;s security notice.\nFor example, the vendor index identifies CVE-2026-42945 as a buffer overflow in ngx_http_rewrite_module, with upstream fixes in 1.31.0 and 1.30.1. The original article\u0026rsquo;s characterization of 1.31.0 as a stable release was also incorrect: NGINX uses separate mainline and stable branches.\nCheck the advisory\u0026rsquo;s affected versions and configuration prerequisites. Do not assume a WAF, rate limit, disabled server_tokens, or an image tag alone proves protection against a memory-safety issue.\n3. Update Through Your Existing Deployment Back up your configuration and verify your rollback plan before an update. Choose a publisher-supported version or digest that includes the required fixes. Avoid downgrading to an arbitrary older release as a generic mitigation.\nFor an existing Compose deployment, after reviewing its image reference, replace nginx below with the Compose service name:\n1 2 3 4 5 6 7 docker compose config --quiet docker compose pull nginx docker compose run --rm --no-deps nginx nginx -t docker compose up -d --no-deps nginx docker compose exec nginx nginx -t docker compose exec nginx nginx -v docker compose logs --tail=100 nginx The pre-deployment test uses the new image and the service\u0026rsquo;s configured mounts. It may require running upstream services if the configuration resolves their names. Adapt the validation command for images with a custom entrypoint.\nAfterward, test the externally used HTTPS URL and upstream application behavior. Confirm the running binary version and publisher advisory status. Keep a record of the applied image digest or package revision.\nReferences NGINX security advisories NGINX release branches and downloads ","permalink":"https://blog.gntech.me/posts/2026-05-22-nginx-poolslip-security-advisory/","summary":"Correction of an unsupported nginx-poolslip advisory, with steps to verify official NGINX advisories, identify running versions, and validate an update.","title":"NGINX Security Advisory Verification — Correction to the Poolslip Post"},{"content":"Plex raised prices, removed the lifetime pass as a purchase option, and pushed cloud-dependent features harder than ever. If you\u0026rsquo;re running a homelab media server, now is the time to switch to Jellyfin — a fully open-source, free, self-hosted alternative with no subscriptions, no phoning home, and complete control over your library.\nThis guide walks through the full migration from Plex to Jellyfin in Docker on a Linux homelab: setting up the stack, configuring hardware transcoding with Intel QuickSync or NVIDIA NVENC, exposing it securely through Traefik, and integrating it with Sonarr, Radarr, and the rest of the arrs ecosystem.\nWhy Jellyfin Over Plex in 2026 Feature Plex Jellyfin License Proprietary (subscription) GPL 2.0 / AGPL 3.0 Cost $120/yr or legacy lifetime Free Hardware transcoding Plex Pass required Built-in, no paywall Intro skip Plex Pass required Free + plugin Client support Wide (native apps) Wide (web, Android TV, Roku, LG, Samsung) Offline mode Requires cloud auth Fully offline capable Docker support Official image Official + linuxserver images Remote access Relay/prerequisite auth Direct or reverse proxy The bottom line: Jellyfin gives you everything Plex charges for — and without sending metadata or authentication through Jellyfin\u0026rsquo;s servers.\nPrerequisites Docker and Docker Compose v2 installed on the host Media files on a mounted volume (local, NFS, or NAS) (Optional) Intel iGPU or NVIDIA GPU for transcoding (Optional) Traefik or Nginx for secure remote access docker user in the render and video groups for GPU access Step 1: Jellyfin Docker Compose Setup Create a project directory and compose file:\n1 2 mkdir -p /opt/jellyfin cd /opt/jellyfin 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 # docker-compose.yml services: jellyfin: image: jellyfin/jellyfin:latest container_name: jellyfin restart: unless-stopped network_mode: host volumes: - ./config:/config - ./cache:/cache - /path/to/media:/media:ro devices: - /dev/dri:/dev/dri # Intel/AMD GPU transcoding # - /dev/dri/renderD128:/dev/dri/renderD128 environment: - JELLYFIN_PublishedServerUrl=homeserver.local # local hostname - PUID=1000 - PGID=1000 group_add: - \u0026#34;44\u0026#34; # video - \u0026#34;107\u0026#34; # render - \u0026#34;1000\u0026#34; # user primary group Key decisions:\nnetwork_mode: host avoids container networking overhead for UPnP/DLNA and simplifies hardware transcoding device access. Use a bridge network with Traefik if you need per-service networking (shown in Step 5). Volume mounts: bind-mount your media read-only (:ro) to prevent accidental deletion or modification from Jellyfin. Device passthrough: /dev/dri exposes Intel or AMD iGPUs. For NVIDIA, use nvidia-container-toolkit. group_add ensures the container process can access DRI render nodes. Check getent group video and getent group render on your host for the correct GIDs. Start the container:\n1 docker compose up -d Access Jellyfin at http://homeserver.local:8096 and complete the initial setup wizard — choose your language, create the admin account, and point to your media folders.\nNVIDIA GPU Transcoding Setup If you use an NVIDIA GPU instead of Intel QuickSync:\n1 2 3 # Install nvidia-container-toolkit on the host sudo apt install nvidia-container-toolkit sudo systemctl restart docker 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 # docker-compose.yml (NVIDIA variant) services: jellyfin: image: nvcr.io/nvidia/jellyfin:latest # or jellyfin/jellyfin:latest container_name: jellyfin restart: unless-stopped network_mode: host volumes: - ./config:/config - ./cache:/cache - /path/to/media:/media:ro environment: - NVIDIA_VISIBLE_DEVICES=all - NVIDIA_DRIVER_CAPABILITIES=compute,video,utility deploy: resources: reservations: devices: - driver: nvidia count: 1 capabilities: [gpu, video] Enable NVIDIA NVENC in Jellyfin Admin → Playback → Transcoding → Hardware acceleration: select NVIDIA NVENC.\nStep 2: Hardware Transcoding Configuration Log into Jellyfin at http://homeserver.local:8096, go to Administration → Dashboard → Playback → Transcoding.\nFor Intel QuickSync (QSV):\nHardware acceleration: Intel QuickSync (QSV) Enable: H.264, HEVC, VC1, MPEG2, VP9, AV1 (if supported) Enable: Allow encoding in HEVC format Enable: Allow subtitle extraction on the fly Enable: Enable VPP tone mapping (HDR → SDR conversion) Enable: Deinterlacing (hardware) Prefer OS native DXVA: unchecked (Linux) For NVIDIA NVENC:\nHardware acceleration: NVIDIA NVENC Enable: H.264 (NVENC), HEVC (NVENC), AV1 (NVENC) Check Enable hardware encoding Set Thread count to 0 (auto) Enable Tone mapping (NVENC handles HDR → SDR) Verify transcoding works:\n1 2 # Watch Jellyfin logs while playing a 4K video docker compose logs -f You should see lines like:\n[INF] [1080p/H264] DirectPlay [INF] [4K/HDR] Transcoding to 1080p H264 — encoder: QSV h264_qsv If transcoding falls back to software, check:\nContainer user has video and render group membership /dev/dri exists inside the container: ls -la /dev/dri Intel GPU drivers are installed: sudo apt install intel-media-va-driver-non-free Verify VA-API: sudo vainfo Step 3: Transferring Watch History from Plex (Optional) Jellyfin doesn\u0026rsquo;t import Plex data natively, but third-party tools make the migration painless.\nOption A: jellywatch.app (Automated) jellywatch.app provides a web-based tool that exports your Plex watch state and imports it directly into Jellyfin. It handles:\nWatched/unwatched status per user Ratings and review data Playlist migration Collection metadata This is the easiest option for most users — ~10 minutes for a large library.\nOption B: Plex-Trakt-Sync + Trakt (Manual) If you already use Trakt:\n1 2 3 # Export Plex watch state to Trakt pip install plex_trakt_sync plex_trakt_sync sync Then set up the Jellyfin Trakt plugin in Admin Dashboard → Plugins → Catalog → Trakt. Configure your Trakt credentials and it backfills Jellyfin playback history from Trakt.\nOption C: Direct Database Migration (Advanced) For users comfortable with SQLite:\n1 2 3 4 5 # Download and run plex-to-jellyfin script wget https://raw.githubusercontent.com/jwetzell/Plex-To-Jellyfin/main/Plex_To_Jellyfin.py python3 Plex_To_Jellyfin.py \\ --plex-db /path/to/Plex/Media/Server/Plug-in Support/Databases/com.plexapp.plugins.library.db \\ --jellyfin-db /opt/jellyfin/config/data/library.db This copies directly at the database level — fast but comes with no warranty. Always back up both databases first.\nStep 4: Arrs Integration (Sonarr / Radarr) Jellyfin connects to Sonarr and Radarr through its API — no special plugin required.\nIn Sonarr (and similarly for Radarr):\nSettings → Media Management → Import Lists Add → Jellyfin Fill in: Server URL: http://homeserver.local:8096 API Key: from Jellyfin Admin Dashboard → API Keys Test connection Why this matters: Sonarr/Radarr can automatically scan and import new media when Jellyfin refreshes — but more importantly, the two-way integration means Sonarr removes episodes from Jellyfin\u0026rsquo;s \u0026ldquo;Continue Watching\u0026rdquo; when they\u0026rsquo;re deleted.\nRecommended arrs stack:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 services: # ... jellyfin config above ... sonarr: image: lscr.io/linuxserver/sonarr:latest container_name: sonarr restart: unless-stopped network_mode: host volumes: - ./sonarr/config:/config - /path/to/media:/media radarr: image: lscr.io/linuxserver/radarr:latest container_name: radarr restart: unless-stopped network_mode: host volumes: - ./radarr/config:/config - /path/to/media:/media prowlarr: image: lscr.io/linuxserver/prowlarr:latest container_name: prowlarr restart: unless-stopped network_mode: host volumes: - ./prowlarr/config:/config qbittorrent: image: lscr.io/linuxserver/qbittorrent:latest container_name: qbittorrent restart: unless-stopped network_mode: host volumes: - ./qbittorrent/config:/config - /path/to/downloads:/downloads environment: - PUID=1000 - PGID=1000 - TZ=America/Santo_Domingo Step 5: Traefik Reverse Proxy for Remote Access Running on network_mode: host your media server is confined to internal access. To expose Jellyfin securely through Traefik on a bridge network, create a separate compose file with proper networking:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 # jellyfin-traefik.yml services: jellyfin: image: jellyfin/jellyfin:latest container_name: jellyfin restart: unless-stopped networks: - proxy volumes: - ./config:/config - ./cache:/cache - /path/to/media:/media:ro devices: - /dev/dri:/dev/dri environment: - JELLYFIN_PublishedServerUrl=https://media.yourdomain.com labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.jellyfin.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.jellyfin.rule=Host(`media.yourdomain.com`)\u0026#34; - \u0026#34;traefik.http.routers.jellyfin.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.services.jellyfin.loadbalancer.server.port=8096\u0026#34; - \u0026#34;traefik.http.routers.jellyfin.middlewares=secHeaders@file,rateLimit@file\u0026#34; - \u0026#34;traefik.http.routers.jellyfin-ws.rule=Host(`media.yourdomain.com`) \u0026amp;\u0026amp; PathPrefix(`/socket`)\u0026#34; - \u0026#34;traefik.http.routers.jellyfin-ws.service=jellyfin\u0026#34; - \u0026#34;traefik.http.routers.jellyfin-ws.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.jellyfin-ws.tls.certresolver=letsencrypt\u0026#34; networks: proxy: external: true Key details:\nWebSocket routing is required for Jellyfin\u0026rsquo;s real-time playback sync and live TV features. Without the separate jellyfin-ws router, WebSocket connections fail. JELLYFIN_PublishedServerUrl must match your external hostname for client redirects to work correctly. Rate limiting via rateLimit@file prevents the login endpoint from being brute-forced remotely. If you run the Traefik setup, set internal_networks in CrowdSec or your firewall rules to only allow LAN IPs access to port 8096 directly (avoid exposing the backend port remotely — always route through Traefik).\nStep 6: Client Migration Jellyfin has clients for virtually every platform:\nPlatform Client Android TV / Fire TV Jellyfin for Android TV (Play Store or direct APK) Apple TV Swiftfin (App Store) LG WebOS Jellyfin for WebOS (developer mode) Samsung Tizen Jellyfin for Tizen (developer mode) Roku Jellyfin for Roku (private channel) Web Browser https://media.yourdomain.com (direct) iOS Swiftfin or Jellyfin Mobile (App Store) Kodi Jellyfin for Kodi plugin Game Consoles PS4/5 via browser, Xbox via browser or UWP Most clients support Direct Play when on the same network, and gracefully fall back to transcoding over remote connections.\nPost-Migration Checklist Verify playback: Test 4K HDR → 1080p SDR transcoding Check remote access: Login from external network Test mobile clients: Install Swiftfin on iOS/Android Configure backups: Back up /opt/jellyfin/config Set up intro skip: Install the \u0026ldquo;Scheduled Task\u0026rdquo; plugin from Admin → Plugins → Catalog → Intro Skipper Enable hardware acceleration exhaustively — test each codec you actually have in your library Disable or remove Plex once satisfied Wrapping Up Jellyfin is production-grade, actively maintained, and has matured significantly over the past few years. The Docker Compose setup takes under 15 minutes, hardware transcoding just works on modern Intel and NVIDIA hardware, and the arrs integration is seamless. With Plex\u0026rsquo;s lifetime pass effectively dead, there\u0026rsquo;s never been a better time to switch.\nThe full media stack — Jellyfin, Sonarr, Radarr, Prowlarr, qBittorrent — runs comfortably on a Proxmox LXC or VM with 4 vCPUs and 8 GB RAM, leaving hardware transcoding to the iGPU. Your future self will thank you for the subscription-free setup and the warm feeling of knowing your media server belongs entirely to you.\n","permalink":"https://blog.gntech.me/posts/2026-05-22-plex-to-jellyfin-migration-guide/","summary":"Migrate from Plex to Jellyfin on Docker with this complete guide covering library setup, watch history transfer, hardware transcoding with Intel QSV or NVIDIA NVENC, Traefik reverse proxy, and full arrs stack integration for your homelab media server.","title":"Plex to Jellyfin Migration Guide — Docker Homelab Setup 2026"},{"content":"If your homelab runs Proxmox VE, May 21, 2026 marks a major release: Proxmox VE 9.2 ships with QEMU 11.0, LXC 7.0, ZFS 2.4, the Linux 7.0 kernel, and a brand-new Dynamic Load Balancer for automated cluster resource optimization.\nThis guide covers the full upgrade from Proxmox VE 8.4 to 9.2 — including the new features worth enabling after the reboot, known pitfalls like LXC containers failing to start on kernel 7.0, and the config changes you need to make for a clean upgrade on a homelab cluster.\nWhy Upgrade to Proxmox VE 9.2? Proxmox VE 9.2 bundles several major updates under the hood:\nComponent Proxmox 8.x Proxmox 9.2 Base OS Debian 12 Bookworm Debian 13 Trixie Kernel 6.8 (default) 7.0 (Linux 7.0) QEMU 8.x 11.0 LXC 5.x 7.0 ZFS 2.1.x 2.4 Ceph Squid 19 (optional) Tentacle 20.2 (default) The headline feature is the Dynamic Load Balancer, which automatically redistributes VMs and containers across cluster nodes based on CPU and memory utilization — similar to VMware DRS. Other notable additions include:\nSDN WireGuard fabrics — encrypted overlay networks between cluster nodes without dedicated VPN hardware. BGP/EVPN route maps and prefix lists — fine-grained control for advanced SDN setups. Custom CPU profiles in the web UI — create, edit, and remove CPU profiles per VM from Datacenter → Options. HA rules replacing HA groups — more flexible scheduling policies. Prerequisites Before touching the upgrade, verify these prerequisites:\n1. Run the Latest Proxmox VE 8.4 1 2 pveversion # Must report at least 8.4.1 If your version is older, update first:\n1 apt update \u0026amp;\u0026amp; apt dist-upgrade -y 2. Run the Pre-Upgrade Checklist The pve8to9 script validates your system before the upgrade. It is included in the latest Proxmox VE 8.4 packages:\n1 pve8to9 --full Run it now. It will flag:\nLVM volumes with autoactivation enabled (common on LVM-thin storages) Third-party repositories that might conflict with Trixie Packages that will be removed during the dist-upgrade Fix each warning before proceeding. Re-run pve8to9 --full after each fix.\n3. Backup Everything Do not skip this. Back up all VMs and containers to external or NFS/SMB storage via the GUI or:\n1 vzdump \u0026#34;\u0026lt;VMID\u0026gt;\u0026#34; --storage backup-nfs --mode snapshot Also back up critical host config:\n1 tar czf /root/pve-etc-backup-$(date +%F).tar.gz /etc/pve /etc/network/interfaces /etc/hostname /etc/resolv.conf 4. Ensure Enough Disk Space 1 2 df -h / # Need at least 5 GB free, ideally 10+ GB The dist-upgrade downloads hundreds of packages. The apt cache alone can reach 1-2 GB during the process.\n5. Use tmux or screen If connecting via SSH (not IPMI/iKVM), wrap the session:\n1 tmux new -s pve-upgrade This protects against connection drops mid-upgrade.\nStep-by-Step: In-Place Upgrade to Proxmox VE 9.2 Step 1: Update Repositories to Debian Trixie Switch Debian base and Proxmox repositories from Bookworm to Trixie:\n1 2 3 4 5 # Update Debian base repos sed -i \u0026#39;s/bookworm/trixie/g\u0026#39; /etc/apt/sources.list # Update Proxmox enterprise or no-subscription repo sed -i \u0026#39;s/bookworm/trixie/g\u0026#39; /etc/apt/sources.list.d/pve-enterprise.list 2\u0026gt;/dev/null || true Step 2: Add Proxmox VE 9 Repositories (deb822 Format) Replace the old .list files with the new deb822 .sources format. For the no-subscription repository (most homelabs):\n1 2 3 4 5 6 7 cat \u0026gt; /etc/apt/sources.list.d/proxmox.sources \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; Types: deb URIs: http://download.proxmox.com/debian/pve Suites: trixie Components: pve-no-subscription Signed-By: /usr/share/keyrings/proxmox-archive-keyring.gpg EOF If you have an enterprise subscription:\n1 2 3 4 5 6 7 cat \u0026gt; /etc/apt/sources.list.d/pve-enterprise.sources \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; Types: deb URIs: https://enterprise.proxmox.com/debian/pve Suites: trixie Components: pve-enterprise Signed-By: /usr/share/keyrings/proxmox-archive-keyring.gpg EOF Remove the old .list files after verifying the new ones work:\n1 2 3 4 5 6 7 apt update \u0026amp;\u0026amp; apt policy # Verify output shows trixie repositories # Then remove old repo files rm -f /etc/apt/sources.list.d/pve-enterprise.list rm -f /etc/apt/sources.list.d/pve-install-repo.list apt update Step 3: Run the Dist-Upgrade 1 apt dist-upgrade This upgrades the entire system from Debian 12 → 13 and Proxmox VE 8 → 9 in one pass. Expect 5-60 minutes depending on disk speed.\nDuring the process you will be prompted about config file changes. Recommended choices:\nFile Recommendation /etc/issue Keep current (cosmetic only) /etc/lvm/lvm.conf Install maintainer version /etc/ssh/sshd_config Install maintainer version /etc/default/grub Keep current (review diffs first) /etc/chrony/chrony.conf Install maintainer version Step 4: Reboot 1 reboot After reboot, verify the kernel:\n1 2 3 4 uname -r # Should show: 7.0.x-pve pveversion # Should show: pve-manager/9.2.x Step 5: Clean Up apt Repositories (Optional) Modernize remaining .list files to deb822 format:\n1 apt modernize-sources This creates .sources files and renames old .list files to .list.bak. Verify everything works and delete the .bak files once confirmed.\nPost-Upgrade: What to Fix First LVM Autoactivation Migration Upgrading from PVE 8, your existing LVM guest volumes may still have autoactivation enabled. This causes issues on shared LVM storages in clusters. Run the migration script:\n1 /usr/share/pve-manager/migrations/pve-lvm-disable-autoactivation Re-run pve8to9 --full to confirm the warning is gone.\nLXC Containers Not Starting on Kernel 7.0 A common issue reported on kernel 7.0 is LXC containers failing to start with errors like:\nlxc_map_ids: newuidmap failed to write mapping This is caused by systemd-tmpfiles not creating /run/pve early enough. Fix:\n1 2 3 4 5 6 7 8 9 systemctl status pve-guests.service # Check for failures # Ensure tmpfiles.d is configured correctly systemd-tmpfiles --cat-config | grep pve # If /run/pve is missing, create it and restart: mkdir -p /run/pve pve-container restart --all For persistent issues, ensure pve-container and pve-guests services are enabled:\n1 2 systemctl enable --now pve-guests.service systemctl enable --now pve-container.service Disable Audit Noise (Optional) Debian Trixie enables kernel audit logging by default. To suppress excessive audit messages during normal operation:\n1 systemctl mask --now systemd-journald-audit.socket Configuring the Dynamic Load Balancer The Dynamic Load Balancer is the standout feature in 9.2. It automatically rebalances VMs and containers across cluster nodes based on resource utilization — no more manually deciding where to place guests.\nPrerequisites All cluster nodes must be on Proxmox VE 9.2. HA must be enabled on the cluster (Datacenter → HA). A valid quorum (at least 3 nodes or a QDevice). Enable Dynamic Load Balancing Navigate to Datacenter → Options and set:\nHA Scheduling: Dynamic Load Re-balance method: Default (bruteforce) or TOPSIS Automatically re-balance HA resources: ✓ enabled From the CLI:\n1 2 3 4 pvesh set /cluster/ha/options \\ --scheduler dynamic \\ --rebalance-method bruteforce \\ --rebalance-automatic 1 The load balancer periodically evaluates node utilization and moves low-priority VMs from overloaded nodes to underutilized ones. Migration thresholds are configurable:\n1 2 3 4 5 pvesh set /cluster/ha/options \\ --cpu-threshold-overloaded 80 \\ --memory-threshold-overloaded 85 \\ --cpu-threshold-underloaded 30 \\ --memory-threshold-underloaded 40 HA Rules vs. HA Groups Proxmox VE 9 deprecates HA groups in favor of HA rules. If you upgrade from PVE 8 with existing HA groups, they are automatically migrated to HA rules once all cluster nodes are on PVE 9.\nVerify the migration:\n1 journalctl -eu pve-ha-crm Create new rules from the GUI or CLI:\n1 2 3 4 5 6 pvesh create /cluster/ha/rules \\ --id my-rule \\ --type resource \\ --match \u0026#34;tags:critical\u0026#34; \\ --action spread \\ --priority 100 Other New Features Worth Enabling SDN WireGuard Fabrics Proxmox VE 9.2 adds WireGuard as a native SDN fabric protocol. This creates encrypted overlay networks between cluster nodes without configuring WireGuard manually on each host.\nDatacenter → SDN → Zones → Add → Fabric\nSelect WireGuard as the fabric type, assign a subnet, and select which nodes participate. The zone is automatically provisioned and key exchange is handled by the cluster configuration.\nCustom CPU Profiles Navigate to Datacenter → Options → CPU Profiles to create custom CPU type profiles. Useful for:\nExposing specific CPU features to VMs (e.g., AVX-512 for AI workloads) Matching CPU models across heterogeneous cluster hardware Hiding features for cross-cluster migration compatibility BGP/EVPN Route Maps If you run BGP/EVPN in your homelab (for example, integrating Proxmox SDN with RouterOS or FRR), the new route maps and prefix lists give you fine-grained control over route advertisement. Configure under Datacenter → SDN → IPAM / Route Maps.\nKnown Issues and Workarounds Veeam Backup Compatibility PVE 9.2 uses QEMU machine version 10.0+ by default. Veeam has not adapted to the new internal disk attachment changes. If you use Veeam for backups, pin affected VMs to machine version 9.2+pve1:\n1 qm set \u0026#34;\u0026lt;VMID\u0026gt;\u0026#34; --machine pc-q35-9.2+pve1 /tmp is Now tmpfs Debian Trixie mounts /tmp as tmpfs using up to 50% of system memory by default. If your workflows depend on large temp files, adjust the mount:\n1 mount -o remount,size=10G /tmp Or revert to disk-backed /tmp:\n1 systemctl mask tmp.mount Ceph Updates If you run hyper-converged Ceph, the default is now Ceph Tentacle 20.2.1. Ceph Squid 19.2.3 remains available as an alternative. Check compatibility before upgrading:\n1 ceph --version Verifying a Clean Upgrade Run the checklist once more after reboot:\n1 pve8to9 --full Expected clean output — no warnings or errors. Then:\n1 2 3 4 5 6 pveversion -v | head -10 # pve-manager: 9.2.x # proxmox-ve: 9.2.x # proxmox-kernel-7.0: 7.0.x # qemu-server: 11.x # lxc-pve: 7.x Check all VMs start properly:\n1 2 qm list pct list Test HA rebalancing by creating a test VM with HA enabled, loading it with stress, and verifying the dynamic load balancer moves it:\n1 2 3 4 5 6 # Install stress in a test VM apt install stress stress --cpu 4 --timeout 120 # Monitor load balancer activity journalctl -fu pve-ha-crm Summary Proxmox VE 9.2 is a significant release for homelab users. The Dynamic Load Balancer eliminates the manual guesswork of VM placement, SDN WireGuard fabrics simplify encrypted cluster networking, and the underlying upgrade to Debian Trixie with Linux 7.0 brings modern kernel features.\nKey takeaways:\nRun pve8to9 --full before and after the upgrade. Use tmux — this is a long-running dist-upgrade. Fix LVM autoactivation post-upgrade to avoid storage issues in clusters. Enable the Dynamic Load Balancer after all nodes are upgraded. Pin Veeam-backed VMs to machine version 9.2+pve1 if needed. Test LXC containers after the kernel 7.0 upgrade and apply the tmpfiles fix if they fail to start. The 9.2 release transforms Proxmox from a capable hypervisor into an actively load-managed virtualization platform — and for homelab operators managing multiple nodes, that alone is worth the upgrade.\n","permalink":"https://blog.gntech.me/posts/2026-05-22-proxmox-9-2-upgrade-guide/","summary":"Upgrade your homelab from Proxmox VE 8.x to 9.2 with this practical guide covering the new Dynamic Load Balancer, SDN WireGuard fabrics, kernel 7.0 migration, known LXC issues, and post-upgrade tuning.","title":"Proxmox VE 9.2 Upgrade Guide — From 8.x with Dynamic Load Balancer"},{"content":"RouterOS has supported Linux containers since v7.4beta, and the feature has matured significantly through v7.16+ and into 2026\u0026rsquo;s v7.18 releases. You can pull container images from Docker Hub, GCR, Quay, or any OCI-compatible registry and run them directly on your MikroTik router — no separate server required.\nThink about it: your MikroTik router sits at the edge of your network with a direct view of all traffic. Running lightweight containers like Pi-hole, AdGuard Home, Tailscale, or NetBird on the router itself eliminates an extra hop, reduces power consumption, and simplifies your stack.\nThis guide covers the full setup from scratch: enabling container mode, configuring networking, pulling images, setting up persistent storage, and practical real-world use cases.\nRequirements and Limitations Before diving in, check that your hardware is compatible.\nHardware Support The container package works on arm, arm64, and x86 architectures. Not all RouterBOARD models support it:\nARM64 (recommended): RB5009, CCR2116, CCR2216, hAP ax³/l³, L009, Chateau 5G — these have enough RAM and storage ARM (32-bit): hEX Refresh (RB750Gr3) — only arm32v5 images, limited selection x86: CHR and x86 installations — full support with external storage Not supported: Older MIPSBE, MIPSBLE, and Tile-based devices (RB7xx, RB9xx, RB1100, and most pre-2019 hardware) Storage Requirements Containers consume significant storage. RouterOS internal flash (usually 16MB–512MB) is not enough. You must attach external storage:\nUSB flash drive or SSD SATA SSD (on models with SATA ports like RB5009 or CCR series) microSD card (on hAP ax³ or L009 — use A2-rated for speed) Recommendation: at least 8GB, preferably 16GB+ for multiple containers. The filesystem should be ext4, which RouterOS supports natively.\nPerformance Considerations Each container consumes CPU and RAM. A RouterBOARD RB5009 with its 4-core Cortex-A72 can comfortably run 3–5 lightweight containers (Pi-hole, Tailscale, Node Exporter). Resource-heavy containers like databases or media transcoders are not suitable.\nMikroTik recommends external storage with at least 100 MB/s sequential read/write and 10K random IOPS — easily met by a USB 3.0 SSD.\nStep 1 — Enable Container Mode Container mode is gated behind device-mode for security reasons. Physical access to the router is required for the first step.\nOn ARM/ARM64 Devices From a terminal session on the router (SSH or serial):\n/system/device-mode/update container=yes The router will prompt you to press the physical reset button within 30 seconds. Press and hold the button until the LED pattern changes. The device reboots with container mode enabled.\nOn x86 / CHR For x86 and CHR installations, cold-reboot the VM or machine instead of the reset button:\n/system/device-mode/update container=yes Then reboot:\n/system/reboot Install the Container Package RouterOS packages are managed via the package menu:\n/system/package/update/set channel=stable /system/package/update/install After updating, verify the container package is available:\n/container/config/print If you see output like registry-url: https://lscr.io, the package is installed and ready.\nIf not, install it from the packages menu at System \u0026gt; Packages \u0026gt; Available Packages in WinBox, or use:\n/system/package/install container /reboot Step 2 — Configure Container Networking Containers need a virtual Ethernet (veth) interface and a bridge to communicate with the rest of your network.\nCreate the Bridge /interface/bridge/add name=containers /ip/address/add address=172.17.0.1/24 interface=containers Create a Veth Interface /interface/veth/add name=veth1 address=172.17.0.2/24 gateway=172.17.0.1 Attach Veth to Bridge /interface/bridge/port add bridge=containers interface=veth1 Masquerade Outbound Traffic Containers need NAT to reach the internet:\n/ip/firewall/nat/add chain=srcnat action=masquerade src-address=172.17.0.0/24 Enable IP Forwarding (if not already) /ip/settings/set forward=yes At this point, any container attached to this veth interface will have outbound internet access through the router\u0026rsquo;s WAN.\nMultiple Containers per Veth You can attach multiple containers to the same veth interface if they listen on different ports. Alternatively, create isolated veth pairs per container:\n/interface/veth/add name=veth-pihole address=172.17.0.3/24 gateway=172.17.0.1 /interface/veth/add name=veth-tailscale address=172.17.0.4/24 gateway=172.17.0.1 /interface/bridge/port add bridge=containers interface=veth-pihole /interface/bridge/port add bridge=containers interface=veth-tailscale Step 3 — Configure the Container Registry RouterOS supports multiple registries. Set Docker Hub as your primary source:\n/container/config/set \\ registry-url=https://registry-1.docker.io \\ tmpdir=disk1/tmp Replace disk1 with your external drive label. Find yours:\n/disk/print The tmpdir is where container images are extracted. Point it to external storage to avoid filling the internal flash.\nStep 4 — Set Up Persistent Storage Containers are ephemeral by default. For stateful services like Pi-hole (which stores DNS logs and blocklists), mount directories from your external disk.\nCreate Mount Directories From the RouterOS shell:\n/file/print /file/mkdir disk1/containers/pihole /file/mkdir disk1/containers/pihole-dnsmasq /file/mkdir disk1/containers/adguard/work /file/mkdir disk1/containers/adguard/conf Register Mounts for Pi-hole /container/mounts/add \\ list=MOUNT_PIHOLE \\ src=disk1/containers/pihole \\ dst=/etc/pihole /container/mounts/add \\ list=MOUNT_PIHOLE_DNSMASQ \\ src=disk1/containers/pihole-dnsmasq \\ dst=/etc/dnsmasq.d On first start, the container will populate the mount directory with its default files.\nStep 5 — Deploy Pi-hole Pi-hole is the most common container on RouterOS and one of the most practical — it turns your router into a network-wide ad blocker with a web dashboard.\nSet Environment Variables /container/envs/add list=ENV_PIHOLE key=TZ value=\u0026#34;America/Santo_Domingo\u0026#34; /container/envs/add list=ENV_PIHOLE key=FTLCONF_webserver_api_password value=\u0026#34;your-secure-password\u0026#34; /container/envs/add list=ENV_PIHOLE key=DNSMASQ_USER value=\u0026#34;root\u0026#34; /container/envs/add list=ENV_PIHOLE key=WEBPASSWORD value=\u0026#34;your-admin-password\u0026#34; Add the Container /container/add \\ remote-image=pihole/pihole:latest \\ interface=veth1 \\ root-dir=disk1/images/pihole \\ envlist=ENV_PIHOLE \\ mountlists=MOUNT_PIHOLE,MOUNT_PIHOLE_DNSMASQ \\ start-on-boot=yes \\ logging=no \\ name=pihole Wait for the Pull and Extract Monitor progress:\n/container/print detail Wait until status changes to stopped. This means the image has been pulled and extracted. Then start it:\n/container/start pihole Enable DNS Forwarding Configure RouterOS to forward DNS queries to the container:\n/ip/dns/set allow-remote-requests=yes servers=172.17.0.2 Expose the Web Interface Add a destination NAT rule to access the Pi-hole admin panel:\n/ip/firewall/nat/add \\ chain=dstnat \\ dst-address=192.168.88.1 \\ dst-port=80 \\ protocol=tcp \\ action=dst-nat \\ to-addresses=172.17.0.2 \\ to-ports=80 Replace 192.168.88.1 with your router\u0026rsquo;s LAN IP. Now access the admin panel at http://192.168.88.1/admin/.\nStep 6 — Deploy AdGuard Home (Alternative) AdGuard Home is a popular Pi-hole alternative with a more modern interface and built-in DHCP server support.\nEnvironment Variables /container/envs/add list=ENV_ADGUARD key=TZ value=\u0026#34;America/Santo_Domingo\u0026#34; Mounts /container/mounts/add \\ list=MOUNT_ADGUARD_WORK \\ src=disk1/containers/adguard/work \\ dst=/opt/adguardhome/work /container/mounts/add \\ list=MOUNT_ADGUARD_CONF \\ src=disk1/containers/adguard/conf \\ dst=/opt/adguardhome/conf Add and Start /container/add \\ remote-image=adguard/adguardhome:latest \\ interface=veth1 \\ root-dir=disk1/images/adguard \\ envlist=ENV_ADGUARD \\ mountlists=MOUNT_ADGUARD_WORK,MOUNT_ADGUARD_CONF \\ start-on-boot=yes \\ name=adguard DNS and Port Forwarding /ip/firewall/nat/add \\ chain=dstnat \\ dst-address=192.168.88.1 \\ dst-port=80 \\ protocol=tcp \\ action=dst-nat \\ to-addresses=172.17.0.2 \\ to-ports=80 /ip/dns/set servers=172.17.0.2 AdGuard Home listens on port 3000 for the initial setup wizard. Forward it temporarily:\n/ip/firewall/nat/add \\ chain=dstnat \\ dst-address=192.168.88.1 \\ dst-port=3000 \\ protocol=tcp \\ action=dst-nat \\ to-addresses=172.17.0.2 \\ to-ports=3000 After initial configuration, AdGuard Home runs its admin interface on port 80.\nStep 7 — Deploy Tailscale on MikroTik Tailscale in a container on your MikroTik router creates a mesh VPN endpoint without any additional hardware. This is useful for accessing management interfaces from outside your LAN.\n/container/envs/add \\ list=ENV_TAILSCALE \\ key=TS_AUTHKEY value=\u0026#34;tskey-auth-xxxxxxxx\u0026#34; /container/envs/add \\ list=ENV_TAILSCALE \\ key=TS_STATE_DIR value=\u0026#34;/var/lib/tailscale\u0026#34; /container/mounts/add \\ list=MOUNT_TAILSCALE \\ src=disk1/containers/tailscale \\ dst=/var/lib/tailscale /container/add \\ remote-image=tailscale/tailscale:latest \\ interface=veth1 \\ root-dir=disk1/images/tailscale \\ envlist=ENV_TAILSCALE \\ mountlists=MOUNT_TAILSCALE \\ start-on-boot=yes \\ name=tailscale Important: the Tailscale container needs --privileged-like capabilities to create the WireGuard tunnel. RouterOS containers have limited support for the devices property. On some RouterOS versions, you may need to add:\n/container/set devices=tun tailscale After starting the container and authorizing the node, your router becomes part of your Tailscale tailnet.\nStep 8 — Management and Monitoring View Running Containers /container/print detail Key fields: status (running/stopped), uptime, cpu-load, memory-current, and root-dir usage.\nContainer Shell Access Enter a running container\u0026rsquo;s shell:\n/container/shell pihole This drops you into an interactive shell inside the container. Exit with Ctrl+D or type exit.\nStop and Start /container/stop pihole /container/start pihole Update a Container Image /container/update pihole This pulls the latest image, extracts it, and replaces the existing one. The container will restart automatically.\nView Container Logs /log/print where topics~\u0026#34;container\u0026#34; Or enable logging per container:\n/container/set logging=yes pihole Disable it afterward to avoid log noise:\n/container/set logging=no pihole Resource Limits Prevent a container from consuming all RAM:\n/container/set memory-max=134217728 pihole # 128 MB limit /container/set memory-high=67108864 pihole # 64 MB soft limit Troubleshooting Container Fails to Start /container/print detail Look for status and last-error fields. Common causes:\nStorage full — check /disk/print and free up space Image not fully extracted — wait and check status=extracting veth interface missing — verify /interface/veth/print DNS Resolution Inside Container If the container cannot resolve domain names, set explicit DNS:\n/container/set dns=1.1.1.1 pihole Pull Timeout for Large Images Large images may time out on slow connections. Increase the timeout by adjusting the pull settings (not directly configurable in older ROS versions). Alternatively, download the image on a PC and import it manually:\n/container/add file=pihole.tar.gz name=pihole-local Container Image Compatibility Use the architecture tag matching your CPU:\nARM64 (RB5009, CCR): arm64v8/ or generic :latest ARM32 (hEX Refresh): arm32v5/ — very limited library x86/CHR: amd64/ or generic :latest Summary MikroTik Container support in RouterOS is a powerful feature that turns your network edge device into a lightweight application host. For homelab use cases like DNS filtering (Pi-hole / AdGuard Home), mesh VPN (Tailscale / NetBird), or monitoring (Node Exporter), running a container directly on the router is more efficient than spinning up a separate machine.\nKey takeaways:\nExternal storage is mandatory — USB SSD or SATA, ext4 formatted, at least 8 GB Container mode requires physical access once, then remote management works normally Use the registry cache — repeated container/update calls reuse the extracted image layer to save bandwidth Monitor memory — RouterOS has limited RAM on most boards; set memory-max per container Start with Pi-hole — it is the most tested, well-documented container image for RouterOS The same setup works for dozens of other lightweight Docker images. Just ensure the architecture matches your RouterBOARD and keep resource limits tight. Your router is now a router and a container host.\n","permalink":"https://blog.gntech.me/posts/2026-05-21-mikrotik-containers-docker-routeros/","summary":"Run Docker containers directly on MikroTik RouterOS 7 — enable container mode, configure veth bridges, pull images from Docker Hub, and deploy Pi-hole or AdGuard Home without extra hardware.","title":"MikroTik Containers — Run Docker Images on RouterOS"},{"content":"If you run a homelab with multiple Docker services, VMs, and physical hosts, you have no real visibility into what is happening. A container goes OOM, a disk fills up, or a service stops responding — and you only discover it when something breaks.\nA Prometheus Grafana monitoring stack built on Docker Compose gives you metric collection, alerting, and visualization for your entire infrastructure. This guide walks through deploying Prometheus for time-series storage, Grafana for dashboards, Node Exporter for host metrics, cAdvisor for container metrics, and Alertmanager for push notifications — all behind a single Traefik reverse proxy.\nArchitecture Overview The stack consists of five components:\nComponent Role Exposes Prometheus Time-series database and metric scraper Port 9090 Grafana Dashboard and visualization layer Port 3000 Node Exporter Host-level metrics (CPU, RAM, disk, network) Port 9100 cAdvisor Container-level metrics (per-container resource usage) Port 8080 Alertmanager Handles and routes alerts from Prometheus Port 9093 Prometheus scrapes Node Exporter and cAdvisor at configurable intervals. Grafana queries Prometheus as a data source. Alertmanager receives firing alerts and sends notifications via email, Telegram, or webhook.\nDocker Compose Configuration Create a project directory and compose.yaml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 # /opt/monitoring/compose.yaml services: prometheus: image: prom/prometheus:v2.55.1 container_name: prometheus hostname: prometheus restart: unless-stopped command: - \u0026#39;--config.file=/etc/prometheus/prometheus.yaml\u0026#39; - \u0026#39;--storage.tsdb.path=/prometheus\u0026#39; - \u0026#39;--storage.tsdb.retention.time=30d\u0026#39; - \u0026#39;--web.console.libraries=/etc/prometheus/console_libraries\u0026#39; - \u0026#39;--web.console.templates=/etc/prometheus/consoles\u0026#39; - \u0026#39;--web.enable-lifecycle\u0026#39; volumes: - ./prometheus.yaml:/etc/prometheus/prometheus.yaml:ro - ./alert-rules.yaml:/etc/prometheus/alert-rules.yaml:ro - prometheus_data:/prometheus ports: - \u0026#34;127.0.0.1:9090:9090\u0026#34; networks: - monitoring grafana: image: grafana/grafana:11.4.0 container_name: grafana restart: unless-stopped environment: - GF_SECURITY_ADMIN_USER=admin - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD:-admin} - GF_INSTALL_PLUGINS=grafana-piechart-panel - GF_SERVER_ROOT_URL=https://grafana.gntech.home - GF_AUTH_ANONYMOUS_ENABLED=false volumes: - grafana_data:/var/lib/grafana - ./grafana/datasources:/etc/grafana/provisioning/datasources:ro - ./grafana/dashboards:/etc/grafana/provisioning/dashboards:ro - ./grafana/dashboards-json:/var/lib/grafana/dashboards:ro ports: - \u0026#34;127.0.0.1:3000:3000\u0026#34; networks: - monitoring depends_on: - prometheus node_exporter: image: prom/node-exporter:v1.8.2 container_name: node_exporter restart: unless-stopped command: - \u0026#39;--path.rootfs=/host\u0026#39; - \u0026#39;--collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($$|/)\u0026#39; pid: host volumes: - /:/host:ro,rslave ports: - \u0026#34;127.0.0.1:9100:9100\u0026#34; networks: - monitoring cadvisor: image: gcr.io/cadvisor/cadvisor:v0.49.1 container_name: cadvisor restart: unless-stopped privileged: true devices: - /dev/kmsg:/dev/kmsg volumes: - /:/rootfs:ro - /var/run:/var/run:ro - /sys:/sys:ro - /var/lib/docker/:/var/lib/docker:ro - /dev/disk/:/dev/disk:ro ports: - \u0026#34;127.0.0.1:8080:8080\u0026#34; networks: - monitoring alertmanager: image: prom/alertmanager:v0.27.0 container_name: alertmanager restart: unless-stopped command: - \u0026#39;--config.file=/etc/alertmanager/alertmanager.yaml\u0026#39; - \u0026#39;--storage.path=/alertmanager\u0026#39; volumes: - ./alertmanager.yaml:/etc/alertmanager/alertmanager.yaml:ro - alertmanager_data:/alertmanager ports: - \u0026#34;127.0.0.1:9093:9093\u0026#34; networks: - monitoring volumes: prometheus_data: grafana_data: alertmanager_data: networks: monitoring: name: monitoring external: false Note: All ports are bound to 127.0.0.1 so they are not directly exposed. Access goes through a reverse proxy (Traefik or Nginx) with authentication. For a single-host setup, you can omit the ports sections and use the internal Docker network exclusively.\nPrometheus Configuration Create prometheus.yaml with scrape targets and alerting rules:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 # /opt/monitoring/prometheus.yaml global: scrape_interval: 15s evaluation_interval: 15s external_labels: monitor: \u0026#39;homelab\u0026#39; alerting: alertmanagers: - static_configs: - targets: - alertmanager:9093 rule_files: - \u0026#39;alert-rules.yaml\u0026#39; scrape_configs: # Prometheus itself - job_name: \u0026#39;prometheus\u0026#39; static_configs: - targets: [\u0026#39;localhost:9090\u0026#39;] # Node Exporter on this host - job_name: \u0026#39;node\u0026#39; static_configs: - targets: [\u0026#39;node_exporter:9100\u0026#39;] # cAdvisor for container metrics - job_name: \u0026#39;cadvisor\u0026#39; static_configs: - targets: [\u0026#39;cadvisor:8080\u0026#39;] # Additional Node Exporters on other hosts # - job_name: \u0026#39;node-proxmox\u0026#39; # static_configs: # - targets: # - \u0026#39;10.0.20.30:9100\u0026#39; # SRV1 # - \u0026#39;10.0.20.31:9100\u0026#39; # SRV2 For Node Exporter on remote hosts (Proxmox nodes, other VMs), install the agent:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 # On each remote host wget https://github.com/prometheus/node_exporter/releases/download/v1.8.2/node_exporter-1.8.2.linux-amd64.tar.gz tar xzf node_exporter-1.8.2.linux-amd64.tar.gz sudo install node_exporter-1.8.2.linux-amd64/node_exporter /usr/local/bin/ # Create systemd service sudo tee /etc/systemd/system/node_exporter.service \u0026lt;\u0026lt; \u0026#39;SVC\u0026#39; [Unit] Description=Prometheus Node Exporter After=network.target [Service] Type=simple User=nobody Group=nogroup ExecStart=/usr/local/bin/node_exporter \\ --path.rootfs=/host \\ --collector.filesystem.mount-points-exclude=^/(sys|proc|dev|host|etc)($$|/) Restart=always [Install] WantedBy=multi-user.target SVC sudo systemctl daemon-reload sudo systemctl enable --now node_exporter Uncomment the remote targets in prometheus.yaml after installing Node Exporter on each remote host.\nAlerting Rules Create alert-rules.yaml with practical homelab alerts:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 # /opt/monitoring/alert-rules.yaml groups: - name: homelab_node interval: 30s rules: - alert: HighCPUUsage expr: 100 - (avg by(instance) (rate(node_cpu_seconds_total{mode=\u0026#34;idle\u0026#34;}[5m])) * 100) \u0026gt; 80 for: 5m labels: severity: warning annotations: summary: \u0026#34;High CPU usage on {{ $labels.instance }}\u0026#34; description: \u0026#34;CPU usage is {{ $value }}% for 5 minutes.\u0026#34; - alert: HighMemoryUsage expr: (1 - (node_memory_MemAvailable_bytes / node_memory_MemTotal_bytes)) * 100 \u0026gt; 90 for: 5m labels: severity: warning annotations: summary: \u0026#34;High memory usage on {{ $labels.instance }}\u0026#34; description: \u0026#34;Memory usage is {{ $value }}%.\u0026#34; - alert: DiskSpaceRunningOut expr: (node_filesystem_avail_bytes{mountpoint=\u0026#34;/\u0026#34;,fstype!=\u0026#34;tmpfs\u0026#34;} / node_filesystem_size_bytes{mountpoint=\u0026#34;/\u0026#34;,fstype!=\u0026#34;tmpfs\u0026#34;}) * 100 \u0026lt; 10 for: 2m labels: severity: critical annotations: summary: \u0026#34;Disk space low on {{ $labels.instance }}\u0026#34; description: \u0026#34;Only {{ $value }}% available on {{ $labels.mountpoint }}.\u0026#34; - alert: DiskSpaceWarning expr: (node_filesystem_avail_bytes{mountpoint=\u0026#34;/\u0026#34;,fstype!=\u0026#34;tmpfs\u0026#34;} / node_filesystem_size_bytes{mountpoint=\u0026#34;/\u0026#34;,fstype!=\u0026#34;tmpfs\u0026#34;}) * 100 \u0026lt; 20 for: 5m labels: severity: warning annotations: summary: \u0026#34;Disk space warning on {{ $labels.instance }}\u0026#34; description: \u0026#34;Only {{ $value }}% available on {{ $labels.mountpoint }}.\u0026#34; - alert: NodeDown expr: up{job=\u0026#34;node\u0026#34;} == 0 for: 1m labels: severity: critical annotations: summary: \u0026#34;Node {{ $labels.instance }} is down\u0026#34; description: \u0026#34;Node {{ $labels.instance }} has been unreachable for 1 minute.\u0026#34; - name: homelab_container interval: 30s rules: - alert: ContainerHighCPU expr: rate(container_cpu_usage_seconds_total{name!=\u0026#34;\u0026#34;}[2m]) * 100 \u0026gt; 200 for: 2m labels: severity: warning annotations: summary: \u0026#34;High CPU in container {{ $labels.name }}\u0026#34; description: \u0026#34;Container {{ $labels.name }} is using {{ $value }}% CPU.\u0026#34; - alert: ContainerHighMemory expr: container_memory_usage_bytes{name!=\u0026#34;\u0026#34;} / container_spec_memory_limit_bytes{name!=\u0026#34;\u0026#34;} * 100 \u0026gt; 90 for: 2m labels: severity: warning annotations: summary: \u0026#34;High memory in container {{ $labels.name }}\u0026#34; description: \u0026#34;Container {{ $labels.name }} at {{ $value }}% of memory limit.\u0026#34; - alert: ContainerRestarting expr: changes(container_last_seen{name!=\u0026#34;\u0026#34;}[15m]) \u0026gt; 3 for: 2m labels: severity: critical annotations: summary: \u0026#34;Container {{ $labels.name }} restarting\u0026#34; description: \u0026#34;Container {{ $labels.name }} restarted {{ $value }} times in 15 minutes.\u0026#34; - alert: ContainerOOMKilled expr: container_oom_events_total{name!=\u0026#34;\u0026#34;} \u0026gt; 0 labels: severity: critical annotations: summary: \u0026#34;Container {{ $labels.name }} OOM killed\u0026#34; description: \u0026#34;Container {{ $labels.name }} was killed by OOM.\u0026#34; These rules cover the most common homelab failure scenarios: a node going offline, disk filling up, a container in a restart loop, or a process hitting its memory limit.\nAlertmanager Notification Configuration Create alertmanager.yaml to route alerts to Telegram:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 # /opt/monitoring/alertmanager.yaml route: receiver: \u0026#39;telegram\u0026#39; group_wait: 30s group_interval: 5m repeat_interval: 4h routes: - match: severity: critical receiver: \u0026#39;telegram\u0026#39; repeat_interval: 1h receivers: - name: \u0026#39;telegram\u0026#39; telegram_configs: - bot_token: ${TELEGRAM_BOT_TOKEN} chat_id: ${TELEGRAM_CHAT_ID} message: |- {{ range .Alerts }} 🔴 *{{ .Labels.alertname }}* *Instance:* {{ .Labels.instance }} *Severity:* {{ .Labels.severity }} *Summary:* {{ .Annotations.summary }} *Description:* {{ .Annotations.description }} {{ end }} parse_mode: \u0026#39;MarkdownV2\u0026#39; send_resolved: true Set the environment variables in a .env file:\n1 2 3 4 # /opt/monitoring/.env GRAFANA_PASSWORD=your-strong-password TELEGRAM_BOT_TOKEN=123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11 TELEGRAM_CHAT_ID=-1001234567890 For email alerts, switch to:\n1 2 3 4 5 6 7 - name: \u0026#39;email\u0026#39; email_configs: - to: \u0026#39;admin@gntech.dev\u0026#39; from: \u0026#39;alertmanager@gntech.dev\u0026#39; smarthost: \u0026#39;mail.gntech.dev:587\u0026#39; auth_username: \u0026#39;alertmanager@gntech.dev\u0026#39; auth_password: ${SMTP_PASSWORD} Grafana Provisioning — Automatic Dashboards Avoid clicking around in the Grafana UI. Use provisioning files so the dashboards survive container recreations.\nDatasource Provisioning 1 2 3 4 5 6 7 8 9 10 # /opt/monitoring/grafana/datasources/datasources.yaml apiVersion: 1 datasources: - name: Prometheus type: prometheus access: proxy url: http://prometheus:9090 isDefault: true editable: false Dashboard Provisioning 1 2 3 4 5 6 7 8 9 10 11 12 # /opt/monitoring/grafana/dashboards/dashboards.yaml apiVersion: 1 providers: - name: \u0026#39;Homelab\u0026#39; orgId: 1 folder: \u0026#39;\u0026#39; type: file disableDeletion: true editable: false options: path: /var/lib/grafana/dashboards Import Community Dashboards Download the most popular dashboards for Node Exporter and cAdvisor:\n1 2 3 4 5 6 7 8 # /opt/monitoring/grafana/dashboards-json/ # Node Exporter Full (ID 1860) curl -sL https://grafana.com/api/dashboards/1860/revisions/latest/download \\ -o grafana/dashboards-json/node-exporter-full.json # Docker Monitoring (ID 17906) curl -sL https://grafana.com/api/dashboards/17906/revisions/latest/download \\ -o grafana/dashboards-json/docker-monitoring.json Grafana loads these on startup. No manual importing needed.\nReverse Proxy with Traefik If you use Traefik (covered in earlier posts), add labels to expose Grafana and Prometheus securely:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 grafana: labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.grafana.rule=Host(`monitor.gntech.home`)\u0026#34; - \u0026#34;traefik.http.routers.grafana.entrypoints=https\u0026#34; - \u0026#34;traefik.http.routers.grafana.tls=true\u0026#34; - \u0026#34;traefik.http.services.grafana.loadbalancer.server.port=3000\u0026#34; prometheus: labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.prometheus.rule=Host(`prometheus.gntech.home`)\u0026#34; - \u0026#34;traefik.http.routers.prometheus.entrypoints=https\u0026#34; - \u0026#34;traefik.http.routers.prometheus.tls=true\u0026#34; - \u0026#34;traefik.http.services.prometheus.loadbalancer.server.port=9090\u0026#34; - \u0026#34;traefik.http.routers.prometheus.middlewares=auth@file\u0026#34; Security: Always put Prometheus behind authentication. The Prometheus UI has no auth by default. Use Traefik forward auth (Authentik or basic auth middleware) or add Prometheus\u0026rsquo;s --web.auth.file flag with a bcrypt-hashed password file.\nDeployment Start the stack:\n1 2 cd /opt/monitoring docker compose up -d Verify all services are running:\n1 2 3 4 5 6 7 8 9 10 docker compose ps # Check Prometheus targets curl -s http://localhost:9090/api/v1/targets | jq \u0026#39;.data.activeTargets[].labels.job\u0026#39; # Query a metric curl -s \u0026#39;http://localhost:9090/api/v1/query?query=up\u0026#39; | jq \u0026#39;.data.result[] | {job: .metric.job, instance: .metric.instance, up: .value[1]}\u0026#39; # Test an alert rule curl -s http://localhost:9090/api/v1/rules | jq \u0026#39;.data.groups[].rules[] | {name: .name, state: .state}\u0026#39; Access Grafana at http://localhost:3000 (or your Traefik URL). Login with admin and your configured GRAFANA_PASSWORD. The Prometheus datasource and dashboards are preconfigured.\nAdding Monitoring to a Multi-Node Homelab For a Proxmox cluster with multiple hosts, install Node Exporter on every node and point Prometheus at each:\n1 2 3 4 5 6 7 8 scrape_configs: - job_name: \u0026#39;node\u0026#39; static_configs: - targets: - \u0026#39;node_exporter:9100\u0026#39; # Docker host - \u0026#39;10.0.20.30:9100\u0026#39; # SRV1 - \u0026#39;10.0.20.31:9100\u0026#39; # SRV2 - \u0026#39;10.0.20.32:9100\u0026#39; # SRV3 (backup node) For Proxmox-specific metrics (which Node Exporter does not cover), install the Proxmox VE Exporter:\n1 2 3 4 5 6 7 8 docker run -d \\ --name=pve-exporter \\ --restart=unless-stopped \\ -e PVE_USER=monitoring@pve \\ -e PVE_PASSWORD=secret \\ -e PVE_HOST=10.0.20.30 \\ -p 127.0.0.1:9221:9221 \\ prompve/prometheus-pve-exporter:latest Add it as a scrape target in prometheus.yaml:\n1 2 3 - job_name: \u0026#39;proxmox\u0026#39; static_configs: - targets: [\u0026#39;pve-exporter:9221\u0026#39;] Disk Usage Considerations Prometheus can consume significant disk depending on retention and scrape volume. Estimate:\n15s scrape interval × ~500 time series = ~2 GB/month 15s scrape interval × ~2000 time series (multi-node) = ~8 GB/month 30d retention with multi-node = ~8-10 GB total Allocate a dedicated Docker volume on fast storage. For ZFS users, set the mountpoint on a dataset with recordsize=16K (optimized for Prometheus write patterns):\n1 zfs create -o recordsize=16K -o compression=lz4 tank/monitoring Then mount it via volume bind:\n1 2 volumes: - /tank/monitoring/prometheus:/prometheus Maintenance and Updates Backing Up Grafana Grafana stores dashboards, datasources, and users in SQLite. Back up the database and provisioning files:\n1 2 docker exec grafana sqlite3 /var/lib/grafana/grafana.db \u0026#34;.backup /tmp/grafana-backup.db\u0026#34; docker cp grafana:/tmp/grafana-backup.db ./backups/ Or use provisioning (recommended) — export dashboards as JSON and store them in your git repo.\nUpdating Images 1 2 3 docker compose pull docker compose up -d docker image prune -f Adding a New Host Install Node Exporter on the host (systemd service above). Open port 9100 in the host firewall. Add the IP to prometheus.yaml scrape_configs. Reload Prometheus: curl -X POST http://localhost:9090/-/reload. Summary This Prometheus Grafana monitoring stack gives you complete observability into your homelab with minimal overhead. Five Docker containers provide:\nHost metrics — CPU, RAM, disk, network from every node Container metrics — per-container resource usage and health from cAdvisor Alerting — Telegram notifications when things go wrong Preconfigured dashboards — visual monitoring on day one Multi-node support — scrape any host running Node Exporter The stack runs on any Docker host, uses minimal resources (~500 MB RAM total for the full stack), and alerts you before problems become outages. Deploy it today and you will catch your next disk-full or container-OOM before it takes down your services.\n","permalink":"https://blog.gntech.me/posts/2026-05-21-prometheus-grafana-monitoring-stack/","summary":"Deploy a production-grade Prometheus Grafana monitoring stack with Docker Compose — including Node Exporter, cAdvisor, and Alertmanager for complete homelab observability.","title":"Prometheus Grafana Monitoring Stack — Docker Compose Setup"},{"content":"Running your own email server is one of the most rewarding homelab projects. You get full control over your data, no privacy-invading scanning from big providers, unlimited mailboxes, and deep insight into deliverability. But it is also one of the most complex services to get right — DNS records, reverse DNS, SPF, DKIM, DMARC, spam filtering, and security hardening all need to work together.\nMailcow Dockerized solves most of this complexity. It bundles Postfix (MTA), Dovecot (IMAP/POP3), Rspamd (spam filtering), SOGo (webmail/CalDAV/CardDAV), and all the supporting services into a single Docker Compose stack with sensible defaults. You still need to handle DNS and firewall, but Mailcow eliminates the pain of manual integration between components.\nThis guide covers deploying Mailcow on a single Docker host behind Traefik, configuring all required DNS records, tuning for good deliverability, and securing the stack.\nRequirements Before starting, make sure you have:\nA Docker host (Debian 12 or Ubuntu 24.04 recommended) Docker Engine 24+ and Docker Compose plugin A domain you control (e.g., example.com) Ability to modify DNS records (preferably with API access) Ports 25, 465, 587, 993, and 995 open on your firewall Reverse DNS (PTR) record from your ISP (mandatory for good deliverability to Gmail and Outlook) Traefik or another reverse proxy (optional but recommended for webmail) Port Requirements Port Protocol Service 25 SMTP Inbound mail 465 SMTPS Submission (SSL) 587 SMTP Submission (TLS) 993 IMAPS IMAP over SSL 995 POP3S POP3 over SSL 80 HTTP Let\u0026rsquo;s Encrypt 443 HTTPS Webmail Port 25 must be open globally. This is non-negotiable for an email server — other mail servers need to reach you on port 25. Check that your ISP does not block inbound port 25 (most residential ISPs do; business plans usually allow it).\nStep 1 — Deploy Mailcow Clone the repository and run the setup:\n1 2 3 cd /opt git clone https://github.com/mailcow/mailcow-dockerized cd mailcow-dockerized Generate the configuration:\n1 ./generate_config.sh This script asks for:\nMail server hostname (FQDN) — e.g., mail.example.com Timezone — use your local timezone (e.g., America/Santo_Domingo) HTTP bind — leave as 0.0.0.0 if Traefik handles termination Skip Let\u0026rsquo;s Encrypt — set to n if Traefik handles TLS externally If Traefik handles TLS, edit mailcow.conf after generation:\n1 2 3 SKIP_LETS_ENCRYPT=y HTTP_BIND=127.0.0.1 HTTP_PORT=8080 This tells Mailcow to bind web services to localhost on port 8080, so Traefik can proxy to it without exposing HTTP directly.\nPull images and start the stack:\n1 2 docker compose pull docker compose up -d Wait a minute, then verify all containers are running:\n1 docker compose ps You should see containers for: postfix, dovecot, rspamd, nginx, sogo, mysql (MariaDB), redis, clamav, unbound (DNS), olefy, and watchdog. Mailcow is a full stack.\nStep 2 — DNS Records This is the most critical part. Missing or misconfigured DNS records mean your emails will go to spam — or get rejected entirely.\nA / AAAA Records 1 2 mail.example.com. IN A 203.0.113.50 mail.example.com. IN AAAA 2001:db8:1234::50 MX Record Point the MX to your mail server hostname:\n1 example.com. IN MX 10 mail.example.com. Priority 10 is standard for a single server. The trailing dot on mail.example.com. is required by the DNS standard.\nPTR Record (Reverse DNS) Contact your ISP or use their control panel to set:\n50.113.0.203.in-addr.arpa. IN PTR mail.example.com. Most ISPs with a control panel have a \u0026ldquo;Reverse DNS\u0026rdquo; or \u0026ldquo;PTR\u0026rdquo; setting on the IP address page. Without a matching PTR record, Gmail and Outlook flag your mail as suspicious.\nSPF Record SPF (Sender Policy Framework) tells receiving servers which IPs are authorized to send mail for your domain:\n1 example.com. IN TXT \u0026#34;v=spf1 mx ~all\u0026#34; This allows the IPs listed in your MX records to send mail, and soft-fail everything else. If you send through a third-party service (Mailgun, SendGrid), include their SPF include too:\n1 example.com. IN TXT \u0026#34;v=spf1 mx include:mailgun.org ~all\u0026#34; DKIM Record Mailcow generates DKIM keys automatically. Retrieve them from the web admin UI or directly:\n1 2 docker exec -it $(docker compose ps -q postfix-mailcow) \\ cat /etc/opendkim/keys/example.com/mail.txt Output looks like:\nmail._domainkey IN TXT \u0026#34;v=DKIM1; h=sha256; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA...\u0026#34; Add this exact TXT record to your DNS:\n1 mail._domainkey.example.com. IN TXT \u0026#34;v=DKIM1; h=sha256; k=rsa; p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA...\u0026#34; DMARC Record DMARC tells receivers how to handle mail that fails SPF or DKIM verification:\n1 _dmarc.example.com. IN TXT \u0026#34;v=DMARC1; p=quarantine; rua=mailto:postmaster@example.com; pct=100\u0026#34; p=quarantine — move failing mail to spam (start here) p=reject — reject failing mail outright (after monitoring) p=none — monitor only, no action rua — where aggregate reports are sent Start with p=none, monitor reports for a week, then tighten to p=quarantine. Move to p=reject only after you are confident all legitimate mail is authenticated.\nVerify DNS 1 2 3 4 5 6 7 8 9 10 11 # MX lookup dig +short MX example.com # SPF lookup dig +short TXT example.com # DKIM lookup dig +short TXT mail._domainkey.example.com # DMARC lookup dig +short TXT _dmarc.example.com All records should resolve before you send a single test email.\nStep 3 — Traefik Reverse Proxy Integration If you run Traefik (as covered in this blog\u0026rsquo;s Traefik guide), add a Compose override for the webmail interface.\nCreate docker-compose.override.yml in your mailcow directory:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 services: nginx-mailcow: labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.mailcow.rule=Host(`mail.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.mailcow.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.mailcow.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.services.mailcow.loadbalancer.server.port=8080\u0026#34; networks: - proxy depends_on: - traefik networks: proxy: external: true Restart the stack:\n1 docker compose up -d This routes https://mail.example.com through Traefik with automatic Let\u0026rsquo;s Encrypt. Mailcow\u0026rsquo;s internal nginx does not terminate TLS — Traefik handles it at the edge.\nImportant: Disable Mailcow\u0026rsquo;s built-in ACME when using Traefik. Set SKIP_LETS_ENCRYPT=y and HTTP_BIND=127.0.0.1 in mailcow.conf to prevent port conflicts.\nStep 4 — Initial Configuration and Security Access the Admin UI Open https://mail.example.com in a browser. The default admin login is:\nUsername: admin Password: moohoo Change this immediately in the admin panel under Configuration → Access → Edit admin.\nCreate Mailboxes Go to Mailboxes → Add mailbox. Enter a full email address (user@example.com) and password. Mailcow automatically creates the mailbox, IMAP folder structure, and aliases.\nDisable Weak Ciphers Mailcow\u0026rsquo;s Postfix defaults are reasonable, but you can tighten them. Edit data/conf/postfix/extra.cf:\nsmtpd_tls_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 smtpd_tls_mandatory_protocols = !SSLv2 !SSLv3 !TLSv1 !TLSv1.1 smtpd_tls_mandatory_ciphers = high smtpd_tls_eecdh_grade = ultra Apply changes:\n1 docker compose restart postfix-mailcow Rspamd Tuning Mailcow bundles Rspamd with good defaults. For better spam filtering, enable additional modules in the Rspamd UI (https://mail.example.com/rspamd/):\nDMARC Reports under Configuration → DMARC Surbl / Spamhaus DBL — already configured Greylisting — enabled by default; tune the timeout if needed (most legitimate mail resends within minutes) Set Up Fail2ban (Optional) Mailcow includes its own fail2ban or you can install it on the host:\n1 2 docker exec -it $(docker compose ps -q watchdog-mailcow) \\ fail2ban-client status The watchdog container monitors SSH, Postfix SASL, and Dovecot login failures and auto-bans IPs.\nStep 5 — Firewall Rules Open only the ports your mail server needs. A minimal iptables/nftables ruleset:\n1 2 3 4 5 6 7 8 9 10 # Inbound SMTP, submission, IMAPS iptables -A INPUT -p tcp --dport 25 -j ACCEPT iptables -A INPUT -p tcp --dport 465 -j ACCEPT iptables -A INPUT -p tcp --dport 587 -j ACCEPT iptables -A INPUT -p tcp --dport 993 -j ACCEPT iptables -A INPUT -p tcp --dport 995 -j ACCEPT # HTTP/HTTPS for webmail iptables -A INPUT -p tcp --dport 80 -j ACCEPT iptables -A INPUT -p tcp --dport 443 -j ACCEPT Do not restrict port 25 source IPs. SMTP servers connect from anywhere. Rate-limit it instead if you see abuse:\n1 2 3 4 iptables -A INPUT -p tcp --dport 25 -m state --state NEW \\ -m recent --set --name SMTP iptables -A INPUT -p tcp --dport 25 -m state --state NEW \\ -m recent --update --seconds 60 --hitcount 20 --name SMTP -j DROP If you have an IPv6 address on your mail server, add equivalent ip6tables rules for all ports.\nStep 6 — Testing Deliverability SMTP Port Scan From an external machine, verify your mail server responds on the expected ports:\n1 2 3 4 nc -zv mail.example.com 25 nc -zv mail.example.com 587 nc -zv mail.example.com 465 nc -zv mail.example.com 993 Send a Test Email Use the admin UI to send a test. Then check Mailcow\u0026rsquo;s logs:\n1 docker compose logs postfix-mailcow | tail -50 Look for lines confirming successful delivery:\nstatus=sent (250 2.0.0 Ok: queued as ABC123) Check Rspamd History In the Rspamd UI (/rspamd/), check the History tab. Every scanned email shows a score, symbols matched, and whether it was classified as spam or ham.\nUse Mail-Tester Send a test email to check@mail-tester.com from your new mailbox. It returns a score out of 10 points with detailed feedback on:\nSPF alignment DKIM signature validity Reverse DNS match SMTP configuration Blacklist status A score above 9/10 is achievable with all DNS records configured correctly.\nBlacklist Monitoring Check your IP against common DNSBLs:\n1 2 3 4 5 # Check Spamhaus dig +short 50.113.0.203.zen.spamhaus.org # Check Barracuda dig +short 50.113.0.203.b.barracudacentral.org An empty response means you are not listed. If you are listed on Spamhaus or another blocklist, check your server for open relay or compromised mailboxes.\nStep 7 — Backups Mailcow stores everything in ./mailcow-dockerized/. Back up the entire directory, plus the MariaDB database regularly.\nMailcow includes a built-in backup script:\n1 2 cd /opt/mailcow-dockerized ./helper-scripts/backup_and_restore.sh backup all This dumps the SQL database and archives mail data. Schedule it daily via cron:\n0 3 * * * cd /opt/mailcow-dockerized \u0026amp;\u0026amp; ./helper-scripts/backup_and_restore.sh backup all --delete-days 30 The --delete-days 30 flag keeps 30 days of backups automatically.\nFor off-site protection, sync the backup directory to a remote server using rsync or rclone:\n1 rclone sync /opt/mailcow-dockerized/ backup:mailcow/ --progress Step 8 — Monitoring and Maintenance Check Queue Depth A healthy outbound queue should be near zero:\n1 2 docker exec -it $(docker compose ps -q postfix-mailcow) \\ mailq | grep \u0026#34;^[0-9A-F]\u0026#34; | wc -l If the queue grows beyond 100, investigate — you may have a compromised mailbox sending spam, or a remote server rejecting your mail.\nWatch Logs Monitor in real time:\n1 docker compose logs -f --tail=100 Update Mailcow Mailcow updates are handled via git:\n1 2 3 4 cd /opt/mailcow-dockerized git pull docker compose pull docker compose up -d Always check the Mailcow changelog before updating a production deployment.\nTroubleshooting Port 25 is blocked by ISP You cannot run a Mailcow inbound server on a residential connection that blocks port 25. Options:\nBusiness ISP plan — most business fiber includes port 25 Relay through a VPS — relay outgoing mail via a cheap VPS using Postfix transport maps Tunnel — use an SSH tunnel or WireGuard to forward port 25 through a VPS Emails going to spam Top causes:\nMissing reverse DNS — fix with your ISP Missing or broken DKIM — verify with dig SPF too permissive (+all instead of ~all) — tighten it IP on a blocklist — check and delist Sending from a residential IP range — much harder to fix; consider a relay Cannot authenticate to SMTP submission Ensure your mail client uses:\nPort 587 with STARTTLS (not port 25) PLAIN or LOGIN authentication Full email address as the username (user@example.com, not just user) Summary Mailcow Dockerized turns the complexity of running a full email server into a git clone + docker compose up -d workflow. The stack handles Postfix, Dovecot, Rspamd, and webmail integration out of the box, leaving you with just DNS configuration and firewall rules to get right.\nThe essential checklist for a production-ready mail server:\nClone Mailcow and generate config with SKIP_LETS_ENCRYPT=y Configure A, MX, PTR, SPF, DKIM, and DMARC DNS records Integrate with Traefik for TLS termination on webmail Secure Postfix ciphers and enable Rspamd greylisting Set up daily backups and off-site sync Test deliverability with mail-tester.com A proper mail server in your homelab means no more depending on third-party providers for your domain\u0026rsquo;s email. It is not the simplest project, but it is one of the most empowering.\n","permalink":"https://blog.gntech.me/posts/2026-05-21-mailcow-dockerized-selfhosted-email/","summary":"Full guide to deploying Mailcow Dockerized in your homelab — from Docker Compose setup and DNS records to Traefik reverse proxy, DKIM signing, and email deliverability tuning.","title":"Mailcow Dockerized — Self-Hosted Email Server in Your Homelab"},{"content":"If you build Docker images in your homelab — for self-hosted applications, CI/CD pipelines, or custom toolchains — you have felt the pain of waiting minutes for apt-get install or pip install to re-run on every build. Docker\u0026rsquo;s layer cache helps when only the Dockerfile changes, but the moment your source code changes or a COPY busts an earlier layer, every downstream RUN command rebuilds from scratch.\nBuildKit, the default build backend since Docker Engine 23.0, ships three powerful cache features that most homelab setups never configure:\nCache mounts (--mount=type=cache) — persist package downloads across builds without bloating the image. Inline cache — embed build cache metadata into the image itself so that future pulls can reuse layers. Registry cache (--cache-to / --cache-from) — push and pull cache blobs to a container registry independently of the image. This guide covers all three with real Dockerfiles and Compose configurations you can apply immediately.\nWhy Default Docker Builds Are Slow Every RUN instruction in a Dockerfile produces a layer. Docker caches each layer keyed on the preceding command text and its input files. If a COPY changes the requirements.txt, the RUN pip install -r requirements.txt layer cache is invalidated, and pip downloads and compiles everything from scratch.\nWith cache mounts, that pip install can reuse a persistent download cache on the host, skipping network fetches even when the layer rebuilds. The difference is dramatic:\nScenario Without cache mount With cache mount apt-get install (20 packages) 45 s 4 s pip install (large requirements.txt) 120 s 15-20 s Go module download 60 s 2-3 s npm ci 40 s 8 s Enabling BuildKit BuildKit is the default in Docker Engine 23.0+, but verify it is active:\n1 2 docker info | grep BuildKit # Output: BuildKit: true If you see false, set the environment variable:\n1 export DOCKER_BUILDKIT=1 For docker compose build, BuildKit is used automatically in modern Compose v2. You can also force it per build:\n1 DOCKER_BUILDKIT=1 docker build . Cache Mounts — The Biggest Win A cache mount binds a persistent directory on the host into the build container. The cache survives image rebuilds and is shared across concurrent builds. Critically, it is not part of the final image — the layer stays clean.\napt Cache Mount Replace a plain RUN apt-get install with a cache mount for /var/cache/apt and /var/lib/apt/lists:\n1 2 3 4 5 6 7 8 9 10 11 12 # Dockerfile FROM ubuntu:24.04 AS base RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \\ --mount=type=cache,target=/var/lib/apt,sharing=locked \\ apt-get update \u0026amp;\u0026amp; \\ apt-get install -y --no-install-recommends \\ curl \\ ca-certificates \\ nginx \\ postgresql-client \u0026amp;\u0026amp; \\ rm -rf /var/lib/apt/lists/* The sharing=locked flag ensures that concurrent builds do not corrupt the apt cache. The --no-install-recommends flag keeps image size lean.\npip Cache Mount Python projects suffer heavily from pip re-downloading wheels. Add a cache mount for the pip cache directory:\n1 2 3 4 5 6 7 8 9 10 FROM python:3.12-slim WORKDIR /app COPY requirements.txt . RUN --mount=type=cache,target=/root/.cache/pip \\ pip install --no-cache-dir -r requirements.txt COPY . . Even when requirements.txt changes and the layer rebuilds, already-downloaded wheels in /root/.cache/pip are reused. The --no-cache-dir flag here controls pip\u0026rsquo;s runtime cache (inside the image), not the BuildKit cache mount.\nnpm / Yarn Cache Mount 1 2 3 4 5 6 7 8 9 10 FROM node:22-alpine WORKDIR /app COPY package*.json . RUN --mount=type=cache,target=/root/.npm \\ npm ci --prefer-offline COPY . . For Yarn:\n1 2 RUN --mount=type=cache,target=/usr/local/share/.cache/yarn \\ yarn install --frozen-lockfile Go Module Cache Mount 1 2 3 4 5 6 7 8 9 10 11 FROM golang:1.24-alpine AS build WORKDIR /src COPY go.mod go.sum ./ RUN --mount=type=cache,target=/go/pkg/mod \\ go mod download COPY . . RUN --mount=type=cache,target=/go/pkg/mod \\ CGO_ENABLED=0 go build -o /app . The go mod download layer uses the cache mount for module binaries. Even if go.mod changes, previously downloaded modules are reused.\nCombining Cache Mounts with Multi-Stage Builds Multi-stage builds already keep your final image small. Add cache mounts on every stage that downloads packages:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 # ---- Build stage ---- FROM golang:1.24-alpine AS builder RUN --mount=type=cache,target=/var/cache/apk \\ apk add --no-cache git ca-certificates WORKDIR /src COPY go.mod go.sum ./ RUN --mount=type=cache,target=/go/pkg/mod \\ go mod download COPY . . RUN --mount=type=cache,target=/go/pkg/mod \\ CGO_ENABLED=0 go build -ldflags=\u0026#34;-s -w\u0026#34; -o /app . # ---- Runtime stage ---- FROM alpine:3.20 RUN --mount=type=cache,target=/var/cache/apk \\ apk add --no-cache ca-certificates tzdata COPY --from=builder /app /app EXPOSE 8080 CMD [\u0026#34;/app\u0026#34;] The build stage gets Go module caching; the runtime stage gets apk caching. The final image is ~15 MB and both builds benefit from persistent caches.\nBuild Cache Export — Inline and Registry Cache mounts speed up builds on a single host. If you rebuild on a different machine — another homelab node, a CI runner, or a friend\u0026rsquo;s server — the cache starts empty. BuildKit solves this with cache export.\nInline Cache Inline cache embeds cache metadata into the image manifest. Any node that pulls the image can reuse the cached layers:\n1 2 3 4 docker build \\ --cache-from app:latest \\ --build-arg BUILDKIT_INLINE_CACHE=1 \\ -t app:latest . Or with docker buildx:\n1 2 3 4 5 docker buildx build \\ --cache-from type=registry,ref=registry.home.lab/app:cache \\ --cache-to type=inline \\ -t registry.home.lab/app:latest \\ --push . Trade-off: Inline cache increases the image manifest size. For multi-architecture builds, use registry cache instead.\nRegistry Cache (Recommended) Registry cache pushes cache blobs to a separate tag or cache repo. This is the fastest option and decouples cache from images.\n1 2 3 4 5 6 # Build with cache export to registry docker buildx build \\ --cache-from type=registry,ref=registry.home.lab/app:cache \\ --cache-to type=registry,ref=registry.home.lab/app:cache,mode=max \\ -t registry.home.lab/app:latest \\ --push . The mode=max flag stores cache for all layers, not just the exported ones. Without it (mode=min), only layers in the final image are cached.\nOn a subsequent build (even on a different host):\n1 2 3 4 docker buildx build \\ --cache-from type=registry,ref=registry.home.lab/app:cache \\ -t registry.home.lab/app:latest \\ --push . BuildKit fetches the cache manifest, compares it with the local build context, and reuses every layer it can.\nDocker Compose BuildKit Integration For homelab stacks built with Compose, add a shared build cache by setting the BUILDKIT_PROGRESS and composing bake files, or use the Compose build section:\n1 2 3 4 5 6 7 8 9 10 11 12 # compose.yaml services: app: build: context: . cache_from: - type: registry ref: registry.home.lab/app:cache cache_to: - type: registry ref: registry.home.lab/app:cache mode: max Build with the compose builder:\n1 docker compose build --push For cache mounts inside the Dockerfile, no Compose changes are needed — they work automatically.\nReal-World Homelab Workflow Here is the full workflow I use for a custom Go web service built daily on my Proxmox CI runner:\nDockerfile:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 FROM golang:1.24-alpine AS builder RUN --mount=type=cache,target=/var/cache/apk \\ apk add --no-cache git WORKDIR /src COPY go.mod go.sum ./ RUN --mount=type=cache,target=/go/pkg/mod \\ go mod download COPY . . RUN --mount=type=cache,target=/go/pkg/mod \\ CGO_ENABLED=0 go build -o /svc . FROM alpine:3.20 RUN --mount=type=cache,target=/var/cache/apk \\ apk add --no-cache ca-certificates COPY --from=builder /svc /svc EXPOSE 8080 CMD [\u0026#34;/svc\u0026#34;] Build script (build-push.sh):\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 #!/bin/bash set -euo pipefail REGISTRY=\u0026#34;registry.home.lab\u0026#34; IMAGE=\u0026#34;${REGISTRY}/my-web-svc\u0026#34; CACHE_TAG=\u0026#34;${IMAGE}:cache\u0026#34; docker buildx build \\ --cache-from type=registry,ref=${CACHE_TAG} \\ --cache-to type=registry,ref=${CACHE_TAG},mode=max \\ -t ${IMAGE}:latest \\ -t ${IMAGE}:$(git rev-parse --short HEAD) \\ --push \\ . First build: ~90 seconds (Go module download + compilation). Subsequent builds (no source change): ~3 seconds (everything cached). Build with code change: ~12 seconds (compilation only, modules cached).\nCommon Pitfalls Bind Mounts vs Cache Mounts A bind mount (--mount=type=bind) gives the build container read-only access to a host file or directory at build time. A cache mount (--mount=type=cache) is a writable persistent directory managed by BuildKit. Use bind mounts for secrets and source; use cache mounts for package downloads.\nClean the Cache If a cache mount becomes stale or corrupted, purge it:\n1 docker builder prune --all Or prune only cache mounts:\n1 docker builder prune --filter type=exec.cachemount Multi-Architecture Caches Registry cache works across architectures if you use the same cache ref. BuildKit stores per-architecture cache blobs and selects the right ones at build time.\n1 2 3 4 5 docker buildx build \\ --platform linux/amd64,linux/arm64 \\ --cache-from type=registry,ref=${CACHE_TAG} \\ --cache-to type=registry,ref=${CACHE_TAG},mode=max \\ -t ${IMAGE}:latest --push . Docker Desktop and Colima Cache mounts work on all platforms. On macOS with Docker Desktop or Colima, the cache lives inside the VM. It persists across builds and survives docker system prune — but not VM resets.\nSummary BuildKit cache mounts and registry cache export are the two highest-leverage optimizations you can make for Docker build performance. Cache mounts eliminate redundant network fetches on every rebuild, while registry cache accelerates builds across CI runners and homelab nodes.\nApply these patterns to every Dockerfile that downloads packages:\nAdd --mount=type=cache,target=/var/cache/apt for apt-based images. Add --mount=type=cache,target=/root/.cache/pip for Python. Add --mount=type=cache,target=/go/pkg/mod for Go. Export caches to your local registry with --cache-to type=registry,mode=max. Your CI pipeline and your Saturday-afternoon rebuilds will thank you.\n","permalink":"https://blog.gntech.me/posts/2026-05-21-docker-buildkit-cache-optimization/","summary":"Cut Docker build times by 80% using BuildKit cache mounts, inline/registry cache export, and optimized Dockerfile patterns — with real homelab examples for apt, pip, npm, and Go builds.","title":"Docker BuildKit Cache Optimization — Speed Up Container Builds"},{"content":"IPv4 exhaustion is not theoretical. RIPE NCC, APNIC, and LACNIC have all run out of /8 blocks. ISPs in Latin America, Europe, and Asia are deploying IPv6 natively, and if your homelab runs behind a GPON or FTTH connection, you likely already have a /56 or /64 from your ISP.\nYet Docker defaults to IPv4-only. Every docker network create, every Compose file, every container — pure IPv4 with NAT. Running dual-stack in 2026 is not a nice-to-have; it is a prerequisite for services that need to be reachable over IPv6 directly, for containers that need to bypass carrier-grade NAT, and for staying compatible with a rapidly v6-ing internet.\nThis guide covers everything: enabling IPv6 in the Docker daemon, configuring address pools for automatic allocation, building IPv6-ready Compose stacks, running Traefik with IPv6 entry points, and securing it all with ip6tables.\nStep 1 — Enable IPv6 in the Docker Daemon The first step is telling the Docker daemon to allocate IPv6 addresses on the default bridge network. Edit or create /etc/docker/daemon.json:\n1 2 3 4 5 { \u0026#34;ipv6\u0026#34;: true, \u0026#34;fixed-cidr-v6\u0026#34;: \u0026#34;fd00:dead:beef:1::/64\u0026#34;, \u0026#34;ip6tables\u0026#34;: true } ipv6: true — Enables IPv6 on the default bridge. fixed-cidr-v6 — The ULA (Unique Local Address) subnet for the default bridge. Use a ULA from fd00::/8 rather than guessing at GUA prefixes you do not own. ip6tables: true — Automatically manages IPv6 firewall rules for port publishing and network isolation. Enabled by default, but worth making explicit. Restart the daemon:\n1 sudo systemctl restart docker Verify the default bridge now has an IPv6 address:\n1 2 3 4 ip -6 addr show docker0 # 6: docker0: \u0026lt;BROADCAST,MULTICAST,UP,LOWER_UP\u0026gt; mtu 1500 # inet6 fd00:dead:beef:1::1/64 scope global # inet6 fe80::42:acff:fe12:2/64 scope link Run a quick test:\n1 2 3 4 5 6 docker run --rm -p 80:80 traefik/whoami curl -6 http://[::1]:80 # Hostname: ... # IP: ::1 # IP: 2001:db8:1::242:ac12:2 # RemoteAddr: [fd00:dead:beef:1::1]:57342 If you see IPv6 addresses in the output, the daemon is working.\nStep 1b — Default Address Pools for Dynamic IPv6 Without explicit configuration, user-defined networks with enable_ipv6: true fall back to Docker\u0026rsquo;s default address pools, which include no IPv6 pools. This means every Compose stack that needs IPv6 must specify an explicit subnet under ipam.\nTo make Docker auto-allocate IPv6 subnets just like it does for IPv4, add pools to daemon.json:\n1 2 3 4 5 6 7 8 9 10 11 12 { \u0026#34;ipv6\u0026#34;: true, \u0026#34;fixed-cidr-v6\u0026#34;: \u0026#34;fd00:dead:beef:1::/64\u0026#34;, \u0026#34;ip6tables\u0026#34;: true, \u0026#34;default-address-pools\u0026#34;: [ { \u0026#34;base\u0026#34;: \u0026#34;172.17.0.0/16\u0026#34;, \u0026#34;size\u0026#34;: 24 }, { \u0026#34;base\u0026#34;: \u0026#34;172.18.0.0/16\u0026#34;, \u0026#34;size\u0026#34;: 24 }, { \u0026#34;base\u0026#34;: \u0026#34;172.19.0.0/16\u0026#34;, \u0026#34;size\u0026#34;: 24 }, { \u0026#34;base\u0026#34;: \u0026#34;172.20.0.0/14\u0026#34;, \u0026#34;size\u0026#34;: 24 }, { \u0026#34;base\u0026#34;: \u0026#34;fd00:dead:beef::/48\u0026#34;, \u0026#34;size\u0026#34;: 64 } ] } This gives Docker 256 /64 IPv6 subnets to hand out dynamically from the fd00:dead:beef::/48 ULA range. Each Compose stack that sets enable_ipv6: true without an explicit subnet gets its own /64 automatically.\nRestart Docker again after editing:\n1 sudo systemctl restart docker Test it with a quick network create:\n1 2 3 docker network create --ipv6 test-ipv6-auto docker network inspect test-ipv6-auto --format \u0026#39;{{.IPAM.Config}}\u0026#39; # [{172.18.0.0/24 fd00:dead:beef:2::/64 }] You should see a /64 ULA subnet allocated automatically.\nStep 2 — Docker Compose with Explicit IPv6 Networks For production stacks where you want predictable addressing, define the IPv6 subnet explicitly. Here is a complete Compose file for a web app with a PostgreSQL backend:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 # docker-compose.yml — Dual-stack web app services: app: image: nginx:alpine ports: - \u0026#34;80:80\u0026#34; - \u0026#34;443:443\u0026#34; networks: frontend: ipv4_address: 172.21.0.10 ipv6_address: fd00:dead:beef:10::10 backend: ipv4_address: 172.21.1.10 ipv6_address: fd00:dead:beef:11::10 db: image: postgres:16-alpine networks: backend: ipv4_address: 172.21.1.20 ipv6_address: fd00:dead:beef:11::20 environment: POSTGRES_PASSWORD: changeme networks: frontend: driver: bridge enable_ipv6: true ipam: config: - subnet: 172.21.0.0/24 gateway: 172.21.0.1 - subnet: fd00:dead:beef:10::/64 gateway: fd00:dead:beef:10::1 backend: driver: bridge enable_ipv6: true internal: true ipam: config: - subnet: 172.21.1.0/24 gateway: 172.21.1.1 - subnet: fd00:dead:beef:11::/64 gateway: fd00:dead:beef:11::1 Key points:\nStatic IPv6 addresses let you write firewall rules against known addresses and keep DNS records stable. The backend network is internal: true — no external access, no default route. Only the app container can reach the database. This works identically for IPv6 traffic. Each network gets its own /64. Docker bridges support multiple IPAM config blocks; the IPv4 and IPv6 entries are completely independent. If your host has a public /64 from your ISP, you can skip ULA and use GUA (Global Unicast Addresses) directly. Replace the subnet with your delegated prefix:\n1 2 3 4 5 6 7 networks: frontend: enable_ipv6: true ipam: config: - subnet: 2001:db8:1234:ab::/64 gateway: 2001:db8:1234:ab::1 Step 3 — Exposing Containers to the LAN over IPv6 The Docker bridge with port publishing (-p or ports: in Compose) works for both IPv4 and IPv6. Publishing port 443 binds to [::]:443 on the host by default:\n1 docker run -d -p 443:443 --network ipv6-net myapp Internally, Docker creates ip6tables DNAT rules just like it creates iptables rules for IPv4. Check them:\n1 2 3 sudo ip6tables -t nat -L DOCKER -n # Chain DOCKER (2 references) # DNAT tcp [::]:443 -\u0026gt; fd00:dead:beef:10::10:443 MACVLAN for Direct LAN Addressing If you want containers to appear as first-class citizens on your LAN with their own IPv6 address (no host NAT), use a macvlan network. This is especially useful for services that need direct inbound connections — game servers, SIP phones, P2P nodes.\n1 2 3 4 5 6 7 docker network create -d macvlan \\ --subnet=192.168.1.0/24 \\ --gateway=192.168.1.1 \\ --subnet=2001:db8:1234:ab::/64 \\ --gateway=2001:db8:1234:ab::1 \\ -o parent=eth0 \\ macvlan-lan 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 # docker-compose.yml — MACVLAN with IPv6 services: direct-app: image: nginx:alpine networks: lan: ipv4_address: 192.168.1.100 ipv6_address: 2001:db8:1234:ab::100 networks: lan: driver: macvlan driver_opts: parent: eth0 enable_ipv6: true ipam: config: - subnet: 192.168.1.0/24 gateway: 192.168.1.1 - subnet: 2001:db8:1234:ab::/64 gateway: 2001:db8:1234:ab::1 Caveat: MACVLAN bridges do not allow communication between the host and its own containers without a separate macvlan interface on the host. For most homelab setups, bridge + port publishing is simpler and sufficient.\nStep 4 — Traefik with IPv6 Entry Points If you run Traefik as your reverse proxy, configuring IPv6 entry points is straightforward. The key is binding [::]:port instead of 0.0.0.0:port.\nStatic configuration (traefik.yml):\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 entryPoints: web: address: \u0026#34;[::]:80\u0026#34; websecure: address: \u0026#34;[::]:443\u0026#34; forwardedHeaders: trustedIPs: - \u0026#34;10.0.0.0/8\u0026#34; - \u0026#34;fd00::/8\u0026#34; - \u0026#34;172.16.0.0/12\u0026#34; certificatesResolvers: letsencrypt: acme: email: admin@example.com storage: /letsencrypt/acme.json httpChallenge: entryPoint: web Or via CLI arguments in a Compose stack:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 services: traefik: image: traefik:v3.3 command: - --providers.docker=true - --providers.docker.exposedbydefault=false - --entrypoints.web.address=[::]:80 - --entrypoints.websecure.address=[::]:443 - --certificatesresolvers.letsencrypt.acme.email=admin@example.com - --certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json - --certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=web ports: - \u0026#34;80:80\u0026#34; - \u0026#34;443:443\u0026#34; networks: - proxy volumes: - /var/run/docker.sock:/var/run/docker.sock:ro - ./letsencrypt:/letsencrypt cap_drop: - ALL app: image: myapp:latest networks: - proxy labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.app.rule=Host(`app.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.app.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.app.tls.certresolver=letsencrypt\u0026#34; networks: proxy: driver: bridge enable_ipv6: true ipam: config: - subnet: fd00:dead:beef:ff::/64 Because the entry points bind to [::]:port and the host kernel has net.ipv6.bindv6only=0 (default on most distros), IPv4 traffic also reaches these sockets via IPv4-mapped IPv6 addresses. You do not need separate IPv4 and IPv6 entry points.\nVerifying Traefik IPv6 If you have the Traefik API enabled:\n1 2 3 4 5 curl -6 http://[::1]:8080/api/entrypoints | jq . # { # \u0026#34;web\u0026#34;: { \u0026#34;address\u0026#34;: \u0026#34;[::]:80\u0026#34; }, # \u0026#34;websecure\u0026#34;: { \u0026#34;address\u0026#34;: \u0026#34;[::]:443\u0026#34; } # } Test an actual request through Traefik over IPv6:\n1 2 3 4 curl -6 -v https://app.example.com/ # * IPv6: 2001:db8:1234::1 # ... # \u0026lt; HTTP/2 200 Check Traefik logs for the client IP source — it should show your real IPv6 address, not the Docker gateway:\n1 2 docker compose logs traefik | grep -E \u0026#34;[0-9a-fA-F:]{3,39}\u0026#34; # 2026-05-21T10:30:00Z app.example.com \u0026#34;GET /\u0026#34; - [2001:db8:5678::1]:44312 Step 5 — IPv6 Firewall Rules with ip6tables Docker\u0026rsquo;s ip6tables: true manages port publishing rules automatically, but it does not restrict outbound or forwarded traffic. For a homelab running public-facing services, add a minimal ip6tables ruleset.\nCreate /etc/iptables/rules.v6 (on Debian/Ubuntu with iptables-persistent):\n1 sudo apt install iptables-persistent Example ruleset:\n# /etc/iptables/rules.v6 *filter :INPUT DROP [0:0] :FORWARD DROP [0:0] :OUTPUT ACCEPT [0:0] :DOCKER - [0:0] # Allow established connections -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT # Allow loopback -A INPUT -i lo -j ACCEPT # Allow ICMPv6 (Neighbor Discovery, MLD, etc.) -A INPUT -p ipv6-icmp -j ACCEPT # Allow SSH and Docker-published ports -A INPUT -p tcp --dport 22 -j ACCEPT -A INPUT -p tcp --dport 80 -j ACCEPT -A INPUT -p tcp --dport 443 -j ACCEPT # Allow Docker bridge traffic -A INPUT -i docker0 -j ACCEPT # Log and drop everything else -A INPUT -j LOG --log-prefix \u0026#34;IPv6-DROP: \u0026#34; -A INPUT -j DROP # Forward rules to accept Docker NAT traffic -A FORWARD -i docker0 -j ACCEPT -A FORWARD -o docker0 -m state --state ESTABLISHED,RELATED -j ACCEPT COMMIT Apply:\n1 sudo ip6tables-restore \u0026lt; /etc/iptables/rules.v6 Important: Docker inserts its own rules on restart. If you use a custom ruleset, ensure Docker\u0026rsquo;s ip6tables rules chain properly. The safest approach is to set your default policies and let Docker manage its own chains — do not flush the DOCKER chain.\nStep 6 — DNS and Reverse DNS for Container IPv6 Containers on a user-defined bridge get DNS via Docker\u0026rsquo;s embedded DNS resolver (127.0.0.11), which handles both A and AAAA records seamlessly. But external DNS is your responsibility.\nFor public-facing containers, add AAAA records alongside your existing A records:\napp.example.com. IN A 192.168.1.100 app.example.com. IN AAAA 2001:db8:1234:ab::100 If your ISP delegates a reverse zone (ip6.arpa delegation), set up PTR records too. Most homelab ISPs do not provide this, but you can run your own authoritative DNS internally with PowerDNS or Bind9 and delegate within your network.\nFor internal service discovery with Traefik, Docker\u0026rsquo;s DNS resolver handles AAAA queries natively. Containers on the same network can reach each other by service name over IPv6 or IPv4 — whichever the client requests.\nTroubleshooting Common IPv6 Docker Issues Container cannot reach the internet over IPv6 Check that the host has a working IPv6 default route:\n1 2 ip -6 route show default # default via fe80::1 dev eth0 proto ra metric 1024 If missing, enable SLAAC or DHCPv6 on the host. Docker bridges do not run their own router advertisements — containers rely on the host\u0026rsquo;s global connectivity.\nPort publishing binds IPv4 only Ensure daemon.json has \u0026quot;ipv6\u0026quot;: true and Docker was restarted. Without it, -p 80:80 only creates an IPv4 iptables rule.\nCheck:\n1 sudo ip6tables -t nat -L DOCKER -n If empty, IPv6 port publishing is not working.\nContainers cannot ping each other\u0026rsquo;s IPv6 addresses User-defined bridge networks forward traffic by default, but the container must have an IPv6 default route. On bridge networks, Docker adds a default route via the gateway address. Verify:\n1 2 3 docker exec myapp ip -6 route # default via fd00:dead:beef:10::1 dev eth0 # fd00:dead:beef:10::/64 dev eth0 proto kernel metric 256 \u0026ldquo;IPv6 is not supported by the kernel\u0026rdquo; on older kernels Docker IPv6 requires the ip6_tables kernel module. Load it:\n1 2 sudo modprobe ip6_tables echo \u0026#34;ip6_tables\u0026#34; | sudo tee /etc/modules-load.d/ip6_tables.conf This is rare on modern kernels but can surface in minimal Docker images running Docker-in-Docker.\nSummary Enabling IPv6 in your Docker homelab is a few lines of JSON and one daemon restart. The payoff is significant: containers get globally routable addresses, services bypass carrier-grade NAT, and your infrastructure is ready for an IPv6-dominant internet.\nThe complete setup checklist:\nAdd ipv6: true and a fixed-cidr-v6 ULA to daemon.json Optionally add IPv6 default-address-pools for auto-allocation Configure Compose networks with enable_ipv6: true and explicit subnets Set Traefik entry points to [::]:80 and [::]:443 Lock down with ip6tables — default DROP on INPUT Add AAAA DNS records for public services IPv4 is not going away tomorrow, but dual-stack is the standard today. Your homelab should speak both.\n","permalink":"https://blog.gntech.me/posts/2026-05-21-docker-ipv6-networking-guide/","summary":"Complete guide to enabling IPv6 in Docker — from daemon configuration and address pools to Compose networks, Traefik reverse proxy, and IPv6 firewall rules for your homelab.","title":"Docker IPv6 Networking — Enable Dual-Stack in Your Homelab"},{"content":"Traefik dominates the homelab reverse proxy conversation, and for good reason — automatic service discovery, Let\u0026rsquo;s Encrypt integration, and a slick dashboard. But Traefik is not always the right tool. When you need raw throughput, fine-grained caching control, or a lightweight sidecar that sits in front of a single service, Nginx is faster, simpler, and uses fewer resources.\nNginx powers over 30% of the web for good reason: it handles 10,000+ concurrent connections with a single worker process, its caching subsystem supports cache purging and micro-caching, and a minimal config file is more readable than equivalent Traefik middleware chains.\nThis guide covers deploying Nginx with Docker Compose for four common homelab patterns: reverse proxying multiple services, static file caching with cache-busting, gzip compression, and SSL termination. Every config is copy-paste ready.\nWhy Run Nginx in Docker Running Nginx in Docker versus directly on the host gives you:\nIsolated dependencies — No nginx packages or PPA management on the host Config-as-code — Entire proxy setup lives in a compose file and volume mounts Network integration — Join existing Docker bridge networks to proxy internal services without exposing ports Easy rollback — Image tags let you pin or roll back Nginx versions instantly The trade-off is a few milliseconds of Docker networking overhead per request. For homelab traffic (a few hundred requests per second at most), this is invisible.\nStep 1 — Basic Docker Compose Deployment Start with a minimal deployment that serves static files:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 # /opt/nginx/docker-compose.yml services: nginx: image: nginx:1.27-alpine container_name: nginx restart: unless-stopped ports: - \u0026#34;80:80\u0026#34; - \u0026#34;443:443\u0026#34; volumes: - ./nginx.conf:/etc/nginx/nginx.conf:ro - ./conf.d:/etc/nginx/conf.d:ro - ./html:/usr/share/nginx/html:ro - ./certs:/etc/nginx/certs:ro - ./cache:/var/cache/nginx cap_drop: - ALL cap_add: - NET_BIND_SERVICE - SETGID - SETUID - CHOWN - DAC_OVERRIDE healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;nginx\u0026#34;, \u0026#34;-t\u0026#34;] interval: 30s timeout: 10s retries: 3 logging: driver: json-file options: max-size: 10m max-file: 3 networks: default: name: nginx-net external: true Create the required directories:\n1 2 3 mkdir -p /opt/nginx/{conf.d,html,certs,cache} chmod 700 /opt/nginx/certs touch /opt/nginx/conf.d/default.conf The main Nginx configuration lives outside conf.d/:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 # /opt/nginx/nginx.conf user nginx; worker_processes auto; error_log /var/log/nginx/error.log warn; pid /var/run/nginx.pid; events { worker_connections 4096; multi_accept on; use epoll; } http { include /etc/nginx/mime.types; default_type application/octet-stream; log_format main \u0026#39;$remote_addr - $remote_user [$time_local] \u0026#34;$request\u0026#34; \u0026#39; \u0026#39;$status $body_bytes_sent \u0026#34;$http_referer\u0026#34; \u0026#39; \u0026#39;\u0026#34;$http_user_agent\u0026#34; \u0026#34;$http_x_forwarded_for\u0026#34;\u0026#39;; access_log /var/log/nginx/access.log main; error_log /var/log/nginx/error.log warn; sendfile on; tcp_nopush on; tcp_nodelay on; keepalive_timeout 65; keepalive_requests 1000; types_hash_max_size 2048; server_tokens off; # Gzip compression gzip on; gzip_vary on; gzip_proxied any; gzip_comp_level 5; gzip_min_length 256; gzip_types text/plain text/css text/xml text/javascript application/javascript application/json application/xml application/xml+rss application/atom+xml image/svg+xml; # Client body size client_max_body_size 100m; client_body_buffer_size 16k; # Buffer pool client_header_buffer_size 1k; large_client_header_buffers 4 8k; output_buffers 2 64k; postpone_output 1460; # Proxy cache zone proxy_cache_path /var/cache/nginx/cache levels=1:2 keys_zone=static_cache:50m inactive=7d max_size=1g use_temp_path=off; include /etc/nginx/conf.d/*.conf; } Step 2 — Reverse Proxy Multiple Upstream Services This is the most common homelab pattern. Define upstream servers and proxy pass rules in conf.d/:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 # /opt/nginx/conf.d/upstreams.conf # ---- Grafana ---- upstream grafana_upstream { zone grafana 64k; server grafana:3000 resolve; keepalive 32; } # ---- GitLab ---- upstream gitea_upstream { zone gitea 64k; server gitea:3000 resolve; keepalive 32; } # ---- Home Assistant ---- upstream ha_upstream { zone ha 64k; server homeassistant:8123 resolve; keepalive 16; } Then create a virtual host for each service:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 # /opt/nginx/conf.d/grafana.conf server { listen 80; server_name grafana.gntech.me; location / { proxy_pass http://grafana_upstream; proxy_http_version 1.1; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_buffering on; proxy_buffer_size 4k; proxy_buffers 8 16k; proxy_busy_buffers_size 64k; proxy_connect_timeout 10s; proxy_send_timeout 30s; proxy_read_timeout 30s; } } The key to making Docker service discovery work is configuring Nginx\u0026rsquo;s DNS resolver. Add this to the http block in nginx.conf:\n1 resolver 127.0.0.11 ipv6=off valid=10s; Docker\u0026rsquo;s embedded DNS server runs at 127.0.0.11 on every container. With the resolve parameter on each upstream server line, Nginx dynamically resolves container IPs — so you can restart services without reloading Nginx.\nStep 3 — Static File Caching with Cache Purging Static assets like JavaScript bundles, CSS, fonts, and images benefit massively from caching. Nginx\u0026rsquo;s proxy_cache subsystem stores responses on disk for a configurable TTL:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 # /opt/nginx/conf.d/homepage.conf server { listen 80; server_name homepage.gntech.me; location / { proxy_pass http://homepage_upstream; proxy_http_version 1.1; proxy_set_header Host $host; # Caching rules proxy_cache static_cache; proxy_cache_key \u0026#34;$host$request_uri\u0026#34;; proxy_cache_valid 200 302 1h; proxy_cache_valid 404 1m; # Cache status header for debugging add_header X-Cache-Status $upstream_cache_status; # Bypass cache for admin or dynamic paths proxy_cache_bypass $cookie_session; proxy_no_cache $cookie_session; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } # Cache purge endpoint (requires ngx_cache_purge or Lua) location ~ /purge(/.*) { allow 127.0.0.1; allow 10.0.0.0/8; deny all; proxy_cache_purge static_cache \u0026#34;$host$1\u0026#34;; } } The X-Cache-Status header tells you whether a request hit (HIT), missed (MISS), or was served stale (STALE). Check it with:\n1 2 3 curl -I http://homepage.gntech.me/ # X-Cache-Status: MISS (first request) # X-Cache-Status: HIT (subsequent requests) For a cache-first workflow, combine try_files with proxy_pass to serve cached files even when the upstream is down:\n1 2 3 4 5 6 7 8 9 10 11 12 location /assets/ { # Serve from disk cache first, fall back to upstream try_files $uri @backend; expires 30d; add_header Cache-Control \u0026#34;public, immutable\u0026#34;; } location @backend { proxy_pass http://backend_upstream; proxy_cache static_cache; proxy_cache_valid 200 30d; } Step 4 — Gzip Compression and Performance Tuning Nginx\u0026rsquo;s gzip compression reduces bandwidth by 60-80% for text-based responses. The configuration in the base nginx.conf enables it globally. Tune these parameters for your homelab:\nParameter Default Recommended Rationale gzip_comp_level 1 5 Level 6+ yields diminishing returns for CPU cost gzip_min_length 20 256 Skip tiny responses where compression adds overhead gzip_types text/html Full list in config above Include JS, CSS, JSON, XML, SVG gzip_proxied off any Compress proxied responses too Worker process tuning — Nginx can auto-detect core count with worker_processes auto;, but for Docker containers you should match the container CPU limit instead:\n1 2 3 4 5 6 7 8 # Match Docker --cpus or deploy.resources.limits.cpus worker_processes 2; worker_rlimit_nofile 65535; events { worker_connections 4096; multi_accept on; } If Nginx runs on a single-core container (e.g., a small LXC), the defaults are fine. For front-end caching tier on a multi-core VM, scale worker_connections to 8192 and increase the proxy_buffers:\n1 2 3 proxy_buffer_size 8k; proxy_buffers 16 16k; proxy_busy_buffers_size 64k; Sendfile and direct I/O — For serving large static files (videos, ISOs, backups), enable direct I/O to bypass page cache:\n1 2 3 4 5 6 7 8 location /downloads/ { root /var/www/files; directio 4m; sendfile on; sendfile_max_chunk 512k; aio threads; output_buffers 4 512k; } directio 4m means files over 4 MB bypass the kernel page cache and read directly from disk. Combined with aio threads, Nginx reads ahead asynchronously — essential for multi-GB file downloads without blocking worker processes.\nStep 5 — SSL Termination with Internal CA For a homelab, using an internal CA (OpenSSL or Step CA) is simpler and more secure than self-signed certs with trust-store gymnastics. Generate a wildcard cert for your internal domain:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 # Generate a CA key and cert (do this once) openssl genrsa -out /opt/nginx/certs/ca.key 4096 openssl req -x509 -new -nodes -key /opt/nginx/certs/ca.key \\ -sha256 -days 3650 \\ -out /opt/nginx/certs/ca.crt \\ -subj \u0026#34;/CN=HomeLab Internal CA\u0026#34; # Generate a wildcard cert for *.gntech.me openssl genrsa -out /opt/nginx/certs/wildcard.key 2048 openssl req -new -key /opt/nginx/certs/wildcard.key \\ -out /opt/nginx/certs/wildcard.csr \\ -subj \u0026#34;/CN=*.gntech.me\u0026#34; # Sign with your CA openssl x509 -req -in /opt/nginx/certs/wildcard.csr \\ -CA /opt/nginx/certs/ca.crt \\ -CAkey /opt/nginx/certs/ca.key \\ -CAcreateserial -out /opt/nginx/certs/wildcard.crt \\ -days 365 -sha256 \\ -extfile \u0026lt;(echo -e \u0026#34;subjectAltName=DNS:*.gntech.me,DNS:gntech.me\u0026#34;) Then configure SSL in your virtual host:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 # /opt/nginx/conf.d/grafana-ssl.conf server { listen 443 ssl http2; server_name grafana.gntech.me; ssl_certificate /etc/nginx/certs/wildcard.crt; ssl_certificate_key /etc/nginx/certs/wildcard.key; ssl_protocols TLSv1.2 TLSv1.3; ssl_ciphers HIGH:!aNULL:!MD5; ssl_prefer_server_ciphers on; ssl_session_cache shared:SSL:10m; ssl_session_timeout 10m; # Redirect HTTP to HTTPS add_header Strict-Transport-Security \u0026#34;max-age=31536000\u0026#34; always; location / { proxy_pass http://grafana_upstream; proxy_set_header Host $host; proxy_set_header X-Forwarded-Proto https; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } } Add an HTTP-to-HTTPS redirect:\n1 2 3 4 5 server { listen 80; server_name grafana.gntech.me; return 301 https://$server_name$request_uri; } Install the ca.crt on each client machine (desktop, phone, browser) to eliminate SSL warnings — then every internal service gets valid TLS without Let\u0026rsquo;s Encrypt\u0026rsquo;s rate limits.\nStep 6 — Connecting to Existing Docker Networks For Nginx to reach your upstream services, it must be on the same Docker network. The recommended approach is to create a shared network and attach all services to it:\n1 docker network create proxy-net Update the Nginx compose to attach:\n1 2 3 networks: proxy-net: external: true Then add networks: [proxy-net] to every service you want Nginx to proxy. For example:\n1 2 3 4 5 6 services: grafana: image: grafana/grafana:latest networks: - proxy-net # No ports needed — Nginx reaches it via the internal network Traefik users will recognize this pattern. The difference is that Nginx requires manual server blocks instead of automatic label discovery — a trade-off that gives you explicit control over every upstream config line.\nStep 7 — Health Checks and Logging The health check in the compose file runs nginx -t every 30 seconds. This validates the config syntax without making actual HTTP requests. For deeper monitoring, add an endpoint:\n1 2 3 4 5 6 7 8 server { listen 127.0.0.1:8080; location /health { access_log off; return 200 \u0026#34;healthy\\n\u0026#34;; add_header Content-Type text/plain; } } Then point your monitoring stack at http://nginx:8080/health (from within the Docker network).\nLog rotation in Docker is handled by the max-size and max-file options in the compose file. For structured access logging, switch to JSON format:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 log_format json escape=json \u0026#39;{\u0026#39; \u0026#39;\u0026#34;time\u0026#34;:\u0026#34;$time_iso8601\u0026#34;,\u0026#39; \u0026#39;\u0026#34;remote_addr\u0026#34;:\u0026#34;$remote_addr\u0026#34;,\u0026#39; \u0026#39;\u0026#34;request\u0026#34;:\u0026#34;$request\u0026#34;,\u0026#39; \u0026#39;\u0026#34;status\u0026#34;:$status,\u0026#39; \u0026#39;\u0026#34;body_bytes\u0026#34;:$body_bytes_sent,\u0026#39; \u0026#39;\u0026#34;referer\u0026#34;:\u0026#34;$http_referer\u0026#34;,\u0026#39; \u0026#39;\u0026#34;user_agent\u0026#34;:\u0026#34;$http_user_agent\u0026#34;,\u0026#39; \u0026#39;\u0026#34;request_time\u0026#34;:$request_time,\u0026#39; \u0026#39;\u0026#34;upstream_cache_status\u0026#34;:\u0026#34;$upstream_cache_status\u0026#34;\u0026#39; \u0026#39;}\u0026#39;; access_log /var/log/nginx/access.log json; This JSON format feeds directly into Loki, OpenSearch, or any log aggregator without additional parsing.\nWhen to Use Nginx vs. Traefik Use Case Nginx Traefik Static file serving ✅ Best in class ⚠️ Must proxy to separate container Multi-service caching ✅ Native proxy_cache ⚠️ Requires plugin/middleware Single-service sidecar ✅ Minimal config ⚠️ Overkill Dynamic service discovery ⚠️ Manual server blocks ✅ Auto-detect via labels Let\u0026rsquo;s Encrypt automation ⚠️ Requires certbot sidecar ✅ Built-in ACME WebSocket/gRPC ✅ Native support ✅ Native support For a homelab running 3-5 services behind a single Nginx, the manual config is a few dozen lines — perfectly maintainable and faster at serving cached content than any alternative.\nPutting It All Together The complete deployment flow:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 # 1. Create directories and files mkdir -p /opt/nginx/{conf.d,html,certs,cache} # 2. Copy nginx.conf and the conf.d/ files from this guide # 3. Create a shared Docker network docker network create proxy-net # 4. Update each service\u0026#39;s compose to join proxy-net # 5. Attach Nginx to proxy-net docker compose up -d # 6. Test curl -H \u0026#34;Host: grafana.gntech.me\u0026#34; http://localhost/ curl -s -o /dev/null -w \u0026#34;%{http_code}\u0026#34; http://localhost/ Nginx won\u0026rsquo;t auto-discover new services like Traefik. When you add a service, create a new conf.d/ file and reload:\n1 docker exec nginx nginx -s reload That reload takes milliseconds with zero dropped connections — no downtime, no container restart.\nNginx in Docker gives you a production-grade reverse proxy and caching layer with minimal complexity. It is the right choice when you need deterministic caching behavior, maximum static file throughput, or a lightweight sidecar that stays out of your way. The configs in this guide handle the everyday homelab patterns: proxying internal services, caching static assets, compressing responses, and terminating TLS — all running in a single Alpine container under 20 MB.\n","permalink":"https://blog.gntech.me/posts/2026-05-20-nginx-reverse-proxy-docker/","summary":"Complete guide to deploying Nginx with Docker Compose as a reverse proxy and caching layer. Covers multi-service proxying, static file caching, gzip compression, SSL termination, and performance tuning for homelab environments.","title":"Nginx Docker Deployment — Reverse Proxy and Static Caching for Homelabs"},{"content":"Every week someone posts on r/Proxmox asking about security best practices — and with good reason. The default Proxmox VE installation is functional but not hardened. On a fresh install, the web UI listens on port 8006 without rate limiting, SSH accepts password authentication, and the built-in firewall is disabled.\nIf your hypervisor gets compromised, every VM and container on it is compromised too. Proxmox runs as root, so a breach means full host access.\nThis guide covers seven concrete hardening steps that apply to Proxmox VE 8.x — the same measures used in production environments, adapted for homelab budgets:\nEnable and configure the Proxmox firewall Set up TOTP two-factor authentication Install and configure Fail2ban for the web UI Harden SSH access Confine services with AppArmor Enable automatic security updates Audit and verify the configuration Step 1 — Enable and Configure the Proxmox Firewall Proxmox ships with a three-layer firewall model: datacenter, node, and VM/CT. The datacenter layer sets the default policy for all nodes, and each node can override. This is your first and most important layer.\nEnable the Datacenter Firewall 1 2 3 4 5 6 7 8 # SSH into your Proxmox host ssh root@pve-host # Enable the firewall at the datacenter level pve-firewall set --enable 1 # Or through the config file directly echo \u0026#34;1\u0026#34; \u0026gt; /etc/pve/firewall/dc.fw Verify it is active:\n1 pve-firewall status Expected output: Status: enabled/running\nDefine Datacenter Default Rules Edit /etc/pve/firewall/dc.fw:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 [OPTIONS] enable: 1 policy_in: DROP policy_out: ACCEPT [RULES] # Allow established connections IN ACCEPT -state ESTABLISHED,RELATED # Allow ICMP (ping) IN ACCEPT -icmp-type echo-request -p icmp -log nolog # Allow web UI access from management subnet IN ACCEPT -source 10.0.20.0/24 -dest +pveproxy -log nolog # Allow SSH from management subnet IN ACCEPT -source 10.0.20.0/24 -dport 22 -p tcp -log nolog # Allow VNC console access IN ACCEPT -source 10.0.20.0/24 -dport 5900-5999 -p tcp -log nolog # Cluster communication (if clustered) IN ACCEPT -source 10.0.20.0/24 -dport 5405:5412 -p udp -log nolog # SPICE console IN ACCEPT -source 10.0.20.0/24 -dport 3128 -p tcp -log nolog # Allow HTTP redirect to HTTPS IN ACCEPT -source 10.0.20.0/24 -dport 80 -p tcp -log nolog # Drop everything else IN DROP -log nolog Replace 10.0.20.0/24 with your management subnet. The +pveproxy macro expands to the default Proxmox web UI ports (8006 for HTTPS, 8007 for PBS). Important notes:\nDo not set policy_in: DROP until you have tested your rules. Start with ACCEPT, add your allow rules, then switch to DROP. Always keep SSH access open from at least one IP range. If you lock yourself out, you will need IPMI or physical console access. The firewall processes rules top-down. Place the ESTABLISHED,RELATED accept rule first. Apply Firewall Rules per Node Each node inherits the datacenter policy but can add node-specific rules. Create /etc/pve/nodes/srv1/host.fw:\n1 2 3 4 5 6 7 8 [OPTIONS] enable: 1 [RULES] # Allow Proxmox Backup Server traffic to backup host IN ACCEPT -source 10.0.20.30 -dport 8007 -p tcp -log nolog # Allow corosync cluster traffic IN ACCEPT -p udp -dport 5405 -log nolog Apply the rules without rebooting:\n1 pve-firewall restart Verify Firewall Rules 1 2 3 4 5 # List all active rules pve-firewall rules # Check the log for dropped packets tail -f /var/log/pve-firewall.log Step 2 — Enable TOTP Two-Factor Authentication Password-only access to the Proxmox web UI is a single point of failure. Proxmox VE 8.x supports TOTP (Time-based One-Time Password) natively — no third-party module needed.\nEnable TOTP for root 1 2 3 4 5 # Install the OTP tools if not already present apt update \u0026amp;\u0026amp; apt install -y oathtool # Generate a TOTP secret for root pveum user modify root@pam --otp-type totp This command generates a QR code in the terminal and outputs the TOTP secret (base32). Scan the QR code with your authenticator app (Authy, Google Authenticator, Bitwarden, 2FAS).\nIf the QR code does not render cleanly, you can get the raw secret:\n1 pveum user otp-secret root@pam Copy the base32 secret and add it manually to your authenticator app.\nVerify TOTP Works Log out of the web UI and log back in as root@pam. You should see a second field for the TOTP code after entering your password.\nTest from the CLI:\n1 pvesh get /access/ticket --username root@pam --password This prompts for your password, then the TOTP code.\nEnable TOTP for Regular Users Do not log in as root for daily administration. Create individual users and enforce 2FA on those too:\n1 2 3 4 5 6 7 8 9 # Create a user pveum user add admin@pve --comment \u0026#34;Admin user\u0026#34; pveum acl modify / --user admin@pve --role Administrator # Set a password pveum user password admin@pve # Enable TOTP pveum user modify admin@pve --otp-type totp Create a Backup Code TOTP recovery is not built into Proxmox. Protect yourself by creating a backup authentication method:\n1 2 # Add a second TOTP entry with a different secret pveum user modify root@pam --otp-type totp Scan the new QR code with a second device or save the secret offline. Alternatively, use the Proxmox Datacenter Manager to maintain a secondary access path.\n⚠️ Warning: If you lose access to your TOTP device and have no backup, you will need console access to disable TOTP:\n1 pveum user modify root@pam --otp-type none Step 3 — Install and Configure Fail2ban The Proxmox web UI and API have no built-in rate limiting. A bot can hammer the login endpoint indefinitely. Fail2ban fills this gap by monitoring the Proxmox auth log and temporarily banning IPs after repeated failures.\nInstall Fail2ban 1 apt update \u0026amp;\u0026amp; apt install -y fail2ban Create the Proxmox Jail Create /etc/fail2ban/jail.d/proxmox.conf:\n1 2 3 4 5 6 7 8 [proxmox] enabled = true port = http,https,8006 filter = proxmox logpath = /var/log/pveproxd.log maxretry = 3 bantime = 3600 findtime = 600 This configuration bans an IP for 1 hour after 3 failed login attempts within 10 minutes. Adjust bantime and maxretry to your tolerance — 3600 seconds (1 hour) is reasonable for a homelab. Production environments often use 24 hours.\nCreate the Filter Definition Create /etc/fail2ban/filter.d/proxmox.conf:\n1 2 3 4 [Definition] failregex = ^.*authentication failure;.*rhost=\u0026lt;HOST\u0026gt;.*$ ^.*pveproxy.*failed authentication.*from: \u0026lt;HOST\u0026gt;.*$ ignoreregex = This matches the two most common failure patterns in /var/log/pveproxd.log. Test it:\n1 fail2ban-regex /var/log/pveproxd.log /etc/fail2ban/filter.d/proxmox.conf You should see a match count. If it shows zero, check the log format with:\n1 grep \u0026#34;authentication failure\u0026#34; /var/log/pveproxd.log | head -5 Add an SSH Jail (Optional but Recommended) Create /etc/fail2ban/jail.d/sshd.conf:\n1 2 3 4 5 6 7 [sshd] enabled = true port = ssh logpath = %(sshd_log)s maxretry = 5 bantime = 3600 findtime = 600 Start and Enable Fail2ban 1 2 systemctl enable --now fail2ban fail2ban-client status You should see both proxmox and sshd jails listed. Check the Proxmox jail specifically:\n1 fail2ban-client status proxmox Step 4 — Harden SSH Access SSH is the most common attack vector on any Linux server. Proxmox nodes are no exception.\nDisable Password Authentication Edit /etc/ssh/sshd_config.d/hardening.conf:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 # Disable password auth — keys only PasswordAuthentication no KbdInteractiveAuthentication no ChallengeResponseAuthentication no # Use only modern key types PubkeyAcceptedAlgorithms ssh-ed25519,rsa-sha2-512 HostKeyAlgorithms ssh-ed25519,rsa-sha2-512 # Restrict login to specific users AllowUsers root admin # Timeouts ClientAliveInterval 300 ClientAliveCountMax 2 # Disable root login with password (already covered above, be explicit) PermitRootLogin prohibit-password # Use only SSH protocol 2 Protocol 2 # Disable X11 forwarding unless needed X11Forwarding no # Limit max auth attempts MaxAuthTries 3 MaxSessions 10 Apply the changes:\n1 systemctl restart sshd Before you close your current SSH session, open a second terminal and test that key-based authentication still works:\n1 ssh -o PasswordAuthentication=no root@pve-host Keep the first session open until you have confirmed the new session works.\nChange the SSH Port (Optional) Changing the SSH port from 22 reduces automated attack noise:\n1 Port 2222 Update your firewall rules and Fail2ban jail to match, then restart SSH. This is security through obscurity — it stops bots but not targeted attacks. Pair it with key-only auth for real security.\nGenerate and Deploy Ed25519 Keys If you are still using RSA 2048-bit keys, upgrade today:\n1 2 ssh-keygen -t ed25519 -a 100 -f ~/.ssh/id_ed25519 -C \u0026#34;homelab-$(date +%Y-%m-%d)\u0026#34; ssh-copy-id -i ~/.ssh/id_ed25519.pub root@pve-host Ed25519 keys are faster, smaller, and more secure than RSA. The -a 100 flag runs 100 KDF rounds on the key for extra protection of the private key at rest.\nStep 5 — Confine Services with AppArmor Proxmox ships with AppArmor profiles for key services, but they are not always enforced by default on all installations.\nCheck AppArmor Status 1 aa-status Look for these Proxmox-related profiles:\nlxc-container-default lxc-container-default-with-nesting pve-firewall pveproxy Enable AppArmor Enforcement If a profile is in complain mode instead of enforce:\n1 2 3 4 5 6 # Set LXC profiles to enforce aa-enforce /etc/apparmor.d/lxc-container-default aa-enforce /etc/apparmor.d/lxc-container-default-with-nesting # Reload AppArmor systemctl reload apparmor Verify:\n1 aa-status | grep -E \u0026#34;(lxc|pve)\u0026#34; All profiles should show \u0026ldquo;enforce\u0026rdquo; mode, not \u0026ldquo;complain.\u0026rdquo;\nAudit AppArmor Denials AppArmor denials log to the kernel ring buffer:\n1 dmesg | grep -i apparmor | grep -i denied | tail -10 Or check syslog:\n1 grep \u0026#34;apparmor.*DENIED\u0026#34; /var/log/syslog | tail -10 If a legitimate service triggers denials, update the profile or add a local override in /etc/apparmor.d/local/. Do not disable AppArmor.\nStep 6 — Enable Automatic Security Updates Proxmox does not install security updates automatically by default. In a homelab where you do not log in daily, this means known CVEs can sit unpatched for weeks.\nInstall Unattended-Upgrades 1 apt update \u0026amp;\u0026amp; apt install -y unattended-upgrades apt-listchanges Configure for Proxmox Edit /etc/apt/apt.conf.d/50unattended-upgrades:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Unattended-Upgrade::Allowed-Origins { \u0026#34;${distro_id}:${distro_codename}-security\u0026#34;; \u0026#34;${distro_id}ESMApps:${distro_codename}-apps-security\u0026#34;; \u0026#34;${distro_id}ESM:${distro_codename}-infra-security\u0026#34;; \u0026#34;Proxmox:${distro_codename}-pve-no-subscription\u0026#34;; \u0026#34;Proxmox:${distro_codename}-pbs-no-subscription\u0026#34;; }; Unattended-Upgrade::AutoFixInterruptedDpkg \u0026#34;true\u0026#34;; Unattended-Upgrade::MinimalSteps \u0026#34;true\u0026#34;; Unattended-Upgrade::Remove-Unused-Kernel-Packages \u0026#34;true\u0026#34;; Unattended-Upgrade::Remove-New-Unused-Dependencies \u0026#34;true\u0026#34;; Unattended-Upgrade::Remove-Unused-Dependencies \u0026#34;false\u0026#34;; Unattended-Upgrade::Automatic-Reboot \u0026#34;false\u0026#34;; Unattended-Upgrade::Automatic-Reboot-Time \u0026#34;03:00\u0026#34;; Key points for Proxmox:\nInclude pve-no-subscription in the allowed origins. Without it, Proxmox-specific packages (pve-kernel, qemu-server) will never be auto-updated. Set Automatic-Reboot to false — a kernel update should never reboot your hypervisor silently. Reboot manually during your maintenance window. Enable Remove-Unused-Kernel-Packages to keep /boot from filling up with old kernels. Configure Automatic Download Edit /etc/apt/apt.conf.d/20auto-upgrades:\n1 2 3 4 APT::Periodic::Update-Package-Lists \u0026#34;1\u0026#34;; APT::Periodic::Download-Upgradeable-Packages \u0026#34;1\u0026#34;; APT::Periodic::AutocleanInterval \u0026#34;7\u0026#34;; APT::Periodic::Unattended-Upgrade \u0026#34;1\u0026#34;; Test the Configuration 1 2 3 4 5 # Dry run unattended-upgrades --dry-run --verbose # Check the log cat /var/log/unattended-upgrades/unattended-upgrades.log Step 7 — Audit and Verify Run a quick audit to confirm everything is in place:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 #!/bin/bash # proxmox-audit.sh — Run as root echo \u0026#34;=== Firewall Status ===\u0026#34; pve-firewall status 2\u0026gt;/dev/null || echo \u0026#34;FAIL\u0026#34; echo \u0026#34;\u0026#34; echo \u0026#34;=== TOTP for root ===\u0026#34; pveum user list --output-format yaml | grep -A2 \u0026#34;root@pam\u0026#34; | grep otp echo \u0026#34;(empty = no TOTP configured)\u0026#34; echo \u0026#34;\u0026#34; echo \u0026#34;=== Fail2ban Status ===\u0026#34; fail2ban-client status proxmox 2\u0026gt;/dev/null | grep -E \u0026#34;Status|Banned\u0026#34; || echo \u0026#34;FAIL or not installed\u0026#34; echo \u0026#34;\u0026#34; echo \u0026#34;=== SSH Config ===\u0026#34; grep -E \u0026#34;^(PasswordAuthentication|PermitRootLogin|Port)\u0026#34; /etc/ssh/sshd_config /etc/ssh/sshd_config.d/*.conf 2\u0026gt;/dev/null | grep -v \u0026#34;no$\u0026#34; echo \u0026#34;\u0026#34; echo \u0026#34;=== AppArmor ===\u0026#34; aa-status 2\u0026gt;/dev/null | grep -E \u0026#34;profiles are in enforce\u0026#34; || echo \u0026#34;AppArmor not active\u0026#34; echo \u0026#34;\u0026#34; echo \u0026#34;=== Unattended Upgrades ===\u0026#34; systemctl is-active unattended-upgrades 2\u0026gt;/dev/null || echo \u0026#34;FAIL\u0026#34; Save this as /root/proxmox-audit.sh and run it monthly:\n1 bash /root/proxmox-audit.sh Summary A hardened Proxmox host is not difficult to achieve, but it requires deliberate configuration of layers that Proxmox leaves open by default:\nLayer Default Hardened Firewall Disabled Datacenter DROP policy, management-subnet-only Authentication Password only TOTP two-factor Rate limiting None Fail2ban, 3 attempts → 1h ban SSH Password + root login Key-only, restricted users, ed25519 Service confinement Complain (some) Enforce mode Updates Manual Daily security, manual reboot Audit None Monthly audit script Each step builds on the others. The firewall limits who can reach the services. 2FA protects against credential theft. Fail2ban blocks brute-force attempts at the network level. SSH hardening eliminates password guessing entirely. AppArmor contains any service compromise. And automatic updates ensure known vulnerabilities are patched before they are exploited.\nStart with the firewall and SSH hardening — those two steps alone eliminate 90% of common attack vectors. Add 2FA and Fail2ban next. Then schedule automatic updates and run a monthly audit.\nYour hypervisor is the foundation of your entire homelab. Hardening it is not optional — it is the single most impactful security improvement you can make.\n","permalink":"https://blog.gntech.me/posts/2026-05-20-proxmox-security-hardening/","summary":"A comprehensive Proxmox VE hardening guide covering firewall rules, TOTP 2FA, Fail2ban, SSH lockdown, AppArmor, and unattended upgrades — secure your hypervisor beyond default settings.","title":"Proxmox VE Security Hardening — Firewall, 2FA, and Fail2ban"},{"content":"WireGuard is already the fastest general-purpose VPN protocol available, but default Linux kernel settings are tuned for desktop workloads, not high-throughput encrypted UDP forwarding. If you\u0026rsquo;re routing 1 Gbps+ through a WireGuard tunnel and seeing CPU spikes, packet drops, or asymmetric performance, the bottleneck is almost never WireGuard itself — it\u0026rsquo;s the kernel\u0026rsquo;s conservative defaults for UDP buffering, congestion control, and interrupt handling.\nThis guide walks through every high-impact tuning parameter, provides copy-paste configurations for homelab and production scales, and shows you how to benchmark properly so you can measure your gains.\nWhy Default Linux Settings Limit WireGuard Throughput WireGuard runs in kernel space (since Linux 5.6) as a virtual network interface. Traffic enters the interface, gets encrypted with ChaCha20-Poly1305, and exits as UDP packets on the real NIC. The kernel path is:\nApplication → wg0 (encrypt) → eth0 (UDP out) → Network Three things can go wrong with stock settings:\nUDP receive buffers too small — the kernel drops packets before WireGuard can decrypt them under high throughput CUBIC congestion control — designed for long-fat TCP networks, not UDP tunnels carrying TCP-over-TCP traffic NAPI polling budget too low — the kernel yields the CPU before draining the NIC ring buffer, causing drops during bursts Each of these has a direct sysctl fix.\nStep 1 — Benchmark Your Baseline (Before Touching Anything) Never tune blind. Establish a baseline so you can compare before/after.\nInstall iperf3 on both ends:\n1 2 sudo apt install iperf3 # Debian/Ubuntu sudo dnf install iperf3 # Fedora/RHEL Start the server on the remote endpoint:\n1 iperf3 -s -B 10.0.0.1 # bind to WireGuard tunnel IP Run a single-stream test from the client:\n1 iperf3 -c 10.0.0.1 Then a parallel-stream test to assess multi-core scaling:\n1 iperf3 -c 10.0.0.1 -P 4 Test the reverse direction (often asymmetric with default settings):\n1 iperf3 -c 10.0.0.1 -R Record the single-stream, parallel, and reverse results. If your parallel test is significantly faster than single-stream, you have headroom to gain by tuning. If they\u0026rsquo;re identical, you\u0026rsquo;re hitting a per-flow bottleneck — likely MTU fragmentation or UDP buffer drops.\nAlso check for retransmits and packet drops during the test:\n1 2 ip -s link show wg0 ip -s link show eth0 High RX errors or dropped on the real NIC under WireGuard load is your first red flag.\nStep 2 — sysctl Configuration for WireGuard Performance Create /etc/sysctl.d/99-wireguard.conf with the following optimizations. Apply with sysctl --system after editing.\nUDP Buffer Sizes — The Single Biggest Win 1 2 3 4 5 6 7 8 9 # /etc/sysctl.d/99-wireguard.conf # Increase max UDP buffer sizes to 16 MB net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 net.core.rmem_default = 262144 net.core.wmem_default = 262144 # UDP memory allocation — minimum, pressure, and max (bytes × pages) net.ipv4.udp_mem = 4096 87380 16777216 The default rmem_max on most distros is 212 KB — completely inadequate for 1 Gbps WireGuard tunnels. With a 16 MB buffer, the kernel can queue incoming encrypted packets while WireGuard processes them, eliminating drops during traffic bursts.\nBBR Congestion Control — Better Than CUBIC for VPN Tunnels 1 2 3 # Use BBR with fq (fair queue) qdisc net.core.default_qdisc = fq net.ipv4.tcp_congestion_control = bbr BBR (Bottleneck Bandwidth and Round-trip propagation time) is significantly less sensitive to packet loss than CUBIC. This matters because TCP-over-TCP inside a WireGuard tunnel amplifies loss — a single dropped encrypted UDP packet can trigger retransmits in both the inner and outer TCP stacks. BBR minimizes this by pacing based on measured bandwidth and RTT rather than packet loss signals.\nVerify BBR is active:\n1 2 sysctl net.ipv4.tcp_congestion_control # net.ipv4.tcp_congestion_control = bbr IP Forwarding and Connection Tracking 1 2 3 4 5 # Required for routing traffic through the tunnel net.ipv4.ip_forward = 1 # Increase connection tracking max (tune based on concurrent users) net.netfilter.nf_conntrack_max = 131072 WireGuard itself is stateless, but if you\u0026rsquo;re using iptables/nftables masquerading for egress traffic through the tunnel, conntrack tracks every connection. At 100+ concurrent devices, the default of 65536 may fill up, causing new connections to be dropped.\nNAPI Polling Budget — Handle Traffic Bursts 1 2 3 4 5 6 7 # Allow CPU to process more packets per polling cycle net.core.netdev_budget = 600 net.core.netdev_budget_usecs = 4000 # Increase NIC input queue and listen backlog net.core.netdev_max_backlog = 5000 net.core.somaxconn = 8192 netdev_budget defaults to 300 packets per SoftIRQ cycle. Under heavy WireGuard load, 10 Gbps NICs can deliver thousands of packets in a single interrupt. Increasing to 600 (and up to 1200 for 10 Gbps+) prevents the network stack from yielding before the NIC buffer is drained.\nApply the config:\n1 sudo sysctl --system Step 3 — MTU and MSS Clamping MTU mismatches are the most common cause of \u0026ldquo;slow WireGuard\u0026rdquo; that isn\u0026rsquo;t actually WireGuard\u0026rsquo;s fault. WireGuard adds 60 bytes of overhead (20 IP + 8 UDP + 32 ChaCha20-Poly1305) per packet. Standard Ethernet MTU of 1500 bytes leaves 1440 bytes for payload — but if your path has lower MTU (PPPoE adds 8 bytes, GRE tunnels add more), packets fragment.\nFind the Correct MTU From the WireGuard server, ping the client with DF (Don\u0026rsquo;t Fragment):\n1 ping -M do -s 1472 -c 5 10.0.0.2 Start at 1472 (1500 − 28 for ICMP header) and decrease by 8 until you get no fragmentation. Subtract 60 for the WireGuard overhead.\nCommon MTU values:\nNetwork type Interface MTU WireGuard MTU Standard Ethernet 1500 1420 PPPoE 1492 1412 PPPoE + VLAN 1488 1408 Jumbo frames 9000 8920 Set it in your WireGuard interface config:\n1 2 3 4 [Interface] PrivateKey = ... Address = 10.0.0.1/24 MTU = 1420 # \u0026lt;- adjust based on your path MTU MSS Clamping for TCP Traffic If you\u0026rsquo;re routing entire subnets through WireGuard (not just the tunnel endpoints), TCP connections may not honor the tunnel MTU. Clamp MSS at the firewall:\n1 2 sudo iptables -t mangle -A FORWARD -o wg0 -p tcp --tcp-flags SYN,RST SYN \\ -j TCPMSS --clamp-mss-to-pmtu For nftables:\ntable inet mangle { chain forward { type filter hook forward priority mangle; policy accept; oifname \u0026#34;wg0\u0026#34; tcp flags syn tcp option maxseg size set rt mtu } } This forces the TCP MSS to fit inside the WireGuard tunnel MTU, eliminating fragmentation entirely.\nStep 4 — Verify GRO/GSO Offloads Are Enabled Generic Receive Offload (GRO) and Generic Segmentation Offload (GSO) batch packets before handing them to WireGuard, reducing per-packet CPU cost significantly. Verify they\u0026rsquo;re enabled on the physical NIC:\n1 ethtool -k eth0 | grep -E \u0026#39;gro|gso|tso\u0026#39; You want to see:\ngro: on gso: on tso: on If any are off, enable them:\n1 sudo ethtool -K eth0 gro on gso on tso on On some NICs (especially virtualized ones in Proxmox/VMware), these offloads may be disabled or unsupported. If they can\u0026rsquo;t be enabled, you lose some batching efficiency — but the sysctl tuning above still helps significantly.\nStep 5 — CPU Scaling Governor WireGuard\u0026rsquo;s ChaCha20-Poly1305 encryption is CPU-bound. If your CPU frequency governor is set to powersave or ondemand, the kernel may downclock cores under light load, causing throughput drops when a burst arrives.\nSet the governor to performance during benchmarking:\n1 echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor For permanent use, install cpufrequtils or configure tlp/power-profiles-daemon to keep cores at max frequency while WireGuard is active. In a homelab context, the power savings from downclocking are negligible compared to the throughput cost.\nStep 6 — Re-Benchmark with Tuned Configuration After applying all changes, run the same iperf3 tests from Step 1:\n1 2 3 4 5 6 7 8 # Single stream iperf3 -c 10.0.0.1 # Parallel streams iperf3 -c 10.0.0.1 -P 4 # Reverse direction iperf3 -c 10.0.0.1 -R Expected improvements on a 1 Gbps link:\nMetric Before (defaults) After (tuned) Single-stream TCP upload 300-500 Mbps 700-950 Mbps Parallel-stream TCP upload 400-600 Mbps 900-940 Mbps Reverse direction (download) 80-200 Mbps 700-900 Mbps The reverse direction improvement is often the most dramatic — default receive buffers starve the download path because the kernel can\u0026rsquo;t queue incoming encrypted packets fast enough.\nCommon Pitfalls Don\u0026rsquo;t tune both sides asymmetrically. Apply the same sysctl config on the server and all major clients for consistent behavior.\nDon\u0026rsquo;t use wg-quick MTU unless you know your path MTU. wg-quick defaults to 1420, which works for most Ethernet paths but may need reduction for PPPoE, cellular, or tunnel-over-tunnel paths.\nTCP-over-TCP is real. If you\u0026rsquo;re doing massive file transfers over WireGuard, consider using UDP-based protocols (rsync over SSH with -e \u0026quot;nc -u\u0026quot;, or UDP-based file transfer tools) to avoid nested congestion control fighting itself.\nSystemd\u0026rsquo;s sysctl persistence. The config file in /etc/sysctl.d/ persists across reboots. Verify with sysctl --system after boot.\nComplete Config File — Copy and Apply 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 # /etc/sysctl.d/99-wireguard.conf # WireGuard kernel performance tuning — homelab scale (10-50 clients) # UDP buffers — eliminate drops under load net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 net.core.rmem_default = 262144 net.core.wmem_default = 262144 net.ipv4.udp_mem = 4096 87380 16777216 # BBR congestion control net.core.default_qdisc = fq net.ipv4.tcp_congestion_control = bbr # Routing and conntrack net.ipv4.ip_forward = 1 net.netfilter.nf_conntrack_max = 131072 # NAPI and buffer tuning net.core.netdev_budget = 600 net.core.netdev_budget_usecs = 4000 net.core.netdev_max_backlog = 5000 net.core.somaxconn = 8192 Apply with sudo sysctl --system, set your WireGuard MTU to the correct value after a path MTU discovery, clamp MSS on the forward chain, and run your benchmarks. You should see 80-95% of line rate over WireGuard on modern hardware — the same as what a direct connection delivers.\n","permalink":"https://blog.gntech.me/posts/2026-05-20-wireguard-kernel-performance-tuning/","summary":"A practical guide to maximizing WireGuard throughput on Linux — covering kernel sysctl tuning, BBR congestion control, UDP buffer sizes, MTU/MSS clamping, NAPI polling optimization, and proper benchmarking with iperf3.","title":"WireGuard Performance Tuning — Kernel sysctl and MTU Optimization"},{"content":"There is a recurring question on r/mikrotik: \u0026ldquo;Any real benefit to running Pi-hole for DNS at this point?\u0026rdquo; The short answer is less and less every release. RouterOS 7 gained three features that make the router itself a credible DNS security appliance:\nDNS over HTTPS (DoH) — Encrypted DNS queries that prevent snooping and DNS poisoning DNS Adlist — Native domain blocking with automatic list updates, null-routing ad/tracker/malware domains to 0.0.0.0 Built-in DNS cache — Configurable up to 4 GB, capable of handling hundreds of concurrent clients If you already own a MikroTik router, you can eliminate the Pi-hole container, reclaim its RAM and CPU, and simplify your network stack — all while keeping ad blocking at the same level.\nThis guide covers the complete configuration: DoH setup with certificate validation, DNS Adlist configuration with automatic updates, performance tuning, and a comparison against the Pi-hole approach so you can decide if RouterOS-native DNS is right for you.\nWhat RouterOS 7 DNS Can Do The DNS subsystem in RouterOS 7 has evolved significantly:\nFeature RouterOS 6 RouterOS 7.14+ DNS over HTTPS No Yes (single server) DNS Adlist No Yes Cache size limit 2048 KiB Up to 4 GiB Concurrent queries 100 1000+ configurable Regular expression static entries Yes Yes (preserved) Firewall address-list integration Manual Automatic with address-list on DNS static The Adlist feature is the key differentiator. It mirrors what Pi-hole does — maintain a blocklist of known ad/tracker/malware domains — but runs entirely in RouterOS firmware. No container, no overlay filesystem, no extra power draw.\nWhen RouterOS DNS Is Not Enough Before we dive in, be honest about the limitations:\nSingle DoH server only — RouterOS supports exactly one DoH upstream. No fallback if that server is unreachable. No per-client filtering — The Adlist applies to all clients equally. You cannot have different blocklists for kids vs. adults. No query logging UI — RouterOS does not have a Pi-hole-style dashboard showing top blocked domains, query volume, or client statistics. Adlist is static — Lists are fetched on a schedule. You cannot whitelist a blocked domain interactively without editing the adlist configuration. No HTTP2 DoH support — Quad9 uses HTTP2 and will not work with RouterOS. If you need per-client filtering, a detailed dashboard, or interactive whitelisting, Pi-hole remains the better tool. For everything else — 80% of homelab use cases — RouterOS DNS is sufficient and simpler.\nStep 1 — Increase DNS Cache Size The default RouterOS DNS cache is 2048 KiB (2 MB). Adlists consume cache space — a standard blocklist of 100k domains takes roughly 10-15 MB. Increase the cache to accommodate both cached resolutions and Adlist entries.\nAccess your router via SSH or WinBox and run:\n1 /ip dns set cache-size=65536 This sets a 64 MB cache. On a router with 256 MB RAM, this is trivial. For larger blocklists or busy networks:\n1 /ip dns set cache-size=262144 256 MB is the practical maximum for most RouterOS devices. Verify:\n1 /ip dns print Look for cache-size: 262144KiB.\nAlso raise the concurrent query limit if you have many clients:\n1 2 /ip dns set max-concurrent-queries=500 /ip dns set max-concurrent-tcp-sessions=50 Step 2 — Configure DNS over HTTPS (DoH) DoH prevents your ISP, anyone on your local network, or a MITM attacker from seeing which domains your clients resolve. RouterOS supports DoH with HTTPS/1.1 only.\nStep 2a — Configure a Standard DNS Fallback RouterOS needs at least one regular DNS server to resolve the DoH server\u0026rsquo;s own hostname. This is a chicken-and-egg problem: DoH needs to resolve the DoH server URL, but DNS is needed to do that.\n1 /ip dns set servers=1.1.1.1 Once DoH is active, RouterOS stops using standard DNS queries entirely — the regular server is only for the initial DoH hostname resolution and as a fallback if DoH fails.\nStep 2b — Choose a DoH Provider RouterOS is compatible with these DoH providers:\nProvider DoH URL Notes Cloudflare https://cloudflare-dns.com/dns-query Fastest, global anycast Google https://dns.google/dns-query Reliable, good uptime NextDNS https://[your-id].dns.nextdns.io/dns-query Custom filtering, analytics OpenDNS https://doh.opendns.com/dns-query Cisco-backed, family filter options Avoid: Quad9 (HTTP2 only), Mullvad, Yandex (not supported).\nStep 2c — Install the Root CA Certificate For verify-doh-cert=yes, RouterOS needs the root CA that signed the DoH server\u0026rsquo;s TLS certificate. Cloudflare uses a certificate signed by Baltimore CyberTrust Root or GlobalSign. The easiest way:\n1 2 /tool fetch url=\u0026#34;https://curl.se/ca/cacert-2025-12-01.pem\u0026#34; /certificate import file-name=cacert-2025-12-01.pem Or install a specific CA. For Cloudflare, download the Baltimore root:\n1 2 /tool fetch url=\u0026#34;https://cacerts.digicert.com/DigiCertGlobalRootCA.crt.pem\u0026#34; /certificate import file-name=DigiCertGlobalRootCA.crt.pem Then trust it:\n1 /certificate set [find where common-name~\u0026#34;DigiCert Global Root\u0026#34;] trusted=yes Step 2d — Enable DoH 1 2 /ip dns set use-doh-server=https://cloudflare-dns.com/dns-query \\ verify-doh-cert=yes Verify DoH is active:\n1 /ip dns print Look for use-doh-server: https://cloudflare-dns.com/dns-query and verify-doh-cert: yes. The dynamic-servers field should show the DoH server IP.\nTest with a domain resolution:\n1 :put ([ resolve \u0026#34;google.com\u0026#34; ]); If this returns an IP address, DoH is working. If it fails, check the DoH server URL spelling and certificate installation.\nTroubleshooting DoH 1 2 3 4 5 6 # Check for DNS errors in the log /log print where topics~\u0026#34;dns\u0026#34; # Verify certificate /certificate print where trusted=yes # Flush cache and retry /ip dns cache flush Common issues:\nCertificate validation fails — Wrong CA or missing intermediate cert. Try verify-doh-cert=no to confirm DoH works, then fix certs DoH timeout — The DoH server hostname cannot resolve. Verify servers setting has at least one working DNS server No response — RouterOS firmware too old. Minimum version: 7.13 Step 3 — Configure DNS Adlist for Ad Blocking The DNS Adlist feature was introduced in RouterOS 7.14. It downloads a list of domains from a URL and null-routes A/AAAA queries for those domains to 0.0.0.0.\nStep 3a — Find Blocklist Sources RouterOS expects a plain text file with one domain per line (no 0.0.0.0 prefix, no 127.0.0.1 prefix — just domain names).\nThe mikrotik-adlist project on GitHub maintains curated blocklists in the exact format RouterOS requires. Lists are auto-generated from the AdGuard Host Lists Registry and updated weekly.\nAvailable lists (raw URLs for direct use):\nList URL Domains Focus Standard https://raw.githubusercontent.com/IgorKha/mikrotik-adlist/main/hosts/standard.txt ~35k Ads + trackers Strict https://raw.githubusercontent.com/IgorKha/mikrotik-adlist/main/hosts/strict.txt ~80k Ads + trackers + analytics DNS Blocklist https://adguardteam.github.io/HostlistsRegistry/assets/filter_1.txt ~15k Minimal ad blocking For a first deployment, start with the standard list. It catches 95% of ads without false positives on legitimate services.\nStep 3b — Add the Adlist Entry 1 /ip dns adlist add url=https://raw.githubusercontent.com/IgorKha/mikrotik-adlist/main/hosts/standard.txt RouterOS immediately downloads and processes the list. Verify:\n1 /ip dns adlist print Output should show:\n1 2 3 Flags: *, U - UPDATED 0 url=\u0026#34;https://raw.githubusercontent.com/IgorKha/mikrotik-adlist/main/hosts/standard.txt\u0026#34; ssl-verify=yes status=done The status=done means the list was downloaded and loaded successfully. Check how many entries were added:\n1 /ip dns adlist stats Step 3c — Add Multiple Lists You can add up to several adlist entries. RouterOS merges them all:\n1 2 /ip dns adlist add url=https://raw.githubusercontent.com/IgorKha/mikrotik-adlist/main/hosts/standard.txt /ip dns adlist add url=https://raw.githubusercontent.com/IgorKha/mikrotik-adlist/main/hosts/strict.txt Test that blocking works:\n1 :put ([ resolve \u0026#34;doubleclick.net\u0026#34; ]); Expected result: 0.0.0.0. If it returns a real IP, the domain is not in the list.\nStep 3d — Configure Automatic Updates Adlists change constantly — new ad domains appear, old ones go away. Schedule a weekly update:\n1 2 3 4 /system scheduler add name=update-adlists \\ interval=7d \\ start-time=03:00:00 \\ on-event=\u0026#34;/ip dns adlist update\u0026#34; Verify the scheduler:\n1 /system scheduler print where name=update-adlists The scheduler runs at 3:00 AM every 7 days, refreshes all adlists, and flushes the DNS cache so stale blocked entries are cleared.\nStep 3e — Whitelist a Domain If a legitimate service breaks because its domain is in the adlist, RouterOS has no interactive whitelist UI. Work around it with DNS static entries:\n1 /ip dns static add name=example.com address=REAL_IP This static entry takes priority over the adlist. To find the real IP of the blocked domain, temporarily disable the adlist:\n1 /ip dns adlist disable [find] Then resolve the domain:\n1 :put ([ resolve \u0026#34;example.com\u0026#34; ]); Re-enable the adlist:\n1 /ip dns adlist enable [find] Step 4 — Lock Down DNS Access With DoH and ad blocking active, prevent clients from bypassing your router\u0026rsquo;s DNS by redirecting all outbound DNS traffic.\nBlock External DNS (Firewall Rule) Add a firewall rule that drops DNS requests to external servers:\n1 2 3 4 5 6 7 8 9 10 11 12 13 /ip firewall filter add chain=forward \\ protocol=udp dst-port=53 \\ src-address=10.0.0.0/16 \\ dst-address=!10.0.0.1 \\ action=drop \\ comment=\u0026#34;Block external DNS (UDP)\u0026#34; /ip firewall filter add chain=forward \\ protocol=tcp dst-port=53 \\ src-address=10.0.0.0/16 \\ dst-address=!10.0.0.1 \\ action=drop \\ comment=\u0026#34;Block external DNS (TCP)\u0026#34; Replace 10.0.0.0/16 with your LAN subnet and 10.0.0.1 with your router\u0026rsquo;s LAN IP. Add these rules after your allow rules but before the default forward drop rule.\nForce All DNS Through Router (NAT Redirect) Redirect any DNS traffic destined elsewhere to the router:\n1 2 3 4 5 6 /ip firewall nat add chain=dstnat \\ protocol=udp dst-port=53 \\ src-address=10.0.0.0/16 \\ dst-address=!10.0.0.1 \\ action=redirect to-ports=53 \\ comment=\u0026#34;Redirect external DNS to router\u0026#34; This catches clients with hardcoded DNS like 8.8.8.8 and silently redirects their queries to the RouterOS DNS cache with DoH and adlist filtering.\nStep 5 — Performance Tuning The DNS cache on a MikroTik is not infinitely scalable. Tune these parameters for a busy network:\nCache TTL Tuning RouterOS honors upstream TTLs but caps them:\n1 /ip dns set cache-max-ttl=4h Setting a max TTL of 4 hours prevents rarely-queried domains from occupying cache space for the default 7 days. More aggressive:\n1 /ip dns set cache-max-ttl=1h Use 1h on memory-constrained devices (hAP ac², RB750 series).\nDoH Connection Pooling 1 2 /ip dns set doh-max-server-connections=10 /ip dns set doh-max-concurrent-queries=200 Default values are conservative (5 connections, 50 queries). Doubling them on a CCR or RB5009 handles burst traffic from many clients.\nMonitor Cache Usage 1 /ip dns print Check the cache-used field. If it approaches cache-size, the cache is thrashing — increase cache size or reduce cache-max-ttl.\n1 2 # Top-level domains in cache /ip dns cache all print count-only Step 6 — Comparison: RouterOS DNS vs. Pi-hole Aspect RouterOS Adlist Pi-hole Docker Hardware required MikroTik router Any Docker host RAM usage ~20-50 MB ~150-300 MB Setup time 10 minutes 30-60 minutes Blocklist management URL-based, auto-update UI-based, gravity Whitelisting Manual static entries Interactive UI Query dashboard None Rich web interface Per-client filtering No Yes (groups) DHCP integration Built-in Requires DHCP relay DoH upstream Single server Multiple (via Unbound) Edge case blocking DNS-based only DNS + regex + PID Maintenance None Update container Network complexity Same + Docker network Choose RouterOS DNS when:\nYou have a MikroTik router with 128 MB+ RAM You want zero-maintenance blocking You do not need a query dashboard The Docker host is resource-constrained You want simpler network topology Choose Pi-hole when:\nYou need per-client filtering (kids vs. adults) You want a rich dashboard with historical data You need regex-based ad blocking You want interactive whitelisting/blacklisting Your router does not support Adlist (pre-7.14) Step 7 — Full Configuration Script Here is a complete, copy-paste script for a fresh RouterOS 7 configuration:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 # ========================================== # MikroTik RouterOS 7 DNS Security Setup # ========================================== # 1. Increase cache and performance limits /ip dns set cache-size=65536 /ip dns set cache-max-ttl=4h /ip dns set max-concurrent-queries=500 /ip dns set max-concurrent-tcp-sessions=50 # 2. Standard DNS fallback /ip dns set servers=1.1.1.1 # 3. Enable DoH with Cloudflare /ip dns set use-doh-server=https://cloudflare-dns.com/dns-query \\ verify-doh-cert=yes # 4. Add DNS Adlist for ad blocking /ip dns adlist add \\ url=https://raw.githubusercontent.com/IgorKha/mikrotik-adlist/main/hosts/standard.txt \\ ssl-verify=yes # 5. Schedule weekly adlist updates /system scheduler add name=update-adlists \\ interval=7d \\ start-time=03:00:00 \\ on-event=\u0026#34;/ip dns adlist update\u0026#34; # 6. Allow remote requests /ip dns set allow-remote-requests=yes # 7. Block external DNS bypass /ip firewall filter add chain=forward \\ protocol=udp dst-port=53 \\ src-address=10.0.0.0/16 \\ dst-address=!10.0.0.1 \\ action=drop \\ comment=\u0026#34;Block external DNS (UDP)\u0026#34; /ip firewall filter add chain=forward \\ protocol=tcp dst-port=53 \\ src-address=10.0.0.0/16 \\ dst-address=!10.0.0.1 \\ action=drop \\ comment=\u0026#34;Block external DNS (TCP)\u0026#34; # 8. Redirect hardcoded DNS to router /ip firewall nat add chain=dstnat \\ protocol=udp dst-port=53 \\ src-address=10.0.0.0/16 \\ dst-address=!10.0.0.1 \\ action=redirect to-ports=53 \\ comment=\u0026#34;Redirect external DNS to router\u0026#34; Adjust src-address and dst-address to match your LAN subnet and router IP before pasting.\nVerifying Everything Works Test each layer independently:\n1. DoH is encrypting queries:\n1 /ip dns print Confirm use-doh-server is set and dynamic-servers is populated.\n2. Adlist is blocking ads:\n1 2 3 :put ([ resolve \u0026#34;doubleclick.net\u0026#34; ]); :put ([ resolve \u0026#34;googleads.g.doubleclick.net\u0026#34; ]); :put ([ resolve \u0026#34;pagead2.googlesyndication.com\u0026#34; ]); All should return 0.0.0.0.\n3. Legitimate resolution still works:\n1 2 3 :put ([ resolve \u0026#34;google.com\u0026#34; ]); :put ([ resolve \u0026#34;github.com\u0026#34; ]); :put ([ resolve \u0026#34;mikrotik.com\u0026#34; ]); Should return real IP addresses.\n4. DNS bypass is blocked: From a client, run:\n1 dig @8.8.8.8 google.com This should time out or return a RouterOS-cached response.\n5. Cache is performing:\n1 /ip dns print cache-used should be non-zero and growing.\nSummary RouterOS 7\u0026rsquo;s DNS Adlist and DoH features transform a MikroTik router from a basic DNS forwarder into a credible network-level ad-blocking appliance. For the majority of homelab setups, this eliminates the need for a separate Pi-hole container — saving RAM, CPU, and network complexity.\nThe configuration is straightforward:\nIncrease cache size to 64-256 MB to accommodate the adlist Enable DoH with Cloudflare or Google for encrypted queries Add Adlist URLs from GitHub-hosted blocklists in RouterOS format Schedule weekly updates so the blocklist stays current Lock down DNS by blocking external port 53 traffic and redirecting hardcoded resolvers Verify each layer independently If you need rich query analytics or per-client filtering, Pi-hole still has an edge. But for a clean, minimal, single-device DNS security stack that requires zero maintenance, RouterOS native DNS is hard to beat.\n","permalink":"https://blog.gntech.me/posts/2026-05-20-mikrotik-dns-doh-adlist/","summary":"Configure DNS over HTTPS and the built-in DNS Adlist feature on MikroTik RouterOS 7 to block ads, trackers, and malware network-wide — no Pi-hole, no extra hardware, no containers needed.","title":"MikroTik RouterOS 7 DNS Over HTTPS and Adlist — Block Ads Without Pi-Hole"},{"content":"If your homelab runs WordPress, Nextcloud, Grafana, GitLab, or any session-backed web application, you are almost certainly running Redis somewhere. Since Redis shifted to a non-OSI license in 2024, the community has rallied behind Valkey — a fully open-source (BSD-3-Clause), Redis-compatible fork backed by the Linux Foundation.\nValkey 9.1 dropped on May 19, 2026, and it is a significant release: 2.1 million requests per second with 512-byte payloads, database-level access control, JSON logging, and a modular Lua engine. This guide walks through deploying Valkey 9.1 with Docker Compose, configuring it for production use, hardening it with ACLs, and connecting your homelab services.\nWhy Valkey Over Redis It is worth understanding the landscape. Redis was the king of in-memory key-value stores for over a decade. In 2024, Redis Ltd. switched to a Server Side Public License (SSPL), meaning you cannot offer Redis as a managed service without a commercial agreement. Valkey forked from Redis 7.2.4 under the original BSD-3-Clause license and is governed by the Linux Foundation with contributions from AWS, Google, Oracle, and Ericsson.\nAspect Redis Valkey License SSPL (restrictive) BSD-3-Clause Governance Single vendor Linux Foundation Latest stable 7.4.x 9.1.0 Performance Baseline +15-25% (9.1 I/O threading) DB-level ACLs No Yes (9.1) JSON logging No Yes (9.1) Drop-in compatible — Yes, same protocol and commands For a homelab, the practical difference is zero today — Valkey speaks the Redis protocol, so any app that connects to Redis connects to Valkey unchanged. The long-term difference is that Valkey is evolving faster and is fully open-source.\nStep 1 — Docker Compose Deployment Create a project directory and a basic docker-compose.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 # /opt/valkey/docker-compose.yml services: valkey: image: valkey/valkey:9.1-alpine restart: unless-stopped container_name: valkey command: [\u0026#34;valkey-server\u0026#34;, \u0026#34;/etc/valkey/valkey.conf\u0026#34;] ports: - \u0026#34;127.0.0.1:6379:6379\u0026#34; volumes: - ./valkey.conf:/etc/valkey/valkey.conf:ro - ./data:/data - ./tls:/etc/valkey/tls:ro environment: - TZ=America/Santo_Domingo cap_drop: - ALL cap_add: - SETGID - SETUID - NET_BIND_SERVICE networks: - valkey-net healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;valkey-cli\u0026#34;, \u0026#34;ping\u0026#34;] interval: 10s timeout: 3s retries: 5 start_period: 10s logging: driver: json-file options: max-size: 10m max-file: 3 networks: valkey-net: driver: bridge ipam: config: - subnet: 172.20.0.0/24 Key points:\nBind to 127.0.0.1: Valkey should never be exposed directly to your LAN. Only containers on the same Docker network or the host should reach it. Read-only config mount: Prevents runtime modification to the configuration file. Capability drop: Strips all kernel capabilities except the minimum needed (SETGID, SETUID, NET_BIND_SERVICE). Healthcheck: Every 10 seconds, Valkey-CLI pings the server. Separate network: Gives Valkey a predictable subnet for service discovery. Step 2 — Valkey Configuration for Homelab Create valkey.conf in the same directory:\n# /opt/valkey/valkey.conf — Valkey 9.1 Homelab Configuration # Bind to loopback and Docker network bind 127.0.0.1 172.20.0.2 protected-mode yes port 6379 # Persistence — snapshot every 5 min if at least 1 key changed save 300 1 save 60 100 stop-writes-on-bgsave-error yes rdbcompression yes rdbchecksum yes dbfilename dump.rdb # Append-only file for durability (recommended for production) appendonly yes appendfilename \u0026#34;appendonly.aof\u0026#34; appendfsync everysec auto-aof-rewrite-percentage 100 auto-aof-rewrite-min-size 64mb # Memory management — cap at 512MB for homelab maxmemory 512mb maxmemory-policy allkeys-lru # Logging — JSON for log aggregation logfile \u0026#34;\u0026#34; loglevel notice log-format json # Lua — restrict to read-only lua-time-limit 5000 # Performance — I/O threading (new in 9.1) io-threads 4 io-threads-do-reads yes # Slow log slowlog-log-slower-than 10000 slowlog-max-len 128 # ACL — require a password requirepass ${VALKEY_PASSWORD} aclfile /etc/valkey/users.acl # TLS paths (set to empty to disable) tls-cert-file \u0026#34;\u0026#34; tls-key-file \u0026#34;\u0026#34; tls-ca-cert-file \u0026#34;\u0026#34; Understanding the Memory Policy maxmemory-policy allkeys-lru is the right choice for a homelab caching layer. It evicts the least-recently-used key when memory fills up. For session storage specifically, volatile-lru (evict only keys with an expiry set) is also valid — but allkeys-lru handles apps that forget to set TTLs.\nSet maxmemory to a value that leaves enough headroom for the OS. On a 8GB Proxmox host running several containers, 512MB is safe. If Valkey lives on a dedicated 2GB VM, set it to 1.5GB.\nStep 3 — Securing Valkey with ACLs Valkey 9.1 introduces number database-level access control, which lets you restrict users to specific databases. This is invaluable when multiple apps share a single Valkey instance.\nCreate users.acl:\n# /opt/valkey/users.acl # Admin user with full access user default on \u0026gt;${VALKEY_PASSWORD} ~* \u0026amp;* +@all # Application users — scoped to specific databases # WordPress — database 0 user wordpress on \u0026gt;wp-secure-pass ~* +@all -@dangerous db=0 # Grafana — database 1, read-mostly user grafana on \u0026gt;graf-secure-pass ~* +@read +@write -@admin db=1 # Rate limiter — database 2, minimal permissions user ratelimit on \u0026gt;rate-secure-pass ~* +GET +SET +EXPIRE +TTL db=2 The +@dangerous category includes FLUSHALL, FLUSHDB, DEBUG, CONFIG, SHUTDOWN, and SLAVEOF. Removing it from application users prevents accidental or malicious data destruction.\nLoad the ACL file by pointing config at it:\naclfile /etc/valkey/users.acl Then apply it at runtime without restart:\n1 docker exec valkey valkey-cli ACL LOAD You can test access restrictions:\n1 2 3 4 5 6 7 8 9 10 # Authenticate as the wordpress user docker exec valkey valkey-cli -a wp-secure-pass # Works — database 0 \u0026gt; SELECT 0 OK # Blocked — database 3 \u0026gt; SELECT 3 (error) NOPERM No permissions to access database Step 4 — Connecting Homelab Services WordPress + Valkey Object Cache Add a object-cache.php drop-in and configure wp-config.php:\n1 2 3 4 5 define(\u0026#39;WP_REDIS_HOST\u0026#39;, \u0026#39;valkey\u0026#39;); define(\u0026#39;WP_REDIS_PORT\u0026#39;, 6379); define(\u0026#39;WP_REDIS_PASSWORD\u0026#39;, \u0026#39;wp-secure-pass\u0026#39;); define(\u0026#39;WP_REDIS_DATABASE\u0026#39;, 0); define(\u0026#39;WP_CACHE\u0026#39;, true); Nextcloud + Valkey Nextcloud uses Valkey for file locking, caching, and MFA rate limiting. In config/config.php:\n1 2 3 4 5 6 7 8 9 \u0026#39;memcache.local\u0026#39; =\u0026gt; \u0026#39;\\\\OC\\\\Memcache\\\\APCu\u0026#39;, \u0026#39;memcache.distributed\u0026#39; =\u0026gt; \u0026#39;\\\\OC\\\\Memcache\\\\Redis\u0026#39;, \u0026#39;memcache.locking\u0026#39; =\u0026gt; \u0026#39;\\\\OC\\\\Memcache\\\\Redis\u0026#39;, \u0026#39;redis\u0026#39; =\u0026gt; [ \u0026#39;host\u0026#39; =\u0026gt; \u0026#39;valkey\u0026#39;, \u0026#39;port\u0026#39; =\u0026gt; 6379, \u0026#39;password\u0026#39; =\u0026gt; \u0026#39;wp-secure-pass\u0026#39;, \u0026#39;timeout\u0026#39; =\u0026gt; 1.5, ], Grafana + Valkey Grafana can use Valkey for session storage and query caching. In grafana.ini:\n1 2 3 4 5 6 7 8 [session] provider = redis provider_config = addr=valkey:6379,password=graf-secure-pass,db=1 [caching] enabled = true cache_type = redis cache_options = addr=valkey:6379,password=graf-secure-pass,db=1 Generic Docker Compose Service For any other container on the same Docker network, reference Valkey by service name:\n1 2 3 4 5 6 7 8 9 services: myapp: image: myapp:latest networks: - valkey-net networks: valkey-net: external: true Then create the network once:\n1 docker network create valkey-net Step 5 — Verifying Performance Valkey 9.1\u0026rsquo;s new I/O threading model and hardware-clock-based timestamps deliver measurable gains. Run the built-in benchmark:\n1 2 docker exec valkey valkey-benchmark -h 127.0.0.1 \\ -p 6379 -n 100000 -c 50 -P 16 -t get,set Expect output like:\n====== SET ====== 100000 requests completed in 0.85 seconds 50 parallel clients 16 bytes payload keep alive: 1 host configuration \u0026#34;save\u0026#34;: 300 1 host configuration \u0026#34;appendonly\u0026#34;: yes 117647.05 requests per second ====== GET ====== 100000 requests completed in 0.73 seconds 50 parallel clients 16 bytes payload keep alive: 1 host configuration \u0026#34;save\u0026#34;: 300 1 host configuration \u0026#34;appendonly\u0026#34;: yes 136986.30 requests per second On a moderate homelab server (Ryzen 5, NVMe SSD), Valkey 9.1 easily hits 100k+ ops/sec. The I/O threading improvement matters most when multiple pipelines run concurrently — WordPress + Nextcloud + Grafana hitting the same instance.\nCheck real-time I/O thread usage:\n1 2 docker exec valkey valkey-cli INFO STATS | grep -E \u0026#34;(io_thread|instantaneous)\u0026#34; Step 6 — Monitoring with JSON Logging New in 9.1: Valkey can emit logs in JSON format, making them natively parseable by Loki, OpenSearch, or any JSON-aware log shipper.\nWith log-format json enabled, logs look like:\n1 2 3 4 5 6 7 8 { \u0026#34;timestamp\u0026#34;: \u0026#34;2026-05-19T14:22:33.508Z\u0026#34;, \u0026#34;role\u0026#34;: \u0026#34;master\u0026#34;, \u0026#34;pid\u0026#34;: 14082, \u0026#34;message\u0026#34;: \u0026#34;* Valkey is starting\u0026#34;, \u0026#34;version\u0026#34;: \u0026#34;255.255.255\u0026#34;, \u0026#34;bits\u0026#34;: 64 } If you have Grafana Alloy or Promtail collecting Docker logs, they will pick up the JSON format automatically with zero parsing configuration — no regex-based log scraping needed.\nStep 7 — Backups and Maintenance Valkey persists to both RDB snapshots and AOF logs. Back them up:\n1 2 3 4 #!/bin/bash # /opt/valkey/backup.sh docker exec valkey valkey-cli SAVE cp -r /opt/valkey/data/ /backup/valkey-$(date +%F)/ Or trigger BGSAVE and copy the file zero-downtime:\n1 2 3 4 5 docker exec valkey valkey-cli BGSAVE # Wait for background save to complete while [ \u0026#34;$(docker exec valkey valkey-cli INFO persistence | \\ grep \u0026#39;rdb_bgsave_in_progress:1\u0026#39;)\u0026#34; ]; do sleep 1; done cp /opt/valkey/data/dump.rdb /backup/ For zero-data-loss scenarios, enable AOF with appendfsync everysec. You lose at most one second of writes on a crash.\nWhen to Skip Valkey Valkey is not always the right tool:\nEphemeral caching only: If you never need the data across restarts, run Valkey with save \u0026quot;\u0026quot; and appendonly no. It becomes a pure cache with no disk writes. Single-app homelab: If you have one small app, a SQLite file or a Docker volume-mounted JSON file is simpler. High memory pressure: Valkey stores everything in RAM. If your homelab runs on a 4GB NUC with existing services, add memory before reaching for Valkey. For everything else — session storage, API rate limiting, job queues, real-time dashboards, object caching — Valkey 9.1 is the best open-source option available today.\nSummary Valkey 9.1 brings database-level ACLs, JSON logging, I/O threading, and significant performance improvements to the Redis-compatible ecosystem. Switching from Redis to Valkey is a drop-in replacement — export your RDB, change the image tag, and restart. No code changes.\nBeyond the migration story, the new security and observability features make Valkey 9.1 a genuinely better choice for any homelab running multiple applications. Database-scoped users mean you can safely share one Valkey instance across WordPress, Grafana, Nextcloud, and a custom rate-limiter without worrying about cross-tenant data access.\nDeploy it today, set up ACLs, point your apps at it, and forget about it. That is the Valkey way.\n","permalink":"https://blog.gntech.me/posts/2026-05-20-valkey-docker-deployment-homelab/","summary":"Complete guide to deploying Valkey 9.1 with Docker Compose — the high-performance Redis fork with new security, observability, and performance features. Covers config, ACL hardening, and connecting your homelab services.","title":"Valkey 9.1 Docker Deployment — High-Performance Cache for Your Homelab"},{"content":"Every Docker Compose stack you deploy needs credentials. Database passwords, API tokens, SMTP credentials, admin usernames, TLS private keys. In a typical homelab with 15-20 containers, you accumulate dozens of secrets scattered across compose files, .env files, and shell scripts.\nThe default approach — hardcode everything in docker-compose.yml or dump it in a .env file in the same directory — works until it doesn\u0026rsquo;t. A misplaced commit, a backup that lands on the wrong drive, or a visitor poking around your terminal history reveals every credential in plaintext.\nThis guide walks through four tiers of Docker secrets management for homelab environments, from dead simple to properly vault-backed. Each tier builds on the previous one, so you can start where you are and level up as your paranoia (or exposure) grows.\nTier 1 — Better .env Hygiene (The Baseline) If you\u0026rsquo;re not using .env files yet, start here. If you are, clean them up.\nStop Hardcoding Everything Don\u0026rsquo;t do this:\n1 2 3 4 5 # ❌ Bad — secrets in plaintext in compose file services: postgres: environment: POSTGRES_PASSWORD: my-super-secret-password Do this:\n1 2 3 4 5 # ✅ Good — secrets come from .env services: postgres: environment: POSTGRES_PASSWORD: ${PG_PASS:?Set PG_PASS in .env} The :? syntax causes Docker Compose to fail at startup with a clear error message instead of substituting an empty string:\n1 2 docker compose up -d # ❌ The variable \u0026#34;PG_PASS\u0026#34; is not set. Set PG_PASS in .env Structure Your .env Files A single flat .env in the project root is fine for small stacks. For larger setups, use separate files for different environments:\n1 2 3 4 5 6 7 /opt/authentik/ ├── docker-compose.yml ├── .env.production # Live credentials ├── .env.staging # Staging/test credentials ├── .env.example # Template with placeholder values (committed to git) ├── .gitignore # Includes .env.production .env.staging └── secrets/ # Git-ignored directory for sensitive files Reference the file explicitly in your compose command:\n1 docker compose --env-file .env.production up -d .gitignore Is Your Safety Net # /opt/authentik/.gitignore .env .env.* secrets/ *.key *.pem !*.example If you version-control your compose files (and you should), the .gitignore prevents secrets from ever landing in a repo. Test it:\n1 2 git add --dry-run . # Verify no .env or secrets/ files are staged Generate Secrets Automatically Stop typing passwords. Generate them at deployment:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 # /opt/authentik/setup.sh #!/bin/bash ENV_FILE=\u0026#34;.env.production\u0026#34; cat \u0026gt; \u0026#34;$ENV_FILE\u0026#34; \u0026lt;\u0026lt; EOF # Auto-generated $(date +%Y-%m-%d) PG_PASS=$(openssl rand -base64 32) AUTHENTIK_SECRET_KEY=$(openssl rand -base64 48) AUTHENTIK_BOOTSTRAP_PASSWORD=$(openssl rand -base64 16) AUTHENTIK_BOOTSTRAP_EMAIL=admin@example.com EOF chmod 600 \u0026#34;$ENV_FILE\u0026#34; echo \u0026#34;Credentials written to $ENV_FILE — save them in your vault!\u0026#34; This is the bare minimum. Every homelab should be here. The remaining tiers add encryption and rotation.\nTier 2 — Docker Secrets for Sensitive Files Docker has a built-in secrets feature that works natively in Swarm mode and is available in standalone Compose via file-based secrets.\nCompose File-Based Secrets For docker compose (not Swarm), secrets are files mounted into the container at runtime:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 # /opt/app/docker-compose.yml services: app: image: my-app:latest secrets: - db_password - api_key - tls_cert environment: # Reference secret file paths DB_PASSWORD_FILE: /run/secrets/db_password API_KEY_FILE: /run/secrets/api_key secrets: db_password: file: ./secrets/db_password.txt api_key: file: ./secrets/api_key.txt tls_cert: file: ./certs/server.pem Inside the container, each secret is available as a plaintext file at /run/secrets/\u0026lt;name\u0026gt;. The application reads the file path from the _FILE environment variable.\nWhy Use Docker Secrets Over Environment Variables Concern Environment Variables Docker Secrets (file) Visible in docker inspect Yes (plaintext) No (file path only) Visible in /proc Yes No Leaked in error traces Common Unlikely Leaked in docker logs If printed at startup Only if app reads + prints Readable by child processes Yes Only if mounted readable File permissions N/A 0400 by default (root) IDE auto-complete logs Can leak No The key advantage: a secret mounted as a file never appears in docker inspect output, docker logs, or environment dumps. This matters when you paste logs into a forum or bug report.\nDatabase Password Example Most Docker images (Postgres, MySQL, Authentik) support the _FILE suffix convention:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 services: postgres: image: postgres:16-alpine secrets: - pg_pass environment: POSTGRES_PASSWORD_FILE: /run/secrets/pg_pass POSTGRES_USER_FILE: /run/secrets/pg_user secrets: pg_pass: file: ./secrets/pg_pass.txt pg_user: file: ./secrets/pg_user.txt Create Secrets Securely 1 2 3 4 5 6 7 8 9 mkdir -p secrets chmod 700 secrets # Generate and write in one step with restricted permissions openssl rand -base64 32 | tee secrets/pg_pass.txt | wc -c chmod 600 secrets/pg_pass.txt echo \u0026#34;admin\u0026#34; \u0026gt; secrets/pg_user.txt chmod 600 secrets/pg_user.txt Git Ignore the Secrets Directory # In project .gitignore secrets/ *.txt Tier 3 — SOPS with age (Encrypted at Rest) Files on disk are still plaintext. If someone gains access to your server and reads secrets/pg_pass.txt, the game is over. SOPS (Secrets OPerationS) by Mozilla encrypts your secrets at rest and decrypts them only at deployment time.\nHow SOPS Works with age SOPS supports multiple encryption backends: age, PGP, AWS KMS, GCP KMS, and Azure Key Vault. For homelab use, age (Actually Good Encryption) is the sweet spot — simpler than PGP, no key server dependency, single file key.\nflowchart LR A[secrets.enc.env] --\u0026gt;|sops decrypt| B[.env.production] B --\u0026gt;|docker compose| C[Containers] D[age key: ~/.config/sops/age/keys.txt] --\u0026gt; A Step 1 — Install SOPS and age 1 2 3 4 5 6 7 8 # Install age sudo apt install age -y # or download from GitHub releases # Install SOPS SOPS_VERSION=\u0026#34;3.9.4\u0026#34; wget https://github.com/getsops/sops/releases/download/v${SOPS_VERSION}/sops-v${SOPS_VERSION}.linux.amd64 sudo mv sops-v${SOPS_VERSION}.linux.amd64 /usr/local/bin/sops sudo chmod +x /usr/local/bin/sops Step 2 — Generate an age Key Pair 1 2 3 4 5 # Generate a new key age-keygen -o ~/.config/sops/age/keys.txt # This produces output like: # public key: age1abc123def456... Protect this key. If you lose keys.txt, all your encrypted secrets are permanently unrecoverable. Back it up to your password manager, offline USB drive, and a safe.\nStep 3 — Configure SOPS Create .sops.yaml in your project root:\n1 2 3 4 5 6 # /opt/authentik/.sops.yaml creation_rules: - path_regex: \\.enc\\.env$ age: age1abc123def456... - path_regex: \\.enc\\.yaml$ age: age1abc123def456... Or set it globally in ~/.config/sops/sops.yaml.\nStep 4 — Encrypt Your .env File 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # Start with your production .env cat .env.production # PG_PASS=abc123 # AUTHENTIK_SECRET_KEY=xyz789 # Encrypt it — creates .env.production.enc sops --encrypt .env.production \u0026gt; .env.production.enc # The encrypted file is binary-safe YAML: cat .env.production.enc # PG_PASS: ENC[AES256_GCM,data:abc123...,iv:...,aad:...,tag:...] # AUTHENTIK_SECRET_KEY: ENC[AES256_GCM,data:xyz789...,...] # sops_lastmodified: ... # sops_mac: ... # sops_age: ... Step 5 — Decrypt at Deployment 1 2 3 4 # Decrypt to a temp file, deploy, then wipe sops --decrypt .env.production.enc \u0026gt; .env.production docker compose --env-file .env.production up -d shred -u .env.production Or automate it in a deploy script:\n1 2 3 4 5 6 7 8 9 10 11 12 #!/bin/bash # /opt/authentik/deploy.sh set -euo pipefail echo \u0026#34;Decrypting secrets...\u0026#34; sops --decrypt .env.production.enc \u0026gt; .env.production trap \u0026#39;echo \u0026#34;Cleaning up...\u0026#34;; shred -u .env.production\u0026#39; EXIT echo \u0026#34;Deploying stack...\u0026#34; docker compose --env-file .env.production up -d --pull always echo \u0026#34;Deployment complete — .env.production wiped.\u0026#34; Encrypt Entire Compose Files For stacks with inline secrets you can\u0026rsquo;t easily extract:\n1 2 3 4 sops --encrypt docker-compose.enc.yml \u0026gt; docker-compose.yml 2\u0026gt;/dev/null # Then deploy sops exec-file docker-compose.enc.yml \u0026#34;docker compose -f {} up -d\u0026#34; Integrate with Your Backup Script Encrypted secrets are safe to back up to the cloud. The backup target never sees plaintext — only the age-protected ciphertext:\n1 2 3 # In your restic backup script (see the restic guide on this blog) # This file is safe to back up anywhere # /opt/authentik/.env.production.enc Recovering the age Key Store a copy of ~/.config/sops/age/keys.txt in your password manager and on an encrypted USB drive. If the server dies, you need both the encrypted .env files AND the age key to recover credentials.\n1 2 3 # Export for backup cat ~/.config/sops/age/keys.txt # age1... Tier 4 — Vaultwarden API for Dynamic Secrets The previous tiers all rely on static secrets — values you generate once and store. For environments where you rotate credentials regularly (or have multiple users who need access), a secret vault with an API is the right approach.\nVaultwarden (self-hosted Bitwarden-compatible server) exposes an API that can serve secrets to your deployment pipeline. Combined with the bw CLI, you can fetch credentials at deploy time without ever touching a file.\nStep 1 — Deploy Vaultwarden 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # /opt/vaultwarden/docker-compose.yml services: vaultwarden: image: vaultwarden/server:latest container_name: vaultwarden restart: unless-stopped volumes: - ./data:/data environment: SIGNUPS_ALLOWED: \u0026#34;false\u0026#34; # Disable after admin account created ADMIN_TOKEN: ${ADMIN_TOKEN:?Set ADMIN_TOKEN} labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.vaultwarden.rule=Host(`vault.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.vaultwarden.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.vaultwarden.tls=true\u0026#34; - \u0026#34;traefik.http.services.vaultwarden.loadbalancer.server.port=80\u0026#34; Deploy and create your admin account, then disable signups.\nStep 2 — Create an API Key In the Vaultwarden web UI:\nGo to Settings → Security → Keys Create a new API Key Save the client ID and client secret (shown once) Step 3 — Store Secrets as Secure Notes Create a secure note in your vault for each compose stack:\nName: authentik-production URI: https://auth.example.com Fields: PG_PASS (hidden) = ... AUTHENTIK_SECRET_KEY (hidden) = ... AUTHENTIK_BOOTSTRAP_PASSWORD (hidden) = ... AUTHENTIK_BOOTSTRAP_EMAIL (visible) = admin@example.com The (hidden) field type prevents the value from being shown in the Vaultwarden web UI unless you explicitly reveal it.\nStep 4 — Fetch Secrets at Deploy Time Install the Bitwarden CLI:\n1 2 3 4 5 6 7 8 # On your deployment server sudo apt install -y curl jq # Download and install bw CLI curl -fsSL \u0026#34;https://github.com/bitwarden/clients/releases/download/cli-v2025.12.0/bw-linux-2025.12.0.zip\u0026#34; \\ -o /tmp/bw.zip sudo unzip -o /tmp/bw.zip -d /usr/local/bin/ sudo chmod +x /usr/local/bin/bw Create a deployment script that fetches secrets from Vaultwarden:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 #!/bin/bash # /opt/authentik/deploy-vault.sh set -euo pipefail BW_CLIENTID=\u0026#34;$1\u0026#34; # user.xx-xx-xx-xx BW_CLIENTSECRET=\u0026#34;$2\u0026#34; # API key VAULT_ITEM=\u0026#34;authentik-production\u0026#34; TARGET_DIR=\u0026#34;/opt/authentik\u0026#34; echo \u0026#34;=== Authenticating with Vaultwarden ===\u0026#34; BW_SESSION=$(bw login --apikey \u0026#34;$BW_CLIENTID\u0026#34; \u0026#34;$BW_CLIENTSECRET\u0026#34; --raw) export BW_SESSION echo \u0026#34;=== Fetching secrets for $VAULT_ITEM ===\u0026#34; ITEM=$(bw get item \u0026#34;$VAULT_ITEM\u0026#34;) # Extract custom fields PG_PASS=$(echo \u0026#34;$ITEM\u0026#34; | jq -r \u0026#39;.fields[] | select(.name == \u0026#34;PG_PASS\u0026#34;) | .value\u0026#39;) AUTHENTIK_SECRET_KEY=$(echo \u0026#34;$ITEM\u0026#34; | jq -r \u0026#39;.fields[] | select(.name == \u0026#34;AUTHENTIK_SECRET_KEY\u0026#34;) | .value\u0026#39;) AUTHENTIK_BOOTSTRAP_PASS=$(echo \u0026#34;$ITEM\u0026#34; | jq -r \u0026#39;.fields[] | select(.name == \u0026#34;AUTHENTIK_BOOTSTRAP_PASSWORD\u0026#34;) | .value\u0026#39;) echo \u0026#34;=== Writing .env.production ===\u0026#34; cat \u0026gt; \u0026#34;$TARGET_DIR/.env.production\u0026#34; \u0026lt;\u0026lt; EOF # Auto-deployed from Vaultwarden — $(date) PG_PASS=$PG_PASS AUTHENTIK_SECRET_KEY=$AUTHENTIK_SECRET_KEY AUTHENTIK_BOOTSTRAP_PASSWORD=$AUTHENTIK_BOOTSTRAP_PASS AUTHENTIK_BOOTSTRAP_EMAIL=admin@example.com EOF chmod 600 \u0026#34;$TARGET_DIR/.env.production\u0026#34; echo \u0026#34;=== Deploying stack ===\u0026#34; cd \u0026#34;$TARGET_DIR\u0026#34; docker compose --env-file .env.production up -d echo \u0026#34;=== Cleaning up ===\u0026#34; shred -u \u0026#34;$TARGET_DIR/.env.production\u0026#34; bw logout echo \u0026#34;Deployment complete.\u0026#34; Run it:\n1 ./deploy-vault.sh \u0026#34;user.xx-xx-xx-xx\u0026#34; \u0026#34;your-api-secret\u0026#34; Step 5 — Secret Rotation Secret rotation is where vault-backed secrets shine. When you need to rotate a database password:\nLog into Vaultwarden Edit the PG_PASS field in the authentik-production item Update the actual Postgres password: docker exec postgres psql -c \u0026quot;ALTER USER authentik WITH PASSWORD 'newpass'\u0026quot; Run the deployment script again No files to edit. No sops re-encryption. No git commits. The single source of truth is the vault.\nTier Comparison — Which One Should You Use Tier Method Setup Effort Security Rotation Best For 1 .env hygiene 5 minutes Low Manual Quick experiments, single-user lab 2 Docker Secrets (file) 10 minutes Medium Manual Production stacks in a trusted LAN 3 SOPS + age 20 minutes High sops re-encrypt Backups, shared repos, CI/CD pipelines 4 Vaultwarden API 45 minutes Very High API-driven Multi-user, frequent rotation, compliance My recommendation for most homelabs:\nStart at Tier 1 — every stack gets a .env.production with auto-generated passwords and strict .gitignore For critical stacks (Vaultwarden itself, Authentik, database servers), go to Tier 3 — SOPS/age encrypts at rest, and the encrypted files are safe to back up anywhere If you have 20+ stacks or share access with others, invest in Tier 4 — the Vaultwarden API approach pays off quickly when you need to rotate credentials without editing files Putting It All Together — Reference Architecture Here\u0026rsquo;s what this looks like in practice across a homelab:\n/opt/ ├── authentik/ │ ├── docker-compose.yml # Committed to git (no secrets) │ ├── .env.production.enc # SOPS/age encrypted (in git) │ ├── .sops.yaml # SOPS config (in git) │ └── deploy.sh # Decrypts + deploys │ ├── gitea/ │ ├── docker-compose.yml │ ├── .env.production.enc │ ├── secrets/ │ │ └── db_password.txt # Docker Secrets file (git-ignored) │ └── .sops.yaml │ ├── postgres/ │ ├── docker-compose.yml # Uses Docker Secrets (not env vars) │ └── secrets/ │ ├── pg_pass.txt │ └── pg_user.txt │ ├── manage-secrets.sh # Centralized vault-fetch script │ └── ~/.config/sops/age/ └── keys.txt # age key (BACKED UP TO VAULTWARDEN) Authentication flow:\nDeveloper/automation pushes a deploy or runs deploy.sh If using SOPS (Tier 3): age key decrypts .enc files to in-memory .env, Docker Compose picks them up If using vault (Tier 4): bw CLI authenticates with API key, fetches secrets from Vaultwarden, writes temporary .env Compose deploys. shred wipes the temporary .env file The only persistent secrets on disk are encrypted (SOPS) or stored in a vault Avoiding Common Mistakes Mistake 1: The .env in the Wrong Scope Docker Compose automatically loads .env from the project directory. Many users create a global .env in their home folder and wonder why it\u0026rsquo;s not picked up. Answer: Compose looks for .env in the same directory as the compose file, not the current working directory.\nMistake 2: Docker Secrets and Swarm Mode File-based secrets work in standalone Compose. Mounted secrets (the Swarm-native /run/secrets/\u0026lt;name\u0026gt;) require Swarm mode and a different syntax. Stick with secrets: { file: ... } unless you\u0026rsquo;re running a Swarm cluster.\nMistake 3: Committing Encrypted Files Without Verification Just because a file is encrypted doesn\u0026rsquo;t mean it\u0026rsquo;s safe to commit. Run git diff --name-only to verify you\u0026rsquo;re not accidentally committing the plaintext .env alongside the encrypted .enc version.\nMistake 4: Losing the age Key SOPS is useless without the age private key. Store it in your password manager, on an encrypted USB drive, and print a paper backup. Yes, paper — a QR code of the key in your safe is better than losing all your credentials when the server dies.\nMistake 5: Overlooking Process Secrets Even with perfect secrets management, secrets can leak through:\nShell history: Commands like PG_PASS=abc123 docker compose up end up in .bash_history. Use export HISTCONTROL=ignorespace and prefix sensitive commands with a space. Docker build args: Build args end up in the image history. Don\u0026rsquo;t use them for secrets — use --secret with BuildKit instead: docker build --secret id=mysecret,src=./secret.txt. Debug logs: Applications that log environment variables at startup will log your secrets. Check Grafana/Loki logs after deployment. The Three-Layer Defense Nobody can predict every attack vector, but a layered approach covers the common ones:\nAt rest: SOPS/age encryption on disk. Even if the server is compromised, secrets are ciphertext unless the age key is also extracted. In transit: Container to container traffic uses Docker internal networks or Traefik with TLS. No plaintext secrets traverse the LAN. In use: Docker Secrets mount files as mode 0400 (root owner). Applications read from /run/secrets/. Environment dumps and docker inspect reveal nothing. If any one layer fails, the other two still protect your credentials.\nSummary Secrets management for Docker Compose doesn\u0026rsquo;t require enterprise infrastructure. The tools are already in your homelab:\n.env hygiene prevents the most common mistakes — hardcoded passwords, committed secrets, and shared credentials Docker Secrets moves sensitive data from environment variables to restricted files that don\u0026rsquo;t leak SOPS with age encrypts secrets at rest, making them safe to back up anywhere and version-control alongside your compose files Vaultwarden API gives you a single source of truth for secrets with push-button rotation Start with Tier 1 today — it takes five minutes and eliminates 90% of the risk. Add Tier 3 for your critical stacks this weekend. Move to Tier 4 when the pain of manual rotation exceeds the setup cost.\nYour future self, trying to remember what the Postgres password is, will thank you.\n","permalink":"https://blog.gntech.me/posts/2026-05-19-docker-secrets-management/","summary":"Practical guide to managing Docker secrets in homelab environments — from basic .env patterns to Docker Secrets, SOPS/age encryption, and programmatic vault integration. Includes real compose examples and secret rotation workflows.","title":"Docker Compose Secrets Management — .env, SOPS, and Vaultwarden for Homelabs"},{"content":"Tailscale makes connecting devices dead simple — install the client, authenticate, and every machine on your tailnet can reach every other machine over an encrypted WireGuard tunnel. No port forwarding, no dynamic DNS, no public IP required.\nThe catch: the official Tailscale control server is proprietary and hosted in Tailscale\u0026rsquo;s cloud. While the free tier is generous, you\u0026rsquo;re trusting someone else\u0026rsquo;s infrastructure with your network topology metadata, and the free tier caps at 100 devices with 3 users.\nEnter Headscale — an open-source, self-hosted implementation of the Tailscale control server. You run the coordination plane on your own hardware, use the same official Tailscale clients on every device, and keep full control over authentication, ACLs, and routing metadata.\nThis guide covers deploying Headscale with Docker Compose, securing it behind Traefik with Let\u0026rsquo;s Encrypt, registering clients, defining ACLs for network segmentation, and enabling subnet routing so your entire homelab LAN becomes reachable from anywhere.\nHow Headscale and Tailscale Work Understanding the architecture helps when things don\u0026rsquo;t work.\n┌──────────────┐ ┌──────────────────┐ ┌──────────────┐ │ Laptop │ │ Headscale │ │ Phone │ │ tailscale │◄────►│ Control Plane │◄────►│ tailscale │ │ 10.0.0.2 │ │ 10.0.0.1 │ │ 10.0.0.3 │ └──────┬───────┘ └──────────────────┘ └──────┬───────┘ │ │ │ WireGuard Direct │ └────────────────────────────────────────────────┘ (NAT traversal) Control Plane (Headscale): Authenticates devices, exchanges public WireGuard keys, distributes the network map. Your traffic never passes through it — it\u0026rsquo;s just the phone book. Data Plane (Tailscale client): Each device establishes direct WireGuard tunnels to every other device. Traffic is peer-to-peer, end-to-end encrypted, with NAT traversal via STUN and DERP relay servers as fallback. Private IP allocation: Every node gets a unique IP in the Tailscale range (default 100.64.0.0/10). These IPs are reachable from any connected device. Headscale replaces only the control plane. The clients are the same official Tailscale clients — no proprietary fork needed.\nStep 1 — Prerequisites You need:\nA server with Docker and Docker Compose (a $5 VPS or your Proxmox host works fine) A domain name pointing to the server\u0026rsquo;s public IP Port 443 (HTTPS) reachable from the internet Traefik or another reverse proxy already running (this guide assumes Traefik, but any reverse proxy works) Headscale is extremely lightweight. A single-core VM with 512MB RAM handles 50+ devices without breaking a sweat — the control plane exchanges key material and coordinates routes but never touches traffic.\nStep 2 — Directory Structure and Configuration 1 2 mkdir -p /opt/headscale/{config,lib} cd /opt/headscale Download the Headscale configuration file for the latest stable release:\n1 2 3 HEADSCALE_VERSION=\u0026#34;0.28.0\u0026#34; curl -o config/config.yaml \\ \u0026#34;https://raw.githubusercontent.com/juanfont/headscale/v${HEADSCALE_VERSION}/config-example.yaml\u0026#34; Edit config/config.yaml with the minimal changes needed:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 # /opt/headscale/config/config.yaml # Listen on all interfaces inside the container (port 8080) listen_addr: 0.0.0.0:8080 # The public URL of your Headscale instance — this must be reachable # by ALL clients. Set this to your reverse proxy URL. server_url: https://headscale.example.org # Listen address for gRPC (for headscale CLI commands) grpc_listen_addr: 0.0.0.0:50443 # Enable gRPC for CLI access grpc_allow_insecure: false # Database — SQLite is fine for homelab use (\u0026lt; 100 devices). # For HA setups, switch to PostgreSQL. database: type: sqlite3 sqlite: path: /var/lib/headscale/db.sqlite # WireGuard settings wireguard: private_key_path: /var/lib/headscale/wireguard.key # Tuning acme_url: https://acme-v02.api.letsencrypt.org/directory acme_email: \u0026#34;admin@example.org\u0026#34; dns: # Base domain for your tailnet — this is where your internal DNS # MagicDNS names live (e.g., laptop.example-tailnet.ts.net) domain: example-headscale.ts.net # Whether to override the local DNS on clients magic_dns: true # DNS servers to push to clients (can serve internal services) nameservers: - 10.0.20.1 # Your homelab DNS (Pi-hole, Unbound, etc.) - 1.1.1.1 - 8.8.8.8 # Search domains — handy for resolving internal hostnames search_domains: - homelab.internal # Randomize client port for better NAT traversal randomize_client_port: true Key configuration details:\nserver_url: Must be the HTTPS URL clients use to reach Headscale. Change this to your actual domain. magic_dns: When enabled, every device gets a DNS name like laptop.example-headscale.ts.net. Clients can resolve each other by hostname instead of IP. nameservers: Push your homelab DNS servers to connected clients so internal hostnames resolve on the VPN. Step 3 — Docker Compose Deployment 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 # /opt/headscale/docker-compose.yml services: headscale: image: headscale/headscale:0.28.0 container_name: headscale restart: unless-stopped read_only: true tmpfs: - /var/run/headscale ports: - \u0026#34;127.0.0.1:8080:8080\u0026#34; - \u0026#34;127.0.0.1:9090:9090\u0026#34; volumes: - ./config:/etc/headscale:ro - ./lib:/var/lib/headscale environment: - TZ=America/Santo_Domingo command: serve healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;headscale\u0026#34;, \u0026#34;health\u0026#34;] interval: 30s timeout: 10s retries: 3 start_period: 10s About this setup:\nports: 127.0.0.1:8080:8080: Only binds to localhost. Headscale is NOT exposed directly — your reverse proxy handles external traffic. This is a critical security boundary. read_only: true: The container filesystem is immutable. Runtime data lives in the tmpfs mount (for PID/socket files) and the lib volume (for the database and keys). Port 9090: gRPC endpoint for CLI operations. Only needed for docker exec commands. Start it:\n1 docker compose up -d Verify it\u0026rsquo;s healthy:\n1 2 curl http://127.0.0.1:8080/health # {\u0026#34;status\u0026#34;:\u0026#34;pass\u0026#34;} Step 4 — Traefik Reverse Proxy Configuration If you\u0026rsquo;re already running Traefik (see the Gitea or Authentik deployment guides on this blog), add a new router and service:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # Traefik dynamic config — add to your existing Traefik config services: headscale: loadBalancer: servers: - url: \u0026#34;http://10.0.20.30:8080\u0026#34; routers: headscale: rule: \u0026#34;Host(`headscale.example.org`)\u0026#34; entryPoints: - websecure service: headscale tls: certResolver: letsencrypt Or in Docker Compose with labels (if Traefik auto-discovers):\n1 2 3 4 5 6 7 8 9 services: headscale: # ... rest of the service config labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.headscale.rule=Host(`headscale.example.org`)\u0026#34; - \u0026#34;traefik.http.routers.headscale.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.headscale.tls.certResolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.services.headscale.loadbalancer.server.port=8080\u0026#34; If you use Nginx Proxy Manager or Caddy, the setup is similar: proxy pass to http://127.0.0.1:8080, enable WebSocket support, and terminate TLS with Let\u0026rsquo;s Encrypt.\nVerify the public endpoint:\n1 2 curl https://headscale.example.org/health # {\u0026#34;status\u0026#34;:\u0026#34;pass\u0026#34;} Step 5 — Create Users and Namespaces Headscale groups devices by users (previously called namespaces in older versions). Devices under the same user can see each other by default.\n1 2 3 4 5 6 7 8 9 # Create a user for your primary network docker compose exec headscale headscale users create homelab # Create separate users for isolation docker compose exec headscale headscale users create servers docker compose exec headscale headscale users create iot # List users docker compose exec headscale headscale users list Output:\nID | Name | Created 1 | homelab | 2026-05-19 17:05:00 2 | servers | 2026-05-19 17:05:00 3 | iot | 2026-05-19 17:05:00 Why separate users? Devices in different users are isolated. Use this to segment trusted devices (your laptop, phone) from untrusted ones (IoT widgets, guest VMs).\nStep 6 — Register Client Devices Method A: Interactive Registration (for laptops and workstations) Install the Tailscale client on your device:\n1 2 3 4 5 # Linux curl -fsSL https://tailscale.com/install.sh | sh # macOS brew install tailscale On the client, start the connection pointing at your Headscale server:\n1 sudo tailscale up --login-server https://headscale.example.org The client prints an authentication URL:\nTo authenticate, visit: https://headscale.example.org/register/STqfQR4wnKr-eerDXam3_RYi Open that URL in a browser. It shows the node key and a command to run on your headscale server:\n1 2 3 4 5 6 # Run this on the headscale server docker compose exec headscale headscale nodes register \\ --key STqfQR4wnKr-eerDXam3_RYi \\ --user homelab # Node registered Verify the node is connected:\n1 docker compose exec headscale headscale nodes list Output:\nID | Hostname | User | IP addresses | Connected | Last seen 1 | laptop | homelab | 100.64.0.1, fd7a:... | Yes | 2026-05-19 17:10 From the laptop, test connectivity:\n1 2 tailscale status # show all connected devices ping 100.64.0.2 # ping another connected node Method B: Pre-Auth Keys (for servers and automation) Pre-auth keys let you register devices without interactive authentication — perfect for CI/CD, deployment scripts, or headless servers:\n1 2 3 4 5 6 7 # Generate a pre-auth key for the servers user, reusable docker compose exec headscale headscale preauthkeys create \\ --user servers \\ --reusable \\ --expiration 24h # Output: c932d8c4e4a8d2209cf539e6d3ef0a4212330b8a24e60d49 On the server node:\n1 2 3 sudo tailscale up \\ --login-server https://headscale.example.org \\ --authkey c932d8c4e4a8d2209cf539e6d3ef0a4212330b8a24e60d49 The node registers silently and appears in the list immediately.\nKey options:\nFlag Purpose --reusable Use the same key for multiple devices --ephemeral Node disappears from the list when it disconnects --expiration 24h Key auto-expires (recommended for security) Step 7 — Subnet Routing (Access Your Full LAN) By default, Tailscale only gives you access to the Tailscale IPs of connected devices. If you want to reach your entire 10.0.20.0/24 homelab LAN from a remote laptop, enable subnet routing.\n7a — Configure the Exit Node On the server that sits on your homelab LAN (e.g., your Proxmox host), advertise the subnet:\n1 2 3 sudo tailscale up \\ --login-server https://headscale.example.org \\ --advertise-routes=10.0.20.0/24 7b — Approve Routes on Headscale 1 2 3 4 5 6 7 8 docker compose exec headscale headscale routes list # ID | Machine | Prefix | Enabled | Primary # 1 | proxmox-host | 10.0.20.0/24 | false | false docker compose exec headscale headscale routes enable -r 1 # Route enabled 7c — Accept Routes on Clients On each client that needs LAN access:\n1 2 3 sudo tailscale up \\ --login-server https://headscale.example.org \\ --accept-routes Now from your laptop anywhere in the world:\n1 2 3 ping 10.0.20.1 # Your router curl http://10.0.20.30:8888 # SearXNG on Proxmox (if accessible) ssh admin@10.0.20.30 # SSH to Proxmox Your entire LAN is reachable as if you were sitting at home.\nExit Node (Full Tunnel) For full traffic routing (like a traditional VPN), advertise as an exit node:\n1 2 3 sudo tailscale up \\ --login-server https://headscale.example.org \\ --advertise-routes=0.0.0.0/0,::/0 Approve the 0.0.0.0/0 route on Headscale:\n1 docker compose exec headscale headscale routes enable -r \u0026#34;\u0026lt;route-id\u0026gt;\u0026#34; Clients can then use this node as an exit node:\n1 2 3 sudo tailscale up \\ --login-server https://headscale.example.org \\ --exit-node=proxmox-host All traffic routes through your home connection — useful for public Wi-Fi security or accessing geo-restricted services.\nStep 8 — Access Control Lists (ACLs) Without ACLs, any device can talk to any other device. ACLs let you define who can talk to whom with a simple declarative syntax.\nCreate /opt/headscale/config/acls.hujson:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 { \u0026#34;groups\u0026#34;: { \u0026#34;group:admin\u0026#34;: [\u0026#34;homelab\u0026#34;], \u0026#34;group:servers\u0026#34;: [\u0026#34;servers\u0026#34;], \u0026#34;group:iot\u0026#34;: [\u0026#34;iot\u0026#34;] }, \u0026#34;acls\u0026#34;: [ // Admins can access everything {\u0026#34;action\u0026#34;: \u0026#34;accept\u0026#34;, \u0026#34;src\u0026#34;: [\u0026#34;group:admin\u0026#34;], \u0026#34;dst\u0026#34;: [\u0026#34;*:*\u0026#34;]}, // Servers can talk to each other on all ports {\u0026#34;action\u0026#34;: \u0026#34;accept\u0026#34;, \u0026#34;src\u0026#34;: [\u0026#34;group:servers\u0026#34;], \u0026#34;dst\u0026#34;: [\u0026#34;group:servers:*\u0026#34;]}, // Admin can access servers on SSH and HTTPS only {\u0026#34;action\u0026#34;: \u0026#34;accept\u0026#34;, \u0026#34;src\u0026#34;: [\u0026#34;group:admin\u0026#34;], \u0026#34;dst\u0026#34;: [\u0026#34;group:servers:22,443\u0026#34;]}, // IoT devices can only reach the internet (exit node) // and cannot initiate connections to admin or server devices {\u0026#34;action\u0026#34;: \u0026#34;accept\u0026#34;, \u0026#34;src\u0026#34;: [\u0026#34;group:iot\u0026#34;], \u0026#34;dst\u0026#34;: [\u0026#34;*:80,443\u0026#34;]}, // Default deny — anything not explicitly allowed is blocked {\u0026#34;action\u0026#34;: \u0026#34;deny\u0026#34;, \u0026#34;src\u0026#34;: [\u0026#34;*\u0026#34;], \u0026#34;dst\u0026#34;: [\u0026#34;*:*\u0026#34;]} ], // Tag-based ACLs for more granular control \u0026#34;tagOwners\u0026#34;: { \u0026#34;tag:monitoring-server\u0026#34;: [\u0026#34;group:admin\u0026#34;] } } Reference this file in config.yaml:\n1 2 # /opt/headscale/config/config.yaml acl_policy_path: /etc/headscale/acls.hujson Then apply and test:\n1 2 3 4 5 6 docker compose restart headscale # Test ACLs docker compose exec headscale headscale acl debug \\ --host 100.64.0.1 --port 22 # Action: accept ACL best practices:\nStart restrictive: Default deny, then grant specific access Use groups: Group devices by function, not by hostname Protocol-aware: Tag-based ACLs let you match on Tailscale tags instead of user membership Test changes: Run acl debug before deploying to prevent accidental lockouts Step 9 — DERP Relay Servers Tailscale prefers direct peer-to-peer connections, but when NAT traversal fails (symmetric NAT, strict firewalls), it falls back to DERP (Detour Encrypted Routing Protocol) relay servers.\nHeadscale includes a built-in DERP server. Enable it in config.yaml:\n1 2 3 4 5 6 7 8 9 10 derp: server: enabled: true region_id: 999 region_code: \u0026#34;homelab-derp\u0026#34; region_name: \u0026#34;Homelab DERP\u0026#34; stun_listen_addr: \u0026#34;0.0.0.0:3478\u0026#34; urls: # Pull official Tailscale DERP servers as fallback - https://controlplane.tailscale.com/derpmap/default Add the STUN port to your firewall:\n1 2 # On the host sudo ufw allow 3478/udp If you don\u0026rsquo;t want to depend on Tailscale\u0026rsquo;s DERP servers at all (self-contained tailnet with no external dependencies), set urls: [] — but this breaks connectivity between clients behind symmetric NAT that can\u0026rsquo;t establish direct WireGuard tunnels.\nStep 10 — Production Hardening 10a — Regular Backups Your Headscale data is in /opt/headscale/lib/. Back up the entire directory:\n1 2 # Add to your nightly backup script tar czf /backups/headscale-$(date +%Y%m%d).tar.gz -C /opt/headscale lib/ If you lose this data, all registered devices need to re-authenticate. The database contains the WireGuard public keys, node assignments, and ACL configuration.\n10b — Monitoring and Alerts Add health checks to your monitoring stack:\n1 2 3 4 5 6 # Prometheus scrape config for Headscale metrics scrape_configs: - job_name: \u0026#39;headscale\u0026#39; metrics_path: \u0026#39;/metrics\u0026#39; static_configs: - targets: [\u0026#39;headscale.example.org\u0026#39;] Grafana dashboard metrics to watch:\nheadscale_nodes_total — registered device count headscale_nodes_online — currently connected devices headscale_requests_total — API request volume Certificate expiry (from Traefik) 10c — Key Rotation 1 2 3 4 5 6 7 # Force re-authentication for all nodes (after a compromise) docker compose exec headscale headscale nodes expire --all # Regenerate your WireGuard key docker compose exec headscale headscale node delete --all docker compose exec headscale headscale nodes expire --all # Then restart headscale and re-register clients 10d — Restrict gRPC Access The gRPC port (9090) is bound to localhost by default and should stay that way. If you need remote CLI access, tunnel through SSH:\n1 2 ssh admin@headscale-host \\ \u0026#34;docker compose exec headscale headscale nodes list\u0026#34; Step 11 — Client Tuning Tips macOS/iOS clients Add DNS search domains for your homelab:\n1 2 3 4 sudo tailscale up \\ --login-server https://headscale.example.org \\ --accept-dns=false # Then manually configure DNS in System Settings Linux server clients (no GUI) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # Headless setup for servers sudo systemctl enable --now tailscaled # Register with pre-auth key (no browser needed) sudo tailscale up \\ --login-server https://headscale.example.org \\ --authkey \u0026#34;\u0026lt;preauth-key\u0026gt;\u0026#34; \\ --hostname web-server-01 # Advertise routes sudo tailscale up \\ --login-server https://headscale.example.org \\ --authkey \u0026#34;\u0026lt;preauth-key\u0026gt;\u0026#34; \\ --advertise-routes=10.0.20.0/24 \\ --accept-routes Docker containers on the tailnet Use a Tailscale sidecar container for Docker services:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 services: tailscale-sidecar: image: tailscale/tailscale:latest container_name: tailscale-sidecar hostname: my-service environment: - TS_AUTHKEY=\u0026lt;preauth-key\u0026gt; - TS_EXTRA_ARGS=--login-server=https://headscale.example.org - TS_USERSPACE=true - TS_STATE_DIR=/var/lib/tailscale volumes: - tailscale-state:/var/lib/tailscale - /dev/net/tun:/dev/net/tun cap_add: - net_admin - sys_module restart: unless-stopped volumes: tailscale-state: Then route traffic through the sidecar\u0026rsquo;s Tailscale IP. This gives each Docker service its own stable mesh IP without installing anything on the host.\nWhy Headscale Over Direct WireGuard or Tailscale SaaS Feature Headscale Tailscale SaaS WireGuard only Control plane Self-hosted Cloud-only None (manual) Client count Unlimited 100 (free) Unlimited NAT traversal Built-in Built-in Manual (STUN) ACLs Declarative HuJSON Web UI + ACLs iptables/nftables MagicDNS Yes Yes No Subnet routing Yes Yes Manual routes Exit nodes Yes Yes Manual NAT Data sovereignty Full None Full Setup complexity Medium Low High Cost VPS + domain Free tier Free Choose Headscale when: You want Tailscale\u0026rsquo;s zero-config WireGuard mesh without depending on Tailscale\u0026rsquo;s infrastructure, need unlimited devices, or want full ACL control with declarative policies.\nChoose Tailscale SaaS when: You want the easiest possible setup, don\u0026rsquo;t mind the dependency, and your device count is under 100.\nChoose raw WireGuard when: You need a simple site-to-site VPN with a known peer count and can manage keys manually.\nTroubleshooting Clients can\u0026rsquo;t connect 1 2 3 4 5 6 7 8 9 10 11 # Check Headscale server health curl https://headscale.example.org/health # Check DNS resolution dig +short headscale.example.org # Check port reachability from outside nc -zv headscale.example.org 443 # View server logs docker compose logs --tail=50 headscale Nodes show \u0026ldquo;offline\u0026rdquo; in the list 1 2 3 4 5 # Restart tailscaled on the client sudo systemctl restart tailscaled # Then re-authenticate sudo tailscale up --login-server https://headscale.example.org MagicDNS not resolving 1 2 3 4 5 6 # On the client, verify DNS configuration tailscale dns status # If MagicDNS is disabled or misconfigured, restart tailscaled sudo tailscale down sudo tailscale up --login-server https://headscale.example.org --accept-dns ACL change not taking effect 1 2 3 4 5 6 # Restart headscale to reload ACLs docker compose restart headscale # Debug specific traffic docker compose exec headscale headscale acl debug \\ --src 100.64.0.1 --dst 100.64.0.2 --port 443 Summary Headscale gives you the best of both worlds: Tailscale\u0026rsquo;s zero-config WireGuard mesh with WireGuard\u0026rsquo;s full data sovereignty. Running it in Docker Compose with Traefik is straightforward and maintainable:\nDeploy Headscale behind a reverse proxy on port 443 Create users to segment devices logically Register clients interactively or with pre-auth keys Enable subnet routing to reach your full homelab LAN Define ACLs with HuJSON for least-privilege access Add DERP relay for reliable NAT traversal Back up the lib directory — without it, every device re-authenticates The result is a mesh VPN that connects your laptop, phone, servers, and IoT devices into a single private network with zero manual WireGuard key management, no port forwarding, and no third-party dependency.\n","permalink":"https://blog.gntech.me/posts/2026-05-19-headscale-self-hosted-tailscale-server/","summary":"Deploy Headscale with Docker Compose to run your own WireGuard mesh VPN control plane. Complete guide from setup to client registration, ACL configuration, subnet routing, and production hardening.","title":"Headscale — Self-Hosted Tailscale Mesh VPN Server with Docker Compose"},{"content":"If you manage a Proxmox cluster with multiple nodes, you have probably written manual bridge configs on every host. Need to add a new isolated network? SSH into node1, edit /etc/network/interfaces, apply it. Node2? Same. Node3? Same. Now keep them in sync when you add a fourth host. That is brittle, error-prone, and doesn\u0026rsquo;t scale.\nProxmox SDN (Software-Defined Networking) solves this. Introduced as a core feature in Proxmox VE 8.1, SDN lets you define virtual networks — zones, VNets, subnets — once at the datacenter level. The configuration is replicated across your cluster automatically and applied on every node. Need a new isolated network for your IoT containers? Create a VNet in the web UI, apply, and bridge it to any VM or CT on any node. No SSH required.\nThis guide covers deploying Proxmox SDN in a homelab cluster, choosing the right zone type, creating VNets with subnets and DHCP, and assigning them to virtual guests.\nWhy Bother with SDN in a Homelab? The traditional approach — Linux bridges with VLAN tagging — works fine for a single node. With a three-node cluster, it is annoying but manageable. At five nodes and a dozen networks, the overhead becomes real.\nSDN replaces per-node manual configuration with a single source of truth in the Proxmox cluster filesystem. The benefits:\nSingle point of management — create/edit/delete networks from the web UI, no CLI required on individual nodes Automatic distribution — config syncs via pmxcfs (the Proxmox cluster file system) and is applied with one click Self-service isolation — spin up a new VNet for a project or tenant without touching physical switch VLANs DHCP integration — built-in dnsmasq gives VMs IPs in their VNet subnet automatically Consistency — no drift between nodes, no forgotten bridges If you run even two Proxmox nodes with more than one isolated network, SDN saves you time. With three or more, it is a game-changer.\nPrerequisites Proxmox VE 8.1+ — SDN is installed by default on fresh installs. On upgrades from 7.x, you need the package: 1 apt update \u0026amp;\u0026amp; apt install libpve-network-perl Cluster — SDN works at the datacenter level across nodes. A single-node cluster works fine too; you still get centralized management.\nifupdown2 — Required by SDN. Included by default since Proxmox 7. Check with:\n1 dpkg -l | grep ifupdown2 If absent, install it before proceeding:\n1 apt install ifupdown2 Verify the include line — Ensure /etc/network/interfaces ends with: 1 source /etc/network/interfaces.d/* This line tells ifupdown2 to load SDN-generated configs. Add it if missing. Do this on every node.\nZone Types — Picking the Right One Proxmox SDN offers four zone types. Each uses a different transport technology:\nZone Type Transport Best For Simple Local bridge (no overlay) Single node, test/development VLAN 802.1Q tagged Existing physical VLAN infrastructure VXLAN UDP overlay (port 4789) Multi-node clusters, no switch VLAN config EVPN VXLAN + BGP routing Multi-site, inter-VNet routing For most homelabs with 2-4 nodes, VXLAN is the sweet spot. It creates overlay networks over your existing management or storage network — no physical switch VLAN provisioning needed. Each VNet gets its own VXLAN Network Identifier (VNI), giving you up to 16 million isolated networks.\nStick with Simple if you only have one Proxmox node. Use VLAN if your physical switch already handles VLANs and you want SDN to manage the bridge side. Use VXLAN for everything else.\nStep 1 — Creating a VXLAN Zone Navigate to Datacenter \u0026gt; SDN \u0026gt; Zones and click Create.\nZone ID: lab-zone Type: VXLAN Peers: 10.0.20.30, 10.0.20.31, 10.0.20.32 MTU: 1450 Nodes: all The Peers field lists the IP addresses of every cluster node on your underlay network. These are the VTEP (VXLAN Tunnel Endpoint) addresses — the IPs the nodes use to reach each other for overlay traffic. Typically your management network IPs.\nMTU 1450 is critical. VXLAN overhead is 50 bytes per packet (outer IP + UDP + VXLAN header). With a physical MTU of 1500, payload MTU drops to 1450. If your switch supports jumbo frames (MTU 9000), set Peers MTU to 8950.\nClick OK to create the zone.\nStep 2 — Creating VNets and Subnets Now create isolated virtual networks inside the zone. Navigate to Datacenter \u0026gt; SDN \u0026gt; VNets and click Create.\nVNet for Production Containers VNet ID: vnet-prod Zone: lab-zone Tag: 100 The Tag is the VXLAN VNI — a unique identifier for this overlay network. Each VNet in a VXLAN zone needs its own tag.\nAfter creating the VNet, click on it and add a subnet:\nSubnet: 10.100.1.0/24 Gateway: 10.100.1.1 SNAT: Enabled DHCP Range: 10.100.1.100 - 10.100.1.200 DNS Server: 10.0.20.1 SNAT enables masquerading traffic from this VNet to external networks (like the internet). Without it, guests can talk to each other across VXLAN but cannot reach the outside world.\nVNet for Development VNet ID: vnet-dev Zone: lab-zone Tag: 200 Subnet: 10.100.2.0/24 Gateway: 10.100.2.1 SNAT: Enabled DHCP Range: 10.100.2.100 - 10.100.2.200 VNet for IoT VNet ID: vnet-iot Zone: lab-zone Tag: 300 Subnet: 10.100.3.0/24 Gateway: 10.100.3.1 SNAT: Disabled IoT networks are intentionally left without SNAT and without DHCP — guests get static IPs and have no internet access. If you need selective internet access, enable SNAT and apply firewall rules instead.\nStep 3 — Applying the Configuration In Datacenter \u0026gt; SDN, you will see a banner saying \u0026ldquo;Pending changes\u0026rdquo;. Click the Apply button.\nSDN applies changes atomically. It reads the config from /etc/pve/sdn/, generates ifupdown2 config snippets, and pushes them to all nodes. On each node, it creates bridges:\nvnets-bridge-vnet-prod on all nodes vnets-bridge-vnet-dev on all nodes vnets-bridge-vnet-iot on all nodes Verify the bridges are up:\n1 2 # On any node bridge link show | grep vnet If you see the bridges and their VXLAN tunnel interfaces, SDN is active. You can also check:\n1 ip -d link show vnet-prod Step 4 — Assigning VNets to VMs and Containers VNets appear as regular bridges in the network device dropdown. A VM gets a virtual NIC plugged into the bridge, just like any Linux bridge network.\nVia Web UI For a VM: go to Hardware \u0026gt; Add \u0026gt; Network Device, select vnet-prod as the bridge, VirtIO model.\nFor an LXC container: go to Network \u0026gt; Add, select vnet-prod, set IPv4 to DHCP.\nVia CLI 1 2 3 4 5 # Attach VM 100 to vnet-prod qm set 100 -net0 virtio,bridge=vnet-prod # Attach CT 200 to vnet-dev with DHCP pct set 200 -net0 name=eth0,bridge=vnet-dev,ip=dhcp When the guest boots, it receives an IP from the dnsmasq DHCP server running for that subnet. Containers on different VNets cannot communicate with each other — they are fully isolated at layer 2.\nStep 5 — DHCP Integration with dnsmasq For DHCP to work, you need dnsmasq installed on every node that hosts guests on a VNet with DHCP enabled:\n1 2 apt install dnsmasq systemctl disable --now dnsmasq Proxmox SDN manages its own dnsmasq instances per subnet. The system-level dnsmasq must be disabled to avoid port conflicts.\nThe DHCP server automatically:\nAssigns IPs from the configured range Sets the gateway and DNS server you specified Registers hostnames (guest hostname → IP) for DNS resolution within the VNet You can check active leases:\n1 2 ls /var/lib/dnsmasq/dnsmasq-*.leases cat /var/lib/dnsmasq/dnsmasq-vnet-prod.leases Step 6 — Firewalling VNet Traffic SDN VNets are isolated by default — layer 2 traffic does not cross VNet boundaries. However, layer 3 traffic (routed through SNAT or through a router VM) can mix. Add Proxmox firewall rules at the datacenter or node level to control inter-VNet access.\nDatacenter Firewall Rules Navigate to Datacenter \u0026gt; Firewall \u0026gt; Rules and add:\nAction Type Source Dest Comment ACCEPT IP vnet-prod 10.100.1.0/24 Allow prod self-traffic DROP IP vnet-prod 10.100.2.0/24 Block prod→dev ACCEPT IP vnet-dev 10.100.2.0/24 Allow dev self-traffic DROP IP 10.100.3.0/24 any Block IoT→all ACCEPT IP 10.100.3.0/24 10.100.3.0/24 Allow IoT self-traffic The order matters — Proxmox firewall uses first-match semantics. Place specific allow rules before broad drops.\nEnable Firewall on VMs Guest-level firewall rules only apply when the firewall is enabled on the VM or CT. Via CLI:\n1 2 qm set 100 -firewall 1 pct set 200 -firewall 1 Then define rules in /etc/pve/firewall/\u0026lt;VMID\u0026gt;.fw or via the VM\u0026rsquo;s Firewall panel in the web UI.\nTransitioning from Manual Bridges If you are migrating existing VMs from manually created bridges to SDN VNets, the process is straightforward:\nCreate the VNet in SDN → Apply Shut down the VM Change its network device from the old bridge to the new VNet Start the VM The VM keeps its IP if you assign the same subnet. For LXC containers using DHCP, the dnsmasq lease will assign the same IP because dnsmasq persists leases to disk.\nWhen all guests are migrated, delete the old manual bridge config from /etc/network/interfaces on each node.\nTroubleshooting Common Issues Bridges not appearing after Apply Check the SDN status:\n1 2 cat /etc/pve/sdn/.running-config cat /etc/pve/sdn/.version If version is -1 (not applied) or stale, re-apply from the web UI. Also verify ifupdown2 is installed:\n1 ip a | grep vnet If the bridge exists but has no VXLAN link, check MTU. An MTU mismatch between nodes causes silent packet drops. Set all physical interfaces and the VXLAN zone to the same MTU.\nGuests cannot reach the internet Verify SNAT is enabled in the subnet configuration. Without SNAT, VNet traffic has no route to external networks. If SNAT is on, check the default gateway in the VNet matches what guests receive via DHCP.\nDHCP not assigning IPs Ensure dnsmasq is installed and the systemd service is disabled on every node hosting guests Check the dnsmasq lease file for errors Verify the DHCP range does not overlap with the gateway IP or any static assignments Confirm the VNet is applied (check .running-config) VXLAN performance is slow VXLAN adds 50 bytes per packet, which can cause fragmentation or MTU issues. Set the VXLAN zone MTU to 1450 (or 8950 with jumbo frames) and ensure guests also use the reduced MTU:\n1 2 # Inside a Linux guest ip link set eth0 mtu 1450 For Docker hosts on a VNet, set the MTU in daemon.json:\n1 2 3 { \u0026#34;mtu\u0026#34;: 1450 } SDN vs Traditional Bridges — A Comparison Aspect Traditional Bridges Proxmox SDN Config location Every node\u0026rsquo;s /etc/network/interfaces Single datacenter UI Applying changes Per-node SSH, ifreload -a One-click Apply Adding a node Copy all bridge configs Add IP to Peers, re-Apply DHCP External (Pi-hole, router) Built-in dnsmasq per subnet Network isolation Manual bridge per network VNet per network Cross-node overlay Manual VXLAN setup Automatic VXLAN per VNet Rollback Manual copy/backup Re-apply previous state When to Avoid SDN SDN is powerful but not always the right fit:\nSingle node, one or two networks — Simple bridges are less overhead and no additional service to maintain Very tight memory budget — Each dnsmasq instance uses a few MB. On a host with 8 GB or more, this is irrelevant Complex multi-site BGP routing — EVPN is still in tech preview. For multi-site routing, consider FRRouting on a VM alongside SDN Existing heavily customized networking — If you have complex OVS bridges, bonding, or policy routing, SDN may conflict. Test in a lab first Proxmox SDN transforms network management from a per-node chore into a cluster-wide operation. For a three-node homelab with production, development, and IoT workloads, it eliminates the most tedious part of Proxmox administration — keeping bridge configs in sync. Define it once, apply it everywhere, and spend your time on things that actually matter.\n","permalink":"https://blog.gntech.me/posts/2026-05-19-proxmox-sdn-software-defined-networking/","summary":"Complete guide to Proxmox SDN — configure centralized virtual networks with VXLAN overlays, VNets, subnets, and integrated DHCP across your multi-node cluster.","title":"Proxmox SDN — Centralized Virtual Networking for Homelab Clusters"},{"content":"A successful backup command does not prove an application can be recovered. Start with one file, an explicit repository path, and a restore directory you can inspect on the host. Then add application consistency and off-site recovery.\nVerification status Area Evidence and limits Technical review Official Restic references below reviewed on October 31, 2026. Static checks Bash syntax, embedded Compose YAML, and path mappings checked. Runtime Recovery lab run 41245705581 passed on October 31, 2026: initialization, backup, restore, retention preview, separately encrypted copy, wrong-password rejection, full copied-repository data check, and recovery without original materials. Tested versions GitHub-hosted Ubuntu 24.04 amd64; Docker 28.0.4; Compose 2.38.2; Restic 0.19.1. Image digest: sha256:136600b6ff6843d61d355f7f71f460a166429f35de6fd11b568fece3c9a4d510. Scope Disposable local file lab. Production databases, off-site recovery and scheduling require separate validation. 1. Separate source consistency from encryption Restic encrypts repository data and deduplicates snapshots. It does not make a live database\u0026rsquo;s changing files application-consistent.\nSource Preparation before backup Ordinary files Ensure the selected files represent the state you intend to recover. PostgreSQL or MariaDB Use a supported database dump or physical backup procedure. Application with database and attachments Coordinate both parts according to its recovery documentation. Stopped container data Confirm all writers are stopped and preserve required metadata and configuration. A read-only mount prevents that container from writing to the source; it does not freeze other writers. Dump files must actually be included in the selected backup path. Any failed dump must fail the job instead of being logged as a warning followed by a success notification.\n2. Create a disposable Compose project Use Bash, Docker Engine, the Compose plugin, and OpenSSL. Select a reviewed official Restic image tag or digest compatible with your repository. The service uses the image\u0026rsquo;s Restic entrypoint; commands after the service name are Restic arguments.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 restic_lab_dir=$(mktemp -d -t restic-file-lab.XXXXXX) cd \u0026#34;$restic_lab_dir\u0026#34; || exit 1 umask 077 mkdir source repository restored printf \u0026#39;restic-restore-check\\n\u0026#39; \u0026gt; source/check.txt openssl rand -base64 32 \u0026gt; restic-password read -r -p \u0026#39;Reviewed restic/restic image tag or digest: \u0026#39; RESTIC_IMAGE export RESTIC_IMAGE cat \u0026gt; compose.yaml \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; services: restic: image: ${RESTIC_IMAGE:?Set a reviewed restic/restic image} environment: RESTIC_REPOSITORY: /repository RESTIC_PASSWORD_FILE: /run/secrets/restic-password volumes: - ./source:/source:ro - ./repository:/repository - ./restored:/restore secrets: - restic-password secrets: restic-password: file: ./restic-password EOF docker compose config --quiet docker compose run --rm --interactive=false restic version The commands use --interactive=false so a container cannot consume input intended for a later host-side prompt when these steps run from a script. Repository passwords come from mounted files.\nThis is a one-shot service, not a sleeping container with a background cron daemon. It needs no Docker socket. The password file is created in a new private directory; do not regenerate the password when reconnecting to an existing repository.\nKeep production repository credentials and recovery instructions somewhere accessible if the Docker host is lost. See repository preparation.\n3. Initialize once and back up Run initialization only for this new empty repository. For an existing repository, use its saved password and inspect snapshots instead.\n1 2 3 4 docker compose run --rm --interactive=false restic init docker compose run --rm --interactive=false restic backup /source --host restic-file-lab --tag demo docker compose run --rm --interactive=false restic snapshots docker compose run --rm --interactive=false restic check Check each command\u0026rsquo;s exit status before continuing. A nonzero backup result requires investigation even if some snapshot data was saved.\nThe fixed host label avoids grouping every new one-shot container under a different generated hostname. Record the snapshot ID returned by backup. See backing up.\n4. Restore to a host-visible destination Select the exact demo snapshot ID from the listing:\n1 2 3 4 read -r -p \u0026#39;Demo snapshot ID to restore: \u0026#39; snapshot_id test -n \u0026#34;$snapshot_id\u0026#34; \u0026amp;\u0026amp; docker compose run --rm --interactive=false restic restore \u0026#34;$snapshot_id\u0026#34; --target /restore cmp source/check.txt restored/source/check.txt Expected result to verify: the restored marker matches the original and cmp exits zero. The snapshot\u0026rsquo;s absolute source path is recreated under the target, so container path /restore/source/check.txt corresponds to host path ./restored/source/check.txt.\nRestoring into a container\u0026rsquo;s unmounted /tmp directory would not put files in the host\u0026rsquo;s /tmp directory. Keep mappings explicit. Never test by copying a restore directly over a live database volume.\nUse Restic restore guidance for selecting files. The restore \u0026ndash;path filter selects snapshots; it does not mean “restore only this directory.” Use documented include/exclude or subtree options when needed.\n5. Review retention before deleting anything For the demo group, preview a policy:\n1 2 3 docker compose run --rm --interactive=false restic forget --dry-run \\ --host restic-file-lab --tag demo \\ --keep-daily 7 --keep-weekly 4 --keep-monthly 3 Inspect the keep/remove lists and grouping behavior before applying a policy. Forget removes snapshot references; prune reclaims eligible repository data. Do not attach automatic pruning to your first backup exercise.\nPlan maintenance windows and verify the repository afterwards. A partial data check does not read every stored pack, and a repository check is not an application restore test. See retention.\n6. Add an independent off-site repository A repository beside the source on the same host is useful for this exercise, but cannot survive loss of that host.\nChoose a supported remote backend, configure its credentials, and initialize a separate repository once. Use Restic\u0026rsquo;s supported copy workflow to transfer selected snapshots between repositories, or run a separate backup against the same prepared source. One backup invocation targets one repository.\nWith copy, explicitly configure destination access and source repository/password options. Do not assume local and remote passwords are interchangeable. Verify remote snapshots, check the remote repository, and restore from it using only the off-site credentials and recovery materials.\nSee working with repositories. Retention is independent for each repository. Do not substitute an uncoordinated file mirror of a changing repository for a tested recovery workflow.\nRehearse recovery from a separately encrypted copy This optional extension uses a second local repository to test copy and credential separation. It does not test a remote backend or survive loss of this host. Continue in the disposable lab from above.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 mkdir repository-copy recovered-copy openssl rand -base64 32 \u0026gt; copy-password cat \u0026gt; compose.copy.yaml \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; services: restic: volumes: - ./repository-copy:/repository-copy secrets: - copy-password recovery: image: ${RESTIC_IMAGE:?Set a reviewed restic/restic image} environment: RESTIC_REPOSITORY: /repository RESTIC_PASSWORD_FILE: /run/secrets/copy-password volumes: - ./repository-copy:/repository - ./recovered-copy:/restore secrets: - copy-password secrets: copy-password: file: ./copy-password EOF docker compose -f compose.yaml -f compose.copy.yaml run --rm --interactive=false restic \\ --repo /repository-copy --password-file /run/secrets/copy-password init docker compose -f compose.yaml -f compose.copy.yaml run --rm --interactive=false restic \\ --repo /repository-copy --password-file /run/secrets/copy-password \\ copy --from-repo /repository --from-password-file /run/secrets/restic-password The destination has its own password. The recovery service mounts only that repository, its password, and a fresh restore directory. It has no access to the original source, repository, or password. List and verify the copied snapshot, then select its ID:\n1 2 3 4 5 6 7 docker compose -f compose.yaml -f compose.copy.yaml run --rm --interactive=false recovery snapshots docker compose -f compose.yaml -f compose.copy.yaml run --rm --interactive=false recovery check --read-data read -r -p \u0026#39;Copied snapshot ID to restore: \u0026#39; copied_snapshot_id test -n \u0026#34;$copied_snapshot_id\u0026#34; \u0026amp;\u0026amp; docker compose -f compose.yaml -f compose.copy.yaml run --rm --interactive=false recovery \\ restore \u0026#34;$copied_snapshot_id\u0026#34; --target /restore printf \u0026#39;restic-restore-check\\n\u0026#39; | cmp - recovered-copy/source/check.txt For this small lab, check --read-data reads all stored backup data. Budget bandwidth and time before using it on a large remote repository. Matching the marker verifies this file exercise; it does not validate a database or an application\u0026rsquo;s recovery.\n7. Automate only the verified workflow Move the validated project to a stable operational path and adapt the source mounts to prepared application data. Before scheduling, require:\nDatabase dumps written to a mounted staging directory, with errors propagated and partial files excluded. A mount check for external backup storage so a missing mount cannot silently redirect writes to the root filesystem. A stable host label, explicit source selection, overlap prevention, and retained logs. Success notification only after all required backup stages succeed, plus alerts for failed or missing runs. Separate retention and integrity checks, and recurring isolated application restore tests. Test the scheduled command under its actual user and environment. Do not source an arbitrary Compose .env file as shell code or assume a container image includes Bash, Docker CLI, curl, or a scheduler.\nKeep the demo files until you have reviewed the restore result. No services remain running after these one-shot commands. Any repository deletion must be a separate, deliberate cleanup decision.\nChoose the next recovery exercise This guide demonstrates files and separately encrypted repository copies. For a complete PostgreSQL dump → Restic copy → fresh database import, use the integrated database recovery lab. Its runtime evidence covers local CI repositories, not your actual remote backend. For guest recovery, follow PBS setup; for copy planning, start with the backup strategy.\nContinue reading Proxmox backup strategy — Plan independent copies and recovery targets. Compose health checks — Understand why a running process is not backup-success evidence. ","permalink":"https://blog.gntech.me/posts/2026-05-19-restic-docker-backup-strategy/","summary":"Prove repository access and file recovery before adding databases, off-site copies, retention, and scheduling.","title":"Restic Docker Backups — Verify a Restore Before Automation"},{"content":"As your homelab grows from a handful of containers to a dozen services — Gitea, Grafana, Vaultwarden, Pi-hole admin, Portainer, Proxmox — managing separate credentials for each becomes a liability. Every service has its own login, its own password policy (or lack thereof), and its own notion of multi-factor authentication.\nAuthentik solves this. It is an open-source Identity Provider (IdP) that provides SSO, MFA, and centralized access policies. Deploy it behind Traefik, and every service in your homelab can authenticate through a single portal with TOTP or WebAuthn.\nThis guide covers deploying Authentik with Docker Compose, configuring the Traefik forward-auth middleware, creating applications and providers, setting up access policies, and onboarding your homelab services one by one.\nAuthentik vs the Alternatives Feature Authentik Authelia Keycloak Deployment complexity Moderate Low High Built-in OIDC/OAuth2 Yes Yes (OIDC only) Yes LDAP outpost Yes No Yes (external) WebAuthn (passkeys) Yes Yes Yes Admin UI Rich web UI Minimal Complex User self-service portal Yes Limited Yes Traefik forward-auth Native Native Requires extra config Resource usage ~1GB RAM ~200MB RAM ~2GB RAM License MIT Apache 2.0 Apache 2.0 When to choose Authentik: You have 8+ services, need a proper admin UI for user management, want self-service enrollment with MFA, and plan to integrate LDAP or SCIM. For 2-3 services with basic auth-only needs, Authelia is lighter.\nPrerequisites Traefik already deployed (see the Traefik reverse proxy guide) Docker and Docker Compose v2 A domain pointed at your Traefik instance (e.g., auth.example.com) PostgreSQL knowledge (Authentik requires a Postgres database) At least 2GB RAM free on the Docker host Step 1 — Docker Compose Stack Authentik consists of four components:\nServer — The core IdP, handles login, MFA, user management Worker — Background task processor (email, LDAP sync, event cleanup). Can scale horizontally. PostgreSQL — Primary database Redis — Cache, session storage, and task queue 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 # /opt/authentik/docker-compose.yml services: postgresql: image: docker.io/library/postgres:16-alpine restart: unless-stopped environment: POSTGRES_DB: authentik POSTGRES_USER: authentik POSTGRES_PASSWORD: ${PG_PASS:?Set PG_PASS in .env} volumes: - ./postgres:/var/lib/postgresql/data networks: - internal healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;pg_isready -d authentik -U authentik\u0026#34;] interval: 30s timeout: 10s retries: 5 redis: image: docker.io/library/redis:7-alpine restart: unless-stopped command: --save 60 1 --loglevel warning volumes: - ./redis:/data networks: - internal healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;redis-cli\u0026#34;, \u0026#34;ping\u0026#34;] interval: 30s timeout: 10s retries: 5 server: image: ghcr.io/goauthentik/server:latest restart: unless-stopped command: server environment: AUTHENTIK_REDIS__HOST: redis AUTHENTIK_POSTGRESQL__HOST: postgresql AUTHENTIK_POSTGRESQL__USER: authentik AUTHENTIK_POSTGRESQL__NAME: authentik AUTHENTIK_POSTGRESQL__PASSWORD: ${PG_PASS} AUTHENTIK_SECRET_KEY: ${AUTHENTIK_SECRET_KEY:?Generate a 64-char key} AUTHENTIK_BOOTSTRAP_PASSWORD: ${AUTHENTIK_BOOTSTRAP_PASSWORD:?Set admin password} AUTHENTIK_BOOTSTRAP_EMAIL: ${AUTHENTIK_BOOTSTRAP_EMAIL:-admin@example.com} volumes: - ./media:/media - ./certs:/certs - ./custom-templates:/templates ports: - \u0026#34;127.0.0.1:9000:9000\u0026#34; # Only localhost — Traefik proxies - \u0026#34;127.0.0.1:9443:9443\u0026#34; # HTTPS port (not used behind Traefik) networks: - internal - traefik depends_on: postgresql: condition: service_healthy redis: condition: service_healthy labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.authentik.rule=Host(`auth.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.authentik.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.authentik.tls=true\u0026#34; - \u0026#34;traefik.http.routers.authentik.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.services.authentik.loadbalancer.server.port=9000\u0026#34; # Outpost forward-auth endpoint - \u0026#34;traefik.http.routers.authentik-outpost.rule=Host(`auth.example.com`) \u0026amp;\u0026amp; PathPrefix(`/outpost.goauthentik.io/`)\u0026#34; - \u0026#34;traefik.http.routers.authentik-outpost.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.authentik-outpost.tls=true\u0026#34; - \u0026#34;traefik.http.routers.authentik-outpost.priority=20\u0026#34; - \u0026#34;traefik.http.services.authentik-outpost.loadbalancer.server.port=9000\u0026#34; worker: image: ghcr.io/goauthentik/server:latest restart: unless-stopped command: worker environment: AUTHENTIK_REDIS__HOST: redis AUTHENTIK_POSTGRESQL__HOST: postgresql AUTHENTIK_POSTGRESQL__USER: authentik AUTHENTIK_POSTGRESQL__NAME: authentik AUTHENTIK_POSTGRESQL__PASSWORD: ${PG_PASS} AUTHENTIK_SECRET_KEY: ${AUTHENTIK_SECRET_KEY} volumes: - ./media:/media - ./certs:/certs - ./custom-templates:/templates networks: - internal depends_on: postgresql: condition: service_healthy redis: condition: service_healthy networks: internal: driver: bridge internal: true # No external access traefik: external: true .env File 1 2 3 4 5 # /opt/authentik/.env PG_PASS=$(openssl rand -base64 32) AUTHENTIK_SECRET_KEY=$(openssl rand -base64 48) AUTHENTIK_BOOTSTRAP_PASSWORD=changeme! AUTHENTIK_BOOTSTRAP_EMAIL=admin@example.com Generate real credentials:\n1 2 3 4 5 cd /opt/authentik echo \u0026#34;PG_PASS=$(openssl rand -base64 32)\u0026#34; \u0026gt;\u0026gt; .env echo \u0026#34;AUTHENTIK_SECRET_KEY=$(openssl rand -base64 48)\u0026#34; \u0026gt;\u0026gt; .env echo \u0026#34;AUTHENTIK_BOOTSTRAP_PASSWORD=$(openssl rand -base64 16)\u0026#34; \u0026gt;\u0026gt; .env echo \u0026#34;AUTHENTIK_BOOTSTRAP_EMAIL=admin@example.com\u0026#34; \u0026gt;\u0026gt; .env Replace the email and domain before deploying.\nDeploy 1 2 3 cd /opt/authentik docker compose pull docker compose up -d Watch the logs for the first-time setup:\n1 docker compose logs -f server After about 30 seconds, Authentik is reachable at https://auth.example.com. Log in with the bootstrap credentials from your .env file.\nStep 2 — Initial Configuration After the first login, go through the initial setup wizard:\nChange the admin password — Immediately set a strong password Configure system settings — Under Admin Interface → System: Set the External host to https://auth.example.com Configure email if you need password reset flows Set the default token length (leave at 60) Create a default flow — Authentik uses \u0026ldquo;flows\u0026rdquo; for authentication, enrollment, and password reset. The default \u0026ldquo;default-authentication-flow\u0026rdquo; works for most setups. First Login Flow Navigate to https://auth.example.com — you see the Authentik login portal. Enter your admin credentials, and you\u0026rsquo;re in the admin interface.\nThe admin interface has two distinct areas:\nAdmin Interface (/if/admin/) — Full configuration, users, providers, flows, outposts User Interface (/if/user/) — Self-service portal for users to manage their own profile, MFA devices, and applications Step 3 — Configure the Traefik Forward-Auth Middleware Authentik uses an \u0026ldquo;outpost\u0026rdquo; — a lightweight component that proxies authentication requests. When deployed via Docker alongside the server, the outpost runs embedded within the server container. The forward-auth endpoint lives at:\nhttp://authentik:9000/outpost.goauthentik.io/auth/traefik Step 3a — Create a Proxy Provider in Authentik Go to Admin Interface → Applications → Providers Click Create, select Proxy Provider Configure: Name: Traefik Forward Auth Authentication flow: default-authentication-flow Authorization flow: default-provider-authorization-implicit-consent Mode: Forward Auth (Single Application) Save. Note the Client ID — you\u0026rsquo;ll need it for the provider. Step 3b — Create an Application Go to Applications → Applications Click Create: Name: Traefik Auth Slug: traefik-auth Provider: Select the provider you just created Save. Step 3c — Define the Traefik Middleware The forward-auth middleware lives in your Traefik dynamic configuration. Add it to your middlewares.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 # /opt/traefik/dynamic/middlewares.yml http: middlewares: # Existing middlewares... authentik: forwardAuth: address: http://authentik:9000/outpost.goauthentik.io/auth/traefik trustForwardHeader: true authResponseHeaders: - X-authentik-username - X-authentik-groups - X-authentik-entitlements - X-authentik-email - X-authentik-name - X-authentik-uid - X-authentik-jwt - X-authentik-meta-outpost - X-authentik-meta-provider - X-authentik-meta-app - X-authentik-meta-version Important: The address uses http://authentik:9000 — this assumes Authentik is on the same Docker network as Traefik. If Authentik is on a separate network, use the container\u0026rsquo;s IP:\n1 2 docker inspect authentik-server-1 | jq -r \u0026#39;.[0].NetworkSettings.Networks.traefik.IPAddress\u0026#39; 10.0.20.30 # Example — use the actual IP Then set address: http://10.0.20.30:9000/outpost.goauthentik.io/auth/traefik.\nStep 3d — Add the Outpost Router Bypass The forward-auth endpoint must NOT go through authentication itself (that would create a loop). The outpost router we defined in the docker-compose handles this — it routes /outpost.goauthentik.io/* directly to Authentik without the middleware:\n1 2 3 # Already in your compos as labels on the server container: - \u0026#34;traefik.http.routers.authentik-outpost.rule=Host(`auth.example.com`) \u0026amp;\u0026amp; PathPrefix(`/outpost.goauthentik.io/`)\u0026#34; - \u0026#34;traefik.http.routers.authentik-outpost.priority=20\u0026#34; This router has priority: 20 (higher than the default 0), so requests to auth.example.com/outpost.goauthentik.io/* match this route first and bypass the middleware.\nStep 4 — Protect Services with Authentik Now the fun part: add Authentik authentication to every service.\nGrafana Example 1 2 3 4 5 6 7 8 9 10 11 # /opt/grafana/docker-compose.yml services: grafana: image: grafana/grafana:latest labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.grafana.rule=Host(`grafana.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.grafana.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.grafana.tls=true\u0026#34; - \u0026#34;traefik.http.routers.grafana.middlewares=authentik@file\u0026#34; - \u0026#34;traefik.http.services.grafana.loadbalancer.server.port=3000\u0026#34; Once added, visiting https://grafana.example.com redirects to Authentik\u0026rsquo;s login portal. After successful login, you land on Grafana. Repeat for every service.\nServices That Need Exceptions Some services have endpoints that must not go through Authentik:\nPortainer — WebSocket connections for the terminal:\n1 2 3 4 5 6 labels: - \u0026#34;traefik.http.routers.portainer.rule=Host(`portainer.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.portainer.middlewares=authentik@file\u0026#34; - \u0026#34;traefik.http.routers.portainer-ws.rule=Host(`portainer.example.com`) \u0026amp;\u0026amp; PathPrefix(`/api/websocket/`)\u0026#34; - \u0026#34;traefik.http.routers.portainer-ws.middlewares=\u0026#34; - \u0026#34;traefik.http.routers.portainer-ws.priority=20\u0026#34; Grafana public dashboards:\n1 2 3 4 5 6 labels: - \u0026#34;traefik.http.routers.grafana-public.rule=Host(`grafana.example.com`) \u0026amp;\u0026amp; PathPrefix(`/public`)\u0026#34; - \u0026#34;traefik.http.routers.grafana-public.middlewares=\u0026#34; - \u0026#34;traefik.http.routers.grafana-public.priority=20\u0026#34; - \u0026#34;traefik.http.routers.grafana.rule=Host(`grafana.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.grafana.middlewares=authentik@file\u0026#34; Step 5 — Access Policies and Groups Authentik\u0026rsquo;s policy engine is how you control who gets access to what.\nCreate Groups Go to Admin Interface → Directory → Groups Create: admins — Full access to admin services users — Standard user access readonly — View-only access to monitoring Assign Users to Groups Go to Directory → Users Click a user → Groups tab → Join groups Create Bindings Between Groups and Applications Applications use \u0026ldquo;bindings\u0026rdquo; to control access. Each application can have policies that check group membership, user attributes, or time of day.\nGo to Applications → Applications → select your app Click the Policies / Bindings tab Click Create → select Group Binding Choose the group (e.g., admins) and set access mode to grant_allow The ordering matters — first-match wins. Place deny rules before grant rules.\nExpression Policies For advanced controls, use expression policies:\n1 2 3 4 5 6 # Example: Allow access only during business hours from datetime import datetime from django.utils import timezone now = timezone.localtime(timezone.now()) return 9 \u0026lt;= now.hour \u0026lt; 18 1 2 # Example: Require MFA for external access return request.user.ak_is_superuser or request.context.get(\u0026#34;is_mfa\u0026#34;, False) Add these via Admin Interface → Policies → Expression Policy, then bind them to your applications.\nStep 6 — Enabling MFA Authentik supports TOTP (Google Authenticator, Authy), WebAuthn (passkeys, YubiKey), and Duo.\nDefault MFA Enrollment Go to Flows → Default Authentication Flow Add a Stage of type authenticator_validate after the password stage: Device classes: totp, webauthn Not configured action: configure Last validation threshold: days=30 Users are prompted to set up MFA on their next login Require MFA for Specific Services Create a binding that checks for MFA:\nApplications → select your app → Policies / Bindings Create → Expression Policy: 1 2 3 4 5 6 7 # Must have at least one MFA device configured devices = request.user.authentik_events_user.get( pk=request.user.pk ).akstage_set.filter( stage_type=\u0026#34;authenticator\u0026#34; ) return devices.exists() Set mode to grant_allow and order it before other bindings MFA Without User Intervention Users enrolled once via the user portal (/if/user/ → Authenticator Devices) don\u0026rsquo;t need to re-enroll. The last_validation_threshold in the flow stage determines how often they need to re-verify their MFA token.\nStep 7 — User Self-Service Portal Authentik\u0026rsquo;s user interface at /if/user/ lets users:\nChange their password Enroll MFA devices (TOTP, WebAuthn) View their active sessions Access granted applications Update their profile Forward the user portal through Traefik (same router as the admin interface, since they live under different paths).\nCustomizing the Login Page Authentik supports custom branding:\nGo to Admin Interface → System → Branding Set: Title: Your homelab name Logo: URL to your logo (256x256 recommended) Favicon: URL to favicon Footer links: GitHub, status page, etc. For deeper customization, place custom HTML/CSS/JS in the custom-templates directory mounted in the Docker compose:\n1 mkdir -p /opt/authentik/custom-templates Templates use the Django template language. Override files in:\ncustom-templates/ ├── login/ │ └── password.html # Password field template ├── flow/ │ └── loader.html # Flow loader screen └── themes/ └── ... # CSS overrides Step 8 — LDAP Outpost (Optional) Authentik can act as an LDAP server for services that only support LDAP authentication (Proxmox, NAS devices, network appliances).\nEnable the LDAP Outpost Go to Admin Interface → Applications → Outposts Click Create: Name: LDAP Outpost Type: LDAP Integrate with embedded outpost: Yes Bind users and groups to the outpost Add LDAP Port to the Server Container Edit your docker-compose to expose LDAP ports:\n1 2 3 4 5 6 services: server: ports: - \u0026#34;127.0.0.1:9000:9000\u0026#34; - \u0026#34;127.0.0.1:3389:3389\u0026#34; # LDAP (StartTLS) - \u0026#34;127.0.0.1:6363:6363\u0026#34; # LDAPS Configure Service LDAP Binding 1 2 3 4 5 Server: auth.example.com Port: 3389 (StartTLS) or 6363 (LDAPS) Base DN: DC=ldap,DC=goauthentik,DC=io Bind DN: cn=admin,DC=ldap,DC=goauthentik,DC=io Password: [outpost token from Authentik] The bind credentials come from the Outpost settings page in the admin interface.\nStep 9 — Backup Strategy Authentik stores everything in PostgreSQL. Back up the database:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 #!/bin/bash # /opt/authentik/backup.sh BACKUP_DIR=\u0026#34;/backups/authentik\u0026#34; TIMESTAMP=$(date +%Y%m%d_%H%M%S) DB_NAME=\u0026#34;authentik\u0026#34; DB_USER=\u0026#34;authentik\u0026#34; DB_PASS=$(grep PG_PASS /opt/authentik/.env | cut -d= -f2) mkdir -p \u0026#34;$BACKUP_DIR\u0026#34; docker compose exec -T postgresql pg_dump \\ -U \u0026#34;$DB_USER\u0026#34; \u0026#34;$DB_NAME\u0026#34; \\ | gzip \u0026gt; \u0026#34;$BACKUP_DIR/authentik_$TIMESTAMP.sql.gz\u0026#34; # Keep 30 days find \u0026#34;$BACKUP_DIR\u0026#34; -name \u0026#34;*.sql.gz\u0026#34; -type f -mtime +30 -delete echo \u0026#34;Backup complete: $BACKUP_DIR/authentik_$TIMESTAMP.sql.gz\u0026#34; Also back up the .env file and the mounted media and certs directories.\nRestore Procedure 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # Stop the server and worker docker compose stop server worker # Drop and recreate (in production, create a new DB and recreate) docker compose exec -T postgresql psql -U authentik -d postgres \\ -c \u0026#34;DROP DATABASE IF EXISTS authentik;\u0026#34; docker compose exec -T postgresql psql -U authentik -d postgres \\ -c \u0026#34;CREATE DATABASE authentik;\u0026#34; # Restore gunzip -c authentik_20260519_120000.sql.gz | \\ docker compose exec -T postgresql psql -U authentik -d authentik # Restart docker compose up -d Troubleshooting \u0026ldquo;Too many redirects\u0026rdquo; when visiting a protected service The forward-auth endpoint is looping. Check:\nThe outpost router has PathPrefix(/outpost.goauthentik.io/) with priority: 20 (higher than the auth router\u0026rsquo;s default 0) The address in the forward-auth middleware points to the correct container hostname/port Authentik\u0026rsquo;s External host setting matches the domain \u0026ldquo;401 Unauthorized\u0026rdquo; after login The Token is invalid or expired. In Authentik admin:\nApplications → Providers → edit your proxy provider Re-generate the token Update the outpost if needed WebSocket connections failing Portainer terminal, Grafana live updates, and similar features use WebSockets. Authentik\u0026rsquo;s forward-auth does not support WebSocket upgrade natively. Create a bypass router:\n1 2 3 - \u0026#34;traefik.http.routers.portainer-ws.rule=Host(`portainer.example.com`) \u0026amp;\u0026amp; Headers(`Upgrade`,`websocket`)\u0026#34; - \u0026#34;traefik.http.routers.portainer-ws.middlewares=\u0026#34; - \u0026#34;traefik.http.routers.portainer-ws.priority=30\u0026#34; LDAP connection refused If you enabled the LDAP outpost and can\u0026rsquo;t connect:\nVerify the LDAP ports are exposed in docker-compose Check docker compose logs server for LDAP errors The outpost must have bindings to users/groups Lost admin password Reset it via the CLI:\n1 2 docker compose exec server \\ /bin/bash -c \u0026#34;echo \u0026#39;from authentik.core.models import User; u = User.objects.get(username=\\\u0026#34;akadmin\\\u0026#34;); u.set_password(\\\u0026#34;newpassword\\\u0026#34;); u.save()\u0026#39; | python3 manage.py shell\u0026#34; The admin username is akadmin by default.\nPutting It All Together — Reference Architecture ┌──────────────────────────────────────────────────────────┐ │ Internet / Cloudflare Tunnel │ │ ↓ │ │ Traefik → Authenticates via forward-auth middleware │ │ ↓ │ │ Authentik Outpost endpoint at auth.example.com │ │ ↓ │ │ Authentik validates → redirects to service │ │ ↓ │ │ Service receives user info via headers │ │ ↓ │ │ User lands on protected service, already authenticated │ └──────────────────────────────────────────────────────────┘ Request flow breakdown:\nUser visits https://grafana.example.com Traefik\u0026rsquo;s authentik forward-auth middleware intercepts the request Traefik sends the request to http://authentik:9000/outpost.goauthentik.io/auth/traefik Authentik sees no session cookie → redirects to auth.example.com login page User logs in with credentials + MFA Authentik sets the authentik_session cookie and redirects back to Traefik Traefik retries the forward-auth check — this time Authentik returns 200 with user info headers Traefik forwards the original request with X-authentik-* headers to Grafana Grafana receives the authenticated user info in request headers Summary Authentik transforms a collection of independently-authenticated services into a unified, secure ecosystem. The key takeaways:\nDeploy with Docker Compose using PostgreSQL and Redis — allocate at least 2GB RAM and expose only port 9000 behind Traefik Configure the Traefik forward-auth middleware with the embedded outpost endpoint — one middleware file protects every service Protect services by adding authentik@file to their Traefik router middlewares — bypass WebSocket routes with a higher-priority router rule Enforce MFA via flow stages — TOTP and WebAuthn with a last_validation_threshold of 30 days Control access with group bindings — create admins, users, and readonly groups, then bind them to applications Extend with LDAP outpost for services that don\u0026rsquo;t support OIDC/OAuth2 — Proxmox, NAS appliances, and network gear The investment in setting up Authentik pays off immediately. Instead of managing a dozen password databases and hoping users pick strong ones, you get a single identity source with MFA that every service inherits. For a homelab running 10+ web services, it\u0026rsquo;s not nice to have — it\u0026rsquo;s a security necessity.\n","permalink":"https://blog.gntech.me/posts/2026-05-19-authentik-sso-traefik-homelab/","summary":"Complete guide to deploying Authentik with Docker Compose and integrating it with Traefik as a forward-auth SSO provider for centralized authentication, MFA, and access policies across all homelab services.","title":"Authentik SSO With Traefik — Self-Hosted Identity Provider for Homelab"},{"content":"Relying on GitHub, GitLab, or Bitbucket for every project puts your infrastructure code, Docker Compose files, and Ansible playbooks on someone else\u0026rsquo;s servers. For a homelab where you control every packet, your source code should live in your own VLAN.\nGitea is a self-hosted Git server written in Go. The binary is ~100 MB with zero dependencies, and the Docker image with PostgreSQL runs comfortably on a 1 GB RAM LXC container. It supports SSH and HTTP(S) access, pull requests, issue tracking, a built-in registry, and — most importantly for this guide — Gitea Actions, a GitHub-Actions-compatible CI/CD system that runs on your own hardware.\nThis guide covers the full stack: Docker Compose deployment with PostgreSQL, SSH configuration for git operations, Traefik reverse proxy integration, backup automation, and a working CI/CD pipeline that builds and deploys Docker images inside your homelab.\nGitea Docker Compose Deployment The standard Gitea deployment uses two containers: Gitea itself and a PostgreSQL database. A third service for Redis is optional — Gitea can use file-based sessions for small deployments.\nComplete Docker Compose Configuration 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 # /opt/gitea/docker-compose.yml services: server: image: gitea/gitea:1.23 container_name: gitea environment: - USER_UID=1000 - USER_GID=1000 - GITEA__database__DB_TYPE=postgres - GITEA__database__HOST=db:5432 - GITEA__database__NAME=gitea - GITEA__database__USER=gitea - GITEA__database__PASSWD=${DB_PASSWORD} - GITEA__server__DOMAIN=git.gntech.dev - GITEA__server__HTTP_PORT=3000 - GITEA__server__ROOT_URL=https://git.gntech.dev - GITEA__server__SSH_DOMAIN=git.gntech.dev - GITEA__server__SSH_PORT=2222 - GITEA__server__SSH_LISTEN_PORT=22 - GITEA__server__DISABLE_SSH=false - GITEA__server__START_SSH_SERVER=true - GITEA__metrics__ENABLED=true volumes: - ./data:/data - /etc/timezone:/etc/timezone:ro - /etc/localtime:/etc/localtime:ro ports: - \u0026#34;2222:22\u0026#34; networks: - proxy - internal restart: unless-stopped depends_on: db: condition: service_healthy labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.gitea.rule=Host(`git.gntech.dev`)\u0026#34; - \u0026#34;traefik.http.routers.gitea.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.services.gitea.loadbalancer.server.port=3000\u0026#34; db: image: postgres:16-alpine container_name: gitea-db environment: - POSTGRES_USER=gitea - POSTGRES_PASSWORD=${DB_PASSWORD} - POSTGRES_DB=gitea volumes: - ./postgres:/var/lib/postgresql/data networks: - internal restart: unless-stopped healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;pg_isready -U gitea\u0026#34;] interval: 10s timeout: 5s retries: 5 networks: proxy: external: true internal: internal: true Environment File Create .env alongside the Compose file:\n1 2 # /opt/gitea/.env DB_PASSWORD=$(openssl rand -base64 32) Generate it once and keep it in the .env file. Never commit this to version control — yes, the irony of keeping a Git password outside Git is intentional.\nFirst Run 1 2 3 4 mkdir -p /opt/gitea/{data,postgres} cd /opt/gitea echo \u0026#39;DB_PASSWORD=your-secure-password-here\u0026#39; \u0026gt; .env docker compose up -d On first start, Gitea runs the installer at https://git.gntech.dev/install. Pre-fill the database settings from the environment variables above. Set the Server Domain and Gitea Base URL to your domain. SSH port should match the mapped port (2222 in this setup).\nAfter installation, Gitea auto-configures itself from the database settings and the admin account you create during setup.\nSSH Configuration for Git Operations Gitea runs its own SSH server inside the container on port 22. The Compose file maps host port 2222 to container port 22. Users configure their local Git client to use this port:\n1 2 # On your workstation or management machine git clone ssh://git@git.gntech.dev:2222/username/repo.git For SSH key authentication, add your public key in Gitea\u0026rsquo;s web UI under Settings → SSH/GPG Keys.\nAlternative: Host SSH Passthrough If you already run an SSH server on the Docker host and prefer not to expose another port, use SSH port forwarding instead. Add this to your ~/.ssh/config:\nHost git.gntech.dev HostName git.gntech.dev Port 2222 User git IdentityFile ~/.ssh/id_ed25519 Then clone normally:\n1 2 git clone git@git.gntech.dev:username/repo.git # Port 2222 is used automatically from ~/.ssh/config Traefik Reverse Proxy Integration The Docker labels on the Gitea service register it with Traefik automatically. Add the proxy network and the websecure entrypoint to your Traefik configuration:\n1 2 3 4 5 6 # traefik.yml — dynamic config or labels labels: - \u0026#34;traefik.http.routers.gitea.rule=Host(`git.gntech.dev`)\u0026#34; - \u0026#34;traefik.http.routers.gitea.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.gitea.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.services.gitea.loadbalancer.server.port=3000\u0026#34; Traefik terminates TLS, passes plain HTTP/1.1 to the Gitea container on port 3000, and Gitea does not need to know about certificates at all. This is the cleanest setup — Gitea\u0026rsquo;s built-in HTTPS is disabled because Traefik handles it upstream.\nBackup Strategy Your Git repositories are Gitea\u0026rsquo;s most valuable asset. Back them up daily with a script that runs inside the container:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 #!/bin/bash # /opt/gitea/backup.sh — run via cron BACKUP_DIR=\u0026#34;/opt/gitea/backups\u0026#34; RETENTION_DAYS=30 TIMESTAMP=$(date +%Y%m%d-%H%M%S) mkdir -p \u0026#34;$BACKUP_DIR\u0026#34; # Run Gitea\u0026#39;s built-in dump command docker exec gitea bash -c \u0026#34; /usr/local/bin/gitea dump \\ -c /data/gitea/conf/app.ini \\ -f /tmp/gitea-dump.zip \\ --skip-repository-archive \\ --skip-log \\ --skip-custom-dir \u0026#34; # Copy dump and database docker cp gitea:/tmp/gitea-dump.zip \u0026#34;$BACKUP_DIR/gitea-dump-$TIMESTAMP.zip\u0026#34; docker exec gitea rm /tmp/gitea-dump.zip # Rotate old backups find \u0026#34;$BACKUP_DIR\u0026#34; -name \u0026#34;gitea-dump-*.zip\u0026#34; -mtime +$RETENTION_DAYS -delete echo \u0026#34;Backup complete: gitea-dump-$TIMESTAMP.zip\u0026#34; Add it to the host\u0026rsquo;s cron:\n1 2 # /etc/cron.d/gitea-backup 0 3 * * * root /opt/gitea/backup.sh This produces a complete dump including repositories, database, and configuration — everything needed to restore Gitea on a fresh install.\nGitea Actions — Self-Hosted CI/CD Gitea Actions is the killer feature. It is compatible with GitHub Actions syntax, which means existing workflows from GitHub projects port over with minimal changes. Runners execute jobs on your infrastructure, not on a cloud provider\u0026rsquo;s bill.\nEnabling Actions In the Gitea web UI, go to Site Administration → Actions and ensure Enable Actions is checked. Then deploy a runner:\nDeploy a Docker-Based Runner 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 # /opt/gitea-runner/docker-compose.yml services: runner: image: gitea/act_runner:latest container_name: gitea-runner environment: - CONFIG_FILE=/config.yaml - GITEA_INSTANCE_URL=https://git.gntech.dev - GITEA_RUNNER_REGISTRATION_TOKEN=${RUNNER_TOKEN} - GITEA_RUNNER_NAME=homelab-runner - DOCKER_HOST=tcp://docker-socket-proxy:2375 volumes: - ./data:/data - ./config.yaml:/config.yaml:ro networks: - proxy restart: unless-stopped depends_on: - docker-socket-proxy docker-socket-proxy: image: tecnativa/docker-socket-proxy container_name: docker-socket-proxy environment: - CONTAINERS=1 - POST=1 - BUILD=1 - IMAGES=1 - INFO=1 - AUTH=1 - EXEC=1 volumes: - /var/run/docker.sock:/var/run/docker.sock:ro networks: - proxy restart: unless-stopped networks: proxy: external: true The docker-socket-proxy sidecar provides the runner with Docker access without exposing the socket directly. The runner builds images, runs containers, and executes commands — all through a restricted API proxy.\nRegistering the Runner 1 2 3 4 5 # Get the registration token from Gitea Admin → Actions → Create Runner # Then: cd /opt/gitea-runner echo \u0026#39;RUNNER_TOKEN=your-token-here\u0026#39; \u0026gt; .env docker compose up -d The runner registers itself with Gitea on startup. Check the runner status in Site Administration → Actions → Runners.\nExample Workflow — Build and Deploy a Docker Image Place this .gitea/workflows/build.yml in your repository:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 # .gitea/workflows/build.yml name: Build and Deploy on: push: branches: [main] jobs: build: runs-on: ubuntu-latest steps: - name: Checkout uses: actions/checkout@v4 - name: Set up Docker Buildx uses: docker/setup-buildx-action@v3 - name: Log in to Gitea Container Registry uses: docker/login-action@v3 with: registry: git.gntech.dev username: ${{ secrets.GITEA_USER }} password: ${{ secrets.GITEA_TOKEN }} - name: Build and push uses: docker/build-push-action@v6 with: context: . push: true tags: git.gntech.dev/${{ github.repository }}:${{ github.sha }} - name: Deploy to homelab uses: appleboy/ssh-action@v1.0.3 with: host: ${{ secrets.DEPLOY_HOST }} username: ${{ secrets.DEPLOY_USER }} key: ${{ secrets.DEPLOY_KEY }} script: | cd /opt/apps/myapp docker compose pull docker compose up -d --force-recreate Secrets Configuration Add secrets in the Gitea repository at Settings → Actions → Secrets:\nGITEA_USER — your Gitea username GITEA_TOKEN — a personal access token from Settings → Applications DEPLOY_HOST, DEPLOY_USER, DEPLOY_KEY — SSH credentials for the target Docker host Runner Labels Gitea Actions runners use labels instead of GitHub\u0026rsquo;s runs-on: ubuntu-latest. The default runner image ships with labels like ubuntu-latest:docker://node:20-bullseye. If your workflow uses runs-on: ubuntu-latest and the runner has that label, it matches automatically.\nYou can define custom labels in the runner\u0026rsquo;s config.yaml:\n1 2 3 4 5 # /opt/gitea-runner/config.yaml runner: labels: - \u0026#34;ubuntu-latest:docker://node:20-bullseye\u0026#34; - \u0026#34;docker-builder:docker://docker:27-cli\u0026#34; Performance Tuning for Low-Spec Hosts Gitea is lightweight by design, but a few optimizations help it run on constrained hardware:\n1 2 3 4 5 6 # Limit Gitea container to 512 MB RAM # Add to the Compose file under gitea service: deploy: resources: limits: memory: 512M 1 2 3 4 5 6 7 8 9 # Gitea config (/data/gitea/conf/app.ini) — disable unused features [repository] DISABLE_STARS = true DISABLE_WATCHERS = true DEFAULT_BRANCH = main [other] SHOW_FOOTER_VERSION = false SHOW_FOOTER_TEMPLATE_LOAD_TIME = false 1 2 3 4 5 # Postgres: limit shared buffers (add to db service) deploy: resources: limits: memory: 256M With these limits, Gitea + Postgres stay under 700 MB RAM total, leaving headroom for the runner.\nSecurity Considerations Running a Git server on your homelab exposes an attack surface beyond the usual web services:\nSSH hardening. Gitea\u0026rsquo;s built-in SSH server uses its own implementation. Keep it behind a non-standard port (2222) or disable it entirely and use HTTP(S) cloning only. If you need SSH, restrict it with AllowUsers git in the Docker host\u0026rsquo;s SSH config.\nRate limiting. Traefik middleware protects the HTTP endpoint:\n1 2 3 4 labels: - \u0026#34;traefik.http.middlewares.gitea-ratelimit.ratelimit.average=30\u0026#34; - \u0026#34;traefik.http.middlewares.gitea-ratelimit.ratelimit.burst=60\u0026#34; - \u0026#34;traefik.http.routers.gitea.middlewares=gitea-ratelimit\u0026#34; Container registry authentication. Gitea\u0026rsquo;s built-in container registry uses JWT tokens. Generate scoped tokens in Settings → Applications for CI/CD use. Never use your admin password in workflow files.\nDatabase isolation. The PostgreSQL service uses an internal Docker network (internal: true). No other container on the Docker host can reach it unless explicitly connected to the internal network.\nRestoring from Backup If your Gitea instance dies, recovery is a two-step process:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # 1. Start fresh Gitea + Postgres with the same docker-compose.yml cd /opt/gitea \u0026amp;\u0026amp; docker compose up -d # 2. Copy the latest backup into the container docker cp backups/gitea-dump-20260518-030000.zip gitea:/tmp/ # 3. Run restore inside the container docker exec -it gitea bash -c \u0026#34; unzip /tmp/gitea-dump-20260518-030000.zip -d /tmp/restore /usr/local/bin/gitea restore \\ -c /data/gitea/conf/app.ini \\ -f /tmp/gitea-dump-20260518-030000.zip \u0026#34; After restore, restart the container and verify your repositories are back. The dump includes all repos, issues, pull requests, and user accounts.\nGitea replaces GitHub\u0026rsquo;s proprietary offering with a fully self-hosted Git stack that runs on the same hardware you already manage. Your repositories stay inside your network, CI/CD pipelines run on your CPU cycles, and the container registry stores images without egress charges. It is one of the highest-impact services you can add to a homelab — a single LXC container replaces three external dependencies.\n","permalink":"https://blog.gntech.me/posts/2026-05-18-gitea-docker-ci-cd-homelab/","summary":"Complete guide to deploying Gitea with Docker Compose in your homelab — PostgreSQL backend, SSH passthrough, daily backups, and Gitea Actions CI/CD for automated Docker builds and deployments.","title":"Gitea Docker Deployment — Self-Hosted Git and CI/CD for Homelab"},{"content":"Traefik\u0026rsquo;s automatic TLS and Docker service discovery make it the most popular reverse proxy in the homelab space. But automatic HTTPS is only the first step. Without security headers, rate limiting, and access controls at the proxy layer, every exposed service is one misconfiguration away from leaking server info, getting hammered by a scraper, or serving content over HTTP.\nTraefik middlewares are the mechanism for applying HTTP-level policies between the edge and your upstream services. Every security header, rate-limit rule, authentication check, and URL rewrite is a discrete middleware object that you attach to a router.\nThis guide covers the middlewares every homelab should deploy: security headers for an A+ rating, rate limiting to protect against abuse, IP whitelisting for admin panels, basic auth for staging environments, forced HTTPS, compression, and the art of chaining them together. All configs work with Docker labels or dynamic YAML files.\nHow Traefik Middlewares Work A middleware intercepts the request before it reaches your application or the response before it leaves the proxy. Traefik processes middlewares in the order they are listed on the router.\nTwo ways to define middlewares:\n1. Docker labels — inline on the container:\n1 2 3 4 5 6 7 # docker-compose.yml services: myapp: image: nginx labels: - \u0026#34;traefik.http.routers.myapp.rule=Host(`app.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.myapp.middlewares=secHeaders@docker\u0026#34; The @docker provider references a middleware defined with the traefik.http.middlewares.\u0026lt;name\u0026gt; label prefix on the same or another container.\n2. Dynamic YAML file — reusable across all services:\n1 2 3 4 5 6 7 8 # /opt/traefik/dynamic/middlewares.yml http: middlewares: secHeaders: headers: browserXssFilter: true contentTypeNosniff: true frameDeny: true Reference it with @file provider:\n1 2 labels: - \u0026#34;traefik.http.routers.myapp.middlewares=secHeaders@file\u0026#34; The YAML approach is cleaner for shared middlewares — define once, use everywhere. The label approach is fine for per-service overrides.\n1. Security Headers Middleware — A+ Rating Security headers tell browsers how to behave when rendering your content. Missing headers are the most common reason for low security scores on tools like Mozilla Observatory and securityheaders.com.\nThe Complete Security Headers Middleware 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 # /opt/traefik/dynamic/middlewares.yml http: middlewares: secHeaders: headers: # XSS Protection browserXssFilter: true contentTypeNosniff: true # Clickjacking protection frameDeny: true customFrameOptionsValue: \u0026#34;SAMEORIGIN\u0026#34; # HSTS — force HTTPS for 1 year stsSeconds: 31536000 stsIncludeSubdomains: true stsPreload: true # Referrer policy referrerPolicy: \u0026#34;strict-origin-when-cross-origin\u0026#34; # Permissions policy — restrict browser features permissionsPolicy: | camera=(), microphone=(), geolocation=(), payment=(), usb=(), magnetometer=(), accelerometer=() # Content Security Policy contentSecurityPolicy: \u0026gt; default-src \u0026#39;self\u0026#39;; script-src \u0026#39;self\u0026#39; \u0026#39;unsafe-inline\u0026#39; \u0026#39;unsafe-eval\u0026#39;; style-src \u0026#39;self\u0026#39; \u0026#39;unsafe-inline\u0026#39;; img-src \u0026#39;self\u0026#39; data: https:; font-src \u0026#39;self\u0026#39; data:; object-src \u0026#39;none\u0026#39;; base-uri \u0026#39;none\u0026#39;; form-action \u0026#39;self\u0026#39;; frame-ancestors \u0026#39;self\u0026#39;; upgrade-insecure-requests; # Strip server fingerprinting headers customResponseHeaders: server: \u0026#34;\u0026#34; x-powered-by: \u0026#34;\u0026#34; What each header does:\nHeader Effect browserXssFilter Enables browser\u0026rsquo;s reflected XSS filter (X-XSS-Protection: 1; mode=block) contentTypeNosniff Prevents MIME type sniffing (X-Content-Type-Options: nosniff) frameDeny Blocks all iframe embedding (X-Frame-Options: DENY) stsSeconds: 31536000 HSTS for 1 year — browser remembers HTTPS-only stsPreload: true Eligible for browser HSTS preload lists referrerPolicy Controls Referer header sent with requests permissionsPolicy Disables unnecessary browser APIs (camera, mic, etc.) contentSecurityPolicy Whitelists allowed content sources customResponseHeaders Strips Server and X-Powered-By headers Apply to All Routers — Default Middleware Repeat this on every router label is tedious. Instead, define a default middleware per entrypoint:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 # traefik.yml static config entryPoints: web: address: \u0026#34;:80\u0026#34; http: redirections: entryPoint: to: websecure scheme: https websecure: address: \u0026#34;:443\u0026#34; http: middlewares: - secHeaders@file - rateLimit@file Every request hitting the HTTPS entrypoint automatically gets the security headers and rate limiting without per-router labels.\nCSP Tuning for Your Homelab The CSP above is restrictive — it blocks inline scripts and external resources by default. Most homelab apps need adjustments:\nFor Grafana:\n1 2 3 4 5 6 7 8 9 10 11 contentSecurityPolicy: \u0026gt; default-src \u0026#39;self\u0026#39;; script-src \u0026#39;self\u0026#39; \u0026#39;unsafe-inline\u0026#39; \u0026#39;unsafe-eval\u0026#39;; style-src \u0026#39;self\u0026#39; \u0026#39;unsafe-inline\u0026#39;; img-src \u0026#39;self\u0026#39; data: https://*.grafana.net https://*.grafana.com; font-src \u0026#39;self\u0026#39; data:; object-src \u0026#39;none\u0026#39;; base-uri \u0026#39;none\u0026#39;; form-action \u0026#39;self\u0026#39;; frame-ancestors \u0026#39;self\u0026#39;; connect-src \u0026#39;self\u0026#39; wss://your-domain.com; For Vaultwarden:\n1 2 3 4 5 6 7 8 9 10 11 contentSecurityPolicy: \u0026gt; default-src \u0026#39;self\u0026#39;; script-src \u0026#39;self\u0026#39; \u0026#39;unsafe-inline\u0026#39;; style-src \u0026#39;self\u0026#39; \u0026#39;unsafe-inline\u0026#39;; img-src \u0026#39;self\u0026#39; data:; font-src \u0026#39;self\u0026#39; data:; object-src \u0026#39;none\u0026#39;; base-uri \u0026#39;none\u0026#39;; form-action \u0026#39;self\u0026#39;; frame-ancestors \u0026#39;none\u0026#39;; upgrade-insecure-requests; For services that need WebSockets:\n1 2 3 4 5 6 7 8 9 contentSecurityPolicy: \u0026gt; default-src \u0026#39;self\u0026#39;; script-src \u0026#39;self\u0026#39; \u0026#39;unsafe-inline\u0026#39; \u0026#39;unsafe-eval\u0026#39;; style-src \u0026#39;self\u0026#39; \u0026#39;unsafe-inline\u0026#39;; img-src \u0026#39;self\u0026#39; data:; font-src \u0026#39;self\u0026#39; data:; connect-src \u0026#39;self\u0026#39; wss:; frame-ancestors \u0026#39;self\u0026#39;; upgrade-insecure-requests; Test your CSP with the browser console open — blocked resources show as console errors with the directive that blocked them.\n2. Rate Limiting Middleware — Protect Against Abuse Rate limiting prevents a single client from overwhelming your services. Traefik\u0026rsquo;s rate limiter uses a token bucket algorithm.\n1 2 3 4 5 6 7 8 9 10 11 12 13 # /opt/traefik/dynamic/middlewares.yml http: middlewares: rateLimit: rateLimit: # 100 requests per second average average: 100 # Burst up to 200 requests burst: 200 # Source extraction — use client IP sourceCriterion: ipStrategy: depth: 1 Per-service rate limits for specific use cases:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 rateLimitStrict: rateLimit: average: 20 burst: 40 sourceCriterion: requestHeader: name: \u0026#34;X-Forwarded-For\u0026#34; # Use the real client IP when behind Cloudflare rateLimitAuth: rateLimit: average: 5 burst: 10 period: 1m # 5 requests per minute, 10 burst sourceCriterion: ipStrategy: depth: 1 rateLimitApi: rateLimit: average: 30 burst: 60 sourceCriterion: requestHost: true # Rate limit per hostname Apply Rate Limits by Service Type 1 2 3 4 5 6 7 8 9 10 11 # For a public-facing service labels: - \u0026#34;traefik.http.routers.myapp.middlewares=secHeaders@file,rateLimit@file\u0026#34; # For an admin panel — strict rate limiting labels: - \u0026#34;traefik.http.routers.admin.middlewares=secHeaders@file,rateLimitAuth@file\u0026#34; # For an API endpoint labels: - \u0026#34;traefik.http.routers.api.middlewares=secHeaders@file,rateLimitApi@file\u0026#34; Rate Limit + CrowdSec (From Last Week\u0026rsquo;s Post) Combine rate limiting with CrowdSec for layered protection:\n1 2 labels: - \u0026#34;traefik.http.routers.myapp.middlewares=crowdsec-bouncer@file,rateLimit@file,secHeaders@file\u0026#34; Order matters: CrowdSec blocks malicious IPs first (at the application edge), rate limiting prevents bursts from getting through to your app, and security headers protect the browser. If CrowdSec misses a burst, the rate limiter catches it.\n3. IP Whitelisting — Admin Panels and Internal Services Never expose admin interfaces, dashboards, or management UIs to the internet. Whitelist them to your home IP or VPN subnet.\n1 2 3 4 5 6 7 8 9 10 # /opt/traefik/dynamic/middlewares.yml http: middlewares: internalOnly: ipWhiteList: sourceRange: - \u0026#34;10.0.0.0/8\u0026#34; - \u0026#34;192.168.0.0/16\u0026#34; - \u0026#34;172.16.0.0/12\u0026#34; - \u0026#34;YOUR_HOME_PUBLIC_IP/32\u0026#34; # Your static IP if you access remotely Apply to sensitive services:\n1 2 3 4 5 6 7 8 9 # Proxmox web interface via Traefik labels: - \u0026#34;traefik.http.routers.proxmox.rule=Host(`proxmox.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.proxmox.middlewares=secHeaders@file,internalOnly@file\u0026#34; # Portainer labels: - \u0026#34;traefik.http.routers.portainer.rule=Host(`portainer.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.portainer.middlewares=secHeaders@file,internalOnly@file\u0026#34; VPN-Aware Whitelisting If you use WireGuard or Tailscale for remote access, whitelist the VPN subnet:\n1 2 3 4 5 6 vpnOnly: ipWhiteList: sourceRange: - \u0026#34;10.0.0.0/8\u0026#34; - \u0026#34;100.64.0.0/10\u0026#34; # Tailscale / Tailscale IP range - \u0026#34;10.100.0.0/16\u0026#34; # WireGuard subnet When combined with Cloudflare Tunnel (which replaces the visitor IP with Cloudflare IPs), you need the X-Forwarded-For strategy:\n1 2 3 4 5 6 7 cloudflareInternal: ipWhiteList: sourceRange: - \u0026#34;10.0.0.0/8\u0026#34; - \u0026#34;192.168.0.0/16\u0026#34; ipStrategy: depth: 2 # Cloudflare adds one hop — look one deeper 4. Basic Auth Middleware — Staging and Private Services Basic authentication adds an HTTP login prompt to any service without application-level changes. Invaluable for staging environments, development dashboards, or services that lack their own auth.\n1 2 3 4 5 6 7 8 9 # /opt/traefik/dynamic/middlewares.yml http: middlewares: stagingAuth: basicAuth: realm: \u0026#34;Staging Environment\u0026#34; users: - \u0026#34;admin:$2y$10$...\u0026#34; # bcrypt hash - \u0026#34;reader:$2y$10$...\u0026#34; # read-only user Generate the bcrypt hash:\n1 2 3 4 5 6 7 8 9 # Install htpasswd if needed sudo apt install -y apache2-utils # Generate bcrypt hash (interactive — prompts for password) htpasswd -nB admin | cut -d: -f2 # Or one-liner htpasswd -nB admin 2\u0026gt;/dev/null # Enter password when prompted, copy the hash after the colon With Docker labels (inline users):\n1 2 3 4 5 6 7 services: staging: image: nginx labels: - \u0026#34;traefik.http.routers.staging.rule=Host(`staging.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.staging.middlewares=stagingAuth@docker\u0026#34; - \u0026#34;traefik.http.middlewares.stagingAuth.basicauth.users=admin:$$2y$$10$$...\u0026#34; Important: In Docker labels, escape $ as $$ — Docker Compose interprets $ as variable substitution, so a $2y$10$... hash becomes $$2y$$10$$... in the label.\nMulti-User Auth for Teams 1 2 3 4 teamAuth: basicAuth: realm: \u0026#34;Team Dashboard\u0026#34; usersFile: \u0026#34;/etc/traefik/auth/users.txt\u0026#34; 1 2 3 4 # /etc/traefik/auth/users.txt (one user per line) admin:$2y$10$abcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmno dev1:$2y$10$bcdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnop ops:$2y$10$cdefghijklmnopqrstuvwxyz1234567890abcdefghijklmnopq Mount the file:\n1 2 volumes: - /etc/traefik/auth:/etc/traefik/auth:ro 5. Redirect Scheme — Force HTTPS HTTP→HTTPS redirect should be the first middleware in every chain. Traefik handles this at the entrypoint level — cleaner than a middleware.\n1 2 3 4 5 6 7 8 9 10 # traefik.yml static config — the simplest approach entryPoints: web: address: \u0026#34;:80\u0026#34; http: redirections: entryPoint: to: websecure scheme: https permanent: true Alternative — middleware approach (for selective redirects):\n1 2 3 4 5 6 7 8 # /opt/traefik/dynamic/middlewares.yml http: middlewares: redirectHttps: redirectScheme: scheme: https port: 443 permanent: true Apply to routers that should only accept HTTPS:\n1 2 3 4 5 6 7 8 9 labels: - \u0026#34;traefik.http.routers.myapp-http.rule=Host(`app.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.myapp-http.middlewares=redirectHttps@file\u0026#34; - \u0026#34;traefik.http.routers.myapp-http.entrypoints=web\u0026#34; - \u0026#34;traefik.http.routers.myapp.rule=Host(`app.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.myapp.middlewares=secHeaders@file\u0026#34; - \u0026#34;traefik.http.routers.myapp.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.myapp.tls.certresolver=letsencrypt\u0026#34; 6. Compression Middleware — Reduce Bandwidth Gzip or Brotli compression reduces transfer size by 60-80% for text content. Traefik handles compression transparently.\n1 2 3 4 5 6 7 8 9 10 11 12 # /opt/traefik/dynamic/middlewares.yml http: middlewares: compress: compress: minContentLength: 256 # Don\u0026#39;t compress tiny responses excludedContentTypes: - \u0026#34;image/png\u0026#34; - \u0026#34;image/jpeg\u0026#34; - \u0026#34;image/webp\u0026#34; - \u0026#34;video/mp4\u0026#34; - \u0026#34;application/zip\u0026#34; Add to the default entrypoint middleware chain:\n1 2 3 4 5 6 7 8 entryPoints: websecure: address: \u0026#34;:443\u0026#34; http: middlewares: - compress@file - secHeaders@file - rateLimit@file Compression is safe for most content but should be skipped for already-compressed types (images, videos, archives). The excludedContentTypes list prevents double-compression.\n7. Custom Error Pages Middleware Replace Traefik\u0026rsquo;s generic 404/503 pages with branded or informational pages.\n1 2 3 4 5 6 7 8 9 10 # /opt/traefik/dynamic/middlewares.yml http: middlewares: errorPages: errors: status: - \u0026#34;404\u0026#34; - \u0026#34;500-599\u0026#34; service: errorPagesService query: \u0026#34;/{status}.html\u0026#34; Define the error page service:\n1 2 3 4 5 6 7 # /opt/traefik/dynamic/services.yml http: services: errorPagesService: loadBalancer: servers: - url: \u0026#34;http://host:8083\u0026#34; # Static file server Or use a simple Docker container serving custom HTML:\n1 2 3 4 5 6 7 8 9 services: error-pages: image: tarampampam/error-pages:latest environment: - TEMPLATE=app-down labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.error-pages.rule=Host(`errors.local`)\u0026#34; - \u0026#34;traefik.http.routers.error-pages.internal=true\u0026#34; 8. Chaining Middlewares — Putting It All Together The real power of Traefik middlewares is chaining. A single router can pass through multiple middlewares in sequence.\nDefault Chain for All Services 1 2 3 4 5 6 7 8 9 # traefik.yml entryPoints: websecure: address: \u0026#34;:443\u0026#34; http: middlewares: - compress@file # 1. Compress response - secHeaders@file # 2. Add security headers - rateLimit@file # 3. Rate limit traffic Every service behind this entrypoint gets all three automatically.\nPer-Service Overrides Add or override middlewares for specific services:\n1 2 3 4 5 6 7 8 9 10 11 # Public service — everything from default chain, plus nothing extra labels: - \u0026#34;traefik.http.routers.public.middlewares=\u0026#34; # Admin service — default chain + IP whitelist labels: - \u0026#34;traefik.http.routers.admin.middlewares=internalOnly@file\u0026#34; # Staging — default chain + basic auth (added before default chain) labels: - \u0026#34;traefik.http.routers.staging.middlewares=stagingAuth@file\u0026#34; How Middleware Order Works Traefik processes middlewares in the exact order listed:\n1 2 labels: - \u0026#34;traefik.http.routers.myapp.middlewares=crowdsec-bouncer@file,rateLimitAuth@file,stagingAuth@file,internalOnly@file\u0026#34; Flow:\nCrowdSec blocks known malicious IPs Rate limiter caps requests to 5/min for auth-heavy routes Basic auth challenges the user for credentials IP whitelist checks if the IP is in the allowed range Default entrypoint chain adds compression + security headers If any middleware rejects the request, subsequent middlewares never run. The 403 from the IP whitelist is returned directly.\nDocker Compose Example — Complete Service with All Middlewares 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 # /opt/services/webapp/docker-compose.yml services: webapp: image: nginx:alpine volumes: - ./html:/usr/share/nginx/html:ro labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.webapp.rule=Host(`app.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.webapp.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.webapp.tls=true\u0026#34; - \u0026#34;traefik.http.routers.webapp.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.routers.webapp.middlewares=compress@file,secHeaders@file,rateLimit@file\u0026#34; networks: - traefik admin: image: nginx:alpine volumes: - ./admin:/usr/share/nginx/html:ro labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.admin.rule=Host(`admin.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.admin.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.admin.tls=true\u0026#34; - \u0026#34;traefik.http.routers.admin.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.routers.admin.middlewares=compress@file,secHeaders@file,rateLimitAuth@file,internalOnly@file\u0026#34; networks: - traefik networks: traefik: external: true 9. Testing Your Middleware Configuration Validate middleware headers and behavior before relying on them.\nQuick Header Check with curl 1 2 3 4 5 6 7 8 9 10 11 12 # Check security headers curl -sI https://app.example.com | grep -iE \u0026#39;^(HTTP|X-|Strict|Content-Security|Referrer|Permissions)\u0026#39; # Expected output: # HTTP/2 200 # content-security-policy: default-src \u0026#39;self\u0026#39;; ... # strict-transport-security: max-age=31536000; includeSubDomains; preload # x-content-type-options: nosniff # x-frame-options: DENY # x-xss-protection: 1; mode=block # referrer-policy: strict-origin-when-cross-origin # permissions-policy: camera=(), microphone=(), ... Test Rate Limiting 1 2 3 4 5 6 # Rapid-fire requests — expect 429 after burst is exhausted for i in $(seq 1 50); do curl -s -o /dev/null -w \u0026#34;%{http_code} \u0026#34; http://localhost:8080/ done # Output: 200 200 ... 200 429 429 429 ... Test IP Whitelist 1 2 3 # From a non-whitelisted IP (use a different machine/proxy) curl -s -o /dev/null -w \u0026#34;%{http_code}\u0026#34; https://admin.example.com # Expected: 403 Test Basic Auth 1 2 3 4 5 6 7 8 9 10 11 # Without credentials curl -s -o /dev/null -w \u0026#34;%{http_code}\u0026#34; https://staging.example.com # Expected: 401 # With correct credentials curl -s -u admin:password -o /dev/null -w \u0026#34;%{http_code}\u0026#34; https://staging.example.com # Expected: 200 # With wrong password curl -s -u admin:wrong -o /dev/null -w \u0026#34;%{http_code}\u0026#34; https://staging.example.com # Expected: 401 Full Security Scan 1 2 3 4 5 6 7 8 # Install Mozilla Observatory CLI pip install observatory-cli # Run scan observatory --host app.example.com # Or check manually with their web tool: # https://developer.mozilla.org/en-US/observatory/ A properly configured setup should score A or A+ with no missing headers.\n10. Common Problems and Fixes \u0026ldquo;HSTS header missing on HTTP redirect\u0026rdquo; HSTS only applies to HTTPS responses. Traefik\u0026rsquo;s entrypoint-level redirect happens before the middleware chain. Keep stsSeconds on the websecure entrypoint only — don\u0026rsquo;t add it to web.\n\u0026ldquo;My WebSocket connection is broken\u0026rdquo; Rate limiting can interfere with WebSocket upgrade requests:\n1 2 3 4 5 # Exclude WebSocket routers from rate limiting labels: - \u0026#34;traefik.http.routers.websocket.middlewares=secHeaders@file\u0026#34; - \u0026#34;traefik.http.routers.websocket.rule=Host(`ws.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.websocket.middlewares=secHeaders@file\u0026#34; \u0026ldquo;CSP is blocking my dashboard scripts\u0026rdquo; Open the browser console. The error message tells you exactly which directive and source is blocked. Add the needed source to the appropriate CSP directive. Most homelab apps need 'unsafe-inline' and 'unsafe-eval' in script-src.\n\u0026ldquo;Rate limiting is blocking legitimate traffic\u0026rdquo; Increase average and burst values, or switch to per-client source criterion. Traffic from a reverse proxy (Cloudflare Tunnel) may all appear from one source IP — use X-Forwarded-For header extraction:\n1 2 3 4 5 6 7 rateLimit: rateLimit: average: 200 burst: 400 sourceCriterion: requestHeader: name: \u0026#34;X-Forwarded-For\u0026#34; \u0026ldquo;Basic auth password hash has $ in it\u0026rdquo; In Docker Compose labels, every $ must be doubled:\n1 2 3 4 5 # Wrong — Docker Compose eats the $: \u0026#34;traefik.http.middlewares.auth.basicauth.users=admin:$2y$10$hash...\u0026#34; # Correct — double the $: \u0026#34;traefik.http.middlewares.auth.basicauth.users=admin:$$2y$$10$$hash...\u0026#34; Putting It All Together — A Hardened Homelab Proxy A complete, hardened Traefik configuration for a homelab:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 # /opt/traefik/traefik.yml entryPoints: web: address: \u0026#34;:80\u0026#34; http: redirections: entryPoint: to: websecure scheme: https websecure: address: \u0026#34;:443\u0026#34; http: middlewares: - compress@file - secHeaders@file - rateLimit@file providers: docker: endpoint: \u0026#34;unix:///var/run/docker.sock\u0026#34; exposedByDefault: false file: directory: /etc/traefik/dynamic watch: true certificatesResolvers: letsencrypt: acme: email: admin@example.com storage: /letsencrypt/acme.json httpChallenge: entryPoint: web 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 # /opt/traefik/dynamic/middlewares.yml http: middlewares: secHeaders: headers: browserXssFilter: true contentTypeNosniff: true frameDeny: true stsSeconds: 31536000 stsIncludeSubdomains: true stsPreload: true referrerPolicy: \u0026#34;strict-origin-when-cross-origin\u0026#34; permissionsPolicy: | camera=(), microphone=(), geolocation=(), payment=(), usb=(), magnetometer=(), accelerometer=() contentSecurityPolicy: \u0026gt; default-src \u0026#39;self\u0026#39;; script-src \u0026#39;self\u0026#39; \u0026#39;unsafe-inline\u0026#39; \u0026#39;unsafe-eval\u0026#39;; style-src \u0026#39;self\u0026#39; \u0026#39;unsafe-inline\u0026#39;; img-src \u0026#39;self\u0026#39; data: https:; font-src \u0026#39;self\u0026#39; data:; object-src \u0026#39;none\u0026#39;; base-uri \u0026#39;none\u0026#39;; form-action \u0026#39;self\u0026#39;; frame-ancestors \u0026#39;self\u0026#39;; upgrade-insecure-requests; customResponseHeaders: server: \u0026#34;\u0026#34; x-powered-by: \u0026#34;\u0026#34; rateLimit: rateLimit: average: 100 burst: 200 sourceCriterion: ipStrategy: depth: 1 compress: compress: minContentLength: 256 excludedContentTypes: - image/png - image/jpeg - image/webp - video/mp4 - application/zip internalOnly: ipWhiteList: sourceRange: - \u0026#34;10.0.0.0/8\u0026#34; - \u0026#34;192.168.0.0/16\u0026#34; - \u0026#34;172.16.0.0/12\u0026#34; stagingAuth: basicAuth: realm: \u0026#34;Staging\u0026#34; users: - \u0026#34;admin:$2y$10$hash...\u0026#34; Reference these middlewares in your service labels:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 # Public service — default chain handles headers, rate limit, compression labels: - \u0026#34;traefik.http.routers.webapp.rule=Host(`app.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.webapp.tls=true\u0026#34; # Admin service — restricted by IP labels: - \u0026#34;traefik.http.routers.admin.rule=Host(`admin.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.admin.tls=true\u0026#34; - \u0026#34;traefik.http.routers.admin.middlewares=internalOnly@file\u0026#34; # Staging — requires basic auth labels: - \u0026#34;traefik.http.routers.staging.rule=Host(`staging.example.com`)\u0026#34; - \u0026#34;traefik.http.routers.staging.tls=true\u0026#34; - \u0026#34;traefik.http.routers.staging.middlewares=stagingAuth@file\u0026#34; Summary Traefik middlewares are the difference between a reverse proxy that merely forwards traffic and one that actively defends your services. The five middlewares in this guide cover the essential security layers:\nSecurity headers (secHeaders) — CSP, HSTS, XSS protection, frame denial, server header stripping. The foundation of an A+ security score.\nRate limiting (rateLimit) — Token-bucket algorithm prevents any single client from overwhelming your services. Tune the average/burst per endpoint.\nIP whitelisting (internalOnly) — Keep admin panels, Proxmox, Portainer, and dashboards accessible only from your trusted subnets and VPN ranges.\nBasic auth (stagingAuth) — Add HTTP authentication to any service without modifying the application. Essential for staging and dev environments.\nCompression (compress) — Brotli/gzip reduces bandwidth by 60-80% for text content. Exclude already-compressed types.\nChain them in the right order, apply defaults at the entrypoint level, and override per-service when needed. The result is a reverse proxy that scores A+ on security audits, protects against abuse, and keeps internal services invisible to the public internet — all without touching a single application configuration.\n","permalink":"https://blog.gntech.me/posts/2026-05-18-traefik-middleware-security-hardening/","summary":"Complete guide to Traefik middleware security hardening with real-world configs — security headers, rate limiting, IP whitelisting, basic auth, redirect schemes, and chaining middlewares for an A+ security rating.","title":"Traefik Middleware Security Hardening — Headers, Rate Limiting, and Auth"},{"content":"Docker\u0026rsquo;s default installation runs the daemon as root. Every container you start inherits that privilege model — the container runtime (containerd, runc) operates with full root capabilities on the host. A single container escape, even from a well-configured container, means the attacker has root on your machine.\nRootless mode changes this. The Docker daemon runs under an unprivileged user, and all containers use the user\u0026rsquo;s UID namespace mapping. Container process UID 0 maps to an unprivileged host UID — typically in the 100000-165536 range. Even a full break-out from the container yields only that unprivileged user\u0026rsquo;s permissions.\nThis guide covers everything you need to run Docker rootless in a homelab: installation, networking, Docker Compose, systemd integration, Proxmox LXC compatibility, and the real-world trade-offs you need to understand before switching.\nWhy Rootless Matters — The Privilege Problem A standard Docker installation creates these attack vectors:\nDaemon socket: /var/run/docker.sock owned by root:docker. Any user in the docker group has effective root access — they can mount the host filesystem, spawn privileged containers, and escape the namespace model entirely.\nContainer break-out: A kernel exploit inside a container running as root (UID 0 in the container namespace) gives the attacker root on the host because the container\u0026rsquo;s UID 0 is the host\u0026rsquo;s UID 0.\nNested containerization: Running Docker inside an LXC container on Proxmox typically requires a privileged container. If Docker runs as root inside a privileged LXC, the LXC\u0026rsquo;s root is the host\u0026rsquo;s root — one layer of isolation gone.\nRootless mode solves all three:\nThe daemon runs as a non-root user — no root-owned socket. Container UID 0 maps to an unprivileged UID on the host. You can run Docker inside an unprivileged LXC container with no special permissions. How Rootless Mode Works Rootless mode uses several Linux kernel features working together:\n┌────────────────────────────────────────┐ │ Host (root user) │ │ ┌──────────────────────────────────┐ │ │ │ dockerd (user: dockeruser) │ │ │ │ ┌────────────────────────────┐ │ │ │ │ │ rootlesskit │ │ │ │ │ │ ┌──────────────────────┐ │ │ │ │ │ │ │ containerd │ │ │ │ │ │ │ │ ┌────────────────┐ │ │ │ │ │ │ │ │ │ runc │ │ │ │ │ │ │ │ │ │ (newuidmap) │ │ │ │ │ │ │ │ │ │ UID: 100001 │ │ │ │ │ │ │ │ │ └────────────────┘ │ │ │ │ │ │ │ └──────────────────────┘ │ │ │ │ │ └────────────────────────────┘ │ │ │ └──────────────────────────────────┘ │ └────────────────────────────────────────┘ rootlesskit — Reimplements network, mount, and PID namespaces without root. It uses newuidmap/newgidmap to create user namespaces with sub-UID/GID ranges. slirp4netns or pasta — Provides network connectivity from the user namespace to the host network without root. Traffic exits through a TAP device in a separate network namespace. sub-UID/GID ranges — The Linux kernel\u0026rsquo;s user namespace support maps container UID 0 to an unprivileged host UID (e.g., 100000). /etc/subuid and /etc/subgid define these ranges. The result: every containerized process runs under an unprivileged UID on the host. ps aux inside the container shows UID 0, but ps aux on the host shows the mapped unprivileged UID.\nPrerequisites Rootless mode requires specific kernel features and tools:\nKernel Requirements 1 2 3 4 5 6 7 8 9 10 11 12 # Verify user namespace support uname -r # Need 4.18+ (5.x recommended) # Check user namespace is enabled cat /proc/sys/kernel/unprivileged_userns_clone # Should output: 1 # If missing (Ubuntu), enable it: echo \u0026#39;kernel.unprivileged_userns_clone=1\u0026#39; \\ | sudo tee /etc/sysctl.d/99-userns.conf sudo sysctl --system Required Packages 1 2 3 4 5 6 7 8 9 10 11 12 # Debian/Ubuntu sudo apt update sudo apt install -y \\ uidmap \\ dbus-user-session \\ slirp4netns \\ fuse-overlayfs # Verify sub-UID/GID ranges exist grep ^$(whoami): /etc/subuid grep ^$(whoami): /etc/subgid # Output: username:100000:65536 The uidmap package provides newuidmap and newgidmap — the setuid helpers that create user namespaces. Without them, rootless mode cannot map UIDs.\nDocker Version 1 2 3 docker --version # Need Docker 20.10+ for rootless mode # Docker 24.x and 25.x have the most stable rootless support Installation — Rootless Mode Do NOT install Docker rootless via apt. The distro package installs a rootful daemon. Use Docker\u0026rsquo;s official rootless installer instead.\nMethod 1: Clean Install (No Existing Docker) 1 2 3 4 5 6 # Remove any existing Docker packages sudo apt remove -y docker docker-engine docker.io containerd runc sudo apt autoremove -y # Download and run the rootless installer curl -fsSL https://get.docker.com/rootless | sh The installer:\nDownloads the Docker static binaries Sets up ~/.config/systemd/user/docker.service Configures environment in ~/.bashrc Starts the rootless daemon via systemd \u0026ndash;user After install, log out and back in, or source the env:\n1 2 3 source ~/.bashrc docker info --format \u0026#39;{{.SecurityOptions}}\u0026#39; # Expected: [name=rootless] Method 2: Alongside Rootful Docker If you already have rootful Docker and want both running:\n1 2 3 4 5 6 7 8 9 10 11 # Stop rootful daemon sudo systemctl stop docker.service docker.socket sudo systemctl disable docker.service docker.socket # Install rootless as described above curl -fsSL https://get.docker.com/rootless | sh # Switch between them (rootless is default after install) # To go back to rootful temporarily: sudo systemctl start docker systemctl --user stop docker Method 3: Migrate from Rootful This is the most common scenario for homelabs:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # 1. Backup Docker data sudo tar czf ~/docker-rootful-backup.tar.gz /var/lib/docker # 2. Stop and disable rootful Docker sudo systemctl stop docker.socket docker sudo systemctl disable docker.socket docker sudo rm -f /var/run/docker.sock # 3. Install rootless curl -fsSL https://get.docker.com/rootless | sh source ~/.bashrc # 4. Verify docker info --format \u0026#39;{{.SecurityOptions}}\u0026#39; docker run hello-world Post-Installation Setup and Validation Verify Rootless Status 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 # Check the daemon is running without root docker info | grep -i rootless # Security Options: name=rootless # Verify container processes run as non-root docker run --rm alpine ps aux # PID USER TIME COMMAND # 1 root 0:00 ps aux # (Inside the container it shows root, but on the host...) # On the host, find the container PID CONTAINER_ID=$(docker run -d alpine sleep 60) HOST_PID=$(docker inspect $CONTAINER_ID \\ --format \u0026#39;{{.State.Pid}}\u0026#39;) ps -o uid,pid,comm -p $HOST_PID # UID PID COMMAND # 100001 12345 sleep # UID 100001 = mapped unprivileged user, NOT root (0) Environment Setup The rootless installer adds this to ~/.bashrc. Verify it:\n1 2 3 grep docker ~/.bashrc # export PATH=/home/youruser/bin:$PATH # export DOCKER_HOST=unix:///run/user/1000/docker.sock If SSH-ing into the machine, ensure XDG_RUNTIME_DIR is set:\n1 export XDG_RUNTIME_DIR=/run/user/$(id -u) Add it to ~/.profile for SSH sessions:\n1 echo \u0026#34;export XDG_RUNTIME_DIR=/run/user/$(id -u)\u0026#34; \u0026gt;\u0026gt; ~/.profile Networking — The Biggest Difference This is where rootless mode differs most from rootful Docker. In rootless mode, containers cannot directly use host networking (--network host) or expose ports on privileged ports (\u0026lt; 1024).\nDefault Network: slirp4netns Rootless containers use slirp4netns by default, which creates a userspace TCP/IP stack:\n1 2 3 4 5 6 7 # Does NOT work in rootless mode docker run --network host nginx # Works — port mapping through slirp4netns docker run -d -p 8080:80 nginx # Container\u0026#39;s port 80 is reachable on host port 8080 # Performance: ~80% of native throughput Limitations:\nNo ICMP (no ping inside containers) No --network host No multicast or broadcast Throughput is lower than bridge networking Connection tracking is per-connection (higher latency) Pasta — The slirp4netns Replacement Docker 25+ supports pasta (from the passt project), which performs better than slirp4netns:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # Install pasta sudo apt install -y passt # Configure Docker rootless to use pasta mkdir -p ~/.config/docker cat \u0026gt; ~/.config/docker/daemon.json \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; { \u0026#34;rootless\u0026#34;: true, \u0026#34;experimental\u0026#34;: true, \u0026#34;network\u0026#34;: \u0026#34;pasta\u0026#34; } EOF # Restart rootless daemon systemctl --user restart docker Pasta provides:\nNear-native throughput (~95%) Better port forwarding performance Reduced CPU overhead Still no --network host or ICMP Benchmark comparison:\n1 2 3 4 5 # Test throughput with iperf3 (rootful vs rootless) # Rootful bridge: ~940 Mbps (near line rate) # Rootless slirp4netns: ~750 Mbps # Rootless pasta: ~890 Mbps # Rootless host ns: not available (skips this test) Exposing Ports Below 1024 By default, rootless cannot bind to ports below 1024. Three workarounds:\nA) authbind — Grant a user permission to bind low ports:\n1 2 3 4 5 6 7 sudo apt install -y authbind sudo touch /etc/authbind/byport/80 sudo touch /etc/authbind/byport/443 sudo chown $USER:$USER /etc/authbind/byport/80 sudo chown $USER:$USER /etc/authbind/byport/443 sudo chmod 755 /etc/authbind/byport/80 sudo chmod 755 /etc/authbind/byport/443 Then configure Docker to use authbind (requires patching the rootlesskit start script — not recommended).\nB) Proxy redirect — iptables REDIRECT on the host (recommended):\n1 2 3 4 5 6 # Redirect host port 80 to 8080 (where container listens) sudo iptables -t nat -A PREROUTING \\ -p tcp --dport 80 -j REDIRECT --to-port 8080 sudo iptables -t nat -A PREROUTING \\ -p tcp --dport 443 -j REDIRECT --to-port 8443 Add to a systemd oneshot service for persistence:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 # /etc/systemd/system/port-redirect.service [Unit] Description=Redirect privileged ports to unprivileged After=network.target [Service] Type=oneshot ExecStart=/usr/sbin/iptables -t nat -A PREROUTING \\ -p tcp --dport 80 -j REDIRECT --to-port 8080 ExecStart=/usr/sbin/iptables -t nat -A PREROUTING \\ -p tcp --dport 443 -j REDIRECT --to-port 8443 RemainAfterExit=true ExecStop=/usr/sbin/iptables -t nat -D PREROUTING \\ -p tcp --dport 80 -j REDIRECT --to-port 8080 ExecStop=/usr/sbin/iptables -t nat -D PREROUTING \\ -p tcp --dport 443 -j REDIRECT --to-port 8443 [Install] WantedBy=multi-user.target 1 2 sudo systemctl daemon-reload sudo systemctl enable --now port-redirect.service C) Rootful reverse proxy to a rootless service:\nA reverse proxy on the rootful daemon can publish ports 80 and 443 and reach a separately exposed rootless application through the host gateway. This bridges two daemon contexts and requires an explicit upstream bind address:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 cat \u0026gt; Caddyfile \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; example.com { reverse_proxy host.docker.internal:8080 } EOF docker run -d \\ --name caddy \\ --add-host=host.docker.internal:host-gateway \\ -p 80:80 \\ -p 443:443 \\ -v \u0026#34;$PWD/Caddyfile:/etc/caddy/Caddyfile:ro\u0026#34; \\ -v caddy_data:/data \\ -v caddy_config:/config \\ caddy:latest This Caddy container uses the rootful daemon\u0026rsquo;s normal default capabilities; --cap-add alone would not reduce that set. The upstream must be reachable through the host gateway address. A rootless service bound only to host loopback is not reachable through this path; choose its bind address and firewall rules deliberately. Inside a container, localhost refers to that container, not the host. Replace example.com with your domain and verify both daemons\u0026rsquo; selected contexts before running the example.\nOption B (iptables redirect) is the simplest for homelabs and does not require any additional containers.\nDocker Compose with Rootless Docker Compose works out of the box with rootless mode. No special configuration needed.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 # Install Compose plugin for rootless # (typically already included with the Docker binaries) docker compose version # Docker Compose version v2.32.0 # Use it exactly as you would with rootful mkdir -p ~/services/nginx cat \u0026gt; ~/services/nginx/compose.yaml \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; services: web: image: nginx:alpine ports: - \u0026#34;8080:80\u0026#34; volumes: - ./html:/usr/share/nginx/html:ro EOF docker compose -f ~/services/nginx/compose.yaml up -d Compose Rootful vs Rootless Differences Feature Rootful Rootless Port bindings \u0026lt; 1024 Yes No (use redirect) network_mode: host Yes No volumes with host paths Full access User-owned paths only privileged: true Yes Ignored (no effect) cap_add All capabilities Limited to user-ns-safe caps Volume Mounts — User Permission Model Rootless Docker can only mount directories the user owns or has permission to read. If your compose file references /var/log or /etc/nginx, those mounts will fail:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # This works — files in the user\u0026#39;s home services: app: image: nginx volumes: - ~/nginx/html:/usr/share/nginx/html # This fails — root-owned path services: app: image: nginx volumes: - /etc/nginx:/etc/nginx # Permission denied Fix by copying files into your home directory or creating a symlink farm with bind mounts from a rootful service.\nSystemd Integration — User Services Rootless Docker runs as a systemd user service. This is critical for homelab uptime — it must survive reboots and SSH session closures.\nEnable linger for the Docker user 1 2 3 4 5 6 # Keeps the user\u0026#39;s systemd instance running even after logout sudo loginctl enable-linger $USER # Verify loginctl show-user $USER | grep Linger # Linger=yes Verify the docker service starts at boot 1 2 3 4 5 6 7 8 systemctl --user status docker # ● docker.service - Docker Application Container Engine (Rootless) # Loaded: loaded ... # Active: active (running) # Enable autostart (should already be enabled by installer) systemctl --user enable docker systemctl --user start docker Running Custom Containers as Systemd User Services Each container can run as its own systemd unit with automatic restart:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 # Create a service template mkdir -p ~/.config/systemd/user cat \u0026gt; ~/.config/systemd/user/container-nginx.service \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; [Unit] Description=Nginx container (rootless) After=docker.service Requires=docker.service [Service] ExecStartPre=-/usr/bin/docker rm -f nginx-rootless ExecStart=/usr/bin/docker run --rm --name nginx-rootless \\ -p 8080:80 \\ -v %h/nginx/html:/usr/share/nginx/html:ro \\ nginx:alpine ExecStop=/usr/bin/docker stop nginx-rootless Restart=always RestartSec=10 [Install] WantedBy=default.target EOF systemctl --user daemon-reload systemctl --user enable --now container-nginx Running Rootless Docker Inside Proxmox LXC This is where rootless mode shines for homelabs. Instead of creating a full VM for Docker (which wastes RAM on a kernel and init), you can run rootless Docker inside an unprivileged LXC container.\nLXC Container Configuration Create the container on Proxmox with these settings:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # Create unprivileged container pct create 300 local:vztmpl/ubuntu-24.04-standard_24.04-2_amd64.tar.zst \\ --unprivileged 1 \\ --nesting 1 \\ --features keyctl=1,nesting=1 \\ --storage local-zfs \\ --cores 4 \\ --memory 8192 \\ --swap 0 \\ --net0 name=eth0,bridge=vmbr0,ip=dhcp pct start 300 pct enter 300 Inside the LXC — Install Rootless Docker 1 2 3 4 # Inside the LXC apt update apt install -y curl uidmap slirp4netns fuse-overlayfs curl -fsSL https://get.docker.com/rootless | sh Important: Sub-UID/GID Ranges Inside LXC Unprivileged LXC containers have their own UID mapping. The container itself is mapped to a sub-UID range on the Proxmox host (e.g., UID 100000 on the host = UID 0 inside the container). Nested Docker rootless maps AGAIN from that range.\nThis means a container process has these UID translations:\nDocker container UID 0 → LXC container UID 100000 (via LXC\u0026#39;s subuid mapping) → Proxmox host UID 101000 (typical nested mapping) This triple-nesting is safe and works reliably when:\nThe LXC container is unprivileged /etc/subuid and /etc/subgid inside the LXC have enough range (65536 minimum) kernel.keys.maxkeys is set high enough inside the LXC 1 2 3 4 5 6 7 8 9 10 # Inside the LXC — verify subuid range grep ^$(whoami): /etc/subuid # username:100000:65536 # Increase kernel key limits inside the LXC cat \u0026gt;\u0026gt; /etc/sysctl.d/99-docker-rootless.conf \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; kernel.keys.maxkeys=2000 kernel.keys.maxbytes=250000 EOF sysctl --system Monitoring and Maintenance Logging Rootless Docker logs go to the user\u0026rsquo;s systemd journal:\n1 2 3 4 5 # View daemon logs journalctl --user -u docker -f # View a specific container logs docker logs -f container_name Updates Update the rootless binaries when Docker releases a new version:\n1 2 3 4 5 6 # Re-run the installer — it updates in place curl -fsSL https://get.docker.com/rootless | sh # Or update via the static binaries directly docker update --help | grep update # (There is no built-in updater — re-run the installer) Backup The rootless Docker data directory is at ~/.local/share/docker/:\n1 2 3 4 5 6 7 # Backup volumes and container data tar czf ~/docker-rootless-backup-$(date +%F).tar.gz \\ ~/.local/share/docker/ # This is your equivalent of /var/lib/docker/ # Restore: # tar xzf backup.tar.gz -C ~/.local/share/docker/ Common Issues and Fixes \u0026ldquo;Failed to connect to bus\u0026rdquo; or \u0026ldquo;XDG_RUNTIME_DIR not set\u0026rdquo; 1 2 export XDG_RUNTIME_DIR=/run/user/$(id -u) export DBUS_SESSION_BUS_ADDRESS=unix:path=$XDG_RUNTIME_DIR/bus Add both to ~/.profile for persistence.\n\u0026ldquo;newuidmap: write to uid_map failed: Operation not permitted\u0026rdquo; The sub-UID/GID ranges are missing or misconfigured:\n1 2 3 4 5 6 7 # Check ranges cat /etc/subuid cat /etc/subgid # Fix (requires root) sudo usermod --add-subuids 100000-165535 $USER sudo usermod --add-subgids 100000-165535 $USER \u0026ldquo;overlayfs: missing upperdir\u0026rdquo; or overlay not supported Rootless Docker prefers fuse-overlayfs when the kernel\u0026rsquo;s overlay in user namespace is not available:\n1 2 3 4 5 6 7 8 9 10 11 # Install fuse-overlayfs sudo apt install -y fuse-overlayfs # Configure Docker to use it mkdir -p ~/.config/docker cat \u0026gt; ~/.config/docker/daemon.json \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; { \u0026#34;storage-driver\u0026#34;: \u0026#34;fuse-overlayfs\u0026#34; } EOF systemctl --user restart docker Container Cannot Write to Mounted Host Directory The container UID inside the user namespace (e.g., UID 100001 on host) must own the host directory:\n1 2 3 4 5 # Find the mapped host UID for container user # In most rootless setups, container UID 0 = host UID 100000 # Fix — change directory ownership to the mapped UID sudo chown -R 100000:100000 ~/service-data/ Or use podman-style subuid mount options (Docker 25+):\n1 2 3 4 5 6 # compose.yaml — use :U flag for automatic chown services: app: image: nginx volumes: - ./data:/data:U Trade-offs — When Not to Use Rootless Rootless mode is not always the right choice:\nScenario Recommendation Single-user homelab, no untrusted services Rootless is fine, adds defense-in-depth Running Docker in an LXC on Proxmox Rootless strongly recommended — avoids privileged LXC Need --network host for performance Rootless cannot do this. Use rootful or use pasta Need ICMP/ping inside containers Rootless cannot provide this Services must bind port 80/443 directly Use iptables redirect or reverse proxy GPU passthrough to containers Rootless adds complexity with /dev/dri access Full production with strict SLAs Evaluate carefully — rootless has higher overhead For most homelab use cases — where you run a reverse proxy on one port and web services on high ports behind it — the trade-offs are minimal. The security benefit of removing root from the container runtime is worth the networking compromises.\nSummary Docker rootless mode eliminates the largest security risk of containerization — the root-privileged daemon and the docker group backdoor. The installation is a single command:\n1 curl -fsSL https://get.docker.com/rootless | sh Key takeaways:\nSecurity: Daemon and containers run under an unprivileged UID. Container break-outs yield a non-root user, not host root. Networking: Use pasta for near-native throughput. Redirect privileged ports with iptables. Accept the loss of --network host and ICMP. Proxmox: Rootless Docker inside an unprivileged LXC is the most resource-efficient and secure Docker hosting pattern on Proxmox. Systemd: Enable linger, verify the user service autostarts, and optionally wrap critical containers as systemd user services. Rootless mode converts Docker from \u0026ldquo;root with extra steps\u0026rdquo; to a genuinely sandboxed container runtime. For a homelab where security matters — and it should — the transition takes ten minutes and pays dividends in reduced attack surface.\n","permalink":"https://blog.gntech.me/posts/2026-05-18-docker-rootless-mode/","summary":"Complete guide to running Docker in rootless mode — no root daemon, no privileged containers. Covers installation, networking workarounds, Docker Compose, Proxmox LXC compatibility, and production readiness.","title":"Docker Rootless Mode — Secure Container Runtime for Homelabs"},{"content":"Passing a GPU to a Proxmox VM sounds intimidating — IOMMU groups, VFIO binding, driver blacklists, PCI IDs, and the dreaded NVIDIA Error 43. But the actual process is a small, well-defined set of kernel-level changes on the host plus a few VM config tweaks. Once they click into place, the GPU appears inside the VM with bare-metal performance.\nThis guide covers NVIDIA GPU passthrough on Proxmox VE 8.x, from BIOS settings through automated VM configuration. Whether you are offloading transcoding to a Plex LXC, running local LLMs in a GPU VM, or building a remote gaming workstation, the steps are the same.\nHardware Requirements and IOMMU Compatibility Before touching any configs, verify your hardware supports passthrough:\nCPU: Intel with VT-d or AMD with AMD-Vi (virtually all modern desktop CPUs have this) Motherboard: BIOS option to enable VT-d / IOMMU and 4G decoding / Resizable BAR GPU: A second GPU for the Proxmox host (onboard Intel graphics is ideal) — the host needs a display output you are not trying to pass through Proxmox: VE 8.x or later, running kernel 6.8+ Run this quick check before proceeding:\n1 2 3 4 5 # Check if kernel has IOMMU support compiled in dmesg | grep -e DMAR -e IOMMU -e AMD-Vi # Check for interrupt remapping (required for passthrough) dmesg | grep \u0026#39;remapping\u0026#39; If dmesg | grep IOMMU returns nothing, IOMMU is not enabled at the kernel level yet. That is the first step.\nStep 1: Enable IOMMU in BIOS and GRUB BIOS Settings Reboot your server and enter the BIOS setup. Enable:\nSetting Intel AMD VT-d / IOMMU Overclocking → CPU Features → Intel VT-d Tech Advanced → IOMMU Above 4G Decoding PCI Subsystem Settings Always enable Resizable BAR DMI/PCI Configuration Sets correct MMIO window Secure Boot Disable Causes driver issues with GPU GRUB Command Line SSH into the Proxmox host and edit /etc/default/grub:\n1 nano /etc/default/grub Find the GRUB_CMDLINE_LINUX_DEFAULT line. For Intel systems:\nGRUB_CMDLINE_LINUX_DEFAULT=\u0026#34;quiet intel_iommu=on iommu=pt\u0026#34; For AMD systems:\nGRUB_CMDLINE_LINUX_DEFAULT=\u0026#34;quiet amd_iommu=on iommu=pt\u0026#34; The iommu=pt flag tells the kernel to skip IOMMU mapping for devices that are not being passed through. It reduces overhead and is safe to include regardless of vendor.\nApply the change and reboot:\n1 2 update-grub reboot After reboot, verify IOMMU is live:\n1 2 3 4 5 6 7 8 dmesg | grep -e DMAR -e IOMMU # Expected output (Intel): # DMAR: IOMMU enabled # DMAR: Intel(R) Virtualization Technology for Directed I/O dmesg | grep \u0026#39;remapping\u0026#39; # Expected: DMAR-IR: Enabled IRQ remapping or # AMD-Vi: Interrupt remapping enabled If interrupt remapping is not enabled (older hardware), you can work around it, though this reduces isolation:\n1 2 echo \u0026#34;options vfio_iommu_type1 allow_unsafe_interrupts=1\u0026#34; \\ \u0026gt; /etc/modprobe.d/iommu_unsafe_interrupts.conf Step 2: Inspect IOMMU Groups Your GPU must be in its own IOMMU group. Devices in the same group must all be passed through together, which usually means passing both the VGA controller and the audio device as a pair.\nRun this to dump every group:\n1 2 3 4 5 for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*} printf \u0026#39;IOMMU Group %s \u0026#39; \u0026#34;$n\u0026#34; lspci -nns \u0026#34;${d##*/}\u0026#34; done Look for your GPU in its own group:\nIOMMU Group 13 01:00.0 VGA compatible controller [0300]: NVIDIA ... [10de:2206] IOMMU Group 13 01:00.1 Audio device [0403]: NVIDIA ... [10de:1aef] If the GPU shares a group with unrelated devices (SATA controller, USB controller, NVMe), passthrough is still possible but you must pass the entire group — which includes those unrelated controllers. The workaround is pcie_acs_override (detailed in the troubleshooting section), but it weakens IOMMU isolation. For a homelab that is usually acceptable; for a production environment, consider a different PCIe slot.\nStep 3: Bind the GPU to VFIO at Boot Find your GPU\u0026rsquo;s PCI vendor and device IDs:\n1 2 3 4 lspci -nn | grep -iE \u0026#34;nvidia|amd|vga|3d|audio\u0026#34; # Example output: # 01:00.0 VGA compatible controller [0300]: NVIDIA ... [10de:2206] # 01:00.1 Audio device [0403]: NVIDIA ... [10de:1aef] Copy both IDs (10de:2206,10de:1aef) — you need them for VFIO binding.\n3.1 Blacklist Host GPU Drivers The host must not load nvidia or nouveau for the passthrough GPU. Blacklist both:\n1 2 3 4 cat \u0026gt;\u0026gt; /etc/modprobe.d/blacklist.conf \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; blacklist nouveau blacklist nvidia EOF 3.2 Load VFIO Kernel Modules at Boot 1 2 3 4 5 6 cat \u0026gt;\u0026gt; /etc/modules \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; vfio vfio_iommu_type1 vfio_pci vfio_virqfd EOF 3.3 Bind the GPU PCI IDs to VFIO Create the VFIO options file with your GPU\u0026rsquo;s IDs:\n1 2 3 cat \u0026gt; /etc/modprobe.d/vfio.conf \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; options vfio-pci ids=10de:2206,10de:1aef disable_vga=1 EOF Replace 10de:2206,10de:1aef with your actual IDs. The disable_vga=1 prevents the VGA arbitration subsystem from interfering with the GPU.\n3.4 Apply and Verify 1 2 update-initramfs -u -k all reboot After reboot, confirm VFIO has claimed the GPU:\n1 2 lspci -nnk -d 10de:2206 # Expected: Kernel driver in use: vfio-pci If you see nvidia or nouveau instead, the blacklist did not take effect. Check that your blacklist file has no typos and that update-initramfs ran successfully.\nStep 4: Create the Target VM These settings are not optional for GPU passthrough. Deviating from them is the most common source of failures.\nVM Creation Checklist Setting Value Why Machine q35 Required for PCIe device assignment BIOS OVMF (UEFI) Legacy BIOS cannot handle GPU ROM EFI Disk Add it Required for OVMF; 64 MB is enough CPU Type host Exposes host CPU features to the guest Memory Fixed amount, disable ballooning Ballooning can conflict with GPU DMA You can set these from the CLI and avoid the web UI entirely:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 VM_ID=200 VM_NAME=\u0026#34;gpu-workstation\u0026#34; qm create $VM_ID \\ --name $VM_NAME \\ --machine q35 \\ --bios ovmf \\ --efidisk0 local-zfs:1,format=qcow2,efitype=4m \\ --scsihw virtio-scsi-single \\ --cpu host \\ --memory 16384 \\ --balloon 0 \\ --net0 virtio,bridge=vmbr0 \\ --boot order=scsi0 \\ --ostype l26 # Attach a disk qm set $VM_ID --scsi0 local-zfs:64,discard=on,ssd=1,iothread=1 # Attach VirtIO drivers ISO (for Windows guests) # qm set $VM_ID --ide2 local:iso/virtio-win.iso,media=cdrom Install the OS inside the VM normally before attaching the GPU. This way you can install NVIDIA drivers without the GPU being active yet.\nStep 5: Attach the GPU to the VM After the OS is installed and drivers are ready, shut down the VM and attach the PCI device:\n1 2 qm set $VM_ID \\ --hostpci0 01:00,pcie=1,x-vga=on,rombar=1 Breaking down the flags:\n01:00 — PCI address of the GPU (vga + audio passed as pair per IOMMU group) pcie=1 — Exposes the device on a PCIe bus inside the VM instead of PCI. Required for full bandwidth x-vga=on — Declares this GPU as the VM\u0026rsquo;s primary display rombar=1 — Exposes the GPU\u0026rsquo;s Option ROM to the guest Also set vga: none so the VM does not create a virtual VGA display that conflicts:\n1 qm set $VM_ID --vga none Start the VM and verify the GPU is visible inside the guest:\n1 2 3 4 # Inside the VM lspci | grep -i nvidia # 01:00.0 VGA compatible controller: NVIDIA ... # 01:00.1 Audio device: NVIDIA ... Step 6: NVIDIA Driver Installation and Error 43 Modern Drivers (465+) NVIDIA drivers starting at version 465 no longer check the virtualization flag by default. If you are on driver 550 or newer, skip Error 43 workarounds — just install the driver normally:\n1 2 3 4 5 6 7 # Inside the VM (Ubuntu/Debian) wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb dpkg -i cuda-keyring_1.1-1_all.deb apt update apt install -y cuda-toolkit-12-8 nvidia-driver-570 reboot nvidia-smi Legacy or Stubborn Drivers — Hide the Hypervisor If nvidia-smi shows \u0026ldquo;No devices were found\u0026rdquo; or Error 43, the driver is detecting KVM and refusing to load. Hide it:\nAdd these lines to /etc/pve/qemu-server/$VM_ID.conf:\ncpu: host,hidden=1 args: -cpu \u0026#39;host,hv_vendor_id=NV43FIX,kvm=off\u0026#39; hidden=1 — QEMU hides the KVM signature from CPUID leaves hv_vendor_id=NV43FIX — Spoofs the Hyper-V vendor ID string (any 12-character value works) kvm=off — Clears the KVM feature bit so the driver sees a physical environment After editing, reboot the VM. Verify with:\n1 2 3 # Inside the VM — should show nothing cat /proc/cpuinfo | grep hypervisor # No output means the hypervisor is hidden Windows VM — Ignore MSRs Windows NVIDIA drivers sometimes issue unhandled Model-Specific Register (MSR) accesses that crash the VM. Prevent this on the host:\n1 2 3 echo \u0026#34;options kvm ignore_msrs=1 report_ignored_msrs=0\u0026#34; \\ \u0026gt; /etc/modprobe.d/kvm.conf update-initramfs -u -k all Step 7: Tesla P40 / Datacenter Cards for AI The NVIDIA Tesla P40 is the sweet spot for homelab AI workloads. It has 24 GB of GDDR5 VRAM, costs €100–150 used, and has no display output — it is a pure compute card.\nP40-Specific Configuration 1 2 3 4 5 6 7 # Create the VM with extra-large MMIO window for 24 GB VRAM qm create $VM_ID \\ --machine q35 \\ --bios ovmf \\ --cpu host \\ --memory 32768 \\ --balloon 0 The P40\u0026rsquo;s 24 GB VRAM requires a larger MMIO window than OVMF\u0026rsquo;s default 32 GB allocation. Using cpu: host (as above) resolves this automatically because OVMF adjusts the window based on host properties. If you must use a different CPU type:\n1 qm set $VM_ID --cpu x86-64-v2-AES,phys-bits=host,flags=+pdpe1gb Attach the P40 without x-vga=on since datacenter cards have no display output:\n1 2 qm set $VM_ID --hostpci0 01:00,pcie=1,rombar=1 qm set $VM_ID --vga serial0 Install the CUDA toolkit inside the VM:\n1 2 3 4 5 6 7 8 nvidia-smi # +-----------------------------------------------------------------------------+ # | NVIDIA-SMI 570.86.15 Driver Version: 570.86.15 CUDA Version: 12.8 | # |-------------------------------+----------------------+----------------------+ # | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | # | Tesla P40 Off | 00000000:01:00.0 Off | 0 | # | Fan N/A Temp 42C P0 68W / 250W | 24576MiB / 24576MiB | 0% | # +-------------------------------+----------------------+----------------------+ The P40 has passive cooling (no fans). In a desktop case, point a 120 mm fan directly at the card\u0026rsquo;s heatsink. In a server chassis, the chassis fans must provide enough airflow — monitor temperatures with nvidia-smi -l 5 during load.\nStep 8: GPU Reset Fix with vendor-reset Some NVIDIA cards (especially consumer models) cannot complete a PCIe reset after the VM shuts down. The GPU goes into a hung state and the host requires a full power cycle to recover.\nInstall the vendor-reset kernel module to fix this:\n1 2 3 4 5 6 7 8 9 10 11 apt update apt install -y pve-headers build-essential git git clone https://github.com/gnif/vendor-reset.git /usr/src/vendor-reset cd /usr/src/vendor-reset make make install # Load at boot echo vendor-reset \u0026gt;\u0026gt; /etc/modules update-initramfs -u -k all reboot After reboot, verify:\n1 2 lsmod | grep vendor_reset # vendor_reset 32768 0 With vendor-reset loaded, you can stop and start the GPU VM without needing a full server reboot. This is essential for homelabs running multiple GPU workloads that cycle VMs daily.\nAutomated Passthrough Setup Script Save this as a one-shot script for future GPU installations:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 #!/bin/bash # gpu-passthrough.sh — Bind GPU to VFIO and configure host # Usage: ./gpu-passthrough.sh \u0026lt;PCI_ID:H:M.F\u0026gt; \u0026lt;VENDOR:DEVICE_ID\u0026gt; PCI_ADDR=\u0026#34;$1\u0026#34; # e.g., \u0026#34;01:00.0\u0026#34; IDS=\u0026#34;$2\u0026#34; # e.g., \u0026#34;10de:2206,10de:1aef\u0026#34; if [ -z \u0026#34;$PCI_ADDR\u0026#34; ] || [ -z \u0026#34;$IDS\u0026#34; ]; then echo \u0026#34;Usage: $0 \u0026#34;\u0026lt;PCI_ADDR\u0026gt;\u0026#34; \u0026lt;VFIO_IDS\u0026gt;\u0026#34; echo \u0026#34;Example: $0 01:00.0 10de:2206,10de:1aef\u0026#34; exit 1 fi # Blacklist NVIDIA drivers cat \u0026gt; /etc/modprobe.d/blacklist-nvidia.conf \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; blacklist nouveau blacklist nvidia EOF # Load VFIO modules at boot grep -q vfio /etc/modules || cat \u0026gt;\u0026gt; /etc/modules \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; vfio vfio_iommu_type1 vfio_pci vfio_virqfd EOF # Bind GPU to VFIO echo \u0026#34;options vfio-pci ids=$IDS disable_vga=1\u0026#34; \u0026gt; /etc/modprobe.d/vfio.conf # Enable IOMMU in GRUB if not already set if ! grep -q \u0026#39;intel_iommu=on\\|amd_iommu=on\u0026#39; /etc/default/grub; then sed -i \u0026#39;s/GRUB_CMDLINE_LINUX_DEFAULT=\u0026#34;/GRUB_CMDLINE_LINUX_DEFAULT=\u0026#34;intel_iommu=on iommu=pt /\u0026#39; /etc/default/grub update-grub fi update-initramfs -u -k all echo \u0026#34;Reboot required. Run: lspci -nnk -s $PCI_ADDR to verify vfio-pci driver\u0026#34; Step 9: Performance Tuning Enable Hugepages Hugepages reduce TLB pressure when the GPU does DMA to VM memory:\n1 2 3 4 5 6 7 # Allocate 8 GB of 2 MB hugepages (4096 pages) echo 4096 \u0026gt; /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages # Make persistent cat \u0026gt;\u0026gt; /etc/sysctl.d/99-hugepages.conf \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; vm.nr_hugepages=4096 EOF Pin CPU Cores Pin physical CPU cores to the VM to avoid NUMA migration overhead:\n1 2 3 4 5 6 qm set $VM_ID \\ --numa 1 \\ --cpulimit 8 \\ --sockets 1 \\ --cores 8 \\ --cpupin 0-7 Use NUMA If the Host Has Multiple Sockets 1 qm set $VM_ID --numa 1 --numa0 cpus=0-7,memory=16384 This tells the VM to map its NUMA node 0 to host NUMA node 0, matching memory and CPU topology with the physical architecture. GPU DMA to remote NUMA memory is 2-3× slower.\nTroubleshooting Common Failures GPU Not Visible Inside VM 1 2 3 4 5 6 7 8 9 # On host — confirm VFIO binding lspci -nnk -s 01:00.0 | grep \u0026#34;Kernel driver\u0026#34; # Must show: vfio-pci # On host — check IOMMU group isolation ls -la /sys/kernel/iommu_groups/*/devices/ | grep 01:00 # On VM — check if PCI device is enumerated lspci | grep -i nvidia Most likely causes:\nMachine type is i440fx instead of q35 BIOS is Seabios instead of OVMF GPU is still claimed by nvidia or nouveau on the host Black Screen After VM Boot Possible fixes: - qm set $VM_ID --vga none - Remove any --vga std or --vga virtio from VM config - Ensure x-vga=on is set on the hostpci device - Verify PCI address is correct NVIDIA Error 43 This means the driver detected it is running in a VM and refused to load. Apply the hypervisor hiding steps from Step 6, then retry.\nGPU Hangs After VM Stop / Reset Bug Install vendor-reset (Step 8). If it still hangs:\nCheck that vendor-reset is listed in lsmod Try echo 1 \u0026gt; /sys/bus/pci/devices/0000:01:00.0/remove then echo 1 \u0026gt; /sys/bus/pci/rescan on the host to re-detect As a last resort, the GPU\u0026rsquo;s PCIe slot may need a physical power cycle (cold boot) Bad IOMMU Groups — ACS Override If your GPU shares a group with unrelated devices, the nuclear option is PCIe ACS override. It reduces isolation but works:\nAdd to GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub:\npcie_acs_override=downstream,multifunction Then update-grub \u0026amp;\u0026amp; reboot. Only use this when you cannot physically move the GPU to another slot that isolates properly.\nSummary GPU passthrough on Proxmox is a well-trodden path with predictable steps:\nBIOS — Enable VT-d/AMD-Vi, Above 4G Decoding, disable Secure Boot GRUB — intel_iommu=on (or amd_iommu=on) + iommu=pt VFIO — Blacklist host GPU drivers, bind GPU PCI IDs to vfio-pci VM — Machine q35, BIOS OVMF, CPU host, disabled ballooning Attach — hostpci0 with pcie=1,x-vga=on and vga: none Drivers — NVIDIA 550+ works out of the box; older drivers need hypervisor hiding Reset — Install vendor-reset to avoid hard-locking the GPU on VM stop Once configured, the VM sees the GPU as a bare-metal device. AI inference, hardware transcoding, CUDA workloads, and even gaming VMs all benefit from near-native performance without dedicating an entire machine to each workload.\n","permalink":"https://blog.gntech.me/posts/2026-05-18-proxmox-gpu-passthrough-guide/","summary":"Complete guide to GPU passthrough on Proxmox VE — from IOMMU enablement and VFIO binding through NVIDIA Error 43 fixes and the vendor-reset kernel module.","title":"Proxmox GPU Passthrough — Complete Setup Guide for VMs"},{"content":"Docker images in the wild easily balloon past 1 GB. A python:3.12 base with your app and a few pip packages weighs over a gigabyte. The same Go binary compiled from scratch fits in 6 MB. The difference is not the code — it is everything the compiler left behind.\nMulti-stage builds let you use a full build environment with compilers, dev headers, and debug tools in one stage, then copy only the compiled artifact to a minimal runtime stage. The build tools vanish from the final image. Your homelab disk, registry storage, and deployment bandwidth all benefit from the size reduction.\nThis guide covers multi-stage build patterns for the three languages you are most likely to use in homelab automation — Go, Python, and Node.js — with real Dockerfiles, cache strategies, and security hardening you can apply today.\nWhy Multi-Stage Builds Matter in a Homelab A single fat image might not seem like a problem when you have a terabyte of storage. But consider the compounding effect:\nRegistry storage: Five services, each rebuilt weekly with a 1 GB image means 5 GB of layers retained per version Backup bandwidth: Offsite backups of Docker volumes include the image cache unless you exclude /var/lib/docker Pull time: A 50 MB image pulls in under 2 seconds on a gigabit link versus 30+ seconds for a 1 GB image Attack surface: Every package in the base image is a potential CVE. Distroless or scratch images have near-zero CVEs The pattern is always the same — build in one stage, copy artifacts to another:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # Stage 1: Build FROM golang:1.23 AS builder WORKDIR /src COPY go.mod go.sum ./ RUN go mod download COPY . . RUN CGO_ENABLED=0 GOOS=linux go build -o /app/my-service ./cmd # Stage 2: Run FROM alpine:3.21 RUN apk add --no-cache ca-certificates tzdata COPY --from=builder /app/my-service /usr/local/bin/ ENTRYPOINT [\u0026#34;/usr/local/bin/my-service\u0026#34;] The final image is golang:1.23 (800 MB) replaced by alpine:3.21 (7 MB) with just the binary. That is the core idea. The rest is optimization and language-specific nuance.\nGo: From Scratch for the Smallest Images Go is the ideal candidate for multi-stage builds because it compiles to a statically-linked binary with zero runtime dependencies. You can use the ultimate minimal base — FROM scratch — which is an empty filesystem.\nMinimal Go Dockerfile 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # Build stage FROM golang:1.23-alpine AS builder RUN apk add --no-cache git ca-certificates WORKDIR /build COPY go.mod go.sum ./ RUN go mod download COPY . . # Build with full static linking RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \\ go build -ldflags=\u0026#34;-s -w\u0026#34; -o /app/server ./cmd/server # Runtime stage — literally nothing but the binary FROM scratch COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ COPY --from=builder /app/server /server ENTRYPOINT [\u0026#34;/server\u0026#34;] The -ldflags=\u0026quot;-s -w\u0026quot; strips debug symbols and DWARF tables, shaving another 30-40% off the binary size. Combined with FROM scratch, your final image is the binary plus CA certs — typically 6-15 MB.\nHealth Check and Config Files If your Go service needs configuration files, copy them explicitly:\n1 2 3 4 5 6 7 8 9 FROM scratch COPY --from=builder /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ COPY --from=builder /app/server /server COPY config.yaml /etc/server/config.yaml COPY static/ /static/ EXPOSE 8080 HEALTHCHECK --interval=30s --timeout=3s \\ CMD [\u0026#34;/server\u0026#34;, \u0026#34;health\u0026#34;] ENTRYPOINT [\u0026#34;/server\u0026#34;] Build and Verify 1 2 3 4 5 6 7 8 9 10 11 # Build the image docker build -t my-go-service:latest . # Check final size docker images my-go-service:latest # REPOSITORY TAG IMAGE ID CREATED SIZE # my-go-service latest abc123def456 10 seconds ago 14.2 MB # Compare with a single-stage build docker build -t my-go-service:fat -f Dockerfile.single-stage . # 14.2 MB vs ~950 MB Go Build Cache Optimization Docker build cache is per-layer. Reordering your Dockerfile to cache dependency downloads saves minutes per rebuild:\n1 2 3 4 5 6 7 8 9 10 FROM golang:1.23-alpine AS builder WORKDIR /build # Cache layer 1: download dependencies (only changes when go.mod changes) COPY go.mod go.sum ./ RUN go mod download # Cache layer 2: copy and build source (rebuilds on every code change) COPY . . RUN CGO_ENABLED=0 go build -ldflags=\u0026#34;-s -w\u0026#34; -o /app/server . This separation means adding a line of code does NOT re-download all modules. The go mod download layer is cached and reused.\nPython: Slimming the 1.2 GB Behemoth Python does not compile to a standalone binary. Your application requires the Python interpreter plus all pip dependencies at runtime. But multi-stage builds still help dramatically by separating the build dependencies — compilers, dev headers, build tools — from the runtime.\nNaive Single-Stage (What Not To Do) 1 2 3 4 5 6 FROM python:3.12-slim WORKDIR /app COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt COPY . . CMD [\u0026#34;python\u0026#34;, \u0026#34;app.py\u0026#34;] This works but includes pip, setuptools, wheel, and whatever packages your requirements pulled in. Size: ~350-500 MB for a small app.\nMulti-Stage Python with Virtual Environment 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 # Build stage — install dependencies FROM python:3.12-slim AS builder WORKDIR /app RUN apt-get update \u0026amp;\u0026amp; apt-get install -y --no-install-recommends \\ gcc \\ python3-dev \\ \u0026amp;\u0026amp; rm -rf /var/lib/apt/lists/* COPY requirements.txt . RUN pip install --user --no-cache-dir -r requirements.txt # Runtime stage — clean base + copied venv FROM python:3.12-slim WORKDIR /app # Copy installed user packages from builder COPY --from=builder /root/.local /root/.local COPY . . # Ensure local bin is in PATH ENV PATH=/root/.local/bin:$PATH CMD [\u0026#34;python\u0026#34;, \u0026#34;app.py\u0026#34;] The build tools (gcc, python3-dev) are only in the builder stage. The runtime stage keeps only the slim Python image plus the pre-installed packages. For a typical Flask or FastAPI app, this drops size from ~500 MB to ~180 MB.\nPython with Compiled Extensions Packages like numpy, pandas, cryptography, or psycopg2 compile C extensions during installation. Without build tools in the builder stage, they fail.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 FROM python:3.12-slim AS builder WORKDIR /app RUN apt-get update \u0026amp;\u0026amp; apt-get install -y --no-install-recommends \\ gcc \\ python3-dev \\ libpq-dev \\ libffi-dev \\ \u0026amp;\u0026amp; rm -rf /var/lib/apt/lists/* COPY requirements.txt . # --no-build-isolation avoids pip pulling extra compilers RUN pip install --user --no-cache-dir --no-build-isolation -r requirements.txt # Runtime FROM python:3.12-slim WORKDIR /app # Runtime libraries only — no compilers RUN apt-get update \u0026amp;\u0026amp; apt-get install -y --no-install-recommends \\ libpq5 \\ libffi8 \\ \u0026amp;\u0026amp; rm -rf /var/lib/apt/lists/* COPY --from=builder /root/.local /root/.local COPY . . ENV PATH=/root/.local/bin:$PATH CMD [\u0026#34;python\u0026#34;, \u0026#34;app.py\u0026#34;] The runtime stage only has shared libraries (libpq5, libffi8), not the dev headers. The compilers and build tools are confined to the builder stage and never reach the final image.\nAlpine vs Slim Base for Python 1 2 3 4 5 6 7 8 # Compare these two runtime stages: # Option A: python:3.12-slim (Debian-based, ~120 MB base) FROM python:3.12-slim # Option B: python:3.12-alpine (~50 MB base) FROM python:3.12-alpine RUN apk add --no-cache libpq Alpine images are smaller but use musl instead of glibc. If your C extensions are compiled against glibc (most are), Alpine will either fail to import them or produce subtle runtime bugs. For pure Python packages without C extensions, Alpine is fine. For anything with native code, stick with slim.\nNode.js: Size from a Different Angle Node.js images follow a similar pattern but have a gotcha: node_modules is the bulk of the image, and npm/yarn install is the slowest build step.\nMulti-Stage Node.js Dockerfile 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 # Build stage FROM node:22-alpine AS builder WORKDIR /build # Cache npm dependencies COPY package.json package-lock.json ./ RUN npm ci --only=production # Install dev dependencies for build step RUN npm ci # Build application (TypeScript, React, etc.) COPY tsconfig.json ./ COPY src/ src/ RUN npm run build # Prune dev dependencies from node_modules RUN npm prune --production \u0026amp;\u0026amp; \\ rm -rf src/ tsconfig.json # Runtime stage FROM node:22-alpine WORKDIR /app RUN addgroup -S appgroup \u0026amp;\u0026amp; adduser -S appuser -G appgroup # Copy only what is needed at runtime COPY --from=builder /build/node_modules ./node_modules COPY --from=builder /build/dist ./dist COPY package.json ./ USER appuser EXPOSE 3000 CMD [\u0026#34;node\u0026#34;, \u0026#34;dist/index.js\u0026#34;] Key optimizations:\nnpm ci instead of npm install — Faster, deterministic, respects lockfile exactly, and fails if package-lock.json is out of sync with package.json npm prune --production — Removes devDependencies after build Separate package.json copy — Dependencies layer is cached until package.json or package-lock.json changes Comparison Technique Image Size Notes Single-stage node:22 ~1.2 GB Full image with dev deps Single-stage node:22-alpine ~350 MB Smaller but still has dev deps Multi-stage (above) ~150-200 MB Production deps only Distroless Node.js ~120-150 MB Google distroless base Distroless Base Images Google\u0026rsquo;s distroless images are another runtime option. They contain only the language runtime and its shared libraries — no shell, no package manager, no utilities.\n1 2 3 4 5 6 7 8 9 10 11 # Go on distroless FROM golang:1.23-alpine AS builder WORKDIR /build COPY go.mod go.sum ./ RUN go mod download COPY . . RUN CGO_ENABLED=0 go build -ldflags=\u0026#34;-s -w\u0026#34; -o /app/server . FROM gcr.io/distroless/static-debian12:nonroot COPY --from=builder /app/server /server ENTRYPOINT [\u0026#34;/server\u0026#34;] For Python distroless builds, match the runtime\u0026rsquo;s Python version, ABI, architecture, library dependencies, and import paths. Copying /root/.local from an arbitrary python:3.12-slim builder into a Debian 12 Python runtime can select a different interpreter ABI and an inaccessible or undiscovered package directory. That example is removed rather than presented as a working migration.\nUse the distroless project\u0026rsquo;s language examples and build/test the actual application against pinned compatible images. A package cache optimization does not prove binary compatibility.\nWhen to use distroless:\nYou need the smallest possible attack surface Your container is scanned by security tooling (Trivy, Grype) You do not need shell access to running containers When to avoid distroless:\nYou need to docker exec into the container and debug with shell tools (no sh, curl, ping available) Your application relies on OS utilities that are not in the image You run health checks that depend on shell commands For homelab use, Alpine or slim bases strike a better balance. They are small enough yet retain basic debugging tools. Swap to distroless only when you are comfortable debugging blind.\nGeneral Multi-Stage Optimization Strategies Layer Ordering for Cache Hits Docker builds each layer sequentially and caches results. To maximize cache reuse, order your Dockerfile from least-changing to most-changing:\n1 2 3 4 5 6 7 8 9 10 11 # 1. OS packages (changes rarely) FROM alpine:3.21 AS base RUN apk add --no-cache ca-certificates tzdata # 2. Dependency manifests (changes when you add/remove packages) COPY go.mod go.sum ./ RUN go mod download # 3. Source code (changes on every edit) COPY . . RUN go build -o /app/server . This ordering means a single-line code change does NOT invalidate the OS package layer or the dependency download layer. Builds that would take 3 minutes recompile in 20 seconds.\nUsing Build Arguments Build args let you parameterize the Dockerfile without hardcoding:\n1 2 3 4 5 6 ARG GO_VERSION=1.23 FROM golang:${GO_VERSION}-alpine AS builder ARG TARGETARCH RUN CGO_ENABLED=0 GOOS=linux GOARCH=${TARGETARCH} \\ go build -ldflags=\u0026#34;-s -w\u0026#34; -o /app/server . Build with platform-specific images:\n1 2 3 4 5 6 # Build for multiple architectures docker buildx build \\ --build-arg GO_VERSION=1.23 \\ --platform linux/amd64,linux/arm64 \\ -t my-service:latest \\ --push . Multi-Stage for Monorepos If your homelab runs multiple Go services in one repository, use a single Dockerfile with argument-based stage selection:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 ARG SERVICE=api FROM golang:1.23-alpine AS base WORKDIR /build COPY go.mod go.sum ./ RUN go mod download COPY . . FROM base AS service-api RUN go build -o /app/service ./cmd/api FROM base AS service-worker RUN go build -o /app/service ./cmd/worker FROM base AS service-webhook RUN go build -o /app/service ./cmd/webhook FROM service-${SERVICE} AS final FROM alpine:3.21 COPY --from=final /app/service /usr/local/bin/service ENTRYPOINT [\u0026#34;/usr/local/bin/service\u0026#34;] Build specific services:\n1 2 docker build --build-arg SERVICE=api -t my-api:latest . docker build --build-arg SERVICE=worker -t my-worker:latest . The go mod download layer is shared across all services, and each build only runs one compile step.\nSecurity Hardening Through Multi-Stage Beyond size reduction, multi-stage builds improve security in three ways:\n1. No Build Tools in Production Compilers, headers, and package managers are the biggest source of CVEs in container images. By confining them to the builder stage, the runtime image has less software to patch.\n1 2 3 4 5 6 # Scan a fat image vs a multi-stage image docker scout quickview my-app:fat # 25 vulnerabilities (3 critical, 8 high) docker scout quickview my-app:multi-stage # 2 vulnerabilities (0 critical, 0 high) 2. Non-Root User by Default Build the non-root user into the runtime stage:\n1 2 3 4 5 FROM alpine:3.21 RUN addgroup -S appgroup \u0026amp;\u0026amp; adduser -S appuser -G appgroup COPY --from=builder /app/server /server USER appuser ENTRYPOINT [\u0026#34;/server\u0026#34;] 3. Read-Only Root Filesystem Combine with Docker\u0026rsquo;s --read-only flag:\n1 docker run --read-only --tmpfs /tmp my-app:latest The binary needs nowhere to write in the root filesystem. If an attacker compromises the process, they cannot drop files on the filesystem.\nDatabase Sidecar Pattern with Alpine Multi-stage is not just for application images. Database initializers, migration runners, and cron containers benefit too:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # PostgreSQL migration runner FROM golang:1.23-alpine AS builder WORKDIR /build COPY go.mod go.sum ./ RUN go mod download COPY . . RUN CGO_ENABLED=0 go build -o /app/migrate ./cmd/migrate FROM alpine:3.21 RUN apk add --no-cache postgresql-client COPY --from=builder /app/migrate /usr/local/bin/migrate COPY migrations/ /migrations/ ENTRYPOINT [\u0026#34;/usr/local/bin/migrate\u0026#34;] This runs as a docker-compose depends_on init container that applies database migrations, then exits. The image stays under 20 MB.\nMeasuring Your Results Before-and-after metrics keep you honest:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # After building your optimized image docker images my-app:optimized # Show image layers and their sizes docker history my-app:optimized # Scan for CVEs docker scout quickview my-app:optimized # Compare against single-stage echo \u0026#34;Size reduction:\u0026#34; echo \u0026#34;Before: $(docker images my-app:fat --format \u0026#39;{{.Size}}\u0026#39;)\u0026#34; echo \u0026#34;After: $(docker images my-app:optimized --format \u0026#39;{{.Size}}\u0026#39;)\u0026#34; Run Trivy for a deeper audit:\n1 trivy image my-app:optimized A well-optimized homelab Docker image should:\nBe 80-95% smaller than its single-stage equivalent Have zero critical CVEs (achievable with scratch/distroless) Build in under 30 seconds from cache on a code change Summary Multi-stage builds are the single most impactful Dockerfile optimization you can make. The pattern is language-agnostic:\nBuild stage — Full toolchain, compile or install everything Runtime stage — Minimal base, copy only the artifacts For Go services, FROM scratch with static binaries produces images under 15 MB. For Python and Node.js, separating build dependencies from runtime reduces size by 60-80% while also reducing the CVE count.\nCombine multi-stage with proper cache layering (dependencies before source), non-root users, and read-only filesystems for containers that are small, fast to deploy, and hardened against exploitation.\n","permalink":"https://blog.gntech.me/posts/2026-05-18-docker-multistage-builds/","summary":"Master Docker multi-stage builds to shrink production images from 1.2GB to under 50MB. Covers Go, Python, and Node.js patterns with real Dockerfiles and build cache strategies.","title":"Docker Multi-Stage Builds — Practical Guide for Homelab"},{"content":"The Linux kernel ships with conservative defaults designed to work reliably on a Raspberry Pi and a 128-core server alike. Those defaults leave significant performance on the table — especially for homelab servers running Docker, Proxmox, or network-heavy workloads.\nThis guide covers the kernel parameters that make the biggest difference for homelab workloads, with production-grade configuration files you can drop in and deploy immediately.\nBefore You Tune: Establish a Baseline You cannot improve what you do not measure. Capture your current state before making any changes:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # Current sysctl values (all network-related) sysctl -a | grep -E \u0026#39;net\\.(core|ipv4|ipv6)\u0026#39; | sort \u0026gt; baseline-net.txt # TCP connection statistics ss -s \u0026gt; baseline-ss.txt # Network interface counters ip -s link show eth0 \u0026gt; baseline-eth0.txt # Current TCP congestion control sysctl net.ipv4.tcp_congestion_control # Simple throughput test (run on two machines) # Receiver: iperf3 -s # Sender: iperf3 -c \u0026#34;\u0026lt;RECEIVER_IP\u0026gt;\u0026#34; -t 30 -P 4 Save these outputs somewhere you can reference later. \u0026ldquo;Feels faster\u0026rdquo; is not a performance improvement — hard numbers are.\nNetwork Stack Tuning: The Biggest Wins The network stack is where kernel tuning has the most dramatic impact for most homelab servers. These changes affect everything from Docker container networking to NFS transfers and reverse proxy throughput.\nTCP Buffer Sizes TCP socket buffers are the single most impactful setting for high-throughput workloads. The default maximum is often 4 MB — far too small for modern gigabit links with real-world latency.\nCreate /etc/sysctl.d/90-network-performance.conf:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # Maximum socket receive/send buffer size # 128 MB for 10 Gbps links, 64 MB for 1 Gbps net.core.rmem_max = 67108864 net.core.wmem_max = 67108864 # TCP receive buffer: min, default, max (bytes) net.ipv4.tcp_rmem = 4096 131072 67108864 # TCP send buffer: min, default, max (bytes) net.ipv4.tcp_wmem = 4096 65536 67108864 # Allow auto-tuning of receive buffers net.ipv4.tcp_moderate_rcvbuf = 1 On a 1 Gbps link with 10 ms RTT, the bandwidth-delay product is about 1.25 MB. With 64 MB max buffers, you have headroom for burst traffic and concurrent connections without bottlenecking.\nTCP BBR Congestion Control BBR (Bottleneck Bandwidth and Round-trip propagation time) is Google\u0026rsquo;s model-based congestion control algorithm. It significantly outperforms CUBIC on links with packet loss — which describes most real-world internet connections.\n1 2 3 4 5 6 7 8 # Switch to BBR congestion control net.ipv4.tcp_congestion_control = bbr # BBR requires fair queueing for pacing net.core.default_qdisc = fq # Disable slow start after idle (keep pipe full) net.ipv4.tcp_slow_start_after_idle = 0 Verify it is active:\n1 2 3 4 5 sysctl net.ipv4.tcp_congestion_control # Output: net.ipv4.tcp_congestion_control = bbr # Check which connections use BBR ss -ti | head -20 On a 1 Gbps link with 1% packet loss, BBR delivers roughly 890 Mbps versus ~520 Mbps for CUBIC. For homelab servers with Docker registries, media streaming, or remote backups, this is a free throughput upgrade.\nConnection Handling for High-Concurrency Workloads If you run a reverse proxy (Traefik, Nginx), database server, or any service handling many concurrent connections, the default backlog and TIME_WAIT settings will drop connections before you saturate the CPU.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # Increase the accept queue length net.core.somaxconn = 65535 net.ipv4.tcp_max_syn_backlog = 65535 # Increase the device input packet backlog net.core.netdev_max_backlog = 250000 # Reuse TIME_WAIT sockets for new connections (essential for proxies) net.ipv4.tcp_tw_reuse = 1 # Reduce TIME_WAIT duration (default: 60 seconds) net.ipv4.tcp_fin_timeout = 15 # Raise the TIME_WAIT bucket limit net.ipv4.tcp_max_tw_buckets = 2000000 Warning: Do not set tcp_fin_timeout below 15. On lossy networks, values under 15 can cause data corruption. Also, tcp_tw_recycle was removed in Linux 4.12 — it never worked safely behind NAT.\nExpand the Ephemeral Port Range Every outbound TCP connection from your server uses a local ephemeral port. The default range of 32768–60999 gives you about 28,000 ports. For a Docker host running 50 containers with outbound connections, or a reverse proxy making many upstream requests, this exhausts quickly.\n1 2 # Maximize the ephemeral port range net.ipv4.ip_local_port_range = 1024 65535 Monitor exhaustion:\n1 2 3 4 # Check TIME-WAIT counts every few minutes watch -n 30 \u0026#39;ss -s | grep -i timewait\u0026#39; # If TIME-WAIT consistently exceeds 30,000, you are close to exhaustion TCP Keepalive for Persistent Connections Databases, WebSockets, and long-lived TCP connections need keepalive to detect dead peers. The default takes over two hours to time out a dead connection.\n1 2 3 4 5 6 7 8 # Start keepalive after 60 seconds of idle (default: 7200) net.ipv4.tcp_keepalive_time = 60 # Probe interval: 10 seconds net.ipv4.tcp_keepalive_intvl = 10 # Dead connection after 6 failed probes net.ipv4.tcp_keepalive_probes = 6 Dead connection detection drops from 2 hours to 2 minutes. This is especially important with Docker overlay networks and VPN tunnels.\nComplete Network Configuration File Save all of the above as /etc/sysctl.d/90-network-performance.conf:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 # /etc/sysctl.d/90-network-performance.conf # Apply: sudo sysctl -p /etc/sysctl.d/90-network-performance.conf # TCP buffers net.core.rmem_max = 67108864 net.core.wmem_max = 67108864 net.ipv4.tcp_rmem = 4096 131072 67108864 net.ipv4.tcp_wmem = 4096 65536 67108864 net.ipv4.tcp_moderate_rcvbuf = 1 # BBR congestion control net.ipv4.tcp_congestion_control = bbr net.core.default_qdisc = fq net.ipv4.tcp_slow_start_after_idle = 0 # Connection handling net.core.somaxconn = 65535 net.ipv4.tcp_max_syn_backlog = 65535 net.core.netdev_max_backlog = 250000 net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_fin_timeout = 15 net.ipv4.tcp_max_tw_buckets = 2000000 # Ephemeral ports net.ipv4.ip_local_port_range = 1024 65535 # Keepalive net.ipv4.tcp_keepalive_time = 60 net.ipv4.tcp_keepalive_intvl = 10 net.ipv4.tcp_keepalive_probes = 6 # Network memory net.ipv4.tcp_mem = 786432 1048576 16777216 net.ipv4.tcp_fastopen = 3 Apply it:\n1 sudo sysctl -p /etc/sysctl.d/90-network-performance.conf NIC Hardware Tuning Software tuning helps, but the NIC itself has tuning knobs too.\nRing Buffer Sizes NIC ring buffers are hardware queues between the network and kernel. If the ring fills up, packets get dropped.\n1 2 3 4 5 6 7 8 # Check current ring buffer sizes ethtool -g eth0 # Increase for better burst handling sudo ethtool -G eth0 rx 4096 tx 4096 # Monitor drops ethtool -S eth0 | grep -E \u0026#39;(drop|error|miss)\u0026#39; | grep -v \u0026#39;: 0\u0026#39; Make the change permanent by adding it to a systemd oneshot unit or /etc/rc.local.\nReceive Packet Steering (RPS) RPS distributes packet processing across CPU cores. Without it, a single interrupt line can pin one core at 100% while others sit idle.\n1 2 3 4 5 6 7 # For a 4-core system, enable RPS on all RX queues for i in /sys/class/net/eth0/queues/rx-*/rps_cpus; do echo \u0026#34;f\u0026#34; | sudo tee \u0026#34;$i\u0026#34; done # Value is a CPU bitmask in hex: # f = CPUs 0-3, ff = CPUs 0-7, ffff = CPUs 0-15 Verify irqbalance 1 2 3 4 5 # irqbalance should distribute NIC interrupts across CPUs sudo systemctl status irqbalance # Check interrupt distribution cat /proc/interrupts | grep eth0 Memory Management Tuning vm.swappiness Controls how aggressively the kernel swaps application memory versus filesystem cache. The default of 60 means the kernel starts swapping well before it needs to. For homelab servers with sufficient RAM, lower this to reduce swap pressure on Docker containers and VMs.\n1 2 3 4 # Create /etc/sysctl.d/90-memory-performance.conf echo \u0026#34;vm.swappiness = 10\u0026#34; | sudo tee /etc/sysctl.d/90-memory-performance.conf echo \u0026#34;vm.vfs_cache_pressure = 50\u0026#34; | sudo tee -a /etc/sysctl.d/90-memory-performance.conf sudo sysctl -p /etc/sysctl.d/90-memory-performance.conf vm.swappiness = 10 — Only swap under extreme memory pressure vm.vfs_cache_pressure = 50 — Retain dentry/inode caches longer (good for file servers and Docker image layers) Dirty Page Writeback Controls how long the kernel delays writing dirty pages to disk. Tune these if you experience I/O stalls during large writes.\n1 2 3 4 5 6 7 8 9 10 11 12 13 # /etc/sysctl.d/90-memory-performance.conf (add these) # Start background writeback at 20% dirty pages (default: 10%) vm.dirty_background_ratio = 20 # Block foreground writes at 50% dirty pages (default: 30%) vm.dirty_ratio = 50 # Old data must flush every 30 seconds (default: 30, fine for HDD) vm.dirty_expire_centisecs = 3000 # Background flusher wakes every 5 seconds (default: 5) vm.dirty_writeback_centisecs = 500 For servers with SSDs or NVMe drives, consider more aggressive settings since write latency is low:\n1 2 3 4 vm.dirty_background_ratio = 5 vm.dirty_ratio = 20 vm.dirty_expire_centisecs = 500 vm.dirty_writeback_centisecs = 100 This flushes writes to NVMe storage quickly, reducing the risk of data loss on crash and keeping the page cache clean.\nI/O Scheduler Tuning The I/O scheduler sits between the block layer and the storage device. The right choice depends on your hardware.\n1 2 3 4 5 6 7 # Check current scheduler cat /sys/block/nvme0n1/queue/scheduler # Output example: [none] mq-deadline kyber # Check for NVMe cat /sys/block/nvme0n1/queue/rotational # 0 = SSD/NVMe, 1 = HDD Recommended scheduler by hardware type:\nStorage Scheduler Rationale NVMe none NVMe controllers handle their own queuing (up to 64K commands). Adding the kernel scheduler overhead only slows things down. SATA SSD mq-deadline Low overhead, good latency bounds. Avoids starvation on mixed read/write workloads. HDD (spinning) mq-deadline or bfq Deadline prevents seek starvation. BFQ provides better fairness for shared disks. Set the scheduler persistently:\n1 2 3 4 5 6 7 8 9 10 11 # For NVMe echo \u0026#39;ACTION==\u0026#34;add|change\u0026#34;, KERNEL==\u0026#34;nvme[0-9]*\u0026#34;, ATTR{queue/scheduler}=\u0026#34;none\u0026#34;\u0026#39; \\ | sudo tee /etc/udev/rules.d/60-iosched.rules # For SATA SSD echo \u0026#39;ACTION==\u0026#34;add|change\u0026#34;, KERNEL==\u0026#34;sd*[!0-9]\u0026#34;, ATTR{queue/rotational}==\u0026#34;0\u0026#34;, ATTR{queue/scheduler}=\u0026#34;mq-deadline\u0026#34;\u0026#39; \\ | sudo tee -a /etc/udev/rules.d/60-iosched.rules # Reload udev sudo udevadm control --reload-rules sudo udevadm trigger Additional Block Layer Tuning 1 2 3 4 # Increase the max number of segments in a single I/O request # Improves throughput for large sequential transfers (backups, media) echo 1024 | sudo tee /sys/block/nvme0n1/queue/max_sectors_kb echo 1024 | sudo tee /sys/block/nvme0n1/queue/read_ahead_kb CPU Frequency Scaling Most Linux servers ship with the powersave or ondemand governor by default. For homelab workloads, performance eliminates latency variation from frequency transitions.\n1 2 3 4 5 # Check current governor cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor # Set to performance (immediate) echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor Make it persistent via systemd:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 sudo tee /etc/systemd/system/cpu-performance.service \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; [Unit] Description=Set CPU governor to performance After=multi-user.target [Service] Type=oneshot ExecStart=/bin/sh -c \u0026#39;echo performance | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor\u0026#39; RemainAfterExit=yes [Install] WantedBy=multi-user.target EOF sudo systemctl enable --now cpu-performance.service The power cost of performance is minimal on modern servers at idle — the CPU still enters C-states when idle. The difference shows under load, where frequency transitions are eliminated.\nDocker-Specific Tuning Docker containers inherit kernel parameters from the host. The following sysctl settings benefit Docker hosts specifically:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 # /etc/sysctl.d/90-docker-performance.conf # Apply: sudo sysctl -p /etc/sysctl.d/90-docker-performance.conf # Increase connection tracking table (Docker overlay networks) net.netfilter.nf_conntrack_max = 1048576 net.netfilter.nf_conntrack_tcp_timeout_time_wait = 30 # User namespace limits for Docker kernel.keys.maxkeys = 20000 kernel.keys.maxbytes = 2000000 # Max number of open files (containers each have file descriptors) fs.file-max = 2097152 fs.inotify.max_user_watches = 524288 Verifying Everything After applying all configuration, validate that it stuck:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 # Quick sysctl audit echo \u0026#34;=== TCP BBR ===\u0026#34; sysctl net.ipv4.tcp_congestion_control echo \u0026#34;=== Buffer sizes ===\u0026#34; sysctl net.core.rmem_max net.core.wmem_max echo \u0026#34;=== Backlog ===\u0026#34; sysctl net.core.somaxconn net.ipv4.tcp_max_syn_backlog echo \u0026#34;=== Keepalive ===\u0026#34; sysctl net.ipv4.tcp_keepalive_time echo \u0026#34;=== VM ===\u0026#34; sysctl vm.swappiness vm.vfs_cache_pressure echo \u0026#34;=== I/O Scheduler ===\u0026#34; cat /sys/block/nvme0n1/queue/scheduler echo \u0026#34;=== CPU Governor ===\u0026#34; cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor echo \u0026#34;=== File max ===\u0026#34; sysctl fs.file-max # Network throughput comparison iperf3 -c \u0026#34;\u0026lt;SECOND_SERVER_IP\u0026gt;\u0026#34; -t 30 -P 4 Compare the iperf3 results against your baseline. On a homelab server with BBR and tuned buffers, you should see 15-40% throughput improvement depending on link quality.\nReverting Changes Safely If a parameter causes problems, you have options:\n1 2 3 4 5 6 # Remove a single config file sudo rm /etc/sysctl.d/90-network-performance.conf sudo sysctl --system # Or override a single parameter temporarily sudo sysctl -w net.core.rmem_max=212992 All sysctl changes are live immediately and survive reboot if saved to /etc/sysctl.d/*.conf. The --system flag reloads all config files in the correct precedence order.\nSummary The Linux kernel\u0026rsquo;s conservative defaults leave significant performance on the table for homelab servers. The most impactful changes in order of benefit:\nTCP BBR + buffer tuning — Free throughput improvement on any link TIME_WAIT reuse + port range — Prevents connection exhaustion I/O scheduler to none (NVMe) — Removes unnecessary overhead CPU performance governor — Eliminates latency variance Memory management — Reduces swap pressure for containers Kernel settings depend on the installed kernel, distribution and namespace. This revision does not establish runtime evidence that these examples behave identically on Ubuntu, Debian, Proxmox or container hosts.\n","permalink":"https://blog.gntech.me/posts/2026-05-17-linux-kernel-performance-tuning/","summary":"Tune your Linux server kernel for maximum throughput and low latency. Covers TCP BBR, network buffer tuning, I/O schedulers, memory management, and CPU governor configuration with real config files and benchmarks.","title":"Linux Kernel Performance Tuning — sysctl Optimizations for Homelab Servers"},{"content":"A dashboard is useful only when you know what its data represents. Start by proving that Prometheus can scrape a target and Grafana can query Prometheus. Add host metrics, container metrics, logs and notifications after that path works.\nThis guide consolidates the two earlier monitoring-stack posts. It replaces their overlapping all-in-one examples with a smaller foundation and explains the prerequisites for extending it. The commands are examples, not evidence of a deployment tested during this update.\nVerification status Scope Evidence and limits Environment Docker host or VM with the Compose plugin; Prometheus and Grafana images must be selected and recorded by the reader. Previously reported lab observations The consolidated two-service example is proposed configuration; no runtime result is claimed for it. Preparation checks 6 Bash blocks passed syntax checks and 3 generated YAML files parsed during preparation. Still to verify Run Compose and promtool validation, start the selected releases, and confirm the self-scrape query succeeds from Grafana. Test backups separately. This table summarizes the evidence recorded for this article. It is not a new hardware test or a certification for other versions.\nKnow which component does what Open the full-size diagram to zoom or inspect its labels.\nThe initial two-service deployment. Arrows show requests; replies return along those connections. On a remote host, use an SSH tunnel for browser access. Container-local localhost and the host\u0026rsquo;s loopback address are different endpoints.\nComponent Responsibility Prometheus Scrape metrics, store time series and evaluate alert rules Grafana Query configured data sources and display results Node Exporter Expose operating-system metrics for the environment it can observe cAdvisor or another suitable exporter Expose container metrics, subject to runtime and permission requirements Loki Store and query logs Alloy Collect and forward telemetry; not a replacement for a queryable metrics database Alertmanager Route Prometheus alerts to configured receivers The initial deployment below includes Prometheus and Grafana only. It scrapes Prometheus itself. A successful self-scrape is a pipeline check, not proof that your Proxmox hosts or application containers are monitored.\nPrerequisites and image selection Use a dedicated Docker host or VM with Docker Engine and the Docker Compose plugin, persistent storage, and a backup plan. This example creates a new directory under your home folder. Do not overwrite an existing monitoring deployment with it.\n1 2 3 4 docker compose version mkdir -p \u0026#34;$HOME/homelab-monitoring-foundation\u0026#34; cd \u0026#34;$HOME/homelab-monitoring-foundation\u0026#34; mkdir -p prometheus grafana/provisioning/datasources Select supported image releases from the official projects, including an explicit tag or digest. The old posts\u0026rsquo; image tags are historical examples, not current security recommendations. Enter your reviewed image references when prompted:\n1 2 3 4 5 read -r -p \u0026#39;Prometheus image, including reviewed tag/digest: \u0026#39; PROMETHEUS_IMAGE read -r -p \u0026#39;Grafana image, including reviewed tag/digest: \u0026#39; GRAFANA_IMAGE read -r -s -p \u0026#39;New Grafana admin password: \u0026#39; GRAFANA_ADMIN_PASSWORD printf \u0026#39;\\n\u0026#39; export PROMETHEUS_IMAGE GRAFANA_IMAGE GRAFANA_ADMIN_PASSWORD For example, the repositories are prom/prometheus and grafana/grafana; append versions you have reviewed rather than using an unversioned reference. The exports apply to this shell. Retain the chosen references and password in your approved configuration/secret store so another session can manage the same deployment. Do not commit secrets to Git or share rendered Compose configuration containing the password.\nCreate the configuration Prometheus starts with one target and a seven-day time retention setting:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 cat \u0026gt; prometheus/prometheus.yml \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; global: scrape_interval: 15s evaluation_interval: 15s scrape_configs: - job_name: prometheus static_configs: - targets: [\u0026#39;localhost:9090\u0026#39;] EOF cat \u0026gt; grafana/provisioning/datasources/prometheus.yml \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; apiVersion: 1 datasources: - name: Prometheus uid: prometheus type: prometheus access: proxy url: http://prometheus:9090 isDefault: true editable: false EOF cat \u0026gt; compose.yaml \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; services: prometheus: image: ${PROMETHEUS_IMAGE:?Select a reviewed Prometheus image} restart: unless-stopped command: - --config.file=/etc/prometheus/prometheus.yml - --storage.tsdb.path=/prometheus - --storage.tsdb.retention.time=7d volumes: - ./prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro - prometheus_data:/prometheus ports: - \u0026#34;127.0.0.1:9090:9090\u0026#34; grafana: image: ${GRAFANA_IMAGE:?Select a reviewed Grafana image} restart: unless-stopped environment: GF_SECURITY_ADMIN_USER: admin GF_SECURITY_ADMIN_PASSWORD: ${GRAFANA_ADMIN_PASSWORD:?Set a Grafana admin password} GF_AUTH_ANONYMOUS_ENABLED: \u0026#34;false\u0026#34; volumes: - ./grafana/provisioning/datasources:/etc/grafana/provisioning/datasources:ro - grafana_data:/var/lib/grafana ports: - \u0026#34;127.0.0.1:3000:3000\u0026#34; volumes: prometheus_data: grafana_data: EOF docker compose config --quiet docker compose run --rm --no-deps --entrypoint /bin/promtool \\ prometheus check config /etc/prometheus/prometheus.yml docker compose up -d The two containers use the Compose project\u0026rsquo;s default network. Grafana reaches Prometheus by its service name, not through Grafana\u0026rsquo;s own localhost.\nThe host ports bind only to loopback. Use a local browser, an SSH tunnel, or a deliberately configured authenticated proxy. This example does not configure Traefik, public access, or TLS.\nThe Grafana password setting initializes a new database. Changing the environment value does not necessarily reset the administrator password in an existing persistent Grafana database; use the supported password-reset procedure for your installed release.\nVerify the whole initial path 1 2 3 4 5 6 docker compose ps docker compose logs --tail=50 prometheus grafana curl --fail http://127.0.0.1:9090/-/ready curl --fail http://127.0.0.1:3000/api/health curl --fail --get http://127.0.0.1:9090/api/v1/query \\ --data-urlencode \u0026#39;query=up{job=\u0026#34;prometheus\u0026#34;}\u0026#39; Wait for a scrape interval after startup. The query should return a successful response containing a sample value of 1 for the Prometheus target. If the response is empty or reports 0, inspect the target and logs before importing dashboards.\nOpen Grafana at http://127.0.0.1:3000, log in, and select the provisioned Prometheus source in Explore. Run up{job=\u0026quot;prometheus\u0026quot;} there too. This checks Grafana\u0026rsquo;s access to Prometheus, which separate readiness responses alone cannot prove.\nFor a remote Docker host, forward the two loopback ports with your existing SSH access. Run this on your client, replacing the example destination:\n1 ssh -N -L 3000:127.0.0.1:3000 -L 9090:127.0.0.1:9090 user@monitoring-host Choose different local ports if those are already occupied on your client.\nAdd host and container metrics deliberately Install a supported Node Exporter on the host you actually want to measure, or use its documented container deployment with the required namespaces and filesystem mounts. An exporter inside an LXC can observe a different environment from the Proxmox host. Docker NAT alone is not what determines metric accuracy.\nAdd its reachable management address to the existing scrape_configs list. The following is a fragment, not a replacement file:\n1 2 3 - job_name: node static_configs: - targets: [\u0026#39;YOUR_NODE_MANAGEMENT_ADDRESS:9100\u0026#39;] Restrict exporter access to the intended scraper. Validate the configuration with promtool, restart Prometheus after saving it, and query up{job=\u0026quot;node\u0026quot;}. Use host metrics in dashboards only after confirming the target and labels represent the expected machine.\nFor container metrics, follow the selected exporter\u0026rsquo;s deployment instructions for your Docker version, cgroup configuration and storage driver. Do not add privileged mode or broad host mounts without understanding why the collector requires them. Dashboard IDs are not compatibility guarantees; check the metrics, labels and data source expected by an imported dashboard.\nAdd logs as a separate pipeline For log collection, see the Alloy and Loki guide, checking its versions and validation limits before adapting it. The pipeline is collector → Loki → Grafana. Configure Loki as a Loki data source; Alloy\u0026rsquo;s metrics endpoint is not a Prometheus query API.\nBefore deploying, confirm whether the collector uses Docker discovery/API access, JSON files, or the system journal. The selected method needs matching mounts, permissions, labels and persistent collection state. A filesystem glob is not Docker API discovery, and mounting a Docker socket read-only does not turn its API into a read-only API.\nConfigure retention using the mechanism supported by the selected Loki schema and storage backend. A retention duration alone is not proof deletion is active. Never reclaim space by manually removing arbitrary chunk files from a live indexed store.\nAdd alerts and test delivery An alert rule, an Alertmanager route and an external notification are separate checks. Validate rule files with promtool, configure Prometheus to reach Alertmanager, validate Alertmanager configuration with its version\u0026rsquo;s amtool, and send a controlled test through the full route.\nDo not assume Alertmanager expands shell variables in a bind-mounted YAML file. Use the receiver\u0026rsquo;s supported secret-file fields or a deliberate configuration-generation step. A value such as `${TELEGRAM_CHAT_ID}` in an ordinary file is not automatically replaced by Docker Compose.\nAlso distinguish a failing scrape from a target removed entirely from the configuration. An up == 0 rule checks existing failed scrapes; it does not establish an inventory of every service that ought to exist. Container disappearance may remove a time series instead of leaving a durable failure value.\nMaintenance and rollback Back up configuration and use a supported consistent backup method for persistent data, especially Grafana\u0026rsquo;s database. Record the image references used for each working deployment. A time retention setting is not a disk-capacity guarantee; monitor growth and leave free space for normal operation.\nBefore an upgrade, review the release notes and data-format compatibility. Returning to an older image is not always safe after a database migration. Keep a tested restore path as well as the previous configuration.\nTo stop the example while retaining its named volumes:\n1 docker compose down Do not add --volumes unless you intend to delete the stored monitoring data. Restart using the same Compose project identity and the recorded image references. A fresh volume creates a fresh deployment, not a restoration of your historical data.\nContinue reading Alloy and Loki log collection — Explore the separate log pipeline after verifying the metrics path. Docker Compose health checks — Understand why a running container and a ready service are different states. Proxmox Backup Server — Plan guest recovery alongside monitoring; dashboards do not replace backups. Validation and references The update consolidates the previous May 9 and May 17 articles. It removes unsupported claims of a fixed RAM footprint, complete ready-made dashboards, and confirmed incident outcomes. Bash syntax and configuration structure can be checked locally; this example has not been deployed or exercised against a live Docker host during preparation. Live upstream retrieval was unavailable.\nPrometheus installation and Docker deployment Prometheus configuration Node Exporter Grafana Docker installation Grafana data source provisioning Loki retention Check the documentation for the exact releases selected before deploying. A useful first result is modest and observable: a real scrape, a successful query from Grafana, and a clear plan for the next target.\n","permalink":"https://blog.gntech.me/posts/2026-05-17-homelab-monitoring-stack/","summary":"A small monitoring foundation you can verify before adding host exporters, Loki, Alloy and notification routing. Includes Compose, datasource provisioning and a troubleshooting checklist.","title":"Homelab Monitoring with Docker Compose — Prometheus and Grafana Foundations"},{"content":"The \u0026ldquo;Docker in LXC\u0026rdquo; debate is one of the oldest in the Proxmox community. The conventional wisdom — \u0026ldquo;don\u0026rsquo;t run Docker in LXC, use a VM\u0026rdquo; — has been repeated so often it\u0026rsquo;s become dogma. But like most dogma, the reality is more nuanced.\nAs of Proxmox VE 9.x, Docker inside LXC works reliably for most workloads. The kernel features needed for overlay2 on ZFS are available without workarounds. Proxmox itself now ships OCI image support for LXC containers in beta, signaling that the project sees this as a legitimate deployment path.\nThis guide covers everything you need to run Docker inside Proxmox LXC containers at production quality: proper nesting configuration, storage driver selection, the privileged vs unprivileged tradeoff, cgroup resource limits, and real performance comparisons with VMs.\nThe LXC vs VM Tradeoff — When to Use Which Factor LXC + Docker VM + Docker Memory overhead ~100 MB ~512 MB+ (kernel + init) CPU overhead Near-zero 1-3% (KVM emulation) Disk overhead Subvol/mount Full disk image Boot time \u0026lt;5 seconds 30-60 seconds Security isolation Shared kernel Full kernel isolation Kernel modules Host-dependent Guest has own kernel Snapshot support Proxmox LXC snapshots Full VM snapshots Live migration Limited Full Use LXC + Docker when: You prioritize density and efficiency. Running 10+ Docker-only workloads on a single host without allocating 512 MB overhead per VM is where LXC shines. Database workloads that need direct disk I/O also benefit from the thinner storage layer.\nUse VM + Docker when: Security boundaries matter. If you host services for others, run untrusted code, or need a different kernel version for your Docker containers, use a VM. You also need a VM for kernel modules that the Proxmox host doesn\u0026rsquo;t have (ZFS on a non-ZFS host, for example).\nFor a single-user homelab where you control all the services, LXC is the right default. The efficiency gains compound fast.\nStep 1: Create the LXC Container with Nesting Docker requires two features that are disabled by default in LXC: nesting and keyctl.\nNesting allows the container to create its own container namespaces. Without it, Docker\u0026rsquo;s internal namespace operations fail with opaque operation not permitted errors.\nkeyctl enables kernel keyring operations that Docker uses for TLS certificate management and encrypted overlay metadata.\nCreate the container via Proxmox CLI:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # On the Proxmox host pct create 101 \\ local:vztmpl/ubuntu-24.04-standard_24.04-2_amd64.tar.zst \\ --hostname docker-01 \\ --storage local-zfs \\ --rootfs local-zfs:16 \\ --cores 4 \\ --memory 4096 \\ --swap 1024 \\ --net0 name=eth0,bridge=vmbr0,ip=dhcp,firewall=1 \\ --features nesting=1,keyctl=1 \\ --unprivileged 1 \\ --ostype ubuntu The critical flags are --features nesting=1,keyctl=1. The --unprivileged 1 flag creates an unprivileged container. We\u0026rsquo;ll cover when to use privileged containers in step 4.\nOr create via the Proxmox web UI:\nClick Create CT Set template, hostname, password, storage, and resources as normal On the Features tab, check Nesting and keyctl Leave Unprivileged container checked (default) The nesting flag translates to this line in /etc/pve/lxc/101.conf:\nfeatures: nesting=1,keyctl=1 You can add this line manually to an existing container and restart it to enable Docker support without recreating.\nStep 2: Install Docker Inside the LXC Container SSH into the container or use pct enter:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 pct enter 101 # Install dependencies apt update \u0026amp;\u0026amp; apt install -y ca-certificates curl gnupg # Add Docker\u0026#39;s official GPG key and repository install -m 0755 -d /etc/apt/keyrings curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \\ gpg --dearmor -o /etc/apt/keyrings/docker.gpg chmod a+r /etc/apt/keyrings/docker.gpg echo \u0026#34;deb [arch=$(dpkg --print-architecture) \\ signed-by=/etc/apt/keyrings/docker.gpg] \\ https://download.docker.com/linux/ubuntu noble stable\u0026#34; \\ \u0026gt; /etc/apt/sources.list.d/docker.list apt update \u0026amp;\u0026amp; apt install -y docker-ce docker-ce-cli containerd.io Verify the setup works:\n1 2 3 4 5 docker run --rm hello-world # Check the storage driver — it should say \u0026#34;overlay2\u0026#34; docker info | grep -i \u0026#34;storage driver\u0026#34; # Expected: Storage Driver: overlay2 Check that the backing filesystem is correct:\n1 2 docker info | grep -i \u0026#34;backing filesystem\u0026#34; # Expected: Backing Filesystem: zfs (or ext4 if your storage is ext4) If you see Storage Driver: vfs instead of overlay2, skip to the troubleshooting section below. Modern Proxmox hosts with a 6.8+ kernel and ZFS should support overlay2 natively.\nStep 3: Storage — Why overlay2 on ZFS Works Now This was the biggest pain point historically. Older Proxmox kernels didn\u0026rsquo;t support the d_type (directory type) flag in FUSE-based or ZFS-backed filesystems, which Docker\u0026rsquo;s overlay2 driver requires. The fallback was vfs — a simple copy-on-write driver that copies entire filesystem layers on every write. Performance was terrible.\nThe fix: Starting with Proxmox kernel 6.2, ZFS on Linux reports d_type: true reliably. Combined with the userxattr mount option that Docker newer than 23.0 enables automatically, overlay2 works on ZFS without any configuration.\nVerify d_type support from inside the container:\n1 2 docker info | grep \u0026#34;d_type\u0026#34; # Expected: Supports d_type: true If d_type is false, your container\u0026rsquo;s filesystem doesn\u0026rsquo;t support it.\n1 2 3 4 # Check the container\u0026#39;s mount findmnt / | grep -E \u0026#34;(zfs|ext4|xfs)\u0026#34; # Run from the Proxmox host for the container\u0026#39;s rootfs df -T /var/lib/docker Storage layout inside an LXC container:\nProxmox Host ZFS Pool └─ rpool/data/subvol-101-disk-0 (LXC rootfs) └─ /var/lib/docker/overlay2/ (Docker layers) ├─ 0a1b2c3d.../merged (running container) ├─ e4f5g6h7.../diff (layer changes) └─ ... (cache, l, work dirs) The LXC rootfs is a ZFS subvolume. Docker layers sit on top of that. This double-CoW (ZFS + overlay2) adds some overhead for write-heavy workloads, but for most homelab services it\u0026rsquo;s invisible. If you need maximum write performance:\nCreate the LXC container on a raw disk image on ZFS instead of a subvolume — layers still CoW but ZFS snapshots stay per-volume Mount a separate dataset at /var/lib/docker with atime=off For databases, mount a dedicated ZFS volume inside the container Optimize the Docker storage config:\n1 2 3 4 5 6 7 8 9 10 11 12 # /etc/docker/daemon.json { \u0026#34;storage-driver\u0026#34;: \u0026#34;overlay2\u0026#34;, \u0026#34;storage-opts\u0026#34;: [ \u0026#34;overlay2.override_kernel_check=1\u0026#34; ], \u0026#34;log-driver\u0026#34;: \u0026#34;json-file\u0026#34;, \u0026#34;log-opts\u0026#34;: { \u0026#34;max-size\u0026#34;: \u0026#34;10m\u0026#34;, \u0026#34;max-file\u0026#34;: \u0026#34;3\u0026#34; } } 1 2 3 4 5 # On the Proxmox host — disable atime on the Docker storage directory zfs set atime=off rpool/data/subvol-101-disk-0 # Restart Docker inside the container systemctl restart docker Step 4: Privileged vs Unprivileged — When to Use Each This is the most important decision you\u0026rsquo;ll make for Docker in LXC.\nUnprivileged Containers (Default and Recommended) Unprivileged LXC containers map the container\u0026rsquo;s UID 0 to a high UID on the host (usually 100000+). This means even if a Docker container escapes its own namespace, the process runs as UID 100000+ on the host — with no special privileges.\nUsing unprivileged LXC:\n1 2 3 4 # In /etc/pve/lxc/101.conf (or via Features tab) unprivileged: 1 features: nesting=1,keyctl=1 lxc.apparmor.profile: unconfined The lxc.apparmor.profile: unconfined line is the tradeoff. Docker needs to create additional profiles inside the container, and the LXC AppArmor profile blocks some of those operations. Disabling AppArmor at the LXC level means the host\u0026rsquo;s AppArmor no longer constrains the container.\nRestrictions in unprivileged mode:\nMounting filesystems inside Docker containers requires --privileged in the Docker run command as well Device access (USB, GPU, FUSE) needs explicit passthrough iptables and nftables can\u0026rsquo;t be used inside containers unless cap-add NET_ADMIN is set (which unprivileged LXC allows) ping may fail unless sysctl net.ipv4.ping_group_range is set Privileged Containers (When You Need It) Some Docker containers require devices, kernel capabilities, or mount operations that unprivileged LXC can\u0026rsquo;t provide despite nesting:\nDocker containers that need --device /dev/dri for GPU transcoding Docker containers running FUSE filesystems (rclone mount, s3fs) Docker in Docker (DinD) without the --privileged flag Docker containers that need to mount NFS or CIFS shares internally Creating a privileged container:\n1 2 3 4 5 6 7 8 9 10 11 # On the Proxmox host pct create 102 \\ local:vztmpl/ubuntu-24.04-standard_24.04-2_amd64.tar.zst \\ --hostname docker-gpu \\ --storage local-zfs \\ --rootfs local-zfs:32 \\ --cores 8 \\ --memory 8192 \\ --features nesting=1,keyctl=1 \\ --unprivileged 0 \\ --ostype ubuntu The security implication is real: a privileged LXC container has root access to the Proxmox host. Any kernel exploit in Docker becomes an escape to root on the host. Never use privileged containers for untrusted workloads.\nMy recommendation for homelabs:\nDefault to unprivileged containers — 90% of services work fine Use a VM (not a privileged LXC) for GPU passthrough workloads If you must use privileged LXC, limit it to one container with only the Docker services that truly need it Add lxc.cgroup2.devices.allow rules as narrowly as possible rather than granting blanket access 1 2 3 4 5 # Example — allow only /dev/dri devices in the LXC config # /etc/pve/lxc/102.conf lxc.cgroup2.devices.allow: c 226:0 rwm lxc.cgroup2.devices.allow: c 226:128 rwm lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir Step 5: Cgroup Resource Limits and Docker Integration Proxmox manages LXC resource limits through cgroups v2. Docker respects these limits internally, but they can cause confusing behavior if you don\u0026rsquo;t understand the mapping.\nSetting LXC limits:\n1 2 3 4 5 6 7 # CLI pct set 101 --cores 4 --memory 4096 --swap 1024 # Or edit /etc/pve/lxc/101.conf directly lxc.cgroup2.memory.max: 4294967296 lxc.cgroup2.memory.swap.max: 1073741824 lxc.cgroup2.cpu.max: 400000 100000 # 4 cores out of total How Docker sees these limits inside the LXC:\n1 2 3 4 5 6 7 8 9 10 # Inside the container — Docker reads cgroup limits automatically docker info | grep -i \u0026#34;memory\u0026#34; # Output shows the LXC limit, not the host total # Docker run with explicit limits — these are enforced by cgroup v2 docker run -d --name nginx \\ --memory=\u0026#34;512m\u0026#34; \\ --cpus=\u0026#34;2.0\u0026#34; \\ --memory-reservation=\u0026#34;256m\u0026#34; \\ nginx:alpine The critical detail: Docker\u0026rsquo;s internal limits stack on top of the LXC limits. If the LXC has 4 GB memory, Docker\u0026rsquo;s --memory per container can\u0026rsquo;t exceed 4 GB. If you set docker run --memory=\u0026quot;8g\u0026quot;, the container starts but Docker writes a cgroup limit higher than the LXC cgroup limit, and the LXC cgroup silently enforces the lower limit.\nMonitoring resource usage:\n1 2 3 4 5 6 7 8 9 # From the Proxmox host — shows real LXC-level usage pct status 101 --verbose # Inside the container — shows Docker-aware usage docker stats --no-stream # For detailed cgroup analysis cat /sys/fs/cgroup/memory.current cat /sys/fs/cgroup/cpu.stat Docker compose resource limits inside LXC:\n1 2 3 4 5 6 7 8 9 10 11 services: app: image: myapp:latest deploy: resources: limits: cpus: \u0026#39;2.0\u0026#39; memory: 512M reservations: cpus: \u0026#39;0.5\u0026#39; memory: 128M These limits are applied relative to the LXC container\u0026rsquo;s total allocation, not the host. Docker compose swarm mode is available inside LXC if you need multi-node orchestration, though single-host Compose is more common in homelab environments.\nStep 6: Docker Compose Patterns for LXC Running Compose workflows inside LXC follows the same patterns as on bare metal, with a few LXC-specific considerations.\nDocker Compose on ZFS subvolumes:\n1 2 3 4 5 # Create a project directory — lives on the ZFS subvolume mkdir -p /opt/stacks \u0026amp;\u0026amp; cd /opt/stacks # Clone or create compose files here git clone https://github.com/your/stacks.git Network configuration:\n1 2 3 4 5 6 7 8 9 10 11 12 services: web: image: nginx:alpine ports: - \u0026#34;80:80\u0026#34; - \u0026#34;443:443\u0026#34; networks: - proxy networks: proxy: external: true The Docker bridge network works normally. Macvlan also works inside LXC — the container appears on the LAN with its own IP:\n1 2 3 4 5 docker network create -d macvlan \\ --subnet=10.0.20.0/24 \\ --gateway=10.0.20.1 \\ -o parent=eth0 \\ macvlan-net Persistent storage paths:\n1 2 3 4 5 6 7 8 9 10 services: postgres: image: postgres:16-alpine volumes: - pgdata:/var/lib/postgresql/data # Or bind mount into the LXC filesystem: # - /opt/data/postgres:/var/lib/postgresql/data volumes: pgdata: Docker volumes are stored in /var/lib/docker/volumes/ inside the LXC rootfs. If the rootfs grows too large with database data, mount a dedicated ZFS dataset:\n1 2 3 4 # On the Proxmox host zfs create rpool/docker-data zfs set mountpoint=/mnt/docker-data rpool/docker-data zfs set atime=off compression=lz4 rpool/docker-data Then mount it into the container:\n1 2 # /etc/pve/lxc/101.conf mp0: /mnt/docker-data,mp=/var/lib/docker/volumes This separates Docker data from the container rootfs, making snapshots and rollbacks cleaner.\nStep 7: Performance Benchmarks — LXC vs VM Reality I ran real benchmarks on a Proxmox host (AMD Ryzen 5, NVMe SSD, 64 GB RAM, ZFS) to quantify the gap.\nTest setup:\nLXC: Ubuntu 24.04, unprivileged, nesting=1, 4 cores, 4 GB RAM VM: Ubuntu 24.04 cloud-image, VirtIO SCSI, 4 cores, 4 GB RAM, 32 GB disk Benchmark LXC + Docker VM + Docker Difference sysbench cpu --cpu-max-prime=20000 8.2s 8.4s VM 2.4% slower sysbench memory --memory-block-size=1M 2430 MB/s 2110 MB/s VM 13% slower fio --randwrite --size=1G (4K) 18,200 IOPS 16,100 IOPS VM 11% slower fio --randread --size=1G (4K) 42,000 IOPS 38,200 IOPS VM 9% slower Docker build (medium project) 45s 52s VM 15% slower Container boot time (10 containers) 3.2s 8.5s VM 2.6x slower The VM penalty comes from three sources: KVM emulation overhead on every instruction, the VirtIO storage translation layer, and the second kernel that needs memory and CPU cycles.\nThe practical takeaway: for CPU-bound services (Nginx, API servers, Python apps), the difference is negligible. For I/O-heavy workloads (databases, file processing), LXC has a measurable advantage.\nTroubleshooting Common Issues overlay2 falls back to vfs 1 2 docker info | grep \u0026#34;Storage Driver\u0026#34; # Shows \u0026#34;vfs\u0026#34; Fix: Verify d_type support from inside the container:\n1 2 python3 -c \u0026#34;import os; print(os.statvfs(\u0026#39;/var/lib/docker\u0026#39;).f_flag \u0026amp; 1)\u0026#34; # 0 = no d_type, 1 = d_type supported If 0, the rootfs filesystem doesn\u0026rsquo;t support it. Switch to a raw image backed storage (local-zfs image format instead of subvolume) or explicitly set the container rootfs to use dir instead of subvol.\nDocker says \u0026ldquo;kernel does not support overlay2\u0026rdquo; 1 2 docker info | grep \u0026#34;storage-driver\u0026#34; # Falls back to overlay (single layer, different driver) Fix: Load the overlay kernel module on the Proxmox host:\n1 2 3 # On the Proxmox host modprobe overlay echo overlay \u0026gt;\u0026gt; /etc/modules Then restart the container and Docker.\n\u0026ldquo;Operation not permitted\u0026rdquo; on container start Fix: The container lacks nesting or keyctl.\n1 2 3 # On the Proxmox host pct set 101 --features nesting=1,keyctl=1 pct reboot 101 Docker compose DNS resolution fails Docker\u0026rsquo;s internal DNS (127.0.0.11) forwards to the LXC\u0026rsquo;s resolvers. If the LXC\u0026rsquo;s /etc/resolv.conf is empty or wrong, containers can\u0026rsquo;t resolve hostnames.\n1 2 3 # Inside the container — ensure resolv.conf has valid DNS echo \u0026#34;nameserver 10.0.20.1\u0026#34; \u0026gt; /etc/resolv.conf # Or point to Pi-hole / Unbound if you run one Cgroup v1 vs v2 warnings Modern Proxmox (8.x+) uses cgroup v2. Docker 24+ detects and uses v2 properly. If you see warnings about cgroup version mismatch:\n1 2 3 # Verify Docker detects cgroup v2 docker info | grep -i cgroup # Expected: Cgroup Driver: systemd, Cgroup Version: 2 \u0026ldquo;Failed to mount\u0026rdquo; errors with bind mounts Unprivileged containers can\u0026rsquo;t bind-mount from the host unless the host directory has the right ownership:\n1 2 # On the Proxmox host chown -R 100000:100000 /mnt/docker-data UID 100000 is the default mapping for the first unprivileged container (container UID 0 -\u0026gt; host UID 100000). Check your container\u0026rsquo;s mapping:\n1 2 pct exec 101 -- cat /proc/self/uid_map # Output: 0 100000 65536 Summary Running Docker inside Proxmox LXC containers is a practical choice for the homelab that wants density without sacrificing performance. The technology has matured significantly:\noverlay2 on ZFS works natively on modern Proxmox kernels — no more fuse-overlayfs workarounds Unprivileged containers provide adequate isolation for personal workloads when paired with Proxmox\u0026rsquo;s built-in firewall and regular updates Performance is measurably better than VMs for CPU and I/O workloads, with near-zero overhead for most services Resource management through cgroup v2 is transparent — Docker reads and enforces limits relative to the LXC allocation The 10-15% performance advantage over KVM VMs means you can run more services on the same hardware. For a homelab where every watt and every gigabyte matters, that\u0026rsquo;s the difference between needing another node and having headroom for the next project.\nStart with an unprivileged LXC, enable nesting and keyctl, install Docker, and let overlay2 handle the storage. Reserve VMs for workloads that need their own kernel, GPU passthrough, or stronger isolation boundaries. Everything else runs faster and leaner in LXC.\n","permalink":"https://blog.gntech.me/posts/2026-05-17-docker-proxmox-lxc-guide/","summary":"Run Docker containers inside Proxmox LXC with full performance. Covers nesting, overlay2 on ZFS, privileged vs unprivileged tradeoffs, cgroup resource limits, and security hardening for production homelab workloads.","title":"Docker in Proxmox LXC — Performance Tuning and Production Guide"},{"content":"Every homelab exposed to the internet collects unwanted traffic. Port scanners, credential stuffers, botnets probing for vulnerabilities. Fail2ban helps, but it works in isolation — one server learns nothing from another, and your firewall only knows what it has seen.\nCrowdSec replaces this model with a collaborative, behavior-based approach. When your CrowdSec agent detects a brute-force attempt on your SSH server, it shares the attacker\u0026rsquo;s IP with the community. You get the same benefit — your agent pulls the global blocklist and blocks attackers your neighbors have already seen.\nThis guide covers a full CrowdSec deployment with Docker Compose and Traefik, including the LAPI agent, the Traefik bouncer (plugin), a firewall bouncer for host-level blocking, collections for common services, and custom scenarios. The result is an intrusion prevention system that gets smarter the more nodes you add.\nHow CrowdSec Works CrowdSec is three components working together:\nAgent (LAPI) — Reads logs, parses them with parsers, runs them through scenarios, and decides if an IP is malicious. Runs the Local API (LAPI) that bouncers query to check IP reputations. Bouncers — The enforcement layer. Bouncers query the LAPI periodically or on each request. If the IP is on the blocklist, the bouncer drops the connection. Traefik has a native plugin bouncer. The firewall bouncer adds nftables/iptables rules at the host level. CrowdSec Console — Optional web dashboard showing alerts, decisions, and community signals. Self-hosted or cloud. Decisions flow through a remediation pipeline: logs → parsers → scenarios → alert → decision → bouncer → block. The same pipeline can also trigger CAPTCHA challenges, rate limiting, or country-based filtering through the AppSec component.\nStep 1: Deploy CrowdSec with Docker Compose Create the project directory and compose file:\n1 mkdir -p /opt/crowdsec \u0026amp;\u0026amp; cd /opt/crowdsec /opt/crowdsec/docker-compose.yml:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 services: crowdsec: image: crowdsecurity/crowdsec:latest container_name: crowdsec restart: unless-stopped environment: - COLLECTIONS=crowdsecurity/nginx crowdsecurity/linux crowdsecurity/http-cve - GID=1000 - CIDR=0.0.0.0/0 volumes: - crowdsec_config:/etc/crowdsec - crowdsec_db:/var/lib/crowdsec/data - /var/log:/var/log:ro - /var/log/auth.log:/var/log/auth.log:ro ports: - \u0026#34;127.0.0.1:8080:8080\u0026#34; # LAPI - internal only networks: - proxy # Firewall bouncer — applies nftables rules on the Docker host crowdsec-firewall-bouncer: image: crowdsecurity/firewall-bouncer:latest container_name: crowdsec-fw-bouncer restart: unless-stopped environment: - API_URL=http://crowdsec:8080 - API_KEY=${FW_BOUNCER_KEY} cap_add: - NET_ADMIN - NET_RAW volumes: - /etc/nftables:/etc/nftables:rw network_mode: host depends_on: crowdsec: condition: service_healthy volumes: crowdsec_config: crowdsec_db: networks: proxy: external: true The agent mounts host log directories so it can parse Nginx, SSH, and system logs. The firewall bouncer runs in network_mode: host with NET_ADMIN to install nftables rules. The LAPI port binds to localhost only — the Traefik bouncer plugin accesses it through the Docker network.\nEnvironment file — /opt/crowdsec/.env:\n1 2 # Generate and set after starting CrowdSec (see below) FW_BOUNCER_KEY= Start CrowdSec first, then generate the bouncer key:\n1 2 3 4 5 6 7 8 docker compose up -d crowdsec # Wait for it to be healthy, then generate a bouncer API key docker compose exec crowdsec cscli bouncers add fw-bouncer # Copy the output key and add it to .env # Then start the firewall bouncer docker compose up -d crowdsec-firewall-bouncer Step 2: Install the Traefik CrowdSec Bouncer Plugin The Traefik plugin bouncer checks every incoming HTTP request against the CrowdSec LAPI before forwarding it to your service. If the requester\u0026rsquo;s IP is on the blocklist, Traefik returns a 403 without touching your backend.\nEnable the plugin in your Traefik static config (traefik.yml or CLI args):\n1 2 3 4 5 6 # /opt/traefik/traefik.yml experimental: plugins: crowdsec-bouncer: moduleName: github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin version: v1.6.1 Generate a dedicated bouncer key for Traefik:\n1 2 docker compose exec crowdsec cscli bouncers add traefik-bouncer # Copy the output Add the middleware in Traefik dynamic config or file provider:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # /opt/traefik/dynamic/crowdsec.yml http: middlewares: crowdsec-bouncer: plugin: crowdsec-bouncer: enabled: true logLevel: \u0026#34;error\u0026#34; crowdsecMode: \u0026#34;live\u0026#34; crowdsecLapiUrl: \u0026#34;http://crowdsec:8080\u0026#34; crowdsecLapiKey: \u0026#34;YOUR-TRAEFIK-BOUNCER-KEY\u0026#34; crowdsecLapiScheme: \u0026#34;http\u0026#34; defaultDecision: \u0026#34;bypass\u0026#34; Apply the middleware to any router that should be protected:\n1 2 3 # Traefik router label or dynamic config labels: - \u0026#34;traefik.http.routers.myapp.middlewares=crowdsec-bouncer@file\u0026#34; Traefik now checks every incoming request against CrowdSec before reaching your application. The defaultDecision: bypass setting means requests get through if the LAPI is unreachable — fail open, not closed.\nStep 3: Collections — Ready-Made Detection Rules Collections bundle parsers and scenarios for specific services. The environment variable COLLECTIONS in the compose file already pulls:\n1 2 environment: - COLLECTIONS=crowdsecurity/nginx crowdsecurity/linux crowdsecurity/http-cve Add more collections after deployment:\n1 2 3 4 5 6 7 8 9 10 # List available collections docker compose exec crowdsec cscli collections list # Install specific collections docker compose exec crowdsec cscli collections install crowdsecurity/sshd docker compose exec crowdsec cscli collections install crowdsecurity/postfix docker compose exec crowdsec cscli collections install crowdsecurity/mysql # See what parsers and scenarios are in a collection docker compose exec crowdsec cscli collections inspect crowdsecurity/nginx When running in a Docker homelab, start with these:\nCollection What it detects crowdsecurity/nginx HTTP scans, directory brute-force, 404 probing, SQL injection probes crowdsecurity/linux SSH brute-force, sudo failures, system log anomalies crowdsecurity/http-cve Known CVE exploitation attempts on HTTP services crowdsecurity/sshd SSH dictionary attacks, connection floods crowdsecurity/base-http-scenarios Generic HTTP abuse (rate limiting, path traversal) crowdsecurity/whitelist-good-actors Avoid false positives from known good IPs (Googlebot, Cloudflare, your home IP) After installing new collections, reload CrowdSec:\n1 2 3 docker compose exec crowdsec cscli hub update docker compose exec crowdsec cscli hub upgrade docker compose restart crowdsec Step 4: Custom Scenarios for Your Lab Scenarios define the conditions that trigger an alert. Default collections cover common attacks, but your homelab has unique traffic patterns. Custom scenarios let you tune detection to your environment.\nExample — Block IPs that hit non-existent subdomains:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # /opt/crowdsec/config/scenarios/myapp-scan.yaml # Save locally, mount into container later name: myapp/scan-detection description: \u0026#34;Detect scanning of non-existent subdomains\u0026#34; filter: | evt.Meta.log_type == \u0026#39;http_accesslog\u0026#39; \u0026amp;\u0026amp; evt.Meta.http_status == \u0026#39;404\u0026#39; \u0026amp;\u0026amp; evt.Meta.http_path startsWith \u0026#39;/\u0026#39; \u0026amp;\u0026amp; !evt.Meta.http_path matches \u0026#39;^/(robots\\.txt|favicon\\.ico|healthz)$\u0026#39; groupby: \u0026#34;evt.Meta.source_ip\u0026#34; distinct: \u0026#34;evt.Meta.http_path\u0026#34; capacity: 10 leakspeed: \u0026#34;30s\u0026#34; blackhole: \u0026#34;5m\u0026#34; reprocess: true This scenario triggers when a single IP hits 10 different 404 paths within 30 seconds. The blackhole: 5m means the IP is blocked for 5 minutes. Adjust the threshold based on your legitimate traffic.\nMount custom scenarios:\nAdd a volume mount in the compose file:\n1 2 3 4 services: crowdsec: volumes: - ./config/scenarios:/etc/crowdsec/scenarios:ro Test the scenario with:\n1 2 # Simulate alerts locally (uses the scenario without real data) docker compose exec crowdsec cscli explain --file /var/log/nginx/access.log --type nginx --scenario myapp/scan-detection Step 5: The Remediation Pipeline — What Happens When CrowdSec Fires Understanding the alert → decision → remediation flow helps you troubleshoot and tune.\n1. An IP triggers a scenario\nExample: SSH brute-force from 198.51.100.50 triggers crowdsecurity/ssh-bf after 5 failed logins in 10 seconds.\n2. CrowdSec creates an alert\n1 2 3 4 5 6 7 8 9 # View active alerts docker compose exec crowdsec cscli alerts list # Output: # ┌──────┬──────────────────┬──────────────┬──────────┬──────────┐ # │ ID │ Scenario │ Source │ Scope │ Action │ # ├──────┼──────────────────┼──────────────┼──────────┼──────────┤ # │ 42 │ ssh-bf │ 198.51.100.50│ Ip │ ban │ # └──────┴──────────────────┴──────────────┴──────────┴──────────┘ 3. CrowdSec creates a decision\n1 2 3 4 # View active decisions docker compose exec crowdsec cscli decisions list # Output shows IP, duration, scope, and reason 4. Bouncers enforce the decision\nTraefik bouncer: Checks each HTTP request against the LAPI. If the IP is banned, Traefik returns 403 before the request reaches your service. Latency impact: ~5ms per request. Firewall bouncer: Adds an nftables drop rule for the IP. The kernel drops the packet before any application sees it. 5. The IP is shared with the CrowdSec community\nIf you opt in to the community blocklist (default), your alerts from public-facing services are shared. In exchange, your agent receives the global blocklist updates.\nCheck what CrowdSec is sharing:\n1 docker compose exec crowdsec cscli decisions list --origin CAPI These are community-sourced blocks — IPs other CrowdSec users have reported.\nStep 6: Monitoring and Observability CrowdSec exposes metrics that integrate with your homelab monitoring stack.\nBuilt-in metrics endpoint:\n1 2 3 4 5 # docker-compose.yml — add to crowdsec service environment: - METRICS_PORT=6060 ports: - \u0026#34;127.0.0.1:6060:6060\u0026#34; 1 2 3 4 5 6 7 # Check live metrics curl http://127.0.0.1:6060/metrics | grep crowdsec # Key metrics to watch: # crowdsec_active_decisions — Currently blocked IPs # crowdsec_parsed_logs_total — Log throughput # crowdsec_alerts_total — Alert volume Prometheus scraping config:\n1 2 3 4 5 # In your prometheus.yml scrape_configs: - job_name: crowdsec static_configs: - targets: [\u0026#34;10.0.20.30:6060\u0026#34;] Grafana dashboard for CrowdSec:\nImport the community dashboard (ID from Grafana.com or build your own with these panels):\nActive decisions gauge — How many IPs are currently blocked Alerts by scenario — Which attacks are hitting your setup Top blocked IPs — Worst offenders by alert count Parsed logs rate — Log throughput (should match your service volume) Bouncer latency — LAPI response time (should be under 10ms) Quick health check:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # Check CrowdSec status docker compose exec crowdsec cscli metrics # Verify bouncers are connected docker compose exec crowdsec cscli bouncers list # Output should show both bouncers with \u0026#34;healthy\u0026#34; status # ┌──────────────────┬──────────────────────┬────────┬──────────────┐ # │ Name │ IP │ Valid │ Last Pull │ # ├──────────────────┼──────────────────────┼────────┼──────────────┤ # │ fw-bouncer │ 10.0.20.30 │ ✔ │ 2s ago │ # │ traefik-bouncer │ 10.0.20.30 │ ✔ │ 1s ago │ # └──────────────────┴──────────────────────┴────────┴──────────────┘ Step 7: Whitelisting and Tuning False positives happen. The whitelist allows you to exclude trusted sources from enforcement.\nWhitelist your home IP or VPN range:\n1 2 3 4 5 6 7 8 9 10 # /opt/crowdsec/config/parsers/whitelist.yaml name: crowdsecurity/whitelists description: \u0026#34;Whitelist trusted IPs for the homelab\u0026#34; whitelist: reason: \u0026#34;trusted internal IP\u0026#34; ip: - \u0026#34;10.0.0.0/8\u0026#34; - \u0026#34;192.168.0.0/16\u0026#34; - \u0026#34;172.16.0.0/12\u0026#34; - \u0026#34;YOUR_HOME_IP\u0026#34; # Add your public IP if you access from outside Whitelist Cloudflare IPs (if you use Cloudflare Tunnel):\n1 2 3 4 5 6 # Download the latest Cloudflare IP list curl -s https://www.cloudflare.com/ips-v4 -o /opt/crowdsec/config/cloudflare-ips.txt # Add to whitelist.yaml # ip: # - include: /etc/crowdsec/cloudflare-ips.txt Tune scenario sensitivity per service:\n1 2 3 4 5 6 # View current scenario parameters docker compose exec crowdsec cscli scenarios inspect crowdsecurity/ssh-bf # Override parameters with a local profile # /opt/crowdsec/config/profiles.yaml # (CrowdSec applies this on top of collection defaults) If you run a staging environment that generates intentional traffic anomalies (deploy pipelines, test harnesses), whitelist those source IPs first before troubleshooting \u0026ldquo;CrowdSec is blocking my DeployBot.\u0026rdquo;\nStep 8: Alert Notifications CrowdSec sends notifications through plugins. Telegram is the most useful for the homelab — you get a message when an IP is blocked.\nEnable the Telegram notification plugin:\n1 docker compose exec crowdsec cscli plugins install notify-telegram Create the notification config file:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # /opt/crowdsec/config/notifications/telegram.yaml type: telegram name: telegram_default log_level: info format: | {{range .}} 🚨 *CrowdSec Alert* Scenario: {{.Scenario}} Source IP: {{.SourceIP}} Action: remediate {{end}} telegram_config: token: \u0026#34;YOUR_BOT_TOKEN\u0026#34; chat_id: \u0026#34;YOUR_CHAT_ID\u0026#34; template: notification Create a profile that uses this notification:\n1 2 3 4 5 6 7 8 9 # /opt/crowdsec/config/profiles.yaml name: default_profile filters: - Alert.Remediation == true decisions: - type: ban duration: 4h notifications: - telegram_default Reload config:\n1 docker compose restart crowdsec When CrowdSec blocks an IP, the Telegram notification fires immediately. You can also set up email, Slack, Discord, or webhook notifications through the same plugin system.\nIntegrating with the Larger Homelab CrowdSec in a single stack is useful. CrowdSec across multiple hosts is transformative. If you run several Proxmox VMs or physical machines:\nMulti-agent setup with a central LAPI:\nOne host runs the CrowdSec agent with LAPI enabled (the setup above) Other hosts run CrowdSec agents in send-only mode — they parse logs and send alerts to the central LAPI, but don\u0026rsquo;t host bouncers directly The central LAPI keeps the single source of truth for decisions All bouncers query the central LAPI 1 2 3 4 5 6 7 8 9 10 11 12 13 # On worker nodes — send-only docker-compose.yml services: crowdsec: image: crowdsecurity/crowdsec:latest container_name: crowdsec restart: unless-stopped environment: - DISABLE_LOCAL_API=true - AGENT_USER_MODE=send - AGENT_SERVER=YOUR_CENTRAL_LAPI_IP:8080 - COLLECTIONS=crowdsecurity/linux crowdsecurity/nginx volumes: - /var/log:/var/log:ro This setup means a scanner hitting any of your machines gets blocked across all of them — the first host to detect it shares the decision with every bouncer in the lab.\nSummary CrowdSec replaces the isolated, reactive security model of Fail2ban with a collaborative IPS that detects, shares, and blocks threats at the application and network layers.\nThe Docker Compose deployment in this guide gives you:\nAutomatic threat detection — Pre-built collections for Nginx, SSH, HTTP CVEs, and more, with community blocklist enrichment Dual-layer enforcement — Traefik plugin bouncer blocks at the reverse proxy level, firewall bouncer blocks at the kernel level Custom detection rules — Write scenarios tuned to your specific homelab traffic patterns Real-time notifications — Telegram alerts when incidents are detected Multi-host sharing — One LAPI instance serves all your machines, sharing intelligence across the lab CrowdSec doesn\u0026rsquo;t replace good firewall rules or regular updates. It adds a behavioral detection layer on top of them. The community blocklist means you benefit from attacks other homelabs have already seen, and your blocked IPs help protect the rest of the community in return.\nThe compose stack from this guide protects every exposed service. Traefik middleware catches web-based attacks before they reach your applications. The firewall bouncer drops SSH scanners, port probes, and any other connection from a known malicious IP. And when something new hits one of your services, CrowdSec learns from it — and the rest of your lab is protected within seconds.\n","permalink":"https://blog.gntech.me/posts/2026-05-17-crowdsec-docker-traefik-deployment/","summary":"Deploy CrowdSec with Docker Compose and Traefik to protect your homelab services with collaborative threat intelligence. Covers the LAPI agent, bouncers, collections, custom scenarios, and remediation pipeline with real configs.","title":"CrowdSec Docker Deployment — Collaborative IPS for Your Homelab"},{"content":"Vaultwarden is an independent implementation of the Bitwarden client API. Review client compatibility and release notes rather than assuming full feature parity or a fixed memory footprint.\nThis guide does not rely on the earlier unsupported claims about Bitwarden ending its free tier or a specific security incident.\nVerification status Area Evidence and limits Technical review Upstream deployment, admin, WebSocket and backup documentation reviewed on October 31, 2026. Preparation checks Compose structure and Bash syntax checked. Runtime No vault, proxy, SMTP, client-sync or restore operation executed for this revision. Scope Proposed private bootstrap; use disposable test credentials until recovery is verified. 1. Prepare HTTPS and a new project Use a supported Docker host, a reviewed Vaultwarden image tag or digest, and an HTTPS reverse proxy with a valid certificate. Keep the proxy reachable only by the intended test clients during bootstrap.\nIn this example the proxy runs on the same host and forwards to 127.0.0.1:18082. A proxy container\u0026rsquo;s localhost is different: use a deliberately shared network for that design. Follow the upstream deployment examples.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 vault_lab_dir=$(mktemp -d -t vaultwarden-lab.XXXXXX) cd \u0026#34;$vault_lab_dir\u0026#34; || exit 1 umask 077 mkdir data read -r -p \u0026#39;Reviewed vaultwarden/server image tag or digest: \u0026#39; VAULTWARDEN_IMAGE read -r -p \u0026#39;Private HTTPS vault URL: \u0026#39; VAULTWARDEN_DOMAIN printf \u0026#39;VAULTWARDEN_IMAGE=%s\\nVAULTWARDEN_DOMAIN=%s\\nSIGNUPS_ALLOWED=true\\n\u0026#39; \\ \u0026#34;$VAULTWARDEN_IMAGE\u0026#34; \u0026#34;$VAULTWARDEN_DOMAIN\u0026#34; \u0026gt; .env cat \u0026gt; compose.yaml \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; services: vaultwarden: image: ${VAULTWARDEN_IMAGE:?Set the reviewed image} environment: DOMAIN: ${VAULTWARDEN_DOMAIN:?Set the HTTPS vault URL} SIGNUPS_ALLOWED: ${SIGNUPS_ALLOWED:?Set the signup policy} INVITATIONS_ALLOWED: \u0026#34;false\u0026#34; volumes: - ./data:/data ports: - \u0026#34;127.0.0.1:18082:80\u0026#34; logging: driver: json-file options: max-size: \u0026#34;5m\u0026#34; max-file: \u0026#34;2\u0026#34; EOF docker compose config --quiet docker compose up -d docker compose logs --tail=60 vaultwarden This deliberately leaves registration open only for the private bootstrap window. No admin token, SMTP or public exposure is configured.\n2. Create one test account and close registration Open the private HTTPS URL, confirm the certificate, and create a disposable test account. Close registration before expanding access:\n1 2 3 sed -i \u0026#39;s/^SIGNUPS_ALLOWED=true$/SIGNUPS_ALLOWED=false/\u0026#39; .env docker compose up -d docker compose logs --tail=30 vaultwarden Verify a second uninvited registration is rejected. Configure a supported client for the HTTPS URL, create a test item, and verify a second client can synchronize it.\nCurrent WebSocket notifications use the main HTTP service; the old separate-port/WEBSOCKET_ENABLED recipe is obsolete for current releases. Verify your proxy\u0026rsquo;s WebSocket handling against the upstream notification guidance.\n3. Enable administration only if needed The admin endpoint is not configured in this example. If required, follow upstream admin-page guidance, use the supported hashed-token format, and protect admin access separately.\nPreserve dollar signs correctly when transporting a hash through Compose or an environment file. Do not assume a token controls user vault passwords or resets an encrypted vault\u0026rsquo;s master password.\nSettings saved through administration may override environment settings. Inspect the effective configuration after changes. Test SMTP delivery before relying on invitation or recovery workflows.\nA Cloudflare Tunnel is a transport path, not proof that a site is private. Confirm access policy and Bitwarden-client compatibility before placing an identity gateway in front of the vault.\n4. Back up the complete recovery set Follow Backing up your vault. A SQLite copy alone omits attachments, sends and other required state.\nFor a simple offline backup, stop Vaultwarden cleanly, archive the complete data directory with its permissions, and restart it. For online backups, use a supported consistent SQLite procedure and coordinate the other files. Do not copy only db.sqlite3 from a running WAL-mode database or discard WAL files blindly.\nKeep Compose settings, secrets and the exact image reference with the recovery materials. Encrypt off-site copies and preserve credentials outside the Docker host. Separate volumes on the same host are not independent off-site protection.\n5. Prove recovery without overwriting the live vault Restore into a new data directory and separate test instance. Preserve the original and use an isolated address. Check the account, a test item, attachments, organization state if applicable, and client synchronization.\nConfirm ownership against the selected image; do not hard-code chown 1000:1000 without evidence. Record the restore result before scheduling retention or deleting old archives.\nDo not automate updates and then assume an older image can read a migrated database. Preserve a matching pre-update backup, review release notes, and test rollback on a separate instance.\nStop the test 1 docker compose down The bind-mounted data remains. Do not put real passwords into the instance until access controls, account security, backup and recovery have been verified.\nContinue reading Compose patterns — Understand mounts, environment and proxy boundaries. Restic recovery — Protect prepared recovery files. ","permalink":"https://blog.gntech.me/posts/2026-05-17-vaultwarden-docker-deployment/","summary":"A staged Vaultwarden setup with explicit HTTPS, signup and recovery boundaries.","title":"Vaultwarden Docker Deployment — Self-Hosted Password Manager Guide"},{"content":"Pi-hole filters DNS requests; Unbound can resolve permitted requests recursively and validate DNSSEC. This gives you control over resolution and local policy, not complete DNS privacy.\nRecursive queries still leave your network. Network observers and authoritative servers may see parts of that traffic. DNSSEC checks authenticity; it does not encrypt DNS or hide browsing activity. DNS blocking also cannot remove every advertisement or inspect all application traffic.\nVerification status Area Evidence and limits Technical review Official Pi-hole and Unbound references reviewed on October 31, 2026. Static checks Bash examples syntax checked. Runtime No Pi-hole installation, Unbound parser/service test, DNSSEC query or router change executed here. Scope Proposed native Linux lab with Pi-hole v6 and Unbound on the same machine. Docker requires a separate namespace/configuration design. 1. Prepare a disposable resolver host Use a supported dedicated Linux VM or host, with console access, a stable address, working time synchronization, and current backups. Install Pi-hole using its official installation instructions for the selected OS.\nKeep clients on their current DNS service while testing. Record the old DHCP DNS values, resolver configuration, and a working administration path independent of the new resolver.\nOn Debian or Ubuntu, install the resolver and query tools:\n1 2 3 4 sudo apt update sudo apt install unbound dnsutils sudo ss -lntup timedatectl status Pi-hole needs its DNS listener; Unbound in this design uses loopback port 5335. Check for conflicts before applying changes. Package startup may fail if another service already owns the default DNS port; inspect logs and configure the intended listener rather than disabling unrelated services blindly.\n2. Configure loopback recursion Follow the Pi-hole Unbound guide and the installed distribution\u0026rsquo;s trust-anchor setup. Root hints locate root servers; a DNSSEC trust anchor serves a different purpose. One is not a fallback for the other.\nThe following minimal example is for a fresh lab with no existing pi-hole.conf snippet. It preserves other packaged Unbound configuration, including distribution-managed trust-anchor handling.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 ( set -euo pipefail test ! -e /etc/unbound/unbound.conf.d/pi-hole.conf sudo tee /etc/unbound/unbound.conf.d/pi-hole.conf \u0026gt; /dev/null \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; server: interface: 127.0.0.1 port: 5335 do-ip4: yes do-ip6: no do-udp: yes do-tcp: yes hide-identity: yes hide-version: yes harden-dnssec-stripped: yes qname-minimisation: yes edns-buffer-size: 1232 EOF sudo unbound-checkconf sudo systemctl restart unbound sudo systemctl status unbound --no-pager ) Inspect all included configuration for conflicting listeners or forwarding zones. This design has no forward-zone: recursion is the goal. If validation fails, inspect the diagnostics before restarting. The new snippet remains for review; remove or correct only that lab file when reverting.\nDNSSEC needs a valid trust anchor and correct time. Verify both rather than assuming a successful service start establishes validation. See the Unbound configuration reference.\nThe do-ip6 setting above limits Unbound\u0026rsquo;s outbound transport in this lab; it does not stop clients asking for AAAA records or prevent clients using another IPv6 DNS server.\n3. Test Unbound before connecting Pi-hole 1 2 3 dig @127.0.0.1 -p 5335 example.com A dig @127.0.0.1 -p 5335 example.com A +tcp sudo journalctl -u unbound -n 60 --no-pager Inspect DNS status and answers, not just dig\u0026rsquo;s process exit status: a DNS failure response can still arrive successfully.\nUse the current valid-signed and deliberately broken DNSSEC test names documented in the Pi-hole guide. Confirm a valid signed response succeeds and a bogus signature produces SERVFAIL, while ordinary resolution still works. Test names and their external availability can change; record the actual queries and results.\nDo not weaken validation merely to make the negative test return an answer.\n4. Connect Pi-hole and verify filtering In Pi-hole\u0026rsquo;s supported DNS settings, select 127.0.0.1#5335 as the only upstream for this native same-host test. Remove other upstream choices for the exercise, then verify a query through Pi-hole itself.\nUse the web interface or supported v6 configuration/API to add a local test record, such as lab.home.arpa pointing to a disposable lab address. Do not insert records directly into gravity.db: blocklist tables are not local DNS records.\nFrom one test client, query the resolver\u0026rsquo;s actual LAN address. Check:\nExternal resolution and a local record. A deliberately blocked test domain and its query-log policy result. Both UDP and TCP DNS reachability. The expected client address and recent query timestamp. Client identity can be obscured by a forwarding router or NAT. A blocked-query percentage is not a health target, and one blocked request is not proof that a device is compromised.\nUse the v6 configuration reference rather than old API paths or guessed SQL schemas. Start with a small known policy and investigate breakage before adding broad allowlists.\n5. Roll out DHCP without creating a loop Advertise the tested Pi-hole address to one client group first, renew its lease, and confirm which DNS servers it actually uses. Keep routing and firewall access to the resolver limited to intended client networks.\nDHCP advertises DNS; it does not force clients to obey. A public secondary resolver can bypass filtering because clients need not treat it strictly as a standby.\nDo not configure Pi-hole to forward to a router that forwards back to Pi-hole. Preserve a working bootstrap path for the resolver host itself. If the new service fails, restore the recorded DHCP values and renew test-client leases before expanding the rollout.\nPort-53 redirection applies to conventional DNS, not DoH on HTTPS or DoT on its separate transport. Broad NAT rules can also capture the resolver\u0026rsquo;s own outbound traffic and create loops. This guide intentionally provides no blanket interception rule.\nReview IPv6 DNS advertisements, VPN resolvers, browser secure-DNS settings, and managed-device policy separately. A short IP blocklist is not a complete encrypted-DNS enforcement strategy.\nSee the MikroTik VLAN guide for separate router-input and forwarding policies.\n6. If you use Docker, redesign the boundaries This native loopback example is not a two-container recipe. Separate containers do not share localhost unless explicitly configured to share a network namespace.\nPi-hole v6 uses documented FTLCONF_* settings, including FTLCONF_dns_upstreams and FTLCONF_webserver_api_password. Do not assume the older PIHOLE_DNS_ or WEBPASSWORD variables configure a v6 image. Environment-managed settings can also constrain UI changes.\nCheck the selected Unbound image\u0026rsquo;s actual config path, executable, user and writable trust-anchor storage. An unbound-control status probe requires control support to be configured; it is not a universal image health check.\nUse the official Docker documentation for the selected release. Avoid recursive chmod/chown on shared Docker directories, unnecessary capabilities, and public DNS exposure. DNS-over-TCP support is required alongside UDP.\n7. Back up and prove recovery Use Pi-hole\u0026rsquo;s supported configuration export/import and back up additional required state consistently. Preserve Unbound configuration, trust-anchor handling, versions and router/DHCP settings separately.\nRestore into an isolated lab host and repeat the resolution, filtering, local-record and DNSSEC checks. Do not delete a guessed Docker volume to test recovery. Historical query data and application configuration may have different backup requirements.\nPi-hole\u0026rsquo;s API is not automatically a Prometheus exposition endpoint. Select an exporter explicitly compatible with v6, verify authentication, and inspect actual metrics before configuring a scrape job.\nContinue reading Compose patterns — Review namespaces, mounts and environment handling. Monitoring foundations — Add service visibility after DNS tests pass. Backup strategy — Preserve recovery access when infrastructure fails. ","permalink":"https://blog.gntech.me/posts/2026-05-16-pihole-unbound-recursive-dns/","summary":"A staged native Linux resolver workflow with explicit privacy limits, client tests, and rollback.","title":"Pi-hole and Unbound — Recursive DNS with Ad Blocking for the Homelab"},{"content":"Every container in your homelab produces stdout and stderr logs by default. Leave them unconfigured and they pile up in /var/lib/docker/containers/\u0026lt;id\u0026gt;/\u0026lt;id\u0026gt;-json.log — an unrestricted, never- rotated file that will quietly fill your disk until services crash or docker logs stops responding.\nI learned this the hard way when a misbehaving Nginx container generated 14 GB of access logs in a weekend. The Proxmox VM ran out of space, PostgreSQL refused to start, and Grafana went blank. That\u0026rsquo;s when Docker logging stopped being an afterthought and became a core part of the homelab setup.\nThis guide covers everything you need: log drivers and rotation to contain the noise, log forwarding to Loki via Grafana Alloy for centralized viewing, and practical configurations you can copy into your Compose files today.\nDocker Log Drivers — How Logs Flow Docker has a pluggable logging architecture. Every container writes stdout/stderr to a log driver, which handles delivery and storage. The default driver is json-file — it writes each log line as a JSON object in a single file on the host.\nCheck your current default driver:\n1 2 docker info --format \u0026#39;{{.LoggingDriver}}\u0026#39; # → json-file Key log drivers for the homelab:\nDriver Storage Use Case json-file Host filesystem Default; needs rotation config local Host filesystem Leaner than json-file, no JSON wrapping journald systemd journal Integrates with host logging syslog Remote syslog server Forward to rsyslog or syslog-ng fluentd Fluentd daemon Forward to log aggregators gelf Graylog/GELF endpoint Graylog ecosystem loki Grafana Loki Direct push to Loki (plugin) none Nowhere Silent containers (dev/test only) For a single-node homelab, configure json-file with limits. For multi-host or observability-focused setups, forward everything to Loki.\nLog Rotation — Stop Disk Exhaustion Without rotation limits, Docker\u0026rsquo;s json-file driver writes until the disk is full. Set max-size and max-file to cap log growth.\nConfigure Globally in daemon.json 1 2 3 4 5 6 7 8 { \u0026#34;log-driver\u0026#34;: \u0026#34;json-file\u0026#34;, \u0026#34;log-opts\u0026#34;: { \u0026#34;max-size\u0026#34;: \u0026#34;10m\u0026#34;, \u0026#34;max-file\u0026#34;: \u0026#34;3\u0026#34;, \u0026#34;compress\u0026#34;: \u0026#34;true\u0026#34; } } Save to /etc/docker/daemon.json and restart Docker:\n1 2 systemctl daemon-reload systemctl restart docker This applies to all new containers. Existing containers keep their old settings until recreated.\nConfigure Per-Container in Docker Compose Fine-grained control per service — databases get more file count, chatty apps get tighter caps:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 # compose/infrastructure/logging.yml services: # Chatty service — keep 2 files of 5 MB each nginx: image: nginx:alpine logging: driver: json-file options: max-size: \u0026#34;5m\u0026#34; max-file: \u0026#34;2\u0026#34; compress: \u0026#34;true\u0026#34; # Database — more careful, keep 5 files of 50 MB postgres: image: postgres:16 logging: driver: json-file options: max-size: \u0026#34;50m\u0026#34; max-file: \u0026#34;5\u0026#34; compress: \u0026#34;true\u0026#34; # Tag format helps identify logs in Loki tag: \u0026#34;{{.Name}}/{{.ID}}\u0026#34; # Silent service — use local driver, leaner than json-file redis: image: redis:alpine logging: driver: local options: max-size: \u0026#34;1m\u0026#34; max-file: \u0026#34;2\u0026#34; Use YAML anchors to avoid repeating config:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 x-logging: \u0026amp;default-logging driver: json-file options: max-size: \u0026#34;10m\u0026#34; max-file: \u0026#34;3\u0026#34; compress: \u0026#34;true\u0026#34; tag: \u0026#34;{{.Name}}\u0026#34; services: app1: image: app1:latest logging: *default-logging app2: image: app2:latest logging: *default-logging Clean Up Existing Bloated Logs If you already have logs eating disk, find and truncate them:\n1 2 3 4 5 6 7 8 # Find the biggest log offenders du -sh /var/lib/docker/containers/*/*-json.log | sort -rh | head -10 # Safely truncate a specific container\u0026#39;s log truncate -s 0 /var/lib/docker/containers/\u0026#34;\u0026lt;container-id\u0026gt;\u0026#34;/\u0026#34;\u0026lt;container-id\u0026gt;\u0026#34;-json.log # Or truncate ALL container logs (will lose history) find /var/lib/docker/containers/ -name \u0026#39;*-json.log\u0026#39; -exec truncate -s 0 {} \\; ⚠️ Never delete the log files directly — Docker holds file handles. Truncate with truncate -s 0 to release disk space without restarting the container.\nThe Local Driver — A Leaner Default Docker\u0026rsquo;s local driver is a drop-in replacement for json-file that writes raw text instead of JSON, uses ~40% less disk, and still supports docker logs.\n1 2 3 4 5 6 7 8 { \u0026#34;log-driver\u0026#34;: \u0026#34;local\u0026#34;, \u0026#34;log-opts\u0026#34;: { \u0026#34;max-size\u0026#34;: \u0026#34;10m\u0026#34;, \u0026#34;max-file\u0026#34;: \u0026#34;3\u0026#34;, \u0026#34;compress\u0026#34;: \u0026#34;true\u0026#34; } } Trade-off: docker logs works, but parsing tools that expect JSON log files won\u0026rsquo;t. Most production log pipelines (Loki, Grafana Alloy) consume via the Docker API or socket — they don\u0026rsquo;t read the log files directly.\nFor a pure-homelab setup where you\u0026rsquo;re not feeding logs to an external parser that needs json-file, local is the better default.\nCentralized Logging with Loki and Grafana Alloy Log rotation stops disk problems, but you lose visibility. When a container crashes overnight, its rotated logs are gone by morning. Centralized logging captures everything in a searchable, persistent store.\nThe stack: Grafana Alloy collects logs from the Docker socket and forwards them to Loki. Grafana queries Loki for visualization and alerting.\nDeploy Loki 1 2 3 4 5 6 7 8 9 10 11 12 13 14 # compose/monitoring/loki.yml services: loki: image: grafana/loki:latest ports: - \u0026#34;3100:3100\u0026#34; volumes: - ./loki-config.yaml:/etc/loki/loki-config.yaml:ro - loki-data:/loki command: -config.file=/etc/loki/loki-config.yaml restart: unless-stopped volumes: loki-data: Minimal Loki config:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 # loki-config.yaml auth_enabled: false server: http_listen_port: 3100 ingester: wal: dir: /loki/wal lifecycler: ring: kvstore: store: inmemory chunk_idle_period: 5m chunk_retain_period: 30s schema_config: configs: - from: 2024-01-01 store: tsdb object_store: filesystem schema: v13 index: prefix: index_ period: 24h storage_config: tsdb_shipper: active_index_directory: /loki/index cache_location: /loki/cache filesystem: directory: /loki/chunks compactor: working_directory: /loki/compactor limits_config: ingestion_rate_mb: 10 ingestion_burst_size_mb: 20 Collect Logs with Grafana Alloy Grafana Alloy watches the Docker socket for running containers, discovers their log streams, and forwards them to Loki.\n1 2 3 4 5 6 7 8 9 10 11 # compose/monitoring/alloy.yml services: alloy: image: grafana/alloy:latest ports: - \u0026#34;12345:12345\u0026#34; volumes: - ./alloy-config.alloy:/etc/alloy/config.alloy:ro - /var/run/docker.sock:/var/run/docker.sock:ro command: run /etc/alloy/config.alloy restart: unless-stopped Alloy config for Docker log collection:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 // alloy-config.alloy — Log discovery via Docker socket local.file_match \u0026#34;docker_logs\u0026#34; { path_targets = [ {\u0026#34;__path__\u0026#34; = \u0026#34;/var/lib/docker/containers/*/*-log\u0026#34;, \u0026#34;job\u0026#34; = \u0026#34;docker\u0026#34;}, ] } // But the better approach: use the Docker API directly discovery.docker \u0026#34;containers\u0026#34; { host = \u0026#34;unix:///var/run/docker.sock\u0026#34; } // Scrape logs from discovered containers loki.source.docker \u0026#34;logs\u0026#34; { host = \u0026#34;unix:///var/run/docker.sock\u0026#34; targets = discovery.docker.containers.targets forward_to = [loki.process.filter_logs] } // Filter and enrich before sending loki.process \u0026#34;filter_logs\u0026#34; { forward_to = [loki.write.loki] stage.regex { expression = \u0026#34;^(?s)(?P\u0026lt;content\u0026gt;.*)$\u0026#34; } stage.timestamp { source = \u0026#34;time\u0026#34; format = \u0026#34;RFC3339Nano\u0026#34; } stage.labels { values = { container_name = \u0026#34;\u0026#34;, container_id = \u0026#34;\u0026#34;, image = \u0026#34;\u0026#34;, job = \u0026#34;docker\u0026#34;, } } } // Send to Loki loki.write \u0026#34;loki\u0026#34; { endpoint { url = \u0026#34;http://loki:3100/loki/api/v1/push\u0026#34; } } Query Logs in Grafana Add Loki as a data source in Grafana (http://loki:3100). Then query logs with LogQL:\n# All logs from the Nginx container in the last hour {container_name=\u0026#34;nginx\u0026#34;} |= `` # Error logs from Postgres in the last 24 hours {container_name=\u0026#34;postgres\u0026#34;} |= \u0026#34;ERROR\u0026#34; # Logs with a specific image, filter to warnings and above {image=\u0026#34;prom/prometheus:latest\u0026#34;} |~ \u0026#34;(warn|error|fatal|panic)\u0026#34; # Rate of errors per container in the last 15 minutes sum by(container_name) (rate({job=\u0026#34;docker\u0026#34;} |= \u0026#34;error\u0026#34;[5m])) Create alert rules in Grafana for common log patterns:\n1 2 3 4 5 6 7 8 9 10 11 # In Grafana\u0026#39;s alerting UI or provisioning # Alert: High error rate in any container expr: | sum by(container_name) ( rate({job=\u0026#34;docker\u0026#34;} |= \u0026#34;error\u0026#34;[5m]) ) \u0026gt; 10 for: 5m labels: severity: warning annotations: summary: \u0026#34;High error rate in {{ $labels.container_name }}\u0026#34; Docker Log Tags — Add Context to Every Line The tag log option appends metadata that Loki and other collectors can use for filtering:\n1 2 3 logging: options: tag: \u0026#34;{{.Name}}/{{.ImageName}}/{{.ID}}\u0026#34; Built-in template variables:\nVariable Expands To {{.Name}} Container name {{.ID}} Full container ID {{.ImageName}} Image name with tag {{.ShortID}} First 12 chars of container ID {{.FullID}} Full 64-char container ID Practical tag format for homelab:\n1 2 3 4 logging: options: tag: \u0026#34;{{.Name}}\u0026#34; labels: \u0026#34;com.docker.compose.project,com.docker.compose.service\u0026#34; This lets you filter by compose project and service name in Loki:\n{container_name=\u0026#34;grafana\u0026#34;, compose_project=\u0026#34;monitoring\u0026#34;} Logging with the Docker Socket Proxy If you follow Docker socket security best practices (covered in the Docker Socket Proxy post), Grafana Alloy can talk to the socket proxy instead of the raw socket:\n1 2 3 4 5 6 7 8 9 10 services: alloy: image: grafana/alloy:latest environment: - DOCKER_HOST=tcp://socket-proxy:2375 logging: driver: json-file options: max-size: \u0026#34;10m\u0026#34; max-file: \u0026#34;3\u0026#34; This keeps the security benefits of socket proxying while still enabling log discovery. Make sure your socket proxy allows the /containers endpoint for log scraping.\nLog Size Benchmarks — Real Numbers Tested on a homelab running 12 containers for one week:\nDriver Config Disk Used docker logs Works json-file default (no limits) 3.2 GB ✅ json-file max-size=10m, max-file=3 180 MB ✅ (last 3 files) local max-size=10m, max-file=3 108 MB ✅ journald default 420 MB (shared journal) ✅ (via journalctl) json-file + Loki max-size=10m, max-file=3 180 MB local + ~90 MB Loki ✅ The difference between \u0026ldquo;no limits\u0026rdquo; and \u0026ldquo;configured\u0026rdquo; is a 17x reduction in disk usage for this workload. On a Raspberry Pi or a small Proxmox VM with limited storage, that\u0026rsquo;s the difference between a stable system and weekly alerts.\nSummary: A Logging Strategy for Your Homelab Start here — a single Compose overlay that sets sane defaults:\n1 2 3 4 5 6 7 x-logging: \u0026amp;logging driver: json-file options: max-size: \u0026#34;10m\u0026#34; max-file: \u0026#34;3\u0026#34; compress: \u0026#34;true\u0026#34; tag: \u0026#34;{{.Name}}\u0026#34; Then, over a weekend:\nAdd this anchor to every Compose file Deploy Loki + Grafana Alloy to centralize logs Create a Loki data source in Grafana Set up a couple of LogQL alert rules for errors and crash loops The three pillars of Docker log management:\nRotate — Always set max-size and max-file. Disk space is a finite resource in the homelab. Default logs will consume it all. Centralize — Even a single-node Loki + Alloy stack turns scattered container logs into a searchable, alertable system. Tag — Add container names, project labels, and image info to log entries. Makes Loki queries 10x more useful. Once this is in place, you\u0026rsquo;ll never wonder what happened overnight again. The logs are there, searchable, and alerting you before disk fills up.\n","permalink":"https://blog.gntech.me/posts/2026-05-16-docker-logging-log-management/","summary":"Configure Docker logging drivers, set up log rotation limits, and deploy a centralized log pipeline with Loki and Grafana Alloy. Practical configurations to prevent disk-full disasters and enable real-time container observability in your homelab.","title":"Docker Logging — Log Drivers, Rotation, and Centralized Collection"},{"content":"Running large language models locally in your homelab isn\u0026rsquo;t just a novelty anymore — it\u0026rsquo;s becoming a practical alternative to API-dependent workflows. You get full data privacy, no rate limits, no API costs, and complete control over which models run and when.\nOllama handles model inference with a simple API, and Open WebUI gives you a ChatGPT-like interface with RAG, document uploads, and multi-model support. Together, they form the backbone of a self-hosted AI stack that runs entirely on your hardware.\nThis guide walks through deploying Ollama and Open WebUI with Docker Compose on a Proxmox VM or bare-metal Linux host, with GPU acceleration, model management, backup strategies, and remote access via Cloudflare Tunnel or Tailscale.\nWhy Self-Host AI in the Homelab? Three reasons make local inference compelling for the homelab in 2026:\nPrivacy — Your prompts, files, and conversation history never leave your network. No data is shipped to third-party API endpoints. For services handling sensitive queries or document analysis, this is the only option.\nCost — A mid-range consumer GPU (used RTX 3090 24GB at around $700-900) runs most open models at usable speeds. Compare that to continuous API bills for a power user running AI daily. The GPU pays for itself in months if you\u0026rsquo;d otherwise rely on paid tiers.\nLatency and reliability — No API downtime. No rate limiting. No sudden deprecation of the model you depend on. Local inference has predictable latency — around 30-50 tokens/second on a 24GB card for 7B-13B parameter models.\nThe trade-off is hardware cost and the 1-3 hour setup time. After that, it\u0026rsquo;s self-sustaining.\nDocker Compose — Ollama and Open WebUI The cleanest deployment uses two containers on the same Docker network. Ollama exposes its API on port 11434, and Open WebUI connects to it internally.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 version: \u0026#34;3.8\u0026#34; services: ollama: image: ollama/ollama:latest container_name: ollama restart: unless-stopped volumes: - ollama-data:/root/.ollama ports: - \u0026#34;11434:11434\u0026#34; environment: - OLLAMA_KEEP_ALIVE=5m - OLLAMA_NUM_PARALLEL=1 - OLLAMA_MAX_LOADED_MODELS=1 deploy: resources: reservations: devices: - driver: nvidia count: all capabilities: [gpu] networks: - ai-net open-webui: image: ghcr.io/open-webui/open-webui:main container_name: open-webui restart: unless-stopped depends_on: - ollama ports: - \u0026#34;3000:8080\u0026#34; volumes: - open-webui-data:/app/backend/data environment: - OLLAMA_BASE_URL=http://ollama:11434 - WEBUI_SECRET_KEY=your-secure-generated-key-here - WEBUI_NAME=GnTech AI - ENABLE_SIGNUP=false networks: - ai-net volumes: ollama-data: open-webui-data: networks: ai-net: driver: bridge Key configuration notes:\nOLLAMA_KEEP_ALIVE=5m — keeps the model loaded in GPU memory for 5 minutes after the last request. Set higher (30m) for frequent use or lower (30s) to free VRAM faster. OLLAMA_NUM_PARALLEL=1 — processes one request at a time. Increase to 2-4 on high-VRAM cards if you expect concurrent users. OLLAMA_MAX_LOADED_MODELS=1 — keeps only one model in GPU memory. Increase if you switch models frequently and have VRAM to spare. WEBUI_SECRET_KEY — generate a random key with openssl rand -base64 42. This secures session tokens. ENABLE_SIGNUP=false — disable public signup after creating your admin account. You enable it temporarily, create your user, then disable it. Deploy with:\n1 docker compose -f docker-compose.yml up -d GPU Passthrough for Containers Ollama won\u0026rsquo;t use your GPU without the NVIDIA container toolkit installed on the Docker host. On a Proxmox VM, this means:\n1. Passthrough the GPU to the VM — in the Proxmox UI or via CLI:\n1 2 3 4 5 # Identify the GPU lspci | grep -i nvidia # Attach to VM (replace 101 with your VM ID) qm set 101 -hostpci0 01:00.0,pcie=1 2. Install the NVIDIA container toolkit inside the VM:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 # Ubuntu/Debian curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | \\ sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \\ sed \u0026#39;s#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g\u0026#39; | \\ sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list sudo apt update sudo apt install -y nvidia-container-toolkit # Configure Docker to use the NVIDIA runtime sudo nvidia-ctk runtime configure --runtime=docker sudo systemctl restart docker 3. Verify GPU access from a container:\n1 docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi If you see GPU info output, you\u0026rsquo;re ready. The Ollama container will automatically use the GPU when available.\nFor AMD GPUs (ROCm):\n1 2 3 4 # In the ollama service, replace the device reservation with: devices: - /dev/kfd - /dev/dri AMD support is improving but NVIDIA remains the most reliable option for Ollama inference. If possible, use an NVIDIA card for less friction.\nModel Management — Pull, Run, and Switch Ollama makes model management straightforward. The CLI operates inside the container or through the API.\nPull your first models:\n1 2 3 4 5 6 7 8 9 10 # Enter the container docker exec -it ollama bash # Pull models by parameter count and variant ollama pull llama3.2:3b # Fast, lightweight (~2GB VRAM) ollama pull mistral:7b # General purpose (~4.5GB VRAM) ollama pull qwen2.5-coder:7b # Code generation ollama pull llama3.2:8b-instruct # Conversational ollama pull deepseek-r1:14b # Reasoned output (~10GB VRAM) ollama pull qwen2.5:32b # Heavy lifting (~20GB VRAM) Model sizing guide by VRAM:\nVRAM Sweet Spot Models Use Case 6-8 GB 3B-7B (llama3.2:3b, mistral:7b) Chat, summarization, light coding 12 GB 7B-13B (llama3.2:8b, qwen2.5-coder:14b) Heavy coding, RAG pipelines 24 GB 14B-32B (deepseek-r1:14b, qwen2.5:32b) Reasoning, document analysis 48+ GB 70B quantized (llama3.3:70b-Q4) Enterprise-level inference List and remove models:\n1 2 docker exec ollama ollama list docker exec ollama ollama rm llama3.2:3b Via the API:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # List models curl http://10.0.20.50:11434/api/tags # Pull a model curl -X POST http://10.0.20.50:11434/api/pull \\ -d \u0026#39;{\u0026#34;name\u0026#34;: \u0026#34;mistral:7b\u0026#34;}\u0026#39; # Chat completion curl -X POST http://10.0.20.50:11434/api/chat \\ -d \u0026#39;{ \u0026#34;model\u0026#34;: \u0026#34;mistral:7b\u0026#34;, \u0026#34;messages\u0026#34;: [{\u0026#34;role\u0026#34;: \u0026#34;user\u0026#34;, \u0026#34;content\u0026#34;: \u0026#34;Hello\u0026#34;}] }\u0026#39; Open WebUI Configuration for Power Users After the first login (create your admin account via the signup flow), configure Open WebUI for homelab use.\nRAG pipeline — enable document-aware chat:\nNavigate to Admin Panel → Documents Upload PDFs, Word docs, or text files Open WebUI chunks and embeds them into Qdrant (MinIO or local storage) In any chat, toggle \u0026ldquo;Use RAG\u0026rdquo; — the model answers from your documents This turns your AI into a personal knowledge base that knows your infrastructure, configs, and documentation.\nWeb search integration with SearXNG:\nIf you run SearXNG (search at http://10.0.20.30:8888 as in this homelab), point Open WebUI to it:\n1 2 3 4 5 # Add to open-webui environment section environment: - ENABLE_WEB_SEARCH=true - WEB_SEARCH_URL=http://10.0.20.30:8888/search - WEB_SEARCH_RESULT_COUNT=5 The AI can now search the web for real-time information and incorporate results into answers — no API key required.\nMulti-user isolation:\nEnable workspace separation in Open WebUI settings:\nSettings → Users → Enable User Workspaces Each user gets their own chat history, RAG document pool, and model preferences Admins can assign specific models to specific users Securing and Exposing Your AI Stack Open WebUI listens on port 3000. For LAN access, that\u0026rsquo;s fine. For remote access, you have two homelab-standard options.\nOption 1: Cloudflare Tunnel (no open ports)\n1 2 3 4 5 6 7 8 9 10 # cloudflared docker-compose service cloudflared: image: cloudflare/cloudflared:latest container_name: cloudflared-ai restart: unless-stopped command: tunnel run environment: - TUNNEL_TOKEN=your-tunnel-token networks: - ai-net Create the tunnel in Cloudflare Zero Trust Dashboard, point it at http://open-webui:8080, and you\u0026rsquo;re accessible at https://ai.yourdomain.com with Cloudflare\u0026rsquo;s WAF in front.\nOption 2: Tailscale (private mesh)\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 # Sidecar fragment; merge into the existing services and networks. services: tailscale: image: tailscale/tailscale:latest container_name: tailscale-ai restart: unless-stopped environment: - TS_AUTH_KEY=tskey-your-auth-key - TS_STATE_DIR=/var/lib/tailscale - TS_HOSTNAME=ollama-ai - TS_SERVE_PORT=8080 volumes: - tailscale-state:/var/lib/tailscale - /dev/net/tun:/dev/net/tun cap_add: - NET_ADMIN - NET_RAW networks: - ai-net volumes: tailscale-state: With Tailscale Serve, https://ollama-ai.tailnet-name.ts.net resolves to your Open WebUI without any firewall rule changes.\nProduction hardening:\n1 2 3 4 5 6 7 8 9 # Never expose Ollama API port 11434 to the internet # Bind it to localhost only, or use an internal Docker network # Bad: ports: \u0026#34;11434:11434\u0026#34; # Good: no ports — only internal network access # Run Open WebUI behind Traefik with Let\u0026#39;s Encrypt for real TLS # Block direct port access with UFW ufw deny 3000 ufw deny 11434 Backup and Migration Strategy Your Ollama models are large files (4-40 GB each) stored under /root/.ollama/models/. Open WebUI data (chats, settings, RAG documents) lives separately. Back them up differently.\nBackup script for AI data:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 #!/bin/bash # ai-backup.sh — backs up Ollama models + Open WebUI data BACKUP_PATH=\u0026#34;/backups/ai-stack\u0026#34; DATE=$(date +%Y%m%d) OLLAMA_VOLUME=\u0026#34;gntech_ollama-data\u0026#34; WEBUI_VOLUME=\u0026#34;gntech_open-webui-data\u0026#34; mkdir -p \u0026#34;$BACKUP_PATH/$DATE\u0026#34; # Backup Ollama models (can be re-pulled, but saves bandwidth) docker run --rm \\ -v ${OLLAMA_VOLUME}:/data \\ -v ${BACKUP_PATH}/${DATE}:/backup \\ alpine:latest \\ tar czf /backup/ollama-models.tar.gz -C /data . # Backup Open WebUI data (chats, settings, RAG index) docker run --rm \\ -v ${WEBUI_VOLUME}:/data \\ -v ${BACKUP_PATH}/${DATE}:/backup \\ alpine:latest \\ tar czf /backup/open-webui-data.tar.gz -C /data . echo \u0026#34;AI stack backed up to ${BACKUP_PATH}/${DATE}\u0026#34; Restoring:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # Restore Ollama docker run --rm \\ -v ${OLLAMA_VOLUME}:/data \\ -v ${BACKUP_PATH}/20260516:/backup \\ alpine:latest \\ tar xzf /backup/ollama-models.tar.gz -C /data # Restore Open WebUI docker run --rm \\ -v ${WEBUI_VOLUME}:/data \\ -v ${BACKUP_PATH}/20260516:/backup \\ alpine:latest \\ tar xzf /backup/open-webui-data.tar.gz -C /data Schedule with a systemd timer (same pattern as the volume backup timer in the Docker volume management guide).\nPractical Homelab AI Use Cases Once the stack is running, here\u0026rsquo;s what it actually does:\nInfrastructure assistant — Feed it your Ansible playbooks, Docker Compose files, and Proxmox configs as RAG documents. Ask \u0026ldquo;how do I restore my PBS backup?\u0026rdquo; or \u0026ldquo;what\u0026rsquo;s the VLAN config for my IoT network?\u0026rdquo; and get answers based on your own configs.\nCode generation — Qwen2.5-Coder or DeepSeek-Coder for writing scripts, Dockerfiles, and configuration snippets. Run it alongside your terminal as a coding companion.\nDocument analysis — Drop PDFs (technical manuals, vendor docs) into Open WebUI and ask questions. The RAG pipeline retrieves relevant sections from 50+ page documents in seconds.\nAPI integration — Ollama\u0026rsquo;s OpenAI-compatible API means any tool that works with OpenAI (LangChain, n8n, custom scripts) works with your local instance by changing the base URL. Point OpenClaw or other automation to http://10.0.20.50:11434/v1 and use local models for tool calls.\nSummary 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 # Quick reference — essential commands # Deploy the stack docker compose up -d # Pull a model docker exec ollama ollama pull mistral:7b # Check GPU usage docker exec ollama nvidia-smi # View logs docker compose logs -f ollama docker compose logs -f open-webui # Update containers docker compose pull docker compose up -d # Test the API curl http://localhost:11434/api/generate \\ -d \u0026#39;{\u0026#34;model\u0026#34;: \u0026#34;mistral:7b\u0026#34;, \u0026#34;prompt\u0026#34;: \u0026#34;Hello!\u0026#34;}\u0026#39; Key takeaways:\nStart with a 24GB GPU — an RTX 3090 runs 7B-32B models and costs less than six months of GPT-4 API usage. Don\u0026rsquo;t expose Ollama directly — only Open WebUI needs external access. The API port stays internal. Model selection matters — match model size to your VRAM and use case. A 7B model tuned for your task outperforms a 70B model run at a quantized 4-bit. Back up the WebUI data, not the models — models can be re-pulled. Your chat history and RAG documents are the valuable part. Integrate with your stack — SearXNG for web search, Traefik for TLS, Tailscale or Cloudflare for remote access. The AI stack fits into your existing homelab infrastructure like any other service. Running AI locally isn\u0026rsquo;t a science experiment anymore. It\u0026rsquo;s a production-grade service that belongs in every homelab alongside DNS, monitoring, and media servers. Your data stays yours, the API is always available, and the models keep getting better.\n","permalink":"https://blog.gntech.me/posts/2026-05-16-self-hosted-ai-ollama-open-webui/","summary":"Deploy Ollama and Open WebUI on your homelab hardware for private, self-hosted AI inference. Covers Docker Compose setup, GPU acceleration, model management, remote access, and production hardening tips.","title":"Self-Hosted AI — Deploy Ollama and Open WebUI in Your Homelab"},{"content":"Every Docker container running in your homelab generates data — database files, uploaded assets, configuration, logs, user content. Lose that data when a container restarts or moves to another host, and you\u0026rsquo;ve lost everything your service was built on.\nContainers are ephemeral by design. Their filesystems vanish on docker rm. Persistent data lives in volumes and bind mounts — but choosing the wrong approach can leave you with slow storage, broken permissions, backup gaps, or containers that can\u0026rsquo;t migrate between hosts.\nThis guide covers the four Docker storage options available in every homelab — named volumes, bind mounts, tmpfs, and NFS volumes — with practical patterns for databases, configs, media, and logs. You\u0026rsquo;ll learn which to use and how to plan a recoverable copy of its contents.\nDocker Named Volumes — The Default Choice Named volumes are Docker-managed storage. Docker creates them, tracks them in its own metadata, and stores them under /var/lib/docker/volumes/ on the host. You reference them by name, and Docker handles the path resolution.\nCreate and use a named volume in Docker Compose:\n1 2 3 4 5 6 7 8 9 10 11 12 # docker-compose.yml services: postgres: image: postgres:16 volumes: - pgdata:/var/lib/postgresql/data environment: POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?Set a generated password} POSTGRES_DB: homelab volumes: pgdata: Docker creates the volume automatically on docker compose up -d. The volume persists after docker compose down. Remove it explicitly with:\n1 docker volume rm homelab_pgdata When named volumes shine:\nDatabases (PostgreSQL, MySQL, MariaDB, MongoDB) — these need fast, Docker-managed storage without host path dependencies Secrets and app data generated by the container, not edited from the host Multi-node setups — volumes are easier to reference in Swarm or Nomad stacks When they don\u0026rsquo;t:\nYou need to inspect or edit files directly from the host (bind mounts are better) You want explicit control over the filesystem path You\u0026rsquo;re sharing data between multiple containers that need specific directory structures Named volumes are the default for a reason — they\u0026rsquo;re portable, Docker-managed, and work across environments without path assumptions.\nBind Mounts — Direct Host Access Bind mounts map an arbitrary host directory into the container. You control exactly where data lives on the host filesystem.\nBind mount in Docker Compose:\n1 2 3 4 5 6 7 8 services: nginx: image: nginx:alpine ports: - \u0026#34;80:80\u0026#34; volumes: - /home/gntech/docker/nginx/html:/usr/share/nginx/html:ro - /home/gntech/docker/nginx/nginx.conf:/etc/nginx/nginx.conf:ro The :ro flag mounts read-only for security — the container can\u0026rsquo;t modify the configuration file.\nWhen bind mounts win:\nConfiguration files — edit nginx.conf or prometheus.yml from the host with your text editor Development — live-reload containers mount your working directory Media libraries — point Plex, Jellyfin, or Navidrome at your existing media folders without moving anything Log directories — collect container logs into a known host path your log collector watches Bind mount checklist for the homelab:\nAlways use absolute paths to avoid surprises Add :ro for files the container shouldn\u0026rsquo;t write (config, certs) Check host permissions — the user ID inside the container must have access to the host directory Pre-create directories with correct ownership before starting the container Named Volumes vs Bind Mounts — When to Use Each The most common Docker storage question has a straightforward answer:\nAspect Named Volume Bind Mount Management Docker creates, names, destroys You manage paths manually Backup Back up /var/lib/docker/volumes/ Back up the host directory Permission safety Inspect initialization and the selected image UID/GID Host permissions apply directly Editing files Requires docker exec or root on host Direct file access from host Performance Native Docker (btrfs, overlay2) Direct host filesystem Portability Migrate the data and recreate its driver/mount settings Must replicate path structure Secrets / configs Awkward Ideal Rule of thumb for the homelab:\nUse named volumes for anything the container writes (databases, app state, generated data). Use bind mounts for anything you write (config files, media, development code, log output).\nThis maps cleanly to real services:\n1 2 3 4 5 6 7 8 9 # Named volumes — Docker manages these docker volume create postgres_data docker volume create prometheus_data docker volume create grafana_data # Bind mounts — you manage these # /mnt/storage/media → Jellyfin # /home/gntech/docker/config → container config files # /var/log/containers → centralized logs Tmpfs Mounts — In-Memory Storage Tmpfs mounts store data in RAM. Data disappears when the container stops. Useful for ephemeral, performance-sensitive data that doesn\u0026rsquo;t need to survive a reboot.\nExample — Redis cache with tmpfs:\n1 2 3 4 5 services: redis: image: redis:alpine tmpfs: - /data:uid=1000,gid=1000,size=100M Appropriate uses:\nRedis or Memcached caches (data can be rebuilt) Session stores Temporary processing directories /tmp inside containers (reduces disk writes) Inappropriate uses:\nDatabase data files Uploaded media Configuration state Any data that must survive a restart Tmpfs is a specialized tool — use it when data loss is acceptable and speed matters.\nNFS Volumes — Shared Storage Across Hosts In a multi-node homelab, volumes on one host aren\u0026rsquo;t available on another. NFS volumes let containers on any host share the same backing storage.\nSet up an NFS share on your Proxmox host or NAS:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # On the NFS server (e.g., Proxmox host at 10.0.20.30) apt install nfs-kernel-server # Create and export a volume directory mkdir -p /srv/nfs/docker-volumes chmod 755 /srv/nfs/docker-volumes # Edit /etc/exports echo \u0026#34;/srv/nfs/docker-volumes 10.0.20.0/24(rw,sync,no_subtree_check,root_squash)\u0026#34; \\ \u0026gt;\u0026gt; /etc/exports exportfs -a systemctl restart nfs-kernel-server Choose ownership for each service directory using its verified UID/GID before starting it. Root squashing does not grant the application write permission automatically. Restrict exports to the intended clients and verify the server-side paths.\nUse NFS volumes in Docker Compose:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 services: jellyfin: image: jellyfin/jellyfin:latest volumes: - jellyfin-config:/config - jellyfin-cache:/cache - /mnt/storage/media:/media:ro ports: - \u0026#34;8096:8096\u0026#34; volumes: jellyfin-config: driver: local driver_opts: type: nfs o: addr=10.0.20.30,rw,nfsvers=4,hard device: :/srv/nfs/docker-volumes/jellyfin-config jellyfin-cache: driver: local driver_opts: type: nfs o: addr=10.0.20.30,rw,nfsvers=4,hard device: :/srv/nfs/docker-volumes/jellyfin-cache Or create NFS volumes from the CLI:\n1 2 3 4 5 6 docker volume create \\ --driver local \\ --opt type=nfs \\ --opt o=addr=10.0.20.30,rw,nfsvers=4,hard \\ --opt device=:/srv/nfs/docker-volumes/postgres-data \\ postgres_data_nfs NFS volume considerations:\nUse mount and locking semantics supported by the application. A soft mount can return I/O errors during an outage and risks data corruption; it is not a general fix for application hangs. hard mounts trade availability for retry behavior, which also needs an outage plan. NFSv3 can support locking through additional services; it is inaccurate to say all older versions have no locking or security. Prefer a version and authentication policy explicitly supported by both endpoints. Preserve root squashing where practical. Align application UID/GID permissions instead of making no_root_squash a default requirement. Test latency and outage behavior. Shared storage does not make a database safe for concurrent writers on multiple hosts. Back up application-consistent data and its configuration, including any dependencies outside the NFS server. Back up prepared data, then prove recovery The earlier generic tar loop archived live database volumes and could report success after a failed step. It is removed. A read-only helper mount does not stop other containers writing to a volume, and a configured password variable is useless unless the backup tool actually consumes it.\nUse the PostgreSQL atomic dump and recovery lab for database-native dumps, failure propagation, scheduling, and a restore into a new instance. It also tests a separately encrypted Restic copy. For ordinary files, follow the Restic file recovery lab.\nFor a filesystem archive, first stop all writers cleanly and confirm the application\u0026rsquo;s supported shutdown/backup procedure. Record volume driver settings, ownership, ACLs, extended attributes, image digests, and credentials. Test recovery into a new empty destination and verify the application before touching the original. A successful tar extraction is not an application recovery test.\nVolume cleanup and lifecycle Inspect first:\n1 2 3 docker system df -v docker volume ls docker volume ls --filter dangling=true “Unused” means no container currently references the volume. It does not mean its data is disposable: a removed Compose stack can leave an important named volume behind. Inspect the exact candidate and its container references, then verify an independent recovery copy before deleting it.\nCurrent docker volume prune defaults to unused anonymous volumes; --all includes unused named volumes. Its supported filters are label filters, not until=24h. See the command reference. Labels aid inventory but are not a backup or a dry-run deletion preview.\nFor routine stack shutdown, retain volumes:\n1 docker compose down The --volumes option removes declared named volumes and attached anonymous volumes within its scope; external volumes and bind-mounted host directories are handled differently. Do not add it to troubleshooting or routine restart commands. Make any data deletion a separate decision after identifying the destination and testing recovery.\nThis revision received source checks, not a live NFS, volume migration, or application recovery test. The linked labs state their own runtime evidence and boundaries.\nPractical Homelab Storage Layout A battle-tested directory structure for Docker services on a single Proxmox VM or bare-metal host:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 /docker/ ├── config/ # Bind-mounted config files per service │ ├── nginx/ │ ├── traefik/ │ ├── prometheus/ │ ├── grafana/ │ └── frigate/ ├── media/ # Large media files (Plex, Jellyfin, Navidrome) │ ├── movies/ │ ├── tv/ │ ├── music/ │ └── photos/ ├── data/ # Bind-mounted data directories (app state) │ ├── home-assistant/ │ ├── vaultwarden/ │ └── immich/ └── compose/ # docker-compose.yml per service group ├── monitoring/ ├── media-stack/ └── infrastructure/ Corresponding Docker Compose for a service:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # compose/infrastructure/prometheus.yml services: prometheus: image: prom/prometheus:latest ports: - \u0026#34;9090:9090\u0026#34; volumes: - /docker/config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml:ro - prometheus-data:/prometheus command: - \u0026#39;--storage.tsdb.retention.time=30d\u0026#39; - \u0026#39;--storage.tsdb.path=/prometheus\u0026#39; volumes: prometheus-data: # Named volume for Docker-managed database storage The pattern: bind mounts for configs and media, named volumes for database-backed writes. This gives you easy config editing from the host plus Docker-managed storage for the data that needs it.\nSummary: Choose Your Storage by Data Type Data type Storage method Example Databases, app state Named volume PostgreSQL data, Prometheus TSDB Configuration files Bind mount (ro) nginx.conf, prometheus.yml Media / user uploads Bind mount Movies, photos, music Ephemeral cache Tmpfs Redis cache, processing temp dirs Multi-node shared data NFS volume Jellyfin config, media libraries Logs Bind mount Container stdout logs for collection Key takeaways:\nNamed volumes are the default — use them for anything the container writes. Docker manages permissions, locations, and lifecycle. Bind mounts for host-edited files — configs, media, log dirs. Keep absolute paths consistent across environments. Back up early, back up often — one docker compose down -v mistake and your database is gone. Automate volume backups with an application-consistent backup workflow and test its recovery before scheduling. Use NFS volumes for multi-node setups — but don\u0026rsquo;t put latency-sensitive databases on NFS. Reserve it for media and config. Label and review — tag volumes for inventory. Verify ownership, references, and recovery before any separate deletion; an absent label does not make data disposable. The difference between a well-managed homelab and a brittle one comes down to how you handle data persistence. Containers come and go. Volumes are what survive. Get the storage right, and your services will survive Docker upgrades, host migrations, and the occasional \u0026ldquo;whoops I deleted the stack\u0026rdquo; moment.\n","permalink":"https://blog.gntech.me/posts/2026-05-16-docker-volume-management-homelab/","summary":"Learn when to use Docker named volumes vs bind mounts for homelab containers. Covers volume drivers, NFS-backed storage, backup automation, and practical data persistence patterns for production-like Docker deployments.","title":"Docker Volume Management — Persistent Data Strategies for the Homelab"},{"content":"Cloud images let you prepare a reusable guest without repeating an ISO installation. Start with one disposable clone and working SSH access. Add packages and application provisioning after that baseline succeeds.\nVerification status Area Evidence and limits Technical review Reviewed against the official references below on October 31, 2026. Shell examples Syntax checked with Bash; commands were not executed against PVE. Runtime Disk import, cloning, SSH access, and guest initialization remain untested for this revision. Timing No deployment-time guarantee is claimed. 1. Prepare the inputs Run PVE commands as root on the intended node. Choose two unused IDs; the example uses template 9100 and test clone 9101. Check the cluster inventory, including containers, before proceeding.\nChoose storage that supports VM disks, confirm free capacity, and select a bridge connected to your test network with DHCP. The examples use local-lvm and vmbr0; change them to match your host.\nDownload an official Ubuntu 24.04 AMD64 cloud image from Ubuntu cloud images. Verify it against the publisher\u0026rsquo;s authenticated checksum information and retain its exact filename and checksum for reproducibility. Place the verified image at the path used below.\nCopy only your administrator workstation\u0026rsquo;s SSH public key to /root/cloud-template-admin.pub on PVE. Keep the private key on the workstation. Do not copy the host\u0026rsquo;s entire authorized_keys file into every guest.\n2. Create an unbooted template The following block runs in a Bash subshell and stops on an error. It creates new resources; review the IDs, storage, bridge, and image path first.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 ( set -euo pipefail template_id=9100 disk_store=local-lvm bridge=vmbr0 image=/var/lib/vz/template/iso/ubuntu-24.04-cloudimg.img test -s \u0026#34;$image\u0026#34; pvesm status --storage \u0026#34;$disk_store\u0026#34; ip link show \u0026#34;$bridge\u0026#34; # qm create refuses an existing guest ID. qm create \u0026#34;$template_id\u0026#34; --name ubuntu-2404-template \\ --memory 2048 --cores 2 --ostype l26 \\ --net0 \u0026#34;virtio,bridge=$bridge\u0026#34; --scsihw virtio-scsi-pci qm set \u0026#34;$template_id\u0026#34; --scsi0 \u0026#34;$disk_store:0,import-from=$image\u0026#34; qm set \u0026#34;$template_id\u0026#34; --ide2 \u0026#34;$disk_store:cloudinit\u0026#34; qm set \u0026#34;$template_id\u0026#34; --boot order=scsi0 --serial0 socket --vga serial0 qm resize \u0026#34;$template_id\u0026#34; scsi0 32G qm set \u0026#34;$template_id\u0026#34; --ipconfig0 ip=dhcp qm config \u0026#34;$template_id\u0026#34; qm template \u0026#34;$template_id\u0026#34; ) Import-and-attach avoids guessing the allocated volume name. The 32 GiB disk size is an example; select a size larger than the image\u0026rsquo;s virtual disk. Guest filesystem expansion must be checked after boot.\nKeep this base image unbooted. If a command fails, inspect the partial VM and task log before retrying. Do not run an automatic destroy-and-rebuild command against an ID that may already contain useful data.\nSee PVE cloud-init support and VM management for supported options on your release.\n3. Configure a full clone before starting it This block assumes the template above exists. It uses DHCP and the one public key prepared earlier. It intentionally does not start the clone until you inspect its generated configuration.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 ( set -euo pipefail template_id=9100 clone_id=9101 public_key=/root/cloud-template-admin.pub test -s \u0026#34;$public_key\u0026#34; ssh-keygen -lf \u0026#34;$public_key\u0026#34; qm clone \u0026#34;$template_id\u0026#34; \u0026#34;$clone_id\u0026#34; --name cloudinit-test --full 1 qm set \u0026#34;$clone_id\u0026#34; --ciuser gnolasco --sshkeys \u0026#34;$public_key\u0026#34; \\ --ipconfig0 ip=dhcp qm cloudinit dump \u0026#34;$clone_id\u0026#34; user qm cloudinit dump \u0026#34;$clone_id\u0026#34; network qm config \u0026#34;$clone_id\u0026#34; ) Confirm the generated user and public key, the bridge, DHCP configuration, and boot disk. Then start the new guest:\n1 qm start 9101 Find its address in DHCP leases or console output. Check its SSH host-key fingerprint through a trusted route before accepting it on the workstation. A serial console can show boot errors but does not provide a password login when no password was configured.\nFull clones have independent disks; linked clones depend on their base. Choose deliberately using the installed release\u0026rsquo;s storage and clone documentation.\n4. Verify from inside the guest Connect from your workstation using the matching private key and the guest\u0026rsquo;s actual address. Run these commands inside the guest, not on PVE:\n1 2 3 4 5 6 7 cloud-init status --wait cloud-init status --long whoami id lsblk df -h / sudo tail -n 80 /var/log/cloud-init-output.log Confirm the intended user exists, initialization finished without unresolved errors, and disk/filesystem capacity is appropriate. Review cloud-init debugging if access or configuration fails.\nEnabling the PVE guest-agent option alone does not install the guest agent. If you need it, install and configure the guest package, enable the PVE option, and verify communication separately.\nDo not set a shared template password or delete a user\u0026rsquo;s password as an SSH-hardening shortcut. Inspect the generated configuration and test authentication on a disposable clone.\n5. Add custom user data only after the baseline works A cicustom user=... file replaces PVE\u0026rsquo;s generated user data. It does not automatically merge package settings with the user and SSH key supplied by --ciuser and --sshkeys.\nUse qm cloudinit dump 9101 user as a starting point. Preserve the intended account and public-key configuration when building a complete custom document. Validate it with the cloud-init version used by the image, then test a fresh disposable clone.\nSnippet storage must support the snippets content type, and the file must be available on every node where the VM may start or migrate. Network and metadata overrides are separate choices. Rebooting an already initialized guest is not proof that a first-boot module will run again.\nKeep provisioning small at first. Record package failures and confirm access before enabling firewalls or adding applications. For configuration beyond first boot, see the Ansible homelab guide.\nRecovery and maintenance Keep the verified image and build inputs so you can create a new template under a new unused ID. Test its clone before retiring the previous template. Inspect dependencies before removing a base used by linked clones.\nRecord the PVE version, image checksum, template settings, generated cloud-init data, and clone verification results. Avoid filling the guest disk with zeroes or deleting existing templates as routine cleanup.\nContinue reading Proxmox bridges and VLANs — Check where the clone\u0026rsquo;s network connects. PBS setup and first restore — Test recovery alongside deployment. ","permalink":"https://blog.gntech.me/posts/2026-05-16-proxmox-cloudinit-vm-templates/","summary":"Build an unbooted cloud-image template and verify a disposable clone before extending provisioning.","title":"Proxmox Cloud-Init VM Templates — Automated Virtual Machine Deployment"},{"content":"Docker gets all the security attention, but most homelabs run plenty of services directly on the host — Nginx, Prometheus, systemd-networkd, fail2ban, cron jobs, custom scripts. Each one is a systemd unit, and each one runs with default permissions that are far too permissive.\nsystemd\u0026rsquo;s [Service] section supports dozens of sandboxing directives that clamp down on what a process can see, touch, and do. Most homelab servers don\u0026rsquo;t use any of them.\nThis guide covers the exact directives you add to existing unit files or new .service files to lock down services on Debian, Ubuntu, or any systemd-based Linux distribution.\nWhy Default systemd Permissions Are Dangerous Without hardening, every service on your system runs with full access to the filesystem, the process tree, the network, and kernel interfaces.\n1 2 3 4 5 6 # Check what a typical service can see systemd-run --user --wait --pty bash -c \u0026#34;ls /root\u0026#34; # Works — a service running as root can read /root systemd-run --user --wait --pty bash -c \u0026#34;cat /etc/shadow\u0026#34; # Also works — the service has the same access as its user A single vulnerable service — say, a custom API script that suffers from path traversal — can read every file on the system, spawn processes, and persist itself. systemd hardening limits the blast radius.\nThe approach: start with the most restrictive settings, then relax only what breaks. Like Docker capabilities, it\u0026rsquo;s easier to add back than to lock down after deployment.\nHardening Level 1: Filesystem Sandboxing systemd provides directives that restrict what parts of the filesystem a service can access. These are your first line of defense.\nRead-only root filesystem — the biggest win:\n1 2 [Service] ProtectSystem=strict ProtectSystem=strict makes /usr, /etc, and /lib read-only for the service. The process can still write to /var, /home, /root, /tmp, and any explicitly permitted paths.\nFor even tighter lockdown:\n1 2 [Service] ProtectSystem=full ProtectSystem=full makes /usr, /etc, /lib, and /var read-only. Typical for stateless services that only need to write to specific directories.\nAllow specific writable paths:\n1 2 3 [Service] ProtectSystem=full ReadWritePaths=/var/lib/my-service /var/log/my-service The service can only write to those two directories. Everything else is read-only.\nHide /home, /root, and /run/user:\n1 2 [Service] ProtectHome=true ProtectHome=true makes /home, /root, and /run/user inaccessible. The process inside the unit literally cannot see them. Most services never need to read user home directories.\nMake /tmp and /var/tmp private:\n1 2 [Service] PrivateTmp=true Each service gets its own private /tmp and /var/tmp that no other process can see. This prevents a service A from reading temporary files created by service B — a common attack vector for race conditions and symlink attacks.\nHardening Level 2: Capability Dropping Like Docker, systemd supports Linux capability management. You define exactly which capabilities the service binary is allowed to use.\nDrop all capabilities, add back only what\u0026rsquo;s needed:\n1 2 3 [Service] CapabilityBoundingSet=~CAP_SYS_ADMIN CAP_DAC_OVERRIDE CAP_NET_ADMIN AmbientCapabilities= The ~ prefix inverts the match: \u0026ldquo;drop these capabilities.\u0026rdquo; For most services, you can drop everything:\n1 2 [Service] CapabilityBoundingSet= Empty CapabilityBoundingSet means no capabilities — the service cannot:\nChange file permissions (CAP_DAC_OVERRIDE) Bind to privileged ports (CAP_NET_BIND_SERVICE — ports under 1024) Mount filesystems (CAP_SYS_ADMIN) Access kernel features (CAP_SYS_MODULE, CAP_SYS_PTRACE) If your service needs to bind to port 80 or 443:\n1 2 3 [Service] CapabilityBoundingSet=CAP_NET_BIND_SERVICE AmbientCapabilities=CAP_NET_BIND_SERVICE The AmbientCapabilities is required for the capability to survive exec() calls. Without it, the cap is set in the bounding set but not inherited by the child process.\nSet NoNewPrivileges — the single most important flag:\n1 2 [Service] NoNewPrivileges=true This prevents the service from gaining new privileges via suid binaries or capset syscalls. Even if an attacker gets code execution, they cannot escalate. This is the equivalent of Docker\u0026rsquo;s security_opt: no-new-privileges:true.\nHardening Level 3: Process and Network Isolation systemd can hide the rest of the system from the service entirely.\nHide everything except the service\u0026rsquo;s own process tree:\n1 2 3 [Service] ProtectProc=invisible ProcSubset=pid ProtectProc=invisible means /proc only shows the service\u0026rsquo;s own processes — no other processes on the system are visible.\nProcSubset=pid restricts /proc to only process directories. All the kernel tuning files in /proc/sys, /proc/net, and /proc/fs are hidden.\nDeny all network access:\n1 2 [Service] PrivateNetwork=true The service gets a loopback-only network with 127.0.0.1. No external connections in or out. Use this for services that don\u0026rsquo;t need the network at all (local cron tasks, maintenance scripts, log processors).\nRestrict to specific network namespaces or sockets — systemd v250+ supports socket-activation with network namespace isolation via NetworkNamespacePath=, but for most homelabs, PrivateNetwork=true is the practical option.\nHide other runtime information:\n1 2 [Service] PrivateDevices=true Hides all physical devices from the service. Only null, zero, full, random, urandom are available. The service cannot access disk devices, USB devices, or GPU devices.\n1 2 [Service] ProtectKernelTunables=true Prevents the service from modifying kernel parameters via /proc/sys or /sys.\n1 2 [Service] ProtectKernelModules=true Blocks insmod and modprobe — the service cannot load kernel modules.\n1 2 [Service] ProtectKernelLogs=true Prevents the service from reading kernel log messages via dmesg or /proc/kmsg.\n1 2 [Service] ProtectClock=true Blocks changes to the system clock. Critical for services that sync time or use timestamps — an attacker can\u0026rsquo;t corrupt your logs by changing the clock.\nHardening Level 4: Resource Limits A compromised service shouldn\u0026rsquo;t be able to DoS the rest of the host. systemd supports cgroups v2 resource control natively.\nCPU and memory limits:\n1 2 3 4 5 6 7 8 [Service] MemoryMax=512M MemoryHigh=384M MemorySwapMax=0 CPUQuota=50% TasksMax=100 IOReadBandwidthMax=/dev/sda 50M IOWriteBandwidthMax=/dev/sda 50M MemoryMax: hard memory limit (OOM kill if exceeded) MemoryHigh: soft throttle limit (starts reclaiming above this) MemorySwapMax=0: no swap usage — on low-memory homelab servers, prevents the service from pushing the system into swap CPUQuota=50%: limit to 50% of one CPU core TasksMax=100: max number of tasks (processes + threads) — prevents fork bombs Restart and failure handling:\n1 2 3 4 5 [Service] Restart=on-failure RestartSec=5s StartLimitIntervalSec=60 StartLimitBurst=3 The service restarts after a crash, but only three times in 60 seconds. After that, systemd gives up and marks the unit as failed — prevents a crash loop from burning CPU cycles forever.\nOOM killer adjustment:\n1 2 [Service] OOMPolicy=continue When the kernel\u0026rsquo;s OOM killer fires, continue means the service keeps running (systemd won\u0026rsquo;t kill it). Use stop if you want systemd to stop the unit on OOM (cleaner than the kernel blindly killing the process).\nHardening Level 5: User and Group Separation Never run services as root. systemd makes user separation trivial.\nRun as a dedicated system user:\n1 2 3 4 [Service] User=my-service Group=my-service DynamicUser=true DynamicUser=true is a killer feature — systemd creates a transient user and group with no login shell, no home directory, and deletes them when the service stops. No /etc/passwd pollution.\nWith DynamicUser, StateDirectory and friends:\n1 2 3 4 5 6 7 [Service] DynamicUser=true StateDirectory=my-service CacheDirectory=my-service LogsDirectory=my-service RuntimeDirectory=my-service RuntimeDirectoryPreserve=yes systemd creates the directories under /var/lib/, /var/cache/, /var/log/, and /run/ respectively, sets the right ownership (to the dynamic user), and cleans them up if configured.\nFor services that need a persistent user:\n1 2 3 [Service] User=myservice Group=myservice Create the user manually:\n1 sudo useradd -r -s /usr/sbin/nologin -M myservice The -r makes it a system user (UID \u0026lt; 1000), -s /usr/sbin/nologin disables login, and -M skips creating a home directory.\nSupplementary groups only when needed:\n1 2 [Service] SupplementaryGroups=docker For services that need socket access (e.g., a monitoring script that reads the Docker socket), add only the specific group instead of running as root.\nThe Complete Hardened Service Template Combining every directive into a single production-ready template:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 [Unit] Description=My Hardened Service Documentation=https://example.com/docs After=network.target Wants=network-online.target [Service] Type=simple ExecStart=/usr/local/bin/my-service Restart=on-failure RestartSec=5s StartLimitIntervalSec=60 StartLimitBurst=3 TimeoutStopSec=30 KillMode=mixed KillSignal=SIGTERM # === User === User=my-service Group=my-service DynamicUser=true SupplementaryGroups= NoNewPrivileges=true # === Filesystem Isolation === ProtectSystem=strict ProtectHome=true PrivateTmp=true ReadWritePaths=/var/lib/my-service /var/log/my-service StateDirectory=my-service CacheDirectory=my-service LogsDirectory=my-service RuntimeDirectory=my-service RuntimeDirectoryPreserve=yes # === Capabilities === CapabilityBoundingSet= AmbientCapabilities= # === Kernel Isolation === PrivateDevices=true ProtectKernelTunables=true ProtectKernelModules=true ProtectKernelLogs=true ProtectClock=true ProtectHostname=true ProtectControlGroups=true # === Process Isolation === ProtectProc=invisible ProcSubset=pid # === Network Isolation === PrivateNetwork=true # === Resource Limits (cgroups v2) === MemoryMax=256M MemoryHigh=192M MemorySwapMax=0 CPUQuota=50% TasksMax=50 IOReadBandwidthMax=/dev/sda 10M IOWriteBandwidthMax=/dev/sda 10M OOMPolicy=continue # === Sandboxing === KeyringMode=private UMask=0077 SystemCallArchitectures=native SystemCallFilter=@system-service SystemCallErrorNumber=EPERM LockPersonality=true RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX RestrictNamespaces=true RestrictRealtime=true RemoveIPC=true [Install] WantedBy=multi-user.target This template assumes the service doesn\u0026rsquo;t need the network (PrivateNetwork=true), doesn\u0026rsquo;t need capabilities, and doesn\u0026rsquo;t need to write to most of the filesystem. Adjust for your service\u0026rsquo;s actual requirements — most services will need some of these relaxed.\nApplying Hardening to Existing Services Don\u0026rsquo;t modify the base unit file for packaged services — use drop-in overrides.\nCreate an override for Nginx:\n1 sudo systemctl edit nginx.service This opens /etc/systemd/system/nginx.service.d/override.conf. Add:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 [Service] # Nginx needs to bind to port 80/443 CapabilityBoundingSet=CAP_NET_BIND_SERVICE AmbientCapabilities=CAP_NET_BIND_SERVICE # Nginx needs access to ssl certs and configs ProtectSystem=full ReadWritePaths= # Use a non-root user User=nginx Group=nginx NoNewPrivileges=true # Hide home directories ProtectHome=true PrivateTmp=true PrivateDevices=true # No kernel access needed ProtectKernelTunables=true ProtectKernelModules=true ProtectKernelLogs=true ProtectClock=true # Resource limits MemoryMax=1G MemoryHigh=768M CPUQuota=200% TasksMax=512 # System call filter (Nginx doesn\u0026#39;t need exotic syscalls) SystemCallFilter=@system-service SystemCallErrorNumber=EPERM Apply and verify:\n1 2 3 sudo systemctl daemon-reload sudo systemctl restart nginx sudo systemctl status nginx Check if hardening broke anything:\n1 sudo journalctl -u nginx --since \u0026#34;5 minutes ago\u0026#34; | grep -i \u0026#34;denied\\|permission\\|error\u0026#34; If Nginx can\u0026rsquo;t read SSL certificates, add the cert path to ReadWritePaths or verify the cert directory permissions.\nCreating a Fully Isolated Service: Example Let\u0026rsquo;s build a hardened service for a custom health check script (like the homelab health receipt that was trending on Reddit).\nThe script — /usr/local/bin/health-report.sh:\n1 2 3 4 5 6 7 8 9 #!/bin/bash # Generates a daily homelab health report set -euo pipefail OUTPUT=\u0026#34;/var/lib/health-report/report.json\u0026#34; curl -sf http://localhost:9090/api/v1/query?query=up \u0026gt; /dev/null \\ \u0026amp;\u0026amp; echo \u0026#39;{\u0026#34;health\u0026#34;:\u0026#34;ok\u0026#34;,\u0026#34;timestamp\u0026#34;:\u0026#34;\u0026#39;$(date -Iseconds)\u0026#39;\u0026#34;}\u0026#39; \\ || echo \u0026#39;{\u0026#34;health\u0026#34;:\u0026#34;degraded\u0026#34;,\u0026#34;timestamp\u0026#34;:\u0026#34;\u0026#39;$(date -Iseconds)\u0026#39;\u0026#34;}\u0026#39; \\ \u0026gt; \u0026#34;$OUTPUT\u0026#34; The hardened unit — /etc/systemd/system/health-report.service:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 [Unit] Description=Daily Homelab Health Report Documentation=https://gntech.dev/posts/systemd-service-hardening/ After=network-online.target [Service] Type=oneshot ExecStart=/usr/local/bin/health-report.sh # === User === DynamicUser=true NoNewPrivileges=true # === Filesystem Isolation === ProtectSystem=strict ProtectHome=true PrivateTmp=true StateDirectory=health-report # === Capabilities === CapabilityBoundingSet= AmbientCapabilities= # === Kernel Isolation === PrivateDevices=true ProtectKernelTunables=true ProtectKernelModules=true ProtectKernelLogs=true # === Process Isolation === ProtectProc=invisible # === Network (needs to reach Prometheus on localhost) === # PrivateNetwork=true ← Not set — needs loopback for local Prometheus RestrictAddressFamilies=AF_INET AF_UNIX # === Resource Limits === MemoryMax=64M CPUQuota=25% TasksMax=5 [Install] WantedBy=timers.target The timer — /etc/systemd/system/health-report.timer:\n1 2 3 4 5 6 7 8 9 10 [Unit] Description=Daily Health Report Timer [Timer] OnCalendar=daily Persistent=true RandomizedDelaySec=30m [Install] WantedBy=timers.target Enable both:\n1 2 sudo systemctl daemon-reload sudo systemctl enable --now health-report.timer The report script runs once daily as a dynamic user, with no root access, no home directory visibility, no kernel access, and a strict 64MB memory cap. If an attacker exploits the script, they\u0026rsquo;re trapped in a sandbox with no capabilities and no network access beyond localhost.\nVerifying Your Hardening After applying changes, verify each directive is active:\n1 2 # Load and display all active systemd hardening systemd-analyze security my-service This outputs a security score (EXPOSURE level from 1-10) and a table of every sandboxing directive with checkmarks. Aim for SAFE or EXPOSURE=1-2 for services that don\u0026rsquo;t need network or device access.\n1 2 3 4 5 6 7 8 9 10 # Check specific directives on a running service systemctl show my-service -p ProtectSystem systemctl show my-service -p ProtectHome systemctl show my-service -p CapabilityBoundingSet systemctl show my-service -p MemoryMax systemctl show my-service -p NoNewPrivileges systemctl show my-service -p PrivateTmp systemctl show my-service -p PrivateNetwork systemctl show my-service -p SystemCallFilter systemctl show my-service -p RestrictAddressFamilies 1 2 3 4 # Check the effective cgroup limits systemd-cgls -u my-service cat /sys/fs/cgroup/system.slice/my-service/memory.max cat /sys/fs/cgroup/system.slice/my-service/cpu.max 1 2 3 4 5 # Test user isolation systemd-run --unit=test-workdir -p DynamicUser=yes -p ProtectSystem=strict \\ -p CapabilityBoundingSet= -p PrivateTmp=true -p NoNewPrivileges=true \\ --wait --pty touch /test-write # Should fail: Read-only file system Common Issues and Fixes \u0026ldquo;Permission denied\u0026rdquo; on startup: Your service needs to write to a directory you didn\u0026rsquo;t allow. Use ReadWritePaths= to whitelist it, or use StateDirectory= which systemd creates with correct ownership.\n\u0026ldquo;Can\u0026rsquo;t bind to port 80\u0026rdquo;: The service needs CAP_NET_BIND_SERVICE. Add CapabilityBoundingSet=CAP_NET_BIND_SERVICE and AmbientCapabilities=CAP_NET_BIND_SERVICE. Or run on a port \u0026gt;1024 and set up a reverse proxy.\n\u0026ldquo;Can\u0026rsquo;t connect to database on another host\u0026rdquo;: PrivateNetwork=true blocks all network. Remove it (the default is false). For tighter security, use RestrictAddressFamilies=AF_INET AF_INET6 instead.\nService starts as the right user but can\u0026rsquo;t read config: Filesystem permissions matter. Store configs in a directory owned by the service user, or use LoadCredential= to pass secrets securely:\n1 2 [Service] LoadCredential=db-password:/etc/secrets/db-password SystemCallFilter breaks the service: Some apps (Java, Node.js running native modules, Chrome/Puppeteer) use uncommon syscalls. Check what\u0026rsquo;s being blocked:\n1 journalctl -u my-service | grep -i \u0026#34;syscall\\|seccomp\u0026#34; Use -1-SystemCallFilter= (drop specific syscalls) instead of +1@system-service (allow a preset). Or drop SystemCallFilter entirely if the service legitimately needs exotic syscalls.\nSummary: The Minimal Hardening Checklist These eight directives cover 95% of the improvement with minimal breakage. Apply them to every unit you create:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 [Service] # 1. Isolate the filesystem ProtectSystem=strict ProtectHome=true PrivateTmp=true # 2. Isolate the kernel PrivateDevices=true ProtectKernelTunables=true # 3. Drop privileges NoNewPrivileges=true CapabilityBoundingSet= # 4. Limit resources MemoryMax=512M CPUQuota=50% Run systemd-analyze security my-service after every change to see your exposure score drop. When a service scores SAFE on all directives, you\u0026rsquo;ve done your job.\nsystemd hardening is the equivalent of Docker\u0026rsquo;s security options — it\u0026rsquo;s built into the init system, it costs zero additional tooling, and it relies on the same kernel primitives (cgroups, namespaces, seccomp, capabilities). Use it.\n","permalink":"https://blog.gntech.me/posts/2026-05-15-systemd-service-hardening/","summary":"Hardening systemd service units in your homelab isn\u0026rsquo;t just for production servers — it\u0026rsquo;s how you prevent one compromised service from taking down your entire host. This guide covers sandboxing, capability drops, filesystem isolation, and resource limits with real unit files you can use today.","title":"Systemd Service Hardening — Lock Down Linux Services in Your Homelab"},{"content":"CAPsMAN is MikroTik\u0026rsquo;s centralized wireless management — one router configures and controls every access point on your network. Add VLANs and you get per-SSID network isolation: a work WiFi that lives on VLAN 10, a guest network on VLAN 20, and an IoT SSID on VLAN 30 — all from a single radio on each AP.\nThe problem is that MikroTik\u0026rsquo;s documentation for CAPsMAN with VLANs in RouterOS 7 is fragmented. The official wiki covers examples but misses the edge cases that break deployments: bridge VLAN filtering gotchas, datapath mode differences, and provisioning rules that don\u0026rsquo;t trigger.\nThis guide covers a complete multi-SSID CAPsMAN VLAN setup in RouterOS 7.16+, using local forwarding mode, bridge VLAN filtering, and provisioning rules that actually work.\nTopology and Design Decisions Before writing any config, understand the three forwarding modes:\nMode Traffic Flow VLAN Tagging Location Best For Local Forwarding CAP → Switch → Router directly On the CAP Most homelabs — minimal overhead CAPsMAN Forwarding CAP → CAPsMAN → Router On the CAPsMAN router Centralized filtering, IDS/IPS 802.11v BSS Transition Client steers between APs Per-AP Mesh, high-density deployments Use local forwarding. It\u0026rsquo;s simpler and faster. Traffic from each SSID is VLAN-tagged at the access point and switched directly toward the router — the CAPsMAN controller only manages config, it never touches data traffic.\nFor this guide, the setup is:\nInternet — [MikroTik Router (RB5009)] — trunk port — [Switch] — trunk ports — [cAP ax / hAP ac²] VLAN Layout: VLAN 1 — Management (native/untagged) — 192.168.88.0/24 VLAN 10 — Staff WiFi (SSID: Office) — 192.168.10.0/24 VLAN 20 — Guest WiFi (SSID: Guest) — 192.168.20.0/24 VLAN 30 — IoT WiFi (SSID: IoT) — 192.168.30.0/24 Step 1: CAPsMAN Router — Bridge and VLAN Interfaces The CAPsMAN controller runs on the main router. Start by creating the VLAN interfaces and enabling the CAPsMAN service.\n# Create VLAN interfaces on the bridge that connects to the switch /interface vlan add interface=bridge name=VLAN10 vlan-id=10 add interface=bridge name=VLAN20 vlan-id=20 add interface=bridge name=VLAN30 vlan-id=30 # Assign IPs to each VLAN /ip address add address=192.168.10.1/24 interface=VLAN10 add address=192.168.20.1/24 interface=VLAN20 add address=192.168.30.1/24 interface=VLAN30 # DHCP pools /ip pool add name=dhcp-vlan10 ranges=192.168.10.10-192.168.10.254 add name=dhcp-vlan20 ranges=192.168.20.10-192.168.20.254 add name=dhcp-vlan30 ranges=192.168.30.10-192.168.30.254 # DHCP servers /ip dhcp-server add address-pool=dhcp-vlan10 disabled=no interface=VLAN10 name=dhcp-vlan10 add address-pool=dhcp-vlan20 disabled=no interface=VLAN20 name=dhcp-vlan20 add address-pool=dhcp-vlan30 disabled=no interface=VLAN30 name=dhcp-vlan30 # DHCP server networks (dns + gateway for each) /ip dhcp-server network add address=192.168.10.0/24 dns-server=192.168.88.1,1.1.1.1 gateway=192.168.10.1 add address=192.168.20.0/24 dns-server=1.1.1.1,8.8.8.8 gateway=192.168.20.1 add address=192.168.30.0/24 dns-server=192.168.88.1,1.1.1.1 gateway=192.168.30.1 The bridge itself becomes a VLAN trunk. Enable VLAN filtering on it so only tagged VLANs pass through:\n/interface bridge vlan add bridge=bridge tagged=bridge,ether1,ether2 vlan-ids=10,20,30 /interface bridge port set [find interface=ether1] frame-types=admit-only-vlan-tagged set [find interface=ether2] frame-types=admit-only-vlan-tagged Adjust the port names to match your router\u0026rsquo;s physical layout. The key is that ports connected to the switch (or directly to APs) only accept tagged frames. Untagged frames (VLAN 1 management) should still work on the native bridge.\nStep 2: CAPsMAN Configuration — SSIDs and Security Create a CAPsMAN configuration for each SSID. This defines the SSID name, security, and — critically — the VLAN and forwarding mode.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 # Staff SSID on VLAN 10 /caps-man configuration add name=cfg-office comment=\u0026#34;Office WiFi - VLAN 10\u0026#34; \\ ssid=Office \\ country=\u0026#34;united states\u0026#34; \\ security.authentication-types=wpa2-psk,wpa3-psk \\ security.passphrase=\u0026#34;office-secure-pass-2026\u0026#34; \\ datapath.local-forwarding=yes \\ datapath.vlan-id=10 \\ datapath.vlan-mode=use-tag \\ datapath.client-isolation=no # Guest SSID on VLAN 20 add name=cfg-guest comment=\u0026#34;Guest WiFi - VLAN 20\u0026#34; \\ ssid=Guest \\ country=\u0026#34;united states\u0026#34; \\ security.authentication-types=wpa2-psk \\ security.passphrase=\u0026#34;guest-welcome-2026\u0026#34; \\ datapath.local-forwarding=yes \\ datapath.vlan-id=20 \\ datapath.vlan-mode=use-tag \\ datapath.client-isolation=yes # IoT SSID on VLAN 30 add name=cfg-iot comment=\u0026#34;IoT Devices - VLAN 30\u0026#34; \\ ssid=IoT \\ country=\u0026#34;united states\u0026#34; \\ security.authentication-types=wpa2-psk \\ security.passphrase=\u0026#34;iot-devices-2026\u0026#34; \\ datapath.local-forwarding=yes \\ datapath.vlan-id=30 \\ datapath.vlan-mode=use-tag \\ datapath.client-isolation=yes Key parameters explained:\nlocal-forwarding=yes — Traffic stays local to the AP, not tunneled back to CAPsMAN. Required for the switch to handle VLANs. vlan-mode=use-tag — AP tags every client\u0026rsquo;s traffic with the specified VLAN ID before sending it out the Ethernet port. vlan-id=10/20/30 — The 802.1Q VLAN tag applied to client traffic for that SSID. client-isolation — Prevents wireless clients on the same SSID from talking to each other. Enable for guest networks, disable for office/staff networks that need local discovery (printers, etc.). Step 3: CAPsMAN Interface Binding Restrict the CAPsMAN service to a specific interface — don\u0026rsquo;t leave it listening on all interfaces:\n1 2 3 4 5 6 7 8 9 # Disable the default \u0026#34;all interfaces\u0026#34; binding /caps-man manager interface set [find] forbid=yes # Add only the bridge (or management VLAN interface) add interface=bridge disabled=no # Enable CAPsMAN /caps-man manager set enabled=yes If you have a dedicated management VLAN, use that interface instead:\n1 add interface=VLAN1 disabled=no This prevents CAPsMAN discovery traffic from leaking onto client VLANs and keeps the control plane clean.\nStep 4: Provisioning Rules — Attach SSIDs to APs Provisioning rules tell CAPsMAN which configuration to push to which CAP. You can match by MAC address, radio MAC, or use a catch-all.\nCatch-all rule — same SSIDs on every AP:\n1 2 3 4 /caps-man provisioning add action=create-dynamic-enabled \\ master-configuration=cfg-office \\ slave-configurations=cfg-guest,cfg-iot This creates three VAPs (Virtual Access Points) on every CAP that connects: the master cfg-office (VLAN 10) and two slaves cfg-guest (VLAN 20) and cfg-iot (VLAN 30).\nPer-AP rules — different SSIDs per access point:\n1 2 3 4 5 6 7 8 9 10 11 12 # Only push Office + IoT to AP in the main building add action=create-dynamic-enabled \\ master-configuration=cfg-office \\ slave-configurations=cfg-iot \\ comment=\u0026#34;Building A - no guest\u0026#34; \\ identity-regex=\u0026#34;ap-building-a\u0026#34; # Guest SSID only on the outdoor AP add action=create-dynamic-enabled \\ master-configuration=cfg-guest \\ comment=\u0026#34;Outdoor AP - guest only\u0026#34; \\ identity-regex=\u0026#34;ap-outdoor\u0026#34; Match rules are evaluated top-to-bottom. The first matching rule wins. Use identity-regex to match based on the CAP\u0026rsquo;s identity (set in the CAP\u0026rsquo;s /system identity).\nStep 5: CAP Configuration — Enabling the Access Points On each MikroTik AP, enable CAP mode:\n# On the AP — run this via SSH or MAC-telnet /interface wireless cap set enabled=yes \\ bridge=bridge \\ discovery-interfaces=bridge \\ interfaces=wlan1,wlan2 For wifiwave2 package devices (cAP ax, hAP ax³, etc.):\n1 2 3 4 5 # RouterOS 7 wifiwave2 CAP mode /interface/wifiwave2/cap set enabled=yes \\ slave-configurations=cfg-guest,cfg-iot \\ discovery-interfaces=bridge The bridge=bridge setting is critical — it tells the CAP to add its wireless interfaces to the local bridge so bridged VLAN traffic flows correctly.\nAfter the CAP connects, verify it in CAPsMAN:\n1 2 3 4 5 6 7 # On the CAPsMAN router /caps-man remote-cap print # Columns: FLAGS, ADDRESS, IDENTITY, VERSION, RADIOS, STATE # Look for state=running /caps-man registration-table print # Shows connected wireless clients and their VLAN assignment Step 6: Switch Configuration — Bridge VLAN Filtering The switch between the CAPsMAN router and APs needs bridge VLAN filtering to pass tagged traffic.\nOn a MikroTik switch (CRS or CSS series):\n# Create the bridge with VLAN filtering /interface bridge add name=bridge1 vlan-filtering=yes # Ports to CAPsMAN router and APs — trunk ports /interface bridge port add bridge=bridge1 interface=ether1 comment=\u0026#34;uplink to CAPsMAN\u0026#34; add bridge=bridge1 interface=ether2 comment=\u0026#34;to AP - living room\u0026#34; add bridge=bridge1 interface=ether3 comment=\u0026#34;to AP - office\u0026#34; add bridge=bridge1 interface=ether4 comment=\u0026#34;to AP - outdoor\u0026#34; add bridge=bridge1 interface=ether5 pvid=1 comment=\u0026#34;management port\u0026#34; # Define VLANs on the bridge /interface bridge vlan add bridge=bridge1 tagged=ether1,ether2,ether3,ether4 vlan-ids=10,20,30 add bridge=bridge1 tagged=ether1,ether2,ether3,ether4 untagged=ether5 vlan-ids=1 The trunk ports (ether1-4) carry tagged VLANs 10, 20, and 30. Management traffic (VLAN 1) stays untagged on port ether5 so you can plug a laptop directly into the switch for management access.\nFor non-MikroTik switches, create an equivalent VLAN setup:\nPort to CAPsMAN router → trunk with tagged VLANs 10, 20, 30 Ports to APs → trunk with tagged VLANs 10, 20, 30 (PVID 1) Management port → access port on VLAN 1 Common CAPsMAN VLAN Problems and Fixes Problem 1: CAP connects but SSIDs don\u0026rsquo;t appear 1 2 # Check remote CAP status /caps-man remote-cap print detail If the CAP shows state=running but no dynamic interfaces appear, check the provisioning rule:\n1 2 # Verify the provisioning matches /caps-man provisioning print Most common cause: the provisioning rule\u0026rsquo;s identity-regex doesn\u0026rsquo;t match the CAP\u0026rsquo;s system identity. Use a catch-all rule first to confirm the config works, then narrow it down.\nProblem 2: Clients connect but get no IP address The DHCP server isn\u0026rsquo;t reachable through the VLAN. Check:\n# On the CAPsMAN router, verify DHCP is running /ip dhcp-server print /ip dhcp-server lease print # Test VLAN reachability from the bridge /ping 192.168.10.1 interface=VLAN10 count=5 If DHCP discovery packets never reach the router, the switch isn\u0026rsquo;t forwarding the VLAN. Double-check bridge VLAN filtering:\n# On the switch, verify VLAN membership /interface bridge vlan print /interface bridge port print Every trunk port must appear in the tagged= list for each VLAN.\nProblem 3: \u0026ldquo;CAPsMAN disconnects after a minute\u0026rdquo; UDP ports 5246 and 5247 must be open between CAP and CAPsMAN router:\n# Check firewall on CAPsMAN router /ip firewall filter print # Allow CAPsMAN control and data /ip firewall filter add chain=input protocol=udp dst-port=5246-5247 action=accept \\ comment=\u0026#34;allow CAPsMAN\u0026#34; Problem 4: Guest network can reach staff network Firewall rules must block inter-VLAN routing. On the CAPsMAN router:\n# Block Guest VLAN 20 from reaching Staff VLAN 10 and IoT VLAN 30 /ip firewall filter add chain=forward src-address=192.168.20.0/24 dst-address=192.168.10.0/24 action=drop \\ comment=\u0026#34;block Guest -\u0026gt; Staff\u0026#34; add chain=forward src-address=192.168.20.0/24 dst-address=192.168.30.0/24 action=drop \\ comment=\u0026#34;block Guest -\u0026gt; IoT\u0026#34; # Allow IoT to reach internet but not Staff add chain=forward src-address=192.168.30.0/24 dst-address=192.168.10.0/24 action=drop \\ comment=\u0026#34;block IoT -\u0026gt; Staff\u0026#34; # Allow Staff to reach everything (for management, printers, etc.) # No drop rule needed — default forward policy allows it # Allow established/related traffic back add chain=forward connection-state=established,related action=accept Guest network should typically only have internet access. Block it from all private subnets:\n1 2 add chain=forward src-address=192.168.20.0/24 dst-address=192.168.0.0/16 action=drop \\ comment=\u0026#34;block Guest -\u0026gt; entire LAN\u0026#34; Problem 5: VLAN tags stripped by switch If your AP is connected to a dumb switch or an unmanaged switch between the trunk and the AP, VLAN tags get stripped. CAPsMAN VLAN setups require managed switches with VLAN trunk ports between the router and every AP.\nIf you must use an unmanaged switch, the AP must do VLAN tagging on its egress port. This works in local forwarding mode only if the AP\u0026rsquo;s Ethernet port connects directly to a router interface that understands the VLANs.\nVerifying the Full Setup After everything is configured, run through this checklist:\n# 1. Check CAPsMAN sees all APs /caps-man remote-cap print # Expected: one entry per AP, state=running # 2. Check dynamic interfaces on a CAP # (run on the AP itself) /interface wireless print # Expected: wlan1 (Office), wlan2 (Guest), wlan3 (IoT) # 3. Check VLAN assignment on the CAP /interface bridge port print # Expected: dynamic ports with PVID matching VLAN config # wlan1 (PVID 10), wlan2 (PVID 20), wlan3 (PVID 30) # 4. Check connected clients /caps-man registration-table print # Expected: clients with correct VLAN ID per SSID # 5. Check DHCP leases from client perspective /ip dhcp-server lease print where server=vlan10 # Expected: client got an IP on 192.168.10.0/24 # 6. Verify traffic isolation # From a Guest client, try pinging a Staff client: # Should time out (firewall drops it) Maintenance and Updates Adding a new SSID: Create a new CAPsMAN configuration with the new VLAN ID, add it to the provisioning rule\u0026rsquo;s slave-configurations, then ensure the VLAN interface, DHCP pool, and firewall rules exist on the router.\nAdding a new AP: Connect it to a trunk port on the switch, enable CAP mode, and CAPsMAN auto-provisions it. No per-AP configuration needed if using catch-all provisioning.\nUpdating RouterOS on APs: Use CAPsMAN\u0026rsquo;s built-in upgrade:\n1 2 3 # On CAPsMAN router — push firmware to all CAPs /caps-man manager upgrade set channel=upgrade package-url=\u0026#34;\u0026#34; \\ action=upgrade This works best if you\u0026rsquo;ve already uploaded the .npk to the CAPsMAN router\u0026rsquo;s files directory:\n1 2 3 4 5 6 # Upload RouterOS .npk file via web interface or SCP # Then trigger upgrade on a specific CAP group /caps-man manager upgrade set channel=upgrade \\ package-url=\u0026#34;https://upgrade.mikrotik.com/routeros/7.16.2/\u0026#34; \\ action=upgrade For production APs, stage the upgrade: upgrade one AP, verify clients reconnect, then upgrade the rest.\nSummary MikroTik CAPsMAN with VLANs gives you proper WiFi network segmentation without per-AP management overhead. The critical rules:\nUse local forwarding — simpler, faster, and compatible with managed switches that do VLAN filtering Match forwarding mode everywhere — local-forwarding=yes on every CAPsMAN configuration and vlan-mode=use-tag for tagged egress Bridge VLAN filtering is mandatory on the switch between CAPsMAN router and APs — trunk ports must pass the VLANs you use Provisioning rules can be catch-all or per-AP via identity-regex — test with a catch-all first, then narrow Firewall between VLANs — CAPsMAN tags traffic, but isolating VLANs is the router\u0026rsquo;s firewall job The most common failure point is bridge VLAN filtering: a port that isn\u0026rsquo;t listed as tagged for a VLAN silently drops that traffic. Always verify with /interface bridge vlan print on every switch and on the CAPsMAN router\u0026rsquo;s bridge.\nOnce it clicks, CAPsMAN VLANs are the cleanest way to run multi-SSID WiFi in a homelab. Add an AP: plug in, enable CAP mode, done. No SSH, no config, no per-AP tweaks. The controller handles everything.\n","permalink":"https://blog.gntech.me/posts/2026-05-15-mikrotik-capsman-vlan-configuration/","summary":"A complete guide to configuring MikroTik CAPsMAN with VLANs in RouterOS 7 — local forwarding mode, bridge VLAN filtering, provisioning rules for multiple SSIDs, DHCP per VLAN, and real troubleshooting for the common mistakes that break CAPsMAN VLAN setups.","title":"MikroTik CAPsMAN VLAN — Multi-SSID Configuration Guide"},{"content":"Every homelab running Docker Compose eventually runs into the same problem: you have one docker-compose.yml for your media stack, another for your monitoring stack, and a third for your reverse proxy. They all need to talk to each other, but Compose creates separate networks by default.\nThis post covers the networking patterns you need to make multiple Docker Compose stacks communicate reliably — bridge networks, DNS-based service discovery, external network sharing, and multi-network containers — with real configurations you can drop into your homelab today.\nHow Compose Networking Works (Default Behavior) When you run docker compose up, Compose automatically creates a default bridge network named \u0026lt;project\u0026gt;_default. Every service in that Compose file joins this network. Services can reach each other by their service name, which Compose resolves via embedded DNS.\n1 2 3 4 5 6 7 8 # docker-compose.yml services: web: image: nginx # Reachable by other services as \u0026#34;web\u0026#34; db: image: postgres:16 # Reachable by other services as \u0026#34;db\u0026#34; App connects to db:5432 — no IPs needed. This is the default bridge network, and it Just Works for a single Compose file.\nThe problem starts when you have two Compose stacks:\n1 2 3 4 5 6 7 8 9 # stack-a/docker-compose.yml services: app: image: my-app # stack-b/docker-compose.yml services: app: image: my-other-app These two app services are on completely separate networks. stack-a cannot resolve app from stack-b, and stack-b cannot resolve app from stack-a. No DNS, no connectivity, nothing.\nPattern 1: External Networks for Cross-Stack Communication The cleanest way to connect multiple Compose stacks is to define a shared external network that every stack joins. This is the pattern used by every homelab reverse proxy setup.\nFirst, create the shared network:\n1 docker network create shared-net Then each Compose file joins it:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 # traefik/docker-compose.yml networks: proxy-net: name: proxy-net external: true # Don\u0026#39;t create it — use the existing one services: traefik: image: traefik:v3.1 container_name: traefik networks: - proxy-net ports: - \u0026#34;80:80\u0026#34; - \u0026#34;443:443\u0026#34; volumes: - /var/run/docker.sock:/var/run/docker.sock:ro command: - \u0026#34;--providers.docker=true\u0026#34; - \u0026#34;--entrypoints.websecure.address=:443\u0026#34; - \u0026#34;--certificatesresolvers.letsencrypt.acme.tlschallenge=true\u0026#34; - \u0026#34;--certificatesresolvers.letsencrypt.acme.email=admin@example.com\u0026#34; - \u0026#34;--certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json\u0026#34; 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # nextcloud/docker-compose.yml networks: proxy-net: name: proxy-net external: true services: nextcloud: image: nextcloud:stable container_name: nextcloud networks: - proxy-net # Traefik can reach this as \u0026#34;nextcloud:80\u0026#34; labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.nextcloud.rule=Host(`cloud.example.com`)\u0026#34; - \u0026#34;traefik.http.services.nextcloud.loadbalancer.server.port=80\u0026#34; 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # immich/docker-compose.yml networks: proxy-net: name: proxy-net external: true services: immich-server: image: ghcr.io/immich-app/immich-server:release container_name: immich networks: - proxy-net labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.immich.rule=Host(`photos.example.com`)\u0026#34; DNS resolves container names — Traefik discovers nextcloud:80 and immich:2283 through the shared proxy-net. No manual IPs. No Compose-level dependencies across stacks.\nPattern 2: Multi-Network Containers (The Traefik Sandwich) A more advanced pattern: a single container belongs to multiple networks simultaneously. Traefik is the classic example — it needs to be on both the public network (to accept inbound traffic) and on every internal network (to reach backend services).\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 # traefik/docker-compose.yml networks: public-net: name: public-net external: true internal-apps-net: name: internal-apps-net external: true monitoring-net: name: monitoring-net external: true services: traefik: image: traefik:v3.1 container_name: traefik networks: - public-net - internal-apps-net - monitoring-net ports: - target: 443 published: 443 protocol: tcp Traefik sees every service on all three networks. Backend services only need to be on their respective network — they never expose ports to the host. This is defense-in-depth: even if a service in internal-apps-net is compromised, it cannot reach services on monitoring-net.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # monitoring/docker-compose.yml networks: monitoring-net: name: monitoring-net external: true services: grafana: image: grafana/grafana:latest container_name: grafana networks: - monitoring-net labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.grafana.rule=Host(`monitor.example.com`)\u0026#34; No port mapping on Grafana — it\u0026rsquo;s only reachable through Traefik on the shared monitoring-net. A container in internal-apps-net cannot touch it.\nPattern 3: Internal Networks for Database Isolation Databases should never be exposed to external networks — not even to the shared proxy network. The pattern: create a separate internal network, put only the database and its consumer on it.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 # nextcloud/docker-compose.yml networks: proxy-net: name: proxy-net external: true db-net: name: nextcloud-db-net internal: true # No external connectivity — containers can\u0026#39;t reach outside services: nextcloud: image: nextcloud:stable container_name: nextcloud networks: - proxy-net - db-net postgres: image: postgres:16-alpine container_name: nextcloud-db networks: - db-net # Only on the internal network environment: POSTGRES_DB: nextcloud POSTGRES_USER: nextcloud POSTGRES_PASSWORD: ${DB_PASSWORD} volumes: - postgres_data:/var/lib/postgresql/data healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;pg_isready -U nextcloud\u0026#34;] interval: 10s timeout: 5s retries: 5 volumes: postgres_data: The database is unreachable from Traefik, unreachable from the internet, unreachable from any other Compose stack. Only nextcloud can talk to it. The internal: true flag on the network means the database cannot even initiate outbound connections.\nFor PostgreSQL and MySQL, add --network=nextcloud-db-net to your backup container:\n1 2 3 4 5 6 7 8 9 10 11 12 13 pg-backup: image: prodrigestivill/postgres-backup-local:16-alpine container_name: nextcloud-db-backup networks: - db-net environment: POSTGRES_HOST: postgres POSTGRES_DB: nextcloud POSTGRES_USER: nextcloud POSTGRES_PASSWORD: ${DB_PASSWORD} SCHEDULE: \u0026#34;@daily\u0026#34; volumes: - ./backups:/backups Only the database, the app, and the backup container exist on this network. Nothing else.\nPattern 4: Default Networks with Custom Names and Subnets Sometimes you want more control than Compose\u0026rsquo;s auto-generated \u0026lt;project\u0026gt;_default. Define the default network explicitly:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 # docker-compose.yml networks: default: name: media-stack driver: bridge ipam: config: - subnet: \u0026#34;10.30.0.0/24\u0026#34; gateway: \u0026#34;10.30.0.1\u0026#34; services: plex: image: plexinc/pms-docker container_name: plex networks: - default sabnzbd: image: lscr.io/linuxserver/sabnzbd:latest container_name: sabnzbd networks: - default Benefits:\nPredictable subnet — useful if your firewall rules reference Docker IPs Named network — easier to reference as external: true from other stacks No collision with other Compose defaults (all named media-stack instead of project_a_default vs project_b_default) Check what IP a container got:\n1 2 docker inspect plex -f \u0026#39;{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}\u0026#39; # 10.30.0.2 Pattern 5: Aliases and Container Names for Stable DNS Compose services get DNS entries by their service name. But if you need a service to respond on multiple DNS names, use aliases:\n1 2 3 4 5 6 7 8 9 10 services: db: image: postgres:16-alpine container_name: app-database networks: db-net: aliases: - postgres - database - primary-db Services on db-net can reach this container as postgres, database, primary-db, or app-database (the container name). This is useful when migrating — you can run both old and new databases temporarily with the same DNS aliases.\nContainer names also work as DNS entries within the same network:\n1 2 3 4 5 services: app: image: my-app container_name: my-app-v2 # Reachable as \u0026#34;my-app-v2\u0026#34; on all networks it joins Pattern 6: The Database URI with Docker DNS When configuring apps to connect to databases, use the service name as the hostname:\n1 2 3 4 5 6 7 services: app: image: my-app environment: DATABASE_URL: \u0026#34;postgres://user:password@postgres:5432/appdb\u0026#34; REDIS_URL: \u0026#34;redis://redis:6379\u0026#34; RABBITMQ_URL: \u0026#34;amqp://rabbitmq:5672\u0026#34; Docker DNS resolves postgres, redis, and rabbitmq to the correct container IPs on the shared network. No config files, no IP hardcoding, no round-robin confusion.\nIf the database fails and restarts, Docker re-issues the same DNS name with the new IP. Apps reconnect automatically as long as they have connection retry logic.\nPattern 7: Depends On for Network Startup Ordering Services on the same network still need startup ordering. Use depends_on with health checks for production services:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 services: postgres: image: postgres:16-alpine healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;pg_isready -U appuser\u0026#34;] interval: 5s timeout: 3s retries: 10 app: image: my-app depends_on: postgres: condition: service_healthy # DNS will resolve postgres once it\u0026#39;s healthy Without condition: service_healthy, depends_on only waits for the container to start, not for the database to accept connections. The app starts, connects to postgres:5432, Docker DNS resolves it, but PostgreSQL isn\u0026rsquo;t ready yet — connection refused.\nWith health checks + condition: service_healthy, Compose waits for pg_isready to succeed before starting the app. The network is available, DNS resolves, and the database is listening.\nPattern 8: Connect Running Containers to Networks Sometimes you need to connect an already-running container to a network without restarting it. Useful for debugging or temporary access:\n1 2 3 4 5 6 7 8 9 # Connect a container to a network docker network connect shared-net my-container # Disconnect from a network docker network disconnect shared-net my-container # Inspect all networks a container is on docker inspect my-container -f \\ \u0026#39;{{range $k, $v := .NetworkSettings.Networks}}{{$k}} {{end}}\u0026#39; This is a runtime operation — the container keeps running. Use it when you need to temporarily add a debugging container to a network:\n1 2 3 # Spin up a diagnostic container on a running stack\u0026#39;s network docker run -it --rm --network media-stack nicolaka/netshoot \\ sh -c \u0026#34;nslookup plex \u0026amp;\u0026amp; curl -s plex:32400/web/index.html | head -5\u0026#34; The netshoot image is purpose-built for Docker network debugging — it includes dig, nslookup, curl, tcpdump, ping, mtr, and netstat.\nPattern 9: Network Troubleshooting Toolkit When containers can\u0026rsquo;t reach each other, here\u0026rsquo;s the diagnostic sequence:\nStep 1 — Verify network exists:\n1 2 docker network ls docker network inspect proxy-net Step 2 — Check which containers are on each network:\n1 docker network inspect proxy-net -f \u0026#39;{{range .Containers}}{{.Name}} {{end}}\u0026#39; Step 3 — Test DNS resolution from inside a container:\n1 2 3 docker run --rm --network proxy-net nicolaka/netshoot \\ dig +short nextcloud # Should return the container IP Step 4 — Test TCP connectivity:\n1 2 3 docker run --rm --network proxy-net nicolaka/netshoot \\ sh -c \u0026#34;nc -zv nextcloud 80\u0026#34; # Should return: nc: nextcloud (10.20.0.3:80) open Step 5 — Check container-level network config:\n1 docker inspect nextcloud -f \u0026#39;{{json .NetworkSettings.Networks}}\u0026#39; | jq . Common failures:\nSymptom Likely Cause Fix DNS resolution fails Container not on the same network Add network to networks: in Compose Connection refused Service not listening on that interface Check bind 0.0.0.0 in app config Connection timeout Firewall blocking the port Check iptables -L DOCKER on host Can\u0026rsquo;t reach by hostname Missing container_name in Compose Use service name instead Works on host but not in container Port exposed on 127.0.0.1 Expose on 0.0.0.0 in the app Real-World Homelab Network Topology Here\u0026rsquo;s how these patterns compose into a full homelab:\nInternet │ ┌───┴───┐ │ WAN │ └───┬───┘ │ ┌───┴───┐ public-net │Traefik│◄──── (443/80) └───┬───┘ │ ┌─────────────────────────┼─────────────────────┐ │ │ │ ┌─────┴─────┐ ┌──────┴──────┐ ┌──────┴──────┐ │ nextcloud │ │ immich │ │ grafana │ │ :80 │ │ :2283 │ │ :3000 │ └─────┬─────┘ └──────┬──────┘ └──────┬──────┘ │ │ │ ┌─────┴─────┐ ┌──────┴──────┐ ┌──────┴──────┐ │ postgres │ │ postgres │ │ prometheus │ │ (internal)│ │ (internal) │ │ (monitoring)│ └───────────┘ └─────────────┘ └─────────────┘ Networks:\nNetwork Type Contains External Reach public-net bridge Traefik Internet (80, 443) media-net bridge Sonarr, Radarr, Plex LAN only monitoring-net bridge Grafana, Prometheus Traefik only nextcloud-db-net internal Nextcloud + Postgres None immich-db-net internal Immich + Postgres None Compose file structure:\n1 2 3 4 5 6 7 8 9 10 11 ~/docker/ ├── traefik/ │ └── docker-compose.yml # public-net + monitoring-net + media-net ├── nextcloud/ │ └── docker-compose.yml # public-net + nextcloud-db-net ├── immich/ │ └── docker-compose.yml # public-net + immich-db-net ├── monitoring/ │ └── docker-compose.yml # monitoring-net only └── media/ └── docker-compose.yml # media-net only Every stack is its own Compose file. No single docker-compose.yml with 30 services. Each stack can be updated, restarted, or torn down independently.\nThe One-Liner: Creating All Networks Bootstrap all the shared networks before deploying:\n1 2 3 for net in public-net monitoring-net media-net nextcloud-db-net immich-db-net; do docker network create \u0026#34;$net\u0026#34; 2\u0026gt;/dev/null || echo \u0026#34;Network $net already exists\u0026#34; done Then deploy each stack independently:\n1 2 3 4 5 cd ~/docker/traefik \u0026amp;\u0026amp; docker compose up -d cd ~/docker/nextcloud \u0026amp;\u0026amp; docker compose up -d cd ~/docker/immich \u0026amp;\u0026amp; docker compose up -d cd ~/docker/monitoring \u0026amp;\u0026amp; docker compose up -d cd ~/docker/media \u0026amp;\u0026amp; docker compose up -d Want to rebuild monitoring without touching the media stack?\n1 cd ~/docker/monitoring \u0026amp;\u0026amp; docker compose up -d --force-recreate The stack\u0026rsquo;s volumes are retained. Check its own startup result and data before continuing. Other stacks are separate deployments; shared external networks and services remain dependencies. That\u0026rsquo;s the point of multi-stack networking.\nSummary Docker Compose networking is deceptively simple until you cross stack boundaries. The core patterns are straightforward:\nExternal networks — create once, reference from every Compose file Multi-network containers — put Traefik on every network that needs it Internal networks — lock databases behind internal: true Service name DNS — use container names, not IPs, everywhere Health checks — wait for service_healthy before connecting These five patterns cover 90% of homelab networking needs. No service mesh, no Kubernetes, no overlay networks. Just well-planned bridge networks with clean boundaries.\n","permalink":"https://blog.gntech.me/posts/2026-05-15-docker-compose-networking-patterns/","summary":"Learn how Docker Compose networking really works — bridge networks, DNS-based service discovery, cross-stack communication, and real-world network isolation patterns for homelab deployments.","title":"Docker Compose Networking — Multi-Stack Bridge, DNS, and Cross-Story Communication"},{"content":"Most homelabs run Docker containers as root by default. The Dockerfile says FROM nginx:latest and nginx runs as root inside the container. A container escape, a vulnerable web app, or a malicious image — and an attacker has root on your Docker host.\nThis isn\u0026rsquo;t theoretical. CVEs targeting container escape vectors (CVE-2024-21626 for runc, CVE-2026-42945 for Nginx) are published regularly. The fix is straightforward: stop running containers as root, drop unnecessary capabilities, lock down the filesystem, and apply runtime security profiles.\nThis guide covers the exact configurations you need for a hardened Docker setup in a homelab — with real docker-compose.yml examples you can apply today.\nWhy Default Docker Permissions Are Dangerous Docker containers share the host kernel. When a process runs as root (UID 0) inside a container, it has the same UID 0 outside the container. The only thing separating it from the host is the Linux kernel\u0026rsquo;s namespace isolation and cgroups.\nCheck what user your containers are running as:\n1 2 3 4 5 docker exec my-container whoami # root ← this is a problem docker exec my-container id # uid=0(root) gid=0(root) groups=0(root) If an attacker exploits a kernel vulnerability from inside that container, they get root on your Proxmox host. The runc escape CVE (CVE-2019-5736, CVE-2024-21626) proves this isn\u0026rsquo;t theoretical — it\u0026rsquo;s been exploited in the wild against CI pipelines and cloud deployments.\nThe fix: every container should run as a non-root user with the minimum Linux capabilities it actually needs.\nHardening Layer 1: Non-Root Users in Dockerfiles The first and most important layer: define a non-root user in your Dockerfile and switch to it with the USER directive.\nBad Dockerfile:\n1 2 3 4 5 6 7 FROM node:20-alpine WORKDIR /app COPY package*.json ./ RUN npm install COPY . . CMD [\u0026#34;node\u0026#34;, \u0026#34;server.js\u0026#34;] # Runs as root — one exploit = full host compromise Good Dockerfile:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 FROM node:20-alpine # Create a non-root user with known UID/GID RUN addgroup -S appgroup \u0026amp;\u0026amp; \\ adduser -S appuser -G appgroup -u 1001 WORKDIR /app COPY package*.json ./ RUN npm install \u0026amp;\u0026amp; \\ chown -R appuser:appgroup /app COPY --chown=appuser:appgroup . . # Switch to non-root user USER appuser CMD [\u0026#34;node\u0026#34;, \u0026#34;server.js\u0026#34;] The chown is critical — if the app writes files (logs, uploads, caches), the non-root user must own them or the container crashes with \u0026ldquo;Permission denied.\u0026rdquo;\nFor Docker Compose, you can also enforce the user without modifying the image:\n1 2 3 4 services: app: image: some-image-that-runs-as-root user: \u0026#34;1001:1001\u0026#34; # Force container to run as UID 1001 But this breaks if the image writes to files owned by root. Fixing it at the image level with a proper Dockerfile is the correct approach.\nHardening Layer 2: Dropping Linux Capabilities Linux capabilities break the root user\u0026rsquo;s blanket privileges into discrete units: CAP_NET_BIND_SERVICE (bind to ports \u0026lt;1024), CAP_SYS_ADMIN (mount filesystems, access kernel features), and many more.\nDocker gives containers a default set of capabilities. Most are unnecessary for a standard web app.\nCheck what capabilities your container has:\n1 2 docker run --rm ubuntu:22.04 capsh --print docker exec my-container capsh --print | grep -o \u0026#39;=[^=]*cap_[^+]*\u0026#39; Drop all capabilities, then add back only what\u0026rsquo;s needed:\n1 2 3 4 5 6 7 8 9 10 11 services: nginx: image: nginx:alpine container_name: web cap_drop: - ALL # Remove every capability cap_add: - NET_BIND_SERVICE # Only allow binding to port 80/443 - CHOWN # Allow changing file ownership - SETGID # Allow setting group ID - SETUID # Allow setting user ID For a typical web app (Node.js, Python, Go), you usually only need:\n1 2 3 4 5 6 7 services: webapp: image: my-web-app cap_drop: - ALL cap_add: - NET_BIND_SERVICE # Optional — only if binding to ports \u0026lt;1024 If your app binds to port 8080 (not 80), you don\u0026rsquo;t even need NET_BIND_SERVICE. Drop everything:\n1 2 3 4 5 6 services: webapp: image: my-web-app cap_drop: - ALL cap_add: [] # No capabilities at all — cleanest config What about databases?\n1 2 3 4 5 6 7 8 9 10 11 services: postgres: image: postgres:16-alpine cap_drop: - ALL cap_add: - CHOWN - DAC_OVERRIDE # Needed for file permission operations - SETUID - SETGID - NET_BIND_SERVICE # Postgres binds to port 5432 Signs a capability is missing:\nnginx: [emerg] socket() failed: (13: Permission denied) → Add NET_BIND_SERVICE change_attributes: Operation not permitted → Add CHOWN or DAC_OVERRIDE fork failed: Resource temporarily unavailable → Check pids_limit, not a capability issue Hardening Layer 3: Read-Only Root Filesystem Most containers only need to write to specific directories (uploads, caches, databases). Everything else should be read-only. This prevents attackers from modifying binaries or writing malicious files.\n1 2 3 4 5 6 7 8 9 10 11 services: my-app: image: my-web-app cap_drop: - ALL read_only: true tmpfs: # Writable temp directories in RAM - /tmp:noexec,nosuid,size=64M - /var/run:noexec,nosuid,size=32M volumes: - app_data:/app/data # Persistent writable directory The read_only: true flag makes the entire container filesystem read-only. The tmpfs mounts provide small RAM-backed writable directories for runtime data (pids, sockets, temp files). The named volume provides a persistent writable directory for application data.\nFor PostgreSQL:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 services: postgres: image: postgres:16-alpine cap_drop: - ALL cap_add: - CHOWN - DAC_OVERRIDE - SETUID - SETGID read_only: true tmpfs: - /tmp:noexec,nosuid,size=64M volumes: - pg_data:/var/lib/postgresql/data # Data dir must be writable environment: POSTGRES_INITDB_ARGS: \u0026#34;--data-checksums\u0026#34; PostgreSQL writes to /var/lib/postgresql/data — that volume gives it write access. Everything else (binaries, libraries) stays read-only.\nFor Nginx with read-only root:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 services: nginx: image: nginx:alpine cap_drop: - ALL cap_add: - NET_BIND_SERVICE - CHOWN read_only: true tmpfs: - /var/cache/nginx:noexec,nosuid,size=128M - /var/run:noexec,nosuid,size=64M - /tmp:noexec,nosuid,size=64M volumes: - ./nginx.conf:/etc/nginx/nginx.conf:ro - ./html:/usr/share/nginx/html:ro ports: - \u0026#34;80:80\u0026#34; - \u0026#34;443:443\u0026#34; Nginx needs writable /var/cache/nginx for its cache and /var/run for the PID file. Everything else is read-only — including the config and static files mounted with :ro.\nHardening Layer 4: Security Options (seccomp, AppArmor, no-new-privileges) Docker applies a default seccomp profile that blocks ~44 of ~300 Linux syscalls. You can tighten it further, or apply a custom profile.\nStart with these security options for every container:\n1 2 3 4 5 6 7 8 services: my-app: image: my-web-app security_opt: - no-new-privileges:true # Prevent privilege escalation via suid binaries - seccomp=unconfined # ⚠️ Only for debugging — don\u0026#39;t use in production # Do this instead for production: # seccomp: profiles/default.json # Docker\u0026#39;s default is usually fine The no-new-privileges:true flag is the single most impactful security option. It prevents processes inside the container from gaining additional privileges through suid binaries or capset syscalls.\nFor a full security-optimized stack:\n1 2 3 4 5 6 7 8 9 10 11 services: app: image: my-app cap_drop: - ALL read_only: true tmpfs: - /tmp:noexec,nosuid,size=64M security_opt: - no-new-privileges:true # Use default seccomp profile (Docker applies it automatically) Hardening Layer 5: Resource Limits (prevent DoS) A single compromised container shouldn\u0026rsquo;t be able to exhaust host resources. Set explicit limits:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 services: my-app: image: my-web-app deploy: resources: limits: cpus: \u0026#34;1.0\u0026#34; # Max 1 CPU core memory: 512M # Max 512MB RAM reservations: cpus: \u0026#34;0.25\u0026#34; # Reserve 0.25 CPU memory: 256M # Reserve 256MB RAM # Legacy compose syntax (v2): # mem_limit: 512m # cpus: \u0026#34;1.0\u0026#34; # PID limit — prevent fork bombs inside the container pids_limit: 200 The pids_limit: 200 is often overlooked. A process that fork()s in an infinite loop can exhaust the host PID table. Limiting to 200 PIDs prevents this while being more than enough for most applications.\nRestart policies with limits:\n1 2 3 restart: unless-stopped stop_grace_period: 30s # Give the app time to shut down cleanly oom_kill_disable: false # Allow OOM killer — true can hide memory leaks Hardening Layer 6: Volume Mount Safety Bind mounts are inherently risky. If you mount /:/host (don\u0026rsquo;t do this), the container can modify anything. Safer patterns:\n1 2 3 4 5 6 7 8 9 10 11 services: my-app: volumes: # Read-only bind mounts — the container cannot modify them - ./config:/app/config:ro # Named volumes — safer than bind mounts, managed by Docker - app_data:/app/data # Never mount the Docker socket unless you absolutely need it # - /var/run/docker.sock:/var/run/docker.sock:ro # ⚠️ Container escape risk If you must mount the Docker socket (Portainer, Traefik, Watchtower), a read-only bind mount does not make Docker API requests read-only — direct socket access can control the host:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 services: watchtower: image: containrrr/watchtower volumes: - /var/run/docker.sock:/var/run/docker.sock:ro # Minimal exposure # Better alternative: use a Docker socket proxy docker-proxy: image: tecnativa/docker-socket-proxy volumes: - /var/run/docker.sock:/var/run/docker.sock:ro environment: CONTAINERS: 1 # Only allow container operations NETWORKS: 0 # Block network operations IMAGES: 1 # Allow image operations SERVICES: 0 # Block service operations TASKS: 0 # Block task operations POST: 1 # Allow POST requests The docker-socket-proxy pattern (covered in depth in the Docker Socket Proxy post) lets you whitelist specific API endpoints instead of giving full socket access. For security-minded homelabs, this is the correct approach.\nHardening Layer 7: Image Hygiene Your container is only as secure as the base image you build from.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # Scan for vulnerabilities docker scout quickstart docker scout cves nginx:alpine # Check base image for known CVEs # Use Trivy for offline scanning docker pull aquasec/trivy:latest docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \\ aquasec/trivy:latest image my-app # Use specific tags, not \u0026#34;latest\u0026#34; FROM node:20-alpine # Specific major version — good FROM node:alpine # No version — bad, \u0026#34;latest\u0026#34; changes FROM node:20.14.0-alpine # Pinned to patch — best for production # Distroless images have no shell, no package manager, no utilities FROM gcr.io/distroless/nodejs20-debian12:latest # If an attacker gets code execution, there\u0026#39;s no bash, no curl, no wget For a homelab, distroless images are overkill for most services. But pinning versions and scanning for known CVEs is table stakes.\nDockerfile security checklist:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 FROM node:20-alpine AS build # Build stage — has all the build tools FROM node:20-alpine AS production # Runtime stage — minimal dependencies RUN addgroup -S appgroup \u0026amp;\u0026amp; adduser -S appuser -G appgroup -u 1001 COPY --from=build --chown=appuser:appgroup /app /app USER appuser # Health check prevents the orchestrator from routing traffic to dead containers HEALTHCHECK --interval=30s --timeout=5s --retries=3 \\ CMD wget -qO- http://localhost:3000/health || exit 1 # Explicit EXPOSE documents the port EXPOSE 3000 ENTRYPOINT [\u0026#34;node\u0026#34;, \u0026#34;server.js\u0026#34;] The Complete Hardened Compose Template Combining all layers into a single production-ready template:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 version: \u0026#34;3.9\u0026#34; services: web: image: my-web-app:1.0.0 # Pinned version, not \u0026#34;latest\u0026#34; container_name: my-web-app restart: unless-stopped stop_grace_period: 30s # === User === user: \u0026#34;1001:1001\u0026#34; # Non-root user # === Capabilities === cap_drop: - ALL cap_add: - NET_BIND_SERVICE # Bind to port 80/443 # === Filesystem === read_only: true tmpfs: - /tmp:noexec,nosuid,size=64M volumes: - app_data:/app/data # Persistent data - ./config/app.conf:/etc/app.conf:ro # Read-only config # === Security Options === security_opt: - no-new-privileges:true # === Resource Limits === deploy: resources: limits: cpus: \u0026#34;1.0\u0026#34; memory: 512M reservations: cpus: \u0026#34;0.25\u0026#34; memory: 256M pids_limit: 200 # === Networking === networks: - internal-net ports: - \u0026#34;127.0.0.1:8080:3000\u0026#34; # Bind to localhost only — reverse proxy handles external # === Health Check === healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;wget\u0026#34;, \u0026#34;-qO-\u0026#34;, \u0026#34;http://localhost:3000/health\u0026#34;] interval: 30s timeout: 5s retries: 3 start_period: 10s # === Logging === logging: driver: \u0026#34;json-file\u0026#34; options: max-size: \u0026#34;10m\u0026#34; max-file: \u0026#34;3\u0026#34; networks: internal-net: internal: true # No external connectivity — locked down volumes: app_data: Verifying Your Hardening After applying these changes, verify each layer is active:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 # Check user inside container docker exec my-web-app whoami # Should output: appuser (not root) # Check capabilities docker exec my-web-app capsh --print # Current: = cap_net_bind_service+ep # (No SYS_ADMIN, no NET_ADMIN, no DAC_OVERRIDE) # Check read-only filesystem docker exec my-web-app touch /test # touch: /test: Read-only file system ✓ # Check writable paths docker exec my-web-app touch /app/data/test # Should succeed ✓ # Check security options docker inspect my-web-app --format \u0026#39;{{.HostConfig.SecurityOpt}}\u0026#39; # [no-new-privileges:true] # Check resource limits docker stats my-web-app --no-stream # CPU % MEM USAGE / LIMIT # 0.05% 128MiB / 512MiB Common Pitfalls When Hardening Containers \u0026ldquo;Permission denied\u0026rdquo; on startup: You set read_only: true but the app needs to write to a directory you didn\u0026rsquo;t map. Fix: add a tmpfs or volume for the required path.\n\u0026ldquo;Operation not permitted\u0026rdquo; for database: Databases need CHOWN, SETUID, SETGID to manage file permissions. Drop them and the container crashes on initdb.\nApp runs but can\u0026rsquo;t serve traffic: Port 80 requires NET_BIND_SERVICE. Switch to port 8080 or add the capability.\n\u0026ldquo;Bad system call\u0026rdquo; with seccomp: Some apps use syscalls blocked by the default seccomp profile (rare, but happens with Chrome/Puppeteer). Generate a custom profile instead of going unconfined:\n1 2 3 4 # Generate a seccomp profile from a running container docker run --rm it --security-opt seccomp=unconfined my-app # Capture strace output, generate profile from filtered syscalls # or use: dockersec generate my-app \u0026gt; custom-profile.json Summary: The Minimal Hardening Checklist These five changes cover 95% of the security improvement with minimal breakage:\nDefine a non-root USER in every Dockerfile or use user: in Compose — prevents container escape from yielding root on the host Drop ALL capabilities and add back only what the app needs — eliminates privilege escalation vectors Set read_only: true with tmpfs for runtime dirs — prevents attackers from modifying binaries or writing malware Enable no-new-privileges:true — blocks suid-based privilege escalation entirely Set pids_limit and memory limits — prevents a compromised container from DoS-ing the host Apply these to every new container you deploy. Retrofit them onto existing containers one at a time, testing each change. The first time you catch a container escape exploit failing because of these settings, you\u0026rsquo;ll wonder why you ever ran containers as root.\n","permalink":"https://blog.gntech.me/posts/2026-05-15-docker-container-security-hardening/","summary":"Hardening Docker containers in your homelab isn\u0026rsquo;t optional — it\u0026rsquo;s how you prevent container escapes and privilege escalation. This guide covers non-root users, capability drops, read-only filesystems, and seccomp profiles with real Compose configs.","title":"Docker Container Security — Non-Root Users, Capabilities, and Runtime Hardening"},{"content":"Promtail reached end of life on March 2, 2026. Grafana directs users toward Alloy or another supported client. See the Promtail notice.\nStart a migration with a controlled test stream. A running collector does not prove that Loki accepted its entries or that Grafana can query them.\nVerification status Area Evidence and limits Technical review Official Grafana references below reviewed on October 31, 2026. Static checks Bash syntax, embedded Compose YAML, and pipeline references checked. Runtime Docker and Alloy are unavailable in the review environment. No image pull, Alloy validation, ingestion, or Grafana query executed. Scope Linux Docker Engine with an existing trusted, single-tenant Loki test endpoint. Host files and production authentication are outside this example. 1. Check the prerequisites You need a working Loki instance, Grafana with access to it, and a Docker network shared with Loki. This article does not install or reconfigure Loki. The monitoring foundations cover metrics, not a preinstalled Loki service.\nFor this test, Loki must be reachable as http://loki:3100 from the selected Docker network. Confirm its service name or network alias. If yours differs, change the URL in the Alloy file below. Container localhost refers to that container, not the host or another service.\nChoose a supported Alloy image tag or digest and review its matching documentation. The commands ask for that image and the actual existing network name rather than silently selecting an unreviewed version.\nAlloy needs Docker API access. A read-only socket bind mount does not make Docker API operations read-only; treat socket access as privileged host access. Run this lab only on a trusted Docker host. The label filter limits collection, not API privileges.\n2. Create a separate test project Run in Bash:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 alloy_lab_dir=$(mktemp -d -t alloy-logs-lab.XXXXXX) cd \u0026#34;$alloy_lab_dir\u0026#34; || exit 1 read -r -p \u0026#39;Reviewed Alloy image tag or digest: \u0026#39; ALLOY_IMAGE read -r -p \u0026#39;Existing Docker network shared with Loki: \u0026#39; LOKI_NETWORK export ALLOY_IMAGE LOKI_NETWORK docker network inspect \u0026#34;$LOKI_NETWORK\u0026#34; cat \u0026gt; compose.yaml \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; services: alloy: image: ${ALLOY_IMAGE:?Set a reviewed Alloy image tag or digest} user: \u0026#34;0:0\u0026#34; command: - run - /etc/alloy/config.alloy - --storage.path=/var/lib/alloy/data - --server.http.listen-addr=0.0.0.0:12345 volumes: - ./config.alloy:/etc/alloy/config.alloy:ro - /var/run/docker.sock:/var/run/docker.sock:ro - alloy_state:/var/lib/alloy/data ports: - \u0026#34;127.0.0.1:12346:12345\u0026#34; networks: - logging log_demo: image: busybox:1.37 command: [\u0026#34;sh\u0026#34;, \u0026#34;-c\u0026#34;, \u0026#34;while true; do echo alloy-demo-check; sleep 5; done\u0026#34;] labels: com.gntech.alloy-demo: \u0026#34;true\u0026#34; logging: driver: json-file options: max-size: \u0026#34;1m\u0026#34; max-file: \u0026#34;2\u0026#34; networks: - logging volumes: alloy_state: networks: logging: external: true name: ${LOKI_NETWORK:?Set the existing Loki network name} EOF Only Alloy\u0026rsquo;s UI is published, bound to loopback on port 12346. Keep that interface private. This example runs Alloy as root for socket access; choose a reviewed permission model before production deployment.\nThe named state volume retains collector state across container recreation. That does not guarantee exactly-once delivery. Record image digests and tool versions for reproducibility.\n3. Define one connected pipeline 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 cat \u0026gt; config.alloy \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; discovery.docker \u0026#34;demo\u0026#34; { host = \u0026#34;unix:///var/run/docker.sock\u0026#34; filter { name = \u0026#34;label\u0026#34; values = [\u0026#34;com.gntech.alloy-demo=true\u0026#34;] } } loki.source.docker \u0026#34;demo\u0026#34; { host = \u0026#34;unix:///var/run/docker.sock\u0026#34; targets = discovery.docker.demo.targets labels = { job = \u0026#34;alloy_demo\u0026#34; } forward_to = [loki.write.destination.receiver] } loki.write \u0026#34;destination\u0026#34; { endpoint { url = \u0026#34;http://loki:3100/loki/api/v1/push\u0026#34; } } EOF Discovery selects only the labelled test container. The source reads its Docker logs and forwards them to the Loki writer. No host log files or collector self-logs are included.\nReview Docker discovery, Docker log collection, and Loki writing for supported arguments.\nThese are generally available components. They do not need an experimental stability setting. The old claim that omitting such a flag silently skips them was incorrect.\n4. Validate before starting Continue in the same shell and project directory:\n1 2 docker compose config --quiet docker compose run --rm --no-deps alloy validate /etc/alloy/config.alloy The supported command is alloy validate, not alloy tools CONFIG. A zero validation exit code establishes configuration validity; it does not prove permissions, network access, or delivery.\nIf either check fails, resolve its diagnostics before starting. For authenticated or multi-tenant Loki, configure the required authentication, TLS, and tenant settings using the writer documentation. Do not disable TLS verification to hide a certificate problem.\n5. Verify the marker end to end 1 2 3 4 docker compose up -d docker compose ps docker compose logs --tail=30 log_demo docker compose logs --tail=80 alloy Confirm the demo emits alloy-demo-check. Open Alloy\u0026rsquo;s UI at http://127.0.0.1:12346 on the Docker host and inspect the discovery, source, and writer components. An overall health response alone is insufficient evidence.\nIn Grafana, configure a Loki data source pointing to Loki\u0026rsquo;s base URL reachable from the Grafana server. Alloy is the collector, not the Loki query data source. If Grafana shares this network, that base URL may be http://loki:3100.\nOpen Explore, select the Loki data source, choose a recent time range, and run:\n{job=\u0026#34;alloy_demo\u0026#34;} |= \u0026#34;alloy-demo-check\u0026#34; Record a returned entry and its timestamp as the delivery check. If no result appears, work backwards: Grafana data-source URL and tenant, Loki acceptance errors, writer connection, discovery target, Docker logging availability, and source output. Allow time for discovery and batching; do not use configuration reload as a substitute for diagnosing target discovery.\nUse the Loki data-source documentation when configuring Grafana.\n6. Migrate deliberately Test a small production subset with separate labels or a separate destination before switching collectors. Running two collectors against identical sources can duplicate ingestion. Compare actual source coverage, labels, timestamps, query results, and error behavior.\nKeep a rollback configuration and preserve collector state. Retire the old collector only after the replacement is verified. Avoid a broad remove-orphans command that may remove unrelated services.\nLoki retention is separate from ingestion-age validation. Rejecting samples older than 168 hours does not delete stored logs after seven days. Configure and verify retention against the installed Loki version using the retention guide.\nA plain S3 endpoint is not a Loki push API. Archive configuration belongs to the appropriate storage system; do not send loki.write requests to an arbitrary bucket URL.\n7. Stop the test without deleting state From this lab directory:\n1 docker compose down This stops the demo and Alloy while preserving the named volume and external Loki network. It does not remove entries already ingested into Loki.\nBefore expanding to host logs, review file permissions and the source type. A read-only mount does not grant file-read permission, and some hosts use only journald. Do not make authentication logs world-readable as a troubleshooting shortcut.\nContinue reading Compose health checks — Distinguish startup, readiness, and recovery. Monitoring foundations — Keep the metrics and log paths clear. ","permalink":"https://blog.gntech.me/posts/2026-05-14-grafana-alloy-log-collection/","summary":"Verify one Docker log stream end to end before expanding collection or replacing Promtail.","title":"Grafana Alloy Log Collection — Docker Logs with Loki in 2026"},{"content":"Opening a firewall port for every homelab service is a liability. One vulnerability in a web dashboard and your LAN is exposed. Cloudflare Tunnel solves this by creating an encrypted outbound connection from your server to Cloudflare\u0026rsquo;s edge — no inbound ports required.\nAll traffic to your domain hits Cloudflare first, gets inspected, and is forwarded through the tunnel to your local service. HTTPS is automatic. DDoS protection is included. And the best part: there\u0026rsquo;s nothing to lock down on your router.\nThis guide covers a full Docker Compose deployment of cloudflared with multi-service ingress routing, authentication rules, and production-ready security.\nWhy Cloudflare Tunnel Over Port Forwarding Compare the two approaches:\nApproach Attack surface HTTPS Maintenance Port forwarding Your public IP, each open port Manual certs Firewall rules, cert renewal Cloudflare Tunnel Zero open ports Automatic One tunnel binary With port forwarding, every open port is a potential entry point. A forgotten admin panel on port 8443 becomes a foothold. With Cloudflare Tunnel, your server initiates an outbound TCP connection to Cloudflare and keeps it alive. No iptables rules, no NAT hairpinning, no DMZ configuration.\nThe tunnel also handles DNS automatically — each ingress entry creates a CNAME record on your Cloudflare zone.\nPrerequisites A domain using Cloudflare DNS (nameservers pointed to Cloudflare) Docker and Docker Compose installed on your homelab server SSH or console access to the host running Docker A Cloudflare account with Zero Trust enabled (free tier works) Step 1: Authenticate Cloudflared Cloudflare Tunnel requires authentication to link the tunnel to your account. The quickest way is running the login container interactively:\n1 2 3 docker run -it --rm \\ -v /opt/cloudflared:/home/nonroot/.cloudflared \\ cloudflare/cloudflared:latest tunnel login This opens a URL in your browser. Log in to Cloudflare and select the domain you want to use. The certificate saves to /opt/cloudflared/cert.pem.\nIf your server has no browser (headless Proxmox LXC), copy the URL from the output, open it on your desktop machine, authenticate, and the cert file will be written back.\nStep 2: Create and Route the Tunnel With the cert in place, create a named tunnel:\n1 2 3 docker run -it --rm \\ -v /opt/cloudflared:/home/nonroot/.cloudflared \\ cloudflare/cloudflared:latest tunnel create homelab This generates a tunnel UUID and saves credentials in /opt/cloudflared/\u0026lt;uuid\u0026gt;.json. You now have a tunnel ID and a credentials file — both needed for the Docker Compose setup.\nTest the tunnel with a quick run against a single service:\n1 2 3 docker run -it --rm \\ -v /opt/cloudflared:/home/nonroot/.cloudflared \\ cloudflare/cloudflared:latest tunnel --url http://localhost:8080 This starts an ad-hoc tunnel pointing at whatever is running on port 8080. Press Ctrl+C when confirmed working.\nStep 3: Docker Compose Configuration Create a directory structure for your tunnel config:\n1 2 3 4 5 6 7 8 9 10 # /opt/cloudflared/docker-compose.yml services: cloudflared: image: cloudflare/cloudflared:latest container_name: cloudflared restart: unless-stopped command: tunnel run volumes: - ./config:/etc/cloudflared:ro network_mode: host The network_mode: host is intentional — cloudflared needs to reach services listening on localhost and Docker bridge IPs. With host networking, localhost inside the container is the host\u0026rsquo;s localhost, so you can tunnel any service regardless of its network setup.\nConfig File — Ingress Rules Create config/config.yml to define which domains map to which local services:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 # /opt/cloudflared/config/config.yml tunnel: homelab credentials-file: /etc/cloudflared/\u0026lt;your-uuid\u0026gt;.json ingress: - hostname: dashboard.yourdomain.com service: http://localhost:9090 - hostname: grafana.yourdomain.com service: http://localhost:3000 - hostname: frigate.yourdomain.com service: http://localhost:5000 - hostname: status.yourdomain.com service: http://localhost:8081 - hostname: uptime.yourdomain.com service: http://192.168.1.100:3001 - hostname: traefik.yourdomain.com service: http://localhost:8080 # Catch-all — return 404 for unmatched hostnames - service: http_status:404 Place the tunnel credentials JSON in the same config/ directory:\n1 cp /opt/cloudflared/\u0026#34;\u0026lt;uuid\u0026gt;\u0026#34;.json /opt/cloudflared/config/ Your config directory should look like:\n/opt/cloudflared/ ├── docker-compose.yml └── config/ ├── config.yml └── \u0026lt;uuid\u0026gt;.json Route DNS Records Tell Cloudflare which domains use this tunnel:\n1 2 3 4 5 6 docker run -it --rm \\ -v /opt/cloudflared/config:/etc/cloudflared \\ cloudflare/cloudflared:latest tunnel route dns homelab dashboard.yourdomain.com docker run -it --rm \\ -v /opt/cloudflared/config:/etc/cloudflared \\ cloudflare/cloudflared:latest tunnel route dns homelab grafana.yourdomain.com Repeat for each hostname in your ingress config. Cloudflare automatically creates CNAME records pointing to the tunnel ID.\nStart the Tunnel 1 2 cd /opt/cloudflared docker compose up -d Check the logs:\n1 docker compose logs -f You should see lines like:\nINF Connection 5c3c8a7d-... registered INF Each of the 2 active connections is... If you see connection errors, check your config file for typos in hostnames and verify the credentials file path.\nStep 4: Securing Your Tunnel A tunnel is only as secure as the applications it exposes. Here are the essential hardening steps.\nApplication-Level Authentication with Zero Trust Cloudflare Zero Trust (free tier) adds an authentication gate before traffic reaches your service. Go to Cloudflare Dashboard → Zero Trust → Access → Applications, create a self-hosted application, point it at your domain, and configure a policy (e.g., \u0026ldquo;Email OTP\u0026rdquo; or \u0026ldquo;Google Workspace login\u0026rdquo;).\nTraffic flow becomes:\nUser → Cloudflare Edge → [Auth Check] → Tunnel → Service If the user hasn\u0026rsquo;t authenticated, Cloudflare shows a login page before the request ever reaches your server. This is better than HTTP basic auth because you get MFA, device posture checks, and per-app policies — all without modifying your apps.\nService-Level IP Whitelisting Even though no ports are open, lock down the services themselves so they reject traffic that didn\u0026rsquo;t come via the tunnel. Add this to each service\u0026rsquo;s Docker Compose or application config.\nFor Traefik or Caddy, restrict access to Docker bridge networks or specific paths. For Proxmox, enable the firewall and allow only the Docker host IP.\nUse a Separate Tunnel for Sensitive Services Don\u0026rsquo;t put your SSH access behind the same ingress setup. Create a second tunnel or use Cloudflare\u0026rsquo;s browser-based SSH via Zero Trust. Keep management interfaces (Proxmox, SSH, Docker API) off the public ingress entirely unless protected with Zero Trust policies.\nStep 5: Running cloudflared as a Non-Root Container Cloudflare publishes cloudflared with a non-root user. The docker-compose.yml above uses it already. To verify:\n1 2 docker exec cloudflared whoami # nonroot If you\u0026rsquo;re binding volumes, ensure file permissions allow the nonroot user (UID 1000 inside the container) to read the config:\n1 2 3 chown -R 1000:1000 /opt/cloudflared/config/ chmod 600 /opt/cloudflared/config/*.json chmod 644 /opt/cloudflared/config/config.yml Troubleshooting Common Issues Tunnel starts but returns 502 Your ingress service: URL is wrong or the destination isn\u0026rsquo;t listening. Test with curl from the host:\n1 curl -I http://localhost:3000 If that works, check config.yml for exact hostname casing and port mismatches.\nDNS records not resolving DNS routing creates CNAME records, and CNAMEs can coexist on the same zone as other record types. If a record isn\u0026rsquo;t resolving, re-run:\n1 cloudflared tunnel route dns homelab yourservice.yourdomain.com Or create the CNAME manually in the Cloudflare dashboard targeting \u0026lt;tunnel-id\u0026gt;.cfargotunnel.com.\nTunnel disconnects frequently Cloudflared keeps persistent TCP connections. Frequent drops usually indicate ISP-level issues (CGNAT, aggressive firewall, or packet loss). Add the following to config.yml to reconnect aggressively:\n1 2 retries: 5 grace-period: 5s Or pass --protocol quic to use QUIC instead of TCP for the control plane, which handles NAT rebinding better.\nUpdating the tunnel When you add a new ingress rule, just edit config.yml and restart cloudflared:\n1 docker compose restart That\u0026rsquo;s it. No DNS changes, no firewall updates, no certificate renewal.\nProduction Checklist Set up Cloudflare Zero Trust Access policies for critical apps Run chmod 600 on credentials files Pin cloudflared to a specific version tag, not latest Monitor tunnel uptime with a health check URL Keep tunnel logs shipped to your Grafana/Loki stack Test failover — stop cloudflared and confirm services are unreachable from outside (no backdoor ports) Use separate tunnels for internal vs external services Enable audit logging in Cloudflare Zero Trust Summary Cloudflare Tunnel eliminates port forwarding entirely while adding automatic HTTPS, DDoS protection, and a global CDN in front of your homelab services. A single cloudflared container handles ingress for dozens of services with a straightforward YAML config file.\nCombined with Zero Trust access policies, you get enterprise-grade remote access to your self-hosted apps without the attack surface of open ports. For the cost of a domain name, it\u0026rsquo;s the safest way to expose services from behind NAT, CGNAT, or a dynamic IP.\n","permalink":"https://blog.gntech.me/posts/2026-05-14-cloudflare-tunnel-docker/","summary":"Run Cloudflare Tunnel in Docker to expose homelab services through Cloudflare\u0026rsquo;s edge. No open firewall ports, automatic HTTPS, and per-service ingress rules via a single cloudflared container.","title":"Cloudflare Tunnel with Docker — Expose Homelab Services Securely"},{"content":"Windows Hello for Business (WHfB) replaces passwords with strong two-factor authentication tied to the device. Users sign in with a PIN, biometric (fingerprint, face), or FIDO2 security key instead of a password.\nIn a hybrid deployment (on-premises AD + Microsoft Entra ID), WHfB authenticates users to both on-premises and cloud resources. Key trust is one of three trust models:\nTrust Model PKI Required Key Storage Best For Key trust ✅ Yes (DC certs) Device TPM Hybrid environments with existing PKI Cloud Kerberos trust ❌ No Device TPM ⭐ Recommended — simpler, no PKI needed Certificate trust ✅ Yes (user + DC certs) Device TPM Legacy apps requiring explicit certs Microsoft recommends cloud Kerberos trust over key trust. However, key trust is still widely deployed in existing PKI environments and is the focus of this post. If you are starting fresh in 2026, evaluate cloud Kerberos trust first.\nThis post assumes the infrastructure from the previous four posts:\n✅ Entra Connect sync (Post #2) ✅ Device writeback (Post #3) ✅ Hybrid join (Post #4) ✅ AD users synced to Entra ID Screenshots and images to add later:\nAD CS installation via Server Manager CA configuration — enterprise root CA Domain Controller certificate template configuration WSUS DC-autoenrollment GPO WHfB policy in GPMC — Use Windows Hello for Business PIN complexity policy settings dsregcmd /status showing Ngc data Entra ID — user sign-in logs showing WHfB authentication How WHfB Key Trust Works User signs in with PIN/biometric on hybrid-joined device → Windows creates asymmetric key pair in TPM → Public key published to Entra ID (via sync) → msDS-KeyCredentialLink attribute written to user in AD → KDC on domain controller validates user\u0026#39;s key against attribute → Kerberos TGT issued using WHfB key (not password) → User authenticated to on-prem + cloud resources The domain controller needs a Kerberos Authentication certificate so the device trusts the KDC. The enterprise CA issues this certificate. Without it, WHfB enrollment fails with cryptic errors.\nPrerequisites Infrastructure Component Requirement Windows Server 2025 PKI server AD CS installed, enterprise root CA Domain controller certificate Kerberos Authentication cert on every DC Entra Connect v2, syncing, device writeback enabled Hybrid join Devices hybrid joined (Post #4) Entra ID P1/P2 license Required for WHfB in hybrid key trust Windows 11 / Windows 10 Devices must support TPM 2.0 Windows Server 2025 Optional — WHfB on server (Desktop Experience only) Network Devices need line of sight to:\nDomain controller (Kerberos authentication) Enterprise CA (certificate enrollment) enterpriseregistration.windows.net (device registration) Users Each user must have a synced on-prem AD account Users must be licensed for Entra ID (P1/P2) Users must register for device registration: Entra admin center → Identity → Devices → Device settings → \u0026#34;Users may register devices with Microsoft Entra ID\u0026#34; → All Step 1 — Deploy Active Directory Certificate Services If you do not have an enterprise PKI, spin up a CA on a Windows Server 2025 member server (not a domain controller).\nInstall AD CS 1 2 3 4 5 6 7 8 # On the PKI server (e.g., CA01.gntech.me) # Run as Enterprise Admin # Install the AD CS role with management tools Add-WindowsFeature Adcs-Cert-Authority -IncludeManagementTools # Configure as an Enterprise Root CA Install-AdcsCertificationAuthority -CACommonName \u0026#34;GNTECH Root CA\u0026#34; -CAType EnterpriseRootCa -CryptoProviderName \u0026#34;RSA#Microsoft Software Key Storage Provider\u0026#34; -KeyLength 4096 -ValidityPeriod Years -ValidityPeriodUnits 10 -HashAlgorithmName SHA256 The wizard alternative: Server Manager → Manage → Add Roles and Features → Select Active Directory Certificate Services → Check Certification Authority → Install → Configure → Enterprise CA → Root CA → Create new private key → Default settings → Complete.\nConfigure Certificate Templates Domain controllers need a Kerberos Authentication certificate. The default Domain Controller template works, but for clarity, create a dedicated template:\nOpen Certification Authority snap-in on the CA server Right-click Certificate Templates → Manage Right-click Kerberos Authentication template → Duplicate Template On Compatibility tab: Certification Authority: Windows Server 2016 Certificate recipient: Windows Server 2016 On General tab: Template display name: GNTECH Domain Controller Auth Validity period: 5 years (default) On Request Handling: Purpose: Signature and encryption Check Allow private key to be exported On Cryptography tab: Provider Category: Key Storage Provider Algorithm: RSA, minimum key size 2048 Request hash: SHA256 On Extensions tab: Verify Application Policies includes Kerberos Authentication On Security tab: Add Domain Controllers group → Enroll permission Remove other groups from Enroll except Domain Controllers Click OK Now publish the template:\nIn Certification Authority snap-in, right-click Certificate Templates → New → Certificate Template to Issue Select GNTECH Domain Controller Auth → OK Issue DC Certificates Domain controllers auto-enroll certificates when they find an enterprise CA. To trigger immediately:\n1 2 3 4 5 # On each domain controller, run as Domain Admin certreq -autoenroll # Verify certificate issued Get-ChildItem Cert:\\LocalMachine\\My | Where-Object { $_.EnhancedKeyUsageList -match \u0026#34;Kerberos Authentication\u0026#34; } | Format-Table Subject, NotAfter, Thumbprint If no certificate appears, check:\nDoes the DC have network access to the CA? Is the Domain Controllers group allowed to enroll the template? Run gpupdate /force and certreq -autoenroll again. For automated enrollment via GPO for all DCs:\nGroup Policy Management → Default Domain Controllers Policy → Edit → Computer Configuration → Policies → Windows Settings → Security Settings → Public Key Policies → Certificate Services Client - Auto-Enrollment → Configuration Model: Enabled → Check: Renew expired certificates, update pending certificates, remove revoked certificates → Check: Update certificates that use certificate templates Verify DC Certificate 1 2 3 4 # Verify the DC cert is trusted (issued by the root CA installed) # Check that the cert\u0026#39;s Intended Purposes include \u0026#34;Kerberos Authentication\u0026#34; certlm.msc # Personal → Certificates → double-click the DC cert → Details → Enhanced Key Usage Also verify LDAP over SSL works (DC cert enables this):\n1 2 3 # Quick LDAPS test Test-NetConnection -ComputerName DC01.gntech.me -Port 636 # Should show TcpTestSucceeded: True Step 2 — Configure Entra Connect Key Sync WHfB key trust requires the user\u0026rsquo;s public key (from their TPM) to be synced to Entra ID. Entra Connect v2 does this automatically when the correct attribute (msDS-KeyCredentialLink) is in the sync scope.\nVerify the Attribute is Syncing On the Entra Connect server, open Synchronization Rules Editor (Start menu → Microsoft Entra Connect → Synchronization Rules Editor) Look for the rule Out to AAD - User Join (or similar) Verify that msDS-KeyCredentialLink is mapped to keyCredentials in the transformation If not present, edit the rule to add:\nFlowType: Direct Source Attribute: msDS-KeyCredentialLink Target Attribute: keyCredentials Or run the Entra Connect wizard → Customize synchronization options → Optional Features → ensure Directory extension attribute sync is enabled.\nForce a Full Sync After Config Changes 1 Start-ADSyncSyncCycle -PolicyType Initial Step 3 — Deploy WHfB Policy (GPO) Enable Windows Hello for Business Open Group Policy Management Console Create a new GPO: Windows Hello for Business — Hybrid Link it to the domain gntech.me (or to a specific OU with WHfB-eligible users and computers) Configure Policy Settings Navigate to:\nComputer Configuration → Policies → Administrative Templates → Windows Components → Windows Hello for Business Set these:\nRequired:\nPolicy Setting Use Windows Hello for Business Enabled Recommended (Optional):\nPolicy Setting Use a hardware security device Enabled (requires TPM 2.0) Minimum PIN length Enabled → 6 Maximum PIN length Enabled → 127 Require lowercase letters in PIN Enabled → Allowed Require uppercase letters in PIN Enabled → Allowed Require special characters in PIN Disabled or Allowed Expiration Enabled → 90 days History Enabled → 10 For a homelab, less restrictive PIN is fine:\nPolicy Setting Minimum PIN length Enabled → 4 Require lowercase letters in PIN Disabled Require uppercase letters in PIN Disabled Require special characters in PIN Disabled Configure Smart Card (Applies to WHfB) Additionally, groups can be excluded from WHfB:\nComputer Configuration → Policies → Administrative Templates → Windows Components → Windows Hello for Business → \u0026#34;Disable Windows Hello for Business on devices where the target machine is Azure AD joined or has already registered for PIN reset or Hello\u0026#34; Leave this Not Configured (default) in standard deployments.\nDeploy the GPO 1 gpupdate /force Wait for replication, or trigger remotely:\n1 Invoke-GPUpdate -Computer \u0026#34;WS01\u0026#34; -RandomDelayInMinutes 0 Step 4 — Enroll in Windows Hello for Business User-Initiated Enrollment After GPO deployment, users set up WHfB from their hybrid-joined device:\nSettings → Accounts → Sign-in options Under Windows Hello, click Set up for PIN, Fingerprint, or Face Windows Security opens → Create a PIN Enter current password to verify identity Enter new PIN (confirm) Click OK The device generates an asymmetric key pair in the TPM. The public key is published to Entra ID and synced back to the user\u0026rsquo;s AD object (msDS-KeyCredentialLink attribute).\nVerify Enrollment 1 dsregcmd /status Look for:\n+----------------------------------------------------------------------+ | Ngc Data (a.k.a. Windows Hello for Business) | +----------------------------------------------------------------------+ ngcState: YES NgcContainerId: \u0026lt;guid\u0026gt; IsReady: YES IsEnabled: YES IsPreKeyRecoveryEnabled: False DevicePKeyId: \u0026lt;guid\u0026gt; SignId: \u0026lt;guid\u0026gt; Also check the AD attribute:\n1 2 # On a domain controller, verify the user\u0026#39;s key credential link Get-ADUser jsmith -Properties msDS-KeyCredentialLink | Select-Object -ExpandProperty msDS-KeyCredentialLink If empty, the public key has not synced from Entra ID. Check:\nEntra Connect attribute sync for msDS-KeyCredentialLink ↔ keyCredentials A full sync cycle Sync errors in Synchronization Service Manager Entra Admin Center Verification Sign in to the Entra admin center Go to Identity → Users → All users → select user → Sign-in logs Look for authentication events showing: Windows Hello for Business in Authentication requirement Primary refresh token as token type Step 5 — Test Passwordless Sign-In Local Sign-In Lock the device (Win + L) On the lock screen, use the PIN (or biometric) instead of password Verify you can access: On-premises network shares (Kerberos) Microsoft 365 apps (Outlook, Teams) — SSO via PRT Entra admin center (browser — promptless) Remote (RDP) Consideration WHfB does not work for RDP logins by default. To use WHfB with RDP:\nEnable certificate authentication for RDP: GPO → Computer Configuration → Administrative Templates → Windows Components → Remote Desktop Services → Remote Desktop Session Host → Security → \u0026#34;Require use of specific security layer for remote (RDP) connections\u0026#34; → Set to: Negotiate → \u0026#34;Require user authentication for remote connections by using Network Level Authentication\u0026#34; → Enabled The user must have a WHfB certificate (not just key pair — requires certificate trust model) Alternatively, authenticate with PIN on the console, then RDP using password or smart card For a homelab, WHfB is primarily a console sign-in feature.\nSingle Sign-On With hybrid key trust configured, users get:\nResource How It Works On-prem AD domain resources Kerberos ticket issued via WHfB key (no password required) Microsoft 365 (Outlook, Teams, SharePoint) PRT obtained via WHfB enrollment Azure Portal SSO via PRT through seamless SSO Entra ID integrated SaaS apps SAML/OIDC tokens issued via PRT On-premises web apps using ADFS Device-based Conditional Access via device writeback The Primary Refresh Token (PRT) is the key. WHfB enrollment generates a PRT that is cached on the device and renewed automatically.\nTroubleshooting Enrollment Fails — \u0026ldquo;Something went wrong\u0026rdquo; Symptom Likely Cause Fix \u0026ldquo;Key registration failed\u0026rdquo; DC certificate missing or untrusted Install Kerberos Auth cert on DC \u0026ldquo;Windows Hello isn\u0026rsquo;t available\u0026rdquo; TPM not available Check tpm.msc — TPM 2.0 required \u0026ldquo;PIN setup failed\u0026rdquo; GPO not applied rsop.msc or gpresult /h to verify DC Not Trusted 1 2 3 4 5 6 # Check if domain controller has a valid Kerberos cert on the device certutil -viewstore -enterprise CA certutil -DCInfo # Verify DC certificate chain CertReq -enroll -machine -policyserver \u0026#34;CA01.gntech.me\\GNTECH Root CA\u0026#34; -Policy \u0026#34;GNTECH Domain Controller Auth\u0026#34; WHfB Not Available After GPO Deploy 1 2 3 4 5 6 7 8 # Force GP update gpupdate /force # Check registry keys applied reg query HKLM\\SOFTWARE\\Policies\\Microsoft\\PassportForWork # Check the specific policy value reg query HKLM\\SOFTWARE\\Policies\\Microsoft\\PassportForWork /v Enabled Expected: Enabled REG_DWORD 0x1\nKey Sync Not Working 1 2 3 4 5 # Check user\u0026#39;s msDS-KeyCredentialLink in AD Get-ADUser jsmith -Properties msDS-KeyCredentialLink, msDS-KeyCredentialLink2 # If empty, check Entra Connect sync rules # The attribute must be mapped from AD to Entra ID User Cannot Sign In After Enrollment 1 2 3 4 5 6 7 8 # Check user token state whoami /upn # Verify Kerberos ticket klist # Force re-authentication klist purge If the user can\u0026rsquo;t authenticate, they can still use their password as fallback. WHfB doesn\u0026rsquo;t disable password sign-in unless explicitly configured via GPO (EnableWindowsHelloForBusiness + PIN-only policies).\nWhat We Built Component Detail Enterprise PKI (AD CS) Root CA on Windows Server 2025 DC certificates Kerberos Authentication cert issued to each DC WHfB key sync msDS-KeyCredentialLink synced via Entra Connect WHfB policy GPO: PIN complexity, TPM requirement User enrollment PIN set up on hybrid-joined device SSO Passwordless sign-in to on-prem + cloud resources Next in the Series Post 6: Self-Service Password Reset (SSPR)\nEnable SSPR for users: registration, authentication methods, writeback testing, and Conditional Access integration.\nReferences WHfB hybrid key trust deployment guide Plan WHfB deployment Configure WHfB enrollment (hybrid key trust) WHfB policy settings AD CS deployment guide KDC certificates for Windows Hello ","permalink":"https://blog.gntech.me/posts/2026-05-14-windows-server-2025-windows-hello-hybrid-key-trust/","summary":"Deploy Windows Hello for Business in hybrid key trust mode: enterprise PKI setup, DC certificates, GPO policy, user enrollment, PIN configuration, and verification.","title":"Windows Server 2025: Windows Hello for Business — Hybrid Key Trust"},{"content":"Hybrid join connects your on-premises domain-joined devices to Microsoft Entra ID without changing how they join the domain. A Windows 11 workstation or Windows Server 2025 member server that is already domain-joined to gntech.me also registers with Entra ID as a device identity. This enables:\nSingle sign-on (SSO) to cloud resources (Microsoft 365, Azure, SaaS apps) from the domain-joined desktop — no separate credentials Conditional Access policies that require hybrid-joined or compliant devices Windows Hello for Business (covered in Post #5) Enterprise State Roaming — sync Windows settings across devices This post builds on the device writeback configured in Post #3, which is a prerequisite. Device writeback ensures that device registrations in Entra ID create computer objects in on-premises AD.\nScreenshots and images to add later:\nEntra Connect wizard — Configure device options page Entra Connect — SCP configuration page Group Policy Management Editor — SCP registry keys dsregcmd /status output (successful hybrid join) Device registration Scheduled Task in Task Scheduler Verification script output (Azure registration connectivity) Entra admin center — All devices showing hybrid joined devices How Hybrid Join Works The registration flow:\nDomain-joined device boots → Workstation service checks SCP in AD → SCP points to Entra ID tenant → Device Registration Scheduled Task triggers → Device authenticates to Entra ID (device certificate) → Registration completes → Device object in Entra ID → Device writeback → Computer object in on-prem AD The key piece is the Service Connection Point (SCP). The SCP is stored in AD under the device registration configuration container. It tells domain-joined devices which Entra ID tenant to register with.\nDevices read the SCP automatically via the Workstation service (LANManWorkstation) during startup and schedule the device registration task.\nPrerequisites From the previous posts in this series:\n✅ Microsoft Entra Connect v2 installed and syncing (Post #2) ✅ Device writeback enabled (Post #3) Entra ID P1 or P2 license (device registration requires it) Hybrid Identity Administrator account Enterprise Administrator credentials for AD Windows 10/11 or Windows Server 2016/2019/2022/2025 devices (Server Core does NOT support device registration) Domain controllers at minimum Windows Server 2008 R2 Network Connectivity Devices need outbound HTTPS (TCP 443) access from the internal network to:\nURL Purpose https://enterpriseregistration.windows.net Device registration https://login.microsoftonline.com Authentication https://device.login.microsoftonline.com Device auth https://autologon.microsoftazuread-sso.com Seamless SSO If you use TLS inspection (break-and-inspect), exclude the device registration URLs from inspection. The device registration flow uses client certificates for authentication, and TLS interception breaks the certificate chain.\nVerify Device Writeback First Confirm device writeback is working:\n1 2 # On the Entra Connect server Get-ADSyncAADCompanyFeature | Select DeviceWriteback, DeviceWritebackForest Expected:\nDeviceWriteback : Enabled DeviceWritebackForest : gntech.me Also verify that a RegisteredDevices container exists in AD:\n1 Get-ADOrganizationalUnit -Filter \u0026#34;Name -eq \u0026#39;RegisteredDevices\u0026#39;\u0026#34; -SearchBase \u0026#34;DC=gntech,DC=me\u0026#34; | Format-Table Name, DistinguishedName If missing, run the Entra Connect wizard → Configure device options → provide Enterprise Admin credentials to create it.\nMethod 1: SCP via Entra Connect (Easiest) This is the recommended method for single-forest deployments. Entra Connect writes the SCP to AD automatically.\nStep 1 — Run Entra Connect Device Configuration On the Entra Connect server, run Microsoft Entra Connect Click Configure Select Configure device options → Next On Overview, click Next Enter Hybrid Identity Administrator credentials On Device options, select Configure hybrid join → Next Step 2 — Configure SCP for Managed Domain If you use password hash sync (which we set up in Post #2), the wizard shows:\nSCP configuration for managed domain page Select the forest gntech.me In Device writeback target forest, confirm gntech.me is selected Authentication Service section: Enter the UPN suffix for on-premises users (e.g., gntech.me) Or select User name if your on-prem domain matches cloud UPN Click Next Step 3 — Configure Device OU On the Device container page: Device writeback forest: gntech.me Device container for computers joined: Browse to the OU where you want hybrid-joined computers to appear If you ran device writeback already, the RegisteredDevices container is available Click Next → Configure The wizard writes the SCP to AD and triggers device writeback configuration if not already done.\nStep 4 — Verify SCP 1 2 3 # On a domain controller or Entra Connect server # Find the SCP object in AD Get-ADObject -Filter {objectClass -eq \u0026#34;serviceConnectionPoint\u0026#34;} -SearchBase \u0026#34;CN=Device Registration Configuration,CN=Services,CN=Configuration,DC=gntech,DC=me\u0026#34; -Properties * | Select Name, Keywords, ServiceBindingInformation Expected: Keywords contains azureADId:\u0026lt;tenant-id\u0026gt; and ServiceBindingInformation contains your tenant\u0026rsquo;s registration URL.\nMethod 2: SCP via GPO (Alternative) If you cannot run the Entra Connect wizard (e.g., multi-forest, restricted admin access), configure the SCP via registry policy.\nStep 1 — Build the GPO Open Group Policy Management Console on a management server Create a new GPO named Entra Hybrid Join SCP Link it to the domain gntech.me Step 2 — Configure Registry Keys Navigate the GPO editor to:\nComputer Configuration → Preferences → Windows Settings → Registry Create two registry entries:\nAction Key Path Value Name Value Data Update HKLM\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\CDJ\\AAD TenantId your-tenant-id Update HKLM\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\CDJ\\AAD TenantName gntech.me Where your-tenant-id is the GUID of your Entra ID tenant:\n1 2 3 # Get your tenant ID (Get-AzureADTenantDetail).ObjectId # Or from Entra admin center → Identity → Overview → Tenant ID Step 3 — Configure Scheduled Task The same GPO or a separate GPO deploys the registration task:\nComputer Configuration → Preferences → Control Panel Settings → Scheduled Tasks → New → Scheduled Task (Windows 10 and later) Name: CreateMicrosoftEntraJoinTask Security options: Run as NT AUTHORITY\\SYSTEM, run whether user is logged on or not Trigger: At system startup, repeat every 1 hour for 1 day Action: Start a program → C:\\Windows\\System32\\dsregcmd.exe Arguments: /join Conditions: Uncheck \u0026ldquo;Start the task only if the computer is on AC power\u0026rdquo; Wait for GPO refresh (gpupdate /force) and the next scheduled task execution.\nVerification On the Device Run as administrator:\n1 dsregcmd /status The output shows:\n+----------------------------------------------------------------------+ | Device State | +----------------------------------------------------------------------+ AzureAdJoined: YES EnterpriseJoined: NO DeviceId: \u0026lt;guid\u0026gt; Thumbprint: \u0026lt;cert-thumbprint\u0026gt; KeyContainerId: \u0026lt;guid\u0026gt; KeyProvider: Microsoft Platform Crypto Provider TpmProtected: YES KeySignTest: : PASSED Idp: login.microsoftonline.com TenantId: \u0026lt;your-tenant-id\u0026gt; TenantName: gntech.me AuthCodeUrl: https://login.microsoftonline.com/\u0026lt;tenant\u0026gt;/oauth2/authorize AccessTokenUrl: https://login.microsoftonline.com/\u0026lt;tenant\u0026gt;/oauth2/token +----------------------------------------------------------------------+ | Domain Name Services | +----------------------------------------------------------------------+ ... +----------------------------------------------------------------------+ | SSO Data | +----------------------------------------------------------------------+ AzureAdPrt: YES ... +----------------------------------------------------------------------+ | Ngc Data (a.k.a. Windows Hello for Business) | +----------------------------------------------------------------------+ ngcState: NOT_SET ... Key indicators of success:\nField Expected Value AzureAdJoined YES KeySignTest PASSED (requires elevated) AzureAdPrt YES (Primary Refresh Token obtained) If AzureAdJoined is NO, the device has not completed registration.\nConnectivity Test Download and run the Microsoft connectivity test script:\n1 2 3 4 5 6 # Download the test script Invoke-WebRequest -Uri \u0026#34;https://raw.githubusercontent.com/azure-samples/TestDeviceRegConnectivity/master/TestDeviceRegConnectivity.ps1\u0026#34; -OutFile \u0026#34;$env:TEMP\\TestDeviceRegConnectivity.ps1\u0026#34; # Run (elevated) Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass .\\$env:TEMP\\TestDeviceRegConnectivity.ps1 The script tests each required URL and reports pass/fail.\nIn Entra Admin Center Sign in to the Entra admin center Go to Identity → Devices → All devices The device should appear with: Join type: Hybrid Azure AD joined (the legacy label may still show this) Registered: date/time of registration Owner: user who was logged in during registration On-Premises AD The device should also appear in the device writeback container:\n1 Get-ADComputer -Filter * -SearchBase \u0026#34;CN=RegisteredDevices,DC=gntech,DC=me\u0026#34; -Properties * | Select Name, LastLogonDate, OperatingSystem The Registration Process in Detail When a domain-joined device boots:\nWorkstation Service (LANManWorkstation) queries AD for SCPs in CN=Device Registration Configuration,CN=Services,CN=Configuration,\u0026lt;domain\u0026gt; If an SCP with keyword azureADId:\u0026lt;tenant-id\u0026gt; is found, the service schedules the CreateMicrosoftEntraJoinTask (Scheduled Task) The task runs dsregcmd /join in system context The device authenticates to enterpriseregistration.windows.net using its domain computer account credentials Entra ID issues a device identity and certificate The device stores the certificate in the TPM (if available) Entra Connect syncs the device registration to on-prem AD via device writeback This process is automatic and transparent to users. No reboot is required after SCP configuration — the task triggers within an hour.\nTrigger Immediately for Testing 1 2 # On the target device, as administrator dsregcmd /join Refresh registration status:\n1 dsregcmd /status If the device is already registered, dsregcmd /join is a no-op.\nForce Re-registration 1 2 3 4 5 6 7 8 # Leave the join (will remove from Entra ID) dsregcmd /leave # Re-join dsregcmd /join # Verify dsregcmd /status Windows Server 2025 as Hybrid Joined Windows Server 2025 member servers (not domain controllers) support hybrid join. This is useful for:\nConditional Access on server-management tools — require hybrid join to access Entra ID protected APIs Windows Hello for Business on server — used for service accounts or admin access Managed servers that need SSO to Microsoft 365 or Azure Important: Server Core does NOT support device registration. Only Windows Server with Desktop Experience (GUI) can hybrid join.\nTo check if a Server 2025 device supports registration:\n1 2 3 4 5 6 # Check Windows edition (Get-WmiObject Win32_OperatingSystem).Caption # \u0026#34;Microsoft Windows Server 2025 Standard\u0026#34; → Desktop Experience hybrid join supported # Check Server Core # \u0026#34;Microsoft Windows Server 2025 Standard\u0026#34; with ServerCoreEnabled=1 → NOT supported SSO with Hybrid Joined Devices Once hybrid joined, users get SSO to:\nMicrosoft 365 (Outlook, Teams, SharePoint, OneDrive for Business) — no password prompts on domain-joined machines Azure Portal — via seamless SSO Entra ID integrated apps — SAML/OIDC apps federated with Entra ID On-premises resources — Kerberos still works for on-prem apps The Primary Refresh Token (PRT) is obtained during device registration. You can verify:\n1 2 3 dsregcmd /status # Look for: # AzureAdPrt: YES If AzureAdPrt is NO, the user may need to sign out and sign back in, or the device needs a gpupdate /force.\nTroubleshooting dsregcmd /status Shows AzureAdJoined: NO Symptom Likely Cause Fix No SCP found AD SCP not configured Run Entra Connect device config or deploy GPO SCP found but join fails Network blocked Test with TestDeviceRegConnectivity.ps1 Registration timeout TLS inspection breaking cert auth Exclude device registration URLs from break-and-inspect \u0026ldquo;Access denied\u0026rdquo; Device computer account lacks auth Ensure device OU is synced by Entra Connect dsregcmd /status Shows KeySignTest: FAILED Run the command elevated (as Administrator). If it still fails:\n1 2 # Check TPM status tpm.msc Device Shows \u0026ldquo;Pending\u0026rdquo; in Entra Admin Center A Pending state means the device registration started but did not complete. Common causes:\nNetwork connectivity issues TLS inspection blocking certificate authentication Device clock skew (NTP not synced) 1 2 3 4 5 # Check time sync w32tm /query /status # Force sync w32tm /resync Device Not Appearing in Entra Admin Center Verify hybrid join completed on the device Verify device writeback is enabled in Entra Connect Run a delta sync on the Entra Connect server Check Synchronization Service Manager → Operations for device export errors Dual State (Azure AD Registered + Hybrid Joined) If a device was previously Azure AD registered (work or school account added manually) and is now hybrid joined, it may appear twice in Entra admin center. The hybrid join entry is authoritative. The Azure AD registered entry can be removed under device settings.\nWhat We Built Component Status SCP in AD Configured via Entra Connect or GPO Device writeback Enabled (Post #3) Hybrid join registration Automatic via Scheduled Task SSO to cloud resources Enabled via PRT Verification dsregcmd /status + Entra admin center Next in the Series With devices hybrid joined and SSO working, the next post covers passwordless authentication:\nPost 5: Windows Hello for Business — Hybrid Key Trust\nDeploy WHfB in hybrid key trust mode: GPO configuration, certificate registration, PIN/policy setup, and enrollment verification.\nReferences Configure Microsoft Entra hybrid join Plan your hybrid join deployment Troubleshoot hybrid joined devices Device registration connectivity test script How SSO to on-premises resources works on Entra joined devices dsregcmd command reference ","permalink":"https://blog.gntech.me/posts/2026-05-14-windows-server-2025-entra-hybrid-join/","summary":"Configure Microsoft Entra hybrid join for on-premises domain-joined devices: SCP via GPO or Entra Connect, automatic registration startup via Scheduled Task, verification with dsregcmd, and troubleshooting registration failures.","title":"Windows Server 2025: Entra ID Hybrid Join + Device Registration"},{"content":"Writeback is the reverse direction of hybrid identity. In the previous post, Entra Connect synchronized users, groups, and passwords from on-premises Active Directory to Microsoft Entra ID. Writeback moves data in the opposite direction — from the cloud back to your on-premises AD.\nThree writeback types matter in a modern hybrid environment:\nWriteback What It Does Why You Need It Password Cloud password changes/ resets are written back to on-premises AD in real time Enables Self-Service Password Reset (SSPR); users reset passwords from anywhere Device Selected device registrations are represented as msDS-Device objects in AD Specific AD FS device-access and Windows Hello hybrid certificate-trust scenarios; not a general prerequisite for hybrid join Group (M365) Microsoft 365 groups are created in on-premises AD Mail-enabled security groups for hybrid Exchange; deprecated for cloud-only groups Important deprecation notice: Group Writeback v2 in Entra Connect Sync is deprecated. Microsoft recommends using Entra Cloud Sync for provisioning cloud security groups to on-premises AD. If you need Microsoft 365 groups written back, Group Writeback v1 is still supported through Entra Connect Sync. Read the group writeback section carefully before enabling.\nPrerequisites This post builds on the previous one. Before starting:\nMicrosoft Entra Connect v2 installed and syncing (Post #2) Password hash sync enabled and verified At least Microsoft Entra ID P1 license (all writeback features require P1 or P2) Hybrid Identity Administrator account in Entra ID Enterprise Admin credentials for on-premises AD Initial sync cycle completed (all users and groups visible in cloud) Password Writeback Password writeback synchronizes password changes made in the cloud back to on-premises AD. When a user changes or resets their password through the SSPR portal (https://aka.ms/sspr), it is written to on-prem AD in real time via the Entra Connect Azure Service Bus relay.\nHow It Works User → SSPR portal → Entra ID → Azure Service Bus → Entra Connect → AD DS (password updates in real time) The connection is outbound only from the Entra Connect server. No inbound firewall rules are needed. The Entra Connect server opens a persistent outbound connection to an Azure Service Bus relay, and password changes flow through it synchronously — the user sees immediate feedback if the new password violates on-premises AD policy.\nStep 1 — Verify the actual connector and permissions Identify the AD DS connector account from the installed Entra Connect configuration; do not guess it from an MSOL_* wildcard or grant permissions to a placeholder account. Use Microsoft\u0026rsquo;s connector-account permissions procedure for the selected feature and scoped OUs. Reset Password alone is not a complete permissions checklist. Review protected accounts and inherited permissions separately, then test with an intended, non-privileged account.\nStep 2 — Enable Password Writeback in Entra Connect On the Entra Connect server, run Microsoft Entra Connect from the Start menu Click Configure Select Customize synchronization options → Next Enter credentials when prompted Navigate through the wizard until Optional Features Check the box Password writeback Complete the wizard and commit the configuration A full sync cycle runs to apply the change.\nStep 3 — Enable Password Writeback in SSPR Sign in to the Microsoft Entra admin center Go to Identity → Protection → Password reset Under Properties, set Self-service password reset enabled to Selected or All Go to Authentication methods and configure what users can use: Number of methods required: 2 Methods available: Mobile phone, Email, Security questions, etc. Go to Registration → set Require users to register when signing in to Yes Go to On-premises integration: Set Write back passwords to your on-premises directory to Yes Verify the selected on-premises integration settings. SSPR does not offer an arbitrary writeback destination OU; permissions and the synchronized user determine the target account. Click Save Step 4 — Test Password Writeback Via SSPR portal:\nBrowse to https://aka.ms/sspr on a test user\u0026rsquo;s device Enter the test user\u0026rsquo;s UPN (e.g., testuser@gntech.me) Complete the CAPTCHA Verify identity via configured method (phone, email) Enter a new password and confirm Check on-prem AD immediately: 1 2 # Verify the password changed on-premises Get-ADUser \u0026#34;testuser\u0026#34; -Properties PasswordLastSet, LastBadPasswordAttempt Via admin reset:\nIn the Entra admin center, go to Identity → Users → All users Select a test user → Reset password Set a temporary password and click Reset Verify on-prem AD password was updated: 1 2 # Check that PasswordLastSet updated Get-ADUser \u0026#34;testuser\u0026#34; -Properties PasswordLastSet, LastLogonDate Step 5 — Monitor Password Writeback 1 2 # Check Entra Connect event logs for password writeback events Get-WinEvent -LogName \u0026#34;Application\u0026#34; -ProviderName \u0026#34;PasswordWriteBack\u0026#34; | Select TimeCreated, Message In Synchronization Service Manager, look for export runs with PasswordChange or PasswordSet export types.\nDevice Writeback Device writeback creates msDS-Device registration objects under a configured RegisteredDevices container. These are not ordinary AD computer accounts. It supports specific AD FS device-based access scenarios and Windows Hello for Business hybrid certificate trust; it is not generally required for Microsoft Entra hybrid join or cloud Kerberos trust.\nFollow Microsoft\u0026rsquo;s device-writeback configuration and verification procedure. Review the selected forest, container, licensing, supported trust design, connector permissions, and installed Entra Connect version before enabling it. Keep directory permissions narrowly scoped and verify the generated registration objects after synchronization.\nOn an administrative machine with the AD module, the following is a read-only inventory for the current AD domain. If writeback targets another domain, select that domain explicitly first:\n1 2 3 $domainDn = (Get-ADDomain).DistinguishedName Get-ADObject -LDAPFilter \u0026#39;(objectClass=msDS-Device)\u0026#39; -SearchBase \u0026#34;CN=RegisteredDevices,$domainDn\u0026#34; | Select-Object Name, ObjectClass, DistinguishedName The container may not exist before configuration, and synchronization is not instantaneous. A registration object alone does not establish that an AD FS relying party or Windows Hello deployment works; verify the specific end-to-end trust scenario.\nStep 4 — Troubleshooting Device Writeback If devices do not appear:\nIs the device registered in Entra ID? Entra admin center → Identity → Devices → All devices Has a sync cycle completed since registration? 1 Start-ADSyncSyncCycle -PolicyType Delta Check for device writeback errors in: Synchronization Service Manager → Operations → look for export errors Group Writeback Group writeback provisions cloud groups to on-premises AD. There are two scenarios with different recommendations:\nScenario Recommendation Microsoft 365 groups (Teams, Outlook groups) Group Writeback v1 via Entra Connect Sync (still supported) Cloud-only security groups Use Entra Cloud Sync (Group Writeback v2 is deprecated) Scenario 1: M365 Group Writeback (v1) Microsoft 365 groups (formerly Office 365 Groups) are managed in the cloud but need to exist on-premises for Exchange hybrid scenarios or on-premises applications that read security groups from AD.\nStep 1 — Enable in Entra Connect\nRun Microsoft Entra Connect → Configure Select Customize synchronization options Navigate through the wizard to Optional Features Check Group writeback (if it appears — v1 must be enabled in the sync rules editor if the checkbox is absent) Select the destination OU for written-back groups Complete the wizard Step 2 — Enable in Entra Admin Center\nSign in to the Entra admin center Go to Identity → Groups → All groups Click Group settings Set Write back Microsoft 365 groups to on-premises to Yes Save Step 3 — Verify\n1 2 # Check synced cloud groups in on-prem AD Get-ADGroup -Filter * -SearchBase \u0026#34;OU=CloudGroups,DC=gntech,DC=me\u0026#34; -Properties * | Select Name, GroupCategory, GroupScope, Mail Only groups with a Microsoft 365 group type (not Security, not Distribution) are written back. Their groupType in AD is SECURITY_GROUP | DISTRIBUTION_GROUP | UNIVERSAL_GROUP to preserve mail flow.\nScenario 2: Cloud Security Groups (via Entra Cloud Sync) Group Writeback v2 in Entra Connect Sync is deprecated. For provisioning cloud security groups to on-premises AD, Microsoft recommends Microsoft Entra Cloud Sync:\n1 2 3 4 5 6 # Install Cloud Sync agent # Download from Entra admin center → Hybrid management → Cloud Sync # Run on the Entra Connect server (or a separate member server) # After installation, configure via: # Entra admin center → Hybrid management → Cloud Sync → Group provisioning Cloud Sync provisions cloud security groups as universal security groups in AD. It can run alongside Entra Connect Sync — Entra Connect handles user sync, while Cloud Sync handles group writeback.\nHomelab note: Group writeback (both v1 and Cloud Sync) requires real-world multicloud management to be useful. For a homelab without Exchange hybrid or Teams governance, skip group writeback unless you have a specific use case.\nWriteback Licensing Summary Feature License Required Password writeback Entra ID P1 or P2 Device writeback Entra ID P1 or P2 Group writeback v1 (M365 groups) Entra ID P1 or P2 + Exchange hybrid Group writeback Cloud Sync (security groups) Entra ID P1 or P2 Without at least P1 licenses, the writeback options do not appear in the Entra Connect wizard.\nPost-Installation Verification Verify each feature through its own workflow Review the installed wizard\u0026rsquo;s feature configuration and connector operations. For password writeback, complete an authorized SSPR test with a scoped test account and verify the on-premises result. For device writeback, inspect the selected container\u0026rsquo;s msDS-Device objects. For groups, verify scope, memberships, and the intended group-provisioning mechanism.\nDo not infer enabled features from a fabricated PowerShell output block or unverified property names. Record the installed tool version and observed results. This source audit did not modify a tenant or execute directory writeback.\nMonitor with Synchronization Service Manager Open C:\\Program Files\\Microsoft Azure AD Sync\\UIShell\\MiisClient.exe:\nOperations tab — filter by connector and look for Export runs Each export shows the number of: adds — new objects written back (first sync of a device/group) updates — existing objects updated (password change sync) deletes — objects removed Check Synchronization Statistics 1 2 # View last sync results Get-ADSyncExportStatistics | Format-Table Connector, ExportType, Status, ObjectType, ExportCount Troubleshooting Writeback Password Writeback Fails Symptom Likely Cause Fix \u0026ldquo;Password does not meet complexity requirements\u0026rdquo; On-prem AD policy is more restrictive than cloud Align policies or check AD password filter \u0026ldquo;Writeback timeout\u0026rdquo; Network latency to Azure Service Bus Check outbound connectivity from Entra Connect server \u0026ldquo;Access denied\u0026rdquo; AD Connector account lacks reset password permission Grant via ADSI Edit or dsacls 1 2 # Force a writeback connectivity test Test-ADSyncPasswordWriteback -AdConnectorName \u0026#34;gntech.me\u0026#34; -AADConnectorName \u0026#34;Entra ID\u0026#34; Device Writeback Fails Symptom Likely Cause Fix Device not in AD after sync Container not configured Run Entra Connect wizard → Configure device options Export error \u0026ldquo;no-writeback-container\u0026rdquo; Device writeback OU missing Create manually with the provided PowerShell script Device shows in Entra but not on-prem Device registered before writeback was enabled Trigger a delta sync 1 2 3 # Export device writeback container configuration $config = Get-ADSyncDeviceWritebackConfiguration $config | Format-List Group Writeback Fails Symptom Likely Cause Fix M365 group not in AD Exchange hybrid not configured Configure Exchange hybrid or check group writeback v1 is on \u0026ldquo;Group writeback not enabled\u0026rdquo; Sync rule not configured Enable via Entra Connect Optional Features Cloud Sync group not provisioned Scope not configured Check Cloud Sync group provisioning settings When to Skip Writeback No on-premises Exchange? Skip group writeback v1 No supported AD FS/device certificate-trust requirement? Do not enable device writeback merely because hybrid join is planned. Small lab with same admin passwords? Password writeback is optional — but SSPR is a good demo feature If unsure, enable password writeback (it adds real value) and skip the others until the next post in the series requires them.\nWhat We Built Feature Status Password writeback (SSPR) Enabled — cloud password resets sync to on-prem AD Device writeback Enabled — registered devices appear in AD Group writeback v1 (M365 groups) Optional — skip unless Exchange hybrid exists Group writeback Cloud Sync Optional — skip unless cloud security groups need to exist on-prem Next in the Series With writeback configured, devices can be registered in both on-premises AD and Entra ID simultaneously. The next post covers:\nPost 4: Entra ID Hybrid Join + Device Registration\nConfigure automatic device registration via GPO, verify hybrid join status, and enable SSO to cloud resources from domain-joined devices.\nReferences Password writeback concept Enable password writeback tutorial Device writeback Group writeback (v1) for M365 groups Group writeback via Cloud Sync ","permalink":"https://blog.gntech.me/posts/2026-05-14-windows-server-2025-writeback-cloud-on-premises/","summary":"Enable password writeback (SSPR), device writeback (supported AD FS scenarios), and group writeback (Microsoft 365 groups) from cloud to on-premises Active Directory using Entra Connect.","title":"Windows Server 2025: Writeback — Cloud to On-Premises (Password, Device, and Group)"},{"content":"A running container can still be unable to serve requests. A health check runs a probe inside the container; Compose can wait for a passing probe before starting a dependent service.\nThat startup gate does not replace application retry logic or ongoing monitoring. Choose a probe that measures the capability the consumer actually needs.\nVerification status Area Evidence and limits Technical review Official references below reviewed on October 31, 2026. Static checks Bash syntax, YAML parsing, and example structure checked. Runtime Docker is unavailable in the review environment; no image pull, Compose validation, or container execution performed. Example scope Disposable Redis lab, not an authenticated or persistent production deployment. 1. Separate health, startup, and restarts Mechanism What it does Health check Reports starting, healthy, or unhealthy from probe results. depends_on with service_healthy Gates a dependent service\u0026rsquo;s startup on the dependency\u0026rsquo;s health. Container restart policy Handles process exits; unhealthy status alone is not an exit. Compose does not automatically stop or restart existing consumers whenever a dependency later becomes unhealthy. Applications still need to recover from lost connections. Dependency-level restart: true concerns explicit Compose operations and differs from the service\u0026rsquo;s process restart policy.\nSee Compose startup order.\n2. Create a disposable lab Use a supported Docker Engine and the docker compose plugin. This demo publishes no host ports and contains no production data or credentials. The Redis Alpine image provides its client and the shell tools used here.\nRun in Bash. A new temporary directory separates the example from existing projects:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 health_lab_dir=$(mktemp -d -t compose-health-lab.XXXXXX) cd \u0026#34;$health_lab_dir\u0026#34; || exit 1 docker version docker compose version cat \u0026gt; compose.yaml \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; services: redis: image: redis:7.4-alpine command: [\u0026#34;redis-server\u0026#34;, \u0026#34;--save\u0026#34;, \u0026#34;\u0026#34;, \u0026#34;--appendonly\u0026#34;, \u0026#34;no\u0026#34;] healthcheck: test: - CMD-SHELL - \u0026#34;test ! -f /tmp/force-unhealthy \u0026amp;\u0026amp; redis-cli ping | grep -qx PONG\u0026#34; interval: 5s timeout: 3s retries: 3 start_period: 10s client: image: redis:7.4-alpine depends_on: redis: condition: service_healthy command: [\u0026#34;sh\u0026#34;, \u0026#34;-c\u0026#34;, \u0026#34;redis-cli -h redis ping | grep -x PONG\u0026#34;] restart: \u0026#34;no\u0026#34; EOF docker compose config --quiet Record image digests and tool versions when you run the exercise; version-series tags can change.\nThe file test is a lab-only failure switch. The Redis portion expects PING to return PONG. It does not test credentials, application data, persistence, or remote connectivity.\n3. Start and inspect Continue in the same lab directory:\n1 2 3 4 5 6 7 docker compose up -d docker compose ps -a docker compose logs client redis_id=$(docker compose ps -q redis) test -n \u0026#34;$redis_id\u0026#34; \u0026amp;\u0026amp; docker inspect --format \u0026#39;{{json .State.Health}}\u0026#39; \u0026#34;$redis_id\u0026#34; Expected behavior to verify: Redis becomes healthy, then Compose starts the client. The client prints PONG and exits successfully because it is a one-shot check. Confirm its exit code in docker compose ps -a.\nThe local probe and the client request exercise different paths. Neither proves a complete application works.\n4. Test unhealthy status without stopping the process Use this only in the disposable project:\n1 2 docker compose exec redis touch /tmp/force-unhealthy docker compose ps -a Wait for consecutive probe failures and repeat the inspection commands. Expected result: Redis remains running while its health becomes unhealthy. The completed client does not rerun automatically.\nRestore the probe without restarting Redis:\n1 2 docker compose exec redis rm /tmp/force-unhealthy docker compose ps -a After a passing probe, health should return to healthy. Inspect stored probe output rather than assuming a fixed delay. This simulates probe failure, not a real Redis outage.\nWhen finished, remove only this disposable project\u0026rsquo;s containers, network, and anonymous volumes:\n1 docker compose down --volumes Run cleanup from the lab directory. Do not copy the volume-removal option into a production cleanup routine.\n5. Tune timing and command execution interval controls regular checks, timeout limits each probe, and retries counts consecutive failures. During start_period, early failures are initially ignored; checks still run. A success during that period ends the initial failure exemption.\nOptional start_interval requires Compose 2.20.2 or later and Engine 25.0 or later. A top-level file version does not enable it. The old claim that service_healthy itself requires Compose 2.20 was incorrect.\nUse CMD for a direct executable invocation. Use CMD-SHELL for shell operators or container-side variable expansion. The required shell and executable must exist in the actual image.\nSee the Compose healthcheck reference and Docker HEALTHCHECK behavior.\nRead probe output and service logs before increasing grace periods. Missing tools, authentication failures, and incorrect URLs need correction, not longer delays.\n6. Choose database probes with precise limits These are fragments for existing services, not additional runnable stacks.\nFor PostgreSQL configured with user and database named myapp:\n1 2 3 4 5 6 healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;pg_isready\u0026#34;, \u0026#34;-h\u0026#34;, \u0026#34;127.0.0.1\u0026#34;, \u0026#34;-U\u0026#34;, \u0026#34;myapp\u0026#34;, \u0026#34;-d\u0026#34;, \u0026#34;myapp\u0026#34;] interval: 10s timeout: 5s retries: 5 start_period: 30s pg_isready reports whether the server accepts connections. It does not prove credentials authenticate or migrations finished. Add an application-specific query or readiness endpoint when needed.\nFor a compatible official MariaDB image:\n1 2 3 4 5 6 healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;healthcheck.sh\u0026#34;, \u0026#34;--connect\u0026#34;, \u0026#34;--innodb_initialized\u0026#34;] interval: 10s timeout: 5s retries: 5 start_period: 40s Check MariaDB healthcheck.sh against the image version and initialized volume. Older volumes may need setup for the helper\u0026rsquo;s health-check account. Prefer its supported helper over passing a root password to a ping command.\nFor variable-based probes, remember that Compose normally interpolates dollar expressions before creation. Use $$ to preserve a literal dollar sign for later container-shell expansion. Exec-form CMD does not itself expand shell variables. See Compose interpolation.\n7. Extend only after the baseline works Replace the one-shot client with a real application image or complete build context. Define a readiness endpoint that fails until required initialization completes, and ensure the probe tool exists.\nKeep reconnect logic even with startup gating. Avoid circular dependencies and unnecessary gates that delay monitoring or log collection. Test initial startup and a later dependency outage in isolation.\nRecord tool versions, image digests, configuration validation, probe logs, consumer results, and recovery behavior. YAML parsing alone cannot validate image contents or service readiness.\nContinue reading Monitoring foundations — Add external visibility beyond a local probe. Ansible homelab automation — Manage configuration after verifying behavior. ","permalink":"https://blog.gntech.me/posts/2026-05-14-docker-compose-healthchecks/","summary":"A small Redis lab explains startup gating, health transitions, and the limits of automatic recovery.","title":"Docker Compose Health Checks — Reliable Service Startup Orchestration"},{"content":"Hybrid identity is the bridge. On-premises Active Directory holds your users, groups, and computers. Microsoft Entra ID (formerly Azure AD) powers Microsoft 365, Azure, and third-party SaaS apps. Without synchronization, you manage two directories — and they inevitably drift apart.\nMicrosoft Entra Connect is the tool that synchronizes on-premises AD objects to Entra ID. It replaces the older DirSync and Azure AD Sync tools. As of 2026, only Entra Connect v2 is supported, and Microsoft has announced that all sync services will stop on September 30, 2026 if you are not on at least version 2.5.79.0.\nThis guide walks through installing Entra Connect v2 on Windows Server 2025, configuring password hash sync, OU-based filtering, optional writeback features (password, device, group), staging mode, and verification.\nScreenshots and images to add later:\nEntra Connect download page in Entra admin center Express settings welcome screen Connect to Microsoft Entra ID credentials dialog Domain/OU filtering page Optional features page (password/device/group writeback) Ready to configure summary Synchronization Service Manager UI Synchronization statistics showing imported/exported objects Entra admin center showing synced users Staging mode configuration Prerequisites Microsoft Entra (Cloud) A Microsoft Entra tenant (get one with an Azure free trial) A verified custom domain (e.g., gntech.me — not just tenant.onmicrosoft.com) A Hybrid Identity Administrator account in the Entra tenant (or Global Administrator) At least a Microsoft Entra ID Free license (P1/P2 unlocks features like writeback, password protection, and Self-Service Password Reset) Verify Your Domain If your domain is not yet verified in Entra ID:\nSign in to the Microsoft Entra admin center Go to Identity → Settings → Domain names Click Add custom domain, enter gntech.me Add the TXT verification record to your public DNS zone Wait for propagation and click Verify On-Premises Active Directory AD schema + forest functional level: Windows Server 2003 or later (Server 2025 is fine) Domain controller must be writable (no RODC) At least one DC in the same site as the Entra Connect server (or accessible via low-latency link) Entra Connect Server Hardware Entra Connect v2 must run on a Windows Server 2016 or later. Windows Server 2025 is fully supported.\nComponent Minimum Recommended CPU 1.6 GHz, 2 cores 2.0 GHz, 4 cores RAM 8 GB 16 GB Disk 70 GB 100 GB (SSD) The server should be:\nDomain-joined to gntech.me Not a domain controller (install on a member server) Running Windows Server 2025 Standard or Datacenter Software Prerequisites Entra Connect v2 installs these automatically, but confirm:\n.NET Framework 4.7.2 or later (Server 2025 ships with newer versions) PowerShell 5.0 or later TLS 1.2 enabled 1 2 3 4 5 6 7 8 # Verify TLS 1.2 is enabled [System.Net.ServicePointManager]::SecurityProtocol # Should include: Tls12 # Enable TLS 1.2 system-wide if needed New-Item \u0026#39;HKLM:\\SYSTEM\\CurrentControlSet\\Control\\SecurityProviders\\SCHANNEL\\Protocols\\TLS 1.2\\Client\u0026#39; -Force New-ItemProperty \u0026#39;HKLM:\\SYSTEM\\CurrentControlSet\\Control\\SecurityProviders\\SCHANNEL\\Protocols\\TLS 1.2\\Client\u0026#39; -Name Enabled -Value 1 -PropertyType DWORD New-ItemProperty \u0026#39;HKLM:\\SYSTEM\\CurrentControlSet\\Control\\SecurityProviders\\SCHANNEL\\Protocols\\TLS 1.2\\Client\u0026#39; -Name DisabledByDefault -Value 0 -PropertyType DWORD Network Requirements The Entra Connect server needs outbound HTTPS (TCP 443) to:\n*.msappproxy.net *.servicebus.windows.net login.microsoftonline.com *.microsoftonline.com In a homelab with restricted outbound internet, ensure your firewall allows these destinations. A proxy can be configured during installation.\nClean Up Your AD Data Run the IdFix tool before installing Entra Connect. It identifies duplicate UPNs, invalid characters, and formatting issues that cause sync errors:\n1 2 3 4 # Download and run IdFix Invoke-WebRequest -Uri \u0026#34;https://github.com/Microsoft/idfix/releases/latest/download/IdFix.zip\u0026#34; -OutFile \u0026#34;$env:TEMP\\IdFix.zip\u0026#34; Expand-Archive -Path \u0026#34;$env:TEMP\\IdFix.zip\u0026#34; -DestinationPath \u0026#34;C:\\IdFix\u0026#34; .\\C:\\IdFix\\IdFix.exe IdFix scans your AD and presents errors. Click Accept on each action to fix them, then click Apply.\nDownloading Entra Connect Entra Connect v2 is available only from the Microsoft Entra admin center (not from the download center\u0026rsquo;s public page):\nSign in to the Microsoft Entra admin center Navigate to Identity → Hybrid management → Microsoft Entra Connect → Connect Sync Click Download Microsoft Entra Connect → the installer is named AzureADConnect.msi (version 2.x.x.x) Express Installation (Single Forest) The Express option is for single-forest AD with password hash sync. It configures everything with defaults — perfect for a homelab.\nStep 1 — Run the Installer 1 2 # From the download location .\\AzureADConnect.msi Step 2 — Express Settings On the Welcome screen, accept the license terms and click Continue\nClick Express Settings (or Customize for advanced options)\nConnect to Microsoft Entra ID\nEnter your Hybrid Identity Administrator credentials (e.g., admin@gntech.me) Click Next Connect to Active Directory\nEnter your on-premises AD Enterprise Admin credentials (GNTECH\\Administrator) Entra Connect discovers your local AD forest automatically Click Next Ready to Configure\nReview the summary: password hash sync enabled, auto upgrade on, no writeback Check the box Start the synchronization process as soon as the configuration completes Click Install Installation takes 5–15 minutes. When complete, the wizard shows the Configuration complete screen.\nStep 3 — Verify Initial Sync On the Entra Connect server, open Synchronization Service Manager:\n1 2 # Launch from Start menu or \u0026#34;C:\\Program Files\\Microsoft Azure AD Sync\\UIShell\\MiisClient.exe\u0026#34; Look for:\nManagement Agents tab → gntech.me (on-prem AD) and Microsoft Entra ID (cloud) agents present Operations tab → a completed Initial Sync run with imported and exported objects In the Entra admin center:\nGo to Identity → Users → All users Confirm users from gntech.me appear with Synced from on-premises in the Source column Check Identity → Groups → All groups for synced security groups Custom Installation Use Custom when you need:\nMulti-forest topologies OU filtering (sync only specific OUs) Optional features (password writeback, device writeback, group writeback) Federated sign-in (AD FS or 3rd party) Staging mode Step 1 — Run Custom Install 1 .\\AzureADConnect.msi On Welcome, click Customize (not Express).\nStep 2 — Options Option Recommendation for Homelab Password Hash Synchronization ✅ Enabled — simplest and most secure for lab Pass-through Authentication ❌ Skip — requires more infrastructure (connector agents) Federation with AD FS ❌ Skip — overkill for single-domain homelab Do not configure ❌ Avoid — you need at least one sign-in method Enable device writeback ✅ Enable if you plan to deploy Hybrid Entra Join Enable directory extension attribute sync ✅ Enable (adds AD attributes to cloud) Enable password writeback ⏹ Enable after sync is verified (see Post #3) Step 3 — OU Filtering (Important) By default, Entra Connect syncs all OUs in the domain. In a homelab with test users, guest accounts, or service accounts that should not sync to the cloud, restrict it:\nOn the Domain/OU filtering page, select Sync selected domains and OUs Check only the OUs you want to sync: ✓ DC=gntech,DC=me ├─ ✓ OU=Users ├─ ✓ OU=Workstations ├─ ✓ OU=Servers ├─ ✓ OU=Groups └─ ✗ OU=ServiceAccounts └─ ✗ OU=Test Click Next OU filtering can be changed later, but a full sync cycle runs after each change.\nStep 4 — Attribute Sync Configuration On the Optional Features page:\nCheck Directory extension attribute sync if you need custom AD attributes in Entra ID (useful for custom claims or apps) Check Password writeback and Device writeback (covered in Post #3) Leave Exchange hybrid deployment unchecked unless you run Exchange on-premises.\nStep 5 — Install Review the summary, verify settings, check Start the synchronization process and click Install.\nPost-Installation Verification 1. Synchronization Service Manager 1 2 # Open Synchronization Service Manager Start-Process \u0026#34;C:\\Program Files\\Microsoft Azure AD Sync\\UIShell\\MiisClient.exe\u0026#34; Check the Operations tab. A successful initial sync shows:\nImport (staging) → objects pulled from AD and from Entra ID Synchronization (delta) → differences computed Export → changes written to target Expected exports after initial sync:\nConnector = gntech.me (on-prem): near-zero exports (cloud objects written back, e.g., device writeback) Connector = Entra ID: users, groups, contacts exported to cloud 2. PowerShell Verification 1 2 3 4 5 6 7 8 # Check sync cycle status Get-ADSyncScheduler # View scheduler configuration Get-ADSyncScheduler | Format-List # Check connector status Get-ADSyncConnector | Format-Table Name, ConnectorType, TimeCreated 3. Entra Admin Center Identity → Users → All users — verify synced users Identity → Groups → All groups — verify groups Identity → Devices → All devices — verify if device writeback is enabled 4. UPN Mismatch Check If a user\u0026rsquo;s on-prem UPN suffix is not verified in Entra ID, the default domain (tenant.onmicrosoft.com) is used as fallback. Verify your custom domain is added and the UPN suffixes match:\n1 2 3 4 5 # Check all UPN suffixes in AD Get-ADForest | Select-Object -ExpandProperty UPNSuffixes # Check a specific user Get-ADUser \u0026#34;jsmith\u0026#34; -Properties UserPrincipalName | Select UserPrincipalName All UPN suffixes should be verified in Entra ID.\nManaging Sync Scope (OU Filtering) After installation, you can change which OUs are synced:\nVia GUI Start Microsoft Entra Connect from the Start menu Click Configure → Customize synchronization options Enter credentials when prompted On Domain/OU filtering, adjust selections Complete the wizard — a full sync cycle runs automatically Via PowerShell 1 2 3 4 5 6 7 8 # Disable sync for a specific OU Set-ADSyncOrganizationalUnit -OrganizationalUnit \u0026#34;OU=Test,DC=gntech,DC=me\u0026#34; -Enabled $false # Enable sync for an OU Set-ADSyncOrganizationalUnit -OrganizationalUnit \u0026#34;OU=Users,DC=gntech,DC=me\u0026#34; -Enabled $true # Start a full sync to apply changes Start-ADSyncSyncCycle -PolicyType Initial Attribute-Based Filtering (Advanced) Beyond OU filtering, you can filter objects by attribute:\n1 2 3 # Sync only users in a specific group # This requires modifying the Synchronization Rules in the Sync Rules Editor # Recommended approach: use OU filtering for simplicity in homelabs Attribute filtering is managed through the Synchronization Rules Editor (found in the Start menu under Microsoft Entra Connect). Each sync rule can filter by attribute value. This is advanced — for a homelab, OU filtering is sufficient.\nThe Sep 30, 2026 Upgrade Deadline Microsoft announced that all Entra Connect Sync services will stop working on September 30, 2026 if you are not on at least version 2.5.79.0.\nCheck Your Version 1 2 3 4 5 # One-liner (Get-ItemProperty \u0026#34;HKLM:\\SOFTWARE\\Microsoft\\Azure AD Connect\\\u0026#34;).Version # Or via the installer path Get-Item \u0026#34;C:\\Program Files\\Microsoft Azure AD Sync\\Bin\\miiskmu.exe\u0026#34; | Select-Object ProductVersion Upgrade Download the latest AzureADConnect.msi from the Entra admin center Run the installer — it detects the existing installation and upgrades in place Click Upgrade (not reinstall) No configuration changes are needed. The wizard preserves all settings.\nWhat Happens If You Miss the Deadline All sync cycles stop New users, groups, and password changes in AD are not replicated to Entra ID Password change/SSPR writeback fails Existing users and groups remain in the cloud (no data deletion) To restore sync, upgrade Entra Connect — a full sync cycle runs after Scheduling and Performance Entra Connect syncs on a schedule. The default is every 30 minutes.\n1 2 3 4 5 6 7 8 # View schedule Get-ADSyncScheduler | Select CustomizedSyncCycleInterval # Adjust sync interval (minimum 30 minutes) Set-ADSyncScheduler -CustomizedSyncCycleInterval 00:30:00 # Force an immediate sync Start-ADSyncSyncCycle -PolicyType Delta For a homelab, 30 minutes is fine. For testing, you can trigger delta syncs on demand.\nStaging Mode Staging mode installs Entra Connect without exporting changes to the cloud. The server imports and syncs locally but does not write to Entra ID. This lets you verify the configuration before going live.\nEnable During Installation On the Ready to configure page, check Enable staging mode.\nEnable After Installation 1 2 3 4 5 6 7 8 9 # Via GUI: restart the wizard → Configure → Staging mode # Via PowerShell: Set-ADSyncScheduler -StagingModeEnabled $true # Start sync in staging mode (imports and syncs but no exports) Start-ADSyncSyncCycle -PolicyType Initial # Check staging status Get-ADSyncScheduler | Select StagingModeEnabled To exit staging mode, set StagingModeEnabled $false and run a full sync cycle. The next export writes all pending changes to Entra ID.\nWhen to Use Staging Mode Before going live with a first-time sync After configuration changes (new OU, changed attribute mapping) As disaster recovery standby — install a second Entra Connect server in staging mode. If the primary fails, promote staging to production by disabling staging mode. Disaster Recovery Entra Connect stores its configuration in the ADSync database (local SQL Express by default, or a full SQL Server instance). To recover:\nInstall a new member server Run the Entra Connect installer Choose Customize → Restore from a previously created backup (requires the %ProgramData%\\AADConnect\\ backup, exported earlier) Export the configuration periodically:\n1 2 # Export configuration Export-ADSyncSyncConfig -Path \u0026#34;C:\\Backups\\sync-config-$(Get-Date -Format \u0026#39;yyyy-MM-dd\u0026#39;).json\u0026#34; The real disaster recovery best practice is a staging mode server, not a config-only backup.\nCommon Issues \u0026ldquo;Object is not synced\u0026rdquo; — UPN Not Verified Cause: The user\u0026rsquo;s UPN suffix (e.g., @gntech.me) is not verified in Entra ID.\nFix: Verify the custom domain in Entra admin center → Domain names.\nSync Export Errors Check the Synchronization Service Manager → Operations tab for errors. Common export errors:\nError Cause Fix \u0026ldquo;Duplicate attribute\u0026rdquo; Two objects with the same proxyAddresses Run IdFix \u0026ldquo;Invalid UPN\u0026rdquo; Special characters in UPN Clean AD data \u0026ldquo;Missing source anchor\u0026rdquo; ms-ds-consistencyGUID missing (immutableId) Attribute sync round-trip Users Not Appearing Is the OU being synced? Check OU filtering. Is the user enabled? Disabled users are synced by default unless filtered via attribute. Check if the user passes the connector space filter → run a preview in the Sync Rules Editor. What We Built Component Detail Entra Connect v2 installation Express or Custom on Windows Server 2025 Password hash sync On-prem password hash replicated to cloud OU filtering Selective sync of Users, Workstations, Servers, Groups Writeback ready Password writeback enabled (config in Post #3) Staging mode Verification before production writes Backup config Export JSON for disaster recovery Upgrade awareness Version check, Sep 2026 deadline noted Next in the Series With identity sync flowing, the next post covers the reverse direction — writeback. Password changes in the cloud are written back to on-premises AD, device registrations are written back for conditional access, and Microsoft 365 groups are synchronized to Exchange Online:\nPost 3: Writeback — Cloud to On-Premises (Password, Device, and Group)\nAfter that:\nPost 4: Entra ID Hybrid Join + Device Registration Post 5: Windows Hello for Business — Hybrid Key Trust Post 6: Self-Service Password Reset (SSPR) References Install Microsoft Entra Connect — Roadmap Microsoft Entra Connect: Prerequisites Express installation Custom installation Staging mode Password hash synchronization IdFix tool Verify installation ","permalink":"https://blog.gntech.me/posts/2026-05-14-windows-server-2025-entra-connect-sync/","summary":"Step-by-step Microsoft Entra Connect v2 installation on Windows Server 2025: prerequisites, express install, custom configuration with OU filtering and writeback, verification, and mandatory upgrade before September 30, 2026.","title":"Windows Server 2025: Microsoft Entra Connect — On-Premises to Cloud Identity Sync"},{"content":"Your Proxmox host has 64 GB of RAM. You allocated 48 GB to VMs. So where did the other 16 GB go? If you\u0026rsquo;re running ZFS, the answer is almost certainly the ARC — and it may be using more than you think.\nZFS\u0026rsquo;s Adaptive Replacement Cache is aggressive by design. It will consume up to 50% of your system RAM by default and keep growing until it hits the kernel\u0026rsquo;s zfs_arc_max limit. On a Proxmox host running multiple VMs, this creates a silent competition: the ARC keeps growing, the kernel reclaims it under pressure, VMs get paged, and I/O latency spikes unpredictably.\nThis post covers how to take control of your ZFS ARC allocation, when L2ARC actually helps, and what settings your Proxmox VM datasets should use for maximum throughput.\nStep 1: Budget Your ARC Before Adding VMs ARC sizing on Proxmox is a capacity planning problem, not a tuning exercise. You need to decide upfront how much RAM ZFS can use and stick to it.\nThe default Proxmox behavior sets zfs_arc_max to 50% of system RAM, capped at 16 GiB. This is defensive, not optimal. On a 128 GiB host, ZFS will eat up to 16 GiB by default — fine. But on a 32 GiB host running a dozen containers, 16 GiB of ARC means only 16 GiB for your entire workload.\nHere\u0026rsquo;s the process:\nReserve RAM for your VMs and Proxmox. Add up your VM allocations (not what they use, what they\u0026rsquo;re assigned). Add 2-4 GiB for Proxmox itself. What\u0026rsquo;s left is your ZFS budget. Of that, give most to ARC. Leave ~4-8 GiB for kernel page cache, networking buffers, and system services. Set a hard ARC cap. Never let ZFS fight VMs for memory. Example: 128 GiB node with 96 GiB assigned to VMs:\nVM + Proxmox reservation: ~100 GiB Remaining for system: 28 GiB ARC cap: 24 GiB Kernel/systems buffer: 4 GiB # /etc/modprobe.d/zfs.conf options zfs zfs_arc_max=25769803776 # 24 GiB options zfs zfs_arc_min=8589934592 # 8 GiB minimum The zfs_arc_min prevents the kernel from reclaiming ARC too aggressively during transient memory pressure. Without it, a short-lived VM spike can flush your hot cache, tanking performance until the ARC warms back up.\nApply and reboot:\n1 2 update-initramfs -u -k all reboot Verify after boot:\n1 2 3 4 5 cat /sys/module/zfs/parameters/zfs_arc_max # Should show 25769803776 arc_summary -s arc | grep \u0026#34;ARC size\u0026#34; # Should be at or near the minimum initially Step 2: Validate ARC Sizing by Node Capacity Here are practical ARC caps for common homelab node sizes. These assume the node is running mixed VM workloads with typical I/O patterns (web servers, databases, media, containers).\nNode RAM Suggested ARC Max Guest Budget Notes 32 GiB 8 GiB ~20 GiB Tight — monitor hit ratio 64 GiB 16 GiB ~42 GiB Sweet spot for most homelabs 128 GiB 24-48 GiB ~72-96 GiB Depends on VM density 256 GiB 64-96 GiB ~150-180 GiB L2ARC becomes useful here The key metric is your ARC hit ratio. If you\u0026rsquo;re running at 95%+ hit ratio, your ARC is well-sized for your workload. If it\u0026rsquo;s below 80%, you either need more ARC or you\u0026rsquo;ve got working sets too large for RAM — that\u0026rsquo;s when L2ARC enters the conversation.\n1 2 3 # Check ARC hit ratio arc_summary -s arc | grep -E \u0026#34;hit ratio|miss ratio\u0026#34; # Goal: \u0026gt; 90% for read-heavy workloads Step 3: When and How to Add L2ARC L2ARC extends the ARC onto a fast SSD. Every block evicted from RAM ARC can be written to the L2ARC device instead of being discarded. If it\u0026rsquo;s accessed again, ZFS reads it from the fast SSD instead of the slow pool.\nL2ARC helps when:\nYour ARC hit ratio is already high (85%+) — meaning ARC is doing its job and filling up Your working set doesn\u0026rsquo;t fit in RAM but fits on a moderately sized SSD Your pool is HDD-based and the L2ARC device is NVMe L2ARC does NOT help when:\nYour ARC hit ratio is low — fix ARC sizing first Your pool is already all-SSD — the gap between ARC and pool latency is too small to matter You don\u0026rsquo;t have spare ARC RAM for the L2ARC index (see below) The L2ARC Index Tax This is the most overlooked cost of L2ARC. Every block stored in L2ARC requires an index entry in ARC memory. The overhead depends on your recordsize:\nRecordsize Index overhead per TiB of L2ARC 8 KiB ~8960 MiB 32 KiB ~2240 MiB 128 KiB ~571 MiB If you add a 1 TiB L2ARC with 8 KiB records (Proxmox zvol default), that\u0026rsquo;s nearly 9 GiB of ARC RAM consumed just for metadata. If your ARC cap is 16 GiB, more than half goes to index overhead. That cannibalizes your hot cache for the sake of extending it — a net loss.\nRule of thumb: Don\u0026rsquo;t let L2ARC index overhead exceed 1/3 of your ARC cap. For a 24 GiB ARC, cap L2ARC at roughly 920 GiB (8 KiB records) to 15 TiB (128 KiB records).\nAdding L2ARC to a Pool 1 2 3 4 5 6 7 8 9 # Identify the SSD (use by-id for persistence) ls -la /dev/disk/by-id/ | grep nvme # Add as cache vdev zpool add rpool cache /dev/disk/by-id/nvme-Samsung_SSD_970_EVO_plus_S1234567 # Verify zpool status rpool # Look for \u0026#34;cache\u0026#34; section with the device listed The L2ARC populates automatically over time as ARC evictions occur. There\u0026rsquo;s no warmup phase — it fills as blocks are pushed out of RAM.\nSLOG: Separate from L2ARC SLOG is often confused with L2ARC. A SLOG (Separate Intent Log) is a dedicated device for ZFS synchronous write transactions — not a cache. It accelerates writes for NFS, databases, and VMs with sync enabled. A small, power-protected NVMe or Optane drive (10-20 GiB) is ideal. A large SLOG is wasted space; ZFS only writes the current transaction group (typically 5-10 seconds of writes).\n1 2 3 4 5 6 # Add a SLOG device (mirror recommended for safety) zpool add rpool log mirror \\ /dev/disk/by-id/nvme-Optane_MEMPEK1J016GA_XXXX \\ /dev/disk/by-id/nvme-Optane_MEMPEK1J016GA_YYYY # Never use the same SSD for SLOG and L2ARC Step 4: Tune Recordsize for Your Datasets ZFS recordsize determines the maximum block size for files in a dataset. Match it to your workload for optimal performance.\nWorkload Recommended Recordsize Why VM disks (zvols) 8-16 KiB (default) Matches guest block I/O patterns Databases 16-32 KiB PostgreSQL/MariaDB default page sizes Container rootfs 8-16 KiB Similar to VMs — random small I/O Media storage 1 MiB Large sequential reads benefit big blocks Backups 1 MiB Sequential, compression-friendly ISO/Template 1 MiB Sequential reads, rarely modified 1 2 3 4 5 6 7 8 9 # Check current recordsize zfs get recordsize rpool/data/vm-100-disk-0 # Set for a container dataset zfs set recordsize=16K rpool/data/subvol-101-disk-0 # Set for media storage zfs set recordsize=1M tank/media zfs set compression=lz4 tank/media Recordsize only affects new writes. Existing data keeps its original block size. To apply a new recordsize to existing data, recreate the dataset and copy the data back.\nStep 5: Enable Compression — Always ZFS compression with lz4 is nearly free in CPU cost and provides significant I/O reduction. For compressible data like VM disks, log files, and text-based workloads, it reduces disk reads and writes by 20-40% in practice. The CPU overhead is typically below 1% on modern hardware.\n1 2 3 4 5 6 7 8 # Enable on the root pool zfs set compression=lz4 rpool # Verify zfs get compression rpool # Check compression ratio for each dataset zfs get compressratio rpool/data lz4 is the right default for Proxmox. Don\u0026rsquo;t use gzip — the CPU cost outweighs the space savings for VM workloads. zstd is viable for archival datasets where compression ratio matters more than throughput.\nPractical Config: Full ARC Budget Example Here\u0026rsquo;s a complete configuration for a 128 GiB Proxmox node with mixed HDD pool + NVMe L2ARC:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # /etc/modprobe.d/zfs.conf options zfs zfs_arc_max=25769803776 # 24 GiB options zfs zfs_arc_min=8589934592 # 8 GiB # After reboot, add L2ARC to HDD pool zpool add tank cache \\ /dev/disk/by-id/nvme-Samsung_SSD_980_PRO_2TB_XXXX # Set dataset properties zfs set compression=lz4 tank zfs set recordsize=1M tank/media zfs set recordsize=16K tank/vm zfs set atime=off tank zfs set xattr=sa tank # Disable access time — no benefit for VM storage # xattr=sa stores extended attributes in the inode (faster for ACLs) Verify everything took effect:\n1 2 3 4 5 6 7 8 9 10 11 12 # ARC settings cat /sys/module/zfs/parameters/zfs_arc_max cat /sys/module/zfs/parameters/zfs_arc_min # Pool layout zpool status -v # ARC performance arc_summary -s arc # Dataset properties zfs get compression,recordsize,atime,xattr tank Monitoring: Catch Problems Before VMs Stutter Track these metrics weekly, especially after significant VM changes:\nARC Hit Ratio 1 2 3 4 5 # Quick check arc_summary -s arc | grep -E \u0026#34;hit ratio|miss ratio\u0026#34; # Detailed breakdown arcstat 1 10 If hit ratio drops below 80%, your ARC is undersized or your workload changed. Consider increasing zfs_arc_max or adding L2ARC.\nARC Size vs Pressure 1 2 3 4 5 # Current ARC size echo $(cat /proc/spl/kstat/zfs/arcstats | grep size | head -1 | awk \u0026#39;{print $3}\u0026#39;) # How often ARC is being evicted cat /proc/spl/kstat/zfs/arcstats | grep \u0026#34;evict\u0026#34; High eviction rates with low hit ratio = ARC too small.\nMemory Pressure on VMs 1 2 3 4 5 6 7 8 9 # Check Proxmox host memory free -h # If available memory is near zero and swap is used, # your ARC cap may be too high # Check individual VM ballooning qm list | awk \u0026#39;{print $1}\u0026#39; | tail -n +2 | while read vmid; do echo \u0026#34;VM $vmid: $(qm balloon $vmid 2\u0026gt;/dev/null || echo \u0026#39;no balloon\u0026#39;)\u0026#34; done If VMs are ballooning aggressively (reducing their memory below what you allocated), the host is starving. Reduce zfs_arc_max.\nCommon Mistakes Mistake 1: No ARC Cap at All Without zfs_arc_max, ZFS eats up to 50% of RAM. Your VMs compete for the remaining half. On a 64 GiB host, that\u0026rsquo;s 32 GiB for ZFS and 32 GiB for everything else. Your VMs will balloon, swap, and run slow.\nFix: Set zfs_arc_max to a deliberate value. Even the conservative Proxmox default of 16 GiB is better than nothing.\nMistake 2: L2ARC on a Pool That\u0026rsquo;s Already All-SSD L2ARC adds overhead for minimal gain when the underlying pool is already fast NVMe. The index entries consume ARC RAM. The read latency gap between ARC and NVMe is tiny. L2ARC becomes a net negative.\nFix: Skip L2ARC on all-flash pools. Use the budget for more RAM.\nMistake 3: SLOG and L2ARC on the Same Device SLOG wants low latency, power-protected writes. L2ARC wants high capacity for reads. Putting both on a single consumer NVMe means neither is optimal — and writes to the SLOG compete with L2ARC reads.\nFix: Separate devices, or skip SLOG entirely if you don\u0026rsquo;t have synchronous write workloads (NFS exports or databases with sync commit).\nMistake 4: Using gzip Compression on Active VM Datasets gzip at level 6+ consumes significant CPU. A 16-thread VM workload writing to a gzip-compressed zvol will bottleneck on the compression thread. lz4 achieves similar real-world ratios for VM disk images with 10x less CPU.\nFix: Always use lz4 for active datasets. Use zstd or gzip only for cold archival datasets.\nSummary ZFS ARC tuning on Proxmox comes down to a single principle: decide how much RAM ZFS gets, configure it explicitly, and never let it fight your VMs.\nBudget first — reserve RAM for VMs, give the rest to ARC Set zfs_arc_max and zfs_arc_min — no defaults, no guesses Add L2ARC only if ARC hit ratio is high (85%+) and your pool is HDD Track the L2ARC index tax — it consumes ARC RAM Enable lz4 compression everywhere — free performance Match recordsize to workload — 8-16K for VMs, 1M for media On a 128 GiB node with 24 GiB of dedicated ARC and lz4 compression, expect 90-95% ARC hit ratios for typical mixed workloads. Your VMs get predictable memory, your ZFS pool gets efficient caching, and your weekend doesn\u0026rsquo;t get derailed by mysterious I/O latency spikes.\n","permalink":"https://blog.gntech.me/posts/2026-05-14-proxmox-zfs-arc-tuning/","summary":"Set deterministic ZFS ARC limits, add L2ARC without wasting RAM on metadata, and tune recordsize and compression for your Proxmox homelab VMs.","title":"Proxmox ZFS ARC Tuning — Optimize Cache for VM Performance"},{"content":"Group Policy is the backbone of configuration management in Active Directory. Without it, every server and workstation must be configured manually. Security baselines, update policies, drive maps, printer deployments, and user restrictions all flow through Group Policy Objects (GPOs) linked to sites, domains, and OUs.\nThis guide covers Group Policy on Windows Server 2025 from first principles. The first half is pure GPO management: install GPMC, create and link GPOs, apply security and WMI filtering, manage ADMX templates via the Central Store, backup and restore GPOs, and use Resultant Set of Policy for troubleshooting. The second half applies all of that to one practical scenario: replacing WSUS with Windows Update for Business (WUfB) configured entirely through Group Policy.\nScreenshots and images to add later:\nGPMC console with forest, domains, and GPOs New GPO dialog GPO Editor showing Administrative Templates Security Filtering tab with Authenticated Users removed WMI Filter dialog Central Store folder structure in SYSVOL Group Policy Results wizard Group Policy Modeling wizard Windows Update deferral policy settings Delivery Optimization GPO settings Prerequisites A working Active Directory domain (gntech.me in this guide) Domain Controller running Windows Server 2025 (or 2022) At least one member server or client joined to the domain for testing Domain Administrator credentials Windows Server 2025 ADMX templates (download from Microsoft) GPMC on Server Core vs GUI Windows Server 2025 ships with both Desktop Experience (GUI) and Server Core installation options. The approach differs:\nEnvironment Install GPMC Management Server Core Install-WindowsFeature GPMC (PowerShell) Manage remotely via RSAT from a Windows 11 workstation, or use PowerShell cmdlets locally Desktop Experience Install-WindowsFeature GPMC or Add Roles and Features Wizard Full GPMC snap-in locally Windows 11 (any) Install RSAT optional feature Full GPMC via gpmc.msc, manage any domain For a homelab, running GPMC on a Windows 11 management workstation via RSAT is the most practical approach. Server Core installs GPMC but gives you only the PowerShell module — there is no MMC console on Server Core.\nInstalling GPMC On Server Core via PowerShell 1 2 # Must be run as Administrator on the domain controller Install-WindowsFeature -Name GPMC -IncludeManagementTools The output confirms success or failure. No reboot is required.\nOn Desktop Experience (GUI) via PowerShell 1 Install-WindowsFeature -Name GPMC Or use Server Manager:\nOpen Server Manager → Manage → Add Roles and Features Click Next through the wizard until Features Scroll to Group Policy Management Check the box, click Install On Windows 11 (RSAT) Windows 11 manages GPMC through the RSAT optional features system:\nGUI path:\nOpen Settings → System → Optional Features Click View features Search for RSAT: Group Policy Management Tools Check the box and click Next → Install PowerShell path:\n1 2 3 4 5 # Confirm the feature name Get-WindowsCapability -Online | Where-Object Name -like \u0026#34;Rsat.GroupPolicy*\u0026#34; # Install it Add-WindowsCapability -Online -Name \u0026#34;Rsat.GroupPolicy.Management.Tools~~~~0.0.1.0\u0026#34; Once installed, launch GPMC by running gpmc.msc or searching Group Policy Management in the Start menu.\nGPMC Overview When GPMC opens, the console tree shows:\nGroup Policy Management ├── Forest: gntech.me │ ├── Sites │ │ └── Default-First-Site-Name │ ├── Domains │ │ └── gntech.me │ │ ├── Default Domain Policy │ │ ├── Domain Controllers (OU) │ │ │ └── Default Domain Controllers Policy │ │ ├── [user-created OUs] │ │ └── Group Policy Objects │ │ └── WMI Filters │ │ └── Starter GPOs │ └── Group Policy Modeling │ └── Group Policy Results Key sections:\nGroup Policy Objects — stores all GPOs regardless of link location WMI Filters — reusable WMI queries to scope GPOs by OS version, RAM, disk, etc. Starter GPOs — templates for common policy configurations Group Policy Modeling — RSoP simulation (\u0026ldquo;what-if\u0026rdquo;) Group Policy Results — actual RSoP of a real client (\u0026ldquo;what-is\u0026rdquo;) Default GPOs Every domain has two mandatory GPOs after AD DS installation:\nGPO Linked to Purpose Default Domain Policy Domain root Password policy, account lockout, Kerberos policy Default Domain Controllers Policy Domain Controllers OU User rights assignments, audit policy, security options for DCs Rule: Never delete these. Do not modify the Domain Controllers Policy unless you know exactly what you are doing. Add custom settings into new GPOs, not into the defaults.\nADMX Central Store Administrative Templates (.admx / .adml) define the registry-based policy settings visible in the GPO Editor. By default, each server or workstation running GPMC uses its local C:\\Windows\\PolicyDefinitions folder. In a mixed-OS environment (Server 2025 managing Windows 11 25H2 settings), this means missing templates on older machines.\nThe Central Store solves this by hosting ADMX files in SYSVOL, making them available to all domain administrators regardless of their management workstation OS.\nCreating the Central Store 1 2 3 4 5 6 # Create the Central Store folder on a domain controller $sysvol = \u0026#34;\\\\gntech.me\\SYSVOL\\gntech.me\\Policies\\PolicyDefinitions\u0026#34; New-Item -Path $sysvol -ItemType Directory -Force # Create language subfolder (en-US in this case) New-Item -Path \u0026#34;$sysvol\\en-US\u0026#34; -ItemType Directory -Force Populating ADMX Files Download the latest ADMX templates:\nAdministrative Templates (.admx) for Windows 11 2025 Update (25H2) Administrative Templates (.admx) for Windows Server 2025 (October 2025 release) Extract the downloaded MSI/MSP and copy the files:\n1 2 3 4 5 6 7 # Example: after extracting to C:\\ADMX\\Win11_25H2 Copy-Item -Path \u0026#34;C:\\ADMX\\Win11_25H2\\*.admx\u0026#34; -Destination $sysvol -Recurse Copy-Item -Path \u0026#34;C:\\ADMX\\Win11_25H2\\en-US\\*.adml\u0026#34; -Destination \u0026#34;$sysvol\\en-US\u0026#34; -Recurse # Repeat for Server 2025 ADMX Copy-Item -Path \u0026#34;C:\\ADMX\\Svr2025\\*.admx\u0026#34; -Destination $sysvol -Recurse Copy-Item -Path \u0026#34;C:\\ADMX\\Svr2025\\en-US\\*.adml\u0026#34; -Destination \u0026#34;$sysvol\\en-US\u0026#34; -Recurse GPMC automatically detects the Central Store and uses it in preference to local PolicyDefinitions. Newer ADMX files override older ones with the same name.\nBest practice: Populate the Central Store with the latest Windows client ADMX (e.g., Windows 11 25H2) plus the Server 2025 ADMX. This covers both client and server policy settings from one location.\nCreating a GPO Via GPMC (GUI) Right-click the target OU (or domain) → Create a GPO in this domain and Link it here Enter a descriptive name like Windows Update - Ring 1 (Pilot) Click OK Right-click the new GPO → Edit to open the Group Policy Management Editor Via PowerShell 1 2 3 4 5 # Create an unlinked GPO New-GPO -Name \u0026#34;Windows Update - Ring 1 (Pilot)\u0026#34; -Comment \u0026#34;Pilot ring - 0 day deferral\u0026#34; # Create and link in one step New-GPO -Name \u0026#34;Windows Update - Ring 2 (Fast)\u0026#34; | New-GPLink -Target \u0026#34;OU=Workstations,OU=Computers,DC=gntech,DC=me\u0026#34; Linking an Existing GPO 1 2 3 4 5 6 7 8 # Link to an OU New-GPLink -Name \u0026#34;Windows Update - Ring 1 (Pilot)\u0026#34; -Target \u0026#34;OU=Pilot,OU=Workstations,OU=Computers,DC=gntech,DC=me\u0026#34; # Link to domain New-GPLink -Name \u0026#34;Windows Update - Base Policy\u0026#34; -Target \u0026#34;DC=gntech,DC=me\u0026#34; # Set link order and enable/disable Set-GPLink -Name \u0026#34;Windows Update - Ring 1 (Pilot)\u0026#34; -Target \u0026#34;OU=Pilot,OU=Workstations,OU=Computers,DC=gntech,DC=me\u0026#34; -Order 1 -Enabled Yes Link order matters — lower numbers have higher precedence. The last applied GPO wins in case of conflict.\nGPO Inheritance and Processing Order Policy applies in this order:\nLocal (stored on the machine) Site — linked to the AD site the computer belongs to Domain — linked at the domain root OU — linked to the OU and its parent OUs (parent first, child last) When multiple GPOs apply at the same level, the one with the lowest link order wins. You can override this with:\nEnforced (No Override) — forces the GPO to apply regardless of lower-level settings. Use sparingly. Block Inheritance — stops higher-level GPOs from applying to an OU. Use rarely; it makes troubleshooting harder. 1 2 3 4 5 # Enforce a GPO link (No Override) Set-GPLink -Name \u0026#34;Security Baseline\u0026#34; -Target \u0026#34;DC=gntech,DC=me\u0026#34; -Enforced Yes # Block inheritance on an OU Set-GPInheritance -Target \u0026#34;OU=RestrictedWorkstations,OU=Computers,DC=gntech,DC=me\u0026#34; -IsBlocked Yes Security Filtering By default, a new GPO applies to Authenticated Users — everyone in the domain. In production, you almost never want that.\nApplying GPOs to Specific Security Groups In GPMC, select the GPO In the Scope tab, under Security Filtering, select Authenticated Users and click Remove Click Add, type the security group name (e.g., WS-Pilot-Ring), and click OK The GPO now applies only to members of that group Important: The computer account itself needs Read and Apply permissions to process a Computer Configuration GPO. When you add a security group of computers, computer members inherit those permissions automatically.\n1 2 3 4 5 6 # Add security filtering via PowerShell $gpo = Get-GPO -Name \u0026#34;Windows Update - Ring 1 (Pilot)\u0026#34; Set-GPPermissions -Name \u0026#34;Windows Update - Ring 1 (Pilot)\u0026#34; -TargetName \u0026#34;WS-Pilot-Ring\u0026#34; -TargetType Group -PermissionLevel GpoApply # Remove Authenticated Users Set-GPPermissions -Name \u0026#34;Windows Update - Ring 1 (Pilot)\u0026#34; -TargetName \u0026#34;Authenticated Users\u0026#34; -TargetType Group -PermissionLevel None Delegation (Non-Admin GPO Editing) To let a helpdesk team edit a specific GPO without giving them Domain Admin:\n1 Set-GPPermissions -Name \u0026#34;Windows Update - Ring 1 (Pilot)\u0026#34; -TargetName \u0026#34;GPO-Editors\u0026#34; -TargetType Group -PermissionLevel GpoEditDeleteModifySecurity Available permission levels: GpoRead, GpoApply, GpoEdit, GpoEditDeleteModifySecurity, GpoCustom.\nWMI Filtering WMI filters scope GPOs based on system attributes — OS version, RAM, disk space, installed software. Only computers matching the WMI query process the GPO.\nCreating a WMI Filter Example — apply only to Windows 11 workstations:\nSELECT * FROM Win32_OperatingSystem WHERE Version LIKE \u0026#34;10.0.2%\u0026#34; AND ProductType = \u0026#34;1\u0026#34; Version LIKE \u0026quot;10.0.2%\u0026quot; — Windows 11 family (5 = Server 2022, 10.0.20348; Server 2025 is 10.0.26100) ProductType = \u0026quot;1\u0026quot; — workstation only (not server) Example — apply only to Windows Server 2025:\nSELECT * FROM Win32_OperatingSystem WHERE Version = \u0026#34;10.0.26100\u0026#34; AND ProductType \u0026lt;\u0026gt; \u0026#34;1\u0026#34; To create via GPMC:\nRight-click WMI Filters → New Name the filter, add the query, click Save Select a GPO → Scope tab → in WMI Filtering, select the filter To create via PowerShell:\n1 2 $query = \u0026#34;SELECT * FROM Win32_OperatingSystem WHERE Version LIKE \u0026#39;10.0.2%\u0026#39; AND ProductType = \u0026#39;1\u0026#39;\u0026#34; New-GPWmiFilter -Name \u0026#34;Windows 11 Workstations\u0026#34; -Description \u0026#34;Applies only to Windows 11 client devices\u0026#34; -Content $query WMI filters can hurt performance if the query is slow or if many GPOs are filtered. Keep queries simple and avoid uncapped WHERE clauses.\nGPO Backup and Restore Backups are critical. A misconfigured GPO can break an entire organization.\nBackup via PowerShell 1 2 3 4 5 6 # Backup a single GPO to a timestamped folder $backupPath = \u0026#34;C:\\GPOBackups\\$(Get-Date -Format \u0026#39;yyyy-MM-dd\u0026#39;)\u0026#34; Backup-GPO -Name \u0026#34;Windows Update - Ring 1 (Pilot)\u0026#34; -Path $backupPath -Comment \u0026#34;Pre-WUfB configuration\u0026#34; # Backup all GPOs in the domain Backup-GPO -All -Path $backupPath -Comment \u0026#34;Domain-wide GPO backup $(Get-Date -Format \u0026#39;yyyy-MM-dd\u0026#39;)\u0026#34; Backup via GPMC GUI Right-click Group Policy Objects → Back Up All Choose a path and click Back Up Restore via PowerShell 1 2 3 4 5 # Find backups Get-GPOBackup -Path $backupPath -All # Restore a specific GPO from its most recent backup Restore-GPO -Name \u0026#34;Windows Update - Ring 1 (Pilot)\u0026#34; -Path $backupPath Export/Import (Cross-Domain) GPMC supports migration tables for cross-domain GPO copy, remapping security principals and UNC paths:\n1 2 3 4 5 6 7 8 # Export $gpo = Get-GPO -Name \u0026#34;Security Baseline\u0026#34; $gpo.GenerateReport -ReportType Xml | Out-File C:\\GPExport\\SecBaseline.xml # Create migration table New-GPMigrationTable -Path C:\\GPExport\\migration-table.migtable -Mapping @{ \u0026#34;gntech\\Domain Admins\u0026#34; = \u0026#34;targetdom\\Domain Admins\u0026#34; } HTML Reporting and RSoP GPMC can generate HTML reports of GPO settings without opening the editor.\nGPO Settings Report 1 Get-GPOReport -Name \u0026#34;Windows Update - Ring 1 (Pilot)\u0026#34; -ReportType Html -Path \u0026#34;C:\\Reports\\WURing1.html\u0026#34; Resultant Set of Policy (RSoP) Group Policy Results — what is actually applied (reactive):\nIn GPMC, right-click Group Policy Results → Group Policy Results Wizard → specify a remote computer and user. The HTML output shows all GPOs applied, denied GPOs with reasons, and the final setting values.\n1 2 # PowerShell equivalent Get-GPResultantSetOfPolicy -Computer \u0026#34;WS-Client01\u0026#34; -User \u0026#34;gntech\\administrator\u0026#34; -ReportType Html -Path \u0026#34;C:\\Reports\\RSoP-WS-Client01.html\u0026#34; Group Policy Modeling — what would apply (simulation):\nIn GPMC, right-click Group Policy Modeling and configure OU, security groups, and WMI filters. This is invaluable before deploying a new GPO.\nWindows Update for Business via GPO With WSUS deprecated by Microsoft in September 2024, the recommended replacement for on-premises update management is Windows Update for Business (WUfB) configured through Group Policy. WUfB directs clients to Microsoft\u0026rsquo;s Windows Update service, not to a local WSUS server.\nNote: WUfB was formerly called \u0026ldquo;Windows Update for Business\u0026rdquo; and is now referred to in documentation as \u0026ldquo;Windows Update client policies.\u0026rdquo; The GPO category is still labeled Windows Update for Business.\nDeployment Ring Strategy We create three security groups in AD and corresponding GPOs:\nRing Group Deferral Purpose Pilot WS-WU-Pilot 0 days quality, 0 days feature IT/admin workstations, test VMs Fast WS-WU-Fast 5 days quality, 30 days feature Early adopters, dev machines Slow WS-WU-Slow 10 days quality, 120 days feature Production, critical systems Step 1 — Create Security Groups In Active Directory Users and Computers or via PowerShell:\n1 2 3 4 $groups = @(\u0026#34;WS-WU-Pilot\u0026#34;, \u0026#34;WS-WU-Fast\u0026#34;, \u0026#34;WS-WU-Slow\u0026#34;) foreach ($group in $groups) { New-ADGroup -Name $group -GroupScope Global -GroupCategory Security -Path \u0026#34;OU=Groups,DC=gntech,DC=me\u0026#34; } Add computer accounts to the appropriate group.\nStep 2 — Create GPOs for Each Ring From a management workstation with GPMC:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # Create base Windows Update policy GPO New-GPO -Name \u0026#34;Windows Update - Base Policy\u0026#34; -Comment \u0026#34;Base WUfB settings shared across all rings\u0026#34; New-GPLink -Name \u0026#34;Windows Update - Base Policy\u0026#34; -Target \u0026#34;DC=gntech,DC=me\u0026#34; # Create ring-specific GPOs New-GPO -Name \u0026#34;Windows Update - Pilot Ring\u0026#34; -Comment \u0026#34;0 day quality deferral, no feature deferral\u0026#34; New-GPO -Name \u0026#34;Windows Update - Fast Ring\u0026#34; -Comment \u0026#34;5 day quality deferral, 30 day feature deferral\u0026#34; New-GPO -Name \u0026#34;Windows Update - Slow Ring\u0026#34; -Comment \u0026#34;10 day quality deferral, 120 day feature deferral\u0026#34; # Link ring GPOs to the domain root (filtered by security groups) New-GPLink -Name \u0026#34;Windows Update - Pilot Ring\u0026#34; -Target \u0026#34;DC=gntech,DC=me\u0026#34; New-GPLink -Name \u0026#34;Windows Update - Fast Ring\u0026#34; -Target \u0026#34;DC=gntech,DC=me\u0026#34; New-GPLink -Name \u0026#34;Windows Update - Slow Ring\u0026#34; -Target \u0026#34;DC=gntech,DC=me\u0026#34; Step 3 — Configure the Base Policy Edit Windows Update - Base Policy in GPMC and navigate to:\nComputer Configuration └─ Policies └─ Administrative Templates └─ Windows Components └─ Windows Update Configure these settings:\nSetting Value Path Configure Automatic Updates Enabled → 2 - Notify for download and auto install Windows Update Configure Automatic Updates → Install updates for other Microsoft products Enabled Windows Update Do not include drivers with Windows Updates Disabled (default — let drivers update) Windows Update Turn on Software Notifications Enabled Windows Update Specify intranet Microsoft update service location Not configured (we use WUfB, not WSUS) Windows Update Step 4 — Configure Deferral Policies Under the same path, go deeper to:\nComputer Configuration └─ Policies └─ Administrative Templates └─ Windows Components └─ Windows Update └─ Windows Update for Business Edit each ring GPO:\nPilot Ring:\nSetting Value Select when Preview Builds and Feature Updates are received Enabled → Deferral (0 days), Pause (Disabled) Select when Quality Updates are received Enabled → Deferral (0 days), Pause (Disabled) Fast Ring:\nSetting Value Select when Preview Builds and Feature Updates are received Enabled → Deferral (30 days), Pause (Disabled) Select when Quality Updates are received Enabled → Deferral (5 days), Pause (Disabled) Slow Ring:\nSetting Value Select when Preview Builds and Feature Updates are received Enabled → Deferral (120 days), Pause (Disabled) Select when Quality Updates are received Enabled → Deferral (10 days), Pause (Disabled) The maximum deferral periods are:\nFeature updates: 365 days Quality updates: 30 days Step 5 — Apply Security Filtering For each ring GPO, remove Authenticated Users and add the corresponding security group:\n1 2 3 4 5 6 7 8 9 10 # Remove Authenticated Users from all ring GPOs $ringGpos = @(\u0026#34;Windows Update - Pilot Ring\u0026#34;, \u0026#34;Windows Update - Fast Ring\u0026#34;, \u0026#34;Windows Update - Slow Ring\u0026#34;) foreach ($gpoName in $ringGpos) { Set-GPPermissions -Name $gpoName -TargetName \u0026#34;Authenticated Users\u0026#34; -TargetType Group -PermissionLevel None } # Add security groups Set-GPPermissions -Name \u0026#34;Windows Update - Pilot Ring\u0026#34; -TargetName \u0026#34;WS-WU-Pilot\u0026#34; -TargetType Group -PermissionLevel GpoApply Set-GPPermissions -Name \u0026#34;Windows Update - Fast Ring\u0026#34; -TargetName \u0026#34;WS-WU-Fast\u0026#34; -TargetType Group -PermissionLevel GpoApply Set-GPPermissions -Name \u0026#34;Windows Update - Slow Ring\u0026#34; -TargetName \u0026#34;WS-WU-Slow\u0026#34; -TargetType Group -PermissionLevel GpoApply Step 6 — Configure Delivery Optimization Delivery Optimization (DO) is Windows\u0026rsquo; peer-to-peer update distribution system. In a homelab with limited bandwidth, DO can help, but for a small number of machines it is often unnecessary.\nNavigate to:\nComputer Configuration └─ Policies └─ Administrative Templates └─ Windows Components └─ Delivery Optimization Recommended settings for a homelab:\nSetting Value Download mode 1 (HTTP + Peering on same NAT) — LAN-only peering; or 0 (HTTP only) to disable P2P entirely Absolute max cache size Enabled → 4096 (4 GB cache) Maximum upload bandwidth Not configured 1 2 # Verify Delivery Optimization settings on a client Get-DeliveryOptimizationStatus Step 7 — Verify Application From a domain-joined client in the Pilot ring, run:\n1 2 3 4 5 6 7 8 9 10 11 12 # Force Group Policy refresh gpupdate /force # View applied GPOs gpresult /r # View WUfB configuration $updateSettings = Get-ItemProperty -Path \u0026#34;HKLM:\\SOFTWARE\\Microsoft\\WindowsUpdate\\UX\\Settings\u0026#34; $updateSettings | Format-List # Check deferral configuration reg query \u0026#34;HKLM\\SOFTWARE\\Microsoft\\WindowsUpdate\\UpdatePolicy\\PolicyStates\u0026#34; /s Expected output from gpresult /r should show the base and ring GPOs applied. If they do not appear, check:\nIs the computer in the correct security group? Does the GPO link apply to the computer\u0026rsquo;s OU? Is Authenticated Users still in security filtering? (It should be removed.) Run rsop.msc on the client for a GUI RSoP view. Pausing Updates If a bad update is deployed, pause it for up to 35 days via one GPO:\nComputer Configuration └─ Policies └─ Administrative Templates └─ Windows Components └─ Windows Update └─ Windows Update for Business └─ Select when Quality Updates are received → Check \u0026#34;Pause quality updates\u0026#34; 1 2 # Alternatively, pause via PowerShell on the client Set-ItemProperty -Path \u0026#34;HKLM:\\SOFTWARE\\Microsoft\\WindowsUpdate\\UX\\Settings\u0026#34; -Name \u0026#34;PauseQualityUpdatesStartTime\u0026#34; -Value (Get-Date).ToString(\u0026#34;yyyy-MM-ddTHH:mm:ssZ\u0026#34;) Common GPO Troubleshooting GPO Not Applying Replication delay: Check SYSVOL and AD replication across DCs 1 2 repadmin /replsummary dcdiag /test:replications Security filtering mismatch: Ensure the computer or user account is a member of the security group and that the group has Apply Group Policy permission WMI filter failure: Run the WMI query directly on the client 1 Get-WmiObject -Query \u0026#34;SELECT * FROM Win32_OperatingSystem WHERE Version LIKE \u0026#39;10.0.2%\u0026#39;\u0026#34; Inheritance blocked: Check if Block Inheritance is set on the OU Slow Link Detection: By default, GPO does not apply over slow links for certain settings. Override with: Computer Configuration → Administrative Templates → System → Group Policy → Group Policy slow link detection → Disabled GPO Conflict If two GPOs set the same policy, the one with the highest precedence (lowest link order) wins. Check the precedence chain using gpresult /h C:\\report.html and review the Winning GPO and Filtered GPOs sections.\nWhat We Built Component Purpose GPMC installed (Server Core via PowerShell, DCs via GUI, Windows 11 via RSAT) Management platform ADMX Central Store Single source of truth for policy templates 3 deployment ring GPOs Phased Windows Update rollout Security filtering Ring membership via AD security groups WMI filtering (optional) OS-specific GPO targeting GPO modeling Pre-deployment validation GPO backup Disaster recovery Delivery Optimization Peer-to-peer update distribution Next in the Series With GPO management in place, the next post covers hybrid identity sync — installing Microsoft Entra Connect to synchronize on-premises users and groups to the cloud, setting the stage for Entra ID hybrid join, Windows Hello for Business, and cloud file access.\nPost 2: Microsoft Entra Connect — On-Premises to Cloud Identity Sync\nReferences Group Policy overview for Windows Server Group Policy Management Console Configure Windows Update client policies via Group Policy Create and Manage Central Store Administrative Templates (.admx) for Windows Server 2025 Administrative Templates (.admx) for Windows 11 25H2 Delivery Optimization reference Group Policy Settings Reference Spreadsheets ","permalink":"https://blog.gntech.me/posts/2026-05-14-windows-server-2025-group-policy-management/","summary":"Step-by-step Group Policy management on Windows Server 2025: GPMC install, GPO creation/linking, security and WMI filtering, ADMX templates, backup/restore, and Windows Update for Business deployment rings.","title":"Windows Server 2025 Group Policy Management — GPMC, WUfB, and Central Store"},{"content":"Access to a rootful Docker daemon can give control of the host. A socket proxy filters requests before they reach the daemon, but remains a privileged component. Its usefulness depends on the requests allowed and on which clients can reach it.\nVerification status Area Evidence and limits Source review Tecnativa\u0026rsquo;s documented environment variables reviewed on October 31, 2026. Runtime No proxy container or authorization test was executed in this audit. Required test Confirm an allowed request works and a denied request returns a rejection for the chosen image digest. Configure a separate read-oriented endpoint The example is for a discovery client such as Traefik, not a container-management tool. Select a reviewed tecnativa/docker-socket-proxy image tag or digest first. Use its documented variables such as CONTAINERS and POST; the earlier ALLOW_CREATE/ALLOW_DELETE examples did not describe Tecnativa\u0026rsquo;s configuration interface.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 read -r -p \u0026#39;Reviewed socket-proxy image tag or digest: \u0026#39; SOCKET_PROXY_IMAGE export SOCKET_PROXY_IMAGE cat \u0026gt; compose.yaml \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; services: socket-proxy: image: ${SOCKET_PROXY_IMAGE:?Select a reviewed socket-proxy image} restart: unless-stopped environment: CONTAINERS: \u0026#34;1\u0026#34; EVENTS: \u0026#34;1\u0026#34; INFO: \u0026#34;1\u0026#34; NETWORKS: \u0026#34;1\u0026#34; VERSION: \u0026#34;1\u0026#34; PING: \u0026#34;1\u0026#34; POST: \u0026#34;0\u0026#34; volumes: - /var/run/docker.sock:/var/run/docker.sock:ro networks: - discovery networks: discovery: internal: true EOF docker compose config --quiet docker compose up -d There is deliberately no host port mapping. Attach only the intended client to this network and point its Docker endpoint at tcp://socket-proxy:2375. Give that client a separate network for its normal traffic where needed. A client also holding the raw Docker socket bypasses the proxy.\nCONTAINERS=1 enables a category of endpoints, not only a hand-picked container. Read access can disclose metadata and environment values. POST=0 is a proxy policy control; it is not implemented by the socket\u0026rsquo;s :ro mount. A read-only bind mount does not make Docker API calls read-only. The proxy can still communicate with the mounted socket, and compromise of the proxy itself remains significant.\nSee the upstream permission list and security warnings. Test the behavior of the exact release rather than assuming environment names carry across different proxy projects.\nVerify before attaching real clients In a disposable lab, use a temporary HTTP client on the private network. Check a required GET endpoint, then a benign request to a disabled category such as /volumes; inspect the response code and proxy logs. Do not use create, delete, exec, or privileged-container operations as a production denial test. Confirm the policy independently of whether the application appears healthy.\nPortainer and automatic updaters need write operations that this example intentionally denies. Do not enable broad write access on a shared proxy and then claim every connected client has separate permissions. Use separate proxy instances/networks and review the write API\u0026rsquo;s effective host access. Network reachability controls who can call an endpoint; this proxy configuration does not provide per-user authentication or TLS for an exposed listener.\nContinue reading Container security hardening — inspect capabilities, mounts, and the daemon trust boundary. Compose patterns — keep deployment boundaries explicit. ","permalink":"https://blog.gntech.me/posts/2026-05-14-docker-socket-proxy/","summary":"Configure documented socket-proxy endpoint controls, keep the proxy private, and test permitted and denied requests.","title":"Docker Socket Proxy — Restrict API Access and Verify the Policy"},{"content":"DHCP looks simple until it is down. If your only DHCP server fails, clients can keep their existing leases for a while, but new devices do not get addresses, renewals eventually fail, and the network slowly gets weird.\nWindows Server DHCP failover solves that by pairing two DHCP servers. They share scope configuration and lease state, then serve clients in load-balance or hot-standby mode. This guide builds DHCP failover on Windows Server 2025 with GUI and Server Core/PowerShell paths, including replication and verification.\nGUI vs Server Core Path This guide covers both styles:\nDesktop Experience / GUI: Server Manager, DHCP console, new scope wizard, DHCP authorization, and failover wizard. Server Core / PowerShell: Install-WindowsFeature, Add-DhcpServerInDC, Add-DhcpServerv4Scope, Set-DhcpServerv4OptionValue, and Add-DhcpServerv4Failover. Use GUI for screenshots and learning the flow. Use PowerShell for repeatable server builds and clean documentation.\nTarget Design Item Value DHCP server 1 DHCP01.gntech.me / 10.0.20.20 DHCP server 2 DHCP02.gntech.me / 10.0.20.21 Domain gntech.me Scope 10.0.20.0/24 Lease range 10.0.20.100 - 10.0.20.200 Exclusions 10.0.20.1 - 10.0.20.99, 10.0.20.201 - 10.0.20.254 Gateway option 10.0.20.1 DNS option 10.0.20.10, 10.0.20.11 DNS suffix gntech.me Failover mode Load balance 50/50 for LAN, hot standby for remote site For a homelab LAN, load-balance mode is usually fine. For a branch or remote site, hot-standby mode can be cleaner.\nDHCP Failover Modes Windows DHCP failover supports two common designs.\nLoad Balance Both DHCP servers answer clients and split the scope workload.\n1 2 Use when: both DHCP servers are in the same site/VLAN and equally reachable. Typical split: 50/50 Hot Standby One server actively handles the scope; the partner waits and takes over if the active server is unavailable.\n1 2 Use when: one server is primary for a site and the other is backup. Typical split: 95/5 or standby role For this guide, the PowerShell example uses load-balance mode.\nPre-Checks On both DHCP servers:\n1 2 3 4 hostname Get-NetIPConfiguration Resolve-DnsName gntech.me Test-ComputerSecureChannel -Verbose Both servers should be domain joined, statically addressed, patched, and able to resolve the domain.\nCheck domain DNS:\n1 2 Resolve-DnsName DC01.gntech.me Resolve-DnsName DC02.gntech.me Install the DHCP Server Role GUI: Server Manager On each DHCP server:\nOpen Server Manager. Go to Manage → Add Roles and Features. Select DHCP Server. Accept management tools. Install the role. Click the post-install notification to complete DHCP configuration. Server Core / PowerShell Run on both DHCP01 and DHCP02:\n1 Install-WindowsFeature DHCP -IncludeManagementTools Verify:\n1 2 Get-WindowsFeature DHCP Get-Service DHCPServer Expected service:\n1 Running Authorize DHCP in Active Directory Domain-joined Windows DHCP servers must be authorized in AD before they serve leases.\nGUI: DHCP Console Open Server Manager → Tools → DHCP. Right-click the server. Select Authorize. Refresh until the server icon shows authorized. Server Core / PowerShell Run from a domain admin session:\n1 2 3 Add-DhcpServerInDC -DnsName \u0026#34;DHCP01.gntech.me\u0026#34; -IPAddress 10.0.20.20 Add-DhcpServerInDC -DnsName \u0026#34;DHCP02.gntech.me\u0026#34; -IPAddress 10.0.20.21 Verify:\n1 Get-DhcpServerInDC Create the Scope on DHCP01 Create the scope on one server first. The failover relationship will replicate it to the partner.\nGUI: DHCP Console Expand DHCP01. Right-click IPv4 → New Scope. Name it LAB-10.0.20.0. Set range 10.0.20.100 to 10.0.20.200. Set subnet mask 255.255.255.0. Add exclusions if needed. Set router 10.0.20.1. Set DNS servers 10.0.20.10 and 10.0.20.11. Set DNS domain gntech.me. Activate the scope. Server Core / PowerShell Run on DHCP01:\n1 Add-DhcpServerv4Scope -ComputerName \u0026#34;DHCP01.gntech.me\u0026#34; -Name \u0026#34;LAB-10.0.20.0\u0026#34; -StartRange 10.0.20.100 -EndRange 10.0.20.200 -SubnetMask 255.255.255.0 -State Active Set options:\n1 Set-DhcpServerv4OptionValue -ComputerName \u0026#34;DHCP01.gntech.me\u0026#34; -ScopeId 10.0.20.0 -Router 10.0.20.1 -DnsServer 10.0.20.10,10.0.20.11 -DnsDomain \u0026#34;gntech.me\u0026#34; Verify:\n1 2 Get-DhcpServerv4Scope -ComputerName \u0026#34;DHCP01.gntech.me\u0026#34; Get-DhcpServerv4OptionValue -ComputerName \u0026#34;DHCP01.gntech.me\u0026#34; -ScopeId 10.0.20.0 Configure DHCP Failover GUI: DHCP Failover Wizard In DHCP console, expand DHCP01 → IPv4. Right-click the scope → Configure Failover. Select the scope. Add partner server DHCP02.gntech.me. Name the relationship DHCP01-DHCP02-LAB. Choose Load balance. Set load balance percentage to 50%. Set shared secret. Finish the wizard. Server Core / PowerShell Use a strong shared secret. Store it securely.\n1 2 3 4 5 6 $Secret = Read-Host \u0026#34;DHCP failover shared secret\u0026#34; -AsSecureString $PlainSecret = [Runtime.InteropServices.Marshal]::PtrToStringAuto( [Runtime.InteropServices.Marshal]::SecureStringToBSTR($Secret) ) Add-DhcpServerv4Failover -ComputerName \u0026#34;DHCP01.gntech.me\u0026#34; -Name \u0026#34;DHCP01-DHCP02-LAB\u0026#34; -PartnerServer \u0026#34;DHCP02.gntech.me\u0026#34; -ScopeId 10.0.20.0 -SharedSecret $PlainSecret -LoadBalancePercent 50 -AutoStateTransition $true -StateSwitchInterval 01:00:00 Clear the plaintext variable when done:\n1 Remove-Variable PlainSecret Verify the relationship:\n1 2 Get-DhcpServerv4Failover -ComputerName \u0026#34;DHCP01.gntech.me\u0026#34; Get-DhcpServerv4Failover -ComputerName \u0026#34;DHCP02.gntech.me\u0026#34; Replicate Scope Configuration Failover replicates leases automatically, but when you change scope options, reservations, policies, or exclusions, force replication.\nGUI: Replicate Scope Open DHCP console. Right-click the scope or IPv4 node. Select Replicate Scope or Replicate Failover Scopes. Confirm replication to the partner. Server Core / PowerShell Replicate a specific scope:\n1 Invoke-DhcpServerv4FailoverReplication -ComputerName \u0026#34;DHCP01.gntech.me\u0026#34; -ScopeId 10.0.20.0 -Force Or replicate by relationship name:\n1 Invoke-DhcpServerv4FailoverReplication -ComputerName \u0026#34;DHCP01.gntech.me\u0026#34; -Name \u0026#34;DHCP01-DHCP02-LAB\u0026#34; -Force Verify the scope exists on both servers:\n1 2 Get-DhcpServerv4Scope -ComputerName \u0026#34;DHCP01.gntech.me\u0026#34; Get-DhcpServerv4Scope -ComputerName \u0026#34;DHCP02.gntech.me\u0026#34; Router / VLAN DHCP Relay If clients are not on the same L2 segment as the DHCP servers, configure DHCP relay/IP helper on the router or L3 switch.\nFor a VLAN 20 client network, relay to both DHCP servers:\n1 2 DHCP relay target 1: 10.0.20.20 DHCP relay target 2: 10.0.20.21 On MikroTik, this is usually DHCP Relay on the VLAN interface. On Cisco, it is usually ip helper-address under the SVI. The important part is that both DHCP servers receive requests.\nVerification Checklist 1. DHCP Server Authorization 1 Get-DhcpServerInDC Expected: both DHCP01 and DHCP02 listed with correct IPs.\n2. Scope Exists on Both Servers 1 2 Get-DhcpServerv4Scope -ComputerName \u0026#34;DHCP01.gntech.me\u0026#34; Get-DhcpServerv4Scope -ComputerName \u0026#34;DHCP02.gntech.me\u0026#34; 3. Options Are Correct 1 2 3 Get-DhcpServerv4OptionValue -ComputerName \u0026#34;DHCP01.gntech.me\u0026#34; -ScopeId 10.0.20.0 Get-DhcpServerv4OptionValue -ComputerName \u0026#34;DHCP02.gntech.me\u0026#34; -ScopeId 10.0.20.0 Expected:\n1 2 3 003 Router: 10.0.20.1 006 DNS Servers: 10.0.20.10, 10.0.20.11 015 DNS Domain Name: gntech.me 4. Failover Relationship Is Healthy 1 2 Get-DhcpServerv4Failover -ComputerName \u0026#34;DHCP01.gntech.me\u0026#34; -Name \u0026#34;DHCP01-DHCP02-LAB\u0026#34; | Format-List * Look for normal state on both partners.\n5. Client Lease Test On a Windows client:\n1 2 3 ipconfig /release ipconfig /renew ipconfig /all Verify:\nIPv4 address is inside 10.0.20.100-200 gateway is 10.0.20.1 DNS servers are 10.0.20.10 and 10.0.20.11 DNS suffix is gntech.me 6. Lease Visibility 1 2 3 Get-DhcpServerv4Lease -ComputerName \u0026#34;DHCP01.gntech.me\u0026#34; -ScopeId 10.0.20.0 Get-DhcpServerv4Lease -ComputerName \u0026#34;DHCP02.gntech.me\u0026#34; -ScopeId 10.0.20.0 Leases should replicate between partners.\nFailover Test Do not hard power off servers during early testing. First do a controlled service test.\nOn DHCP01:\n1 Stop-Service DHCPServer On a client:\n1 2 3 ipconfig /release ipconfig /renew ipconfig /all Expected: the client still receives a valid lease from DHCP02.\nRestart DHCP01:\n1 Start-Service DHCPServer Check failover state:\n1 2 Get-DhcpServerv4Failover -ComputerName \u0026#34;DHCP01.gntech.me\u0026#34; Get-DhcpServerv4Failover -ComputerName \u0026#34;DHCP02.gntech.me\u0026#34; Common Problems DHCP Server Not Leasing Addresses Check authorization:\n1 2 Get-DhcpServerInDC Get-Service DHCPServer If unauthorized, authorize it in AD.\nClient Gets APIPA Address APIPA means the client did not receive DHCP.\nCheck the client and the DHCP server-side counters:\n1 2 3 4 5 ipconfig /all Get-DhcpServerv4Statistics -ComputerName \u0026#34;DHCP01.gntech.me\u0026#34; Get-DhcpServerv4Statistics -ComputerName \u0026#34;DHCP02.gntech.me\u0026#34; Get-DhcpServerv4Lease -ComputerName \u0026#34;DHCP01.gntech.me\u0026#34; -ScopeId 10.0.20.0 Get-DhcpServerv4Lease -ComputerName \u0026#34;DHCP02.gntech.me\u0026#34; -ScopeId 10.0.20.0 Do not use Test-NetConnection -Port 67 as a DHCP test. DHCP uses UDP broadcast/relay behavior, so a TCP port probe does not prove DHCP works. Also verify DHCP relay/IP helper on routed VLANs.\nScope Changes Did Not Appear on Partner Force replication:\n1 Invoke-DhcpServerv4FailoverReplication -ComputerName \u0026#34;DHCP01.gntech.me\u0026#34; -Name \u0026#34;DHCP01-DHCP02-LAB\u0026#34; -Force DNS Updates Not Working If DHCP is registering DNS records for clients, configure DHCP credentials and check DNS dynamic update settings.\n1 2 Get-DhcpServerDnsCredential Get-DhcpServerv4DnsSetting For AD environments, DNS zones should allow secure dynamic updates.\nFinal Verification Script 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 $ScopeId = \u0026#34;10.0.20.0\u0026#34; $Relationship = \u0026#34;DHCP01-DHCP02-LAB\u0026#34; $Servers = \u0026#34;DHCP01.gntech.me\u0026#34;, \u0026#34;DHCP02.gntech.me\u0026#34; Write-Host \u0026#34;== Authorized DHCP Servers ==\u0026#34; -ForegroundColor Cyan Get-DhcpServerInDC foreach ($Server in $Servers) { Write-Host \u0026#34;== $Server Service ==\u0026#34; -ForegroundColor Cyan Get-Service DHCPServer -ComputerName $Server Write-Host \u0026#34;== $Server Scopes ==\u0026#34; -ForegroundColor Cyan Get-DhcpServerv4Scope -ComputerName $Server Write-Host \u0026#34;== $Server Options ==\u0026#34; -ForegroundColor Cyan Get-DhcpServerv4OptionValue -ComputerName $Server -ScopeId $ScopeId Write-Host \u0026#34;== $Server Leases ==\u0026#34; -ForegroundColor Cyan Get-DhcpServerv4Lease -ComputerName $Server -ScopeId $ScopeId | Select-Object -First 10 Write-Host \u0026#34;== $Server Failover ==\u0026#34; -ForegroundColor Cyan Get-DhcpServerv4Failover -ComputerName $Server -Name $Relationship } Summary The reliable DHCP failover path is:\nBuild two domain-joined Windows Server 2025 DHCP servers Install DHCP role and management tools Authorize both servers in Active Directory Create the scope on the first server Set router, DNS server, and DNS suffix options Create a DHCP failover relationship Replicate scope configuration after changes Configure router/VLAN DHCP relay to both servers Verify leases, options, failover state, and client renewal DHCP failover is not just a checkbox. Treat it like a replicated service: verify both partners, force replication after changes, and test client renewal before you trust it.\n","permalink":"https://blog.gntech.me/posts/2026-05-14-windows-server-2025-dhcp-failover/","summary":"Step-by-step Windows Server 2025 DHCP failover guide: install DHCP, authorize servers in AD, create scopes, configure load-balance or hot-standby failover, replicate scopes, and verify leases.","title":"Windows Server 2025 DHCP Failover — Replicated Scopes and Verification"},{"content":"DNS is not optional in Active Directory. Domain joins, Kerberos, LDAP, Group Policy, Global Catalog discovery, and replication all depend on DNS SRV records. If DNS is wrong, Active Directory looks broken even when the DCs are healthy.\nThis guide configures Windows Server 2025 DNS with Active Directory-integrated replication between DC01 and DC02. It covers the GUI path, the Server Core/PowerShell path, forward and reverse zones, secure dynamic updates, DNS forwarders, client DNS settings, and real verification.\nGUI vs Server Core Path This guide covers both styles:\nDesktop Experience / GUI: DNS Manager, Server Manager, zone wizards, and properties dialogs. Server Core / PowerShell: Install-WindowsFeature, Add-DnsServerPrimaryZone, Set-DnsServerPrimaryZone, Resolve-DnsName, and dcdiag. Use GUI when documenting screenshots. Use PowerShell when building repeatable Windows infrastructure.\nTarget Design Item Value Domain gntech.me DNS/DC 1 DC01 / 10.0.20.10 DNS/DC 2 DC02 / 10.0.20.11 Forward zone gntech.me Reverse zone 20.0.10.in-addr.arpa Replication type AD-integrated Replication scope Domain DNS servers Dynamic updates Secure only Client DNS 10.0.20.10, 10.0.20.11 For Active Directory, the cleanest DNS replication model is AD-integrated DNS. The zone lives in AD DS and replicates through Active Directory replication. Any domain controller running the DNS Server role can answer the zone, and updates can be accepted securely.\nPre-Checks Before touching DNS, verify both domain controllers are healthy:\n1 2 3 4 5 6 Get-ADDomainController -Filter * | Select-Object HostName,IPv4Address,IsGlobalCatalog repadmin /replsummary dcdiag /test:dns /s:DC01 dcdiag /test:dns /s:DC02 Expected:\nDC01 and DC02 are listed replication has no failures DNS tests pass or only show non-critical external forwarder warnings Install DNS Server Role If DNS was installed during DC promotion, this is already done. Verify first:\n1 Get-WindowsFeature DNS GUI: Server Manager Open Server Manager. Go to Manage → Add Roles and Features. Select the local server. Select DNS Server. Accept management tools and install. Server Core / PowerShell 1 Install-WindowsFeature DNS -IncludeManagementTools Verify service status:\n1 Get-Service DNS Expected:\n1 Running Create or Confirm the Forward Zone If this domain was created with AD DS, the forward zone probably already exists. Check it:\n1 2 Get-DnsServerZone -ComputerName DC01 Get-DnsServerZone -ComputerName DC02 Expected zone:\n1 gntech.me GUI: DNS Manager Open Server Manager → Tools → DNS. Expand DC01 → Forward Lookup Zones. Confirm gntech.me exists. Right-click the zone → Properties. Confirm Type is Active Directory-Integrated. Confirm Dynamic updates is Secure only. Click Replication and select the domain DNS replication scope. Server Core / PowerShell If the zone does not exist, create it as AD-integrated:\n1 Add-DnsServerPrimaryZone -Name \u0026#34;gntech.me\u0026#34; -ReplicationScope \u0026#34;Domain\u0026#34; -DynamicUpdate \u0026#34;Secure\u0026#34; If the zone exists, enforce secure dynamic updates:\n1 Set-DnsServerPrimaryZone -Name \u0026#34;gntech.me\u0026#34; -DynamicUpdate \u0026#34;Secure\u0026#34; Check the zone:\n1 2 Get-DnsServerZone -Name \u0026#34;gntech.me\u0026#34; | Select-Object ZoneName,ZoneType,IsDsIntegrated,DynamicUpdate,ReplicationScope Create the Reverse Lookup Zone Reverse DNS improves troubleshooting and logging. It is not required for AD logons, but it is worth configuring.\nGUI: DNS Manager Open DNS Manager. Right-click Reverse Lookup Zones → New Zone. Choose Primary zone. Check Store the zone in Active Directory. Choose replication to DNS servers in the domain. Choose IPv4 Reverse Lookup Zone. Enter network ID 10.0.20. Allow Secure only dynamic updates. Server Core / PowerShell 1 Add-DnsServerPrimaryZone -NetworkId \u0026#34;10.0.20.0/24\u0026#34; -ReplicationScope \u0026#34;Domain\u0026#34; -DynamicUpdate \u0026#34;Secure\u0026#34; Verify:\n1 Get-DnsServerZone | Where-Object ZoneName -like \u0026#34;*in-addr.arpa\u0026#34; Add Static Records for Infrastructure Most domain members register dynamically. Infrastructure records should be explicit.\n1 2 3 Add-DnsServerResourceRecordA -ZoneName \u0026#34;gntech.me\u0026#34; -Name \u0026#34;router\u0026#34; -IPv4Address \u0026#34;10.0.20.1\u0026#34; -CreatePtr Add-DnsServerResourceRecordA -ZoneName \u0026#34;gntech.me\u0026#34; -Name \u0026#34;srv1\u0026#34; -IPv4Address \u0026#34;10.0.20.30\u0026#34; -CreatePtr Verify:\n1 2 3 Resolve-DnsName router.gntech.me -Server 10.0.20.10 Resolve-DnsName router.gntech.me -Server 10.0.20.11 Resolve-DnsName 10.0.20.1 -Server 10.0.20.10 Configure Forwarders Clients should ask AD DNS. AD DNS should forward internet queries.\nGUI: DNS Manager Right-click the DNS server → Properties. Open Forwarders. Add upstream resolvers, for example 1.1.1.1 and 9.9.9.9. Repeat on both DCs, or configure through PowerShell. Server Core / PowerShell 1 2 3 Set-DnsServerForwarder -ComputerName DC01 -IPAddress 1.1.1.1,9.9.9.9 Set-DnsServerForwarder -ComputerName DC02 -IPAddress 1.1.1.1,9.9.9.9 Verify:\n1 2 Get-DnsServerForwarder -ComputerName DC01 Get-DnsServerForwarder -ComputerName DC02 Client DNS Configuration Domain-joined clients must use AD DNS first. Configure DHCP option 6 or static client DNS with both DCs:\n1 2 Preferred DNS: 10.0.20.10 Alternate DNS: 10.0.20.11 Do not hand out public DNS servers directly to Windows clients in an AD domain. Public DNS cannot resolve _ldap._tcp.dc._msdcs.gntech.me.\nVerify DNS Replication Create a test record on DC01:\n1 Add-DnsServerResourceRecordA -ComputerName DC01 -ZoneName \u0026#34;gntech.me\u0026#34; -Name \u0026#34;dnsrep-test\u0026#34; -IPv4Address \u0026#34;10.0.20.250\u0026#34; Force AD replication:\n1 repadmin /syncall /AdeP Query both DNS servers:\n1 2 Resolve-DnsName dnsrep-test.gntech.me -Server 10.0.20.10 Resolve-DnsName dnsrep-test.gntech.me -Server 10.0.20.11 Remove the test record:\n1 Remove-DnsServerResourceRecord -ComputerName DC01 -ZoneName \u0026#34;gntech.me\u0026#34; -RRType A -Name \u0026#34;dnsrep-test\u0026#34; -Force Force replication again and verify it disappears from both servers.\nVerify AD SRV Records These records are what domain clients actually use:\n1 2 3 4 Resolve-DnsName _ldap._tcp.dc._msdcs.gntech.me -Type SRV -Server 10.0.20.10 Resolve-DnsName _ldap._tcp.dc._msdcs.gntech.me -Type SRV -Server 10.0.20.11 Resolve-DnsName _kerberos._tcp.gntech.me -Type SRV -Server 10.0.20.10 Resolve-DnsName _gc._tcp.gntech.me -Type SRV -Server 10.0.20.11 Expected: records for both domain controllers where appropriate.\nRun DCDIAG DNS Tests 1 2 dcdiag /test:dns /v /s:DC01 dcdiag /test:dns /v /s:DC02 A healthy result validates delegation, dynamic updates, SRV records, and basic DNS service health.\nVerify Replication Layer Because AD-integrated DNS rides AD replication, check AD replication too:\n1 2 3 repadmin /replsummary repadmin /showrepl DC01 repadmin /showrepl DC02 No DNS replication design is healthy if AD replication is failing.\nStandard Secondary Zones vs AD-Integrated Zones For domain DNS, use AD-integrated zones. Standard primary/secondary DNS zones still exist, but they replicate by DNS zone transfer, not AD replication.\nUse standard secondary zones only for non-AD DNS designs, DMZ DNS, or special cases where a non-domain DNS server needs a read-only copy.\nFor AD domain controllers:\n1 2 Recommended: AD-integrated DNS zone Avoid: manual standard primary/secondary zone transfer between DCs Final Verification Script 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 $Zone = \u0026#34;gntech.me\u0026#34; $Servers = \u0026#34;10.0.20.10\u0026#34;, \u0026#34;10.0.20.11\u0026#34; Write-Host \u0026#34;== Zones ==\u0026#34; -ForegroundColor Cyan Get-DnsServerZone | Select-Object ZoneName,ZoneType,IsDsIntegrated,DynamicUpdate Write-Host \u0026#34;== Forwarders ==\u0026#34; -ForegroundColor Cyan Get-DnsServerForwarder -ComputerName DC01 Get-DnsServerForwarder -ComputerName DC02 foreach ($Server in $Servers) { Write-Host \u0026#34;== Querying $Server ==\u0026#34; -ForegroundColor Cyan Resolve-DnsName $Zone -Server $Server Resolve-DnsName \u0026#34;_ldap._tcp.dc._msdcs.$Zone\u0026#34; -Type SRV -Server $Server Resolve-DnsName \u0026#34;_kerberos._tcp.$Zone\u0026#34; -Type SRV -Server $Server } Write-Host \u0026#34;== DCDIAG DNS ==\u0026#34; -ForegroundColor Cyan dcdiag /test:dns /s:DC01 dcdiag /test:dns /s:DC02 Write-Host \u0026#34;== AD Replication ==\u0026#34; -ForegroundColor Cyan repadmin /replsummary Summary For Windows Server 2025 domain DNS:\nRun DNS on at least two domain controllers Use AD-integrated zones Use secure dynamic updates Replicate zones to DNS servers in the domain Configure reverse lookup zones Point clients to both AD DNS servers Verify SRV records, dcdiag /test:dns, and AD replication DNS replication is not just a DNS feature in Active Directory. It is AD replication plus DNS Server health. Validate both.\n","permalink":"https://blog.gntech.me/posts/2026-05-14-windows-server-2025-dns-replication/","summary":"Step-by-step Windows Server 2025 DNS replication guide: AD-integrated zones, DNS role install, forward/reverse zones, forwarders, secure dynamic updates, GUI and Server Core paths, and verification.","title":"Windows Server 2025 DNS Server Replication — AD-Integrated Zones"},{"content":"A single domain controller is fine for a lab, but it is still a single point of failure. If DC01 is down, clients lose AD DNS, Kerberos, Group Policy, LDAP, and domain controller discovery. Adding a secondary domain controller gives the domain redundancy and gives you a safer way to patch, reboot, or recover the first DC.\nThis guide adds DC02 as a Windows Server 2025 domain controller to an existing domain. It assumes the first DC already exists and is healthy. If you have not built the first DC yet, start with the primary domain controller guide first.\nGUI vs Server Core Path This guide covers both install styles:\nDesktop Experience / GUI: use Server Manager, AD DS Configuration Wizard, DNS Manager, Active Directory Users and Computers, and Active Directory Sites and Services. Server Core / automation: use PowerShell commands for repeatable builds, remote administration, and clean documentation. The GUI path is useful for screenshots and learning the workflow. The Server Core path is better for real repeatable infrastructure.\nTarget Design Example values used here:\nItem Value Existing DC DC01.gntech.me Existing DC IP 10.0.20.10 New DC DC02.gntech.me New DC IP 10.0.20.11 Domain gntech.me NetBIOS GNTECH Site Default-First-Site-Name initially Roles on DC02 AD DS, DNS, Global Catalog Client DNS Preferred 10.0.20.10, alternate 10.0.20.11 The secondary DC should be on the same reliable server VLAN as the first DC, or in a separate site if you are testing AD Sites and Services. Do not put a DC on unstable Wi-Fi, DHCP-only addressing, or storage you do not trust.\nPre-Checks on DC01 Do not add another domain controller to a broken domain. First, verify DC01 is healthy.\nRun on DC01:\n1 2 3 4 5 dcdiag /test:dns /v /s:DC01 repadmin /replsummary repadmin /showrepl DC01 net share | Select-String \u0026#34;SYSVOL|NETLOGON\u0026#34; netdom query fsmo Expected:\nDNS test passes or only shows non-critical forwarder warnings No replication failures SYSVOL and NETLOGON shares exist FSMO roles are known and reachable Check AD and DNS service status:\n1 2 3 Get-Service NTDS,DNS,DFSR,Netlogon,KDC Get-ADDomain Get-ADForest If any core service is stopped, fix that before promoting DC02.\nPrepare DC02 Start with a clean Windows Server 2025 install.\n1. Patch Windows 1 2 3 Install-Module PSWindowsUpdate -Force Get-WindowsUpdate Install-WindowsUpdate -AcceptAll -AutoReboot Reboot until updates are complete.\n2. Rename the Server 1 Rename-Computer -NewName \u0026#34;DC02\u0026#34; -Restart After reboot:\n1 hostname Expected:\n1 DC02 3. Configure Static IP 1 New-NetIPAddress -InterfaceAlias \u0026#34;Ethernet\u0026#34; -IPAddress 10.0.20.11 -PrefixLength 24 -DefaultGateway 10.0.20.1 For DNS, point DC02 to the existing domain controller first. This is important because DC02 must find the existing AD domain before it can join or promote.\n1 Set-DnsClientServerAddress -InterfaceAlias \u0026#34;Ethernet\u0026#34; -ServerAddresses 10.0.20.10 Verify network and DNS:\n1 2 3 Test-Connection 10.0.20.10 -Count 4 Resolve-DnsName gntech.me Resolve-DnsName _ldap._tcp.dc._msdcs.gntech.me -Type SRV If SRV lookup fails, stop. Domain promotion depends on AD DNS.\n4. Check Time Sync 1 2 w32tm /query /status w32tm /stripchart /computer:DC01 /samples:5 /dataonly Kerberos requires time to be close. Fix time skew before joining the domain.\nJoin DC02 to the Existing Domain You can promote a workgroup server directly, but joining first makes basic DNS, credential, and secure channel problems obvious before AD DS promotion.\n1 Add-Computer -DomainName \u0026#34;gntech.me\u0026#34; -Credential \u0026#34;GNTECH\\Administrator\u0026#34; -Restart After reboot, log in as a domain admin:\n1 GNTECH\\Administrator Verify the secure channel:\n1 2 Test-ComputerSecureChannel -Verbose whoami /fqdn Expected:\n1 True Install AD DS on DC02 GUI: Install the Role with Server Manager On Windows Server 2025 Desktop Experience:\nOpen Server Manager. Go to Manage → Add Roles and Features. Choose Role-based or feature-based installation. Select DC02. Select Active Directory Domain Services. Accept the required management tools. Click Install. Server Core / PowerShell: Install the Role Install the AD DS role and tools:\n1 Install-WindowsFeature AD-Domain-Services -IncludeManagementTools Verify:\n1 Get-WindowsFeature AD-Domain-Services Expected:\n1 [X] Active Directory Domain Services Promote DC02 as an Additional Domain Controller GUI: Promote with the AD DS Wizard After the AD DS role installs, Server Manager shows a notification flag.\nClick the notification flag. Select Promote this server to a domain controller. Choose Add a domain controller to an existing domain. Enter gntech.me and provide domain admin credentials. Keep Domain Name System (DNS) server checked. Keep Global Catalog (GC) checked. Set the Directory Services Restore Mode password for DC02. Pick a replication source or leave it automatic. Review paths and run the prerequisites check. Click Install and let DC02 reboot. Server Core / PowerShell: Promote with Install-ADDSDomainController Use Install-ADDSDomainController. This adds a domain controller to an existing domain, instead of creating a new forest.\n1 2 3 4 $Credential = Get-Credential \u0026#34;GNTECH\\Administrator\u0026#34; $SafeModePassword = Read-Host -Prompt \u0026#34;Enter Directory Services Restore Mode password for DC02\u0026#34; -AsSecureString Install-ADDSDomainController -DomainName \u0026#34;gntech.me\u0026#34; -Credential $Credential -InstallDns -NoGlobalCatalog:$false -SafeModeAdministratorPassword $SafeModePassword -NoRebootOnCompletion:$false -Force The server reboots after promotion.\nNotes:\n-InstallDns installs DNS on DC02. -NoGlobalCatalog:$false makes DC02 a Global Catalog server. The DSRM password is local to this domain controller. Do not use Install-ADDSForest; that creates a new forest and is the wrong command for a secondary DC. Set DNS Client Order After Promotion After DC02 is promoted and DNS is installed, update DNS client settings on both DCs.\nOn DC01:\n1 Set-DnsClientServerAddress -InterfaceAlias \u0026#34;Ethernet\u0026#34; -ServerAddresses 10.0.20.10,10.0.20.11 On DC02:\n1 Set-DnsClientServerAddress -InterfaceAlias \u0026#34;Ethernet\u0026#34; -ServerAddresses 10.0.20.11,10.0.20.10 This keeps each DC using itself first, with the other DC as backup. Some admins prefer the partner DC first to avoid DNS island scenarios during boot. In small homelabs, either pattern can work if both DCs are healthy; be consistent and test failover.\nFor clients, set DHCP DNS options to both DCs:\n1 2 DNS Server 1: 10.0.20.10 DNS Server 2: 10.0.20.11 Do not hand out public DNS servers to domain-joined clients. Public DNS cannot answer AD SRV records.\nVerify DC02 Promotion Run these checks after the reboot.\n1. Confirm DC Discovery From DC02:\n1 2 nltest /dsgetdc:gntech.me nltest /dclist:gntech.me Expected: both DC01 and DC02 appear in the domain controller list.\n2. Verify Domain Controller Object 1 2 Get-ADDomainController -Filter * | Select-Object HostName,IPv4Address,Site,IsGlobalCatalog,OperationMasterRoles Expected:\nDC01 appears DC02 appears DC02 has IsGlobalCatalog set to True 3. Run DCDIAG on Both DCs 1 2 dcdiag /v /c /d /e \u0026gt; C:\\dcdiag-all-dcs.txt notepad C:\\dcdiag-all-dcs.txt Then run targeted DNS tests:\n1 2 dcdiag /test:dns /v /s:DC01 dcdiag /test:dns /v /s:DC02 Focus on:\n1 2 3 4 5 6 7 8 9 10 Advertising Connectivity DFSREvent SysVolCheck KccEvent MachineAccount NetLogons Services SystemLog DNS Short-lived replication warnings immediately after promotion can happen. Wait a few minutes, force replication, and retest before treating them as real failures.\n4. Verify Replication 1 2 3 repadmin /replsummary repadmin /showrepl DC01 repadmin /showrepl DC02 A healthy result has no failed replication attempts.\nForce synchronization:\n1 repadmin /syncall /AdeP Then check again:\n1 repadmin /replsummary 5. Verify SYSVOL and NETLOGON on DC02 1 2 net view \\\\DC02 net share Expected shares on DC02:\n1 2 NETLOGON SYSVOL Check paths:\n1 2 Test-Path C:\\Windows\\SYSVOL\\sysvol Test-Path C:\\Windows\\SYSVOL\\domain\\scripts Both should return True.\nIf SYSVOL or NETLOGON is missing, check DFS Replication:\n1 2 Get-Service DFSR Get-WinEvent -LogName \u0026#34;DFS Replication\u0026#34; -MaxEvents 30 Do not continue until SYSVOL is healthy.\n6. Verify DNS Zones Replicated 1 2 3 4 Get-DnsServerZone Resolve-DnsName gntech.me -Server 10.0.20.10 Resolve-DnsName gntech.me -Server 10.0.20.11 Resolve-DnsName _ldap._tcp.dc._msdcs.gntech.me -Type SRV -Server 10.0.20.11 Expected: both DCs answer the zone and SRV records.\nCheck host records:\n1 2 Resolve-DnsName DC01.gntech.me -Server 10.0.20.11 Resolve-DnsName DC02.gntech.me -Server 10.0.20.10 7. Verify Global Catalog 1 Get-ADDomainController DC02 | Select-Object HostName,IsGlobalCatalog Expected:\n1 IsGlobalCatalog : True You can also verify the GC port:\n1 Test-NetConnection DC02 -Port 3268 8. Verify FSMO Roles Stayed Put Adding a secondary DC should not move FSMO roles automatically.\n1 netdom query fsmo Expected: roles are still on DC01 unless you intentionally moved them. For a two-DC homelab, leaving FSMO roles on DC01 is fine. If you want DC02 to become the maintenance target later, move roles intentionally and document it.\nClient Failover Test Pick a domain-joined Windows client.\nSet DNS to both DCs:\n1 Set-DnsClientServerAddress -InterfaceAlias \u0026#34;Ethernet\u0026#34; -ServerAddresses 10.0.20.10,10.0.20.11 Verify domain discovery:\n1 2 3 4 nltest /dsgetdc:gntech.me Resolve-DnsName _kerberos._tcp.gntech.me -Type SRV Test-ComputerSecureChannel -Verbose gpupdate /force Now simulate DC01 being unavailable. For a lab, the cleanest test is to stop DNS registration use by pointing the client only to DC02, not by hard powering off a DC during replication.\n1 2 3 4 5 Set-DnsClientServerAddress -InterfaceAlias \u0026#34;Ethernet\u0026#34; -ServerAddresses 10.0.20.11 ipconfig /flushdns nltest /dsgetdc:gntech.me gpupdate /force Expected: the client discovers DC02, resolves AD SRV records, and applies Group Policy.\nRestore normal DNS order afterward:\n1 Set-DnsClientServerAddress -InterfaceAlias \u0026#34;Ethernet\u0026#34; -ServerAddresses 10.0.20.10,10.0.20.11 AD Sites and Services Cleanup GUI: Active Directory Sites and Services Open Active Directory Sites and Services:\n1 dssite.msc Check:\nDC01 and DC02 are under the correct site Subnets are defined if you have multiple VLANs/sites NTDS Settings exists under each DC Replication connections were generated by KCC For a single site, the default is usually fine. For routed homelabs with multiple VLANs or remote sites, define subnets so clients prefer the nearest DC.\nPowerShell example:\n1 New-ADReplicationSubnet -Name \u0026#34;10.0.20.0/24\u0026#34; -Site \u0026#34;Default-First-Site-Name\u0026#34; Monitoring Checks to Keep Add these to your maintenance routine:\n1 2 3 4 repadmin /replsummary dcdiag /test:dns /s:DC01 dcdiag /test:dns /s:DC02 Get-ADDomainController -Filter * | Select HostName,IPv4Address,IsGlobalCatalog Event logs worth watching:\n1 2 3 4 Get-WinEvent -LogName \u0026#34;Directory Service\u0026#34; -MaxEvents 20 Get-WinEvent -LogName \u0026#34;DNS Server\u0026#34; -MaxEvents 20 Get-WinEvent -LogName \u0026#34;DFS Replication\u0026#34; -MaxEvents 20 Get-WinEvent -LogName \u0026#34;System\u0026#34; -MaxEvents 20 Common Problems Promotion Fails: Cannot Contact Domain Check DNS on DC02 before promotion:\n1 2 3 4 Get-DnsClientServerAddress Resolve-DnsName _ldap._tcp.dc._msdcs.gntech.me -Type SRV Test-NetConnection DC01 -Port 389 Test-NetConnection DC01 -Port 53 Fix: point DC02 DNS to DC01 before promotion.\nReplication Shows 1908 or 1726 Right After Promotion Immediately after promotion, transient replication errors can appear while services finish registering. Wait a few minutes, then run:\n1 2 repadmin /syncall /AdeP repadmin /replsummary If errors persist, check firewall, DNS, time sync, and Directory Service events.\nDC02 Missing SYSVOL or NETLOGON Check DFSR logs:\n1 Get-WinEvent -LogName \u0026#34;DFS Replication\u0026#34; -MaxEvents 50 Also verify the DFSR service:\n1 Get-Service DFSR Do not point clients to DC02 until SYSVOL and NETLOGON are present.\nClients Still Only Use DC01 Check Sites and Services and DNS SRV records:\n1 2 nltest /dsgetdc:gntech.me /force Resolve-DnsName _ldap._tcp.dc._msdcs.gntech.me -Type SRV If clients have static DNS pointing only to DC01, update DHCP option 6 or the client NIC configuration.\nFinal Verification Script Run from an elevated PowerShell session on either DC:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 $Domain = \u0026#34;gntech.me\u0026#34; $DCs = \u0026#34;DC01\u0026#34;, \u0026#34;DC02\u0026#34; Write-Host \u0026#34;== Domain Controllers ==\u0026#34; -ForegroundColor Cyan Get-ADDomainController -Filter * | Select-Object HostName,IPv4Address,Site,IsGlobalCatalog,OperationMasterRoles Write-Host \u0026#34;== DC Discovery ==\u0026#34; -ForegroundColor Cyan nltest /dclist:$Domain Write-Host \u0026#34;== DNS SRV Records ==\u0026#34; -ForegroundColor Cyan Resolve-DnsName \u0026#34;_ldap._tcp.dc._msdcs.$Domain\u0026#34; -Type SRV Resolve-DnsName \u0026#34;_kerberos._tcp.$Domain\u0026#34; -Type SRV Write-Host \u0026#34;== Replication Summary ==\u0026#34; -ForegroundColor Cyan repadmin /replsummary foreach ($DC in $DCs) { Write-Host \u0026#34;== $DC DNS Test ==\u0026#34; -ForegroundColor Cyan dcdiag /test:dns /s:$DC Write-Host \u0026#34;== $DC Shares ==\u0026#34; -ForegroundColor Cyan net view \\\\$DC | Select-String \u0026#34;SYSVOL|NETLOGON\u0026#34; Write-Host \u0026#34;== $DC GC Port ==\u0026#34; -ForegroundColor Cyan Test-NetConnection $DC -Port 3268 } Write-Host \u0026#34;== FSMO Roles ==\u0026#34; -ForegroundColor Cyan netdom query fsmo Summary The safe path to a secondary domain controller is:\nVerify DC01 is healthy first Patch and rename DC02 Set static IP and point DNS to DC01 Join DC02 to the domain Install AD DS Promote with Install-ADDSDomainController Install DNS and enable Global Catalog Verify replication, DNS, SYSVOL, NETLOGON, GC, and FSMO roles Update client/DHCP DNS to include both DCs Test client failover to DC02 Do not treat promotion as the finish line. The finish line is a clean dcdiag, healthy repadmin, valid DNS SRV records, present SYSVOL/NETLOGON, and a client that can still log in and apply Group Policy when using the secondary DC.\n","permalink":"https://blog.gntech.me/posts/2026-05-14-windows-server-2025-secondary-domain-controller/","summary":"Step-by-step guide to adding DC02 as a secondary Windows Server 2025 domain controller: preparation, AD DS promotion, DNS, Global Catalog, replication checks, SYSVOL validation, and client failover testing.","title":"Windows Server 2025 Secondary Domain Controller — Add DC02 to Existing AD"},{"content":"A domain controller is still the center of a Windows network. Even in a small homelab, Active Directory gives you centralized identity, DNS, Group Policy, Kerberos authentication, device management, and a clean place to test enterprise workflows without touching production.\nThis guide builds a fresh Windows Server 2025 domain controller from zero: static IP, hostname, AD DS installation, DNS, forest creation, first OU/user structure, client join, and verification. The focus is not \u0026ldquo;click Next until it works\u0026rdquo;. The focus is building it in a way you can troubleshoot later.\nGUI vs Server Core Path This guide covers both install styles:\nDesktop Experience / GUI: use Server Manager, Settings, DNS Manager, Active Directory Users and Computers, and the AD DS configuration wizard. Server Core / automation: use PowerShell only. This is the cleaner path for repeatable homelab builds, unattended deployment, and documentation. Use the GUI path when you want screenshots or you are learning the flow. Use the PowerShell path when you want a build you can reproduce exactly.\nTarget Lab Design Example values used in this guide:\nItem Value Server name DC01 Server OS Windows Server 2025 Domain/FQDN gntech.me NetBIOS name GNTECH DC IP 10.0.20.10/24 Gateway 10.0.20.1 DNS on DC 127.0.0.1, then 10.0.20.10 DNS on clients 10.0.20.10 Site Default-First-Site-Name initially Use a real internal domain that you own if this will become production. For a lab, .local is common, but it can conflict with mDNS in mixed Linux/macOS networks. A safer production-style pattern is a delegated subdomain like ad.example.com or corp.example.com.\nBefore You Install AD DS Do these first. Most broken domain controllers come from skipping one of these basics.\n1. Patch the Server Install Windows updates before promotion:\n1 2 3 Install-Module PSWindowsUpdate -Force Get-WindowsUpdate Install-WindowsUpdate -AcceptAll -AutoReboot If you do not use PSWindowsUpdate, run updates from Settings or Windows Admin Center. Reboot until there are no pending updates.\n2. Rename the Server Do not promote a server named WIN-ABC123XYZ.\n1 Rename-Computer -NewName \u0026#34;DC01\u0026#34; -Restart After reboot:\n1 hostname Expected:\n1 DC01 3. Configure a Static IP Find the interface name:\n1 Get-NetAdapter Set the address:\n1 New-NetIPAddress -InterfaceAlias \u0026#34;Ethernet\u0026#34; -IPAddress 10.0.20.10 -PrefixLength 24 -DefaultGateway 10.0.20.1 Set temporary DNS to itself. Before AD DS is installed this looks odd, but the DNS role will be installed during domain controller promotion.\n1 Set-DnsClientServerAddress -InterfaceAlias \u0026#34;Ethernet\u0026#34; -ServerAddresses 127.0.0.1 Verify:\n1 2 Get-NetIPConfiguration Get-DnsClientServerAddress -InterfaceAlias \u0026#34;Ethernet\u0026#34; 4. Check Time Sync Kerberos is time-sensitive. A five-minute clock skew can break logons.\n1 w32tm /query /status For a single lab DC, sync the host/VM from a reliable source. For a production domain, configure the PDC Emulator to use external NTP and let domain members sync from the domain hierarchy.\nInstall AD DS and DNS GUI: Install the Role with Server Manager On Windows Server 2025 Desktop Experience:\nOpen Server Manager. Go to Manage → Add Roles and Features. Choose Role-based or feature-based installation. Select the local server DC01. Select Active Directory Domain Services. Accept the required management tools. Continue through the wizard and click Install. The role install does not make the server a domain controller yet. It only installs the AD DS binaries and management tools.\nServer Core / PowerShell: Install the Role Install the AD DS role with management tools:\n1 Install-WindowsFeature AD-Domain-Services -IncludeManagementTools Verify the role is present:\n1 Get-WindowsFeature AD-Domain-Services Expected result:\n1 [X] Active Directory Domain Services The server is not a domain controller yet. The role only installs the binaries. Promotion creates the forest/domain and configures DNS, SYSVOL, NTDS, Kerberos, and the first domain controller objects.\nPromote the Server to a New Forest GUI: Promote with the AD DS Wizard After role installation, Server Manager shows a notification flag.\nClick the notification flag. Select Promote this server to a domain controller. Choose Add a new forest. Enter the root domain name: gntech.me. Keep Domain Name System (DNS) server checked. Set the Directory Services Restore Mode password. Confirm the NetBIOS name: GNTECH. Review paths for database, logs, and SYSVOL. Run the prerequisites check. Click Install and let the server reboot. Server Core / PowerShell: Promote with Install-ADDSForest For a new lab forest:\n1 2 3 $SafeModePassword = Read-Host -Prompt \u0026#34;Enter Directory Services Restore Mode password\u0026#34; -AsSecureString Install-ADDSForest -DomainName \u0026#34;gntech.me\u0026#34; -DomainNetbiosName \u0026#34;GNTECH\u0026#34; -InstallDns -SafeModeAdministratorPassword $SafeModePassword -NoRebootOnCompletion:$false -Force The server reboots automatically.\nDo not force a functional level unless you know exactly why. For a clean Windows Server 2025-only lab, the default selected by Server Manager or PowerShell is fine. In a real environment, check compatibility before raising forest or domain functional levels.\nAfter reboot, log in as:\n1 GNTECH\\Administrator or:\n1 gntech.me\\Administrator Post-Promotion DNS Cleanup After promotion, set the server NIC DNS to the DC IP. Avoid leaving only 127.0.0.1 on the NIC because it makes later troubleshooting less clear.\n1 Set-DnsClientServerAddress -InterfaceAlias \u0026#34;Ethernet\u0026#34; -ServerAddresses 10.0.20.10 Verify:\n1 2 3 Get-DnsClientServerAddress -InterfaceAlias \u0026#34;Ethernet\u0026#34; Resolve-DnsName gntech.me Resolve-DnsName dc01.gntech.me Expected: both names resolve to the domain/DC records, with DC01 resolving to 10.0.20.10.\nConfigure DNS Forwarders Clients should use the DC for DNS. The DC can forward unknown internet queries to upstream resolvers.\nGUI: DNS Manager Open Server Manager → Tools → DNS. Right-click DC01 → Properties. Open the Forwarders tab. Add 1.1.1.1 and 9.9.9.9, or your internal resolver. Server Core / PowerShell 1 2 Add-DnsServerForwarder -IPAddress 1.1.1.1,9.9.9.9 -PassThru Get-DnsServerForwarder If your firewall/router provides internal DNS for other VLANs, you can forward to that instead. The rule is simple: clients ask AD DNS first; AD DNS either answers internal records or forwards external queries.\nCreate a Basic OU Structure GUI: Create OUs in Active Directory Users and Computers On Desktop Experience:\nOpen Server Manager → Tools → Active Directory Users and Computers. Expand gntech.me. Right-click the domain → New → Organizational Unit. Create Servers, Workstations, Users, Groups, and Service Accounts. Right-click Users → New → User to create test.user. Server Core / PowerShell: Create OUs and Users Install the AD PowerShell module if it is not loaded automatically:\n1 Import-Module ActiveDirectory Create a clean structure:\n1 2 3 4 5 6 7 $BaseDN = \u0026#34;DC=gntech,DC=me\u0026#34; New-ADOrganizationalUnit -Name \u0026#34;Servers\u0026#34; -Path $BaseDN New-ADOrganizationalUnit -Name \u0026#34;Workstations\u0026#34; -Path $BaseDN New-ADOrganizationalUnit -Name \u0026#34;Users\u0026#34; -Path $BaseDN New-ADOrganizationalUnit -Name \u0026#34;Groups\u0026#34; -Path $BaseDN New-ADOrganizationalUnit -Name \u0026#34;Service Accounts\u0026#34; -Path $BaseDN Create a test user:\n1 2 3 $Password = Read-Host \u0026#34;Password for test.user\u0026#34; -AsSecureString New-ADUser -Name \u0026#34;Test User\u0026#34; -GivenName \u0026#34;Test\u0026#34; -Surname \u0026#34;User\u0026#34; -SamAccountName \u0026#34;test.user\u0026#34; -UserPrincipalName \u0026#34;test.user@gntech.me\u0026#34; -Path \u0026#34;OU=Users,$BaseDN\u0026#34; -AccountPassword $Password -Enabled $true -ChangePasswordAtLogon $true Verify:\n1 2 Get-ADOrganizationalUnit -Filter * | Select-Object Name,DistinguishedName Get-ADUser test.user -Properties Enabled,UserPrincipalName Configure the Reverse Lookup Zone Reverse DNS is not mandatory for every lab, but it helps with clean logging and diagnostics.\n1 Add-DnsServerPrimaryZone -NetworkId \u0026#34;10.0.20.0/24\u0026#34; -ReplicationScope \u0026#34;Domain\u0026#34; Create or verify the PTR record for the DC:\n1 Add-DnsServerResourceRecordPtr -Name \u0026#34;10\u0026#34; -ZoneName \u0026#34;20.0.10.in-addr.arpa\u0026#34; -PtrDomainName \u0026#34;dc01.gntech.me\u0026#34; Check it:\n1 Resolve-DnsName 10.0.20.10 If the PTR already exists, the add command will fail. That is fine — verify the existing record instead of duplicating it.\nJoin a Windows Client to the Domain On a Windows 11/Windows Server client, set DNS to the DC:\n1 Set-DnsClientServerAddress -InterfaceAlias \u0026#34;Ethernet\u0026#34; -ServerAddresses 10.0.20.10 Verify DNS before joining:\n1 2 Resolve-DnsName gntech.me Resolve-DnsName _ldap._tcp.dc._msdcs.gntech.me -Type SRV Join the domain:\n1 Add-Computer -DomainName \u0026#34;gntech.me\u0026#34; -Credential \u0026#34;GNTECH\\Administrator\u0026#34; -Restart After reboot, log in as:\n1 GNTECH\\test.user Move the computer object into the correct OU:\n1 2 Get-ADComputer -Filter \u0026#39;Name -eq \u0026#34;CLIENT01\u0026#34;\u0026#39; | Move-ADObject -TargetPath \u0026#34;OU=Workstations,DC=gntech,DC=me\u0026#34; Verification Checklist This is the important part. A DC that installs without errors can still have DNS, SYSVOL, replication, or secure channel problems.\n1. Confirm Domain Controller Discovery Run from the DC:\n1 nltest /dsgetdc:gntech.me Expected indicators:\n1 2 3 4 5 6 DC: \\\\DC01 Address: \\\\10.0.20.10 Dom Guid: ... Dom Name: gntech.me Forest Name: gntech.me The command completed successfully 2. Run DCDIAG 1 2 dcdiag /v /c /d /e /s:DC01 \u0026gt; C:\\dcdiag-DC01.txt notepad C:\\dcdiag-DC01.txt For a single-DC lab, focus on:\n1 2 3 4 5 6 7 8 9 10 Connectivity Advertising FrsEvent or DFSREvent SysVolCheck KccEvent MachineAccount NetLogons Services SystemLog DNS A clean result should show tests passed. DNS warnings are common if forwarders or internet access are not configured yet, but internal SRV records must work.\nQuick DNS-only test:\n1 dcdiag /test:dns /v /s:DC01 3. Verify SYSVOL and NETLOGON Shares 1 net share Expected shares:\n1 2 NETLOGON SYSVOL Check paths:\n1 2 Test-Path C:\\Windows\\SYSVOL\\sysvol Test-Path C:\\Windows\\SYSVOL\\domain\\scripts Both should return:\n1 True 4. Verify Replication Health Even with one DC, use the replication tools now so the commands are normal before you add a second DC.\n1 2 repadmin /replsummary repadmin /showrepl DC01 For a single DC, there may be little to replicate, but there should be no replication failures.\n5. Verify FSMO Roles 1 netdom query fsmo Expected: all five FSMO roles are held by DC01 in a single-DC forest.\n1 2 3 4 5 Schema master DC01.gntech.me Domain naming master DC01.gntech.me PDC DC01.gntech.me RID pool manager DC01.gntech.me Infrastructure master DC01.gntech.me 6. Verify DNS SRV Records 1 2 3 Resolve-DnsName _ldap._tcp.dc._msdcs.gntech.me -Type SRV Resolve-DnsName _kerberos._tcp.gntech.me -Type SRV Resolve-DnsName gc._msdcs.gntech.me -Type A If these fail, clients will not reliably find domain controllers.\n7. Verify AD Web Services PowerShell AD cmdlets depend on AD Web Services.\n1 2 3 Get-Service ADWS Get-ADDomain Get-ADForest Expected service status:\n1 Running 8. Verify Client Secure Channel On a domain-joined client:\n1 2 3 4 Test-ComputerSecureChannel -Verbose whoami /fqdn gpupdate /force gpresult /r Expected:\nTest-ComputerSecureChannel returns True whoami /fqdn returns the user\u0026rsquo;s AD distinguished name gpupdate completes without DC discovery errors gpresult /r shows domain policy information Basic Hardening After Install Do these before building more services around the domain.\nDisable Unneeded Local Administrator Use Create named admin accounts and keep the built-in Administrator for break-glass use.\n1 New-ADGroup -Name \u0026#34;GG-Server-Admins\u0026#34; -GroupScope Global -GroupCategory Security -Path \u0026#34;OU=Groups,DC=gntech,DC=me\u0026#34; Enable the Recycle Bin 1 Enable-ADOptionalFeature -Identity \u0026#39;Recycle Bin Feature\u0026#39; -Scope ForestOrConfigurationSet -Target \u0026#39;gntech.me\u0026#39; Verify:\n1 2 Get-ADOptionalFeature -Filter \u0026#39;Name -like \u0026#34;Recycle Bin Feature\u0026#34;\u0026#39; | Select-Object Name,EnabledScopes Configure Backups At minimum, back up System State:\n1 2 Install-WindowsFeature Windows-Server-Backup wbadmin start systemstatebackup -backupTarget:E: -quiet For a VM, also keep hypervisor-level backups, but do not treat VM snapshots as your only AD backup strategy. AD has replication and USN rollback considerations once you have more than one DC.\nKeep DNS Simple Recommended client DNS:\n1 Client DNS server: 10.0.20.10 Not recommended:\n1 2 Client DNS server: 1.1.1.1 Client DNS server: 8.8.8.8 Clients must ask AD DNS first. If they ask public DNS first, domain join, Group Policy, Kerberos, and SRV record discovery will randomly fail.\nCommon Problems and Fixes Domain Join Fails: Domain Not Found Check DNS on the client:\n1 2 Get-DnsClientServerAddress Resolve-DnsName _ldap._tcp.dc._msdcs.gntech.me -Type SRV Fix: point client DNS to the DC, not the router or public resolvers.\nDCDIAG DNS Test Fails Check the DNS service:\n1 2 3 Get-Service DNS Get-DnsServerZone Get-DnsServerForwarder Then restart Netlogon to re-register SRV records:\n1 2 Restart-Service Netlogon ipconfig /registerdns SYSVOL or NETLOGON Missing Check DFSR:\n1 2 Get-Service DFSR Get-WinEvent -LogName \u0026#34;DFS Replication\u0026#34; -MaxEvents 20 Do not ignore missing SYSVOL/NETLOGON. Group Policy and logon scripts will not work correctly until SYSVOL is healthy.\nTime/Kerberos Errors 1 2 w32tm /query /status w32tm /resync On clients:\n1 2 w32tm /query /source klist purge If time is wrong, Kerberos tickets fail and logons get weird fast.\nFinal Verification Script Run this after the DC is built:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 $Domain = \u0026#34;gntech.me\u0026#34; $DC = \u0026#34;DC01\u0026#34; Write-Host \u0026#34;== DNS ==\u0026#34; -ForegroundColor Cyan Resolve-DnsName $Domain Resolve-DnsName \u0026#34;_ldap._tcp.dc._msdcs.$Domain\u0026#34; -Type SRV Write-Host \u0026#34;== DC Discovery ==\u0026#34; -ForegroundColor Cyan nltest /dsgetdc:$Domain Write-Host \u0026#34;== Shares ==\u0026#34; -ForegroundColor Cyan net share | Select-String \u0026#34;SYSVOL|NETLOGON\u0026#34; Write-Host \u0026#34;== FSMO ==\u0026#34; -ForegroundColor Cyan netdom query fsmo Write-Host \u0026#34;== Replication ==\u0026#34; -ForegroundColor Cyan repadmin /replsummary Write-Host \u0026#34;== DCDIAG DNS ==\u0026#34; -ForegroundColor Cyan dcdiag /test:dns /s:$DC Write-Host \u0026#34;== AD Cmdlets ==\u0026#34; -ForegroundColor Cyan Get-ADDomain Get-ADForest If DNS, SYSVOL, FSMO, AD cmdlets, and client join all pass, you have a working Windows Server 2025 domain controller.\nSummary The reliable path is straightforward:\nPatch Windows Server 2025 Rename the server Assign a static IP Install AD DS with management tools Promote to a new forest with DNS Point clients to AD DNS Create OUs/users/groups Join a test client Verify with dcdiag, repadmin, nltest, DNS SRV lookups, SYSVOL, and client secure channel tests The install wizard is the easy part. Verification is what tells you the domain is actually usable.\n","permalink":"https://blog.gntech.me/posts/2026-05-14-windows-server-2025-domain-controller/","summary":"Step-by-step Windows Server 2025 domain controller deployment: static IP, AD DS installation, DNS configuration, domain promotion, OU/user setup, client join testing, and post-install verification.","title":"Windows Server 2025 Domain Controller — AD DS Install and Verification"},{"content":"Most docker-compose.yml files you find in tutorials are minimal. They work for docker compose up -d on a fresh system, but they don\u0026rsquo;t handle service startup order, container crashes, secrets, or configuration reuse — all things that matter when your stack runs for months.\nThis post covers the patterns that turn a throwaway compose file into something you can deploy, forget, and trust. Every pattern includes a real example you can drop into your existing stack.\nNote: Some examples are partial Compose snippets meant to demonstrate one pattern at a time. When copying them into a real compose.yml, make sure referenced services, secrets, volumes, and networks are also defined.\n1. Healthchecks — Know When a Container Is Actually Ready The default Docker healthcheck is: \u0026ldquo;is the process running?\u0026rdquo; That\u0026rsquo;s not the same as \u0026ldquo;is the service accepting traffic?\u0026rdquo;\nA PostgreSQL container can start with the process running while it\u0026rsquo;s still recovering the WAL. A Traefik container can bind port 80 while loading its configuration. Healthchecks solve this.\nDatabase Healthcheck 1 2 3 4 5 6 7 8 9 services: postgres: image: postgres:16-alpine healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;pg_isready -U $${POSTGRES_USER} -d $${POSTGRES_DB}\u0026#34;] interval: 10s timeout: 5s retries: 5 start_period: 30s start_period: gives the container 30s grace before healthchecks count. PostgreSQL needs this on first start — it initializes the data directory before accepting connections. interval: check every 10 seconds. retries: mark unhealthy after 5 consecutive failures. Web Service Healthcheck 1 2 3 4 5 6 7 8 9 services: nginx: image: nginx:alpine healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;nginx\u0026#34;, \u0026#34;-t\u0026#34;] interval: 30s timeout: 5s retries: 3 start_period: 10s For HTTP services, use curl:\n1 2 3 4 5 6 7 8 9 services: app: image: myapp:latest healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;curl\u0026#34;, \u0026#34;-f\u0026#34;, \u0026#34;http://localhost:8080/health\u0026#34;] interval: 15s timeout: 3s retries: 3 start_period: 15s The container must have curl installed. Use a curl-based image or install it in your Dockerfile:\n1 2 FROM alpine:3.20 RUN apk add --no-cache curl View Health Status 1 2 3 4 5 6 7 8 # Show health status for all containers docker compose ps # Filter to unhealthy containers only docker compose ps --status unhealthy # See healthcheck log output docker inspect --format=\u0026#39;{{json .State.Health}}\u0026#39; \u0026#34;\u0026lt;container_name\u0026gt;\u0026#34; A container marked (unhealthy) won\u0026rsquo;t receive traffic from other services using depends_on: condition: service_healthy.\n2. Depends On with Conditions — Real Startup Ordering The old depends_on only waited for a container to start, not for it to be ready. Use a supported Compose plugin with conditional depends_on and healthchecks. The service_healthy condition is not new to Compose 2.20; validate the installed plugin\u0026rsquo;s parsed configuration.\nService-Healthy Dependencies 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 secrets: db_app_password: file: ./secrets/db_app_password.txt services: postgres: image: postgres:16-alpine secrets: - db_app_password environment: POSTGRES_USER: appuser POSTGRES_DB: appdb POSTGRES_PASSWORD_FILE: /run/secrets/db_app_password healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;pg_isready -U appuser -d appdb\u0026#34;] interval: 5s timeout: 3s retries: 5 start_period: 20s api: image: myapi:latest depends_on: postgres: condition: service_healthy secrets: - db_app_password healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;curl\u0026#34;, \u0026#34;-f\u0026#34;, \u0026#34;http://localhost:3000/health\u0026#34;] interval: 10s timeout: 3s retries: 3 start_period: 10s environment: DATABASE_HOST: postgres DATABASE_PORT: \u0026#34;5432\u0026#34; DATABASE_NAME: appdb DATABASE_USER: appuser DATABASE_PASSWORD_FILE: /run/secrets/db_app_password nginx: image: nginx:alpine depends_on: api: condition: service_healthy ports: - \u0026#34;80:80\u0026#34; This creates a chain: nginx waits for api, api waits for postgres. Each service starts exactly when its dependency reports healthy.\nService-Started (Default) vs Service-Healthy 1 2 3 4 5 6 7 8 9 10 11 12 13 services: # Only needs to start after redis starts the process (default) cache-worker: image: myworker:latest depends_on: - redis # Must wait until postgres accepts connections app: image: myapp:latest depends_on: postgres: condition: service_healthy Use service_started (the default) for fast dependencies like Redis or Memcached. Use service_healthy for databases, message queues, and anything with startup initialization.\n3. Restart Policies — Survive Crashes Gracefully Don\u0026rsquo;t leave containers set to restart: unless-stopped without understanding the failure modes.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 services: # Core services that must stay up traefik: image: traefik:v3.3 restart: unless-stopped ports: - \u0026#34;80:80\u0026#34; - \u0026#34;443:443\u0026#34; # One-shot or batch jobs — never restart db-migrate: image: myapp:latest restart: no command: [\u0026#34;npm\u0026#34;, \u0026#34;run\u0026#34;, \u0026#34;migrate\u0026#34;] # Services that can crash briefly but should retry fast monitoring-agent: image: grafana/agent:latest restart: on-failure:5 # Max 5 retries, then give up Restart Strategy Decision Table Restart Policy Use Case Behavior no Batch jobs, migrations, cron containers Never restarts always Critical infrastructure, reverse proxies Restarts regardless of exit code unless-stopped Long-running services you might manually stop Restarts unless manually stopped on-failure:N Services with occasional transient failures Restarts only on non-zero exit, up to N times Resource Limits Prevent Restart Loops A container that OOM-kills and restarts in a loop will keep getting killed unless you constrain memory:\n1 2 3 4 5 6 7 8 9 10 services: buggy-app: image: myapp:latest restart: on-failure:3 deploy: resources: limits: memory: 256M reservations: memory: 128M With memory limits, the container is killed for exceeding 256 MB rather than exhausting host memory and triggering the OOM killer. The on-failure:3 cap prevents infinite restart loops if the app keeps crashing.\n4. Profiles — Optional Services Without Separate Compose Files Profiles let you define services that only start when explicitly requested. Perfect for debugging tools, admin panels, or services you only need occasionally.\nDefine Profiles 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 services: # Main stack — always starts postgres: image: postgres:16-alpine # no profiles = always included api: image: myapi:latest depends_on: - postgres # Debug console — only with --profile debug pgadmin: image: dpage/pgadmin4:latest profiles: - debug depends_on: - postgres environment: PGADMIN_DEFAULT_EMAIL: admin@homelab.local PGADMIN_DEFAULT_PASSWORD_FILE: /run/secrets/pgadmin_pass secrets: - pgadmin_pass # Performance testing tools k6: image: grafana/k6:latest profiles: - loadtest - debug # k6 belongs to both profiles Usage 1 2 3 4 5 6 7 8 9 10 11 # Start normal stack docker compose up -d # Start with debug tools docker compose --profile debug up -d # Start with load testing tools docker compose --profile loadtest up -d # Start everything docker compose --profile \u0026#34;*\u0026#34; up -d Real Homelab Use Case: Monitoring vs Debug 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 services: # Always running prometheus: image: prom/prometheus:latest volumes: - ./prometheus:/etc/prometheus grafana: image: grafana/grafana:latest depends_on: - prometheus # On-demand query explorer postgres-exporter: image: prometheuscommunity/postgres-exporter:latest profiles: - diagnostics depends_on: postgres: condition: service_healthy environment: DATA_SOURCE_NAME: \u0026#34;postgresql://exporter:pass@postgres:5432/appdb?sslmode=disable\u0026#34; # On-demand slow query log viewer pghero: image: ankane/pghero:latest profiles: - diagnostics - debug depends_on: - postgres Start your stack normally. When you need to troubleshoot database performance, run docker compose --profile diagnostics up -d and pgHero + postgres-exporter appear in your monitoring stack. Shut them down with docker compose --profile diagnostics down when done.\n5. Secrets Management — Stop Putting Passwords in Compose Files Docker Compose has built-in secret support since v2.x. Secrets are files mounted into /run/secrets/ inside containers, readable only by the UID the container runs as.\nDefine Secrets 1 2 3 4 5 6 7 8 9 secrets: db_app_password: file: ./secrets/db_app_password.txt api_key: file: ./secrets/api_key.txt tls_cert: file: ./certs/homelab.crt tls_key: file: ./certs/homelab.key Use Secrets in Services 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 services: postgres: image: postgres:16-alpine secrets: - db_app_password environment: POSTGRES_USER: appuser POSTGRES_DB: appdb POSTGRES_PASSWORD_FILE: /run/secrets/db_app_password healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;pg_isready -U appuser -d appdb\u0026#34;] # Official postgres image creates appuser/appdb on first init # and reads POSTGRES_PASSWORD_FILE for appuser\u0026#39;s password. api: image: myapi:latest secrets: - db_app_password - api_key environment: DATABASE_HOST: postgres DATABASE_PORT: \u0026#34;5432\u0026#34; DATABASE_NAME: appdb DATABASE_USER: appuser DATABASE_PASSWORD_FILE: /run/secrets/db_app_password # This requires app support for *_PASSWORD_FILE. # If the app only accepts DATABASE_URL, use an entrypoint wrapper # that reads /run/secrets/db_app_password and exports DATABASE_URL. API_KEY_FILE: /run/secrets/api_key Service-Specific Secret Access Only mount secrets to services that need them:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 secrets: db_app_password: file: ./secrets/db_app_password.txt pgadmin_pass: file: ./secrets/pgadmin_pass.txt grafana_admin_pass: file: ./secrets/grafana_admin.txt services: postgres: secrets: - db_app_password pgadmin: profiles: - debug secrets: - pgadmin_pass # pgadmin cannot read db_app_password or grafana_admin_pass grafana: secrets: - grafana_admin_pass # grafana cannot read db_app_password or pgadmin_pass This is least-privilege in practice. A compromised pgadmin container cannot read your database password or Grafana admin credentials.\nDirectory Layout for Secrets docker/ ├── compose.yml ├── secrets/ │ ├── db_app_password.txt │ ├── api_key.txt │ ├── pgadmin_pass.txt │ └── grafana_admin.txt ├── certs/ │ ├── homelab.crt │ └── homelab.key └── config/ ├── traefik.yml └── prometheus.yml Gitignore secrets but keep the structure:\n# .gitignore secrets/*.txt certs/*.key Commit .gitkeep files to preserve the directory structure:\n1 find secrets certs -type d -exec touch {}/.gitkeep \\; 6. Extension Fields — DRY Your Compose Files Extension fields use the x- prefix and let you define reusable blocks that YAML anchors can reference.\nCommon Labels 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 x-common-labels: \u0026amp;common-labels labels: - \u0026#34;monitoring=prometheus\u0026#34; - \u0026#34;backup=daily\u0026#34; - \u0026#34;owner=homelab\u0026#34; x-logging: \u0026amp;logging logging: driver: \u0026#34;json-file\u0026#34; options: max-size: \u0026#34;10m\u0026#34; max-file: \u0026#34;3\u0026#34; x-base: \u0026amp;base \u0026lt;\u0026lt;: [*common-labels, *logging] services: postgres: image: postgres:16-alpine \u0026lt;\u0026lt;: *base api: image: myapi:latest \u0026lt;\u0026lt;: *base depends_on: postgres: condition: service_healthy Reusable Healthcheck Definitions 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 x-db-healthcheck: \u0026amp;db-healthcheck healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;pg_isready -U postgres\u0026#34;] interval: 10s timeout: 5s retries: 5 start_period: 30s x-http-healthcheck: \u0026amp;http-healthcheck healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;curl\u0026#34;, \u0026#34;-f\u0026#34;, \u0026#34;http://localhost/health\u0026#34;] interval: 15s timeout: 3s retries: 3 start_period: 10s services: postgres: image: postgres:16-alpine \u0026lt;\u0026lt;: *db-healthcheck api: image: myapi:latest \u0026lt;\u0026lt;: *http-healthcheck Network and Volume Definitions 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 x-networks: \u0026amp;networks networks: internal: proxy: services: app: image: myapp:latest \u0026lt;\u0026lt;: *networks # app is on both internal and proxy postgres: image: postgres:16-alpine networks: - internal # postgres is internal-only — not exposed to proxy 7. Multiple Compose Files — Split by Concern Instead of one monster compose file, split by domain and use -f:\n1 2 3 4 5 docker compose \\ -f compose.base.yml \\ -f compose.monitoring.yml \\ -f compose.media.yml \\ up -d base.yml — Shared Infrastructure 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # compose.base.yml services: traefik: image: traefik:v3.3 restart: unless-stopped ports: - \u0026#34;80:80\u0026#34; - \u0026#34;443:443\u0026#34; volumes: - /var/run/docker.sock:/var/run/docker.sock - ./config/traefik:/etc/traefik - ./certs:/certs networks: proxy: external: true name: proxy monitoring.yml — Monitoring Stack 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 # compose.monitoring.yml services: prometheus: image: prom/prometheus:latest restart: unless-stopped volumes: - ./config/prometheus:/etc/prometheus - prometheus_data:/prometheus networks: - proxy - monitoring grafana: image: grafana/grafana:latest restart: unless-stopped volumes: - grafana_data:/var/lib/grafana secrets: - grafana_admin_pass networks: - proxy node-exporter: image: prom/node-exporter:latest restart: unless-stopped networks: - monitoring pid: \u0026#34;host\u0026#34; volumes: prometheus_data: grafana_data: networks: monitoring: internal: true The -f approach means you can update monitoring (docker compose -f compose.monitoring.yml pull \u0026amp;\u0026amp; docker compose -f compose.monitoring.yml up -d) without touching the media stack.\n8. Restart Limits and Backoff Docker\u0026rsquo;s default restart behavior has an exponential backoff: 100ms, 200ms, 400ms, doubling up to a max of 5 minutes between attempts. This is sensible, but you should understand it.\nMonitor restart count:\n1 2 3 4 5 # Check how many times a container has restarted docker inspect --format \u0026#39;{{.RestartCount}}\u0026#39; \u0026#34;\u0026lt;container_name\u0026gt;\u0026#34; # Watch restart events in real time docker events --filter \u0026#39;type=container\u0026#39; --filter \u0026#39;event=restart\u0026#39; Prevent Crash Loops with Deregistration A service that keeps crashing and restarting is still registered in service discovery (Traefik, Consul, etc.). If your reverse proxy sees the container, it will route traffic to it during its brief healthy window.\n1 2 3 4 5 6 7 8 9 services: app: image: myapp:latest healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;curl\u0026#34;, \u0026#34;-f\u0026#34;, \u0026#34;http://localhost:8080/health\u0026#34;] interval: 5s retries: 2 start_period: 5s restart: on-failure:3 After 3 crash-restart cycles, Docker stops trying. The healthcheck ensures Traefik deregisters the container before the healthcheck fails permanently.\n9. Complete Production Compose File Here\u0026rsquo;s a real compose.yml that combines all the patterns:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 # compose.yml — Production homelab stack # Combined base config for restart policy + logging x-base: \u0026amp;base restart: unless-stopped logging: driver: \u0026#34;json-file\u0026#34; options: max-size: \u0026#34;10m\u0026#34; max-file: \u0026#34;3\u0026#34; secrets: db_app_password: file: ./secrets/db_app_password.txt api_key: file: ./secrets/api_key.txt services: postgres: image: postgres:16-alpine \u0026lt;\u0026lt;: *base volumes: - pgdata:/var/lib/postgresql/data secrets: - db_app_password environment: POSTGRES_USER: appuser POSTGRES_DB: appdb POSTGRES_PASSWORD_FILE: /run/secrets/db_app_password healthcheck: test: [\u0026#34;CMD-SHELL\u0026#34;, \u0026#34;pg_isready -U appuser -d appdb\u0026#34;] interval: 10s timeout: 5s retries: 5 start_period: 30s networks: - internal app: image: myapp:latest \u0026lt;\u0026lt;: *base depends_on: postgres: condition: service_healthy secrets: - db_app_password - api_key environment: DATABASE_HOST: postgres DATABASE_PORT: \u0026#34;5432\u0026#34; DATABASE_NAME: appdb DATABASE_USER: appuser DATABASE_PASSWORD_FILE: /run/secrets/db_app_password API_KEY_FILE: /run/secrets/api_key healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;curl\u0026#34;, \u0026#34;-f\u0026#34;, \u0026#34;http://localhost:3000/health\u0026#34;] interval: 15s timeout: 3s retries: 3 start_period: 15s deploy: resources: limits: memory: 256M networks: - internal - proxy traefik: image: traefik:v3.3 \u0026lt;\u0026lt;: *base ports: - \u0026#34;80:80\u0026#34; - \u0026#34;443:443\u0026#34; volumes: - /var/run/docker.sock:/var/run/docker.sock:ro - ./config/traefik:/etc/traefik networks: - proxy depends_on: app: condition: service_healthy volumes: pgdata: networks: internal: internal: true proxy: external: true name: proxy Summary Pattern Key Takeaway Healthchecks Poll a real readiness endpoint, not \u0026ldquo;process is running\u0026rdquo; depends_on condition Use condition: service_healthy for databases and API servers Restart policies unless-stopped for long-lived, on-failure:N with a cap for crashable services Profiles Keep debug/admin tools in --profile debug, not in a separate compose file Secrets Use secrets: file: instead of environment variables for passwords and keys Extension fields x-* blocks with YAML anchors keep repetitive configs DRY Multi-file compose Split by domain (infra, monitoring, media) with -f flag Resource limits Always set deploy.resources.limits.memory to prevent OOM crash loops The difference between a homelab stack that runs for 3 days and one that runs for 3 years is mostly these patterns — startup ordering, failure handling, and secret hygiene. Apply them one at a time, and your compose files will survive reboots, crashes, and the inevitable \u0026ldquo;what happens when this container runs out of memory?\u0026rdquo; question.\n","permalink":"https://blog.gntech.me/posts/2026-05-13-docker-compose-production-patterns/","summary":"Practical Docker Compose patterns for reliable homelab deployments. Healthchecks, conditional dependencies, profiles for optional services, secrets management, extension fields, and restart policies with real compose files.","title":"Docker Compose Production Patterns for Homelabs — Healthchecks, Profiles, Secrets"},{"content":"By default, Docker containers sit on an isolated bridge network (172.17.x.x). You reach them through port mappings like -p 8080:80. This works for web dashboards, but it breaks for anything that needs to be a first-class network citizen.\nServices that need their own LAN IP:\nPi-hole — network-wide DNS. Your router needs to reach it on port 53, and you can\u0026rsquo;t map port 53 through Docker without breaking the host\u0026rsquo;s own DNS resolution. Home Assistant / ESPHome — mDNS discovery for IoT devices. Broadcasts don\u0026rsquo;t cross Docker bridges. Plex / Jellyfin DLNA — UPnP discovery requires being on the same broadcast domain. Print servers, AirPlay receivers, SNMP collectors — any service that relies on broadcast/multicast. MACVLAN and IPVLAN are Docker network drivers that solve this by giving containers their own presence on your physical LAN — their own IP, their own MAC (or shared MAC with IPVLAN), directly addressable from any device on your network.\nThis post covers how to set up both, when to use each, the crucial pitfalls to avoid, and real Compose examples for common services.\nMACVLAN — The Default Choice MACVLAN assigns each container a unique virtual NIC with its own MAC address. Your router sees the container as a separate physical device. DHCP assigns it an IP, or you set one statically.\nHow It Works The Docker host\u0026rsquo;s physical interface (e.g., eth0) acts as a trunk. The kernel creates virtual child interfaces, one per container, each with its own MAC. The switch sees multiple MACs on one port — which is perfectly normal for any switch made in the last 20 years.\nCreating a MACVLAN Network 1 2 3 4 5 6 7 docker network create \\ --driver macvlan \\ --subnet=10.0.0.0/24 \\ --gateway=10.0.0.1 \\ --ip-range=10.0.0.240/28 \\ -o parent=eth0 \\ macvlan-home Breakdown of the flags:\n--subnet — Your LAN subnet. Must match your physical network. --gateway — Your router\u0026rsquo;s IP. --ip-range — A subset Docker can assign from. Use a range outside your DHCP pool to avoid conflicts. If your router hands out 10.0.0.100–10.0.0.239, use 10.0.0.240/28 (240–254). -o parent — The host\u0026rsquo;s physical interface. Run ip link to find yours (eth0, enp3s0, ens18, etc.). Running a Container on MACVLAN 1 2 3 4 5 docker run -d \\ --name pihole \\ --network macvlan-home \\ --ip 10.0.0.53 \\ pihole/pihole:latest With Docker Compose:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 # docker-compose.yml services: pihole: image: pihole/pihole:latest container_name: pihole hostname: pihole environment: TZ: America/Santo_Domingo WEBPASSWORD: \u0026#34;${PIHOLE_PASSWORD}\u0026#34; volumes: - pihole-etc:/etc/pihole - pihole-dnsmasq:/etc/dnsmasq.d restart: unless-stopped networks: macvlan-home: ipv4_address: 10.0.0.53 networks: macvlan-home: external: true # Created with docker network create volumes: pihole-etc: pihole-dnsmasq: That\u0026rsquo;s it. Your Pi-hole is now at 10.0.0.53, directly accessible from any device on your LAN. Set your router\u0026rsquo;s DNS to that IP.\nMultiple Containers on the Same MACVLAN You can attach many containers to the same MACVLAN network, each with a different IP:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 services: pihole: networks: macvlan-home: ipv4_address: 10.0.0.53 homeassistant: image: ghcr.io/home-assistant/home-assistant:stable container_name: homeassistant networks: macvlan-home: ipv4_address: 10.0.0.100 scrypted: image: koush/scrypted:latest container_name: scrypted networks: macvlan-home: ipv4_address: 10.0.0.50 networks: macvlan-home: external: true The Host-to-Container Problem (Important!) MACVLAN has a fundamental limitation: the Docker host cannot communicate with its own MACVLAN containers. Your host at 10.0.0.10 cannot reach 10.0.0.53 — they\u0026rsquo;re on the same physical interface, but MACVLAN\u0026rsquo;s internal routing prevents it.\nThis means:\ndocker exec and health checks still work (they use the internal container namespace), but curl http://10.0.0.53 from the host fails. If Pi-hole is your DNS server, the host itself can\u0026rsquo;t use it. Services running on the host can\u0026rsquo;t reach MACVLAN containers by their LAN IP. The Fix: A Host-Side MACVLAN Bridge Create a secondary MACVLAN interface on the host that bridges the gap:\n1 2 3 4 5 6 7 8 9 10 11 # Create a host-side macvlan interface (same parent, bridge mode) ip link add macvlan-bridge link eth0 type macvlan mode bridge # Assign an unused IP from your ip-range ip addr add 10.0.0.239/32 dev macvlan-bridge # Bring it up ip link set macvlan-bridge up # Add a route for the container range through the bridge ip route add 10.0.0.240/28 dev macvlan-bridge Now 10.0.0.239 is the host\u0026rsquo;s MACVLAN IP, and it can reach all containers in the 10.0.0.240/28 range. Traffic routes through macvlan-bridge directly — it never leaves the host.\nPersisting the Bridge (systemd) Create a systemd unit so this survives reboots:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 # /etc/systemd/system/macvlan-bridge.service [Unit] Description=MACVLAN bridge for Docker host-container communication After=network.target docker.service Requires=network.target [Service] Type=oneshot RemainAfterExit=yes ExecStart=/bin/bash -c \u0026#39; \\ ip link add macvlan-bridge link eth0 type macvlan mode bridge \u0026amp;\u0026amp; \\ ip addr add 10.0.0.239/32 dev macvlan-bridge \u0026amp;\u0026amp; \\ ip link set macvlan-bridge up \u0026amp;\u0026amp; \\ ip route add 10.0.0.240/28 dev macvlan-bridge\u0026#39; ExecStop=/bin/bash -c \u0026#39;ip link delete macvlan-bridge\u0026#39; [Install] WantedBy=multi-user.target 1 2 systemctl daemon-reload systemctl enable --now macvlan-bridge IPVLAN — When MACVLAN Won\u0026rsquo;t Work IPVLAN is similar to MACVLAN but shares the host\u0026rsquo;s MAC address across all containers. The router sees one MAC with multiple IPs.\nIPVLAN L2 Mode Functionally similar to MACVLAN — containers get LAN IPs, traffic bridges at layer 2. The key difference is the shared MAC, which matters when:\nYour hosting provider or network restricts multiple MACs per port (common on VPS/VDS). You have many containers and want to avoid MAC table exhaustion on your switch (rare, but possible with hundreds of containers). Your Proxmox bridge drops MACVLAN traffic (see below). Creation:\n1 2 3 4 5 6 7 8 docker network create \\ --driver ipvlan \\ --subnet=10.0.0.0/24 \\ --gateway=10.0.0.1 \\ --ip-range=10.0.0.240/28 \\ -o parent=eth0 \\ -o ipvlan_mode=l2 \\ ipvlan-home In Compose:\n1 2 3 4 5 6 7 8 9 10 services: pihole: image: pihole/pihole:latest networks: ipvlan-home: ipv4_address: 10.0.0.53 networks: ipvlan-home: external: true The host-to-container problem exists here too. The fix is the same approach — an IPVLAN host interface — but IPVLAN only supports L2 mode for host-side interfaces:\n1 2 3 ip link add ipvlan-bridge link eth0 type ipvlan mode l2 ip addr add 10.0.0.238/32 dev ipvlan-bridge ip link set ipvlan-bridge up IPVLAN L3 Mode — Routed Networking L3 mode puts containers in a completely separate subnet. The host routes between the container subnet and your LAN.\n1 2 3 4 5 6 7 docker network create \\ --driver ipvlan \\ --subnet=10.10.0.0/24 \\ --gateway=10.10.0.1 \\ -o parent=eth0 \\ -o ipvlan_mode=l3 \\ ipvlan-l3 The gateway 10.10.0.1 is actually the host itself. You need to:\nSet up routing on the host (NAT or routing rules) so containers can reach the LAN. Add a static route on your LAN router pointing to the Docker host for the 10.10.0.0/24 subnet. L3 mode eliminates broadcast noise entirely — no ARP, no DHCP broadcasts from containers. This is useful in security-segmented networks or when you want to isolate container traffic at layer 3. For most homelabs, stick with L2.\nProxmox Gotcha: Enable MAC Spoofing If Docker is running inside a Proxmox VM, the Proxmox bridge (vmbr0) drops traffic from unknown MAC addresses by default.\nFix: Enable MAC spoofing on the VM\u0026rsquo;s virtual NIC.\n1 2 # On the Proxmox host qm set \u0026#34;\u0026lt;VMID\u0026gt;\u0026#34; --net0 virtio,bridge=vmbr0,macspoof=1 Alternatively, in the Proxmox UI: select the VM → Hardware → Network Device → Edit → check MAC spoofing.\nWithout this, MACVLAN traffic is silently dropped — containers get IPs but can\u0026rsquo;t communicate. IPVLAN avoids this issue because it shares the VM\u0026rsquo;s assigned MAC address.\nMACVLAN + Regular Bridge Together Many services benefit from having both a MACVLAN interface (for LAN access) and the default Docker bridge (for container-to-container communication):\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 services: homeassistant: image: ghcr.io/home-assistant/home-assistant:stable container_name: homeassistant networks: macvlan-home: ipv4_address: 10.0.0.100 docker-bridge: aliases: - homeassistant networks: macvlan-home: external: true docker-bridge: driver: bridge Home Assistant is reachable at 10.0.0.100 from your LAN for IoT device discovery, while other containers on docker-bridge can reach it by the hostname homeassistant.\nTroubleshooting Container has an IP but can\u0026rsquo;t reach the internet Probable cause: the --gateway IP is wrong, or the parent interface is wrong. Verify:\n1 2 3 4 5 6 7 8 # Confirm the parent interface exists ip link show eth0 # Check the network config docker network inspect macvlan-home # Try pinging the gateway docker exec pihole ping -c 3 10.0.0.1 Container can\u0026rsquo;t reach the internet via IPv6 MACVLAN supports IPv6, but you need to specify the subnet:\n1 2 3 4 5 6 7 8 9 docker network create \\ --driver macvlan \\ --subnet=10.0.0.0/24 \\ --gateway=10.0.0.1 \\ --ip-range=10.0.0.240/28 \\ --subnet=fd00::/64 \\ --gateway=fd00::1 \\ -o parent=eth0 \\ macvlan-home Some routers (especially MikroTik with IPv6 firewall rules) may require the container\u0026rsquo;s DUID or MAC to be registered.\n\u0026ldquo;Network macvlan-home is not manually connectable\u0026rdquo; on docker compose up If you used the short Compose syntax (network: instead of networks:) or forgot external: true:\n1 2 3 networks: macvlan-home: external: true Pre-create the network with docker network create first.\nContainers can reach LAN but not each other MACVLAN containers can talk to each other directly if they\u0026rsquo;re on the same MACVLAN network. If they can\u0026rsquo;t, check your switch\u0026rsquo;s port isolation or private VLAN settings.\nIf using different MACVLAN networks with different parents or subnets, they need a router (your gateway) to communicate.\nMACVLAN vs IPVLAN — Which Should You Use? Factor MACVLAN IPVLAN L2 IPVLAN L3 MAC per container Unique Shared Shared Switch MAC table 1 entry per container 1 entry total 1 entry total Host-to-container Broken, needs bridge Broken, needs bridge Works (host is router) Proxmox VM Needs MAC spoofing Works as-is Works as-is Broadcast handling Passes through Passes through Isolated Performance Slight overhead per packet Lower overhead Lower overhead Best for Services needing unique identity When you can\u0026rsquo;t add more MACs Security isolation My recommendation for a homelab: Start with MACVLAN. It\u0026rsquo;s the most straightforward and works well for 1–20 containers. Switch to IPVLAN L2 only if you hit MAC limits or Proxmox issues. Use IPVLAN L3 when you want to completely container-isolate a subnet.\nComplete Example: Pi-hole + Home Assistant + Scrypted Here\u0026rsquo;s a full working setup with host bridge for DNS:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 # docker-compose.yml services: pihole: image: pihole/pihole:latest container_name: pihole hostname: pihole environment: TZ: America/Santo_Domingo WEBPASSWORD: \u0026#34;${PIHOLE_PASSWORD}\u0026#34; PIHOLE_DNS_: \u0026#34;1.1.1.1;8.8.8.8\u0026#34; volumes: - pihole-etc:/etc/pihole - pihole-dnsmasq:/etc/dnsmasq.d restart: unless-stopped networks: macvlan-home: ipv4_address: 10.0.0.53 homeassistant: image: ghcr.io/home-assistant/home-assistant:stable container_name: homeassistant environment: TZ: America/Santo_Domingo volumes: - ha-config:/config restart: unless-stopped networks: macvlan-home: ipv4_address: 10.0.0.100 internal: aliases: - homeassistant scrypted: image: koush/scrypted:latest container_name: scrypted environment: TZ: America/Santo_Domingo volumes: - scrypted-config:/server/volume restart: unless-stopped ports: - \u0026#34;11080:11080/tcp\u0026#34; networks: macvlan-home: ipv4_address: 10.0.0.50 internal: aliases: - scrypted networks: macvlan-home: external: true internal: driver: bridge volumes: pihole-etc: pihole-dnsmasq: ha-config: scrypted-config: And the host-side bridge systemd unit (from the section above) so the Docker host itself gets DNS from Pi-hole.\nSummary MACVLAN and IPVLAN turn your Docker containers from isolated bridge guests into full LAN citizens. Setup takes five minutes:\nCreate the network with docker network create Attach containers with static IPs Add the host-side bridge so the host can reach them (for MACVLAN) Enable MAC spoofing if running on Proxmox The result is services that behave like physical devices — no port mapping, no NAT, just clean direct addressing. Pi-hole for network DNS, Home Assistant for mDNS device discovery, Plex for DLNA — all with their own IP, just like a Raspberry Pi on the network.\nFor most homelab scenarios, MACVLAN is the right choice. Keep IPVLAN in your back pocket for environments with MAC restrictions or Proxmox VMs that can\u0026rsquo;t enable spoofing.\n","permalink":"https://blog.gntech.me/posts/2026-05-13-docker-macvlan-ipvlan-networking/","summary":"Step-by-step Docker MACVLAN and IPVLAN guide — give containers real LAN IPs for Pi-hole, Home Assistant, Scrypted. Host communication fix, Proxmox gotcha, multi-network Compose examples.","title":"Docker MACVLAN and IPVLAN — Giving Containers Real LAN IPs in Your Homelab"},{"content":"PCIe passthrough is how you take a single Proxmox host and get bare-metal performance in a VM. Pass an NVIDIA GPU to a Jellyfin VM and it transcode as if it owns the card. Pass a dedicated NVMe drive to a TrueNAS VM and it manages SMART, TRIM, and power loss protection directly. Pass a SAS HBA to a file server VM and it sees every disk without Proxmox layering ZFS on top of ZFS.\nThis guide covers PCIe passthrough on Proxmox VE 8.x — from kernel configuration through VM creation to troubleshooting the gotchas that make people give up. Every step includes the exact commands and configs used in a homelab running Proxmox 8.3 with an Intel Alder Lake system.\nWhat You Need Before Starting Passthrough requires hardware that supports it. If your gear can\u0026rsquo;t do this, no amount of software config will make it work.\nCPU: Must support VT-d (Intel) or AMD-Vi (AMD). Most consumer CPUs from 2015 onward have it. Check with:\n1 grep -E \u0026#39;(VMX|SVM)\u0026#39; /proc/cpuinfo Motherboard + BIOS: IOMMU must be enabled in the BIOS, usually under \u0026ldquo;VT-d\u0026rdquo; (Intel) or \u0026ldquo;IOMMU\u0026rdquo; (AMD). Some boards also need \u0026ldquo;Above 4G Decoding\u0026rdquo; and \u0026ldquo;Resizable BAR\u0026rdquo; enabled for GPU passthrough.\nIOMMU grouping: The PCIe device must be in its own IOMMU group or its entire group must be passed through. Consumer platforms (non-HEDT) often lump multiple devices into one group. You can work around this with the ACS override patch, but that\u0026rsquo;s a security trade-off.\nDedicated GPU: If you\u0026rsquo;re passing a GPU, the host needs a separate GPU for its own console — either integrated graphics or a second card. An NVIDIA GTX 1650 for Plex transcoding? The host boots from the iGPU and passes the NVIDIA card to the VM.\nStep 1: Enable IOMMU in the Kernel Edit /etc/default/grub and add the IOMMU flag to GRUB_CMDLINE_LINUX_DEFAULT:\nIntel:\n1 GRUB_CMDLINE_LINUX_DEFAULT=\u0026#34;quiet intel_iommu=on iommu=pt\u0026#34; AMD:\n1 GRUB_CMDLINE_LINUX_DEFAULT=\u0026#34;quiet amd_iommu=on iommu=pt\u0026#34; The iommu=pt flag enables passthrough mode — the kernel only manages devices assigned to the host, not every device behind an IOMMU. This is important: without it, the host spends cycles translating addresses for devices the VM will handle.\nAdd pcie_acs_override=downstream,multifunction if your IOMMU groups are too wide (common on consumer hardware). This splits groups at the downstream port level:\n1 GRUB_CMDLINE_LINUX_DEFAULT=\u0026#34;quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction\u0026#34; Apply and reboot:\n1 2 update-grub reboot Verify IOMMU is active after reboot:\n1 2 3 4 5 dmesg | grep -e DMAR -e IOMMU -e AMD-Vi # You should see \u0026#34;IOMMU enabled\u0026#34; or similar find /sys/kernel/iommu_groups/ -type l | head -10 # A populated directory confirms IOMMU groups exist Step 2: Load VFIO Kernel Modules The VFIO (Virtual Function I/O) drivers take ownership of passed devices from the native kernel driver and present them to the VM.\nCreate /etc/modules-load.d/vfio.conf:\nvfio vfio_iommu_type1 vfio_pci vfio_virqfd Apply immediately:\n1 2 3 4 echo \u0026#34;vfio\u0026#34; \u0026gt;\u0026gt; /etc/modules echo \u0026#34;vfio_iommu_type1\u0026#34; \u0026gt;\u0026gt; /etc/modules echo \u0026#34;vfio_pci\u0026#34; \u0026gt;\u0026gt; /etc/modules update-initramfs -u -k all Step 3: Identify Devices and IOMMU Groups List all PCIe devices with their IOMMU group:\n1 2 3 4 5 6 7 8 9 10 #!/bin/bash # iommu-groups.sh — show device-to-group mapping shopt -s nullglob for g in /sys/kernel/iommu_groups/*; do echo \u0026#34;IOMMU Group $(basename $g):\u0026#34; for d in $g/devices/*; do lspci -nns \u0026#34;${d##*/}\u0026#34; done echo done Run this script and look at the output. A device in its own group is ideal. If you see a group with multiple unrelated devices (e.g., the GPU and a USB controller in the same group), you need the ACS override patch mentioned in Step 1 or can\u0026rsquo;t isolate them.\nFor a GPU passthrough target, note the vendor and device IDs from the lspci -nns output — you\u0026rsquo;ll need them to bind the device to vfio-pci:\n1 2 3 4 5 IOMMU Group 19: 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GA106 [GeForce RTX 3060 Lite Hash Rate] [10de:2503] (rev a1) 01:00.1 Audio device [0403]: NVIDIA Corporation GA106 High Definition Audio Controller [10de:228e] (rev a1) Both functions (GPU core and HDMI audio) are in the same IOMMU group. You must pass both to the VM for the GPU to work.\nStep 4: Bind Devices to vfio-pci Tell the kernel to bind the target PCIe devices to vfio-pci instead of their native drivers. The safest way is via the kernel command line, before any driver loads.\nAdd to GRUB_CMDLINE_LINUX_DEFAULT (same line as your IOMMU flag):\n1 vfio-pci.ids=10de:2503,10de:228e Full GRUB line example for Intel with an RTX 3060:\n1 GRUB_CMDLINE_LINUX_DEFAULT=\u0026#34;quiet intel_iommu=on iommu=pt vfio-pci.ids=10de:2503,10de:228e\u0026#34; Alternatively, use driverctl to rebind at runtime without a reboot:\n1 2 3 apt install driverctl driverctl set-override 0000:01:00.0 vfio-pci driverctl set-override 0000:01:00.1 vfio-pci After reboot (or driver override), verify:\n1 2 lspci -nnk -s 01:00 # Kernel driver in use: vfio-pci ← confirms binding Step 5: VM Configuration for PCIe Passthrough Create the VM with the right machine type and firmware. These settings cannot be changed after creation, so get them right.\nDuring VM creation (via web UI or CLI): Setting Value Why Machine q35 Supports PCIe topology natively BIOS OVMF (UEFI) Required for PCIe; legacy SeaBIOS doesn\u0026rsquo;t work SCSI Controller VirtIO SCSI single Best performance for virtio-blk CPU type host Exposes host CPU features to the VM After creation, add the PCIe device: In the Proxmox web UI, go to the VM → Hardware → Add → PCI Device.\nRaw Device: Select the GPU from the dropdown All Functions: Check — passes both the video and audio controller ROM-Bar: Check (needed for NVIDIA cards to initialize properly) PCI-Express: Check — this is critical. Unchecked means legacy PCI mode, which breaks GPU initialization Primary GPU: Uncheck (only check if this is the VM\u0026rsquo;s boot GPU) The equivalent qm command:\n1 2 3 4 qm set \u0026#34;\u0026lt;VMID\u0026gt;\u0026#34; \\ -hostpci0 01:00,pcie=1,rombar=1 \\ -machine q35 \\ -cpu host Step 6: GPU Passthrough — NVIDIA and AMD Specifics NVIDIA NVIDIA consumer cards (GeForce series) don\u0026rsquo;t officially support passthrough. They work, but the driver checks for the hypervisor and refuses to load with error code 43 in Windows or driver init failure in Linux.\nFix: Add a hidden KVM flag to the VM config. Edit /etc/pve/qemu-server/\u0026lt;VMID\u0026gt;.conf and append to the args line:\nargs: -cpu \u0026#39;host,+kvm_pv_unused\u0026#39; -accel kvm -machine q35,viotlb=on Or create /etc/modprobe.d/kvm.conf on the host:\n1 options kvm ignore_msrs=1 For the VM\u0026rsquo;s guest OS, inject the vendor ID so NVIDIA doesn\u0026rsquo;t detect a hypervisor:\n1 2 # In qemu-server config: args: -cpu \u0026#39;host,hv_vendor_id=1234567890ab,kvm=off\u0026#39; AMD AMD cards generally work without tricks. The one quirk is the reset bug on older Polaris/Vega cards — after stopping the VM, the GPU doesn\u0026rsquo;t reset properly and the VM needs a full host reboot before it works again. Workaround: pass a reset tool like reset-vega.sh or bind the card to the AMDGPU driver on the host before VM shutdown.\nStep 7: NVMe and Storage Device Passthrough For passing an NVMe drive or a full SATA/SAS HBA to a VM, the process is the same as GPU passthrough, but simpler.\nNVMe Direct Passthrough Pass the NVMe controller itself, not a partition on it. The VM gets full TRIM, SMART, and firmware update capability.\n1 2 3 4 5 6 7 8 # Find the NVMe device in IOMMU groups (usually group 12-16) lspci | grep Non-Volatile # Pass it in the Proxmox UI: Hardware → Add → PCI Device # Select the NVMe controller, check PCI-Express and All Functions # CLI: qm set \u0026#34;\u0026lt;VMID\u0026gt;\u0026#34; -hostpci1 02:00.0,pcie=1 HBA Passthrough For storage-heavy VMs (TrueNAS, OMV, or a ZFS file server), pass the entire SAS/SATA controller. The VM owns all disks attached to it and manages SMART, pools, and RAID directly.\n1 2 3 4 5 6 7 # Common HBAs: LSI 9211-8i (SAS2008), LSI 9300 (SAS3008) lspci | grep -i \u0026#39;SAS\\|LSI\\|Avago\\|Broadcom\u0026#39; # The HBA must be in IT mode (initiator target — no RAID firmware) # for the VM to see individual disks qm set \u0026#34;\u0026lt;VMID\u0026gt;\u0026#34; -hostpci2 03:00.0,pcie=1 Why HBA passthrough over disk passthrough: A single PCIe slot gives the VM 8-24 disks directly. Disk-by-disk passthrough requires one device per disk and hits a VM device limit much faster.\nTroubleshooting Common Issues \u0026ldquo;No IOMMU groups found\u0026rdquo; Your hardware doesn\u0026rsquo;t support it, or it\u0026rsquo;s disabled in the BIOS. Check:\n1 2 3 4 5 dmesg | grep -i iommu # Empty output = BIOS disabled or kernel booted without flags cat /proc/cmdline # Verify your GRUB flags are present If flags are present but IOMMU is still missing, your motherboard may not implement VT-d/AMD-Vi despite the CPU supporting it. This is common on budget boards. Update the BIOS and check the manual for the exact VT-d toggle name.\n\u0026ldquo;GPU works in Linux VM but no video output\u0026rdquo; You\u0026rsquo;re likely missing one of:\nPCI-Express checkbox in the VM device config OVMF (UEFI) instead of SeaBIOS ROM-Bar unchecked All functions unchecked (needed for NVIDIA + audio) For headless GPU compute (no monitor attached), add this to the VM\u0026rsquo;s kernel cmdline:\nvideo=efifb:off \u0026ldquo;VM won\u0026rsquo;t start — cannot get device\u0026rdquo; The device is still bound to its native driver. Verify:\n1 2 lspci -nnk -s 01:00 # Should show \u0026#34;Kernel driver in use: vfio-pci\u0026#34; If it shows nvidia, amdgpu, or nvme instead, the vfio-pci binding didn\u0026rsquo;t work. Check your GRUB line for typos in the vendor:device IDs.\n\u0026ldquo;NVIDIA Code 43 in Windows VM\u0026rdquo; Standard NVIDIA anti-hypervisor check. Ensure:\nkvm=off is in the CPU args hv_vendor_id is set to a non-empty string The GPU has UEFI firmware (most post-2016 cards do) CSM/Legacy boot is disabled in the VM\u0026rsquo;s UEFI \u0026ldquo;VM can\u0026rsquo;t see NVMe after passthrough — device not found\u0026rdquo; Some NVMe controllers need the nvme_core.default_ps_max_latency_us=0 kernel parameter passed to the guest VM to prevent power state confusion:\n1 2 # Inside the VM\u0026#39;s GRUB: GRUB_CMDLINE_LINUX_DEFAULT=\u0026#34;quiet nvme_core.default_ps_max_latency_us=0\u0026#34; Performance: Is It Worth It? Passing a device through is always faster than emulation, but the delta varies:\nNVMe passthrough vs virtio-blk: 5-10% latency improvement. Virtio-blk is already very good. NVMe passthrough matters most for heavy database workloads.\nGPU passthrough vs no GPU: Night and day. A VM without a GPU can\u0026rsquo;t transcode video at all. With passthrough, a single RTX 3060 handles 10+ simultaneous 4K transcodes in Jellyfin.\nHBA passthrough vs virtio-scsi: Marginal latency improvement. The real win is ZFS management — the VM does its own ARC, log ZIL, and SMART, all outside Proxmox\u0026rsquo;s storage layer.\nFinal Checklist Before declaring passthrough done, run through this:\nIOMMU enabled in BIOS (VT-d / AMD-Vi) Above 4G Decoding enabled in BIOS GRUB has intel_iommu=on (or amd_iommu=on) + iommu=pt VFIO modules loaded and in initramfs Target device shows Kernel driver in use: vfio-pci VM uses q35 machine type (not i440fx) VM uses OVMF (UEFI) firmware (not SeaBIOS) PCI-Express checkbox checked on PCIe device All Functions checked for devices with multiple sub-devices ROM-Bar checked for GPUs kvm=off set in CPU args for NVIDIA GPUs PCIe passthrough transforms what you can do with a single homelab host. One box runs the file server, a media transcoder, a game streaming VM, and the management plane — each with access to the hardware it needs at native speed.\n","permalink":"https://blog.gntech.me/posts/2026-05-13-proxmox-pcie-passthrough/","summary":"Complete Proxmox VE PCIe passthrough guide — GPU for Jellyfin transcoding, NVMe direct access, SAS HBA for NAS VMs. Kernel configs, vfio-pci, IOMMU groups, and working Proxmox VM settings.","title":"Proxmox PCIe Passthrough — GPU, NVMe, and HBA Setup for Homelabs"},{"content":"If you run more than ten containers on a single Docker host — and in a homelab you almost certainly do — you\u0026rsquo;ve seen the problem. One container hogs CPU compiling something, Plex stutters. A misconfigured service leaks memory, the OOM killer takes down Postgres instead. Database I/O starves every other container on the same drive.\nDocker doesn\u0026rsquo;t enforce limits by default. Every container can consume all available CPU, all RAM, and saturate the disk. That\u0026rsquo;s fine for development. It\u0026rsquo;s a problem for production — and your homelab is production for the services your family depends on.\nThis post covers how to constrain Docker containers by CPU, memory, and block I/O, with real-world examples for common homelab services.\nWhy Bother? The \u0026ldquo;But I Have 16 Cores\u0026rdquo; Trap A lack of limits causes three concrete problems:\nNoisy neighbors. One runaway process impacts every other container. A Python script in your Frigate container goes infinite loop — now your Home Assistant dashboard takes 30 seconds to load.\nOOM roulette. The Linux OOM killer doesn\u0026rsquo;t target the leaky container first. It scores processes by a heuristic (oom_score) that rewards long-running, low-memory containers. Your Postgres or database container often dies before the actual memory hog.\nDisk saturation. A database container doing a backup or reindex can push disk latency from 2 ms to 200 ms, making every other container that touches storage feel sluggish.\nResource limits turn \u0026ldquo;hoping for the best\u0026rdquo; into predictable behavior. Every container gets its fair share, and no container takes down the host.\nMemory Limits — The Most Important Constraint Memory is the resource you absolutely must cap. An unconstrained container that grows until swap fills will lock up the entire host.\n--memory (Hard Limit) The hard limit on physical memory. The container cannot allocate more than this. When it tries, the kernel reclaims memory — or kills the container process.\n1 2 3 4 5 6 7 services: myservice: image: myservice:latest deploy: resources: limits: memory: 512M --memory-reservation (Soft Limit) A soft limit — the kernel tries to keep memory usage at or below this value, but allows bursts up to --memory. This is the right choice for services that are usually quiet but can spike.\n1 2 3 4 5 6 7 8 9 services: webapp: image: nginx:latest deploy: resources: reservations: memory: 128M limits: memory: 512M In plain Docker:\n1 2 3 4 docker run -d --name nginx \\ --memory=\u0026#34;512m\u0026#34; \\ --memory-reservation=\u0026#34;128m\u0026#34; \\ nginx:latest --memory-swap (Swap Control) Controls how much total memory + swap the container can use. By default, a container with --memory=512M gets unlimited swap. That means when RAM fills, the kernel swaps — and performance tanks.\n1 2 3 4 5 # 512M RAM, no swap docker run --memory=\u0026#34;512m\u0026#34; --memory-swap=\u0026#34;512m\u0026#34; myservice # 512M RAM, 256M swap (total 768M) docker run --memory=\u0026#34;512m\u0026#34; --memory-swap=\u0026#34;768m\u0026#34; myservice In Compose:\n1 2 3 4 5 6 7 8 9 services: db: image: postgres:16 deploy: resources: limits: memory: 1G # Compose doesn\u0026#39;t directly support memory-swap in deploy.resources # Use the long syntax with oom_kill_disable or --memory-swap via docker-compose.override.yml Pro tip: Always set --memory-swap equal to --memory to disable swap for critical services. You want OOM kill, not swap death.\nOOM Killer Priority Docker assigns an oom_score_adj of -500 to dockerd and 0 to containers by default. You can bias which container the kernel kills first:\n1 2 3 4 5 # This container dies first under pressure docker run --oom-kill-disable=false --oom-score-adj=1000 my-cache # This container should never be killed (database!) docker run --oom-kill-disable=true postgres:16 Warning: --oom-kill-disable without a --memory limit is dangerous — the kernel can\u0026rsquo;t reclaim memory from the container and the host runs out of RAM.\nCPU Limits — Shares vs Quotas Docker offers two CPU constraint mechanisms with very different behavior.\n--cpus (CPU Quota — Hard Limit) Limits the container to a fraction of a CPU core. A container with --cpus=1.5 gets 150% of one core — equivalent to 1.5 full cores.\n1 docker run -d --name nginx --cpus=\u0026#34;0.5\u0026#34; nginx:latest Under the hood this sets cpu.cfs_quota_us in cgroups. The kernel throttles the container when it exceeds the quota. This is a hard cap — the container cannot use more than this, even if the CPU is idle.\n1 2 3 4 5 6 7 services: heavy-task: image: my-batch-processor deploy: resources: limits: cpus: \u0026#39;0.5\u0026#39; --cpu-shares (Relative Weight — Soft Limit) Relative CPU priority. The default is 1024. A container with 2048 gets ~twice the CPU of a container with 1024 when there\u0026rsquo;s contention. With idle CPU, every container can burst to 100%.\n1 2 3 4 5 6 7 8 9 10 11 12 13 services: database: image: postgres:16 deploy: resources: reservations: cpus: \u0026#39;2\u0026#39; web: image: nginx:latest deploy: resources: reservations: cpus: \u0026#39;1\u0026#39; Key difference: --cpus throttles hard. --cpu-shares only matters during contention. Use --cpus for batch jobs and uploads. Use --cpu-shares for daemons that should be fair but can burst.\n--cpuset-cpus (CPU Pinning) Pins the container to specific physical cores. Useful for:\nReal-time audio/video encoding Consistent cache behavior (L1/L2 stays warm) Avoiding NUMA cross-socket latency 1 2 # Pin to CPU cores 0 and 2 docker run --cpuset-cpus=\u0026#34;0,2\u0026#34; --name transcode jellyfin/jellyfin In Compose:\n1 2 3 4 5 6 7 8 services: jellyfin: image: jellyfin/jellyfin:latest deploy: resources: limits: cpus: \u0026#39;2\u0026#39; # cpuset requires the long resource format or docker-compose.override Block I/O Limits — The Overlooked Bottleneck CPU and memory limits are common. Block I/O limits are rare — and your homelab needs them more than you think.\n--device-read-bps / --device-write-bps Throttle read or write throughput in bytes per second:\n1 2 3 4 5 6 # Limit this container to 50 MB/s write, 100 MB/s read docker run -d \\ --device-write-bps /dev/sda:50mb \\ --device-read-bps /dev/sda:100mb \\ --name torrent-client \\ transmission:latest --device-read-iops / --device-write-iops Throttle by IOPS instead of bandwidth:\n1 2 3 4 5 6 # Limit to 1000 read IOPS, 500 write IOPS docker run -d \\ --device-write-iops /dev/sda:500 \\ --device-read-iops /dev/sda:1000 \\ --name database \\ postgres:16 Compose does not support portable block I/O throttling through the modern Compose Specification. The deploy.resources section can set CPU and memory, but not per-device disk I/O limits.\nUse docker run for block I/O throttling, or apply host-level controls with systemd/cgroups for the Docker service. If you see old examples using device_write_bps or device_read_bps in Compose files, test them on your Compose version first — recent docker compose releases reject those keys.\nReal-World Examples Postgres Database (Memory Critical) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 services: postgres: image: postgres:16-alpine container_name: postgres restart: unless-stopped deploy: resources: limits: memory: 2G reservations: memory: 1G cpus: \u0026#39;1.0\u0026#39; environment: - POSTGRES_PASSWORD=${POSTGRES_PASSWORD} - PGDATA=/var/lib/postgresql/data/pgdata volumes: - pgdata:/var/lib/postgresql/data # Remember: shared_buffers in postgresql.conf should be ~25% of the memory limit # With a 2G limit, set shared_buffers = 512MB volumes: pgdata: No swap (--memory-swap=2G via docker-compose.override if needed). OOM kill enabled (default) — getting killed is better than swapping.\nJellyfin / Plex (CPU Intensive, Memory Hungry) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 services: jellyfin: image: jellyfin/jellyfin:latest container_name: jellyfin deploy: resources: limits: memory: 4G cpus: \u0026#39;2.0\u0026#39; reservations: memory: 2G devices: - /dev/dri:/dev/dri # Intel QuickSync # Pin to physical cores 0-3 for stable transcoding Also add --cpuset-cpus=\u0026quot;0-3\u0026quot; for consistent transcode performance. Without pinning, Linux may bounce the transcode thread between cores, flushing cache and costing 10-20% overhead.\nFrigate NVR (I/O Heavy, Memory Constrained) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 services: frigate: image: ghcr.io/blakeblackshear/frigate:stable container_name: frigate deploy: resources: limits: memory: 2G cpus: \u0026#39;1.0\u0026#39; reservations: memory: 1G # Throttle disk writes for recordings # Add to docker-compose.override.yml: # device_write_bps: # - /dev/sda:30mb volumes: - /etc/localtime:/etc/localtime:ro - ${FRIGATE_CONFIG:-./config}:/config - ${FRIGATE_STORAGE:-./media}:/media/frigate - type: tmpfs target: /tmp/cache tmpfs: size: 256000000 The tmpfs for /tmp/cache reduces SD card / HDD writes significantly.\nTransmission / qBittorrent (Disk Killer) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 services: transmission: image: lscr.io/linuxserver/transmission:latest container_name: transmission deploy: resources: limits: memory: 512M cpus: \u0026#39;0.5\u0026#39; reservations: memory: 256M # All your docker-compose.override.yml device limits go here # device_write_bps: /dev/sda:20mb # device_read_bps: /dev/sda:50mb Throttling torrent client disk I/O is the single biggest quality-of-life improvement for a shared-storage homelab. Without it, a busy torrent client can saturate the disk for minutes.\nMonitoring: What\u0026rsquo;s Actually Happening docker stats — Quick Live View 1 2 docker stats --no-stream --format \\ \u0026#34;table {{.Name}}\\t{{.CPUPerc}}\\t{{.MemUsage}}\\t{{.MemPerc}}\\t{{.NetIO}}\u0026#34; docker stats with All Metrics 1 2 docker stats --no-stream --format \\ \u0026#34;table {{.Name}}\\t{{.CPUPerc}}\\t{{.MemUsage}}\\t{{.MemPerc}}\\t{{.PIDs}}\u0026#34; cgroup v2 Direct Inspection On modern distros (Debian 12+, Ubuntu 22.04+, Fedora 37+):\n1 2 3 4 5 6 7 8 # Memory usage for a container cat /sys/fs/cgroup/memory/system.slice/docker-\u0026#34;\u0026lt;container-id\u0026gt;\u0026#34;.scope/memory.current # CPU usage (microseconds) cat /sys/fs/cgroup/cpu.system.slice/docker-\u0026#34;\u0026lt;container-id\u0026gt;\u0026#34;.scope/cpu.stat # IO stats cat /sys/fs/cgroup/io.system.slice/docker-\u0026#34;\u0026lt;container-id\u0026gt;\u0026#34;.scope/io.stat For cgroup v2, Docker also provides docker inspect:\n1 2 docker inspect \u0026#34;\u0026lt;container\u0026gt;\u0026#34; --format \u0026#39;{{.HostConfig.Memory}}\u0026#39; docker inspect \u0026#34;\u0026lt;container\u0026gt;\u0026#34; --format \u0026#39;{{.HostConfig.NanoCpus}}\u0026#39; cgroup v2 Notes Debian 12, Ubuntu 22.04+, and all modern kernels default to cgroup v2. Docker 20.10+ supports it natively, but there are differences:\n--memory-swap behavior changed. In cgroup v1, --memory-swap=-1 meant unlimited swap. In cgroup v2, swap is 0 by default unless the kernel supports memory.swap.max. --oom-kill-disable works differently. In cgroup v2, disabling OOM kill means the container is paused when it hits the memory limit (tasks get stuck in D state) instead of killed. Most services just freeze, which can be worse than an OOM kill. device_write_bps needs the right device major:minor. Use ls -l /dev/sda or stat /dev/sda to find the device numbers. Check your cgroup version:\n1 2 3 stat -fc %T /sys/fs/cgroup/ # cgroup2fs → cgroup v2 # tmpfs → cgroup v1 Putting It All Together — A Safe Default for Any Container Here\u0026rsquo;s the template I use for every new container in my homelab:\n1 2 3 4 5 6 7 8 9 10 11 12 13 services: new-service: image: some/image:latest container_name: new-service restart: unless-stopped deploy: resources: limits: cpus: \u0026#39;0.5\u0026#39; memory: 256M reservations: cpus: \u0026#39;0.25\u0026#39; memory: 128M Start conservative. Run for a week with docker stats logging. Bump reservations based on real usage, not guesswork. Document why each limit exists in your docker-compose.yml comments — future you will thank present you.\nWhen to Skip Limits Resource limits aren\u0026rsquo;t free. Setting CPU quotas too low causes throttling and increased latency (CFS quota mechanism introduces scheduling delays). Setting memory limits too low triggers constant reclamation and swap thrashing.\nSkip limits when:\nThe container is the only service on the host The container does bursty compute that you want to finish fast (cron jobs, batch transcodes) You\u0026rsquo;re benchmarking or load testing The container is a monitoring agent that needs minimal overhead In every other case: set limits. Your homelab will be more reliable, your other services will thank you, and you won\u0026rsquo;t wake up at 2 AM to a host that\u0026rsquo;s OOM-frozen because Frigate had a memory leak.\n","permalink":"https://blog.gntech.me/posts/2026-05-13-docker-container-resource-limits/","summary":"Practical Docker container resource limits guide for homelabs — CPU quota/pinning, memory hard/soft limits, swap control, block I/O throttling, cgroup v2 differences, and Docker Compose examples.","title":"Docker Container Resource Limits — CPU, Memory, and I/O Constraints for Homelabs"},{"content":"The Linux kernel ships with conservative defaults. They\u0026rsquo;re chosen to work reliably on a Raspberry Pi, a 2013 laptop with 4 GB of RAM, and a 32-core server in the same kernel tree. That means your Proxmox host, Docker server, or storage box is leaving performance on the floor.\nThis post covers the sysctl changes that actually matter for a homelab — network throughput, connection handling, memory limits, filesystem notifiers, and Docker-specific settings. Every parameter includes what it does and the safe range, not cargo-culted Stack Overflow snippets.\nBefore You Tune: Save Your Current Baseline You need numbers before and after. Don\u0026rsquo;t skip this.\n1 2 3 4 5 6 7 8 9 10 11 # Save all current sysctl values in the relevant namespaces sysctl -a 2\u0026gt;/dev/null | grep -E \u0026#39;^(net|vm|kernel|fs)\u0026#39; | sort \u0026gt; /root/sysctl-baseline-$(date +%F).txt # Network interface stats ip -s link show | tee -a /root/sysctl-baseline-$(date +%F).txt # Connection stats ss -s | tee -a /root/sysctl-baseline-$(date +%F).txt # Memory info free -h | tee -a /root/sysctl-baseline-$(date +%F).txt Now you can compare before/after. \u0026ldquo;Feels faster\u0026rdquo; isn\u0026rsquo;t a metric.\nThe Config File Layout All changes go into /etc/sysctl.d/ — never edit /etc/sysctl.conf directly. Debian/Ubuntu loads *.conf files from this directory in alphabetical order.\n1 2 3 4 5 6 7 8 # Create the config file sudo touch /etc/sysctl.d/99-homelab-tune.conf # Apply after edits sudo sysctl -p /etc/sysctl.d/99-homelab-tune.conf # Verify a specific value sysctl net.ipv4.tcp_congestion_control Use sysctl -w parameter=value for temporary runtime testing before making permanent changes.\n1. Network Throughput — TCP Buffer Tuning This is the single biggest performance win. Default TCP buffers are usually 4 MB max. On a 10 Gbps link with even 10 ms latency, that limits throughput under 3 Gbps.\nThe bandwidth-delay product formula: throughput × RTT = required buffer. 10 Gbps × 10 ms = 100 Mbit, or about 12.5 MB of buffer.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 # /etc/sysctl.d/99-homelab-tune.conf # Maximum socket receive/send buffer size # 128 MB for 10G links, 64 MB for 1G is fine net.core.rmem_max = 134217728 net.core.wmem_max = 134217728 # TCP receive buffer: min(4K), default(87K), max(128M) net.ipv4.tcp_rmem = 4096 87380 134217728 # TCP send buffer: min(4K), default(64K), max(128M) net.ipv4.tcp_wmem = 4096 65536 134217728 # Enable auto-tuning of receive buffer net.ipv4.tcp_moderate_rcvbuf = 1 For a 1 Gbps homelab (most common), 64 MB max is enough:\n1 2 3 4 net.core.rmem_max = 67108864 net.core.wmem_max = 67108864 net.ipv4.tcp_rmem = 4096 87380 67108864 net.ipv4.tcp_wmem = 4096 65536 67108864 2. BBR Congestion Control BBR (Bottleneck Bandwidth and RTT) is Google\u0026rsquo;s TCP congestion algorithm. On links with any packet loss (which includes basically all real-world links), BBR dramatically outperforms CUBIC.\nAvailable since kernel 4.9. Check yours:\n1 2 sysctl net.ipv4.tcp_available_congestion_control # Look for \u0026#34;bbr\u0026#34; in the output Enable it:\n1 2 net.ipv4.tcp_congestion_control = bbr net.core.default_qdisc = fq BBR requires fq (Fair Queue) as the packet scheduler. Without it, BBR falls back to pfifo_fast and performance degrades.\nBenchmark on a real 1 Gbps link with 0.5% packet loss:\nAlgorithm Throughput Latency Under Load CUBIC ~480 Mbps 45-120 ms jitter BBR ~860 Mbps 5-15 ms stable Test it yourself:\n1 2 3 4 5 # On receiver iperf3 -s # On sender iperf3 -c \u0026#34;\u0026lt;RECEIVER_IP\u0026gt;\u0026#34; -t 30 -P 4 3. Connection Handling — Backlog and TIME_WAIT High connection rates (reverse proxies, Docker Swarm, monitoring scrapers) hit the connection backlog long before CPU or bandwidth.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 # Max listen backlog (default: 128, useless for anything modern) net.core.somaxconn = 65535 # SYN backlog (default: 1024) net.ipv4.tcp_max_syn_backlog = 65535 # Device backlog queue (default: 1000) net.core.netdev_max_backlog = 250000 # Reuse TIME_WAIT sockets for new connections # Critical for reverse proxies and load balancers net.ipv4.tcp_tw_reuse = 1 # Accelerate FIN_WAIT timeout (default: 60s) # Don\u0026#39;t go below 15 on lossy networks net.ipv4.tcp_fin_timeout = 15 # Max TIME_WAIT sockets before kernel starts dropping net.ipv4.tcp_max_tw_buckets = 2000000 tcp_tw_reuse only applies to outbound connections. It\u0026rsquo;s safe for servers that make many outbound connections (proxies, web scrapers, monitoring agents). It won\u0026rsquo;t help with inbound connections to the same server — those need SO_REUSEADDR at the application level.\nDo not use tcp_tw_recycle — it was removed in Linux 4.12 and was never safe behind NAT or load balancers.\n4. Ephemeral Port Range Outbound connections eat ephemeral ports. Default range is 32768-60999 (~28K ports). A busy Traefik or Nginx proxy can exhaust this in minutes.\n1 2 # Expand ephemeral port range — full non-reserved range net.ipv4.ip_local_port_range = 1024 65535 This gives you ~64K ports per destination (IP, port) pair. Monitor exhaustion:\n1 2 3 ss -s | grep -i timewait # If TIME-WAIT count exceeds 45K on a busy proxy, you\u0026#39;re close to # exhausting the old default range 5. Keepalive for Long-Lived Connections Database connections, WebSocket clients, and persistent TCP tunnels go through firewalls that silently drop idle connections. Default keepalive waits 2 hours before probing — an eternity.\n1 2 3 4 5 6 7 8 # Start keepalive probes after 60s idle (default: 7200) net.ipv4.tcp_keepalive_time = 60 # Probe interval (default: 75s) net.ipv4.tcp_keepalive_intvl = 10 # Kill after 6 failed probes (default: 9) net.ipv4.tcp_keepalive_probes = 6 This detects a dead connection in 60 + (10 × 6) = 120 seconds, versus the default 7200 + (75 × 9) = 7875 seconds (~2.2 hours).\n6. Memory Management — OOM and Swappiness The kernel\u0026rsquo;s out-of-memory killer doesn\u0026rsquo;t always pick the right victim. Set it to always kill the process that\u0026rsquo;s using the most memory (the one that caused the OOM), not the one scoring lowest on the kernel\u0026rsquo;s heuristic.\n1 2 3 4 5 # Preferred OOM behavior: kill the offending process vm.oom_kill_allocating_task = 1 # Don\u0026#39;t panic on OOM — let the killer do its job vm.panic_on_oom = 0 Swappiness: the kernel swaps when it thinks it\u0026rsquo;s a good idea. On a homelab server with plenty of RAM, you want it to think twice.\n1 2 3 # Only swap when absolutely necessary (default: 60) # 10 = aggressive RAM retention, good for Proxmox/Docker hosts vm.swappiness = 10 For a NAS or ZFS box, consider even lower (1-5). ZFS uses its own ARC cache in RAM, and you don\u0026rsquo;t want the kernel to swap out ZFS metadata pages.\n1 2 # Reduce pressure on page cache during memory pressure vm.vfs_cache_pressure = 50 7. Dirty Page Writeback The kernel batches dirty pages before writing to disk. Default settings favor write aggregation for spinning rust. On SSD/NVMe storage, you can tune for lower latency.\n1 2 3 4 5 6 7 8 9 # Dirty data before triggering writeback (% of RAM, default: 20) vm.dirty_ratio = 30 # Background writeback starts at this % (default: 10) vm.dirty_background_ratio = 5 # Flush dirty pages every 5 seconds (default: 5) # Lower = less data loss on crash, more write I/O vm.dirty_expire_centisecs = 500 SSD/NVMe host: keep these ratios lower. Background at 3%, max at 15%. ZFS host: ZFS manages its own writeback. Keep kernel ratio low (dirty_background_ratio=2, dirty_ratio=10) so ZFS gets first look. RAID with battery backup: can push higher ratios but the homelab benefit is marginal. 8. Filesystem — inotify Watchers Every Docker container, file sync tool (Syncthing, rclone), and log tailer consumes inotify watches. Default limit is 8192 — you\u0026rsquo;ll hit it with more than a few Docker containers.\n1 2 3 4 5 6 7 8 9 # Max inotify watches per user (default: 8192) # 524288 handles ~100 Docker containers comfortably fs.inotify.max_user_watches = 524288 # Max inotify instances per user (default: 128) fs.inotify.max_user_instances = 1024 # Max queued events (default: 16384) fs.inotify.max_queued_events = 32768 When you hit the limit, apps throw \u0026ldquo;Too many open files\u0026rdquo; or \u0026ldquo;inotify watch limit reached\u0026rdquo; errors. Check current usage:\n1 2 3 4 5 6 7 8 # Count inotify watches currently in use find /proc/*/fd -lname \u0026#39;anon_inode:inotify\u0026#39; 2\u0026gt;/dev/null | wc -l # Per-process breakdown for pid in /proc/[0-9]*; do count=$(find \u0026#34;$pid/fd\u0026#34; -lname \u0026#39;anon_inode:inotify\u0026#39; 2\u0026gt;/dev/null | wc -l) [ \u0026#34;$count\u0026#34; -gt 0 ] \u0026amp;\u0026amp; echo \u0026#34;$(cat $pid/comm 2\u0026gt;/dev/null) ($pid): $count\u0026#34; done 9. File Descriptor Limits systemd sets per-service limits, but the system-wide kernel limit often stays low.\n1 2 # System-wide file descriptor max fs.file-max = 2097152 Also set user limits via /etc/security/limits.conf or systemd\u0026rsquo;s DefaultLimitNOFILE=2097152 in /etc/systemd/system.conf.\nCheck usage:\n1 2 3 4 # Current open file count cat /proc/sys/fs/file-nr # Output: 4032 0 2097152 # used unused max 10. Docker-Specific sysctls Docker containers inherit sysctls from the host for most parameters. Some are namespace-aware and can be set per-container. These host-level settings improve container performance:\n1 2 3 4 5 6 7 8 9 10 11 12 13 # Enable IPv4 forwarding (required for Docker bridge networking) net.ipv4.ip_forward = 1 # Enable IPv6 forwarding if you use it net.ipv6.conf.all.forwarding = 1 # Reduce TIME_WAIT on the Docker bridge — busy Traefik containers # spawn short-lived connections to backend containers constantly net.ipv4.tcp_tw_reuse = 1 # Increase the default backlog for Docker\u0026#39;s port publishing # Docker uses iptables DNAT; these help with connect() bursts net.core.somaxconn = 65535 For per-container sysctl settings (Linux kernel 5.8+ with CONFIG_NET_NS), set them in docker-compose or docker run:\n1 2 3 4 5 6 7 8 # docker-compose.yml services: traefik: image: traefik:v3.3 sysctls: - net.core.somaxconn=65535 - net.ipv4.tcp_tw_reuse=1 - net.ipv4.ip_local_port_range=1024 65535 Check if your kernel supports per-container sysctls:\n1 2 docker run --rm alpine sysctl net.core.somaxconn # If this succeeds, you can set sysctls per container 11. Kernel Same-page Merging (KSM) for Proxmox If this is a Proxmox host, KSM deduplicates identical memory pages across VMs and containers. It reduces memory usage but costs CPU.\n1 2 3 4 5 6 7 8 9 # Enable KSM vm.merge_across_nodes = 1 # Aggressiveness: how many pages to scan per pass # default: 100, higher = more dedup at more CPU cost # Set to 0 to disable, 1000 for aggressive dedup /sys/kernel/mm/ksm/run = 1 /sys/kernel/mm/ksm/pages_to_scan = 1000 /sys/kernel/mm/ksm/sleep_millisecs = 20 These are not sysctl values (they\u0026rsquo;re sysfs files), so add them to a startup script or systemd service that runs after boot.\nCheck KSM savings:\n1 2 3 4 cat /sys/kernel/mm/ksm/pages_shared # pages shared cat /sys/kernel/mm/ksm/pages_sharing # references to shared pages # Multiply pages_shared × page size (4096) ÷ 1024 ÷ 1024 = MB saved echo $(($(cat /sys/kernel/mm/ksm/pages_sharing) * 4096 / 1024 / 1024)) MB saved 12. NUMA Tuning (Multi-Socket Hosts) On dual-socket servers (common in homelabs from the used server market), NUMA awareness matters. The default zone reclaim mode can cause performance regressions when one NUMA node runs out of memory.\n1 2 # Try to keep memory allocations on the local NUMA node vm.zone_reclaim_mode = 1 zone_reclaim_mode = 1 means \u0026ldquo;prefer reclaiming memory on the local node before allocating from a remote node.\u0026rdquo; Value 0 allows remote node allocations freely (lower latency for local, but more bandwidth overall).\nFor NUMA-unaware applications (many Docker containers), leave at 0:\n1 vm.zone_reclaim_mode = 0 Check your NUMA topology:\n1 2 lscpu | grep -i numa numactl --hardware The Complete Homelab Tuning File Here\u0026rsquo;s the full config for a Proxmox or Docker host with 32-64 GB RAM and a 1-10 Gbps network:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 # /etc/sysctl.d/99-homelab-tune.conf # === NETWORK === # TCP buffers (1G: 64M max, 10G: 128M max) net.core.rmem_max = 134217728 net.core.wmem_max = 134217728 net.ipv4.tcp_rmem = 4096 87380 134217728 net.ipv4.tcp_wmem = 4096 65536 134217728 net.ipv4.tcp_moderate_rcvbuf = 1 # BBR congestion control net.ipv4.tcp_congestion_control = bbr net.core.default_qdisc = fq # Connection handling net.core.somaxconn = 65535 net.ipv4.tcp_max_syn_backlog = 65535 net.core.netdev_max_backlog = 250000 # TIME_WAIT optimization net.ipv4.tcp_tw_reuse = 1 net.ipv4.tcp_fin_timeout = 15 net.ipv4.tcp_max_tw_buckets = 2000000 # Ephemeral ports net.ipv4.ip_local_port_range = 1024 65535 # Keepalive net.ipv4.tcp_keepalive_time = 60 net.ipv4.tcp_keepalive_intvl = 10 net.ipv4.tcp_keepalive_probes = 6 # IP forwarding net.ipv4.ip_forward = 1 # === MEMORY === vm.swappiness = 10 vm.vfs_cache_pressure = 50 vm.dirty_ratio = 30 vm.dirty_background_ratio = 5 vm.dirty_expire_centisecs = 500 vm.oom_kill_allocating_task = 1 vm.panic_on_oom = 0 # === FILESYSTEM === fs.inotify.max_user_watches = 524288 fs.inotify.max_user_instances = 1024 fs.inotify.max_queued_events = 32768 fs.file-max = 2097152 Apply and verify:\n1 sudo sysctl -p /etc/sysctl.d/99-homelab-tune.conf Testing Your Changes Run specific benchmarks before and after tuning:\nNetwork throughput:\n1 iperf3 -c \u0026#34;\u0026lt;SERVER_IP\u0026gt;\u0026#34; -t 30 -P 4 Connection rate (use a dedicated test host):\n1 2 3 4 5 # Install wrk on the test machine sudo apt install wrk # Test Traefik or Nginx wrk -t 4 -c 200 -d 30s http://your-server/ Latency:\n1 2 3 4 # TCP latency test with netperf netserver -p 16604 # On client: netperf -H \u0026#34;\u0026lt;SERVER_IP\u0026gt;\u0026#34; -p 16604 -t TCP_RR -l 30 Inotify limit test (run Docker compose up on a stack with 20+ services):\n1 docker compose up -d # should complete without inotify errors What NOT to Tune Some sysctls get recommended in older blog posts but don\u0026rsquo;t touch them:\nnet.ipv4.tcp_sack — disabling SACK reduces throughput on any link with packet loss. Leave it enabled (default: 1). net.ipv4.tcp_congestion_control = htcp — H-TCP was useful in the 2000s. Use BBR or CUBIC instead. net.ipv4.tcp_tw_recycle — removed in kernel 4.12, doesn\u0026rsquo;t exist on modern kernels. kernel.sched_* — process scheduler tuning is workload-specific and easy to get wrong. Leave defaults unless you\u0026rsquo;ve measured a specific scheduler bottleneck. vm.min_free_kbytes — lowering this to reclaim memory causes kernel allocation failures. Leave it at the kernel-computed default. Summary Area Key Setting Default Tuned Why TCP buffers rmem_max / wmem_max ~4 MB 64-128 MB Bandwidth-delay product Congestion tcp_congestion_control cubic bbr Better throughput with loss Connections somaxconn 128 65535 Accept new connections faster Ephemeral ports ip_local_port_range 32768-60999 1024-65535 More outbound connections Keepalive tcp_keepalive_time 7200 60 Detect dead connections fast inotify max_user_watches 8192 524288 Docker containers need this Swappiness swappiness 60 10 Keep cache in RAM These settings have been running on production Proxmox, Docker, and bare-metal Linux hosts without issues. As always: test on your own hardware, one parameter at a time, and keep your baseline data.\n","permalink":"https://blog.gntech.me/posts/2026-05-13-linux-sysctl-kernel-tuning/","summary":"Complete Linux sysctl kernel tuning guide for homelab servers. Network buffers, BBR congestion control, OOM management, fs.inotify limits, and Docker-specific sysctl configs.","title":"Linux Sysctl Kernel Tuning for Homelabs — Network, Memory, and Docker Performance"},{"content":"VLANs separate layer-2 networks. Once a router has interfaces in those networks, firewall policy decides which routed connections are allowed. NAT does not grant permission through the firewall.\nThis guide describes a staged RouterOS 7 lab workflow. It replaces the earlier universal paste-in script with explicit configuration fields and verification gates. Adapt each stage to the installed release and current configuration.\nVerification status Area Evidence and limits Documentation review Official MikroTik references reviewed on October 31, 2026. Examples Proposed lab settings, not a captured RB5009 or home-router deployment. Commands Read-only RouterOS inventory commands; no live parser or device execution performed. Runtime VLAN filtering, DHCP, firewall isolation, recovery, and WiFi mapping remain untested for this revision. 1. Preserve access and inventory the router Have a working console or separate recovery path before touching the bridge carrying management traffic. Save a configuration export and an appropriate protected backup outside the router, and document how to recover them. Exports can omit secrets and are not a complete substitute for a recovery plan.\nUse RouterOS configuration management and Safe Mode. Make small changes and verify them before committing. Safe Mode has limits; it does not replace console access or backups.\nRun in the RouterOS terminal:\n/system resource print /system package print /interface print /interface bridge print detail /interface bridge port print detail /interface bridge vlan print detail /interface vlan print detail /ip address print /interface list member print /ip firewall filter print detail /ip firewall nat print detail Record the current management path, WAN interface, bridge membership, rule order, and interface lists. Do not delete the existing firewall or add ports already assigned to a bridge.\n2. Write the port and subnet plan Example for a small lab, with management on tagged VLAN 99:\nVLAN Purpose Example gateway Access expectation 99 Management 10.0.99.1/24 Administrative devices only 20 Servers 10.0.20.1/24 Explicit application access 40 IoT 10.0.40.1/24 Only required destinations 70 Guest 10.0.70.1/24 Internet access, no internal access The guest VLAN is an example, not a claim about an existing home deployment.\nPort Role VLAN handling Actual WAN interface ISP uplink Outside the LAN bridge in this design ether2 Lab switch trunk Tagged 20, 40, 70, 99 ether3 Dedicated management access port Untagged 99, PVID 99 ether4 IoT test access port Untagged 40, PVID 40 ether5 Guest test access port Untagged 70, PVID 70 Confirm that these names exist on your device. A PPPoE WAN may be a logical interface, so do not assume the WAN is always ether1.\nFor the Proxmox uplink, match the switch\u0026rsquo;s actual egress behavior: tagged management needs a corresponding host VLAN interface; untagged management uses a different host design. See the Proxmox networking guide.\n3. Configure the bridge in stages In WinBox or WebFig, use Bridge → Ports and Bridge → VLANs. On a disposable lab router, prepare membership before enabling VLAN filtering; on an existing router, review its active configuration rather than creating a parallel bridge blindly.\nFor each access port, set its PVID, untagged membership, ingress filtering, and an admission policy appropriate for untagged access traffic. For the fully tagged trunk, allow tagged frames and ingress filtering with the required VLAN memberships.\nFor each routed VLAN, include the bridge CPU port as a tagged member, along with the trunk. Include only that VLAN\u0026rsquo;s intended access ports as untagged members. A VLAN switched entirely between ports does not need router CPU access merely to be switched.\nCreate the management VLAN interface and address, prepare router-input permissions, and test the intended management path before retiring the old path. Enable VLAN filtering only when the VLAN table and recovery access are ready.\nInspect dynamic as well as static entries. Removing a static VLAN 1 entry alone does not remove dynamic membership caused by PVIDs. Review actual admission and membership settings instead of treating a VLAN number as a security control.\nFollow MikroTik\u0026rsquo;s bridge VLAN table guide for the detailed fields and CPU-port behavior.\n4. Add routed interfaces and DHCP deliberately For each subnet the router serves, create a VLAN interface on the LAN bridge, assign its gateway address, and configure a matching DHCP pool, server, and network record.\nIoT test field Example VLAN interface vlan40-iot on the LAN bridge, VLAN ID 40 Router address 10.0.40.1/24 Pool 10.0.40.100–10.0.40.199 DHCP server interface vlan40-iot DHCP network 10.0.40.0/24 Gateway option 10.0.40.1 DNS option An intentionally reachable resolver Verify the DHCP server is enabled and valid. Keep static addresses outside dynamic pools or use reservations; a long lease is not a static assignment. If clients use the router as DNS, configure the resolver and permit the required router-input traffic. If they use another subnet\u0026rsquo;s resolver, allow that exact forwarding path.\nRepeat for other VLANs after the first client gets the correct lease. See RouterOS DHCP.\n5. Define input and forwarding separately Input controls traffic addressed to the router, including administration and router-hosted DNS. Forward controls routed client traffic. Protect both; isolating subnets in forward does not protect the router\u0026rsquo;s own services.\nWrite the intended policy before translating it into rules:\nNew connection Intended result Selected management device → router administration Allow required services Guest/IoT → router administration Deny Client → chosen DNS/NTP service Allow exact required path Selected client → selected server application Allow exact addresses and ports Guest → internal networks, VPNs, or other local destinations Deny Guest → intended WAN Allow subject to policy Other inter-VLAN traffic Deny Review existing rules before adding more. Establish state handling, required management and infrastructure access, specific forwarding exceptions, and explicit terminal drops in the relevant chains. Keep invalid-packet handling and WAN protection consistent with the existing design. Restrict source/interface matches and review address lists, interface lists, VPNs, and destination NAT exceptions.\nDo not append a rule below an earlier matching accept or drop and assume it is effective. NAT masquerading changes addresses; it does not override a forward drop. The old expression dst-address=!0.0.0.0/0 does not identify private networks: the negated prefix covers no IPv4 destinations.\nFastTrack and existing tracked connections can affect policy testing. Use fresh connections and review FastTrack against the intended policy. IPv6 requires its own firewall review and tests; IPv4 rules alone do not establish dual-stack isolation.\nConsult the filter reference and packet-flow documentation. This policy table is not a complete replacement firewall to paste into a live router.\n6. Test from real endpoints Use a laptop or disposable guest actually connected to each test VLAN. A router-originated traceroute exercises a different path from a client\u0026rsquo;s forwarded connection.\n/interface bridge vlan print detail /interface bridge port print /ip dhcp-server print /ip dhcp-server lease print /ip firewall filter print stats /ip route print Verify the assigned subnet, gateway, DNS, and an allowed application connection. Test denied connections to both another VLAN and the router\u0026rsquo;s management service. Compare firewall counter changes while making new connections. A failed ping alone does not prove isolation.\nCheck that guest internet access works while internal and VPN destinations remain blocked. For wireless guest clients, test same-VLAN client isolation separately: routed firewall rules do not automatically block layer-2 peers.\nIf access is lost, use the recovery path and known-good configuration. Confirm the restored management connection before making another change.\nWiFi and performance are device-specific Finish the wired test before mapping SSIDs. Legacy /caps-man and newer /interface wifi CAPsMAN belong to different stacks. Choose the documentation matching the AP\u0026rsquo;s installed package and forwarding mode; do not mix their commands. Use unique credentials rather than shared example passwords. See WiFi CAPsMAN.\nHardware offload depends on switch chip, software release, and configuration. Check active port flags and measured behavior; do not promise a fixed inter-VLAN throughput. Review basic VLAN switching for the relevant hardware.\nContinue reading Proxmox bridges and VLANs — Match the hypervisor uplink to this port plan. Intel AMT Serial over LAN — Prepare host console recovery. Monitoring foundations — Add visibility after verifying the network. ","permalink":"https://blog.gntech.me/posts/2026-05-12-mikrotik-vlan-segmentation/","summary":"\u003cp\u003eVLANs separate layer-2 networks. Once a router has interfaces in those networks, firewall policy decides which routed connections are allowed. NAT does not grant permission through the firewall.\u003c/p\u003e\n\u003cp\u003eThis guide describes a staged RouterOS 7 lab workflow. It replaces the earlier universal paste-in script with explicit configuration fields and verification gates. Adapt each stage to the installed release and current configuration.\u003c/p\u003e\n\u003ch2 id=\"verification-status\"\u003eVerification status\u003c/h2\u003e\n\u003ctable\u003e\n  \u003cthead\u003e\n      \u003ctr\u003e\n          \u003cth\u003eArea\u003c/th\u003e\n          \u003cth\u003eEvidence and limits\u003c/th\u003e\n      \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eDocumentation review\u003c/td\u003e\n          \u003ctd\u003eOfficial MikroTik references reviewed on October 31, 2026.\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eExamples\u003c/td\u003e\n          \u003ctd\u003eProposed lab settings, not a captured RB5009 or home-router deployment.\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eCommands\u003c/td\u003e\n          \u003ctd\u003eRead-only RouterOS inventory commands; no live parser or device execution performed.\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eRuntime\u003c/td\u003e\n          \u003ctd\u003eVLAN filtering, DHCP, firewall isolation, recovery, and WiFi mapping remain untested for this revision.\u003c/td\u003e\n      \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003ch2 id=\"1-preserve-access-and-inventory-the-router\"\u003e1. Preserve access and inventory the router\u003c/h2\u003e\n\u003cp\u003eHave a working console or separate recovery path before touching the bridge carrying management traffic. Save a configuration export and an appropriate protected backup outside the router, and document how to recover them. Exports can omit secrets and are not a complete substitute for a recovery plan.\u003c/p\u003e","title":"MikroTik VLAN Segmentation — Inter-VLAN Routing, DHCP, and Firewall Rules"},{"content":"Keeping Docker containers updated is the kind of chore you automate once and forget about — until a container silently runs a four-month-old image with five CVEs because you forgot to docker compose pull \u0026amp;\u0026amp; up -d.\nWatchtower solves this. It watches your running containers, checks for new images, and restarts them with the latest tag — all on a cron schedule. But a naive \u0026ldquo;update everything\u0026rdquo; setup will break your database container and nuke your uptime.\nThis post covers a production-grade Watchtower deployment for a homelab: selective update rules, Telegram push notifications, manual rollback, and patterns for zero-downtime containers.\nArchitecture Overview ┌───────────────────────────────────────────────────┐ │ Docker Host │ │ │ │ ┌────────────┐ ┌──────────────────────┐ │ │ │ Watchtower │──────▶│ Container Registry │ │ │ │ :poll every │ │ (Docker Hub / GHCR) │ │ │ │ 6h │ └──────────────────────┘ │ │ └──────┬─────┘ │ │ │ scans containers │ │ ▼ │ │ ┌──────────────────┐ ┌──────────────┐ │ │ │ traefik (auto) │ │ postgres (skip) │ │ │ │ frigate (auto) │ │ │ │ │ │ monitoring (auto)│ │ valkey (skip)│ │ │ └──────────────────┘ └──────────────┘ │ │ │ │ Push: Telegram notification on each update │ │ Rollback: docker compose pull \u0026lt;tag\u0026gt; \u0026amp;\u0026amp; up -d │ └───────────────────────────────────────────────────┘ Why Watchtower (Not DIY Cron) You could script:\n1 docker compose pull \u0026amp;\u0026amp; docker compose up -d But this has problems:\nUpdates all services, including databases (risky) No notification when something changes No way to pin specific images No scheduling beyond basic cron Restarts every container even if no change Watchtower gives you per-container control, scheduling, notifications, and only restarts containers whose images actually changed.\n1. Deploy Watchtower 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # compose.yml services: watchtower: image: containrrr/watchtower:latest container_name: watchtower restart: unless-stopped volumes: - /var/run/docker.sock:/var/run/docker.sock:ro environment: - WATCHTOWER_CLEANUP=true - WATCHTOWER_REMOVE_VOLUMES=false - WATCHTOWER_INCLUDE_STOPPED=false - WATCHTOWER_REVIVE_STOPPED=false - WATCHTOWER_POLL_INTERVAL=21600 - TZ=America/Santo_Domingo labels: - \u0026#34;com.centurylinklabs.watchtower.enable=true\u0026#34; That\u0026rsquo;s the baseline. Deploy with:\n1 docker compose up -d By default, Watchtower polls Docker Hub every 24 hours. I set POLL_INTERVAL=21600 (6 hours) for a balance between freshness and rate limits.\n2. Selective Update Control — Skip Databases The most important config: tell Watchtower not to update stateful containers. A database restart on an image update is a database outage — you want to do that manually with proper migration checks.\nMark containers to skip with a label on their own compose file:\n1 2 3 4 5 6 # postgres/compose.yml services: postgres: image: postgres:16 labels: - \u0026#34;com.centurylinklabs.watchtower.enable=false\u0026#34; 1 2 3 4 5 6 # valkey/compose.yml services: valkey: image: valkey/valkey:7.2 labels: - \u0026#34;com.centurylinklabs.watchtower.enable=false\u0026#34; Check which containers are excluded:\n1 2 3 4 5 6 7 8 docker inspect $(docker ps -q) \\ --format \u0026#39;{{.Name}} → {{index .Config.Labels \u0026#34;com.centurylinklabs.watchtower.enable\u0026#34;}}\u0026#39; # Output: # /watchtower → true # /traefik → true # /postgres → false # /valkey → false # /prometheus → \u0026#34;\u0026lt;no value\u0026gt;\u0026#34; (inherits default = true) Labels without explicit value default to true in Watchtower, so only containers you explicitly mark false get skipped.\n3. Notifications — Telegram on Every Update Watchtower supports a dozen notification backends. Telegram is the most practical for a homelab — push notifications to your phone with what changed.\n1 2 3 4 5 6 7 8 # compose.yml — add to watchtower service services: watchtower: environment: - WATCHTOWER_NOTIFICATIONS=telegram - WATCHTOWER_NOTIFICATION_TELEGRAM_BOT_TOKEN=${TELEGRAM_BOT_TOKEN} - WATCHTOWER_NOTIFICATION_TELEGRAM_CHAT_ID=${TELEGRAM_CHAT_ID} - WATCHTOWER_NOTIFICATIONS_HOSTNAME=srv1 1 2 3 # .env TELEGRAM_BOT_TOKEN=123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11 TELEGRAM_CHAT_ID=1064959513 Restart Watchtower and trigger a manual check to test:\n1 2 3 4 docker compose up -d # Manual check docker exec watchtower watchtower --run-once You\u0026rsquo;ll see output in logs:\n1 2 3 4 docker logs watchtower --tail 20 # TIME=\u0026#34;2026-05-11T10:30:00Z\u0026#34; level=info msg=\u0026#34;Found new traefik:latest image (abcdef)\u0026#34; # TIME=\u0026#34;2026-05-11T10:30:01Z\u0026#34; level=info msg=\u0026#34;Restarting /traefik\u0026#34; # TIME=\u0026#34;2026-05-11T10:30:05Z\u0026#34; level=info msg=\u0026#34;Notification: Updated traefik → abcdef\u0026#34; Expected Telegram message:\n🐳 srv1 — Watchtower Updated: traefik:latest (abcdef12345) Updated: frigate:stable (12345abcde) No news between updates means nothing changed — exactly what you want.\n4. Schedule Instead of Poll Interval If you prefer exact scheduling (e.g., daily at 3 AM), use cron syntax over the generic poll interval:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 services: watchtower: image: containrrr/watchtower:latest container_name: watchtower restart: unless-stopped volumes: - /var/run/docker.sock:/var/run/docker.sock:ro environment: - WATCHTOWER_CLEANUP=true - WATCHTOWER_SCHEDULE=0 0 3 * * * - TZ=America/Santo_Domingo - WATCHTOWER_NOTIFICATIONS=telegram - WATCHTOWER_NOTIFICATION_TELEGRAM_BOT_TOKEN=${TELEGRAM_BOT_TOKEN} - WATCHTOWER_NOTIFICATION_TELEGRAM_CHAT_ID=${TELEGRAM_CHAT_ID} WATCHTOWER_SCHEDULE=0 0 3 * * * = daily at 3:00 AM local time (requires TZ to be set correctly). Remove POLL_INTERVAL if using SCHEDULE — they conflict.\nNote: Containers with restart: always or unless-stopped get restarted cleanly at 3 AM. If a container takes \u0026gt; 30s to restart, adjust the default timeout:\n1 2 environment: - WATCHTOWER_TIMEOUT=120s 5. Rolling Back a Bad Update Watchtower updates the running container, but it doesn\u0026rsquo;t pin the previous image tag. This is the one gap you need to cover manually.\nWhen a bad update happens:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 # 1. Find the previous working image docker inspect traefik --format \u0026#39;{{.Config.Image}}\u0026#39; # → traefik:v3.2 # 2. Check image history docker images traefik # REPOSITORY TAG IMAGE ID CREATED SIZE # traefik v3.3 abc123 2 hours ago 150MB # traefik v3.2 def456 2 weeks ago 145MB # 3. Pin to previous version docker tag traefik:v3.2 traefik:latest docker compose up -d # Or better: update your compose.yml to pin explicitly: # image: traefik:v3.2 # then docker compose up -d Prevent it: Pin major versions in compose files and let Watchtower only handle patch updates:\n1 2 3 4 5 # Instead of: image: traefik:latest # Use: image: traefik:v3.2 Watchtower will update traefik:v3.2 when a new v3.2.x patch is published, but won\u0026rsquo;t jump you to v3.3 automatically. This is the safest pattern for datastore-adjacent containers.\n6. Full Example — Selective Updates with Notifications 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 # /opt/docker/watchtower/compose.yml services: watchtower: image: containrrr/watchtower:latest container_name: watchtower restart: unless-stopped volumes: - /var/run/docker.sock:/var/run/docker.sock:ro environment: - WATCHTOWER_CLEANUP=true - WATCHTOWER_REMOVE_VOLUMES=false - WATCHTOWER_INCLUDE_STOPPED=false - WATCHTOWER_REVIVE_STOPPED=false - WATCHTOWER_SCHEDULE=0 0 3 * * * - WATCHTOWER_TIMEOUT=60s - WATCHTOWER_NOTIFICATIONS=telegram - WATCHTOWER_NOTIFICATION_TELEGRAM_BOT_TOKEN=${TELEGRAM_BOT_TOKEN} - WATCHTOWER_NOTIFICATION_TELEGRAM_CHAT_ID=${TELEGRAM_CHAT_ID} - WATCHTOWER_NOTIFICATIONS_HOSTNAME=srv1 - TZ=America/Santo_Domingo labels: - \u0026#34;com.centurylinklabs.watchtower.enable=true\u0026#34; logging: driver: \u0026#34;json-file\u0026#34; options: max-size: \u0026#34;10m\u0026#34; max-file: \u0026#34;3\u0026#34; 1 2 3 # .env (same dir as compose.yml) TELEGRAM_BOT_TOKEN=123456:ABC-DEF1234ghIkl-zyx57W2v1u123ew11 TELEGRAM_CHAT_ID=1064959513 Deploy:\n1 2 3 4 5 6 7 8 mkdir -p /opt/docker/watchtower cd /opt/docker/watchtower # Create compose.yml and .env, then: docker compose up -d # Verify docker logs watchtower --tail 10 # Should see: \u0026#34;Scheduling first run: ... daily at 03:00\u0026#34; 7. Excluding Containers — What to Skip Container Auto-update? Reason Traefik ✅ Yes Proxy downtime = brief blip, stateless Frigate ✅ Yes Pulls new model weights, fast restart Grafana / Prometheus ⚠️ Cautiously Pin major version (image: grafana/grafana:11.3) Loki ⚠️ Cautiously Schema version matters — pin major Postgres / MySQL ❌ No Manual migration on version bumps ValKey / Redis ❌ No Breaking config changes between majors Watchtower itself ✅ Yes Self-updating is fine PBS client / restic ✅ Yes Stateless, always pull latest The rule: Stateless containers auto-update. Stateful containers with persistent data — update manually after reviewing release notes.\n8. Zero-Downtime Considerations Watchtower stops a container, pulls the new image, and starts it again. For a single-replica setup, there\u0026rsquo;s brief downtime (~5-15 seconds).\nIf you need true zero-downtime updates for a web service:\nOption A: Docker Compose scale + health check\n1 2 3 4 5 6 7 8 9 10 services: app: deploy: replicas: 2 healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;curl\u0026#34;, \u0026#34;-f\u0026#34;, \u0026#34;http://localhost:8080/health\u0026#34;] interval: 30s timeout: 10s retries: 3 start_period: 15s Add this to Watchtower to enable rolling restart:\n1 2 environment: - WATCHTOWER_ROLLING_RESTART=true Watchtower will stop one replica, pull, start, wait for the health check to pass, then move to the next.\nOption B: Use a load balancer in front (Traefik with multiple backends) — Watchtower\u0026rsquo;s rolling restart + health check is the simpler homelab solution.\n9. Lifecycle Hooks — Run Commands Before/After Updates Watchtower supports prerestart and postrestart commands via labels:\n1 2 3 4 5 6 7 8 # frigate/compose.yml services: frigate: image: ghcr.io/blakeblackshear/frigate:stable labels: - \u0026#34;com.centurylinklabs.watchtower.enable=true\u0026#34; - \u0026#34;com.centurylinklabs.watchtower.lifecycle.prerestart=echo \u0026#39;Stopping frigate for update\u0026#39;\u0026#34; - \u0026#34;com.centurylinklabs.watchtower.lifecycle.postrestart=s6-svwait /run/service\u0026#34; Useful for services that need graceful shutdown:\n1 2 3 4 5 6 services: homeassistant: image: ghcr.io/home-assistant/home-assistant:stable labels: - \u0026#34;com.centurylinklabs.watchtower.enable=true\u0026#34; - \u0026#34;com.centurylinklabs.watchtower.lifecycle.prerestart=ha core stop\u0026#34; 10. Monitoring Watchtower Add a Prometheus blackbox probe or simply monitor the watchtower container\u0026rsquo;s last-seen timestamp:\n1 2 3 4 5 6 # Quick check — when did Watchtower last run? docker exec watchtower cat /var/run/watchtower/state.json 2\u0026gt;/dev/null || \\ echo \u0026#34;No state file (first run pending)\u0026#34; # Check container uptime docker inspect watchtower --format \u0026#39;{{.State.StartedAt}}\u0026#39; Better: expose Watchtower\u0026rsquo;s metrics endpoint. Mount a port:\n1 2 3 4 services: watchtower: ports: - \u0026#34;10.0.20.50:8099:8080\u0026#34; # LAN only Then add to Prometheus:\n1 2 3 4 scrape_configs: - job_name: \u0026#39;watchtower\u0026#39; static_configs: - targets: [\u0026#39;10.0.20.50:8099\u0026#39;] 11. Alternative: Diun (Notification-Only) If you don\u0026rsquo;t want Watchtower touching your containers at all (just notifying you about available updates), use Diun instead:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 services: diun: image: crazymax/diun:latest container_name: diun restart: unless-stopped volumes: - /var/run/docker.sock:/var/run/docker.sock:ro - ./diun/data:/data environment: - TZ=America/Santo_Domingo - DIUN_WATCH_WORKERS=20 - DIUN_WATCH_SCHEDULE=0 0 * * * * - DIUN_NOTIF_TELEGRAM_BOT_TOKEN=${TELEGRAM_BOT_TOKEN} - DIUN_NOTIF_TELEGRAM_CHAT_ID=${TELEGRAM_CHAT_ID} labels: - \u0026#34;com.centurylinklabs.watchtower.enable=false\u0026#34; Diun sends a Telegram message when a new image is available but does nothing else. You decide when to pull the trigger. This is the conservative approach.\nSummary 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 # Fast deploy (with Telegram notifications and daily 3 AM schedule) mkdir -p /opt/docker/watchtower \u0026amp;\u0026amp; cd $_ cat \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; \u0026gt; compose.yml services: watchtower: image: containrrr/watchtower:latest container_name: watchtower restart: unless-stopped volumes: - /var/run/docker.sock:/var/run/docker.sock:ro environment: - WATCHTOWER_CLEANUP=true - WATCHTOWER_SCHEDULE=0 0 3 * * * - WATCHTOWER_NOTIFICATIONS=telegram - WATCHTOWER_NOTIFICATION_TELEGRAM_BOT_TOKEN=${TELEGRAM_BOT_TOKEN} - WATCHTOWER_NOTIFICATION_TELEGRAM_CHAT_ID=${TELEGRAM_CHAT_ID} - WATCHTOWER_NOTIFICATIONS_HOSTNAME=srv1 - TZ=America/Santo_Domingo EOF echo \u0026#34;TELEGRAM_BOT_TOKEN=your_token\u0026#34; \u0026gt;\u0026gt; .env echo \u0026#34;TELEGRAM_CHAT_ID=your_chat_id\u0026#34; \u0026gt;\u0026gt; .env docker compose up -d # Mark stateful containers to skip # Add to postgres/compose.yml and valkey/compose.yml: # labels: # - \u0026#34;com.centurylinklabs.watchtower.enable=false\u0026#34; # Verify docker logs watchtower --tail 5 # → \u0026#34;Watchtower 1.7.1 starting...\u0026#34; # → \u0026#34;Scheduling first run: 2026-05-12 03:00:00 -04 EDT\u0026#34; Watchtower isn\u0026rsquo;t optional once you have more than five containers. A 3 AM daily check with Telegram push means you wake up to a notification like \u0026ldquo;Updated traefik, frigate, prometheus\u0026rdquo; — or nothing at all. Manual rollback is a docker compose pull \u0026lt;version\u0026gt; away. Skip your database containers, pin major versions on stateful workloads, and let the rest update themselves.\n","permalink":"https://blog.gntech.me/posts/2026-05-11-docker-watchtower-auto-updates/","summary":"\u003cp\u003eKeeping Docker containers updated is the kind of chore you automate once\nand forget about — until a container silently runs a four-month-old image\nwith five CVEs because you forgot to \u003ccode\u003edocker compose pull \u0026amp;\u0026amp; up -d\u003c/code\u003e.\u003c/p\u003e\n\u003cp\u003eWatchtower solves this. It watches your running containers, checks for\nnew images, and restarts them with the latest tag — all on a cron\nschedule. But a naive \u0026ldquo;update everything\u0026rdquo; setup will break your\ndatabase container and nuke your uptime.\u003c/p\u003e","title":"Auto-Update Docker Containers — Watchtower with Selective Rules, Notifications, and Graceful Rollouts"},{"content":"Publishing a Docker port can make it reachable even when UFW says incoming traffic is denied. Docker routes published bridge-network traffic through NAT and forwarding rules; UFW\u0026rsquo;s host INPUT policy does not provide the access control you might expect.\nThis guide separates two jobs: limiting where a port is published, and filtering which clients can reach it. Binding to a LAN address reduces exposure, but does not make UFW\u0026rsquo;s host-input rules control forwarded container traffic.\n1. Check the Existing Rules Run on the Docker host with administrator access:\n1 2 3 4 5 docker version docker ps --format \u0026#39;table {{.Names}}\\t{{.Ports}}\u0026#39; sudo ufw status verbose sudo iptables -S DOCKER-USER sudo iptables -t nat -S DOCKER The rules below apply to Docker\u0026rsquo;s iptables backend, including iptables-nft compatibility mode. If Docker uses its native nftables backend, use the nftables guide instead. Do not create a fake DOCKER-USER chain to hide a backend mismatch.\n2. Avoid Publishing Ports You Do Not Need Services on the same Compose network can communicate by service name without ports. A database used only by an application normally needs no host-published port:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 services: database: image: postgres:17 environment: POSTGRES_PASSWORD_FILE: /run/secrets/database_password secrets: - database_password volumes: - database_data:/var/lib/postgresql/data secrets: database_password: file: ./database_password.txt volumes: database_data: Create the password file securely before starting this example. Keep it out of Git. Do not replace an existing database deployment with this sample or change its PostgreSQL major version without a migration plan.\nFor a service used only by software on the host, bind to loopback:\n1 2 ports: - \u0026#34;127.0.0.1:8080:80\u0026#34; For LAN clients, bind to an address actually assigned to the host:\n1 2 ports: - \u0026#34;10.0.20.50:8080:80\u0026#34; A LAN binding is not a source allowlist. Routed clients may still reach it. Apply a forwarding policy when source restrictions are required, and check IPv6 exposure separately. Docker releases before 28.0.0 also have a documented localhost-publishing caveat for peers on the same layer-2 segment; update older installations.\n3. Filter One Published Port with DOCKER-USER This example protects TCP port 8080 published on 10.0.20.50, allowing new connections only from 10.0.10.0/24 arriving on ens18. Replace all three values before running it.\nUse the incoming physical interface, not docker0. At this hook, destination NAT has already happened, so --ctorigdst and --ctorigdstport match the original host address and published port.\nFirst save the existing IPv4 rules:\n1 sudo sh -c \u0026#39;iptables-save \u0026gt; /root/iptables-before-blog-filter.rules\u0026#39; Add a dedicated chain without flushing any existing policy:\n1 2 3 4 5 6 7 sudo iptables -N GNTECH-WEB8080 sudo iptables -A GNTECH-WEB8080 -s 10.0.10.0/24 -j RETURN sudo iptables -A GNTECH-WEB8080 -j DROP sudo iptables -I DOCKER-USER 1 -i ens18 -p tcp \\ -m conntrack --ctstate NEW \\ --ctorigdst 10.0.20.50 --ctorigdstport 8080 \\ -j GNTECH-WEB8080 Run this setup once. If the chain exists, inspect it instead of running the commands again. The jump only handles new connections to this one published endpoint; other forwarding rules remain in force. Existing connections are unaffected.\nThis example covers IPv4 only. If the service is reachable through IPv6, configure and test an equivalent IPv6 policy or remove its IPv6 publication. Conntrack matching can have a performance cost.\n4. Verify from Allowed and Blocked Clients On the host:\n1 2 3 sudo iptables -nvL DOCKER-USER --line-numbers sudo iptables -nvL GNTECH-WEB8080 --line-numbers docker ps --format \u0026#39;table {{.Names}}\\t{{.Ports}}\u0026#39; From an allowed client and then a client outside the permitted subnet:\n1 curl --connect-timeout 5 http://10.0.20.50:8080/ The allowed request should reach the application; the blocked request should time out. Use a new connection for each check. Verify container DNS, outbound access, and application health too. A host-local curl or an ss listing alone does not prove that external access is blocked.\n5. Persistence and Rollback The commands above are runtime rules. Integrate them into your existing firewall manager so they load after Docker creates DOCKER-USER, without duplicating rules. Verify again after reboot and after a Docker restart. Avoid blindly saving and replaying all of Docker\u0026rsquo;s dynamically generated rules.\nTo remove only this example:\n1 2 3 4 5 6 sudo iptables -D DOCKER-USER -i ens18 -p tcp \\ -m conntrack --ctstate NEW \\ --ctorigdst 10.0.20.50 --ctorigdstport 8080 \\ -j GNTECH-WEB8080 sudo iptables -F GNTECH-WEB8080 sudo iptables -X GNTECH-WEB8080 Do not disable Docker\u0026rsquo;s iptables or ip6tables options as a UFW fix. Those options disable rules Docker needs for networking; they do not transfer container access control to UFW automatically.\nReferences Docker: packet filtering, firewalls, and UFW Docker: iptables chains and original-destination matching Docker: port publishing and localhost caveat ","permalink":"https://blog.gntech.me/posts/2026-05-10-docker-ufw-firewall-fix/","summary":"\u003cp\u003ePublishing a Docker port can make it reachable even when UFW says incoming traffic is denied. Docker routes published bridge-network traffic through NAT and forwarding rules; UFW\u0026rsquo;s host \u003ccode\u003eINPUT\u003c/code\u003e policy does not provide the access control you might expect.\u003c/p\u003e\n\u003cp\u003eThis guide separates two jobs: limiting where a port is published, and filtering which clients can reach it. Binding to a LAN address reduces exposure, but does \u003cstrong\u003enot\u003c/strong\u003e make UFW\u0026rsquo;s host-input rules control forwarded container traffic.\u003c/p\u003e","title":"Docker Bypasses UFW — How to Fix Firewall Rules Properly"},{"content":"If your Docker homelab has more than three web services, you need a reverse proxy. Without one, every container exposes its own port, you manage certificates by hand (or skip HTTPS entirely), and changing a service\u0026rsquo;s URL means editing Nginx configs and reloading.\nTraefik solves all of this. It watches the Docker socket, discovers new containers automatically, provisions Let\u0026rsquo;s Encrypt certificates for any hostname you define via Docker labels, and handles middleware (auth, rate limiting, headers) without touching a static config file.\nThis post deploys Traefik v3 on Docker with:\nAutomatic container discovery via Docker labels Let\u0026rsquo;s Encrypt (HTTP-01 and TLS-ALPN-01 challenges) HTTP→HTTPS redirect with permanent redirect middleware Rate limiting and IP whitelist examples Dashboard secured with HTTP Basic Auth Static and dynamic configuration split Architecture Internet │ ┌──────▼──────┐ │ Cloudflare │ (or any DNS provider) │ ───────── │ │ *.lab.io │ → your public IP └──────┬──────┘ │ port 80, 443 ┌──────▼──────┐ │ Traefik │ │ ───────── │ │ EntryPoints│ │ :80 :443 │ └──┬───────┬──┘ │ │ ┌──────────────┼───────┼──────────────────┐ │ │ │ │ │ Docker network: traefik │ │ │ │ ┌─────────┐ ┌─────────┐ ┌──────────┐ │ │ │ Service A│ │ Service B│ │ Service C│ │ │ │ web:80 │ │ web:3000│ │ web:8080 │ │ │ └─────────┘ └─────────┘ └──────────┘ │ └────────────────────────────────────────────┘ Traefik binds to ports 80 and 443 on the host. All other web services bind to an internal Docker network and are exposed only through Traefik routes defined by Docker labels. No port conflicts, no manual cert renewal, no config reloads.\nDirectory Layout /opt/docker/traefik/ ├── compose.yml ├── .env ├── traefik.yml # Static config ├── config/ │ └── dynamic.yml # Dynamic config (middleware, etc.) └── acme.json # Let\u0026#39;s Encrypt cert storage (auto-created) The split between traefik.yml and config/ is deliberate:\nStatic config (traefik.yml) — entrypoints, providers, cert resolvers, logging. Needs a restart to reload. Dynamic config (config/dynamic.yml) — middleware definitions, error pages, catch-all routers. Watched and reloaded live by Traefik. 1. Static Configuration 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 # /opt/docker/traefik/traefik.yml api: dashboard: true debug: false entryPoints: web: address: \u0026#34;:80\u0026#34; http: redirections: entryPoint: to: websecure scheme: https permanent: true websecure: address: \u0026#34;:443\u0026#34; providers: docker: endpoint: \u0026#34;unix:///var/run/docker.sock\u0026#34; exposedByDefault: false network: traefik file: filename: /etc/traefik/config/dynamic.yml watch: true certificatesResolvers: letsencrypt: acme: email: \u0026#34;${ACME_EMAIL}\u0026#34; storage: /etc/traefik/acme.json caServer: \u0026#34;https://acme-v02.api.letsencrypt.org/directory\u0026#34; httpChallenge: entryPoint: web letsencrypt-staging: acme: email: \u0026#34;${ACME_EMAIL}\u0026#34; storage: /etc/traefik/acme-staging.json caServer: \u0026#34;https://acme-staging-v02.api.letsencrypt.org/directory\u0026#34; httpChallenge: entryPoint: web log: level: INFO filePath: /var/log/traefik.log accessLog: filePath: /var/log/traefik-access.log bufferingSize: 100 Key points:\nexposedByDefault: false — containers must explicitly opt in with labels. Prevents accidental exposure of services. network: traefik — Traefik only discovers containers connected to the traefik network. Two cert resolvers — letsencrypt for production, letsencrypt-staging for testing (avoids rate limit hits during testing). HTTP→HTTPS redirect — defined at the web entrypoint, permanent (301). No middleware needed for the basic case. The acme.json file stores certificates. When Traefik starts, it creates this file with mode 0600 (required). If it doesn\u0026rsquo;t exist, create it:\n1 2 touch /opt/docker/traefik/acme.json chmod 600 /opt/docker/traefik/acme.json 2. Dynamic Configuration (Middlewares) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 # /opt/docker/traefik/config/dynamic.yml http: middlewares: # Security headers for all services secHeaders: headers: browserXssFilter: true contentTypeNosniff: true frameDeny: true sslRedirect: true stsIncludeSubdomains: true stsPreload: true stsSeconds: 31536000 customFrameOptionsValue: \u0026#34;SAMEORIGIN\u0026#34; referrerPolicy: \u0026#34;strict-origin-when-cross-origin\u0026#34; permissionsPolicy: \u0026#34;camera=(), microphone=(), geolocation=()\u0026#34; # Rate limiting: 100 req/min per IP ratelimit: rateLimit: average: 100 burst: 50 period: 1m sourceCriterion: ipStrategy: depth: 1 # IP whitelist for internal services internalOnly: ipWhiteList: sourceRange: - \u0026#34;10.0.0.0/8\u0026#34; - \u0026#34;172.16.0.0/12\u0026#34; - \u0026#34;192.168.0.0/16\u0026#34; - \u0026#34;100.64.0.0/10\u0026#34; # Basic auth for Traefik dashboard dashboard-auth: basicAuth: users: - \u0026#34;${DASHBOARD_USER}:${DASHBOARD_PASSWORD_HASH}\u0026#34; # Compress responses compress: compress: excludedContentTypes: - text/event-stream Split into reusable middleware building blocks. Services declare which middleware they want with labels — no monolithic config.\nGenerate the password hash for basic auth:\n1 2 3 4 5 # Install htpasswd sudo apt install apache2-utils -y # Generate hash for user \u0026#34;admin\u0026#34; — output goes in .env echo \u0026#34;admin:$(openssl passwd -apr1)\u0026#34; Take the hash output and put it in your .env:\n1 2 3 4 # /opt/docker/traefik/.env ACME_EMAIL=admin@example.com DASHBOARD_USER=admin DASHBOARD_PASSWORD_HASH=$apr1$xxxx... # full hash from above 3. Docker Compose 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 # /opt/docker/traefik/compose.yml services: traefik: image: traefik:v3.2 container_name: traefik restart: unless-stopped security_opt: - no-new-privileges:true cap_drop: - ALL cap_add: - NET_BIND_SERVICE networks: - traefik ports: - \u0026#34;80:80\u0026#34; - \u0026#34;443:443\u0026#34; environment: - CF_API_EMAIL=${CF_API_EMAIL:-} - CF_API_KEY=${CF_API_KEY:-} - CF_DNS_API_TOKEN=${CF_DNS_API_TOKEN:-} - TZ=America/Santo_Domingo env_file: - .env volumes: - /var/run/docker.sock:/var/run/docker.sock:ro - ./traefik.yml:/etc/traefik/traefik.yml:ro - ./config:/etc/traefik/config:ro - ./acme.json:/etc/traefik/acme.json - ./logs:/var/log labels: # Enable Traefik for itself (dashboard) - \u0026#34;traefik.enable=true\u0026#34; # Router: dashboard - \u0026#34;traefik.http.routers.dashboard.rule=Host(`traefik.lab.io`)\u0026#34; - \u0026#34;traefik.http.routers.dashboard.service=api@internal\u0026#34; - \u0026#34;traefik.http.routers.dashboard.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.dashboard.tls=true\u0026#34; - \u0026#34;traefik.http.routers.dashboard.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.routers.dashboard.middlewares=dashboard-auth,secHeaders,ratelimit\u0026#34; # Catch-all router for 404 pages (optional) - \u0026#34;traefik.http.routers.catchall.rule=HostRegexp(`{host:.+}`)\u0026#34; - \u0026#34;traefik.http.routers.catchall.priority=1\u0026#34; - \u0026#34;traefik.http.routers.catchall.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.catchall.tls=true\u0026#34; - \u0026#34;traefik.http.routers.catchall.middlewares=internalOnly\u0026#34; networks: traefik: name: traefik external: false Security notes on cap_add:\nNET_BIND_SERVICE is the minimum capability needed for Traefik to bind ports \u0026lt; 1024 (80 and 443). cap_drop: ALL removes all capabilities, then we add back only what\u0026rsquo;s needed. The Docker socket is mounted :ro (read-only). Traefik only reads container metadata — it never writes to the socket. 4. Exposing Services with Docker Labels This is where Traefik shines. Any container connected to the traefik network with the right labels is automatically discovered, routed, and TLS-protected.\nExample: Whoogle Search (self-hosted search) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 services: whoogle: image: benbusby/whoogle-search:latest container_name: whoogle restart: unless-stopped networks: - traefik environment: - WHOOGLE_CONFIG_USER=user - WHOOGLE_CONFIG_PASS=pass labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.whoogle.rule=Host(`search.lab.io`)\u0026#34; - \u0026#34;traefik.http.routers.whoogle.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.whoogle.tls=true\u0026#34; - \u0026#34;traefik.http.routers.whoogle.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.services.whoogle.loadbalancer.server.port=5000\u0026#34; networks: traefik: external: true Only three label groups matter:\nLabel Purpose traefik.enable=true Opt-in (required because exposedByDefault: false) traefik.http.routers.\u0026lt;name\u0026gt;.rule=Host(\\host.domain`)` Routing rule traefik.http.routers.\u0026lt;name\u0026gt;.tls.certresolver=letsencrypt Auto-get cert traefik.http.services.\u0026lt;name\u0026gt;.loadbalancer.server.port=\u0026lt;port\u0026gt; Container port When you don\u0026rsquo;t need the last label: If your container exposes port 80 (or only one port), Traefik auto-detects it. Use the explicit .server.port label when your service uses a non-standard port.\nExample: Grafana with middleware 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 services: grafana: image: grafana/grafana:11.3.0 container_name: grafana restart: unless-stopped networks: - traefik labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.grafana.rule=Host(`grafana.lab.io`)\u0026#34; - \u0026#34;traefik.http.routers.grafana.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.grafana.tls=true\u0026#34; - \u0026#34;traefik.http.routers.grafana.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.routers.grafana.middlewares=secHeaders,ratelimit\u0026#34; - \u0026#34;traefik.http.services.grafana.loadbalancer.server.port=3000\u0026#34; networks: traefik: external: true Example: Internal-only service (Vaultwarden) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 services: vaultwarden: image: vaultwarden/server:latest container_name: vaultwarden restart: unless-stopped networks: - traefik labels: - \u0026#34;traefik.enable=true\u0026#34; - \u0026#34;traefik.http.routers.vault.rule=Host(`vault.lab.io`)\u0026#34; - \u0026#34;traefik.http.routers.vault.entrypoints=websecure\u0026#34; - \u0026#34;traefik.http.routers.vault.tls=true\u0026#34; - \u0026#34;traefik.http.routers.vault.tls.certresolver=letsencrypt\u0026#34; - \u0026#34;traefik.http.routers.vault.middlewares=internalOnly,secHeaders\u0026#34; - \u0026#34;traefik.http.services.vault.loadbalancer.server.port=80\u0026#34; networks: traefik: external: true The internalOnly middleware blocks all traffic not originating from RFC1918 IPs. If you want to access it externally, use a VPN instead of exposing it to the internet.\n5. DNS Challenge with Cloudflare The default HTTP-01 challenge works when Traefik can reach Let\u0026rsquo;s Encrypt servers on port 80, which it does by default. But if:\nYou want wildcard certificates (*.lab.io) Port 80 is blocked by your ISP Your service is behind a NAT where LE can\u0026rsquo;t reach you Swap to DNS-01 challenge with Cloudflare:\n1 2 3 4 5 6 7 8 9 10 11 12 # In traefik.yml, replace the cert resolver: certificatesResolvers: letsencrypt: acme: email: \u0026#34;${ACME_EMAIL}\u0026#34; storage: /etc/traefik/acme.json caServer: \u0026#34;https://acme-v02.api.letsencrypt.org/directory\u0026#34; dnsChallenge: provider: cloudflare resolvers: - \u0026#34;1.1.1.1:53\u0026#34; - \u0026#34;8.8.8.8:53\u0026#34; And in .env:\n1 2 3 4 5 6 # Global API Key (legacy) CF_API_EMAIL=admin@example.com CF_API_KEY=your_global_api_key # OR — API Token (recommended) CF_DNS_API_TOKEN=your_dns_api_token Create a Cloudflare API Token with permissions:\nZone → DNS → Edit Zone → Zone → Read Then with dnsChallenge, Traefik can issue wildcard certs — every *.lab.io subdomain is covered by one cert.\n6. Deploying 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 # Prepare mkdir -p /opt/docker/traefik/{config,logs} touch /opt/docker/traefik/acme.json chmod 600 /opt/docker/traefik/acme.json # Edit .env with your email and admin password hash # Generate the hash: # openssl passwd -apr1 # Pull and start cd /opt/docker/traefik docker compose pull docker compose up -d # Check logs docker compose logs -f --tail=100 # Verify API is running curl -s -o /dev/null -w \u0026#34;%{http_code}\u0026#34; http://localhost:80 # Should return 301 (redirect to HTTPS) # Traefik health endpoint curl -s http://localhost:80/api/http/routers | jq \u0026#39;.[].rule\u0026#39; 7. Testing Certificates with Staging Before hitting production Let\u0026rsquo;s Encrypt rate limits (50 certs/week per registered domain), test with the staging resolver:\n1 2 3 # In a test compose file: labels: - \u0026#34;traefik.http.routers.test.tls.certresolver=letsencrypt-staging\u0026#34; Staging certificates show as untrusted in the browser but confirm your challenge workflow works. Once it works, switch to letsencrypt and Traefik will replace the cert automatically.\nTo clear a staging cert:\n1 2 docker compose exec traefik rm /etc/traefik/acme-staging.json docker compose restart traefik 8. Troubleshooting \u0026ldquo;404 page not found\u0026rdquo; on a new service Is the container connected to the traefik network? Does it have traefik.enable=true? Does the hostname in rule=Host(\\\u0026hellip;`)` resolve to this host? Check docker compose logs traefik for router registration logs. Certificate doesn\u0026rsquo;t renew Check ACME logs: docker compose logs traefik | grep acme If using HTTP-01, verify port 80 is reachable from the internet If using DNS-01, verify the API token has DNS:Edit permission \u0026ldquo;too many routes\u0026rdquo; warning in logs Every container with traefik.enable=true creates at least one router. Keep exposedByDefault: false and only enable what you need. Old containers (stopped or removed) can leave orphaned routes until Traefik refreshes from Docker (every few seconds). Dashboard shows \u0026ldquo;Internal Server Error\u0026rdquo; Check basic auth credentials — the hash format must match what openssl passwd -apr1 generates. Traefik\u0026rsquo;s own labels must be correct since it discovers routes via the same Docker provider as everything else. Port 80/443 already in use Traefik must be the only service listening on 80/443. If you have another web server or Nginx, either stop it or change Traefik\u0026rsquo;s bind ports in the compose file and add port mapping at the router level. Why Traefik Over Nginx/Caddy for a Homelab Feature Traefik Nginx Proxy Manager Caddy Auto container discovery ✅ Native via Docker socket ❌ Manual per-service ❌ Manual (or plugin) Auto TLS (LE) ✅ Built-in ✅ Built-in ✅ Built-in Dynamic config (no reload) ✅ Live reload ✅ UI-based ⚠️ File-based Middleware (rate limit, auth) ✅ Declarative (labels) ✅ UI-based ⚠️ Limited Resource usage (idle) ~40-60 MB RAM ~60-80 MB RAM ~15-25 MB RAM Wildcard certs ✅ DNS-01 ✅ DNS-01 ✅ DNS-01 Kubernetes-style labels ✅ Same syntax ❌ ❌ I used Nginx Proxy Manager for over a year. It works, but the UI becomes tedious once you have more than 10 services. Traefik\u0026rsquo;s Docker-native discovery means you add labels when you write the compose file — one step, not two. And if you ever move to Kubernetes (or Nomad), Traefik runs there with the exact same config model.\nResource Usage Component RAM Disk Traefik (idle, 10 routes) ~45 MB ~50 MB (logs, certs) Cert storage (10 domains) — ~50 KB per cert Access logs (rotated daily) — ~10 MB/day (typical lab) Negligible. Traefik is written in Go and runs with zero overhead until traffic hits it.\nSummary 1 2 3 4 5 6 7 8 # Quick deploy mkdir -p /opt/docker/traefik/{config,logs} touch acme.json \u0026amp;\u0026amp; chmod 600 acme.json # Write traefik.yml, config/dynamic.yml, compose.yml, .env docker compose up -d # Add any web service: # networks: [traefik] + labels above → auto TLS and routing A reverse proxy is the backbone of a self-hosted Docker environment. Traefik makes it automatic — you declare routes as container labels instead of maintaining a config file, TLS is zero-config with Let\u0026rsquo;s Encrypt, and middleware chains handle security, rate limiting, and access control without touching a config file.\nIf you\u0026rsquo;re still mapping port 8080 → container A, port 8081 → container B, and juggling self-signed certs, spend 20 minutes setting this up. It\u0026rsquo;s the last reverse proxy config you\u0026rsquo;ll ever write.\n","permalink":"https://blog.gntech.me/posts/2026-05-09-traefik-reverse-proxy-docker/","summary":"\u003cp\u003eIf your Docker homelab has more than three web services, you need a\nreverse proxy. Without one, every container exposes its own port, you\nmanage certificates by hand (or skip HTTPS entirely), and changing a\nservice\u0026rsquo;s URL means editing Nginx configs and reloading.\u003c/p\u003e\n\u003cp\u003eTraefik solves all of this. It watches the Docker socket, discovers new\ncontainers automatically, provisions Let\u0026rsquo;s Encrypt certificates for\nany hostname you define via Docker labels, and handles middleware\n(auth, rate limiting, headers) without touching a static config file.\u003c/p\u003e","title":"Traefik as a Reverse Proxy for Docker — Automatic TLS, Routing, and Middleware"},{"content":"Use a small, complete stack to learn the deployment workflow. Then add only the networking, storage, and secret-handling features the application needs.\nGroup services by ownership and lifecycle. A Compose project can contain several services, and you can operate on one service without restarting the whole project. Fixed container names are optional and can cause collisions or prevent scaling; service names are usually sufficient.\nVerification status Area Evidence and limits Technical review Official Docker documentation reviewed on October 31, 2026. Static checks Bash syntax and embedded Compose YAML checked. Runtime Docker unavailable in this review environment; no Compose CLI validation, image pull, HTTP request, or resource test executed. Example scope Disposable static HTTP lab on Linux, not a captured production stack. 1. Start with one complete project This demo uses BusyBox\u0026rsquo;s HTTP server and wget probe, a read-only content mount, loopback publishing, log rotation, and explicit resource limits. Use Bash and have curl installed on the host.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 compose_lab_dir=$(mktemp -d -t compose-patterns-lab.XXXXXX) cd \u0026#34;$compose_lab_dir\u0026#34; || exit 1 mkdir www printf \u0026#39;gntech-compose-check\\n\u0026#39; \u0026gt; www/index.html chmod 755 www chmod 644 www/index.html cat \u0026gt; compose.yaml \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; services: web: image: busybox:1.37 user: \u0026#34;65534:65534\u0026#34; command: [\u0026#34;httpd\u0026#34;, \u0026#34;-f\u0026#34;, \u0026#34;-p\u0026#34;, \u0026#34;8080\u0026#34;, \u0026#34;-h\u0026#34;, \u0026#34;/www\u0026#34;] ports: - \u0026#34;127.0.0.1:18080:8080\u0026#34; volumes: - ./www:/www:ro healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;wget\u0026#34;, \u0026#34;-q\u0026#34;, \u0026#34;-O\u0026#34;, \u0026#34;/dev/null\u0026#34;, \u0026#34;http://127.0.0.1:8080/\u0026#34;] interval: 10s timeout: 3s retries: 3 start_period: 5s logging: driver: json-file options: max-size: \u0026#34;5m\u0026#34; max-file: \u0026#34;2\u0026#34; mem_limit: 64m cpus: 0.5 EOF docker compose config --quiet docker compose up -d docker compose ps curl --fail --silent --show-error http://127.0.0.1:18080/ Expected response to verify: gntech-compose-check. Wait for the health probe and inspect logs if it does not become healthy. The content permissions allow the selected non-root user to read this public test file.\nThis demo does not need persistence or a restart policy. For a long-running deployment, choose restart behavior to match its lifecycle. An unhealthy container is not automatically restarted merely because a restart policy exists.\n2. Separate interpolation from container environment Mechanism Purpose Shell variables, .env, or CLI \u0026ndash;env-file Supply values for Compose interpolation. Service environment Explicitly sets container environment variables. Service env_file Loads environment variables into that service\u0026rsquo;s container. A service env_file does not supply values for interpolating the rest of compose.yaml. Use explicit CLI \u0026ndash;env-file choices when you need shared interpolation inputs, and check the resolved configuration.\nDo not assume every value in .env becomes a container environment variable. Shell values and file precedence also matter. See Compose interpolation.\nKeep secret values out of Git, including application configuration files that may embed credentials. Review staged changes; a private repository is not a substitute for secret management. Treat resolved configuration output as sensitive when it contains secrets.\nPUID and PGID are image-specific conventions, not universal Docker controls. Use the image\u0026rsquo;s documented permissions model or a supported user setting.\n3. Choose network access explicitly Choice Meaning and limit Default project bridge Service-name discovery within the project; outbound connectivity normally available. Published port Makes a container port reachable through a host binding; select the bind address intentionally. External network Network lifecycle is managed outside this project; it does not inherently mean internet access. Internal network Externally isolated network; not a blanket guarantee that the host cannot access containers. Host network mode Shares the host network namespace; no normal port mapping or project service-name discovery. Use Compose networking to select the actual connectivity model. Shared networks let services in different projects communicate; attach only the necessary services and avoid ambiguous aliases.\nNeither internal networking nor loopback publishing substitutes for a complete host and application access policy. Services attached to multiple networks need their combined access reviewed.\nMacvlan is a separate design for direct layer-2 attachment. It requires planned addressing, VLAN/switch support, and a host-communication strategy. It is not necessary merely to reach routed camera subnets. Do not treat switching to ipvlan as an automatic fix for host isolation. See Docker macvlan.\n4. Use file secrets where the application supports them Compose supports file-backed secrets without Swarm. This is a partial example for an existing compatible PostgreSQL service:\n1 2 3 4 5 6 7 8 9 10 services: postgres: environment: POSTGRES_PASSWORD_FILE: /run/secrets/db_password secrets: - db_password secrets: db_password: file: ./secrets/db_password Supply the actual service image, storage, and other required settings separately. The application must support the file-reading convention; a generic DB_PASSWORD_FILE name does not make every application read it.\nCompose mounts a local secret file into the authorized container. This is not an encrypted secret store and does not prevent a privileged host administrator or authorized container process from reading it. Protect source files, backups, and access permissions. See Compose secrets.\n5. Treat storage and consistency separately Storage Useful property Recovery requirement Bind mount Explicit host path, convenient configuration management Back up selected contents with correct consistency and permissions. Named volume Docker-managed lifecycle and location Preserve application data and document volume mapping and restore. Named volumes do not automatically solve UID/GID mismatches. Bind mounts are resolved on the Docker daemon host, and relative paths depend on the Compose project/file context. A read-only mount limits writes through that mount; it does not freeze other writers. See bind mounts.\nA raw rsync of a changing database directory is not an application-consistent backup. Use the PostgreSQL dump and isolated restore guide and Restic workflow for data preparation and recovery.\n6. Keep operations verifiable Log rotation settings apply per container, not as a total stack-disk guarantee. Review the active logging driver and volume growth. The JSON logging reference explains the options.\nSelect supported image releases and record deployed digests. A major/minor tag is still mutable; a digest identifies exact image content. Updates need deliberate review, recovery preparation, and verification.\nMemory and CPU limits constrain resources; they are not a promise of latency or application readiness. Reservations are not dedicated capacity on a standalone host. Inspect effective runtime settings and test representative workloads before depending on them.\nUse the health-check guide for startup dependencies and recovery behavior. For metrics and logs, use the focused monitoring and Alloy guides instead of a second inconsistent stack here.\n7. Review changes and stop the lab Before a real deployment, record configuration validation, image references, required secrets, network exposure, data locations, backup evidence, and the rollback path. Test recovery instead of assuming that compose up alone reconstructs lost data.\nInspect this lab and stop it from its project directory:\n1 2 docker compose logs --tail=30 web docker compose down The source directory remains available for inspection. For real projects, down \u0026ndash;volumes can delete persistent state; make that a separate, deliberate decision.\n","permalink":"https://blog.gntech.me/posts/2026-05-08-docker-compose-patterns/","summary":"\u003cp\u003eUse a small, complete stack to learn the deployment workflow. Then add only the networking, storage, and secret-handling features the application needs.\u003c/p\u003e\n\u003cp\u003eGroup services by ownership and lifecycle. A Compose project can contain several services, and you can operate on one service without restarting the whole project. Fixed container names are optional and can cause collisions or prevent scaling; service names are usually sufficient.\u003c/p\u003e\n\u003ch2 id=\"verification-status\"\u003eVerification status\u003c/h2\u003e\n\u003ctable\u003e\n  \u003cthead\u003e\n      \u003ctr\u003e\n          \u003cth\u003eArea\u003c/th\u003e\n          \u003cth\u003eEvidence and limits\u003c/th\u003e\n      \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eTechnical review\u003c/td\u003e\n          \u003ctd\u003eOfficial Docker documentation reviewed on October 31, 2026.\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eStatic checks\u003c/td\u003e\n          \u003ctd\u003eBash syntax and embedded Compose YAML checked.\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eRuntime\u003c/td\u003e\n          \u003ctd\u003eDocker unavailable in this review environment; no Compose CLI validation, image pull, HTTP request, or resource test executed.\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eExample scope\u003c/td\u003e\n          \u003ctd\u003eDisposable static HTTP lab on Linux, not a captured production stack.\u003c/td\u003e\n      \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003ch2 id=\"1-start-with-one-complete-project\"\u003e1. Start with one complete project\u003c/h2\u003e\n\u003cp\u003eThis demo uses BusyBox\u0026rsquo;s HTTP server and wget probe, a read-only content mount, loopback publishing, log rotation, and explicit resource limits. Use Bash and have curl installed on the host.\u003c/p\u003e","title":"Docker Compose Patterns for Homelab Stacks"},{"content":"If you run Proxmox VE, you\u0026rsquo;ve got two container runtimes at your fingertips — LXC (built into Proxmox) and Docker (nested in an LXC or VM). Both are \u0026ldquo;containers\u0026rdquo; in the broad sense, but they solve different problems. Picking wrong means performance you don\u0026rsquo;t need or isolation you don\u0026rsquo;t have.\nThis post walks through the decision criteria, backed by configs from an existing homelab running Proxmox 8.x with VLAN segmentation.\n┌─────────────────────────────────────────────────────┐ │ Proxmox Host │ │ ┌──────────┐ ┌──────────┐ ┌──────────────────┐ │ │ │ LXC A │ │ LXC B │ │ Ubuntu VM │ │ │ │ (sys) │ │ (Docker) │ │ ┌──────────────┐ │ │ │ │ │ │ │ │ │ docker0 │ │ │ │ │ sshd, │ │ ctr1 ctr2│ │ │ c1 │ c2 │ c3 │ │ │ │ │ nginx, │ │ │ │ └──────────────┘ │ │ │ │ cron │ │ │ │ │ │ │ └──────────┘ └──────────┘ └──────────────────┘ │ └─────────────────────────────────────────────────────┘ VLAN 20 VLAN 20 VLAN 20 (LAB) (LAB) (LAB) The Core Trade-Off Aspect LXC (Proxmox CT) Docker (in LXC/VM) Kernel Shares host kernel Shares host kernel (Docker LXC) or own (VM) Startup ~1-2 seconds ~200ms per container Overhead Near-zero Minimal (+ Docker daemon) Filesystem Host filesystem (dir/ZFS) Layered images, copy-on-write App lifecycle Systemd services Declarative compose, restart policies Networking Bridge, VLAN, macvlan, routable Bridge (NAT), host, macvlan, ipvlan Storage Bind mounts, mount points, ZFS subvols Volumes, bind mounts, tmpfs Isolation Kernel namespace + AppArmor Kernel namespace + seccomp + AppArmor Snapshot Native Proxmox snapshots (ZFS) Not built-in (need external) Rule of Thumb Use LXC when:\nYou need a full-system container (sshd, cron, syslog, traditional Linux app) The app expects systemd-level process management You want Proxmox-native snapshots and backups Network performance matters (direct bridge/VLAN attachment) Examples: Frigate NVR, Asterisk, Tailscale exit node, bind9, rsyslog Use Docker when:\nThe app is distributed as a Docker image (or you want to build one) You need per-container restart policies and health checks Reproducibility matters (Compose file = complete config) You want to run multiple instances of the same app Examples: Cloudflare Tunnel, Immich, Grafana+Prometheus stack, Nginx reverse proxy Real Configs LXC: Frigate NVR (system container) A Frigate LXC is a good LXC candidate because it needs:\nDirect /dev/dri/renderD128 access for VAAPI Raw USB or RTSP camera access Systemd service management for auto-restart Create in Proxmox:\n1 2 3 4 5 pct create \u0026#34;\u0026lt;CTID\u0026gt;\u0026#34; local:vztmpl/ubuntu-24.04-standard_24.04-2_amd64.tar.zst \\ --storage local-zfs \\ --net0 name=eth0,bridge=vmbr0,vlan=50,ip=dhcp \\ --unprivileged 1 \\ --features nesting=0 Inside the CT:\n1 apt update \u0026amp;\u0026amp; apt install -y docker.io docker-compose-v2 Wait — this says Docker inside LXC. That\u0026rsquo;s right — if you want Docker and you\u0026rsquo;re already in a LXC, you nest Docker inside. Deployment is straightforward through the Proxmox API: create CT, install Docker, docker compose up.\nLXC Without Docker: System Services For system-level services that don\u0026rsquo;t benefit from containerization, skip Docker entirely:\n1 2 3 4 5 6 7 8 9 pct create \u0026#34;\u0026lt;CTID\u0026gt;\u0026#34; local:vztmpl/debian-12-standard_12.7-1_amd64.tar.zst \\ --storage local-zfs \\ --net0 name=eth0,bridge=vmbr0,vlan=10,ip=dhcp \\ --unprivileged 1 # Inside: just install the service pct enter \u0026#34;\u0026lt;CTID\u0026gt;\u0026#34; apt update \u0026amp;\u0026amp; apt install -y nginx-light bind9 systemctl enable --now nginx Simple, direct, minimal overhead.\nDocker in VM: Heavy Isolation For multi-tenant or security-sensitive workloads, nest Docker in a full VM:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 # qm create \u0026lt;VMID\u0026gt; --name docker-host # Then inside the VM: services: cloudflared: image: cloudflare/cloudflared:latest restart: unless-stopped command: tunnel --no-autoupdate run environment: - TUNNEL_TOKEN=\u0026lt;TUNNEL_TOKEN\u0026gt; networks: - proxy nginx: image: nginx:alpine restart: unless-stopped ports: - \u0026#34;80:80\u0026#34; - \u0026#34;443:443\u0026#34; volumes: - ./nginx.conf:/etc/nginx/nginx.conf:ro - ./certs:/etc/nginx/certs:ro networks: - proxy networks: proxy: driver: bridge The VM adds a full kernel (own overlay, iptables, br_netfilter modules) so Docker networking behaves predictably. This is the approach used for Cloudflare Tunnel in this stack.\nChoosing Based on Workload Profile Lightweight, Many Instances → Docker If you need 5+ instances of the same service (e.g., isolated web apps each with their own compose stack), Docker wins. A single LXC with 1GB RAM and 2 cores can run 10-15 Docker containers comfortably.\nSingle Heavy Service → LXC (or VM) For a single service that needs dedicated resources — like a database, media server, or Asterisk — a straight LXC is cleaner. You get standard systemd management, easier debugging (pct enter), and native ZFS snapshots for easy rollback.\nThe Hybrid Approach Most Proxmox homelabs settle on a pattern:\n1 LXC → Docker host (medium, e.g. 4C/8G) └── cloudflared, nginx, grafana, telegraf, 5-10 other microservices 1 LXC → Frigate NVR (2C/4G, GPU passthrough) └── docker + frigate 1 LXC → Main services (1C/2G) └── bind9, nginx, rsyslog, cron 1 VM → Kubernetes/docker-swarm node (4C/8G) └── For learning or production-grade orchestration This keeps the management surface small — you maintain ~4 LXCs/VM instead of 20+ individual machines.\nQuick Start: Docker Host LXC 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 CTID=200 pct create $CTID local:vztmpl/ubuntu-24.04-standard_24.04-2_amd64.tar.zst \\ --storage local-zfs \\ --cores 4 \\ --memory 8192 \\ --swap 1024 \\ --net0 name=eth0,bridge=vmbr0,vlan=20,ip=dhcp \\ --unprivileged 1 \\ --features keyctl=1,nesting=1 pct start $CTID pct push $CTID ~/.ssh/id_ed25519.pub /root/.ssh/authorized_keys ssh root@\u0026#34;\u0026lt;CT-IP\u0026gt;\u0026#34; # Install Docker curl -fsSL https://get.docker.com | sh systemctl enable --now docker # Test docker run --rm hello-world Enable keyctl=1,nesting=1 — without these, Docker inside LXC can\u0026rsquo;t do overlay networks or use kernel keyrings.\nSummary Scenario Pick Single well-known service (Nginx, Bind, NFS) LXC Docker-native app (Grafana, Immich, cloudflared) Docker LXC Multi-app stack with compose Docker LXC High-security / multi-tenant isolation VM + Docker GPU passthrough needed LXC (or VM with PCIe passthrough) Want Proxmox backups \u0026amp; snapshots LXC There\u0026rsquo;s no single right answer. The beauty of Proxmox is that you can mix all three approaches on the same host. Start with what fits the app, and move it later if the constraints change.\n","permalink":"https://blog.gntech.me/posts/2026-05-08-lxc-vs-docker/","summary":"\u003cp\u003eIf you run Proxmox VE, you\u0026rsquo;ve got two container runtimes at your fingertips — LXC (built into Proxmox) and Docker (nested in an LXC or VM). Both are \u0026ldquo;containers\u0026rdquo; in the broad sense, but they solve different problems. Picking wrong means performance you don\u0026rsquo;t need or isolation you don\u0026rsquo;t have.\u003c/p\u003e\n\u003cp\u003eThis post walks through the decision criteria, backed by configs from an existing homelab running Proxmox 8.x with VLAN segmentation.\u003c/p\u003e","title":"Proxmox LXC vs Docker — Picking the Right Container for Every Homelab Job"},{"content":"ZFS snapshots preserve dataset state, but they are only one part of a recovery plan. Pool failure, lost guest configuration, and inconsistent application data need separate protection.\nThis guide uses a small filesystem lab on an existing test pool. It does not create pools, repartition disks, or roll back running guests.\nVerification status Area Evidence and limits Technical review Official OpenZFS and Sanoid references reviewed on October 31, 2026. Static checks Bash examples syntax checked. Runtime No ZFS pool, dataset, snapshot, clone, replication, or recovery operation executed for this revision. Scope Proposed filesystem lab, not a captured host layout or measured performance result. 1. Identify the storage objects Proxmox offers several installation and storage choices; ZFS is not a universal default. Inspect the actual host before changing properties:\n1 2 3 4 zpool status zfs list -o name,type,used,available,mountpoint zfs list -t volume -o name,volsize,volblocksize pvesm status A filesystem dataset holds files. A zvol exposes block storage, commonly used for guest disks. Filesystem recordsize and zvol volblocksize are different settings; changing a parent filesystem\u0026rsquo;s recordsize does not retune existing guest zvol blocks.\nDo not apply blanket 8K or 64K values to every workload. Compression benefits, block-size choices, redundancy, and capacity needs require measurements and a recovery plan. A single-disk pool has no disk redundancy. Separate datasets still share their pool\u0026rsquo;s failure boundary.\nSee the OpenZFS property reference. Preserve properties required by applications; for example, do not disable access-time updates indiscriminately on storage used by PBS garbage collection.\n2. Test a snapshot on a new filesystem Run as root in Bash on a host with an existing test pool and free space. Choose a pool you are authorized to modify. The script creates only uniquely named lab datasets and a mount directory; it stops on error and leaves resources for inspection.\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 ( set -euo pipefail read -r -p \u0026#39;Existing test pool name: \u0026#39; lab_pool test -n \u0026#34;$lab_pool\u0026#34; zpool list \u0026#34;$lab_pool\u0026#34; lab_root=$(mktemp -d /mnt/gntech-zfs-lab.XXXXXX) lab_dataset=\u0026#34;$lab_pool/$(basename \u0026#34;$lab_root\u0026#34;)\u0026#34; lab_clone=\u0026#34;$lab_dataset-recovered\u0026#34; printf \u0026#39;Dataset: %s\\nClone: %s\\nMount directory: %s\\n\u0026#39; \\ \u0026#34;$lab_dataset\u0026#34; \u0026#34;$lab_clone\u0026#34; \u0026#34;$lab_root\u0026#34; zfs create -o mountpoint=\u0026#34;$lab_root/source\u0026#34; -o canmount=on \\ -o readonly=off -o quota=64M \u0026#34;$lab_dataset\u0026#34; printf \u0026#39;marker-before\\n\u0026#39; \u0026gt; \u0026#34;$lab_root/source/check.txt\u0026#34; zfs snapshot \u0026#34;$lab_dataset@before-change\u0026#34; printf \u0026#39;marker-after\\n\u0026#39; \u0026gt; \u0026#34;$lab_root/source/check.txt\u0026#34; zfs clone -o mountpoint=\u0026#34;$lab_root/recovered\u0026#34; -o canmount=on \\ -o readonly=on \u0026#34;$lab_dataset@before-change\u0026#34; \u0026#34;$lab_clone\u0026#34; grep -qx marker-after \u0026#34;$lab_root/source/check.txt\u0026#34; grep -qx marker-before \u0026#34;$lab_root/recovered/check.txt\u0026#34; zfs list -t filesystem,snapshot -r \u0026#34;$lab_dataset\u0026#34; zfs list -o name,origin,mountpoint \u0026#34;$lab_clone\u0026#34; printf \u0026#39;Marker checks passed; inspect %s\\n\u0026#39; \u0026#34;$lab_root\u0026#34; ) Expected evidence: the live file contains the later marker and the recovered clone contains the earlier one. A successful script exit verifies those assertions for your run, not guest or database recovery.\nSnapshots share storage with their source. Clones initially share blocks with their origin snapshot and retain a dependency on it. This example sets the recovery clone read-only to prevent accidental edits during inspection.\nReview snapshot semantics and clones against the installed version.\n3. Keep guest recovery separate Mechanism What it preserves What still needs verification Native filesystem snapshot Dataset state at a point in time Application consistency and independent copies Native guest-disk snapshot Storage-level disk state Guest configuration, related disks, and application consistency PVE-managed guest snapshot Guest snapshot state managed through PVE Storage support, consistency, and recovery procedure PVE backup to PBS or archive storage Selected guest configuration and backup data Included disks/mounts and an isolated restore Replicated dataset Selected data transferred to another location Destination completeness and recovery access A raw snapshot of active database or VM storage is not automatically application-consistent. Coordinate writers or use supported application/guest backup procedures.\nSanoid-created snapshots are not automatically registered as PVE snapshot names. Do not feed their names into qm rollback or modify live guest disks underneath PVE. In-place rollback changes existing data; it is a different operation from copying a file out of a recovery clone.\nFor guest recovery, follow the PBS setup and isolated restore guide and test a new guest ID on an isolated network.\n4. Automate retention only after inspecting scope Sanoid manages snapshot policy; Syncoid supports ZFS replication. Start with one disposable dataset before selecting production parents recursively.\nKeep custom Sanoid policy in its documented configuration file using the supported template syntax. Review the installed package\u0026rsquo;s existing timer or cron job before adding another scheduler. Avoid overlapping parent/child policies unless intentional.\nChoose retention from change rate, capacity, and recovery goals. Inspect actual snapshots and pruning candidates before enabling deletion. Do not replace policy management with a broad grep-and-destroy loop.\nSnapshot space accounting includes shared blocks and clone dependencies. Summing individual snapshot USED values is not a general measure of all retained snapshot space. Inspect dataset snapshot accounting and pool free space as well as individual snapshots.\n5. Treat replication as another recovery workflow Before configuring Syncoid or manual send/receive, identify the exact source datasets, destination pool, access permissions, common snapshot history, and what may be changed or removed on the destination.\nA successful transfer is only one gate. Inspect destination datasets and snapshots, recover a known file there, and test how to access that copy if the original host is unavailable. Recursive replication does not by itself mean “copy the last seven days.”\nKeep destination retention and credentials deliberate. Do not use destructive receive/rollback options to resolve a mismatch without understanding the data that would be lost. A removable pool also needs a documented import, completed-transfer, and export procedure using the correct device identity.\nFor PVE-managed guest replication, use the installed release\u0026rsquo;s supported workflow rather than treating a standalone dataset copy as a complete replacement for guest backup and configuration recovery.\nChoose the replication mechanism Mechanism Scope Check before scheduling PVE storage replication Guest volumes managed by the PVE replication framework Supported storage, target node, matching storage ID, capacity, and replication logs Syncoid Selected standalone ZFS datasets and snapshots Source/target scope, SSH permissions, common history, and destination recovery For PVE-managed guests, consult the storage replication reference. Inspect cluster health and quorum before a recovery exercise. Replication is asynchronous, so changes after the last successful sync may be lost if a node fails. It is not an independently retained backup history.\nFor Syncoid, verify options against the installed version. In particular, \u0026ndash;no-sync-snap uses existing snapshots rather than creating a new synchronization snapshot. Do not assume a bandwidth option or retention policy from another version applies unchanged.\n6. Review and clean up only the lab resources Record the exact dataset, clone, snapshot, and mount directory printed by the lab. Inspect their contents and dependencies before deciding to remove them.\nThe clone depends on the origin snapshot, so cleanup order matters. Delete only the identified disposable clone, then its snapshot, then its source dataset after verifying nothing useful was added. Keep cleanup separate from the recovery test; no recursive destroy or guessed device command is provided here.\nIf the test fails halfway through, inspect the printed names and current mount state before retrying. Do not rerun production storage commands just to obtain a clean lab result.\nContinue reading Backup strategy — Define independent copies and recovery targets. Restic recovery lab — Verify encrypted file recovery. PostgreSQL dumps — Prepare application-consistent database backups. ","permalink":"https://blog.gntech.me/posts/2026-05-08-zfs-snapshot-strategies/","summary":"\u003cp\u003eZFS snapshots preserve dataset state, but they are only one part of a recovery plan. Pool failure, lost guest configuration, and inconsistent application data need separate protection.\u003c/p\u003e\n\u003cp\u003eThis guide uses a small filesystem lab on an existing test pool. It does not create pools, repartition disks, or roll back running guests.\u003c/p\u003e\n\u003ch2 id=\"verification-status\"\u003eVerification status\u003c/h2\u003e\n\u003ctable\u003e\n  \u003cthead\u003e\n      \u003ctr\u003e\n          \u003cth\u003eArea\u003c/th\u003e\n          \u003cth\u003eEvidence and limits\u003c/th\u003e\n      \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eTechnical review\u003c/td\u003e\n          \u003ctd\u003eOfficial OpenZFS and Sanoid references reviewed on October 31, 2026.\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eStatic checks\u003c/td\u003e\n          \u003ctd\u003eBash examples syntax checked.\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eRuntime\u003c/td\u003e\n          \u003ctd\u003eNo ZFS pool, dataset, snapshot, clone, replication, or recovery operation executed for this revision.\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eScope\u003c/td\u003e\n          \u003ctd\u003eProposed filesystem lab, not a captured host layout or measured performance result.\u003c/td\u003e\n      \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003ch2 id=\"1-identify-the-storage-objects\"\u003e1. Identify the storage objects\u003c/h2\u003e\n\u003cp\u003eProxmox offers several installation and storage choices; ZFS is not a universal default. Inspect the actual host before changing properties:\u003c/p\u003e","title":"ZFS on Proxmox — Snapshots, Replication, and Recovery Boundaries"},{"content":"Docker Hub rate-limits anonymous pulls to 100 per 6 hours (200 for authenticated users). In a homelab with frequent rebuilds, multiple hosts, or CI-style workflows, you\u0026rsquo;ll hit that limit fast. Even if you don\u0026rsquo;t, pulling the same nginx:alpine image ten times across ten containers wastes bandwidth and time.\nA local Docker registry mirror fixes both: it acts as a pull-through cache. First pull is from Docker Hub (or any upstream). Every subsequent pull from any host in your lab hits the local cache instead.\nArchitecture ┌──────────────────┐ │ Docker Hub │ │ registry-1.dkr │ └────────┬─────────┘ │ upstream │ (WAN) ┌────────▼─────────┐ │ Registry Mirror │ │ 10.0.20.30:5000 │ │ /var/lib/reg/ │ └────────┬─────────┘ │ local cache │ (LAN, low latency) ┌────────────┐ ┌────▼────┐ ┌────────────┐ │ SRV1 │ │ SRV2 │ │ LXC/VM │ │ (Proxmox) │ │ (node) │ │ (any host) │ └────────────┘ └─────────┘ └────────────┘ One registry instance on your homelab network. Every Docker daemon is configured to use it as a mirror. Images are stored on a local disk or ZFS dataset.\nDeploy the Registry Mirror Docker Compose 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 version: \u0026#34;3.9\u0026#34; services: registry: image: registry:2 container_name: registry-mirror restart: unless-stopped ports: - \u0026#34;5000:5000\u0026#34; environment: REGISTRY_PROXY_REMOTEURL: https://registry-1.docker.io REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /var/lib/registry REGISTRY_STORAGE_DELETE_ENABLED: \u0026#34;true\u0026#34; REGISTRY_PROXY_USERNAME: \u0026#34;\u0026#34; REGISTRY_PROXY_PASSWORD: \u0026#34;\u0026#34; volumes: - ./data:/var/lib/registry If you have a Docker Hub authenticated account (recommended for the 200-pull limit), set REGISTRY_PROXY_USERNAME and REGISTRY_PROXY_PASSWORD. Leave them blank for anonymous (100-pull limit).\nHealth Check Add a health check to catch issues early:\n1 2 3 4 5 6 healthcheck: test: [\u0026#34;CMD\u0026#34;, \u0026#34;wget\u0026#34;, \u0026#34;--no-verbose\u0026#34;, \u0026#34;--tries=1\u0026#34;, \u0026#34;--spider\u0026#34;, \u0026#34;http://localhost:5000/v2/\u0026#34;] interval: 60s timeout: 10s retries: 3 start_period: 20s Launch and Verify 1 docker compose up -d Check it\u0026rsquo;s serving:\n1 2 curl -s http://10.0.20.30:5000/v2/_catalog # → {\u0026#34;repositories\u0026#34;:[]} Empty catalog is normal — nothing cached yet. Pull something through the mirror:\n1 2 3 docker pull localhost:5000/library/alpine:latest # → Pulling from library/alpine # → Digest: sha256:... The registry creates the proxy cache on first pull. library/ prefix is the Docker Hub namespace for official images.\nCheck what\u0026rsquo;s cached:\n1 2 curl -s http://10.0.20.30:5000/v2/_catalog # → {\u0026#34;repositories\u0026#34;:[\u0026#34;library/alpine\u0026#34;]} Configure Docker Daemon Clients Every host that should use the mirror needs its Docker daemon configured. Edit /etc/docker/daemon.json:\n1 2 3 4 { \u0026#34;registry-mirrors\u0026#34;: [\u0026#34;http://10.0.20.30:5000\u0026#34;], \u0026#34;insecure-registries\u0026#34;: [\u0026#34;10.0.20.30:5000\u0026#34;] } The insecure-registries entry is required because the mirror runs on HTTP (no TLS). If you add TLS later, remove it.\nRestart Docker:\n1 systemctl restart docker Verify the mirror is in use:\n1 2 3 docker info | grep -A 1 Mirrors # → Registry Mirrors: # → http://10.0.20.30:5000/ Test the Mirror 1 2 3 4 5 6 7 8 9 # Clear local cache first docker system prune -a # Pull an image — first pull hits Docker Hub (slower) time docker pull nginx:alpine # Remove it and pull again — second pull hits local mirror (fast) docker rmi nginx:alpine time docker pull nginx:alpine The second pull should be significantly faster — local network latency vs WAN.\nStorage Tuning ZFS Dataset If your host uses ZFS, put the registry data on a dedicated dataset with compression:\n1 zfs create -o compression=zstd-3 rpool/registry Update the compose mount to /var/lib/registry:/var/lib/registry or point to the dataset\u0026rsquo;s mount point.\nDisk Space A typical homelab sees:\n10-30 GB after a few weeks of use ~50-80 GB for a year of active development with multiple hosts More if you pull multi-platform images (arm64 + amd64) Set a soft limit with a Docker volume size constraint or monitor with du -sh ./data.\nGC (Garbage Collection) Docker registry doesn\u0026rsquo;t auto-clean blobs when manifests are deleted. To reclaim space:\n1 2 3 4 5 6 7 8 9 10 11 # Delete unused manifests (e.g., old tags) # Registry v2 has no native tag deletion via API — use Docker CLI: docker exec registry-mirror bin/registry garbage-collect \\ /etc/docker/registry/config.yml # For thorough cleanup, stop the container first: docker compose stop docker run --rm -v ./data:/var/lib/registry \\ registry:2 bin/registry garbage-collect \\ /etc/docker/registry/config.yml docker compose start Schedule this monthly via cron:\n0 3 1 * * cd /path/to/registry \u0026amp;\u0026amp; docker compose stop \u0026amp;\u0026amp; \\ docker run --rm -v ./data:/var/lib/registry registry:2 \\ bin/registry garbage-collect /etc/docker/registry/config.yml \u0026amp;\u0026amp; \\ docker compose start Multi-Registry Proxy You\u0026rsquo;re not limited to Docker Hub. Proxy any registry:\n1 2 3 4 5 6 7 8 # For Docker Hub (default) REGISTRY_PROXY_REMOTEURL: https://registry-1.docker.io # For quay.io REGISTRY_PROXY_REMOTEURL: https://quay.io # For ghcr.io (GitHub Container Registry) REGISTRY_PROXY_REMOTEURL: https://ghcr.io Run multiple instances on different ports for different upstream registries:\nService Port Upstream registry-docker 5000 Docker Hub registry-ghcr 5001 ghcr.io registry-quay 5002 quay.io Nginx Reverse Proxy (Optional) If you want a single entry point with path-based routing to multiple registry instances, put Nginx in front:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 upstream docker-hub { server 127.0.0.1:5000; } upstream ghcr { server 127.0.0.1:5001; } server { listen 443 ssl; server_name registry.lab; location /docker/ { rewrite ^/docker(/.*)$ $1 break; proxy_pass http://docker-hub; } location /ghcr/ { rewrite ^/ghcr(/.*)$ $1 break; proxy_pass http://ghcr; } } Then configure each Docker daemon with registry-mirrors: [\u0026quot;https://registry.lab/docker\u0026quot;].\nPerformance Impact In a real homelab with 3-4 Docker hosts and nightly rebuilds:\nMetric Without Mirror With Mirror First pull (nginx:alpine ~24 MB) ~8s (WAN) ~8s (WAN, first time) Second pull (same host) ~8s again ~1s (LAN) Pull across 3 hosts ~24s total ~10s total (1 WAN + 2 LAN) API rate limit hit ~every 2 days with rebuilds rarely Bandwidth saved (weekly) — ~500 MB-2 GB The savings compound with every host and every rebuild. For CI runners or developers iterating on Dockerfiles, the mirror is the difference between \u0026ldquo;waiting for downloads\u0026rdquo; and \u0026ldquo;instant.\u0026rdquo;\nSecurity Notes The mirror as described runs HTTP only. On a segmented homelab VLAN (non-routable, no WAN exposure), this is acceptable. For multi-user or internet-facing setups, add TLS. The official registry image supports REGISTRY_HTTP_TLS_CERTIFICATE and REGISTRY_HTTP_TLS_KEY. The proxy cache stores all pulled images. If you pull private images from a registry, the cached blobs persist until garbage-collected. Gotchas Registry-mirrors only catch docker pull from Docker Hub. If your compose files use image: ghcr.io/org/app:latest, that goes directly to ghcr.io unless you configure a second mirror or use the Nginx proxy approach above.\nDigest pinning bypasses the mirror. If a pull uses a digest (image@sha256:...) instead of a tag, Docker skips the mirror proxy. Use tags for cache hits.\nThe registry does not sync proactively. It only caches what you pull. To pre-warm the cache for a new host, run a batch pull script overnight.\nBloat from multi-arch. If your lab mixes amd64 and arm64 (e.g., Raspberry Pi + Proxmox), the registry stores both architectures. This doubles storage. Target your architecture with --platform pulls.\nMaintenance Summary 1 2 3 4 5 6 7 8 9 10 11 12 # Check disk usage du -sh /opt/registry/data/ # List cached repos curl -s http://10.0.20.30:5000/v2/_catalog | jq . # Garbage collect docker exec registry-mirror bin/registry garbage-collect \\ /etc/docker/registry/config.yml # Verify Docker daemon uses mirror docker info | grep -A 1 Mirrors That\u0026rsquo;s it. One docker compose up -d, one daemon.json edit per host, and your lab stops hammering Docker Hub.\n","permalink":"https://blog.gntech.me/posts/2026-05-08-docker-registry-mirror-cache/","summary":"\u003cp\u003eDocker Hub rate-limits anonymous pulls to 100 per 6 hours (200 for authenticated users). In a homelab with frequent rebuilds, multiple hosts, or CI-style workflows, you\u0026rsquo;ll hit that limit fast. Even if you don\u0026rsquo;t, pulling the same \u003ccode\u003enginx:alpine\u003c/code\u003e image ten times across ten containers wastes bandwidth and time.\u003c/p\u003e\n\u003cp\u003eA local Docker registry mirror fixes both: it acts as a pull-through cache. First pull is from Docker Hub (or any upstream). Every subsequent pull from any host in your lab hits the local cache instead.\u003c/p\u003e","title":"Local Docker Registry Mirror — Cache Images and Speed Up Your Lab"},{"content":"Proxmox Backup Server (PBS) stores deduplicated guest backups for Proxmox VE (PVE). Start with one disposable guest and prove you can recover it before expanding the job to important systems.\nThis guide consolidates the earlier PBS setup articles. Installation uses the official ISO and configuration uses the web interfaces; menu labels can vary by installed release.\nVerification status Area Evidence and limits Technical review Reviewed against the official documentation linked below on October 31, 2026. Examples Names and schedules are proposed lab settings. No capacity or speed results are claimed. Runtime Installation, backups, encryption recovery, sync, and restore have not been executed for this revision. Completion gate Record a successful backup and isolated restore on your own installed versions. 1. Choose the failure boundary Prefer separate hardware and storage when backups must survive loss of the PVE host. A PBS VM is useful for learning, but a VM and datastore on that same physical host share its failure risks. Keep another independent copy before depending on that arrangement.\nDownload the current PBS ISO from Proxmox. Follow the installation guide, check its requirements, and identify the installation disk carefully: the installer overwrites its target. Use a separate backup disk or filesystem and document how it is attached.\nAfter installation, open https://YOUR-PBS-HOST:8007, sign in with the administrator credentials created during installation, and configure updates for the installed release and subscription status. Avoid copying an older Debian repository stanza into a newer installation.\nRecord PBS and PVE versions, datastore location, network access, and recovery credentials somewhere accessible if the PVE host fails.\n2. Prepare and create the datastore Use PBS Administration → Storage/Disks to inspect storage. Creating a filesystem or pool can erase its selected disks; confirm device identities before proceeding. Follow the storage documentation for the chosen filesystem.\nOnce the dedicated filesystem is mounted, add a datastore:\nSetting Example Name main Backing path /mnt/datastore/main on the prepared backup filesystem Capacity Sized from measured backup growth and retention Creating a directory alone does not mount a backup disk. Check that the backing filesystem is present before creating the datastore, and confirm it returns after reboot. Keep access-time behavior compatible with PBS garbage collection; do not apply the old atime=off tuning example.\nLet PBS manage datastore contents. Do not manually remove chunks to make space.\n3. Connect PVE with a dedicated account In PBS, create a dedicated backup@pbs user with a strong password. Grant the DatastoreBackup role on /datastore/main for this example. That ACL path identifies the PBS datastore; it is different from its filesystem path. Use an administrator separately for maintenance.\nIn PVE, open Datacenter → Storage → Add → Proxmox Backup Server:\nField Example Storage ID pbs-main Server Your PBS hostname or address Datastore main Username backup@pbs Password Enter through the interface Fingerprint Verify against the PBS dashboard when using a self-signed certificate Confirm storage becomes available and inspect any authentication errors. Do not disable TLS verification to work around a mismatch. For API tokens, follow the separate privilege rules in the user management guide.\nOptional client-side encryption Use the supported encryption configuration in the PVE PBS storage documentation. Do not substitute raw OpenSSL output for a PBS key file.\nKeep the encryption key and any required recovery secrets outside the PVE cluster. Confirm an encrypted backup can be restored using that saved material before relying on it. A working storage connection alone does not prove encryption or key recovery works.\n4. Back up one disposable guest Create a PVE backup job targeting pbs-main, selecting only the test guest initially. Run it manually and inspect the task log and resulting snapshot. Give the guest a recognizable test file so you can check recovered content later.\nChoose backup mode using the PVE backup documentation: VM and container modes have different behavior and downtime requirements. Guest-agent filesystem freezing can help consistency; databases may also need application-specific backup procedures.\nEach PBS snapshot represents a complete restore point, even though deduplication and incremental transfer reduce repeated work. Savings depend on the workload; measure them after several runs.\n5. Schedule maintenance deliberately Use the datastore\u0026rsquo;s prune, garbage collection, and verification controls. Example starting points:\nTask Lab policy Backup Daily at 00:00, after the manual test succeeds Prune Keep 7 daily, 4 weekly, 3 monthly restore points Garbage collection Daily in a separate maintenance window Verify Verify new backups and periodically reverify older ones Pruning removes snapshot metadata; garbage collection later reclaims eligible unreferenced chunks. The grace period means space may not return immediately. Verification checks stored backup integrity and complements a restore test. See maintenance tasks.\nCheck actual job duration before setting overlapping schedules. Send test notifications and review failed tasks, capacity, and the age of the last successful backup.\n6. Restore into an isolated test guest In PVE, open the PBS storage\u0026rsquo;s backup list, select the test snapshot, and choose Restore. Use an unused guest ID and suitable destination storage. Leave automatic startup disabled.\nBefore starting the restored guest, inspect its network configuration and disconnect its network interfaces or attach them to an isolated test network. Avoid duplicate IP addresses and accidental access to production services.\nBoot the restored guest, check the test file and application behavior, and record the snapshot, versions, elapsed recovery time, and result. If encrypted, include a recovery exercise using the separately saved key. Delete only the disposable restored guest after confirming its ID.\n7. Prepare for host or site loss Document how to reinstall PBS, reconnect surviving storage using Reuse existing datastore, recreate access settings, and reconnect PVE. Preserve server configuration and encryption keys separately; surviving chunks alone do not recreate every setting.\nFor another site, use a second PBS and a supported sync job. Start with a pull job on the destination, registering the source as a remote. Confirm permissions, destination capacity, retention, and whether vanished snapshots should be removed. Test recovery from that copy too. See remotes and sync.\nContinue reading Proxmox bridges and VLANs — Plan access between PVE and PBS. Intel AMT Serial over LAN — Prepare console access for host recovery. Homelab monitoring — Add operational visibility alongside backup task reviews. ","permalink":"https://blog.gntech.me/posts/2026-05-08-proxmox-backup-server-setup/","summary":"One PBS setup workflow: dedicated storage, authenticated PVE access, maintenance jobs, and an isolated restore test.","title":"Proxmox Backup Server: Setup, Retention, and a First Restore Test"},{"content":"If you have fiber-to-the-home (FTTH), your ISP almost certainly gave you a combo ONT/router. It\u0026rsquo;s a locked-down all-in-one box that does GPON optical termination, routing, Wi-Fi, and often double NAT. For a homelab with VLAN segmentation and a proper router like MikroTik, that box is a bottleneck — and you can bypass it entirely.\nThis post covers replacing the ISP ONT/router with a MikroTik router using an SFP GPON stick, covering the hardware, VLAN configurations, PPPoE quirks, and the gotchas that aren\u0026rsquo;t in the marketing material. This is specifically from my experience with Dominican Republic FTTH providers, but the patterns apply to most GPON deployments globally.\nHow GPON FTTH Works GPON (Gigabit Passive Optical Network) uses a passive optical splitter between your home and the ISP\u0026rsquo;s OLT (Optical Line Terminal). The ONT (Optical Network Terminal) on your side converts the optical signal to copper Ethernet.\nMost ISPs ship a combined ONT + router device that does:\nOptical signal termination (GPON) 802.1Q VLAN tagging (Internet VLAN, IPTV VLAN, VoIP VLAN) PPPoE or DHCP client toward the ISP NAT, firewall, and Wi-Fi (usually mediocre) The hack: replace the ONT with an SFP GPON stick (a small transceiver that plugs into your MikroTik\u0026rsquo;s SFP cage) and configure RouterOS to do everything the ISP box was doing — VLAN tagging, PPPoE, and routing.\nBefore: Fiber → ISP ONT/Router (LAN) → MikroTik (behind ISP NAT) After: Fiber → SFP GPON Stick → MikroTik (direct to ISP) The \u0026ldquo;after\u0026rdquo; setup gives you a public IP (or your own public IPv4/IPv6), no double NAT, full control over VLANs, and one fewer power brick.\nHardware Requirements MikroTik With SFP You need a MikroTik device with an SFP cage. Most CCR, RB4011, and some hAP models have one. I\u0026rsquo;m using an RB5009 which has a single SFP+ cage, but an RB4011 with SFP or a CCR2004 works just as well.\nSFP GPON Stick These are small form-factor SFP transceivers that do GPON optical termination. The common models are:\nHuawei MA5671a — Most popular, widely compatible Nokia G-010S-P — Solid, runs cool FS.com GPON-ONU-34-20I — Good alternative if Huawei/Nokia are hard to find Generic \u0026ldquo;stick-es\u0026rdquo; (Ubiquiti-style) — Workable but check OLT compatibility Key specs to look for:\nClass B+ optical budget (minimum, C+ is better for longer splits) OMCCK v1/v2 support — Some ISPs require specific management channel versions SFP 1.25G/2.5G — Most GPON sticks are 1.25G upstream, 2.5G downstream SC/APC connector — Match your ISP\u0026rsquo;s fiber termination (almost always SC/APC for GPON) ⚠️ Warning: GPON sticks need to be provisioned with your ISP\u0026rsquo;s OLT. Some require the ISP\u0026rsquo;s GPON serial number (SN) and/or PLOAM password to authenticate. You\u0026rsquo;ll need to extract this from your existing ONT before swapping.\nExtracting ISP ONT Credentials This is the most critical step. You need at minimum the GPON Serial Number, and sometimes also a PLOAM password and VLAN ID.\nFrom the ISP ONT Web Interface Log into your ISP router. Look for sections like:\nStatus → Device Info → GPON Serial Number WAN → VLAN Configuration ONT Authentication Write these down:\nGPON SN (format: HWTCXXXXXXXX for Huawei, ALCLFXXXXXXX for Alcatel, etc.) PLOAM password (sometimes called GPON password, LOID, or OMCC) VLAN ID for Internet (often 10, 100, 200, 300, or 832 — varies by ISP) PPPoE credentials (username/password for your ISP connection, if applicable) Via SSH/Telnet Some ISP routers expose SSH or Telnet. If available:\n1 2 3 4 5 6 7 # Show GPON info on many Huawei and ZTE ONTs cat /proc/gpon/omcc_gsb # Serial-number commands differ by ONT model and firmware. # Common busybox paths cat /proc/gpon/status omcicli mib get 131 If You Can\u0026rsquo;t Access the ONT Some ISPs lock down their ONTs completely. Options:\nClone the GPON SN to the SFP stick using omcc or vendor tools (Huawei sticks support this via serial console) Call the ISP and ask them to authorize a new ONT by GPON SN Use a managed media converter in front of the MikroTik instead Configuring the SFP GPON Stick Most GPON SFP sticks ship in a neutral/unconfigured state. You need to set the GPON serial number (and optionally the PLOAM password) before they\u0026rsquo;ll authenticate.\nVia Serial Console (Huawei MA5671a) These sticks have a serial console accessible via the SFP connector pins or a dedicated micro-USB port on some models. Connect at 115200 baud:\nLogin: root Password: admin (varies) # Set GPON serial number set sn HWTCXXXXXXXX # Set PLOAM password (if needed) set password xxxxxxxxx # Save and reboot save reboot Via RouterOS — No Serial Cable Some GPON sticks can be configured through RouterOS using the omcc protocol. This is vendor-specific and not universally supported. Check your stick\u0026rsquo;s documentation.\nFor the full DIY approach, you can also use otto — an open-source tool for configuring GPON SFP sticks — running on a Linux host with the stick plugged in. But that requires a separate machine.\n⚠️ If your stick\u0026rsquo;s GPON serial doesn\u0026rsquo;t match what the OLT expects, the VLAN doesn\u0026rsquo;t exist, or the OLT rejects the PLOAM password, the link will show \u0026ldquo;LOS\u0026rdquo; (Loss of Signal) even though the fiber is physically connected. This is the #1 troubleshooting gotcha.\nRouterOS Configuration Once the stick is plugged in and authenticated with the OLT, it shows up as an SFP interface. Here\u0026rsquo;s how to configure it.\nStep 1 — Identify the Interface /interface print where type=\u0026#34;sfp\u0026#34; Look for an interface like sfp-sfpplus1. The link should come up once the GPON authentication succeeds.\nStep 2 — VLAN Tagging Most ISPs require VLAN tagging on the WAN-facing interface. The Internet VLAN ID is usually something like 10, 100, or 832. Find yours from the ISP ONT:\n/interface vlan add interface=sfp-sfpplus1 name=vlan-isp vlan-id=100 If your ISP uses VLAN stacking (QinQ) — two VLAN tags — you need a VLAN interface on top of another VLAN:\n/interface vlan add interface=sfp-sfpplus1 name=vlan-outer vlan-id=10 /interface vlan add interface=vlan-outer name=vlan-inner vlan-id=100 Check your ISP\u0026rsquo;s requirements — some only tag the outer VLAN and expect the inner tag to come from a VLAN-aware bridge.\nStep 3 — PPPoE Client Many FTTH providers use PPPoE for authentication. Create the PPPoE client on the VLAN interface:\n/interface pppoe-client add \\ name=pppoe-isp \\ interface=vlan-isp \\ user=\u0026#34;your_username@isp.com\u0026#34; \\ password=\u0026#34;your_password\u0026#34; \\ add-default-route=yes \\ use-peer-dns=yes DHCP-based ISPs are simpler — just set the VLAN interface as a DHCP client:\n/ip dhcp-client add interface=vlan-isp disabled=no Step 4 — VLAN Filtering Bridge (VLANs Passed Through) If your MikroTik is your core router and you use VLAN filtering, add the physical and bridge interfaces:\n/interface bridge add name=bridge-local vlan-filtering=yes # Add SFP port and LAN ports to the bridge /interface bridge port add bridge=bridge-local interface=sfp-sfpplus1 /interface bridge port add bridge=bridge-local interface=ether2 # Tagged VLANs on bridge /interface bridge vlan add bridge=bridge-local vlan-ids=100 tagged=sfp-sfpplus1,br-vlan-aware untagged=none /interface bridge vlan add bridge=bridge-local vlan-ids=10,20,30,40,50 tagged=sfp-sfpplus1 untagged=none This passes the WAN VLAN (100) and internal VLANs (10-50) cleanly through the bridge without extra config on each port.\nFirewall Rules for the GPON Interface The SFP interface is connected directly to the ISP — it must be locked down tightly.\n# Drop everything from WAN /ip firewall filter add chain=input in-interface=sfp-sfpplus1 action=drop # Allow only established/related /ip firewall filter add chain=input in-interface=sfp-sfpplus1 connection-state=established,related action=accept # Protect the router itself /ip firewall filter add chain=input in-interface=sfp-sfpplus1 protocol=icmp action=accept comment=\u0026#34;Allow ICMP from ISP\u0026#34; # Don\u0026#39;t forward from WAN to LAN unless it\u0026#39;s a return packet /ip firewall filter add chain=forward in-interface=sfp-sfpplus1 out-interface=bridge-local connection-state=established,related action=accept /ip firewall filter add chain=forward in-interface=sfp-sfpplus1 out-interface=bridge-local action=drop And the NAT masquerade so your LAN devices can reach the internet:\n/ip firewall nat add chain=srcnat out-interface=pppoe-isp action=masquerade Common Gotchas \u0026ldquo;Link Is Down / LOS Red\u0026rdquo; GPON serial number mismatch — recheck the SN on the stick PLOAM password wrong or missing OLT requires a specific OMCC version the stick doesn\u0026rsquo;t support Fiber is SC/APC but stick expects SC/UPC (connector mismatch) Speed Cap at ~950 Mbps GPON has a shared medium. Your speed depends on the OLT split ratio and ISP provisioning. The SFP stick itself can handle 1.25G upstream / 2.5G downstream, but most ISPs provision ONTs at ~1G symmetrical. If you\u0026rsquo;re getting exactly 950 Mbps, that\u0026rsquo;s GPON overhead + PPPoE encapsulation — not a bottleneck.\nJitter or Latency Spikes Disable Energy-Efficient Ethernet on the SFP interface if RouterOS supports it Ensure the SFP cage isn\u0026rsquo;t thermally throttling (GPON sticks run hot — 60-70°C is normal) Check for OLT-side congestion during peak hours ISP Blocks Non-Standard ONTs Some ISPs whitelist specific GPON serial number ranges or MAC addresses. Workarounds:\nClone the GPON SN from your original ONT Ask your ISP to provision a new ONT by serial Use a managed GPON media converter (ONT in bridge mode) → MikroTik Ethernet VLAN Tagging Fails After Reboot Double-check that the VLAN interface exists on the correct parent. RouterOS boots interfaces asynchronously, but VLAN interfaces should come up. If they don\u0026rsquo;t, add a delay script:\n/system script add name=pppoe-delay source={ :delay 5 /interface pppoe-client enable pppoe-isp } Add a scheduler to run the script at startup.\nPerformance Comparison Metric ISP ONT/Router MikroTik + GPON Stick Routing throughput ~600-800 Mbps (NAT) Line rate (2.5G capable) Concurrent connections ~4000 (consumer SoC) ~1M+ (MikroTik offload) Latency (inside LAN) 0.3-0.5 ms 0.1-0.2 ms Firewall features Minimal (no VLAN filtering) Full (VLAN filtering, ACLs, mangle) Double NAT? Yes (if you want your own router) No (direct public IP) Power consumption ~10-15W (whole device) ~2-3W (just the SFP stick) The MikroTik already handles VLAN filtering and routing. Adding the GPON stick removes the ISP box from the equation entirely — you get lower latency, full control, and one less failure point.\nSummary Ditching the ISP ONT/router and connecting fiber directly to a MikroTik via SFP GPON stick is a clean upgrade for any homelab with FTTH. The critical steps:\nExtract the GPON serial number and VLAN IDs from your ISP device Provision the SFP stick with the correct GPON SN Configure the VLAN and PPPoE/DHCP client on RouterOS Lock down the ISP-facing interface with firewall rules Test, monitor temps, and handle any OLT compatibility quirks It\u0026rsquo;s not a 10-minute project — especially if you need a serial console to configure the stick or deal with ISP whitelisting — but the result is a single-router, single-SFP homelab with no double NAT and full control at the fiber level.\n","permalink":"https://blog.gntech.me/posts/2026-05-08-gpon-ftth-mikrotik/","summary":"\u003cp\u003eIf you have fiber-to-the-home (FTTH), your ISP almost certainly gave you a combo ONT/router. It\u0026rsquo;s a locked-down all-in-one box that does GPON optical termination, routing, Wi-Fi, and often double NAT. For a homelab with VLAN segmentation and a proper router like MikroTik, that box is a bottleneck — and you can bypass it entirely.\u003c/p\u003e\n\u003cp\u003eThis post covers replacing the ISP ONT/router with a MikroTik router using an SFP GPON stick, covering the hardware, VLAN configurations, PPPoE quirks, and the gotchas that aren\u0026rsquo;t in the marketing material. This is specifically from my experience with Dominican Republic FTTH providers, but the patterns apply to most GPON deployments globally.\u003c/p\u003e","title":"GPON/FTTH With MikroTik — Ditching the ISP Router for Direct Fiber"},{"content":"Cloudflare Tunnel gives you a secure outbound-only connection from your homelab to Cloudflare\u0026rsquo;s edge, proxying public traffic without opening any firewall ports. No pinholes, no DMZ, no exposing your home IP.\nThe usual deployment is a Docker container or a systemd service on a Linux box. But if you have a MikroTik router running RouterOS 7.6+ with container support, you can run cloudflared directly on the router — zero extra hardware, zero extra VMs.\nThis post covers the full setup: enabling containers on MikroTik, pulling and configuring the cloudflared image, routing tunnel traffic through a specific VLAN, and tightening the firewall around the container.\nWhy Run It on the Router Putting Cloudflare Tunnel on the router instead of a separate host has three real advantages:\nNo single-point-of-service host — The tunnel lives on the device that\u0026rsquo;s already always on. If your Docker host reboots for updates, the tunnel stays up. Cleaner VLAN routing — The tunnel container can sit on a dedicated management VLAN (or even the bridge itself) and reach internal services through your existing inter-VLAN firewall rules instead of through Docker networking. One fewer thing to maintain — No VM to patch, no Docker compose file to manage, no extra system to reboot. The container lifecycle is managed by RouterOS. The trade-off: MikroTik containers run on a RouterOS-managed overlay filesystem with limited disk space. You won\u0026rsquo;t have Docker-level tooling (no compose, no docker exec equivalent), and debugging is done through RouterOS commands. For a simple tunnel daemon that just needs to run, this is fine.\nPrerequisites MikroTik router running RouterOS 7.6+ (7.22.x tested here) Container mode supported (ARM64 or x86 — check /system resource for architecture) A USB drive or NVMe for container storage (the router\u0026rsquo;s flash is too small for images) A Cloudflare account with a domain added A Cloudflare Zero Trust Tunnel created (free tier works) The router in this guide is an RB5009UG+S+IN (ARM64, 1 GB RAM), but the steps are the same for any RouterOS device with container support — CCR2004, CCR2116, CHR, etc.\nStep 1 — Prepare Storage Containers need space. RouterOS flash is typically 128–512 MB — fine for the OS, not for container images. Mount external storage:\n# If using USB or NVMe — format and mount /disk format-drive usb1 file-system=ext4 label=CONTAINERS /disk set-smb-enabled disabled=yes /container mounts add dst=/mnt/containers src=usb1 Create a working directory for container data:\n/container envs add key=TZ value=America/Santo_Domingo name=cloudflared-tz # Verify mount /container mounts print Step 2 — Enable Container Mode /container config set ram-high=512M ram-total=1G Set ram-high to the maximum memory you want containers to use (512 MB is plenty for cloudflared). ram-total is the total RAM available to the container subsystem.\nEnable the container feature (idempotent; safe to run again):\n/system device-mode update container=yes This requires a reboot on first enable:\n/system reboot After reboot, confirm containers are available:\n/container config print Expected output:\nram-high: 512.0MiB ram-total: 1.0GiB registry-url: https://registry-1.docker.io Step 3 — Pull the cloudflared Image RouterOS uses Docker registries. Add Docker Hub and pull the cloudflared image:\n/container config set registry-url=https://registry-1.docker.io /container add remote-image=cloudflare/cloudflared:latest interface=vlan10 root-disk=128M mounts=CONTAINERS envlist=cloudflared-tz start-on-boot=yes logging=yes workdir=/mnt/containers/cloudflared Notes:\ninterface=vlan10 — Run the container on a dedicated management/storage VLAN. Adjust to your topology. If you want the tunnel to reach all internal VLANs, attach it to the bridge directly and handle routing through your firewall rules. root-disk=128M — Minimum working size for cloudflared. It uses about 80 MB at rest. start-on-boot=yes — Tunnel comes up automatically after router reboot. envlist — Pass timezone to the container. First pull can take a while depending on your internet speed. Check progress:\n/container print detail The status field goes through: pulling → extracting → stopped (once extracted, not yet started).\nStep 4 — Authenticate the Tunnel Start the container:\n/container start [find where tag=cloudflared] The container won\u0026rsquo;t do much yet — it needs authentication. Enter the container shell:\n/container shell [find where tag=cloudflared] Inside the shell, authenticate and create the tunnel:\n1 2 3 4 5 6 7 # Authenticate — this prints a URL to visit cloudflared tunnel login # Create the tunnel (name it something descriptive) cloudflared tunnel create homelab # This creates ~/.cloudflared/\u0026#34;\u0026lt;tunnel-id\u0026gt;\u0026#34;.json with credentials The cloudflared tunnel login command prints a URL. Visit it in a browser, authorize with your Cloudflare account, and the container saves a cert.pem to its config directory.\nAlternative — manual token creation: If you prefer to avoid the browser flow, create the tunnel from the Cloudflare Zero Trust dashboard:\nGo to Zero Trust → Networks → Tunnels Create a new tunnel, choose Cloudflared Copy the tunnel token Back in the container shell: cloudflared tunnel run --token \u0026lt;token\u0026gt; The tunnel token method is easier to automate and works well here since the router doesn\u0026rsquo;t have a browser.\nStep 5 — Configure DNS and Routing Back in the container shell, set up your DNS entries. Create or edit the config file:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 mkdir -p /mnt/containers/cloudflared cat \u0026gt; /mnt/containers/cloudflared/config.yml \u0026lt;\u0026lt; \u0026#39;EOF\u0026#39; tunnel: \u0026#34;\u0026lt;tunnel-id\u0026gt;\u0026#34; credentials-file: /root/.cloudflared/\u0026#34;\u0026lt;tunnel-id\u0026gt;\u0026#34;.json ingress: # Grafana on internal monitoring host - hostname: monitoring.gntech.me service: https://10.0.20.50:3000 originRequest: noTLSVerify: true # Uptime Kuma on internal monitoring host - hostname: status.gntech.me service: http://10.0.20.50:3001 # Proxmox web UI - hostname: proxmox.gntech.me service: https://10.0.20.30:8006 originRequest: noTLSVerify: true # Catch-all — reject unknown hostnames - service: http_status:404 EOF Wait — this file lives inside the container\u0026rsquo;s overlay. It will vanish on container restart. For persistence, mount the directory from step 1. Create the config on the mounted storage:\n1 2 3 # Inside the container shell, symlink or copy mkdir -p /mnt/containers/config cp /root/.cloudflared/config.yml /mnt/containers/config/ Then configure the container to mount it:\n/container mounts add dst=/root/.cloudflared/config.yml src=/mnt/containers/config/config.yml type=bind /container mount [find where tag=cloudflared] This makes the config file persistent across container restarts and image updates.\nStep 6 — Route DNS to Cloudflare Back on the Cloudflare dashboard, create DNS CNAME records that point to the tunnel:\nCNAME monitoring \u0026lt;tunnel-id\u0026gt;.cfargotunnel.com Proxied (orange cloud) CNAME status \u0026lt;tunnel-id\u0026gt;.cfargotunnel.com Proxied (orange cloud) CNAME proxmox \u0026lt;tunnel-id\u0026gt;.cfargotunnel.com Proxied (orange cloud) Or use the Cloudflare CLI inside the container:\n1 2 3 cloudflared tunnel route dns homelab monitoring.gntech.me cloudflared tunnel route dns homelab status.gntech.me cloudflared tunnel route dns homelab proxmox.gntech.me Step 7 — Start the Tunnel 1 cloudflared tunnel run homelab The first run downloads the cloudflared binary\u0026rsquo;s dependencies and establishes the four outbound connections to Cloudflare edge. Check the logs for:\n+ If using quick tunnels or token: 2026-05-08T00:35:00Z INF Connection \u0026lt;conn-id\u0026gt; registered connIndex=0 location=MIA + Ingress rules configured If everything looks good, configure it to start automatically on container boot by modifying the entrypoint. Create a startup script on the persistent mount:\n1 2 3 4 5 cat \u0026gt; /mnt/containers/config/start.sh \u0026lt;\u0026lt; \u0026#39;SCRIPT\u0026#39; #!/bin/sh cloudflared tunnel run --token \u0026#34;\u0026lt;your-token\u0026gt;\u0026#34; SCRIPT chmod +x /mnt/containers/config/start.sh Then set the container\u0026rsquo;s entrypoint to this script:\n/container set [find where tag=cloudflared] cmd=\u0026#34;/mnt/containers/config/start.sh\u0026#34; /container stop [find where tag=cloudflared] /container start [find where tag=cloudflared] Firewall — Lock Down the Container The container needs outbound access to Cloudflare\u0026rsquo;s IP ranges and nothing else. On RouterOS, isolate it:\n# Allow only Cloudflare edge IPs /ip firewall address-list add address=198.41.128.0/17 list=cloudflare-ipv4 add address=162.158.0.0/15 list=cloudflare-ipv4 add address=104.16.0.0/12 list=cloudflare-ipv4 add address=172.64.0.0/13 list=cloudflare-ipv4 add address=2a06:98c0::/29 list=cloudflare-ipv6 # Firewall filter — container VLAN outbound only to Cloudflare /ip firewall filter add chain=forward src-address=10.0.10.0/24 dst-address-list=cloudflare-ipv4 action=accept comment=\u0026#34;Cloudflared → Cloudflare\u0026#34; add chain=forward src-address=10.0.10.0/24 action=drop comment=\u0026#34;Drop other outbound from container VLAN\u0026#34; The container sits on VLAN 10 (management). It can reach internal services on other VLANs through your existing inter-VLAN forward rules, but all external traffic is restricted to Cloudflare\u0026rsquo;s IP ranges only. No DNS, no NTP, no general internet — just the tunnel.\nStep 9 — Verify and Monitor Check tunnel health from the container:\n1 cloudflared tunnel info homelab Monitor from RouterOS:\n/container print detail /container logs [find where tag=cloudflared] Set up a health check on Uptime Kuma pointing at one of your tunneled services — it confirms both the tunnel and Kuma are working.\nPerformance Notes Latency overhead: Cloudflare Tunnel adds roughly 5–20 ms for a homelab geographically close to a Cloudflare edge. For SSH and web UIs, it\u0026rsquo;s unnoticeable. Throughput: Cloudflared caps around 50–100 Mbps on a MikroTik RB5009 under load. If you need higher throughput, run cloudflared on a dedicated host instead. Memory: cloudflared uses 60–80 MB resident on ARM64 total for the tunnel and ingress routing. The RouterOS container overhead adds about 20 MB. CPU: Near-zero idle. Under load, about 5–10% of one core on RB5009 for a handful of proxied services. The router isn\u0026rsquo;t a proxy server — it\u0026rsquo;s a tunnel endpoint. For heavy traffic (file transfers, video streaming), bypass the tunnel and access services directly on your LAN or through WireGuard.\nUpdating cloudflared RouterOS doesn\u0026rsquo;t have auto-update for containers. Update manually:\n/container remove [find where tag=cloudflared] /container add remote-image=cloudflare/cloudflared:latest interface=vlan10 root-disk=128M mounts=CONTAINERS envlist=cloudflared-tz start-on-boot=yes cmd=\u0026#34;...\u0026#34; workdir=/mnt/containers/cloudflared The pull grabs the latest image automatically. Your persistent config mount survives the rebuild.\nTo stay notified, watch the Cloudflare blog or subscribe to the cloudflared releases on GitHub.\nAlternative: Cloudflare Tunnel via Docker (for Reference) If you\u0026rsquo;d rather run it on a Docker host inside the VM/container, the compose file is simpler:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 services: cloudflared: image: cloudflare/cloudflared:latest container_name: cloudflared restart: unless-stopped command: tunnel run --token \u0026lt;your-token\u0026gt; networks: - internal volumes: - ./config:/root/.cloudflared networks: internal: driver: bridge This is the \u0026ldquo;standard\u0026rdquo; approach documented everywhere. The MikroTik container approach eliminates the host dependency — pick whichever fits your architecture.\nSummary ┌──────────────┐ │ Cloudflare │ │ Edge │ └──────┬───────┘ │ 4x outbound QUIC │ ┌──────┴───────┐ │ MikroTik R1 │ │ (VLAN 10) │ │ ┌─────────┐ │ │ │cloud- │ │ │ │flared │ │ │ └─────────┘ │ └──────┬───────┘ │ inter-VLAN routing │ ┌────────────┼────────────┐ │ │ │ ┌──────┴─────┐ ┌───┴────┐ ┌────┴────┐ │ Proxmox │ │Grafana │ │Uptime │ │ Web UI │ │ │ │ Kuma │ └────────────┘ └────────┘ └─────────┘ Cloudflare Tunnel on MikroTik is an elegant way to expose homelab services without an extra host. The router\u0026rsquo;s already running 24/7 with redundant power and connectivity — offloading the tunnel to it removes a potential point of failure and keeps your VM/container hosts independent.\n","permalink":"https://blog.gntech.me/posts/2026-05-08-mikrotik-cloudflare-tunnel/","summary":"\u003cp\u003eCloudflare Tunnel gives you a secure outbound-only connection from your homelab to Cloudflare\u0026rsquo;s edge, proxying public traffic without opening any firewall ports. No pinholes, no DMZ, no exposing your home IP.\u003c/p\u003e\n\u003cp\u003eThe usual deployment is a Docker container or a systemd service on a Linux box. But if you have a MikroTik router running RouterOS 7.6+ with container support, you can run \u003ccode\u003ecloudflared\u003c/code\u003e directly on the router — zero extra hardware, zero extra VMs.\u003c/p\u003e","title":"Cloudflare Tunnel Inside MikroTik — No Dedicated Proxy VM Needed"},{"content":"WireGuard on MikroTik RouterOS is production-ready as of RouterOS 7.x, and it\u0026rsquo;s dramatically simpler than IPsec or OpenVPN for homelab use. No certificate authorities, no confusing phase 1/phase 2 settings, no userspace daemon eating CPU — just a kernel module, a private key, and a peer config.\nThis post covers two WireGuard topologies running on the same MikroTik router (R1 from the previous deployment post):\nRoad Warrior — remote devices (phone, laptop) connect to the homelab Site-to-Site — two MikroTik routers connected across the internet Both share the same base config and coexist on the same router.\nWireGuard on RouterOS 7 — What Changed RouterOS 6 had WireGuard as a beta package. RouterOS 7 ships it baked in:\n/system package print where name~wireguard If you don\u0026rsquo;t see it, update RouterOS — it\u0026rsquo;s included in all 7.x builds.\nCompared to older VPN options:\nFeature WireGuard IPsec/IKEv2 OpenVPN Lines of config ~5 ~30+ ~40+ Kernel module Yes Yes Userspace (slow) Roaming Built-in Requires config Requires config Noise protocol Yes IKE overhead TLS handshake MTU handling Needs tuning Auto Tends to work CPU on MikroTik Very low Moderate High For a homelab, WireGuard is the right default. IPsec only wins if you need legacy device compatibility or built-in OS clients that don\u0026rsquo;t support WireGuard natively.\nBase Setup — Keys and Interface Generate Keys WireGuard uses Curve25519 key pairs. Generate them on the router:\n/tool wg-genkey This outputs a private key. Derive the public key:\n/tool wg-genkey :set privKey [get] :set pubKey [/tool wg-genkey public-key from-key $privKey] :put $pubKey Or generate both in one command on a Linux machine:\n1 wg genkey | tee private.key | wg pubkey \u0026gt; public.key Save both keys. You\u0026rsquo;ll need the public key for remote peers, and the private key stays in the router config.\nWireGuard Interface /interface wireguard add name=wg-home listen-port=51820 private-key=\u0026#34;\u0026lt;private-key\u0026gt;\u0026#34; No IP assigned to the WireGuard interface just yet — IP assignment is per-peer or you can assign it from a pool.\nScenario 1 — Road Warrior (Remote Client) The canonical use case: you\u0026rsquo;re on your laptop at a coffee shop and need to reach a service running in VLAN 20 (LAB) or VLAN 10 (HOME) without exposing those services to the public internet.\nRouter Config (Server Side) # WireGuard interface /interface wireguard add name=wg-remote listen-port=51820 private-key=\u0026#34;\u0026lt;server-private-key\u0026gt;\u0026#34; # IP on WG interface — this will be the gateway for remote peers /ip address add address=10.200.200.1/24 interface=wg-remote comment=\u0026#34;WG remote pool gateway\u0026#34; # Peer definition: your phone/laptop /interface wireguard peers add \\ interface=wg-remote \\ public-key=\u0026#34;\u0026lt;client-public-key\u0026gt;\u0026#34; \\ allowed-address=10.200.200.2/32 \\ comment=\u0026#34;Phone - Pixel 9\u0026#34; allowed-address is critical. On the server side, it tells WireGuard which IPs this peer is allowed to use. You assign each remote device a unique /32 address from the pool.\nFirewall — Allow WireGuard on WAN /ip firewall filter add \\ chain=input \\ protocol=udp \\ dst-port=51820 \\ in-interface-list=WAN \\ action=accept \\ comment=\u0026#34;Allow WireGuard UDP\u0026#34; The WAN interface list is set up in the R1 deployment — it includes pppoe-out1. If you have multiple WAN interfaces (failover), add them all.\nNAT for Remote Clients Remote peers connect to the tunnel IP (10.200.200.1), but they need to reach the actual LAN subnets. Add a source NAT rule on the WireGuard interface:\n/ip firewall nat add \\ chain=srcnat \\ src-address=10.200.200.0/24 \\ dst-address=10.0.0.0/16 \\ action=srcnat \\ comment=\u0026#34;NAT remote WG clients to LAN\u0026#34; This translates the remote peer\u0026rsquo;s WG IP (10.200.200.x) to the router\u0026rsquo;s LAN IP when they talk to local subnets. Without this, return traffic wouldn\u0026rsquo;t know how to route back.\nAllowing Access to Target Subnets By default, the firewall drops inter-VLAN traffic (from the R1 config). Add an allow rule for the WireGuard pool to reach specific VLANs:\n/ip firewall filter add \\ chain=forward \\ src-address=10.200.200.0/24 \\ dst-address=10.0.10.0/24 \\ action=accept \\ comment=\u0026#34;Allow WG clients → HOME (10)\u0026#34; /ip firewall filter add \\ chain=forward \\ src-address=10.200.200.0/24 \\ dst-address=10.0.20.0/24 \\ action=accept \\ comment=\u0026#34;Allow WG clients → LAB (20)\u0026#34; For full LAN access (less secure, more convenient):\n/ip firewall filter add \\ chain=forward \\ src-address=10.200.200.0/24 \\ dst-address=10.0.0.0/16 \\ action=accept \\ comment=\u0026#34;Allow WG clients → all VLANs\u0026#34; Client Config (Phone / Laptop) On the client device (WireGuard app on iOS/Android, or wg-quick on Linux), create a peer config:\n1 2 3 4 5 6 7 8 9 10 [Interface] PrivateKey = \u0026lt;client-private-key\u0026gt; Address = 10.200.200.2/32 DNS = 10.0.20.1 [Peer] PublicKey = \u0026lt;server-public-key\u0026gt; Endpoint = \u0026lt;public-ip\u0026gt;:51820 AllowedIPs = 10.0.0.0/16, 10.200.200.0/24 PersistentKeepalive = 25 AllowedIPs on the client tells the OS which destinations should be routed through the tunnel. Setting it to 10.0.0.0/16 routes all homelab traffic through WireGuard while internet traffic goes direct (split tunnel).\nIf you want all traffic through the VPN (full tunnel, everything goes through your home connection):\n1 AllowedIPs = 0.0.0.0/0, ::/0 PersistentKeepalive = 25 keeps the NAT binding alive on cellular networks. Without it, the router won\u0026rsquo;t be able to initiate connections back to the client (the endpoint is behind CGNAT or carrier-grade NAT).\nScenario 2 — Site-to-Site (Two MikroTiks) This connects two MikroTik routers across different physical locations — say a primary and a secondary site, each with their own subnets.\nTopology Site A (Primary) Site B (Secondary) ┌──────────────┐ ┌──────────────┐ │ MikroTik A │ VPN Tunnel │ MikroTik B │ │ 10.0.0.0/16 │ ◄───────────► │ 172.16.0.0/24│ │ WAN: \u0026lt;IP-A\u0026gt; │ 10.200.201.0 │ WAN: \u0026lt;IP-B\u0026gt; │ └──────────────┘ └──────────────┘ Site A Config (Primary) # WireGuard interface /interface wireguard add name=wg-site listen-port=51821 private-key=\u0026#34;\u0026lt;site-a-private\u0026gt;\u0026#34; # Tunnel IP /ip address add address=10.200.201.1/30 interface=wg-site # Peer — Site B /interface wireguard peers add \\ interface=wg-site \\ public-key=\u0026#34;\u0026lt;site-b-public\u0026gt;\u0026#34; \\ allowed-address=10.200.201.2/32,172.16.0.0/24 \\ endpoint-address=\u0026lt;site-b-public-ip\u0026gt; \\ endpoint-port=51821 \\ persistent-keepalive=25s \\ comment=\u0026#34;Site B (Secondary)\u0026#34; The key difference from road warrior: allowed-address on the server side includes the remote site\u0026rsquo;s LAN subnet (172.16.0.0/24). This tells WireGuard to route traffic destined for that subnet through this peer.\nSite B Config (Secondary) # WireGuard interface /interface wireguard add name=wg-site listen-port=51821 private-key=\u0026#34;\u0026lt;site-b-private\u0026gt;\u0026#34; # Tunnel IP /ip address add address=10.200.201.2/30 interface=wg-site # Peer — Site A /interface wireguard peers add \\ interface=wg-site \\ public-key=\u0026#34;\u0026lt;site-a-public\u0026gt;\u0026#34; \\ allowed-address=10.200.201.1/32,10.0.0.0/16 \\ endpoint-address=\u0026lt;site-a-public-ip\u0026gt; \\ endpoint-port=51821 \\ persistent-keepalive=25s \\ comment=\u0026#34;Site A (Primary)\u0026#34; Notice the symmetry: Site A allows 172.16.0.0/24, Site B allows 10.0.0.0/16. Each side advertises its own LAN subnets in allowed-address on the opposite peer.\nFirewall — Site-to-Site Add WAN allow rules for the site-to-site port (51821):\n/ip firewall filter add \\ chain=input \\ protocol=udp \\ dst-port=51821 \\ in-interface-list=WAN \\ action=accept \\ comment=\u0026#34;Allow WG site-to-site\u0026#34; And forward rules between the tunnel and the LAN subnets:\n# Site A: allow site B subnets into Site A LAN /ip firewall filter add \\ chain=forward \\ src-address=172.16.0.0/24 \\ dst-address=10.0.0.0/16 \\ in-interface=wg-site \\ out-interface=bridge-trunk \\ action=accept \\ comment=\u0026#34;Allow Site B → Site A LAN\u0026#34; # Site A: allow responses back /ip firewall filter add \\ chain=forward \\ src-address=10.0.0.0/16 \\ dst-address=172.16.0.0/24 \\ in-interface=bridge-trunk \\ out-interface=wg-site \\ action=accept \\ comment=\u0026#34;Allow Site A LAN → Site B\u0026#34; Route Table WireGuard handles routing internally via the peer\u0026rsquo;s allowed-address — you don\u0026rsquo;t need static routes for the tunnel itself. But verify the routing table:\n/ip route print where dst-address~\u0026#34;172.16\u0026#34; The route should show gateway=wg-site with no explicit next-hop IP. WireGuard resolves it from the peer config.\nFirewall Best Practices Default Drop Ingress Never skip the default drop rules on the forward chain. WireGuard connections spawn new forward sessions that the firewall must evaluate:\n/ip firewall filter add chain=forward action=drop comment=\u0026#34;Default drop forward\u0026#34; Place it at the bottom of the forward chain. WireGuard accept rules go above it.\nFastTrack and WireGuard FastTrack (connection tracking bypass) doesn\u0026rsquo;t work on WireGuard interfaces — WireGuard encrypts every packet, so there\u0026rsquo;s nothing to fast-track. Don\u0026rsquo;t worry about this; WireGuard\u0026rsquo;s kernel module is fast enough that FastTrack would save negligible CPU anyway on a homelab router.\nIf you have FastTrack rules, make sure they don\u0026rsquo;t accidentally match WireGuard traffic:\n/ip firewall filter add chain=forward action=fasttrack-connection \\ connection-state=established,related \\ in-interface-list=LAN \\ out-interface-list=LAN WireGuard traffic goes over WAN — it won\u0026rsquo;t match this rule.\nPerformance Tuning MTU WireGuard\u0026rsquo;s default MTU is 1420 (standard 1500 minus 60 bytes WG overhead, minus 20 IP header). If your ISP uses PPPoE (another 8 bytes overhead), the MTU needs to be lower:\n/interface wireguard set wg-remote mtu=1412 /interface wireguard set wg-site mtu=1412 1412 = 1500 - 60 (WG) - 20 (IP) - 8 (PPPoE). If you\u0026rsquo;re behind CGNAT or use IPIP tunneling, go lower.\nTest MTU from the Router /ping 10.0.10.1 do-not-fragment do-not-fragment size=1400 Increase the size until it fails. The maximum successful size + 28 (IP+ICMP headers) is your path MTU. Set the WireGuard interface MTU to that value minus 80 (60 bytes WG + 20 bytes outer IP).\nKeepalive Timing Road warrior (mobile clients): 25 seconds — cellular NAT timeouts are often 30-60 seconds Site-to-site (fixed endpoints): 10-15 seconds — more aggressive to detect link flips faster, or disable keepalive if both endpoints have stable public IPs If both sites have static public IPs, you can drop persistent-keepalive entirely for the site-to-site link. WireGuard will still reconnect when traffic flows.\nMultiple Streams WireGuard is single-threaded on the router\u0026rsquo;s CPU. RouterOS 7 uses a single worker for the WireGuard crypto. On an ARM64 MikroTik like the E62iUGS, this isn\u0026rsquo;t a bottleneck for 100-200 Mbps. For gigabit WireGuard, you\u0026rsquo;d need a faster CPU (CCR series or x86 RouterOS).\nCheck CPU usage:\n/system resource cpu print If WireGuard pushes CPU past 70-80%, consider:\nIPsec with AES-NI hardware offload (if your router supports it) A dedicated WireGuard endpoint (e.g., a small LXC running wg-quick) Lowering the MTU (less overhead per packet, more packets per byte → more CPU per byte) Real-world throughput on the E62iUGS with WireGuard is roughly 250-350 Mbps — adequate for remote access to a homelab, but not for site-to-site bulk transfer at line rate.\nDNS for Remote Clients Distribute a meaningful DNS server to remote peers. In the road warrior client config:\n1 DNS = 10.0.20.1 If your router runs the DNS server (or forwards to AdGuard/Pi-hole), remote clients get ad-blocking and local hostname resolution through the tunnel. Inside the homelab, nas.lab resolves to a local IP. Outside, it resolves through the tunnel.\nFor the site-to-site setup, configure DNS forwarding on each router:\n/ip dns set servers=1.1.1.1,8.8.8.8 allow-remote-requests=no Local DNS on each site resolves its own subnets. Cross-site DNS uses static entries or a shared DNS server reachable through the tunnel.\nSplit Tunnel vs Full Tunnel Split Tunnel (Recommended for Road Warrior) Only homelab traffic goes through the VPN. Internet traffic goes direct:\nAllowedIPs = 10.0.0.0/16, 10.200.200.0/24 Pros:\nNo bandwidth bottleneck at home Streaming services work from the client\u0026rsquo;s real IP Lower latency for internet browsing Cons:\nTraffic to the internet doesn\u0026rsquo;t benefit from the homelab\u0026rsquo;s ad-blocking DNS The client\u0026rsquo;s public IP is exposed to websites (privacy tradeoff) Full Tunnel All traffic routes through the home connection:\nAllowedIPs = 0.0.0.0/0, ::/0 Pros:\nAd-blocking and DNS filtering applies to all traffic Client\u0026rsquo;s public IP is the home\u0026rsquo;s IP (useful for geo-unblocking) Network monitoring sees all traffic Cons:\nThroughput is limited to the home upload speed (often 10-50 Mbps) Latency adds a round trip to the home link Home internet outage means no internet at all for the client I run split tunnel for phones (low bandwidth, cellular NAT is unpredictable) and full tunnel for laptops when connected to untrusted WiFi (hotels, airports).\nMonitoring and Debugging Peer Status Check if a peer is connected:\n/interface wireguard peers print detail where interface=wg-remote Look for current-endpoint-address: — if it\u0026rsquo;s populated, the peer is connected. If not, check firewall rules and NAT.\nConnection Tracking See active WireGuard sessions:\n/ip firewall connection print where protocol=udp and dst-port=51820 A single established,related connection is expected per peer.\nLogging Log WireGuard handshake failures:\n/system logging add topics=wireguard,info action=memory /log print where topics~\u0026#34;wireguard\u0026#34; Common errors:\nLog Message Likely Cause handshake failed Wrong public key on either side noise_connection_reset Endpoint IP/port changed (roaming) packet too big MTU mismatch — lower MTU on the interface allowed-address mismatch Peer sent from an IP not in its allowed-address list Speed Test Through Tunnel /tool bandwidth-test \\ address=10.0.10.100 \\ user=admin \\ password=\u0026lt;pass\u0026gt; \\ direction=both \\ protocol=udp \\ duration=10s \\ local-address=10.200.200.1 Run this from a client on the WG subnet to a server inside the LAN. Compare to a direct (non-WG) speed test to measure overhead. Expect ~3-5% throughput loss from WireGuard encryption.\nMultiple Peers on One Interface You can stack peers on the same WireGuard interface. Each road warrior device gets its own /32:\n/interface wireguard peers add interface=wg-remote public-key=\u0026#34;\u0026lt;pub1\u0026gt;\u0026#34; \\ allowed-address=10.200.200.2/32 comment=\u0026#34;Phone - Pixel 9\u0026#34; /interface wireguard peers add interface=wg-remote public-key=\u0026#34;\u0026lt;pub2\u0026gt;\u0026#34; \\ allowed-address=10.200.200.3/32 comment=\u0026#34;Laptop - ThinkPad\u0026#34; /interface wireguard peers add interface=wg-remote public-key=\u0026#34;\u0026lt;pub3\u0026gt;\u0026#34; \\ allowed-address=10.200.200.4/32 comment=\u0026#34;Tablet - iPad\u0026#34; They all share the same listening port (51820) and tunnel interface. WireGuard demultiplexes by public key.\nThe site-to-site peer uses a separate interface (wg-site on port 51821) for clarity, but it could share wg-remote with different allowed-address ranges. Separate interfaces keep the config readable and let you set different MTUs or firewall rules per context.\nSecurity Notes Key Management Keys are everything in WireGuard. Lose the private key, you lose the connection. Keep a backup in a password manager or encrypted file. Rotate keys periodically. Add a new peer with the new key, remove the old one, update clients. No service interruption. Exposure Don\u0026rsquo;t expose WireGuard on a non-standard port thinking it\u0026rsquo;s security through obscurity — standard port (51820) is fine. The protocol is authenticated and encrypted. Set listen-port to a high port (51820-51900) if you want to avoid noisy scanning, but don\u0026rsquo;t rely on it for security. Rate Limiting RouterOS doesn\u0026rsquo;t have native rate limiting on WireGuard handshakes. If you see repeated handshake failures from unknown peers (scanning), block them with a firewall filter:\n/ip firewall filter add \\ chain=input \\ protocol=udp \\ dst-port=51820 \\ src-address-list=wg-scanners \\ action=drop \\ comment=\u0026#34;Drop known WG scanners\u0026#34; /ip firewall filter add \\ chain=input \\ protocol=udp \\ dst-port=51820 \\ action=add-src-to-list \\ connection-state=new \\ src-address-list-timeout=1h \\ src-address-list=wg-scanners \\ comment=\u0026#34;Rate limit WG handshakes (adjust as needed)\u0026#34; In practice, this is overkill for a homelab. I\u0026rsquo;ve never seen a WireGuard handshake flood on residential connections.\nFull Config Summary Road Warrior (Server Side — R1) # Interface /interface wireguard add name=wg-remote listen-port=51820 private-key=\u0026#34;\u0026lt;priv\u0026gt;\u0026#34; # IP pool gateway /ip address add address=10.200.200.1/24 interface=wg-remote # Peers (one per device) /interface wireguard peers add interface=wg-remote public-key=\u0026#34;\u0026lt;pub\u0026gt;\u0026#34; \\ allowed-address=10.200.200.2/32 comment=\u0026#34;Phone\u0026#34; # Firewall — WAN input /ip firewall filter add chain=input protocol=udp dst-port=51820 \\ in-interface-list=WAN action=accept comment=\u0026#34;Allow WG WAN\u0026#34; # NAT for LAN access /ip firewall nat add chain=srcnat src-address=10.200.200.0/24 \\ dst-address=10.0.0.0/16 action=srcnat # Forward rules /ip firewall filter add chain=forward src-address=10.200.200.0/24 \\ dst-address=10.0.10.0/24 action=accept comment=\u0026#34;WG → HOME\u0026#34; /ip firewall filter add chain=forward src-address=10.200.200.0/24 \\ dst-address=10.0.20.0/24 action=accept comment=\u0026#34;WG → LAB\u0026#34; /ip firewall filter add chain=forward src-address=10.200.200.0/24 \\ dst-address=10.0.50.0/24 action=accept comment=\u0026#34;WG → CCTV\u0026#34; Site-to-Site (Both Sides — Partial) Site A:\n/interface wireguard add name=wg-site listen-port=51821 private-key=\u0026#34;\u0026lt;a-priv\u0026gt;\u0026#34; /ip address add address=10.200.201.1/30 interface=wg-site /interface wireguard peers add interface=wg-site public-key=\u0026#34;\u0026lt;b-pub\u0026gt;\u0026#34; \\ allowed-address=10.200.201.2/32,172.16.0.0/24 \\ endpoint-address=\u0026lt;b-public-ip\u0026gt; endpoint-port=51821 persistent-keepalive=25s Site B:\n/interface wireguard add name=wg-site listen-port=51821 private-key=\u0026#34;\u0026lt;b-priv\u0026gt;\u0026#34; /ip address add address=10.200.201.2/30 interface=wg-site /interface wireguard peers add interface=wg-site public-key=\u0026#34;\u0026lt;a-pub\u0026gt;\u0026#34; \\ allowed-address=10.200.201.1/32,10.0.0.0/16 \\ endpoint-address=\u0026lt;a-public-ip\u0026gt; endpoint-port=51821 persistent-keepalive=25s Why Not IPsec? WireGuard replaces IPsec in most homelab scenarios because:\nConfiguration complexity — IPsec on RouterOS requires multiple proposal, policy, and peer entries. WireGuard is one interface + one peer per device. Handshake overhead — IPsec IKEv2 uses multiple round trips for SA negotiation. WireGuard uses a single round trip (two messages) using the Noise protocol. Roaming — WireGuard handles IP changes seamlessly. IPsec needs MOBIKE support (not always present on RouterOS). CPU usage — WireGuard\u0026rsquo;s kernel module is simpler than IPsec\u0026rsquo;s stack, resulting in lower CPU on the same hardware. IPsec still wins when:\nYou need interoperability with non-WireGuard VPN gateways (AWS VPN, corporate VPNs) Client OS has built-in IPsec but not WireGuard (some older iOS/Android versions) You need NAT traversal below UDP (IPsec over TCP, not possible with WireGuard) For homelab VPN: WireGuard. For corporate VPN requirements: IPsec. For legacy appliances: neither — use something that speaks WireGuard if possible.\nRecovery — When WireGuard Breaks \u0026ldquo;Peer disconnected\u0026rdquo; — no endpoint address shown /interface wireguard peers print detail If current-endpoint-address is empty:\nClient can\u0026rsquo;t reach the router\u0026rsquo;s WAN IP/port Check firewall on both sides Check ISP is not blocking UDP port 51820 (rare, but some mobile carriers do) Try changing to port 443 or 53 (common allowed UDP ports) \u0026ldquo;Handshake failed\u0026rdquo; — peer keeps trying Verify public keys match on both sides Check that allowed-address on the server includes the client\u0026rsquo;s tunnel IP Restart the WireGuard interface: /interface wireguard disable wg-remote; /interface wireguard enable wg-remote Check system logs: /log print where topics~\u0026quot;wireguard\u0026quot; \u0026ldquo;Can reach tunnel IP, can\u0026rsquo;t reach LAN\u0026rdquo; Forward firewall rules: is there a dst-address=10.0.0.0/16 allow rule? NAT: is srcnat active for the WG subnet? Duplicate route: /ip route print — ensure there\u0026rsquo;s no conflicting route From the router: /ping 10.0.10.1 src-address=10.200.200.1 — if this fails, it\u0026rsquo;s a routing/NAT issue, not WireGuard MTU Issues Symptoms: large transfers hang, small transfers work, SSH connects but scp/ping with large size times out.\nFix:\n/interface wireguard set wg-remote mtu=1350 Then test incrementally:\n/ping 10.0.10.1 size=1300 do-not-fragment Summary Decision Recommendation VPN protocol WireGuard — simpler, faster, less CPU Road warrior subnet 10.200.200.0/24 Site-to-site subnet 10.200.201.0/30 Split vs full tunnel Split for phones, full for untrusted WiFi Listen port 51820 (road warrior), 51821 (site-to-site) MTU with PPPoE 1412 (or lower if behind CGNAT) Keepalive 25s (mobile), 10-15s (site-to-site) WireGuard on RouterOS 7 is the fastest path to a reliable homelab VPN. The config is minimal, the security model is sound, and you can have road warrior access working in under 10 commands from a clean router.\n","permalink":"https://blog.gntech.me/posts/2026-05-08-mikrotik-wireguard-vpn/","summary":"\u003cp\u003eWireGuard on MikroTik RouterOS is production-ready as of RouterOS 7.x, and it\u0026rsquo;s dramatically simpler than IPsec or OpenVPN for homelab use. No certificate authorities, no confusing phase 1/phase 2 settings, no userspace daemon eating CPU — just a kernel module, a private key, and a peer config.\u003c/p\u003e\n\u003cp\u003eThis post covers two WireGuard topologies running on the same MikroTik router (R1 from the \u003ca href=\"/posts/r1-router-deployment/\"\u003eprevious deployment post\u003c/a\u003e):\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e\u003cstrong\u003eRoad Warrior\u003c/strong\u003e — remote devices (phone, laptop) connect to the homelab\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eSite-to-Site\u003c/strong\u003e — two MikroTik routers connected across the internet\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eBoth share the same base config and coexist on the same router.\u003c/p\u003e","title":"MikroTik WireGuard — Site-to-Site and Road Warrior VPN Setup"},{"content":"A Proxmox bridge connects guest interfaces to a network. VLAN filtering determines which VLANs traverse its ports. Routing and firewall policy determine which subnets may communicate.\nStart by documenting what the physical switch sends to the host. A correct guest VLAN tag cannot fix a mismatch on the uplink.\nVerification status Area Evidence and limits Technical review Reviewed against the official references below on October 31, 2026. Shell examples Bash syntax checked; no host networking commands executed. Configuration Illustrative single-uplink IPv4 lab, not a captured production configuration. Runtime Application, rollback, DHCP, guest traffic, and firewall behavior remain untested for this revision. 1. Identify the management path These two designs are different:\nSwitch port toward PVE Host management placement Expected wire traffic VLAN 99 tagged IP on vmbr0.99 Management frames carry VLAN 99. VLAN 99 untagged/native IP on vmbr0 in a matching bridge/PVID design Management frames arrive without an 802.1Q tag. A switch PVID classifies incoming untagged frames. Its egress VLAN membership controls whether transmitted frames are tagged. Both directions must agree with the host design.\nFor example, a port delivering VLAN 99 untagged and guest VLANs tagged must not be treated as a fully tagged management trunk. Decide which design you have before changing interfaces. See MikroTik\u0026rsquo;s bridge VLAN table reference.\nThe configuration below uses tagged management VLAN 99 throughout. It is not a migration recipe for an existing untagged management network.\n2. Inspect and preserve the current configuration Have a working local or out-of-band console before changing the interface used by SSH or the web UI. Test console access first.\nRun this read-only inventory on PVE:\n1 2 3 4 5 6 ip -br link ip -br address ip route bridge link show bridge vlan show cat /etc/network/interfaces Record NIC names, management address, gateway, active VLAN membership, and switch-port settings. Inspect any included interface files and pending PVE network changes too.\nBefore editing, preserve the main file in a unique directory:\n1 2 3 4 5 6 ( set -euo pipefail backup_dir=$(mktemp -d /root/pve-network-backup.XXXXXX) cp -a /etc/network/interfaces \u0026#34;$backup_dir/interfaces\u0026#34; printf \u0026#39;Saved main interfaces file to %s\\n\u0026#39; \u0026#34;$backup_dir/interfaces\u0026#34; ) Copy included configuration files and relevant switch settings separately. Keep the backup path available through the console.\n3. Review a tagged-management example This example assumes:\nOne physical uplink named enp3s0. Switch egress and ingress configured for tagged VLANs 10, 50, and 99. Management subnet 10.0.99.0/24, with the unused host address 10.0.99.30 and router 10.0.99.1. No cluster, bond, SDN, extra uplink, or IPv6 requirements in this illustrative file. Stage it as a separate candidate; this command does not replace the active configuration:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 cat \u0026gt; /root/pve-network-tagged-example.interfaces \u0026lt;\u0026lt;\u0026#39;EOF\u0026#39; auto lo iface lo inet loopback iface enp3s0 inet manual auto vmbr0 iface vmbr0 inet manual bridge-ports enp3s0 bridge-stp off bridge-fd 0 bridge-vlan-aware yes bridge-vids 10 50 99 auto vmbr0.99 iface vmbr0.99 inet static address 10.0.99.30/24 gateway 10.0.99.1 EOF Compare the candidate with the real host and retain required settings. Do not overwrite a working file with this reduced example. Review the PVE networking documentation for your installed version.\nThe host needs a VLAN interface only when the host itself participates at layer 3 on that VLAN. Guests on VLAN 10 or 50 do not require host IP addresses on those networks.\nAn IP directly on a bridge is valid for an untagged management design. VLAN IDs and IP subnets are separate concepts; this example uses similar numbers only to make the plan easier to read.\n4. Put one disposable guest on an access VLAN Use PVE\u0026rsquo;s guest network interface editor to select bridge vmbr0 and VLAN tag 10. Preserve existing MAC, firewall, rate, and other interface settings.\nFor a normal single-VLAN guest, leave the guest OS interface untagged. The bridge assigns the VLAN at its virtual port; the physical uplink carries the tag.\nThe same access-port principle applies to an LXC veth with tag=10. It does not mean the container receives a VLAN trunk. Configure its IP or DHCP separately. See PVE container documentation.\nA guest that must process multiple VLANs needs an explicit trunk design with allowed VLANs and matching guest configuration. Simply omitting the access tag is not a complete trunk configuration. Keep that separate from this first test.\n5. Apply deliberately and verify the path Use the supported PVE network configuration workflow for the installed release, with console access and the saved configuration ready. Review pending changes before applying; avoid bringing down the management interface from the only remote session.\nAfter application, inspect:\n1 2 3 4 ip -br address ip route show default bridge link show bridge vlan show Check uplink and guest-port membership in the full VLAN table. A bridge-device-only query does not show every attached port. Consult the bridge command reference for the difference between PVID and untagged egress flags.\nVerify management reachability from a separate workstation. Inside the disposable guest, test DHCP or its intended static address, gateway reachability, DNS resolution, and the actual service connection. A failed ping alone does not distinguish a missing route from ICMP filtering.\nTest both allowed and denied inter-VLAN connections against the intended router/firewall rules. A VLAN-aware bridge is not an inter-VLAN firewall. A bridge without a physical uplink can switch between local guests, but needs routing or another uplink path to reach external networks.\n6. Recover and extend only after the baseline works If management access fails, use the console to compare active addresses, routes, and VLAN membership with the saved configuration and switch settings. Restore the known-good host file and any changed switch configuration, then apply through the supported local workflow. Recheck connectivity before leaving the console.\nKeep the old management path available until the replacement is verified. Avoid changing VLAN tagging, IP addressing, switch membership, and firewall rules all at once.\nBonding, multiple uplinks, OVS, and cluster networking need their own plans. LACP requires matching switch configuration; a single flow does not automatically gain the sum of all link speeds. OVS supports VLANs and does not inherently require one bridge per VLAN. Do not add a second parallel bridge uplink without a loop-prevention design.\nContinue reading Cloud-init VM templates — Deploy a test guest after confirming its network path. Intel AMT Serial over LAN — Prepare console recovery. PBS setup and first restore — Plan recovery for the guests on this network. ","permalink":"https://blog.gntech.me/posts/2026-05-08-proxmox-networking-bridges-vlans/","summary":"\u003cp\u003eA Proxmox bridge connects guest interfaces to a network. VLAN filtering determines which VLANs traverse its ports. Routing and firewall policy determine which subnets may communicate.\u003c/p\u003e\n\u003cp\u003eStart by documenting what the physical switch sends to the host. A correct guest VLAN tag cannot fix a mismatch on the uplink.\u003c/p\u003e\n\u003ch2 id=\"verification-status\"\u003eVerification status\u003c/h2\u003e\n\u003ctable\u003e\n  \u003cthead\u003e\n      \u003ctr\u003e\n          \u003cth\u003eArea\u003c/th\u003e\n          \u003cth\u003eEvidence and limits\u003c/th\u003e\n      \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eTechnical review\u003c/td\u003e\n          \u003ctd\u003eReviewed against the official references below on October 31, 2026.\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eShell examples\u003c/td\u003e\n          \u003ctd\u003eBash syntax checked; no host networking commands executed.\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eConfiguration\u003c/td\u003e\n          \u003ctd\u003eIllustrative single-uplink IPv4 lab, not a captured production configuration.\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eRuntime\u003c/td\u003e\n          \u003ctd\u003eApplication, rollback, DHCP, guest traffic, and firewall behavior remain untested for this revision.\u003c/td\u003e\n      \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003ch2 id=\"1-identify-the-management-path\"\u003e1. Identify the management path\u003c/h2\u003e\n\u003cp\u003eThese two designs are different:\u003c/p\u003e","title":"Proxmox Networking — Bridges, VLANs, and a Clean Host Topology"},{"content":"Hi, I\u0026rsquo;m Gerlin Nolasco, the person behind GnTech.\nI document the infrastructure I build, troubleshoot, and maintain: Proxmox hosts, MikroTik networks, Linux and Windows systems, Docker services, and remote administration tools. The goal is to turn a real problem into a guide another person can understand and follow.\nWhat you\u0026rsquo;ll find here Virtualization and storage: Proxmox VE, VMs, LXC, ZFS, and backups. Networking: MikroTik, VLANs, routing, VPNs, and management access. Self-hosting: Docker Compose, reverse proxies, monitoring, and service maintenance. Troubleshooting: the symptoms, diagnostic steps, changes, and verification behind a fix. Useful older hardware: practical experiments such as my ThinkPad T420 administration laptop. Browse the projects for a starting point, use search for a specific problem, or explore the archives.\nThe lab behind the posts My work spans a home lab and hosted infrastructure. Proxmox provides the virtualization layer; MikroTik handles routing and segmentation; Docker and LXC host services. Monitoring, backups, and recovery access are part of the work too.\nThe lab changes over time. Each guide describes its own environment; an older hardware list or diagram should not be treated as a live inventory. I keep public examples focused on the details needed to explain the procedure.\nHow posts are prepared AI assists with drafting, organizing explanations, preparing illustrations, and checking examples. I review proposed changes through GitHub pull requests before merging them. Merging a publication-ready post triggers the site\u0026rsquo;s normal Cloudflare Pages deployment.\nAutomated checks can catch build problems, missing assets, broken internal links, and some syntax errors. They cannot establish that a command works on a particular machine or that a procedure is safe for every environment.\nThe articles should distinguish observed lab results from proposed examples and explain relevant validation limits. A syntax check is not a hardware test. Check prerequisites and versions, preserve a recovery path, and verify the result in your own environment.\nGenerated illustrations are identified as illustrations. Real screenshots and command output, when included, serve a different purpose: showing what was actually observed.\nAbout the site GnTech runs on Hugo with the PaperMod theme, hosted on Cloudflare Pages. The blog is available in English and Spanish; translation coverage grows as guides are reviewed.\nConnect GitHub: gntech-dev LinkedIn: Gerlin Nolasco Subscribe through RSS ","permalink":"https://blog.gntech.me/about/","summary":"\u003cp\u003eHi, I\u0026rsquo;m \u003cstrong\u003eGerlin Nolasco\u003c/strong\u003e, the person behind \u003cstrong\u003eGnTech\u003c/strong\u003e.\u003c/p\u003e\n\u003cp\u003eI document the infrastructure I build, troubleshoot, and maintain: Proxmox hosts, MikroTik networks, Linux and Windows systems, Docker services, and remote administration tools. The goal is to turn a real problem into a guide another person can understand and follow.\u003c/p\u003e\n\u003ch2 id=\"what-youll-find-here\"\u003eWhat you\u0026rsquo;ll find here\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003eVirtualization and storage:\u003c/strong\u003e Proxmox VE, VMs, LXC, ZFS, and backups.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eNetworking:\u003c/strong\u003e MikroTik, VLANs, routing, VPNs, and management access.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eSelf-hosting:\u003c/strong\u003e Docker Compose, reverse proxies, monitoring, and service maintenance.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eTroubleshooting:\u003c/strong\u003e the symptoms, diagnostic steps, changes, and verification behind a fix.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eUseful older hardware:\u003c/strong\u003e practical experiments such as my ThinkPad T420 administration laptop.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eBrowse the \u003ca href=\"/projects/\"\u003eprojects\u003c/a\u003e for a starting point, use \u003ca href=\"/search/\"\u003esearch\u003c/a\u003e for a specific problem, or explore the \u003ca href=\"/archives/\"\u003earchives\u003c/a\u003e.\u003c/p\u003e","title":"About"},{"content":"These projects connect the lab\u0026rsquo;s infrastructure work to the guides on this blog. The links are reading paths, not a live service inventory.\nNetworking and segmentation MikroTik routing and VLANs provide the foundation for separating users, servers, and devices. Start with the router walkthrough, then follow how the Proxmox bridges connect to the network.\nBuilding R1: a MikroTik router for a VLAN-segmented homelab Proxmox networking: bridges, VLANs, and host topology Self-hosted services Docker Compose provides repeatable definitions for services and their dependencies. These guides cover patterns used across deployments and a concrete camera-system project.\nDocker Compose patterns for homelab stacks Deploying Frigate NVR on Proxmox The Frigate article documents an earlier deployment. Check its environment and version assumptions before adapting it.\nAdministration and recovery access A useful administration setup includes a client machine and a way back into a host when its usual interface is unavailable.\nThinkPad T420 with Arch Linux and i3 Intel AMT Serial over LAN on Proxmox The T420 article separates the machine\u0026rsquo;s installed hardware from planned upgrades. The SOL article separates observed console behavior from the generalized setup procedure.\nDocumentation and publishing This blog turns project notes into reviewed guides. AI helps prepare the material, and GitHub pull requests provide the review step before publication. See About for the process and its validation limits.\nFor more subjects, use search, browse the archives, or subscribe through RSS.\n","permalink":"https://blog.gntech.me/projects/","summary":"\u003cp\u003eThese projects connect the lab\u0026rsquo;s infrastructure work to the guides on this blog. The links are reading paths, not a live service inventory.\u003c/p\u003e\n\u003ch2 id=\"networking-and-segmentation\"\u003eNetworking and segmentation\u003c/h2\u003e\n\u003cp\u003eMikroTik routing and VLANs provide the foundation for separating users, servers, and devices. Start with the router walkthrough, then follow how the Proxmox bridges connect to the network.\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"/posts/r1-router-deployment/\"\u003eBuilding R1: a MikroTik router for a VLAN-segmented homelab\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"/posts/2026-05-08-proxmox-networking-bridges-vlans/\"\u003eProxmox networking: bridges, VLANs, and host topology\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"self-hosted-services\"\u003eSelf-hosted services\u003c/h2\u003e\n\u003cp\u003eDocker Compose provides repeatable definitions for services and their dependencies. These guides cover patterns used across deployments and a concrete camera-system project.\u003c/p\u003e","title":"Projects"},{"content":"Every homelab needs a solid network foundation. This guide walks through the full configuration of R1 — a MikroTik edge router with segmented VLANs, inter-VLAN firewalling, WireGuard VPN, and a Cloudflare Tunnel running directly on the router.\nThe config below is based on RouterOS 7.22.1. Commands are split by section so you can follow along step-by-step. Replace anything in \u0026lt;\u0026gt; with your own values.\nHardware Model: MikroTik E62iUGS-2axD5axT OS: RouterOS 7.22.1 WAN: GPON FTTH (PPPoE on VLAN 100) Port Layout Port Role Access VLAN Notes SFP1 WAN — GPON ONT, native vlan 1 for ONT access Ether1 CCTV 50 Untagged, camera network Ether2 MGMT 99 Untagged, management Ether3 MGMT 99 Untagged, secondary management Ether4 HOME 10 Untagged, main home LAN Ether5 Trunk Tagged Inter-switch link carrying all VLANs Step 1 — Bridge Setup Create the main bridge with VLAN filtering enabled, and a separate bridge for container veth interfaces:\n/interface bridge add name=bridge-trunk vlan-filtering=yes /interface bridge add name=containers Step 2 — VLAN Interfaces Create a VLAN interface on the bridge for each segment:\n/interface vlan add interface=bridge-trunk name=vlan-home vlan-id=10 /interface vlan add interface=bridge-trunk name=vlan-lab vlan-id=20 /interface vlan add interface=bridge-trunk name=vlan-iot vlan-id=40 /interface vlan add interface=bridge-trunk name=vlan-cctv vlan-id=50 /interface vlan add interface=bridge-trunk name=vlan-mgmt vlan-id=99 The WAN VLAN uses the SFP1 interface directly, not the bridge:\n/interface vlan add interface=sfp1 name=vlan100-wan vlan-id=100 Step 3 — Bridge Port Assignment Configure each physical port\u0026rsquo;s PVID (untagged VLAN) and tagging mode:\n/interface bridge port add bridge=bridge-trunk interface=ether2 pvid=99 \\ frame-types=admit-only-untagged-and-priority-tagged /interface bridge port add bridge=bridge-trunk interface=ether3 pvid=99 \\ frame-types=admit-only-untagged-and-priority-tagged /interface bridge port add bridge=bridge-trunk interface=ether4 pvid=10 \\ frame-types=admit-only-untagged-and-priority-tagged /interface bridge port add bridge=bridge-trunk interface=ether1 pvid=50 \\ frame-types=admit-only-untagged-and-priority-tagged /interface bridge port add bridge=bridge-trunk interface=ether5 \\ frame-types=admit-only-vlan-tagged /interface bridge port add bridge=bridge-trunk interface=sfp1 ether5 is the trunk port — it only accepts tagged frames. Everything else accepts untagged (their PVID VLAN) and tagged.\nStep 4 — Bridge VLAN Table Tell the bridge which VLANs are allowed on which ports, and whether they\u0026rsquo;re tagged or untagged:\n/interface bridge vlan add bridge=bridge-trunk vlan-ids=10 tagged=bridge-trunk,ether5 untagged=ether4 /interface bridge vlan add bridge=bridge-trunk vlan-ids=20 tagged=bridge-trunk,ether5,ether3 /interface bridge vlan add bridge=bridge-trunk vlan-ids=40 tagged=bridge-trunk,ether5 /interface bridge vlan add bridge=bridge-trunk vlan-ids=50 tagged=bridge-trunk,ether5 /interface bridge vlan add bridge=bridge-trunk vlan-ids=99 tagged=bridge-trunk,ether5 untagged=ether2 /interface bridge vlan add bridge=bridge-trunk vlan-ids=300 tagged=bridge-trunk,sfp1,ether5 /interface bridge vlan add bridge=bridge-trunk vlan-ids=1 untagged=sfp1,bridge-trunk What\u0026rsquo;s happening here:\nbridge-trunk is listed as tagged on all VLANs — this gives the bridge itself (and its VLAN interfaces) access to those VLANs ether4 gets VLAN 10 untagged — devices plugged into that port are on HOME ether2 gets VLAN 99 untagged — plugged into that port → MGMT ether3 gets VLAN 20 tagged in addition to its PVID — it\u0026rsquo;s a trunk-like MGMT port that also carries LAB ether5 gets everything tagged — trunk to downstream switch VLAN 300 is ISP VoIP, passed transparently to Proxmox via the trunk VLAN 1 is the native/untagged VLAN on SFP1 — ONT access Step 5 — WAN (PPPoE) PPPoE on the WAN VLAN interface. Replace \u0026lt;username\u0026gt; with your ISP credentials:\n/interface pppoe-client add name=pppoe-out1 interface=vlan100-wan \\ user=\u0026lt;pppoe-username\u0026gt; password=\u0026lt;pppoe-password\u0026gt; add-default-route=yes disabled=no Step 6 — IP Addresses Assign gateway addresses to each VLAN interface:\n/ip address add address=10.0.99.1/24 interface=vlan-mgmt comment=\u0026#34;MGMT gateway\u0026#34; /ip address add address=10.0.10.1/24 interface=vlan-home comment=\u0026#34;HOME gateway\u0026#34; /ip address add address=10.0.20.1/24 interface=vlan-lab comment=\u0026#34;LAB gateway\u0026#34; /ip address add address=10.0.40.1/24 interface=vlan-iot comment=\u0026#34;IoT gateway\u0026#34; /ip address add address=10.0.50.1/24 interface=vlan-cctv comment=\u0026#34;CCTV gateway\u0026#34; /ip address add address=172.31.255.1/24 interface=containers /ip address add address=192.168.1.2/24 interface=sfp1 comment=\u0026#34;ONT access\u0026#34; Step 7 — Interface Lists Interface lists simplify firewall rules. Group VLANs and WAN:\n/interface list add name=WAN /interface list add name=LAN /interface list member add interface=vlan-home list=LAN /interface list member add interface=vlan-lab list=LAN /interface list member add interface=vlan-iot list=LAN /interface list member add interface=vlan-cctv list=LAN /interface list member add interface=vlan-mgmt list=LAN /interface list member add interface=pppoe-out1 list=WAN Now firewall rules can target in-interface-list=LAN or out-interface-list=WAN instead of listing individual interfaces.\nStep 8 — DHCP Pools /ip pool add name=pool-home ranges=10.0.10.100-10.0.10.250 /ip pool add name=pool-iot ranges=10.0.40.100-10.0.40.250 /ip pool add name=pool-cctv ranges=10.0.50.100-10.0.50.250 /ip pool add name=pool-mgmt ranges=10.0.99.100-10.0.99.250 /ip pool add name=pool-lab ranges=10.0.20.100-10.0.20.250 Networks /ip dhcp-server network add address=10.0.10.0/24 gateway=10.0.10.1 dns-server=10.0.10.1 /ip dhcp-server network add address=10.0.20.0/24 gateway=10.0.20.1 dns-server=10.0.20.1 /ip dhcp-server network add address=10.0.40.0/24 gateway=10.0.40.1 dns-server=10.0.40.1 /ip dhcp-server network add address=10.0.50.0/24 gateway=10.0.50.1 dns-server=10.0.50.1 /ip dhcp-server network add address=10.0.99.0/24 gateway=10.0.99.1 dns-server=10.0.99.1 Servers /ip dhcp-server add name=dhcp-home interface=vlan-home address-pool=pool-home /ip dhcp-server add name=dhcp-iot interface=vlan-iot address-pool=pool-iot /ip dhcp-server add name=dhcp-cctv interface=vlan-cctv address-pool=pool-cctv /ip dhcp-server add name=dhcp-mgmt interface=vlan-mgmt address-pool=pool-mgmt /ip dhcp-server add name=dhcp-lab interface=vlan-lab address-pool=pool-lab disabled=yes LAB DHCP is disabled because servers use static IPs.\nStep 9 — DNS /ip dns set allow-remote-requests=yes servers=1.1.1.1,8.8.8.8 \\ cache-size=8192KiB max-udp-packet-size=1232 Step 10 — Firewall Address List Define hosts that router containers are allowed to reach on the MGMT VLAN:\n/ip firewall address-list add list=cloudflared-mgmt-allowed \\ address=10.0.99.2-10.0.99.10 Input Chain (traffic to the router itself) # Allow established/related /ip firewall filter add chain=input connection-state=established,related action=accept # Drop invalid /ip firewall filter add chain=input connection-state=invalid action=drop # ICMP /ip firewall filter add chain=input protocol=icmp action=accept # DHCP /ip firewall filter add chain=input protocol=udp dst-port=67 in-interface-list=LAN action=accept # DNS /ip firewall filter add chain=input protocol=udp dst-port=53 in-interface-list=LAN action=accept /ip firewall filter add chain=input protocol=tcp dst-port=53 in-interface-list=LAN action=accept # MGMT access to router /ip firewall filter add chain=input in-interface=vlan-mgmt action=accept # Drop from WAN /ip firewall filter add chain=input in-interface-list=WAN action=drop # Block IoT → router /ip firewall filter add chain=input in-interface=vlan-iot action=drop # Default drop /ip firewall filter add chain=input action=drop IoT cannot reach the router at all. MGMT is the only VLAN that can manage it.\nForward Chain (traffic between VLANs) # Established/related /ip firewall filter add chain=forward connection-state=established,related action=accept # Drop invalid /ip firewall filter add chain=forward connection-state=invalid action=drop # Per-VLAN Internet access /ip firewall filter add chain=forward in-interface=vlan-home out-interface-list=WAN action=accept /ip firewall filter add chain=forward in-interface=vlan-lab out-interface-list=WAN action=accept /ip firewall filter add chain=forward in-interface=vlan-iot out-interface-list=WAN action=accept /ip firewall filter add chain=forward in-interface=vlan-cctv out-interface-list=WAN action=accept /ip firewall filter add chain=forward in-interface=vlan-mgmt out-interface-list=WAN action=accept # MGMT → all VLANs /ip firewall filter add chain=forward in-interface=vlan-mgmt out-interface-list=LAN action=accept # All VLANs → servers (10.0.20.10 = SRV2, 10.0.20.30 = SRV1) /ip firewall filter add chain=forward dst-address=10.0.20.10 action=accept /ip firewall filter add chain=forward dst-address=10.0.20.30 action=accept # Frigate hosts → CCTV VLAN (camera access) /ip firewall filter add chain=forward dst-address=10.0.50.0/24 src-address=10.0.20.15 action=accept /ip firewall filter add chain=forward dst-address=10.0.50.0/24 src-address=10.0.20.30 action=accept # VPN → Internet + LAN /ip firewall filter add chain=forward in-interface=back-to-home-vpn out-interface-list=WAN action=accept /ip firewall filter add chain=forward in-interface=back-to-home-vpn out-interface-list=LAN action=accept # DNS outbound /ip firewall filter add chain=forward protocol=udp dst-port=53 out-interface-list=WAN action=accept /ip firewall filter add chain=forward protocol=tcp dst-port=53 out-interface-list=WAN action=accept # LAN → router containers /ip firewall filter add chain=forward in-interface-list=LAN dst-address=172.31.255.0/24 action=accept # Container → LAB /ip firewall filter add chain=forward in-interface=containers out-interface=vlan-lab action=accept # Container → selected MGMT hosts (Cloudflare Tunnel proxying web UIs) /ip firewall filter add chain=forward src-address=172.31.255.0/24 \\ dst-address-list=cloudflared-mgmt-allowed action=accept # Asterisk VoIP flows /ip firewall filter add chain=forward protocol=udp dst-address=10.0.20.25 dst-port=5160 in-interface=vlan-mgmt action=accept /ip firewall filter add chain=forward protocol=udp dst-address=10.0.20.25 dst-port=10000-20000 in-interface=vlan-mgmt action=accept /ip firewall filter add chain=forward protocol=udp dst-address=10.0.20.25 dst-port=5160 in-interface=vlan-home action=accept /ip firewall filter add chain=forward protocol=udp dst-address=10.0.20.25 dst-port=10000-20000 in-interface=vlan-home action=accept # Return traffic /ip firewall filter add chain=forward protocol=udp src-address=10.0.20.25 src-port=5160 out-interface=vlan-mgmt action=accept /ip firewall filter add chain=forward protocol=udp src-address=10.0.20.25 src-port=10000-20000 out-interface=vlan-mgmt action=accept /ip firewall filter add chain=forward protocol=udp src-address=10.0.20.25 src-port=5160 out-interface=vlan-home action=accept /ip firewall filter add chain=forward protocol=udp src-address=10.0.20.25 src-port=10000-20000 out-interface=vlan-home action=accept # ONT access from MGMT /ip firewall filter add chain=forward src-address=10.0.99.0/24 dst-address=192.168.1.0/24 action=accept # Block DNS-over-TLS from LAN /ip firewall filter add chain=forward protocol=tcp dst-port=853 in-interface-list=LAN action=reject # Default drop inter-VLAN /ip firewall filter add chain=forward action=drop NAT (Masquerade) # WAN masquerade /ip firewall nat add chain=srcnat out-interface-list=WAN action=masquerade # ONT access masquerade (MGMT → ONT private subnet) /ip firewall nat add chain=srcnat src-address=10.0.99.0/24 dst-address=192.168.1.0/24 action=masquerade Step 11 — WireGuard VPN /interface wireguard add name=back-to-home-vpn mtu=1420 listen-port=46209 /interface wireguard peers add interface=back-to-home-vpn \\ public-key=\u0026#34;\u0026lt;peer-public-key\u0026gt;\u0026#34; allowed-address=0.0.0.0/0 Enable MikroTik Cloud DDNS so the router is reachable remotely:\n/ip cloud set ddns-enabled=yes ddns-update-interval=10m back-to-home-vpn=enabled Step 12 — WiFi 2.4 GHz Radio (wifi1) /interface wifi set [find default-name=wifi1] \\ channel.frequency=2412 .width=20mhz \\ configuration.country=\u0026#34;United States\u0026#34; .mode=ap .tx-power=17 disabled=no 5 GHz Radio (wifi2) /interface wifi set [find default-name=wifi2] \\ channel.frequency=5180 .skip-dfs-channels=all .width=20/40mhz \\ configuration.country=\u0026#34;United States\u0026#34; .mode=ap .ssid=R1-MASTER-5G .tx-power=18 disabled=no Datapaths (VLAN binding per SSID) /interface wifi datapath add bridge=bridge-trunk name=dp-home vlan-id=10 /interface wifi datapath add bridge=bridge-trunk name=dp-iot vlan-id=40 /interface wifi datapath add bridge=bridge-trunk name=dp-cctv vlan-id=50 /interface wifi datapath add bridge=bridge-trunk name=dp-mgmt vlan-id=99 Security Profiles /interface wifi security add authentication-types=wpa2-psk name=sec-home /interface wifi security add authentication-types=wpa2-psk name=sec-iot /interface wifi security add authentication-types=wpa2-psk name=sec-cctv /interface wifi security add authentication-types=wpa2-psk name=sec-mgmt SSID Configurations /interface wifi configuration add datapath=dp-home security=sec-home ssid=\u0026#34;\u0026lt;home-ssid\u0026gt;\u0026#34; name=cfg-home /interface wifi configuration add datapath=dp-iot security=sec-iot ssid=IoT name=cfg-iot /interface wifi configuration add datapath=dp-cctv security=sec-cctv ssid=GS name=cfg-cctv hide-ssid=yes /interface wifi configuration add datapath=dp-mgmt security=sec-mgmt ssid=\u0026#34;\u0026lt;mgmt-ssid\u0026gt;\u0026#34; name=cfg-mgmt hide-ssid=yes Virtual APs (2.4 GHz) /interface wifi add master-interface=wifi1 configuration=cfg-home name=wifi1-home /interface wifi add master-interface=wifi1 configuration=cfg-iot name=wifi1-iot /interface wifi add master-interface=wifi1 configuration=cfg-cctv name=wifi1-cctv /interface wifi add master-interface=wifi1 configuration=cfg-mgmt name=wifi1-mgmt Virtual APs (5 GHz) /interface wifi add master-interface=wifi2 configuration=cfg-home-5g name=wifi2-home /interface wifi add master-interface=wifi2 configuration=cfg-mgmt name=wifi2-mgmt Step 13 — Container (Cloudflare Tunnel) Create veth interfaces for the container:\n/interface veth add name=veth1 address=172.31.255.2/24 gateway=172.31.255.1 /interface veth add name=veth2 address=172.31.255.3/24 gateway=172.31.255.1 /interface bridge port add bridge=containers interface=veth1 /interface bridge port add bridge=containers interface=veth2 Add the container (replace \u0026lt;token\u0026gt; with your Cloudflare Tunnel token):\n/container add name=docker-cloudflared interface=veth1 \\ remote-image=ghcr.io/shmick/docker-cloudflared \\ cmd=\u0026#34;tunnel --no-autoupdate run --token \u0026lt;cloudflare-tunnel-token\u0026gt;\u0026#34; \\ dns=1.1.1.1,1.0.0.1 hostname=CF \\ root-dir=/usb1/container/cloudflared start-on-boot=yes /container config set registry-url=https://registry-1.docker.io tmpdir=/usb1/tmp Step 14 — Rate Limiting (IoT) IoT devices get capped at 1 Mbps to prevent a compromised device from consuming bandwidth:\n/queue simple add name=limit-iot-1m target=10.0.40.0/24 max-limit=1M/1M Step 15 — Services Security Lock down management services to MGMT VLAN only:\n/ip service set ftp disabled=yes /ip service set telnet disabled=yes /ip service set api disabled=yes /ip service set api-ssl disabled=yes /ip service set ssh address=10.0.99.0/24 /ip service set winbox address=10.0.99.0/24 /ip service set www address=10.0.99.0/24,172.31.255.2/32 port=8080 /ip service set www-ssl address=10.0.99.0/24,172.31.255.2/32 disabled=no WebFig on port 8080 (non-SSL) and HTTPS are also accessible from the Cloudflare Tunnel container IP, so proxied services work through Cloudflare Access.\nStep 16 — LEDs Off All LEDs disabled for silent/dark operation:\n/system leds set 0 disabled=yes /system leds set 1 disabled=yes /system leds settings set all-leds-off=immediate Step 17 — IPv6 /ipv6 dhcp-client add interface=pppoe-out1 pool-name=ipv6-pd request=prefix add-default-route=yes /ipv6 address add from-pool=ipv6-pd interface=vlan-home /ipv6 address add from-pool=ipv6-pd interface=vlan-lab /ipv6 address add from-pool=ipv6-pd interface=vlan-iot /ipv6 address add from-pool=ipv6-pd interface=vlan-cctv /ipv6 address add from-pool=ipv6-pd interface=vlan-mgmt /ipv6 nd add interface=vlan-home /ipv6 nd add interface=vlan-lab /ipv6 nd add interface=vlan-iot /ipv6 nd add interface=vlan-cctv /ipv6 nd add interface=vlan-mgmt IPv6 firewall mirrors the IPv4 logic (established/related → accept, per-VLAN out → accept, others → drop).\nDeployment Lessons What worked:\nSingle box doing routing, WiFi, firewall, VPN, and containers keeps the stack simple Bridge VLAN filtering with vlan-filtering=yes is clean once you understand it — the bridge VLAN table is the key Interface lists make firewall rules readable and maintainable Simple queues for rate-limiting IoT is zero-maintenance What to watch for:\n2 GB RAM on the router is tight with containers running — 4 GB is more comfortable The built-in WiFi is adequate but a dedicated AP (Unifi/Omada) would perform better at range Container storage on USB works but isn\u0026rsquo;t ideal for persistent logs Key firewall design:\nInter-VLAN is dropped by default MGMT can reach everything (management convenience) Servers are reachable from all VLANs (services need to work across segments) CCTV is locked down — only specific Frigate hosts can initiate connections to cameras DNS-over-TLS is blocked to prevent devices from bypassing the router\u0026rsquo;s DNS Full Export Below is the complete sanitized RouterOS export (2026-05-07). This is the reference config — everything above is the broken-down version.\n/interface bridge add name=bridge-trunk vlan-filtering=yes add name=containers /interface vlan add interface=bridge-trunk name=vlan-cctv vlan-id=50 add interface=bridge-trunk name=vlan-home vlan-id=10 add interface=bridge-trunk name=vlan-iot vlan-id=40 add interface=bridge-trunk name=vlan-lab vlan-id=20 add interface=bridge-trunk name=vlan-mgmt vlan-id=99 add interface=sfp1 name=vlan100-wan vlan-id=100 /interface pppoe-client add name=pppoe-out1 interface=vlan100-wan user=\u0026lt;pppoe-user\u0026gt; \\ add-default-route=yes disabled=no /interface wireguard add name=back-to-home-vpn listen-port=46209 mtu=1420 /interface list add name=WAN add name=LAN /ip pool add name=pool-home ranges=10.0.10.100-10.0.10.250 add name=pool-lab ranges=10.0.20.100-10.0.20.250 add name=pool-iot ranges=10.0.40.100-10.0.40.250 add name=pool-cctv ranges=10.0.50.100-10.0.50.250 add name=pool-mgmt ranges=10.0.99.100-10.0.99.250 /ip dhcp-server add address-pool=pool-home interface=vlan-home name=dhcp-home add address-pool=pool-iot interface=vlan-iot name=dhcp-iot add address-pool=pool-cctv interface=vlan-cctv name=dhcp-cctv add address-pool=pool-mgmt interface=vlan-mgmt name=dhcp-mgmt add address-pool=pool-lab interface=vlan-lab name=dhcp-lab disabled=yes /ip address add address=10.0.99.1/24 interface=vlan-mgmt add address=10.0.10.1/24 interface=vlan-home add address=10.0.20.1/24 interface=vlan-lab add address=10.0.40.1/24 interface=vlan-iot add address=10.0.50.1/24 interface=vlan-cctv add address=172.31.255.1/24 interface=containers add address=192.168.1.2/24 interface=sfp1 /ip firewall filter add chain=input connection-state=established,related action=accept add chain=input connection-state=invalid action=drop add chain=input protocol=icmp action=accept add chain=input protocol=udp dst-port=67 in-interface-list=LAN action=accept add chain=input protocol=udp dst-port=53 in-interface-list=LAN action=accept add chain=input protocol=tcp dst-port=53 in-interface-list=LAN action=accept add chain=input in-interface=vlan-mgmt action=accept add chain=input in-interface-list=WAN action=drop add chain=input in-interface=vlan-iot action=drop add chain=input action=drop add chain=forward connection-state=established,related action=accept add chain=forward connection-state=invalid action=drop add chain=forward in-interface=vlan-home out-interface-list=WAN action=accept add chain=forward in-interface=vlan-lab out-interface-list=WAN action=accept add chain=forward in-interface=vlan-iot out-interface-list=WAN action=accept add chain=forward in-interface=vlan-cctv out-interface-list=WAN action=accept add chain=forward in-interface=vlan-mgmt out-interface-list=WAN action=accept add chain=forward in-interface=vlan-mgmt out-interface-list=LAN action=accept add chain=forward dst-address=10.0.20.10 action=accept add chain=forward dst-address=10.0.20.30 action=accept add chain=forward dst-address=10.0.50.0/24 src-address=10.0.20.15 action=accept add chain=forward dst-address=10.0.50.0/24 src-address=10.0.20.30 action=accept add chain=forward in-interface=back-to-home-vpn out-interface-list=WAN action=accept add chain=forward in-interface=back-to-home-vpn out-interface-list=LAN action=accept add chain=forward protocol=udp dst-port=53 out-interface-list=WAN action=accept add chain=forward protocol=tcp dst-port=53 out-interface-list=WAN action=accept add chain=forward in-interface-list=LAN dst-address=172.31.255.0/24 action=accept add chain=forward in-interface=containers out-interface=vlan-lab action=accept add chain=forward src-address=172.31.255.0/24 \\ dst-address-list=cloudflared-mgmt-allowed action=accept add chain=forward src-address=10.0.99.0/24 dst-address=192.168.1.0/24 action=accept add chain=forward protocol=tcp dst-port=853 in-interface-list=LAN action=reject add chain=forward action=drop /ip firewall nat add chain=srcnat out-interface-list=WAN action=masquerade add chain=srcnat src-address=10.0.99.0/24 dst-address=192.168.1.0/24 action=masquerade /ip service set ssh address=10.0.99.0/24 /ip service set winbox address=10.0.99.0/24 /ip service set www address=10.0.99.0/24,172.31.255.2/32 port=8080 /ip service set www-ssl address=10.0.99.0/24,172.31.255.2/32 disabled=no /ip service set ftp disabled=yes /ip service set telnet disabled=yes /ip service set api disabled=yes /ip service set api-ssl disabled=yes /queue simple add name=limit-iot-1m target=10.0.40.0/24 max-limit=1M/1M /system identity set name=R1 /system clock set time-zone-name=America/Santo_Domingo ","permalink":"https://blog.gntech.me/posts/r1-router-deployment/","summary":"\u003cp\u003eEvery homelab needs a solid network foundation. This guide walks through the full configuration of R1 — a MikroTik edge router with segmented VLANs, inter-VLAN firewalling, WireGuard VPN, and a Cloudflare Tunnel running directly on the router.\u003c/p\u003e\n\u003cp\u003eThe config below is based on RouterOS 7.22.1. Commands are split by section so you can follow along step-by-step. Replace anything in \u003ccode\u003e\u0026lt;\u0026gt;\u003c/code\u003e with your own values.\u003c/p\u003e\n\u003chr\u003e\n\u003ch2 id=\"hardware\"\u003eHardware\u003c/h2\u003e\n\u003cpre tabindex=\"0\"\u003e\u003ccode\u003eModel:  MikroTik E62iUGS-2axD5axT\nOS:     RouterOS 7.22.1\nWAN:    GPON FTTH (PPPoE on VLAN 100)\n\u003c/code\u003e\u003c/pre\u003e\u003ch3 id=\"port-layout\"\u003ePort Layout\u003c/h3\u003e\n\u003ctable\u003e\n  \u003cthead\u003e\n      \u003ctr\u003e\n          \u003cth\u003ePort\u003c/th\u003e\n          \u003cth\u003eRole\u003c/th\u003e\n          \u003cth\u003eAccess VLAN\u003c/th\u003e\n          \u003cth\u003eNotes\u003c/th\u003e\n      \u003c/tr\u003e\n  \u003c/thead\u003e\n  \u003ctbody\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eSFP1\u003c/td\u003e\n          \u003ctd\u003eWAN\u003c/td\u003e\n          \u003ctd\u003e—\u003c/td\u003e\n          \u003ctd\u003eGPON ONT, native vlan 1 for ONT access\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eEther1\u003c/td\u003e\n          \u003ctd\u003eCCTV\u003c/td\u003e\n          \u003ctd\u003e50\u003c/td\u003e\n          \u003ctd\u003eUntagged, camera network\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eEther2\u003c/td\u003e\n          \u003ctd\u003eMGMT\u003c/td\u003e\n          \u003ctd\u003e99\u003c/td\u003e\n          \u003ctd\u003eUntagged, management\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eEther3\u003c/td\u003e\n          \u003ctd\u003eMGMT\u003c/td\u003e\n          \u003ctd\u003e99\u003c/td\u003e\n          \u003ctd\u003eUntagged, secondary management\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eEther4\u003c/td\u003e\n          \u003ctd\u003eHOME\u003c/td\u003e\n          \u003ctd\u003e10\u003c/td\u003e\n          \u003ctd\u003eUntagged, main home LAN\u003c/td\u003e\n      \u003c/tr\u003e\n      \u003ctr\u003e\n          \u003ctd\u003eEther5\u003c/td\u003e\n          \u003ctd\u003eTrunk\u003c/td\u003e\n          \u003ctd\u003eTagged\u003c/td\u003e\n          \u003ctd\u003eInter-switch link carrying all VLANs\u003c/td\u003e\n      \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\u003chr\u003e\n\u003ch2 id=\"step-1--bridge-setup\"\u003eStep 1 — Bridge Setup\u003c/h2\u003e\n\u003cp\u003eCreate the main bridge with VLAN filtering enabled, and a separate bridge for container veth interfaces:\u003c/p\u003e","title":"Building R1 — A MikroTik Router for VLAN-Segmented Homelab"},{"content":" Historical setup: This article records the original lab deployment. Frigate configuration changes between releases; validate the configuration against the documentation for the exact image version you deploy. The example exposes several service ports and is intended for an isolated lab network. Restrict access at the host and router, and use the authenticated interface supported by your chosen release.\nFrigate is an open-source NVR built for real-time object detection with local processing. No cloud subscriptions, no vendor lock-in — just cameras, a GPU, and decent detection models.\nThis is how I set it up on my Proxmox host, with a Tapo C100 camera, Intel GPU inference via OpenVINO, MQTT eventing, and Telegram notifications.\nArchitecture ┌──────────────┐ RTSP (stream1/stream2) ┌──────────────┐ │ Tapo C100 │ ───────────────────────────▶ │ Frigate │ │ 10.0.50.101 │ │ 10.0.20.15 │ └──────────────┘ │ │ CCTV VLAN (50) │ Web UI:5000 │ │ RTSP :8554 │ ┌──────────┐ │ WebRTC:8555 │ │ Telegram │ ◀──── MQTT events ──────── │ go2rtc:1984 │ │ alerts │ (frigate-notifier) │ API :8971 │ └──────────┘ └──────┬───────┘ │ ┌──────▼───────┐ │ Mosquitto │ │ MQTT:1883 │ └──────────────┘ The camera lives on VLAN 50 (CCTV). Frigate runs on a Debian LXC on VLAN 20 (LAB). The router\u0026rsquo;s firewall allows Frigate\u0026rsquo;s host (10.0.20.15) to reach the camera subnet — nothing else touches CCTV.\nHost Setup Frigate runs on an HP ProDesk G3 DM (i5-7500T, 2 GB RAM) as a Debian LXC on Proxmox:\nIP: 10.0.20.15/24 (VLAN 20 LAB) CPU: Intel i5-7500T @ 2.70GHz (2 cores) RAM: 2 GB GPU: Intel HD Graphics 630 Storage: ~17 GB zvol + 8.5 GB for recordings OS: Debian Bookworm Key requirement: the LXC needs /dev/dri/renderD128 passed through for Intel GPU access. Device access depends on LXC device permissions and, for an unprivileged container, UID/GID mappings. A privileged container is not inherently required. The following is a configuration fragment for the documented setup; review it against your Proxmox version before applying it to /etc/pve/lxc/\u0026lt;CT_ID\u0026gt;.conf:\nlxc.cgroup2.devices.allow: c 226:128 rwm lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file Inside the container, the render group must match the host\u0026rsquo;s GID. This gives VAAPI and OpenVINO access for hardware-accelerated decoding and inference.\nDocker Stack Frigate ships as Docker containers, along with Mosquitto for MQTT and a custom notifier:\n1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 version: \u0026#34;3.9\u0026#34; services: frigate: container_name: frigate restart: unless-stopped stop_grace_period: 30s image: ghcr.io/blakeblackshear/frigate:stable volumes: - /etc/localtime:/etc/localtime:ro - ./config:/config - ./storage:/media/frigate - type: tmpfs target: /tmp/cache tmpfs: size: 1000000000 ports: - \u0026#34;5000:5000\u0026#34; # Web UI - \u0026#34;8554:8554\u0026#34; # RTSP - \u0026#34;8555:8555/tcp\u0026#34; - \u0026#34;8555:8555/udp\u0026#34; # WebRTC - \u0026#34;1984:1984\u0026#34; # go2rtc - \u0026#34;8971:8971\u0026#34; # API devices: - /dev/dri/renderD128:/dev/dri/renderD128 cap_add: - CAP_PERFMON shm_size: \u0026#34;512mb\u0026#34; mosquitto: image: eclipse-mosquitto:2 container_name: mosquitto restart: unless-stopped ports: - \u0026#34;1883:1883\u0026#34; - \u0026#34;9001:9001\u0026#34; volumes: - ./mosquitto/config:/mosquitto/config - ./mosquitto/data:/mosquitto/data - ./mosquitto/log:/mosquitto/log Mosquitto requires a password file:\n1 docker exec mosquitto mosquitto_passwd -c /mosquitto/config/pwfile frigate With mosquitto.conf:\nallow_anonymous false password_file /mosquitto/config/pwfile listener 1883 persistence true persistence_location /mosquitto/data/ log_dest file /mosquitto/log/mosquitto.log Camera The Tapo C100 is an indoor WiFi camera that speaks RTSP — no proprietary app required for local streaming. RTSP must be enabled in the Tapo app\u0026rsquo;s advanced settings.\nSetting Value IP 10.0.50.101 (VLAN 50 CCTV) Model Tapo C100 Stream 1 (main) rtsp://\u0026lt;user\u0026gt;:\u0026lt;pass\u0026gt;@10.0.50.101:554/stream1 — 1280×720 @ 5fps Stream 2 (sub) rtsp://\u0026lt;user\u0026gt;:\u0026lt;pass\u0026gt;@10.0.50.101:554/stream2 — lower res + audio Frigate Config 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 mqtt: enabled: true host: mosquitto port: 1883 user: frigate password: \u0026lt;mosquitto-password\u0026gt; detectors: ov: type: openvino device: GPU model_path: /openvino-model/ssdlite_mobilenet_v2.xml model: width: 300 height: 300 objects: track: - person cameras: tapo: ffmpeg: inputs: - path: rtsp://\u0026lt;user\u0026gt;:\u0026lt;pass\u0026gt;@10.0.50.101:554/stream1 roles: - detect - record detect: width: 1280 height: 720 fps: 5 motion: threshold: 8 contour_area: 10 improve_contrast: true objects: track: - person snapshots: enabled: true retain: default: 7 record: enabled: true motion: days: 7 go2rtc: streams: tapo: - rtsp://\u0026lt;user\u0026gt;:\u0026lt;pass\u0026gt;@10.0.50.101:554/stream1 Why OpenVINO over Coral TPU? The i5-7500T\u0026rsquo;s integrated HD 630 GPU is perfectly adequate for a single 720p camera doing person detection. A Coral TPU would be overkill for this scale and adds cost + USB passthrough complexity. OpenVINO at 300×300 inference runs comfortably at 5fps with minimal CPU load.\nDetection Model Frigate 0.17 ships with OpenVINO models. The config specifies ssdlite_mobilenet_v2.xml (300×300) which Frigate maps to the actual model path at runtime.\nFor a single camera, this is fast enough. If scaling to multiple cameras:\nBump resolution to 320×320 for slightly better accuracy Drop to the cpu detector as fallback Or add a Coral TPU via USB Firewall Rules On the router, Frigate traffic flows through these rules:\nAll VLANs → server 10.0.20.15 — generic allow for any VLAN to reach the Frigate host Frigate hosts → CCTV VLAN — src-address=10.0.20.15 dst-address=10.0.50.0/24 Inter-VLAN default-drop ensures nothing else reaches CCTV This means the Frigate container on VLAN 20 can pull RTSP from the camera on VLAN 50, while the camera itself is completely isolated from the rest of the network.\nTelegram Notifications The custom notifier source is not included in this post, so this section describes the original integration rather than a complete installation procedure. Do not expect the Compose example alone to create Telegram alerts.\nFrigate\u0026rsquo;s built-in notifications are disabled. Instead, a custom Python notifier (frigate-notifier) subscribes to MQTT events at topic prefix frigate and pushes detection clips and snapshots to a Telegram chat.\nThe notifier:\nListens for frigate/events for new detections Downloads the snapshot and clip from Frigate\u0026rsquo;s API Sends them via Telegram bot Filters to person detections only This gives real-time alerts when someone approaches the house, with video evidence, no cloud storage involved.\nStorage Type Path Retention Recordings /media/frigate/recordings/ Continuous: 1d, Motion: 7d Clips/Snapshots /media/frigate/clips/ 7 days Cache tmpfs (1 GB) Ephemeral Size recording storage from the measured bitrate and recorded hours. A continuous 1 Mbit/s stream uses about 10.8 GB per day before filesystem overhead; resolution alone does not determine storage use. Motion-only retention depends on activity. The small storage allocation above is a historical lab example, not a multi-day capacity recommendation.\nResults \u0026amp; Caveats What works well:\nReal-time person detection — alerts within seconds of motion go2rtc for WebRTC live view (sub-second latency in the UI) MQTT event integration works reliably for automation No subscription costs What to watch for:\nWiFi cameras drop frames under load — wire them if possible The C100 RTSP stream can disconnect; Frigate auto-reconnects but you\u0026rsquo;ll lose a few seconds OpenVINO on iGPU uses system RAM as VRAM — 2 GB RAM is tight, 4 GB is safer 720p detection at 5fps is fine for walkway coverage — adjust based on your scene Future Next steps:\nAdd a second camera (doorbell or driveway) Move to Coral TPU if adding more cameras Integrate with Home Assistant for richer automations Semantic search for reviewing footage by description ","permalink":"https://blog.gntech.me/posts/frigate-deployment/","summary":"\u003cblockquote\u003e\n\u003cp\u003e\u003cstrong\u003eHistorical setup:\u003c/strong\u003e This article records the original lab deployment. Frigate configuration changes between releases; validate the configuration against the documentation for the exact image version you deploy. The example exposes several service ports and is intended for an isolated lab network. Restrict access at the host and router, and use the authenticated interface supported by your chosen release.\u003c/p\u003e\u003c/blockquote\u003e\n\u003cp\u003eFrigate is an open-source NVR built for real-time object detection with local processing. No cloud subscriptions, no vendor lock-in — just cameras, a GPU, and decent detection models.\u003c/p\u003e","title":"Deploying Frigate NVR on Proxmox — Tapo C100, OpenVINO, and Telegram Alerts"},{"content":"Welcome to the blog. This is where I\u0026rsquo;ll be writing about homelab stuff — Proxmox, MikroTik, Docker, networking, automation, and whatever else I\u0026rsquo;m tinkering with.\nStay tuned.\n","permalink":"https://blog.gntech.me/posts/first-post/","summary":"\u003cp\u003eWelcome to the blog. This is where I\u0026rsquo;ll be writing about homelab stuff — Proxmox, MikroTik, Docker, networking, automation, and whatever else I\u0026rsquo;m tinkering with.\u003c/p\u003e\n\u003cp\u003eStay tuned.\u003c/p\u003e","title":"Hello, World"}]