What systemd Does
systemd is PID 1 — the first process Linux starts, and the parent of everything else. It manages service startup order, dependencies, restarts on failure, and log collection via journald. In a homelab, most of your long-running processes are systemd services.
# Core service commands systemctl status nginx → current state + recent logs systemctl start nginx → start now systemctl stop nginx → stop now systemctl restart nginx → stop then start systemctl reload nginx → reload config without downtime systemctl enable nginx → auto-start on boot systemctl disable nginx → remove from boot systemctl is-active nginx → "active" or "inactive" systemctl is-enabled nginx → "enabled" or "disabled" # List services systemctl list-units --type=service --state=running systemctl list-units --failed
enable vs start — Know the Difference
This trips up almost everyone. start runs the service right now. enable makes it start on every boot. They are independent — a service can be started but not enabled (won't survive reboot), or enabled but not started (will start next boot, but not now).
# The right way to deploy a new service: systemctl enable --now nginx → enable + start in one command # Check what would start on boot: systemctl list-unit-files --type=service | grep enabled
Reading Logs with journalctl
journald collects logs from every systemd service in one place. No more hunting through /var/log/ for each service separately.
journalctl -u nginx → all logs for nginx journalctl -u nginx -f → follow live (like tail -f) journalctl -u nginx --since today → just today's logs journalctl -u nginx -n 50 → last 50 lines journalctl -u nginx -p err → errors only # Multiple services at once journalctl -u nginx -u php-fpm -f # System-wide journalctl -xe → recent errors with context journalctl --since "1 hour ago" journalctl -p 3 -xb → kernel-level errors since boot # Disk usage journalctl --disk-usage journalctl --vacuum-time=7d → delete logs older than 7 days
Anatomy of a Unit File
Services are defined by unit files in /etc/systemd/system/. Understanding their structure lets you write your own — essential for any homelab app that doesn't ship a service file.
# /etc/systemd/system/myapp.service [Unit] Description=My App After=network.target # start after network is up Requires=docker.service # hard dependency [Service] Type=simple # process stays in foreground User=michael # run as this user WorkingDirectory=/opt/myapp ExecStart=/opt/myapp/start.sh ExecReload=/bin/kill -HUP $MAINPID Restart=on-failure # auto-restart if it crashes RestartSec=5s Environment=NODE_ENV=production EnvironmentFile=/opt/myapp/.env # load env vars from file [Install] WantedBy=multi-user.target # start in normal multi-user mode
# After editing a unit file: systemctl daemon-reload → reload systemd config systemctl enable --now myapp → enable and start
Diagnosing a Failed Service
# Step 1 — what's the state? systemctl status myapp # Step 2 — read the logs journalctl -u myapp -n 50 --no-pager # Step 3 — check the unit file for mistakes systemctl cat myapp # Step 4 — validate config (for supported services) nginx -t sshd -t # Step 5 — run it manually as the service user to see raw errors sudo -u michael /opt/myapp/start.sh
≈·*•—[ A|S ]—•*·≈