Running background daemons reliably on Linux means handing them over to init systems rather than keeping them alive with screen sessions or custom shell scripts. Systemd handles service lifecycles, restart loops, and log aggregation out of the box. You'll write a proper systemd service unit file, configure crash recovery, and track output through journald without losing your mind over log rotation.
Where people mess up is treating a background daemon like a standard shell command. If your process exits due to a memory leak or an unhandled exception, you want it to restart immediately, but you also want backoff timers so you don't hammer the CPU in a tight crash loop. Systemd gives you fine-grained control over these behaviors through explicit unit configurations. If you aren't careful with user privileges, you'll end up running persistent background tasks as root, which opens up unnecessary security holes when a dependency gets compromised.
When you look at production setups, managing long-running background tasks effectively is just as important as keeping your web server responsive. If you've ever dealt with email queues or asynchronous processing workloads, you know that missing a single daemon restart can leave your customers hanging. It's the same kind of operational diligence required when handling other infrastructure services like Installing FreeSWITCH on Ubuntu where background audio engines demand clean process supervision to stay alive under heavy traffic spikes.
Writing the Service Unit File
You should place your unit files in /etc/systemd/system/ rather than modifying system-level directories. Let's look at a concrete example for a background worker script. Create a file named /etc/systemd/system/worker.service and populate it with the required directives so that systemd knows how to spawn, track, and stop your application.
[Unit]
Description=Background Worker Daemon
After=network.target
[Service]
Type=simple
User=www-data
Group=www-data
WorkingDirectory=/var/www/html
ExecStart=/usr/bin/python3 /var/www/html/worker.py
Restart=always
RestartSec=5
StandardOutput=journal
StandardError=journal
SyslogIdentifier=worker-daemon
[Install]
WantedBy=multi-user.target
The directives inside this file tell the system how to treat your code. The Type=simple setting assumes the process specified by ExecStart is the main process itself. If your daemon forks into the background, you'll want Type=forking instead, though most modern scripts avoid forking and let systemd handle the process lifecycle directly. The WorkingDirectory directive ensures relative paths resolve correctly, which saves you from silent failures caused by scripts looking for configuration files in the root directory.
Permissions matter here. Don't run your application code as root unless it genuinely needs raw socket binding or low-level system access. Setting User=www-data and Group=www-data limits the blast radius if an attacker manages to exploit a vulnerability in your code. You'll also want to make sure the target user actually owns the log files and working directories, otherwise the service will crash on startup with permission denied errors.
Configuring Restart Policies and Crash Recovery
Processes fail in production. Memory exhaustion, database connection drops, and third-party API timeouts will eventually kill your long-running scripts. That's why your restart policy matters. Setting Restart=always ensures that whenever the process exits—whether cleanly with status zero or with a fatal signal—systemd brings it right back up.
You don't want your server melting down if a script crashes on startup in an infinite loop. That's where RestartSec saves you by introducing a delay between exit and restart. You can also configure rate limiting using directives like StartLimitIntervalSec and StartLimitBurst to stop systemd from restarting a broken service more than a specific number of times within a given window. If you exceed those thresholds, systemd trips a circuit breaker and leaves the service in a failed state so you can investigate.
After you save changes to your unit file, systemd doesn't automatically notice them. You have to reload the daemon manager before restarting your service. Run these commands in your terminal to apply the configuration and enable the service on boot:
sudo systemctl daemon-reload
sudo systemctl enable --now worker.service
If you skip the daemon reload step, systemd will keep running the old cached version of your unit file, leaving you wondering why your configuration changes aren't taking effect. It's a common trap that catches developers off guard when they're rushing through deployments.
Debugging and Journald Logging
Traditional daemons write to flat log files in /var/log/, requiring separate logrotate configurations to stop disk space from filling up. Systemd routes everything through journald by default. When you configure StandardOutput=journal and SyslogIdentifier=worker-daemon, every stdout and stderr stream from your script gets captured with timestamps, process IDs, and service metadata.
Querying these logs is straightforward once you know the journalctl flags. You don't have to tail massive text files or write custom grep chains. Here is how you inspect real-time output for your specific background service:
sudo journalctl -u worker.service -f --since "1 hour ago"
If your script outputs JSON formatted logs, journald preserves the structure, allowing you to filter by specific fields if you forward the logs to an aggregator later. Pay attention to log levels as well. Uncaught exceptions written to stderr automatically show up highlighted in red when you view the journal without pagination, making it obvious when your application is throwing errors.
Managing long-running processes this way keeps your infrastructure predictable. You get clean process supervision, automated restarts with sensible backoff timers, and centralized logging without bolting on extra monitoring daemons. Take the time to write explicit unit files for your workers, and you'll spend less time SSHing into servers to manually restart crashed scripts.
