Default Linux kernel configurations aren't built for general-purpose workloads. They certainly aren't ready for heavy web traffic. When your application starts handling tens of thousands of concurrent connections, you'll hit arbitrary limits imposed by the operating system. You might see errors like Too many open files in your logs. Or worse, experience mysterious connection drops during traffic spikes. Fixing these bottlenecks requires adjusting core sysctl parameters and service limits. It's not magic. It's just telling the OS to get out of your way.
Before you touch any kernel settings, you need to understand how the operating system tracks network connections. Every incoming client socket consumes a file descriptor. The default system-wide limit is often far too low for a production web server handling heavy traffic. If Nginx exhausts its available file descriptors, it'll reject new connections entirely. Even if your CPU and RAM are completely idle, users won't get a page.
Raising File Descriptor Limits
You have to configure both system-wide limits and the specific limits for the Nginx service user. Systemd ignores traditional limits set in /etc/security/limits.conf for system services. This catches many developers off guard. You'll need to override the service configuration explicitly using a drop-in file.
Create a directory for the Nginx systemd override configuration and open a new conf file:
sudo mkdir -p /etc/systemd/system/nginx.service.d
sudo nano /etc/systemd/system/nginx.service.d/override.confAdd the following directives to instruct Systemd to raise the hard and soft limits for open file descriptors:
[Service]
LimitNOFILE=65535
After saving the file, reload the systemd manager configuration and restart Nginx to apply the changes:
sudo systemctl daemon-reload
sudo systemctl restart nginxYou can verify that the new limit took effect by inspecting the active process limits directly from the proc filesystem. Run this command and check the Max open files line:
cat /proc/$(cat /var/run/nginx.pid)/limits | grep "Max open files"If it outputs 65535, your service-level descriptor limit is properly configured. You also need to make sure the main Nginx configuration file matches this capability. Open your nginx.conf file and set the worker connections directive to a high number matching your file descriptor limit:
worker_processes auto;
events {
worker_connections 20480;
use epoll;
multi_accept on;
}
The multi_accept directive tells Nginx to accept all new connections rather than one by one. This significantly reduces latency under heavy load. Combined with the epoll event notification mechanism, Nginx can efficiently poll thousands of file descriptors without burning CPU cycles.
Tuning Network Stack and TCP Buffers
Operating system networking defaults are conservative because they assume limited RAM. High-throughput servers need larger socket buffers to handle high bandwidth-delay products. You can apply these changes persistently by editing your sysctl configuration file.
Open your system sysctl configuration file, typically located at /etc/sysctl.conf or within /etc/sysctl.d/, and append the following parameters:
fs.file-max = 2097152
net.core.somaxconn = 65535
net.core.netdev_max_backlog = 10000
net.ipv4.tcp_max_syn_backlog = 8192
net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 15
Let's look at what these specific keys do. The net.core.somaxconn parameter defines the maximum number of queued connections that can be waiting for acceptance by an application. If this value is too low, the kernel drops incoming SYN packets silently when Nginx can't accept them fast enough, leading to connection timeouts on the client side. Setting it to 65535 matches our connection headroom.
The net.ipv4.tcp_tw_reuse setting allows the kernel to recycle TIME_WAIT sockets for new connections when safe from a protocol perspective. This prevents your server from running out of ephemeral ports during traffic surges. Coupled with a reduced tcp_fin_timeout of 15 seconds instead of the default 60, dead sockets clear out of the connection table much faster.
Apply your sysctl configuration changes immediately without rebooting the server by running the load command:
sudo sysctl -pIf you need deeper context on broader kernel tuning principles, refer to Linux Sysctl Tuning for High-Throughput Web Servers for additional insights on memory management and virtual memory parameters.
Managing Epoll and Worker Scaling
Nginx relies on the asynchronous event notification facility provided by the Linux kernel. The default epoll implementation scales well. You must ensure your worker processes match your actual hardware capabilities. Setting worker processes to auto aligns the worker count with your physical CPU cores, ensuring that Nginx doesn't context-switch unnecessarily across threads.
Verify that your Nginx worker processes are running and properly bound to your CPU cores by inspecting process activity:
ps -ef | grep nginxIf you manage log generation alongside high throughput, remember that disk I/O can also block event loops if logs are written synchronously. Make sure your access logs use buffer options to minimize disk writes, or rotate logs efficiently using strategies detailed in Mastering Logrotate Configuration to prevent log files from filling up your storage partitions and crashing the web server.
Monitor your network stack under load using native diagnostic tools. Watch active socket states using standard network utilities to confirm that your TCP buffers and connection recycling parameters behave correctly under production traffic.
ss -sA healthy high-throughput configuration shows low dropped packet counts and a high volume of established TCP sockets without a buildup of orphaned or time-wait connections. Production environments demand continuous observation, so check these metrics often.
