Compiling and Configuring Nginx with Dynamic Stream Module for Layer 4 Load Balancing on CentOS 7

Introduction

This guide details the process of compiling and installing Nginx on CentOS 7 64-bit to enable and dynamically load the Stream module for Layer 4 (TCP/UDP) load balancing. It covers environment preparation, compilation, service configuration, and final setup.

Environment Preparation and Compilation

It is recommended to use screen or tmux to maintain the compilation session and prevent task termination due to network interruptions.

yum install screen -y
screen -S nginx_compile

Install Build Tools and Dependencies

Install the necessary development tools and libraries.

sudo yum -y groupinstall "Development Tools"
sudo yum -y install epel-release wget gcc autoconf automake pcre-devel zlib-devel openssl-devel perl perl-devel libxslt libxslt-devel libxml2 libxml2-devel gd gd-devel GeoIP GeoIP-devel

Download Source Packages

This example uses Nginx 1.13.2, which is outdated. It is strongly advised to download the latest stable version from the Nginx website. The following commands are for demonstration only.

wget https://nginx.org/download/nginx-1.13.2.tar.gz
tar zxvf nginx-1.13.2.tar.gz
wget https://ftp.pcre.org/pub/pcre/pcre-8.40.tar.gz
tar xzvf pcre-8.40.tar.gz
wget https://www.zlib.net/zlib-1.2.11.tar.gz
tar xzvf zlib-1.2.11.tar.gz
wget https://www.openssl.org/source/openssl-1.1.0f.tar.gz
tar xzvf openssl-1.1.0f.tar.gz
rm -rf *.tar.gz
cd nginx-1.13.2

Configure Compilation Parameters

The key option is --with-stream=dynamic, which compiles the Stream module as a dynamic shared object (.so file) for later loading. Below is a sample configuration with common modules.

./configure --prefix=/etc/nginx 
  --sbin-path=/usr/sbin/nginx 
  --modules-path=/usr/lib64/nginx/modules 
  --conf-path=/etc/nginx/nginx.conf 
  --error-log-path=/var/log/nginx/error.log 
  --pid-path=/var/run/nginx.pid 
  --lock-path=/var/run/nginx.lock 
  --user=nginx 
  --group=nginx 
  --with-threads 
  --with-file-aio 
  --with-http_ssl_module 
  --with-http_v2_module 
  --with-http_realip_module 
  --with-http_sub_module 
  --with-http_gunzip_module 
  --with-http_gzip_static_module 
  --with-http_stub_status_module 
  --with-stream=dynamic 
  --with-stream_ssl_module 
  --with-stream_realip_module 
  --with-pcre=../pcre-8.40 
  --with-pcre-jit 
  --with-zlib=../zlib-1.2.11 
  --with-openssl=../openssl-1.1.0f 
  --with-debug

After running ./configure, check the end of the output for errors. If none, proceed with compilation and installation.

make
sudo make install

Create Symbolic Link and Nginx User

Create a symbolic link for the dynamic modules directory for easier configuration reference.

sudo ln -s /usr/lib64/nginx/modules /etc/nginx/modules

Create the Nginx runtime user if it doesn't exist.

sudo useradd --system --home /var/cache/nginx --shell /sbin/nologin --comment "nginx user" --user-group nginx

Verify Installation and Configure System Service

Check compilation parameters to confirm --with-stream=dynamic is included.

nginx -V 2>&1 | grep stream

Create necessary cache directories and test the configuration.

sudo mkdir -p /var/cache/nginx
sudo nginx -t

Configure the systemd service unit file at /usr/lib/systemd/system/nginx.service.

[Unit]
Description=nginx - high performance web server
Documentation=https://nginx.org/en/docs/
After=network-online.target remote-fs.target nss-lookup.target
Wants=network-online.target

[Service]
Type=forking
PIDFile=/var/run/nginx.pid
ExecStartPre=/usr/sbin/nginx -t -c /etc/nginx/nginx.conf
ExecStart=/usr/sbin/nginx -c /etc/nginx/nginx.conf
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s TERM $MAINPID

[Install]
WantedBy=multi-user.target

Start Nginx and enable it to start on boot.

sudo systemctl start nginx
sudo systemctl enable nginx

Check the service status.

sudo systemctl status nginx
curl -I 127.0.0.1

Firewall Configuration

If the system firewall (firewalld) is active, allow the ports used by Nginx (e.g., 80, 443).

sudo firewall-cmd --permanent --zone=public --add-service=http
sudo firewall-cmd --permanent --zone=public --add-service=https
sudo firewall-cmd --reload

Configure Nginx Stream Layer 4 Load Balancing

The default Nginx configuration file is at /etc/nginx/nginx.conf. To dynamically load the Stream module, use the load_module directive at the top of the main configuration file.

First, back up the original configuration.

sudo cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.backup

Edit /etc/nginx/nginx.conf. A basic TCP load balancing configuration example is shown below.

# Dynamically load the Stream module
load_module modules/ngx_stream_module.so;

worker_processes auto;

events {
    worker_connections 1024;
}

# Stream block for Layer 4 proxying
stream {
    upstream backend {
        hash $remote_addr consistent;
        # Replace with actual server IPs and ports
        server 176.58.111.12:80 weight=5 max_fails=3 fail_timeout=30s;
        server 178.79.163.35:80 weight=5 max_fails=3 fail_timeout=30s;
    }

    server {
        listen 80; # TCP port
        proxy_connect_timeout 1s;
        proxy_timeout 3s;
        proxy_pass backend;
    }
}

After configuration, test the syntax and reload Nginx.

sudo nginx -t
sudo systemctl reload nginx

Management Commands and Path Reference

• Service Management: systemctl start|stop|restart|reload|status nginx
• Check Port Listening: ss -tlnp | grep :80
• Default Web Root: /usr/share/nginx/html
• Main Config File: /etc/nginx/nginx.conf
• Additional Config Directory: /etc/nginx/conf.d/ (typically for HTTP configs)

Conclusion

Compiling and dynamically loading the Stream module allows you to enable or disable Layer 4 proxying without recompiling the entire Nginx binary, providing flexibility for building TCP/UDP load balancers. For production environments, always use updated software versions and adjust configurations according to your network topology and security policies.