Setting Up Real-Time Server Synchronization with Lsyncd (Live Syncing Daemon)

Introduction to Lsyncd

Lsyncd (Live Syncing Daemon) is a lightweight real-time directory synchronization daemon. It monitors a local directory tree (using inotify or fsevents), aggregates events over a period, and then spawns one or more processes (rsync by default) to synchronize the changes. Lsyncd is easy to install, requires no new filesystems or block devices, and does not impact local filesystem performance.

Core Features and Use Cases

Lsyncd combines the inotify mechanism with the rsync tool, wrapped and configured via Lua scripts. Its main advantages are:

• Solves Mass File Sync Issues: By delaying or accumulating events before triggering sync, it avoids the performance problems of frequent file list transfers common in inotify+rsync setups.
• Simple, Flexible Configuration: Uses Lua for configuration files, which are readable and support multiple sync modes (e.g., local copy, local rsync, remote rsync/rsyncssh).
• Design Goal: Ideal for synchronizing a local directory tree to a remote mirror, especially from a secure to a less secure zone.

Compared to other tools:

• DRBD: Operates at the block device level, suitable for high-load systems (like databases), but more complex to configure.
• inotify+rsync / sersync: Less efficient with many or large files; sersync development has ceased.

Prerequisites: SSH Key Authentication

To enable passwordless synchronization, configure SSH key authentication between the source and target servers.

1. Generate Key Pair (on source server):

   ssh-keygen -t rsa -f /root/.ssh/Lsyncd_rsa

2. Distribute Public Key (on source server; replace PUBLIC_IP and PORT with target server details):

   ssh-copy-id -p PORT -i /root/.ssh/Lsyncd_rsa.pub root@PUBLIC_IP

   If the target server already uses key-based login, you may need to manually append the contents of Lsyncd_rsa.pub to the target's /root/.ssh/authorized_keys file.

3. Bidirectional Sync: For two-way synchronization, repeat the above steps on the target server.

Installing Lsyncd and Dependencies

Using CentOS/RHEL as an example, install via the yum package manager.

1. Install rsync:

   yum install rsync -y

2. Install EPEL Repository (if not present):

   yum install epel-release -y
   yum clean all && yum makecache

3. Install Lsyncd and its dependencies:

   yum install lua lua-devel lsyncd -y

Configuring Lsyncd

The main configuration file is /etc/lsyncd.conf. Below is a basic example to sync the local /download directory to the same path on a remote server.

settings {
    logfile      = "/var/log/lsyncd.log",
    statusFile   = "/tmp/lsyncd.stat",
    inotifyMode  = "CloseWrite or Modify",
    maxProcesses = 5,
    maxDelays    = 1
}

sync {
    default.rsync,
    source    = "/download",
    target    = "root@REMOTE_IP:/download",
    rsync     = {
        binary    = "/usr/bin/rsync",
        archive   = true,
        compress  = true,
        verbose   = true
    }
}

Key Configuration Parameters

• settings {}: Global settings.
  • logfile, statusFile: Paths for log and status files.
  • inotifyMode: Types of filesystem events to monitor.
  • maxProcesses: Maximum number of concurrent sync processes.
  • maxDelays: Threshold for accumulated events before triggering a sync.
• sync {}: Sync task definition; multiple can be configured.
  • default.rsync: Sync mode; alternatives include default.direct (local cp/rm) or default.rsyncssh.
  • source: Local source directory.
  • target: Destination address; format varies by mode (local path, user@host:path, host::module).
  • delay: Wait time (seconds) to aggregate multiple changes before syncing.
  • delete: Whether to sync deletions; options: true, false, "startup", "running".
  • rsync {}: Rsync-specific parameters like bwlimit (bandwidth limit) or _extra (extra arguments).

Starting and Managing the Service

1. Start Lsyncd service:

   systemctl start lsyncd

2. Enable auto-start on boot:

   systemctl enable lsyncd

3. Monitor sync logs:

   tail -f /var/log/lsyncd.log

Target Server Configuration (Rsync Daemon Mode)

If using rsync daemon mode (target format: host::module), configure rsyncd on the target server.

1. Install rsync:

   yum install rsync -y

2. Edit configuration file /etc/rsyncd.conf:

   [module_name]
       path = /path/to/target/dir
       comment = Lsyncd Sync Module
       read only = no
       auth users = syncuser
       secrets file = /etc/rsyncd.secrets

3. Create password file and set permissions:

   echo "syncuser:password" > /etc/rsyncd.secrets
   chmod 600 /etc/rsyncd.secrets

4. Start rsync daemon and open firewall port:

   systemctl start rsyncd
   systemctl enable rsyncd
   firewall-cmd --permanent --add-port=873/tcp
   firewall-cmd --reload

Advanced Configuration Examples

The configuration file supports multiple sync blocks for syncing to different targets. Mode selection:

1. Local directory sync (direct mode): Uses cp/rm/mv commands; suitable for many files with infrequent changes.
2. Local directory sync (rsync mode): Uses rsync between local or mounted directories.
3. Remote directory sync (rsync + daemon): Target format user@host::module; requires rsyncd configuration.
4. Remote directory sync (rsync + ssh): Target format user@host:path; utilizes SSH tunnel.
5. Remote directory sync (rsyncssh mode): Similar effect to the previous, with different configuration syntax.

For specific configuration examples, refer to the official Lsyncd manual or community documentation.

Notes and Troubleshooting

• Ensure write permissions on sync directories, especially when running as a non-root user.
• Test key authentication with ssh -v.
• Monitor system logs (/var/log/messages or journalctl -u lsyncd) for troubleshooting.
• Lsyncd version 2.2.1+ requires rsync version ≥ 3.1 on all machines.

Project URL: https://github.com/axkibe/lsyncd