Angie Cluster Setup#

This guide describes the process of creating a fault-tolerant Angie cluster with automatic configuration synchronization and virtual IP address failover.

Related modules

HTTP module: Keepalived queries a server block on a Unix socket.

Preparing Cluster Nodes for Synchronization#

The first step is to prepare all cluster nodes by configuring user accounts and ensuring secure access between servers.

Configuring Users and Access Permissions#

Create a user on all nodes (for example, angie-ha-sync) with sudo privileges:

$ sudo adduser angie-ha-sync

Set a password if necessary:

$ sudo passwd angie-ha-sync

Note

In some operating systems (for example, ALT Linux), you should add the user to the wheel group:

$ sudo usermod -a -G wheel angie-ha-sync

To work with rsync when MAC is enabled in Astra Linux, set the correct integrity level:

$ sudo pdpl-user -i 63 angie-ha-sync

Configure sudo without password:

$ echo "angie-ha-sync ALL=(ALL:ALL) NOPASSWD:ALL" | sudo tee -a /etc/sudoers

On the master node, create SSH keys and copy them to backup nodes:

$ su - angie-ha-sync
$ ssh-keygen -t rsa
$ ssh-copy-id angie-ha-sync@node2_hostname

Warning

Before copying SSH keys, ensure that the /etc/ssh/sshd_config file has the option:

PasswordAuthentication yes

After setting up key-based access, set the value to no to improve security.

Note

For cross-synchronization of Angie configuration, copy the user keys to all nodes:

$ scp -p ~angie-ha-sync/.ssh/id_rsa angie-ha-sync@node2_hostname:.ssh/

Installing Angie and angie-ha-sync#

After preparing the nodes, you need to install the main cluster components: Angie and the configuration synchronization package.

Configure the repository on all nodes according to the instructions for your system packages:

Installing angie-ha-sync#

The configuration synchronization module is available in the following packages:

Angie: angie-ha-sync
Angie PRO: angie-pro-ha-sync

Note

When installing this package on a clean system, the corresponding angie or angie-pro package will also be installed as a dependency.

On all nodes, install the package using your OS package manager, for example:

$ sudo {apk|apt|pkg|yum|zypper} {add|install} angie-pro-ha-sync

Configuring Configuration Synchronization#

The next step is setting up automatic synchronization of configuration files between cluster nodes.

Note

Synchronization principles:

Synchronization is performed via rsync.
Only occurs when the Angie service is running.
Executed manually (command angiehasync -Sd).
Works in one direction: from master node to backup.
rsync runs in daemon mode.

Configuring `rsync`#

Create an rsync configuration (/etc/rsyncd.conf) on the nodes:

[angie] # Directory with Angie configuration
    path = /etc/angie
# User for synchronization
    uid = angie-ha-sync
# User group
    gid = angie-ha-sync
# IP or subnet from which connections are allowed
    hosts allow = 10.21.8.0/24
# Deny all others
    hosts deny = *

Depending on the OS, start the daemon:

$ sudo service rsyncd start # or $ sudo service rsync start

Note

For some systems, there are ready-made instructions:

Configuring the Synchronization File#

Edit /etc/angiehasync/angiehasync.conf:

M_NODE="<node1_hostname>"            # Hostname or IP of this node
TARGET_HOSTS="<node2_hostname>"      # List of hosts/IPs for synchronization (space-separated).
                                     # Can be omitted on backup nodes.
SSH_USER="user"                      # User for synchronization (with administrator privileges)
SSH_ID="/home/$SSH_USER/.ssh/id_rsa" # Path to private key

Note

For cross-synchronization, fill in the TARGET_HOSTS list on all nodes; however, do not include the current node that is currently being configured in the list.

Configuring Health Checks for Angie#

Add a health check server to the http block of the Angie configuration. If you use the packaged /etc/angie/angie.conf, put the server in its own file under /etc/angie/http.d/, for example /etc/angie/http.d/healthcheck.conf, because the http block of the packaged file includes /etc/angie/http.d/*.conf:

server {

    listen unix:/tmp/angie_hcheck.sock; # Unix socket for checking
    access_log off;
    error_log /dev/null;
    default_type text/plain;
    return 200 'ok\n';
}

Check the configuration and restart Angie so that it loads the new file (Debian and Ubuntu packages start Angie on installation, so it may already be running):

$ sudo angie -t && sudo service angie restart

Start synchronization:

$ sudo angiehasync -Sd

Note

The script will automatically check the configuration, perform synchronization with all nodes, and apply it.

Configuring Keepalived#

For automatic failover between cluster nodes, Keepalived is used — a service for managing virtual IP addresses (VIP).

Note

If the keepalived package is not installed — install it:

$ sudo {apk|apt|pkg|yum|zypper} {add|install} keepalived

To bind processes to non-local IP addresses, allow the system to perform corresponding actions:

$ sudo sysctl -w net.ipv4.ip_nonlocal_bind=1

More details: https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt#ip_nonlocal_bind

Suppose VIP 10.21.11.230 is assigned either to the master node (10.21.8.26) or to the backup (10.21.8.27).

If Angie listens on this VIP (listen 10.21.11.230:80;) but the address is not yet assigned, Angie will not be able to start without the ip_nonlocal_bind parameter.

Keepalived Configuration#

On the master node (/etc/keepalived/keepalived.conf):

global_defs {
    enable_script_security
}

vrrp_script angie_check {
    script "/usr/bin/curl -s --connect-timeout 5 -A 'angie_hcheck_script'
    --no-buffer -XGET --unix-socket /tmp/angie_hcheck.sock http://hcheck/"
    interval 5 user angie
}

vrrp_instance angie {
    state MASTER interface enp0s2 virtual_router_id 254 priority 100
    advert_int 2 unicast_src_ip 10.21.8.26

    unicast_peer {
        10.21.8.27
    }

    virtual_ipaddress {
        10.21.11.230
    } track_script {
        angie_check
    }
}

On the backup node:

global_defs {
    enable_script_security
}

vrrp_script angie_check {
    script "/usr/bin/curl -s --connect-timeout 5 -A 'angie_hcheck_script'
    --no-buffer -XGET --unix-socket /tmp/angie_hcheck.sock http://hcheck/"
    interval 5 user angie
}

vrrp_instance angie {
    state MASTER interface enp0s2 virtual_router_id 254 priority 99
    advert_int 2 unicast_src_ip 10.21.8.27

    unicast_peer {
        10.21.8.26
    }

    virtual_ipaddress {
        10.21.11.230
    } track_script {
        angie_check
    }
}

Note

In the vrrp_instance angie section, set the following values:

unicast_src_ip — IP of the current node
unicast_peer — IP of neighboring nodes
virtual_ipaddress — virtual IP (VIP)
interface — network interface

Start the service:

$ sudo keepalived -t && sudo service keepalived start

Keepalived Configuration Breakdown#

Let's examine the main elements of the Keepalived configuration in detail to understand the cluster operation principles.

The configuration includes two parts:

global_defs — global settings
vrrp_instance — VRRP parameters (VIP switching)

Main elements:

enable_script_security — allows execution of health check scripts
vrrp_script — Angie health check script
state MASTER — initial node state
priority — priority (MASTER role is assigned to the highest)
advert_int — VRRP advertisement interval
unicast_src_ip — current node IP
unicast_peer — neighbor IPs
virtual_ipaddress — VIP address
track_script — availability monitoring via health check scripts

Note

If the original master node recovers, it will regain the MASTER role (higher priority). To disable failback, use the nopreempt parameter:

vrrp_instance angie {
    ... nopreempt
}

Testing Cluster Operation#

After completing the configuration, it's necessary to test the cluster operation and ensure correct switching between nodes.

Check VIP status:

$ ip addr show enp0s2 | grep "10.21.11.230"

Test fault tolerance:

Stop Angie on the master node:

$ sudo service angie stop

Check VIP transition to the backup node:

$ ip addr show enp0s2 | grep "10.21.11.230"

Start Angie on the master node again:

$ sudo service angie start

After this, the VIP should return to the master node.

Angie Cluster Setup#

Preparing Cluster Nodes for Synchronization#

Configuring Users and Access Permissions#

Installing Angie and angie-ha-sync#

Installing angie-ha-sync#

Configuring Configuration Synchronization#

Configuring rsync#

Configuring the Synchronization File#

Configuring Health Checks for Angie#

Configuring Keepalived#

Keepalived Configuration#

Keepalived Configuration Breakdown#

Testing Cluster Operation#

Configuring `rsync`#