<!-- review: finished -->

<a id="cluster-setup"></a>

# Angie Cluster Setup

This guide describes the process of creating a fault-tolerant Angie cluster
with automatic configuration synchronization
and virtual IP address failover.

<a id="cluster-preparation"></a>

## Preparing Cluster Nodes for Synchronization

The first step is to prepare all cluster nodes
by configuring user accounts
and ensuring secure access between servers.

<a id="users-permissions"></a>

### Configuring Users and Access Permissions

Create a user on all nodes
(for example, `angie-ha-sync`) with `sudo` privileges:

```console
$ sudo adduser angie-ha-sync
```

Set a password if necessary:

```console
$ sudo passwd angie-ha-sync
```

#### NOTE
In some operating systems (for example, Alt Linux),
you should add the user to the `wheel` group:

```console
$ sudo usermod -a -G wheel angie-ha-sync
```

To work with `rsync` when MAC is enabled in Astra Linux, set
the correct integrity level:

```console
$ sudo pdpl-user -i 63 angie-ha-sync
```

Configure sudo without password:

```console
$ echo "angie-ha-sync ALL=(ALL:ALL) NOPASSWD:ALL" | sudo tee -a /etc/sudoers
```

On the master node, create SSH keys and copy them to backup nodes:

```console
$ su - angie-ha-sync
$ ssh-keygen -t rsa
$ ssh-copy-id angie-ha-sync@node2_hostname
```

#### WARNING
Before copying SSH keys, ensure that the
`/etc/ssh/sshd_config` file has the option:

`PasswordAuthentication yes`

After setting up key-based access, set the value to `no` to improve
security.

#### NOTE
For cross-synchronization of Angie configuration, copy the user keys
to all nodes:

```console
$ scp -p ~angie-ha-sync/.ssh/id_rsa angie-ha-sync@node2_hostname:.ssh/
```

<a id="install"></a>

## Installing Angie and angie-ha-sync

After preparing the nodes, you need to install the main cluster components:
Angie and the configuration synchronization package.

Configure the repository on all nodes according to the instructions for your system packages:

- [Instructions for Angie](https://en.angie.software//angie/docs/installation/oss_packages.md#oss-packages)
- [Instructions for Angie PRO](https://en.angie.software//angie/docs/installation/pro_packages.md#pro-packages)

<a id="install-ha-sync"></a>

### Installing angie-ha-sync

The configuration synchronization module is available in the following packages:

- Angie: `angie-ha-sync`
- Angie PRO: `angie-pro-ha-sync`

#### NOTE
When installing this package on a clean system,
the corresponding `angie` or `angie-pro` package
will also be installed as a dependency.

On all nodes, install the package using your OS package manager, for example:

```console
$ sudo {apk|apt|pkg|yum|zypper} {add|install} angie-pro-ha-sync
```

<a id="sync"></a>

## Configuring Configuration Synchronization

The next step is setting up automatic synchronization of configuration files
between cluster nodes.

#### NOTE
Synchronization principles:

- Synchronization is performed via `rsync`.
- Only occurs when the Angie service is running.
- Executed manually (command `angiehasync -Sd`).
- Works in one direction: from master node to backup.
- `rsync` runs in daemon mode.

<a id="rsync"></a>

### Configuring `rsync`

Create an `rsync` configuration (`/etc/rsyncd.conf`) on the nodes:

```ini
[angie] # Directory with Angie configuration
    path = /etc/angie
# User for synchronization
    uid = angie-ha-sync
# User group
    gid = angie-ha-sync
# IP or subnet from which connections are allowed
    hosts allow = 10.21.8.0/24
# Deny all others
    hosts deny = *
```

Depending on the OS, start the daemon:

```console
$ sudo service rsyncd start # or $ sudo service rsync start
```

#### NOTE
For some systems, there are ready-made instructions:

- [Alt](https://www.altlinux.org/Настройка_rsync-сервера)
- [Astra](https://wiki.astralinux.ru/pages/viewpage.action?pageId=41191598#id-Архивированиеивосстановлениефайловссохранениеммандатныхатрибутов-RSYNC)

<a id="sync-config"></a>

### Configuring the Synchronization File

Edit `/etc/angiehasync/angiehasync.conf`:

```ini
M_NODE="<node1_hostname>"            # Hostname or IP of this node
TARGET_HOSTS="<node2_hostname>"      # List of hosts/IPs for synchronization (space-separated).
                                     # Can be omitted on backup nodes.
SSH_USER="user"                      # User for synchronization (with administrator privileges)
SSH_ID="/home/$SSH_USER/.ssh/id_rsa" # Path to private key
```

#### NOTE
For cross-synchronization,
fill in the `TARGET_HOSTS` list on all nodes;
however, do not include the current node
that is currently being configured in the list.

<a id="healthcheck"></a>

### Configuring Health Checks for Angie

Add a health check block to the Angie configuration
(`/etc/angie/angie.conf`):

```ini
server {

    listen unix:/tmp/angie_hcheck.sock; # Unix socket for checking
    access_log off;
    error_log /dev/null;
    default_type text/plain;
    return 200 'ok\n';
}
```

Start Angie:

```console
$ sudo angie -t && sudo service angie start
```

Start synchronization:

```console
$ sudo angiehasync -Sd
```

#### NOTE
The script will automatically check the configuration, perform synchronization with all
nodes, and apply it.

<a id="keepalived"></a>

## Configuring Keepalived

For automatic failover between cluster nodes, Keepalived is used —
a service for managing virtual IP addresses (VIP).

#### NOTE
If the `keepalived` package is not installed — install it:

```console
$ sudo {apk|apt|pkg|yum|zypper} {add|install} keepalived
```

To bind processes to non-local IP addresses,
allow the system to perform corresponding actions:

```console
$ sudo sysctl -w net.ipv4.ip_nonlocal_bind=1
```

More details:
[https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt#ip_nonlocal_bind](https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt#ip_nonlocal_bind)

Suppose VIP `10.21.11.230`
is assigned either to the master node (`10.21.8.26`)
or to the backup (`10.21.8.27`).

If Angie listens on this VIP (`listen 10.21.11.230:80;`)
but the address is not yet assigned,
Angie will not be able to start without the `ip_nonlocal_bind` parameter.

<a id="keepalived-config"></a>

### Keepalived Configuration

On the master node (`/etc/keepalived/keepalived.conf`):

```ini
global_defs {
    enable_script_security
}

vrrp_script angie_check {
    script "/usr/bin/curl -s --connect-timeout 5 -A 'angie_hcheck_script'
    --no-buffer -XGET --unix-socket /tmp/angie_hcheck.sock http://hcheck/"
    interval 5 user angie
}

vrrp_instance angie {
    state MASTER interface enp0s2 virtual_router_id 254 priority 100
    advert_int 2 unicast_src_ip 10.21.8.26

    unicast_peer {
        10.21.8.27
    }

    virtual_ipaddress {
        10.21.11.230
    } track_script {
        angie_check
    }
}
```

On the backup node:

```ini
global_defs {
    enable_script_security
}

vrrp_script angie_check {
    script "/usr/bin/curl -s --connect-timeout 5 -A 'angie_hcheck_script'
    --no-buffer -XGET --unix-socket /tmp/angie_hcheck.sock http://hcheck/"
    interval 5 user angie
}

vrrp_instance angie {
    state MASTER interface enp0s2 virtual_router_id 254 priority 99
    advert_int 2 unicast_src_ip 10.21.8.27

    unicast_peer {
        10.21.8.26
    }

    virtual_ipaddress {
        10.21.11.230
    } track_script {
        angie_check
    }
}
```

#### NOTE
In the `vrrp_instance angie` section, set the following values:

- `unicast_src_ip` — IP of the current node
- `unicast_peer` — IP of neighboring nodes
- `virtual_ipaddress` — virtual IP (VIP)
- `interface` — network interface

Start the service:

```console
$ sudo keepalived -t && sudo service keepalived start
```

<a id="keepalived-details"></a>

### Keepalived Configuration Breakdown

Let's examine the main elements of the Keepalived configuration in detail
to understand the cluster operation principles.

The configuration includes two parts:

- `global_defs` — global settings
- `vrrp_instance` — VRRP parameters (VIP switching)

Main elements:

- `enable_script_security` — allows execution of health check scripts
- `vrrp_script` — Angie health check script
- `state MASTER` — initial node state
- `priority` — priority (MASTER role is assigned to the highest)
- `advert_int` — VRRP advertisement interval
- `unicast_src_ip` — current node IP
- `unicast_peer` — neighbor IPs
- `virtual_ipaddress` — VIP address
- `track_script` — availability monitoring via health check scripts

#### NOTE
If the original master node recovers,
it will regain the MASTER role (higher priority).
To disable failback, use the `nopreempt` parameter:

```none
vrrp_instance angie {
    ... nopreempt
}
```

<a id="testing"></a>

## Testing Cluster Operation

After completing the configuration, it's necessary to test the cluster operation
and ensure correct switching between nodes.

Check VIP status:

```console
$ ip addr show enp0s2 | grep "10.21.11.230"
```

Test fault tolerance:

Stop Angie on the master node:

```console
$ sudo service angie stop
```

Check VIP transition to the backup node:

```console
$ ip addr show enp0s2 | grep "10.21.11.230"
```

Start Angie on the master node again:

```console
$ sudo service angie start
```

After this, the VIP should return to the master node.