Load Balancing

Angie distributes client requests across a group of upstream servers described by an upstream block. Which server handles a given request is decided by the group's balancing method.

This guide covers HTTP balancing, configured in the HTTP Upstream module. Balancing of TCP and UDP sessions works the same way and is configured in the Stream Upstream module; differences are noted where they matter.

A group uses exactly one balancing method. If a block names more than one, the last one takes effect and Angie logs load balancing method redefined; a block that names none uses Weighted Round Robin.

Related modules

• HTTP Upstream module: methods, server weights, zone, sticky.

• Stream Upstream module: the same methods, except ip_hash.

• HTTP Upstream Probe module: active health checks in Angie PRO.

• HTTP Proxy module: proxy_next_upstream sets the retry conditions.

Choosing a Method

A method determines how Angie picks a server for each request, and each method accepts its own set of server parameters. A parameter that a method does not support is not ignored: Angie refuses to start and reports balancing method does not support parameter. Check this table before combining a method with weight, slow_start, or backup.

Method	Distributes requests by	Level	weight	slow_start	backup
Weighted Round Robin	Position in a weighted rotation. The default; no directive.	L4, L7
hash	Hash of a key you choose, such as a URI or a header.	L4, L7
ip_hash	Hash of the client address.	L7
least_conn	Fewest active connections per unit of weight.	L4, L7
random	Chance, in proportion to weight.	L4, L7
least_time	Chance, in inverse proportion to average response time.	L4, L7
feedback (PRO)	Chance, in proportion to a value the servers report.	L4, L7

Every method above is available in Angie except feedback, which requires Angie PRO. ip_hash has no Stream counterpart; use hash with $remote_addr instead.

Balancing Methods

Weighted Round Robin

Round robin is the default method: Angie uses it whenever a group names no other method, and it has no directive of its own. Angie rotates through the group's servers, giving each one a share of the rotation proportional to its weight:

upstream backend {
    zone backend 1m;
    server backend1.example.com weight=5;
    server backend2.example.com;
    server backend3.example.com;
}

Of every seven requests, five go to backend1.example.com and one to each of the others. The distribution is smooth: a higher-weight server's requests are spread across the rotation rather than sent in a single burst, so the group never sees five consecutive requests for one server followed by two for the rest.

Round robin is also the fallback for other methods. When hash, ip_hash, or random cannot settle on a usable server, they hand the choice to round robin.

hash

The hash method sends all requests whose keys have the same hash to the same server. This helps when the group's servers cache responses themselves, such as caching proxies or memcached nodes. While the group's membership stays the same and each server stands for one address, requests for a given URI always reach the same server, so its response is cached on that one server rather than on every server in the group.

upstream backend {
    zone backend 1m;
    hash $request_uri consistent;
    server backend1.example.com;
    server backend2.example.com;
}

The key is any string of variables. Weights work by giving a heavier server proportionally more of the key space.

Without consistent, Angie maps the key's hash onto a range as long as the group's total weight, where each server holds a segment as long as its weight. Adding or removing a server changes that total and therefore re-maps many keys beyond those assigned to the added or removed server: their cached responses stay on servers that no longer receive requests for them. With consistent, Angie places the servers on a ketama ring instead, and removing a server re-maps only the keys that server held.

Note

A server's ring positions are computed from its name as written in the server directive, not from the addresses the name resolves to; with the service parameter, they are computed from the target name of each SRV record, without its port. All addresses of one name share those positions, so the name occupies as much of the ring as a single server of the same weight. When a key lands on one of the name's positions, Angie picks one of its addresses by round robin. A key is therefore not tied to any single address of a name that resolves to several addresses, and a group whose only server is such a name distributes requests as round robin does.

Without consistent, each address counts as a separate server with its own weight, and the hash is mapped onto the total weight of all addresses. A name that resolves to several addresses therefore receives a share proportional to the number of those addresses. Angie does not sort the addresses, so the address a key reaches can differ between Angie instances.

To tie each key to one address on every Angie instance, with or without consistent, give each address its own server directive, either as the address itself or as a name that resolves to it alone. With consistent, the service parameter does the same when no two SRV records share a target name and each target resolves to a single address.

ip_hash

The ip_hash method keys on the client address: the first three octets of an IPv4 address, or the whole IPv6 address. Requests from one client reach one server for as long as that server is available.

It has no consistent-hashing mode, so changing the set of servers may redirect many clients to other servers. To take a server out of service while preserving the rest of the mapping, mark it down rather than deleting its line:

upstream backend {
    zone backend 1m;
    ip_hash;
    server backend1.example.com;
    server backend2.example.com down;
    server backend3.example.com;
}

least_conn

The least_conn method picks the server with the fewest active connections per unit of weight — it minimizes the ratio of connections to weight, not the raw count. A server at weight=2 holding 15 connections is therefore preferred over one at weight=1 holding 10. When several servers tie, they are tried in turn by weighted round robin.

Only connections currently carrying a request count. Idle keepalive connections do not.

Use it when requests differ widely in how long they occupy a server: large downloads, long polling, WebSocket. Round robin sends each server the same number of requests per unit of weight, but that number says little about load when some requests hold a server for milliseconds and others for minutes; least_conn counts the requests a server is handling right now. If the servers instead differ in speed, use least_time.

random

The random method picks a server at random, in proportion to weight. Over many requests the distribution matches the weights.

The two parameter picks two servers at random and then takes whichever of the two has fewer active connections per unit of weight:

upstream backend {
    zone backend 1m;
    random two;
    server backend1.example.com;
    server backend2.example.com;
    server backend3.example.com;
}

This matters when several balancers front the same servers without sharing state. Each balancer choosing the single least-loaded server drives them all at the same target; choosing the better of two random candidates spreads the load instead. With a single balancer, least_conn is the simpler choice.

Note

random two least_conn is accepted for compatibility and behaves exactly like random two; comparing active connections is the only mode two has.

least_time

The least_time method favors servers that answer quickly. Its mode sets what is measured: header is the time to receive the response header, last_byte the time to receive the whole response.

A group whose servers are compared by the time to the response header:

upstream backend {
    zone backend 1m;
    least_time header;
    server backend1.example.com;
    server backend2.example.com;
}

The measurement is an exponentially weighted moving average, so a single slow response does not move a server far. The response_time_factor directive, or the factor parameter, sets how much weight the accumulated history keeps against each new measurement: a high factor reacts slowly, a low one forgets quickly.

Two properties are worth knowing before you choose it.

It does not simply pick the fastest server. A server's chance of receiving the request is inversely proportional to its average response time. Even a consistently slow server keeps a small share — roughly one percent of what an average server receives, so about one request in 100 x N for a group of N servers. This is deliberate: a server that received nothing would never produce a new measurement, and could never be found to have recovered.

It rejects weight. A group that combines least_time with weight= on any server will not start. Response time already reflects how much load a server is absorbing, so there is nothing left for a weight to express.

In Stream, least_time accepts a different set of modes: connect (the time to establish a connection to the server), first_byte (the time to receive the first byte of data from the server), and last_byte (the time until the session ends); header is not available.

feedback

The feedback (PRO) method lets the servers themselves say how much work they want. Angie reads a value out of each response — typically a header — and gives a server a share of requests in proportion to it, averaged the same way least_time averages response time.

upstream backend {
    zone backend 1m;
    feedback $upstream_http_x_capacity;
    server backend1.example.com;
    server backend2.example.com;
}

Warning

The value is read in the direction you would expect for spare capacity: a higher reported value earns a larger share. If your metric rises as the server gets busier — CPU load, queue depth — add the inverse parameter, or you will send the most traffic to the most loaded server.

The account parameter selects which responses feed the average, which lets you drive balancing from upstream_probe (PRO) responses rather than from client traffic.

Server Weights and Traffic Shares

weight does not mean the same thing to every method, and only for some of them does a server's share of traffic equal its weight divided by the sum of all weights:

Method	What weight does
round robin, random	Sets the share directly. Share equals weight divided by the total weight.
hash, ip_hash	Sets the share of the key space. Traffic matches the weights only if the keys themselves are evenly spread; real client addresses and URIs usually are not.
least_conn	Divides the connection count being compared. A heavier server is driven to a proportionally higher number of concurrent connections, not to a proportionally higher number of requests.
least_time	Nothing — the parameter is rejected at startup.
feedback	Multiplies the value the server reports. Weight is one of two factors, so the share is not predictable from the weight alone.

Once max_conns, slow_start, max_fails, down, or backup servers come into play, no method distributes traffic strictly by weight.

Failure and Recovery

Passive health checks

Angie judges a server by the outcome of requests sent to it. If an attempt matches a retry condition and the attempt limit is not exhausted, Angie can pass the request to a different server. The proxy_next_upstream directive and its counterparts define those conditions; proxy_next_upstream_tries and proxy_next_upstream_timeout limit the number and duration of attempts.

max_fails unsuccessful attempts within fail_timeout mark a server unavailable for that same period, after which it is tried again. The defaults are one attempt and ten seconds. Errors, timeouts, and invalid response headers always count as unsuccessful attempts; a response status counts only if proxy_next_upstream or its counterpart lists it, and 403 and 404 never count.

Setting max_fails=0 disables the check for that server: it is never marked unavailable.

Note

Below the max_fails threshold, an individual failure still reduces a server's share, which recovers gradually. Without slow_start, Angie reduces the effective weight used by round robin and least_conn, then restores it over subsequent requests. With slow_start, each failure moves the server backward on the time-based ramp used by every method that accepts the parameter. Both behaviors require max_fails greater than one: at the default value of one, the first failure marks the server unavailable, so there is no intermediate state.

Active health checks

Passive checks learn about a server only from client requests: a failure is detected after client requests to the server have already failed, and a recovery only when a client request is sent to it again. Active checks, available in Angie PRO, contact the servers on a schedule regardless of client traffic, and notice both failure and recovery without spending client requests on either.

A check is set up by the upstream_probe (PRO) directive in the same location block that passes requests to the group; the group itself needs a shared memory zone:

upstream backend {
    zone backend 1m;
    server backend1.example.com;
    server backend2.example.com;
}

map $upstream_status $good {
    200     "1";
}

server {
    listen 80;

    location / {
        proxy_pass http://backend;
        upstream_probe backend_probe uri=/health interval=5s
                       test=$good fails=3 passes=2;
    }
}

Every five seconds, Angie requests /health from each server in the group. Three failed checks in a row mark a server unhealthy; two passed checks in a row return it to service. Without the test parameter, a check fails only when a client request in this location would count as an unsuccessful attempt. By default, proxy_next_upstream counts only an error or a timeout, so a 500 or 503 response does not fail the check. Here, test=$good passes the check only if the server responds with 200: for any other status, map leaves the variable empty.

Check responses can also feed the feedback method through its account parameter.

A server found unhealthy is excluded from selection by every balancing method. After a configuration reload, checks start over. By default, a server keeps receiving requests without waiting for the first check after the reload. If upstream_probe has the essential parameter, the server receives no requests until the check first finds it healthy (in this example, after two passed checks in a row); adding persistent, which works only together with essential, lifts this wait for servers that were healthy before the reload, because their health flag is kept in the zone (see State Shared Between Worker Processes).

Backup servers

Servers marked backup are held in reserve:

upstream backend {
    zone backend 1m;
    server backend1.example.com;
    server backend2.example.com;

    server backup1.example.com backup;
}

Every request starts at the primary servers and reaches the backup group only when no primary server can take it. There is no lasting failover mode to return from — as soon as a primary server passes its health check again, the next request uses it.

Angie PRO can invert this with backup_switch (PRO) permanent, which keeps sending requests to the group that last served them. Without a time argument the switch never reverts on its own; with permanent=time, Angie sends one request back to the primary group after that interval to see whether it has recovered.

The backup parameter is rejected by hash, ip_hash and random.

Returning a server to service

The slow_start parameter ramps a recovered server back up to its full weight over a period instead of restoring it at once, which keeps a cold cache or a just-started process from being hit at full rate:

server backend1.example.com slow_start=30s;

It most often applies after a server has been marked unavailable and has recovered; an individual failure also moves the server backward on the same ramp (see Failure and Recovery). It works only with the methods marked in the slow_start column under Choosing a Method. Unlike some other implementations, it does not require the zone directive.

To take a server out of rotation deliberately, use down; to stop sending it new traffic while letting bound sessions finish, use drain.

State Shared Between Worker Processes

Balancing is only coherent across an Angie instance if the workers share their view of the group. The zone directive gives them one:

upstream backend {
    zone backend 1m;
    server backend1.example.com;
    server backend2.example.com;
}

Without it, each worker process keeps its own copy of the group's runtime state — failure counters, active connection counts, the position in the round-robin rotation, and response-time averages — and balances independently of the others. With N workers you effectively run N balancers over the same servers: one worker can have a server marked unavailable while the rest keep sending it requests, and a max_conns limit applies per worker, allowing up to N times as many connections as configured.

A zone is also what makes the group visible to upstream metrics, and it is required by the resolve and service parameters of the server directive.

Note

A configuration reload creates the zone again whether or not size changes: counters, passive-check state, and accumulated balancing history start from zero. Two things carry over: the addresses resolved for servers with resolve, and the health flag that active checks with the persistent parameter rely on (see Active health checks).

Session Persistence and Related Controls

Balancing decides where a request goes. Two directives constrain that decision:

sticky	Binds a client session to the server that first served it, identified by a cookie, a route value, or — with learn — a value Angie observes in the traffic. Session binding overrides the balancing method for as long as the session lasts.
queue (PRO)	Holds a request briefly when no server can take it, instead of failing it immediately. Useful when max_conns is reached during short load spikes.

The queue directive requires Angie PRO.

The keepalive directive does not affect which server is chosen: it keeps idle connections to the servers open, so that a request to an already-chosen server reuses one of them instead of opening a new one. The number of such connections is capped per worker process, not in total.

Warning

The sticky, keepalive, and queue directives must appear after the directive that sets the balancing method; otherwise they have no effect.