<!-- review: finished -->

<a id="http-upstream"></a>

# Upstream

Provides a context for describing groups of servers that can be used in the [proxy_pass](https://en.angie.software//angie/docs/configuration/modules/http/http_proxy.md#proxy-pass), [fastcgi_pass](https://en.angie.software//angie/docs/configuration/modules/http/http_fastcgi.md#fastcgi-pass), [uwsgi_pass](https://en.angie.software//angie/docs/configuration/modules/http/http_uwsgi.md#uwsgi-pass), [scgi_pass](https://en.angie.software//angie/docs/configuration/modules/http/http_scgi.md#scgi-pass), [memcached_pass](https://en.angie.software//angie/docs/configuration/modules/http/http_memcached.md#memcached-pass), and [grpc_pass](https://en.angie.software//angie/docs/configuration/modules/http/http_grpc.md#grpc-pass) directives.

<a id="configuration-example-45"></a>

## Configuration Example

```nginx
upstream backend {
    zone backend 1m;
    server backend1.example.com       weight=5;
    server backend2.example.com:8080;
    server backend3.example.com       service=_example._tcp resolve;
    server unix:/tmp/backend3;

    server backup1.example.com:8080   backup;
    server backup2.example.com:8080   backup;
}

resolver 127.0.0.53 status_zone=resolver;

server {
    location / {
        proxy_pass http://backend;
    }
}
```

<a id="directives-48"></a>

## Directives

<a id="index-0"></a>

<a id="u-backup-switch"></a>

### backup_switch (PRO)

#### Versionadded
Added in version 1.9.0: PRO

| [Syntax](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)   | `backup_switch` `permanent`[=time];   |
|------------------------------------------------------------------------------------------|---------------------------------------|
| Default                                                                                  | —                                     |
| [Context](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)  | upstream                              |

The directive enables the ability to start server selection not from the primary group,
but from the *active* group, i.e., the one where a server was successfully found last time.
If a server cannot be found in the active group for the next request,
and the search moves to the backup group,
then this group becomes active,
and subsequent requests are first directed to servers in this group.

If the `permanent` parameter is defined without a [time](https://en.angie.software//angie/docs/configuration/configfile.md#syntax) value,
the group remains active after selection,
and automatic rechecking of groups with lower levels does not occur.
If time is specified,
the active status of the group expires after the specified interval,
and the balancer again checks groups with lower levels,
returning to them if the servers are working normally.

Example:

```nginx
upstream my_backend {
    server primary1.example.com;
    server primary2.example.com;

    server backup1.example.com backup;
    server backup2.example.com backup;

    backup_switch permanent=2m;
}
```

If the balancer switches from primary servers to the backup group,
all subsequent requests are handled by this backup group for 2 minutes.
After 2 minutes elapse, the balancer rechecks the primary servers
and makes them active again if they are working normally.

<a id="index-1"></a>

<a id="u-bind-conn"></a>

### bind_conn (PRO)

| [Syntax](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)   | `bind_conn` value;   |
|------------------------------------------------------------------------------------------|----------------------|
| Default                                                                                  | —                    |
| [Context](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)  | upstream             |

Allows binding a server connection to a client connection when the
value, specified as a string of variables, becomes different from `""` and
`"0"`.

#### WARNING
The `bind_conn` directive must be used after all directives
that set a load balancing method,
otherwise it will not work.
If it is used together with the [sticky](#u-sticky) directive,
then `bind_conn` must come after `sticky`.

#### WARNING
When using the directive, the [Proxy](https://en.angie.software//angie/docs/configuration/modules/http/http_proxy.md#http-proxy) module settings
must allow the use of persistent connections, for example:

```nginx
proxy_http_version 1.1;
proxy_set_header Connection "";
```

A typical use case for the directive is proxying connections with
NTLM authentication, where it is necessary to ensure client-to-server binding at
the beginning of negotiation:

```nginx
map $http_authorization   $ntlm {
    ~*^N(?:TLM|egotiate)  1;
}

upstream ntlm_backend {
    server 127.0.0.1:8080;
    bind_conn $ntlm;
}

server {
    # ...
    location / {
        proxy_pass http://ntlm_backend;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
        # ...
    }
}
```

<a id="index-2"></a>

<a id="u-feedback"></a>

### feedback (PRO)

| [Syntax](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)   | `feedback` variable [`inverse`] [`factor=`number] [`account=`condition_variable] [`last_byte`];   |
|------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------|
| Default                                                                                  | —                                                                                                 |
| [Context](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)  | upstream                                                                                          |

Sets up a feedback-based load balancing mechanism in the `upstream`.
It dynamically adjusts balancing decisions
by multiplying the weight of each proxied server by the average feedback value,
which changes over time depending on the value of the variable
and is subject to an optional condition.

The following parameters can be specified:

| `variable`   | The variable from which the feedback value is taken.<br/>It should represent a performance or health metric;<br/>it is assumed that the server provides it in headers or otherwise.<br/><br/>The value is evaluated with each response from the server<br/>and is factored into the moving average<br/>according to the `inverse` and `factor` settings.                                                                                                                                                                                                                                                                                                                                                                                       |
|--------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `inverse`    | If the parameter is set, the feedback value is interpreted inversely:<br/>lower values indicate better performance.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| `factor`     | The factor by which the feedback value is considered<br/>when calculating the average.<br/>Valid values are integers from 0 to 99.<br/>Default is `90`.<br/><br/>The average is calculated using the [exponential smoothing](https://en.wikipedia.org/wiki/Exponential_smoothing) formula.<br/><br/>The larger the factor, the less new values affect the average;<br/>if `90` is specified, then 90% of the previous value<br/>and only 10% of the new value will be taken.                                                                                                                                                                                                                                                                   |
| `account`    | Specifies a condition variable<br/>that controls which responses are considered in the calculation.<br/>The average value is updated with the feedback value from the response<br/>only if the condition variable of that response<br/>is not equal to `""` or `"0"`.<br/><br/>#### NOTE<br/>By default, responses during [active checks](https://en.angie.software//angie/docs/configuration/modules/http/http_upstream_probe.md#u-upstream-probe)<br/>are not included in the calculation;<br/>combining the [$upstream_probe](https://en.angie.software//angie/docs/configuration/modules/http/http_upstream_probe.md#v-upstream-probe) variable<br/>with `account` allows including these responses<br/>or even excluding everything else. |
| `last_byte`  | Allows processing data from the proxied server after receiving<br/>the complete response, not just the header.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |

Example:

```nginx
upstream backend {

    zone backend 1m;

    feedback $feedback_value factor=80 account=$condition_value;

    server backend1.example.com;
    server backend2.example.com;
}

map $upstream_http_custom_score $feedback_value {
    "high"                      100;
    "medium"                    75;
    "low"                       50;
    default                     10;
}

map $upstream_probe $condition_value {
    "high_priority" "1";
    "low_priority"  "0";
    default         "1";
}
```

This configuration categorizes server responses by feedback levels
based on specific scores from response header fields,
and also adds a condition on [$upstream_probe](https://en.angie.software//angie/docs/configuration/modules/http/http_upstream_probe.md#v-upstream-probe)
to consider only responses from the `high_priority` active check
or responses to regular client requests.

<a id="index-3"></a>

<a id="u-hash"></a>

### hash

| [Syntax](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)   | `hash` key [`consistent`];   |
|------------------------------------------------------------------------------------------|------------------------------|
| Default                                                                                  | —                            |
| [Context](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)  | upstream                     |

Specifies a load balancing method for a server group where the client-server mapping is determined using the hashed key value. The key can contain text, variables, and their combinations. Note that adding or removing a server from the group may result in remapping most of the keys to different servers. The method is compatible with the [Cache::Memcached](https://metacpan.org/pod/Cache::Memcached) Perl library.

```nginx
hash $remote_addr;
```

When using domain names that resolve to multiple IP addresses
(for example, with the `resolve` parameter),
the server does not sort the received addresses, so their order may differ
across different servers, which affects client distribution.
To ensure consistent distribution,
use the `consistent` parameter.

If the `consistent` parameter is specified, the [ketama](https://www.metabrew.com/article/libketama-consistent-hashing-algo-memcached-clients) consistent hashing method will be used instead. The method ensures that only a few keys will be remapped to different servers when a server is added to or removed from the group. This helps to achieve a higher cache hit ratio for caching servers. The method is compatible with the [Cache::Memcached::Fast](https://metacpan.org/pod/Cache::Memcached::Fast) Perl library with the `ketama_points` parameter set to 160.

<a id="index-4"></a>

<a id="u-ip-hash"></a>

### ip_hash

| [Syntax](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)   | `ip_hash`;   |
|------------------------------------------------------------------------------------------|--------------|
| Default                                                                                  | —            |
| [Context](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)  | upstream     |

Specifies a load balancing method for the group where requests are distributed among servers based on client IP addresses. The first three octets of the client's IPv4 address or the entire IPv6 address are used as a hashing key. The method ensures that requests from the same client will always be passed to the same server except when this server is unavailable. In that case, client requests will be passed to another server. Most likely this will also be the same server.

If one of the servers needs to be temporarily removed, it should be marked with the `down` parameter to preserve the current hashing of client IP addresses:

```nginx
upstream backend {
    ip_hash;

    server backend1.example.com;
    server backend2.example.com;
    server backend3.example.com down;
    server backend4.example.com;
}
```

<a id="index-5"></a>

<a id="u-keepalive"></a>

### keepalive

| [Syntax](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)   | `keepalive` connections;   |
|------------------------------------------------------------------------------------------|----------------------------|
| Default                                                                                  | —                          |
| [Context](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)  | upstream                   |

Activates the cache of connections to upstream servers.

The `connections` parameter sets the maximum number of idle keepalive connections to upstream servers that are preserved in the cache of each worker process. When this number is exceeded, the least recently used connections are closed.

#### NOTE
It should be particularly noted that the `keepalive` directive does not limit the total number of connections to upstream servers that Angie worker processes can open. The `connections` parameter should be set low enough to let upstream servers process new incoming connections as well.

#### WARNING
The `keepalive` directive must be used after all directives that set
a load balancing method, otherwise it will not work.

Example configuration of memcached upstream with keepalive connections:

```nginx
upstream memcached_backend {
    server 127.0.0.1:11211;
    server 10.0.0.2:11211;

    keepalive 32;
}

server {
    #...

    location /memcached/ {
        set $memcached_key $uri;
        memcached_pass memcached_backend;
    }

}
```

For HTTP, the [proxy_http_version](https://en.angie.software//angie/docs/configuration/modules/http/http_proxy.md#proxy-http-version) directive should be set to "1.1" and the `Connection` header field should be cleared:

```nginx
upstream http_backend {
    server 127.0.0.1:8080;

    keepalive 16;
}

server {
    #...

    location /http/ {
        proxy_pass http://http_backend;
        proxy_http_version 1.1;
        proxy_set_header Connection "";
    #    ...
    }
}
```

#### NOTE
Alternatively, HTTP/1.0 persistent connections can be used by passing the "Connection: Keep-Alive" header field to an upstream server, though this method is not recommended.

For FastCGI servers, it is required to set [fastcgi_keep_conn](https://en.angie.software//angie/docs/configuration/modules/http/http_fastcgi.md#fastcgi-keep-conn) for keepalive connections to work:

```nginx
upstream fastcgi_backend {
    server 127.0.0.1:9000;

    keepalive 8;
}

server {
    #...

    location /fastcgi/ {
        fastcgi_pass fastcgi_backend;
        fastcgi_keep_conn on;
    #    ...
    }
}
```

#### NOTE
SCGI and uwsgi protocols do not have a concept of keepalive connections.

<a id="index-6"></a>

<a id="u-keepalive-requests"></a>

### keepalive_requests

| [Syntax](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)   | `keepalive_requests` number;   |
|------------------------------------------------------------------------------------------|--------------------------------|
| Default                                                                                  | `keepalive_requests 1000;`     |
| [Context](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)  | upstream                       |

Sets the maximum number of requests that can be served through one keepalive connection. After the maximum number of requests is made, the connection is closed.

Closing connections periodically is necessary to free per-connection memory allocations. Therefore, using too high maximum number of requests could result in excessive memory usage and not recommended.

<a id="index-7"></a>

<a id="u-keepalive-time"></a>

### keepalive_time

| [Syntax](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)   | `keepalive_time` time;   |
|------------------------------------------------------------------------------------------|--------------------------|
| Default                                                                                  | `keepalive_time 1h;`     |
| [Context](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)  | upstream                 |

Limits the maximum time during which requests can be processed through one keepalive connection. After this time is reached, the connection is closed following the subsequent request processing.

<a id="index-8"></a>

<a id="u-keepalive-timeout"></a>

### keepalive_timeout

| [Syntax](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)   | `keepalive_timeout` timeout;   |
|------------------------------------------------------------------------------------------|--------------------------------|
| Default                                                                                  | `keepalive_timeout 60s;`       |
| [Context](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)  | upstream                       |

Sets a timeout during which an idle keepalive connection to an upstream server will stay open.

<a id="index-9"></a>

<a id="u-least-conn"></a>

### least_conn

| [Syntax](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)   | `least_conn`;   |
|------------------------------------------------------------------------------------------|-----------------|
| Default                                                                                  | —               |
| [Context](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)  | upstream        |

Specifies that a group should use a load balancing method where a request is passed to the server with the least number of active connections, taking into account weights of servers. If there are several such servers, they are tried in turn using a weighted round-robin balancing method.

<a id="index-10"></a>

<a id="u-least-time"></a>

### least_time (PRO)

| [Syntax](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)   | `least_time` `header` | `last_byte` [`factor=`number] [`account=`condition_variable];   |
|------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------|
| Default                                                                                  | —                                                                                       |
| [Context](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)  | upstream                                                                                |

Specifies that the group should use a load balancing method where an active
server's chance of receiving the request is inversely proportional to its
average response time; the less it is, the more requests a server gets.

| `header`    | The directive accounts for the average time to receive response headers.   |
|-------------|----------------------------------------------------------------------------|
| `last_byte` | The directive uses the average time to receive the entire response.        |

| `factor`   | Serves the same purpose as [response_time_factor (PRO)](#u-response-time-factor)<br/>and overrides it if the parameter is set.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
|------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `account`  | Specifies a condition variable<br/>that controls which responses are included in the calculation.<br/>The average is updated<br/>only if the condition variable for the response<br/>is not `""` or `"0"`.<br/><br/>#### NOTE<br/>By default, responses during [active health probes](https://en.angie.software//angie/docs/configuration/modules/http/http_upstream_probe.md#u-upstream-probe)<br/>are not included in the calculation;<br/>combining the [$upstream_probe](https://en.angie.software//angie/docs/configuration/modules/http/http_upstream_probe.md#v-upstream-probe) variable<br/>with `account` allows including these responses<br/>or even excluding everything else. |

Current values are presented as `header_time` (headers only)
and `response_time` (entire responses) in the server's `health` object
among the [upstream metrics](https://en.angie.software//angie/docs/configuration/modules/http/http_api.md#api-status-http-upstreams) in the API.

<a id="index-11"></a>

<a id="u-queue"></a>

### queue (PRO)

| [Syntax](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)   | `queue` number [`timeout=`time];   |
|------------------------------------------------------------------------------------------|------------------------------------|
| Default                                                                                  | —                                  |
| [Context](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)  | upstream                           |

If it is not possible to assign a proxied server to a request on the first attempt
(for example, during a brief service interruption
or when there is a surge in load reaching the [max_conns](#u-server) limit),
the request is not rejected;
instead, Angie attempts to enqueue it for processing.

The number parameter of the directive sets the maximum number of requests
in the queue for a [worker process](https://en.angie.software//angie/docs/configuration/modules/core.md#worker-processes).
If the queue is full,
a `502 (Bad Gateway)` error is returned to the client.

#### NOTE
The logic of the [proxy_next_upstream](https://en.angie.software//angie/docs/configuration/modules/http/http_proxy.md#proxy-next-upstream) directive also applies to queued requests.
Specifically, if a server was selected for a request
but it cannot be handed over to it,
the request may be returned to the queue.

If a server is not selected to process a queued request
within the [time](https://en.angie.software//angie/docs/configuration/configfile.md#syntax) set by `timeout`
(default is 60 seconds),
a `502 (Bad Gateway)` error is returned to the client.
Requests from clients that prematurely close the connection are also removed from the queue;
there are counters for the states of requests passing through the queue in the [API](https://en.angie.software//angie/docs/configuration/modules/http/http_api.md#a-queue).

#### WARNING
The `queue` directive must be used after all directives that set the
load balancing method; otherwise, it won't work.

<a id="index-12"></a>

<a id="u-random"></a>

### random

| [Syntax](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)   | `random` [`two`];   |
|------------------------------------------------------------------------------------------|---------------------|
| Default                                                                                  | —                   |
| [Context](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)  | upstream            |

Specifies a load balancing method for the group where a request is passed to a randomly selected server, taking into account server weights.

If the optional `two` parameter is specified, Angie randomly selects two servers, then selects one of them using the [least_conn](#u-least-conn) method, where a request is passed to the server with the least number of active connections.

<a id="index-13"></a>

<a id="u-response-time-factor"></a>

### response_time_factor (PRO)

| [Syntax](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)   | `response_time_factor` number;   |
|------------------------------------------------------------------------------------------|----------------------------------|
| Default                                                                                  | `response_time_factor 90;`       |
| [Context](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)  | upstream                         |

Sets the smoothing factor for the **previous** value when calculating the average
response time for the [least_time (PRO)](#u-least-time) load balancing method using the
[exponentially weighted moving average](https://en.wikipedia.org/wiki/Moving_average#Exponential_moving_average) formula.

The higher the specified number, the less new values affect the average; if
`90` is specified, 90% of the previous value and only 10% of
the new value are taken. Valid values range from 0 to 99 inclusive.

Current calculation results are presented as `header_time`
(headers only) and `response_time` (entire responses) in the server's `health`
object among the [upstream metrics](https://en.angie.software//angie/docs/configuration/modules/http/http_api.md#api-status-http-upstreams) in the API.

#### NOTE
Only successful responses are included in the calculation; what is considered an unsuccessful
response is determined by the [proxy_next_upstream](https://en.angie.software//angie/docs/configuration/modules/http/http_proxy.md#proxy-next-upstream),
[fastcgi_next_upstream](https://en.angie.software//angie/docs/configuration/modules/http/http_fastcgi.md#fastcgi-next-upstream), [uwsgi_next_upstream](https://en.angie.software//angie/docs/configuration/modules/http/http_uwsgi.md#uwsgi-next-upstream),
[scgi_next_upstream](https://en.angie.software//angie/docs/configuration/modules/http/http_scgi.md#scgi-next-upstream), [memcached_next_upstream](https://en.angie.software//angie/docs/configuration/modules/http/http_memcached.md#memcached-next-upstream), and
[grpc_next_upstream](https://en.angie.software//angie/docs/configuration/modules/http/http_grpc.md#grpc-next-upstream) directives. Additionally, the `header_time`
value is recalculated only if all headers are received and processed, and
`response_time` — only if the entire response is received.

<a id="index-14"></a>

<a id="u-server"></a>

### server

| [Syntax](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)   | `server` address [parameters];   |
|------------------------------------------------------------------------------------------|----------------------------------|
| Default                                                                                  | —                                |
| [Context](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)  | upstream                         |

Defines the address and other parameters of a server. The address can be specified as a domain name or IP address, with an optional port, or as a UNIX-domain socket path specified after the `unix:` prefix. If a port is not specified, port 80 is used. A domain name that resolves to multiple IP addresses defines multiple servers at once.

The following parameters can be defined:

| `weight=`number    | Sets the weight of the server. Default is 1.                                                                                                                                                                                                         |
|--------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `max_conns=`number | Limits the maximum number of simultaneous active connections to the proxied server. The default value is `0`, meaning there is no limit. If the group does not reside in the [shared memory zone](#u-zone), the limitation works per worker process. |

#### NOTE
With [idle keepalive connections](#u-keepalive) enabled, multiple [worker processes](https://en.angie.software//angie/docs/configuration/modules/core.md#worker-processes), and a [shared memory zone](#u-zone), the total number of active and idle connections to the proxied server may exceed the `max_conns` value.

<a id="max-fails"></a>

`max_fails=`number — sets the number of unsuccessful attempts to communicate with the server
that should occur during the specified [fail_timeout](#fail-timeout)
period for the server to be considered unavailable;
after this, it will be checked again after the same period.

What is considered an unsuccessful attempt
is defined by the [proxy_next_upstream](https://en.angie.software//angie/docs/configuration/modules/http/http_proxy.md#proxy-next-upstream),
[fastcgi_next_upstream](https://en.angie.software//angie/docs/configuration/modules/http/http_fastcgi.md#fastcgi-next-upstream), [uwsgi_next_upstream](https://en.angie.software//angie/docs/configuration/modules/http/http_uwsgi.md#uwsgi-next-upstream),
[scgi_next_upstream](https://en.angie.software//angie/docs/configuration/modules/http/http_scgi.md#scgi-next-upstream), [memcached_next_upstream](https://en.angie.software//angie/docs/configuration/modules/http/http_memcached.md#memcached-next-upstream), and
[grpc_next_upstream](https://en.angie.software//angie/docs/configuration/modules/http/http_grpc.md#grpc-next-upstream) directives.

When `max_fails` is exceeded, the server is also considered unavailable from the perspective of
[upstream_probe (PRO)](https://en.angie.software//angie/docs/configuration/modules/http/http_upstream_probe.md#u-upstream-probe); client requests will not be directed to it
until checks determine it is available.

#### NOTE
If a `server` directive in a group resolves to multiple servers,
its `max_fails` setting applies to each server separately.

If after resolving all `server` directives
only one server remains in the upstream,
the `max_fails` setting has no effect and will be ignored.

| `max_fails=1`   | Default number of attempts;   |
|-----------------|-------------------------------|
| `max_fails=0`   | Disables attempt accounting.  |

<a id="fail-timeout"></a>

`fail_timeout=`time — sets the period of time during which
a specified number of unsuccessful attempts to communicate with the server
([max_fails](#max-fails)) must occur for the server to be considered unavailable.
The server then remains unavailable for the same period of time
before being checked again.

Default value is 10 seconds.

#### NOTE
If a `server` directive in a group resolves to multiple servers,
its `fail_timeout` setting applies to each server separately.

If after resolving all `server` directives
only one server remains in the upstream,
the `fail_timeout` setting has no effect and will be ignored.

| `backup`      | Marks the server as a backup server. It will be passed requests when the primary servers are unavailable.<br/><br/>If the [backup_switch (PRO)](#u-backup-switch) directive is specified,<br/>its active backup logic also applies.   |
|---------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `down`        | Marks the server as permanently unavailable.                                                                                                                                                                                          |
| `drain` (PRO) | Marks the server as draining; this means<br/>it only receives requests from sessions<br/>previously bound via [sticky](#u-sticky).<br/>Otherwise, the behavior is the same as in `down` mode.                                         |

#### WARNING
The `backup` parameter cannot be used together with the
[hash](#u-hash), [ip_hash](#u-ip-hash), and
[random](#u-random) load balancing methods.

The `down` and `drain` parameters are mutually exclusive.

<a id="reresolve"></a>

| `resolve`      | Allows monitoring changes to the list of IP addresses corresponding to<br/>a domain name and updating it without reloading the configuration.<br/>The group must reside in<br/>[shared memory](#u-zone);<br/>a [resolver](https://en.angie.software//angie/docs/configuration/modules/http/index.md#resolver) must also be defined.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
|----------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `service=`name | Enables resolution of DNS SRV records and sets the service name. For<br/>the parameter to work, the `resolve` parameter must be specified for the server,<br/>without specifying a port in the hostname.<br/><br/>If the service name does not contain a dot, a name is formed according to the RFC standard:<br/>the service name is prefixed with `_`,<br/>then `_tcp` is appended after a dot.<br/>Thus, the service name `http` results in `_http._tcp`.<br/><br/>Angie resolves SRV records<br/>by combining the normalized service name and hostname<br/>and obtaining a list of servers for the resulting combination via DNS,<br/>along with their priorities and weights.<br/><br/>- SRV records with the highest priority<br/>  (those with the lowest priority value)<br/>  are resolved as primary servers,<br/>  while other records become backup servers.<br/>  If `backup` is set with `server`,<br/>  SRV records with the highest priority are resolved as backup servers,<br/>  and other records are ignored.<br/>- Weight is analogous to the `weight` parameter of the `server` directive.<br/>  If weight is specified both in the directive itself and in the SRV record,<br/>  the weight set in the directive is used. |

In this example, a lookup is performed for the record `_http._tcp.backend.example.com`:

```nginx
server backend.example.com service=http resolve;
```

| `sid=`id   | Sets the server ID in the group. If the parameter is not specified,<br/>the ID is set as a hexadecimal MD5 hash<br/>of the IP address and port or UNIX socket path.   |
|------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------|

<a id="slow-start"></a>

| `slow_start=`time   | Sets the [time](https://en.angie.software//angie/docs/configuration/configfile.md#syntax) for gradual recovery of a returning server's weight<br/>when load balancing with the<br/>[round-robin](#round-robin) or [least_conn](#u-least-conn) method.<br/><br/>If the parameter is set<br/>and the server is considered operational again after a failure<br/>from the perspective of [max_fails](#max-fails) and [upstream_probe (PRO)](https://en.angie.software//angie/docs/configuration/modules/http/http_upstream_probe.md#u-upstream-probe),<br/>such a server gradually increases to its specified weight<br/>over the given time period.<br/><br/>If the parameter is not set,<br/>in a similar situation<br/>the server immediately starts operating with its specified weight.   |
|---------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|

#### NOTE
If only one `server` is specified in the upstream,
`slow_start` does not work and will be ignored.

<a id="index-15"></a>

<a id="u-state"></a>

### state (PRO)

| [Syntax](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)   | `state` file;   |
|------------------------------------------------------------------------------------------|-----------------|
| Default                                                                                  | —               |
| [Context](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)  | upstream        |

Specifies a file where the list of upstream servers is persistently stored.
When installing from
[our packages](https://en.angie.software//angie/docs/installation/index.md#install-packages),
the directory
`/var/lib/angie/state/` (`/var/db/angie/state/` on FreeBSD)
is created specifically for storing such files
with appropriate access permissions,
and in the configuration you only need to add the file name:

```nginx
upstream backend {

    zone backend 1m;
    state /var/lib/angie/state/<FILE NAME>;
}
```

The server list here has a format similar to `server`.
The file contents are modified whenever servers are changed in the
[/config/http/upstreams/](https://en.angie.software//angie/docs/configuration/modules/http/http_api.md#api-config-http-upstreams-servers) section
via the configuration API.
The file is read when Angie starts or when the configuration is reloaded.

#### WARNING
To use the `state` directive in an `upstream` block,
there must be no `server` directives in it,
but a shared memory zone ([zone](#u-zone)) is required.

<a id="index-16"></a>

<a id="u-sticky"></a>

### sticky

| [Syntax](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)   | `sticky` cookie name [attr=value]...;<br/><br/>`sticky route` value...;<br/><br/>`sticky learn` `zone=`zone `create=`$create_var1... `lookup=`$lookup_var1... [`header`] [`norefresh`] [`timeout=`time];<br/><br/>`sticky learn` [`zone=`zone] `lookup=`$lookup_var1... `remote_action=`uri `remote_result=`$remote_var [`norefresh`] [`timeout=`time];   |
|------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Default                                                                                  | —                                                                                                                                                                                                                                                                                                                                                         |
| [Context](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)  | upstream                                                                                                                                                                                                                                                                                                                                                  |

Configures session affinity to bind client sessions to proxied servers
in the mode specified by the first parameter;
to drain servers
that have the `sticky` directive configured,
you can use the `drain` (PRO) option
in the [server](#u-server) block.

#### WARNING
The `sticky` directive must be used after all directives
that specify a particular load balancing method,
otherwise it will not work.
If it is used alongside the [bind_conn (PRO)](#u-bind-conn) directive,
then `bind_conn` must come after `sticky`.

`cookie` mode

This mode uses cookies to manage sessions.
It's suitable when cookies are already used for session tracking.

The first client request, before any stickiness applies,
is sent to a backend server according to the configured balancing method.
Angie then sets a cookie identifying the chosen server.

The cookie name (`name`) is defined by the `sticky` directive,
and its value corresponds to the [sid](#reresolve)
from the [server](#u-server) directive.
This value is further hashed if [sticky_secret](#u-sticky-secret) is set.

Subsequent requests with this cookie are routed to the server
specified by its [sid](#reresolve).
If the server is unavailable or can't process the request,
another one is chosen via the configured balancing method.

You can assign cookie attributes in the directive;
by default, only `path=/` is set.
Attribute values can contain variables.
To remove an attribute, set it to an empty value: `attr=`.
For example, `sticky cookie path=` omits the `path` attribute.

This example sets a cookie `srv_id` for 1 hour,
using a domain from a variable:

```nginx
upstream backend {
    server backend1.example.com:8080;
    server backend2.example.com:8080;

    sticky cookie srv_id domain=$my_domain max-age=3600;
}
```

`route` mode

This mode uses predefined route identifiers,
which may come from URLs, cookies, or request arguments.
It's less flexible but good when such identifiers already exist.

The backend server may return a route ID known to both client and server.
This value must match the [sid](#reresolve).

Subsequent requests should carry the route ID,
e.g., via a [cookie](https://en.angie.software//angie/docs/configuration/modules/http/index.md#v-cookie) or [query argument](https://en.angie.software//angie/docs/configuration/modules/http/index.md#v-arg).

The directive takes a list of strings that may include variables
to extract the route ID. The first non-empty result is matched against
[sid](#reresolve).

In this example, Angie checks the `route` cookie first,
then the `route` query argument:

```nginx
upstream backend {
    server backend1.example.com:8080 "sid=server 1";
    server backend2.example.com:8080 "sid=server 2";

    sticky route $cookie_route $arg_route;
}
```

`learn` mode (PRO 1.4.0+)

This mode uses a dynamically generated key to assign a client to a backend.
It's flexible and supports session storage in shared memory
and various identifier sources.

A session is created from the backend server's response.
`create` and `lookup` define how to generate and locate sessions,
and both accept multiple variables.

The session ID is the first non-empty variable from `create`.
For example, it may come from a response cookie.

Sessions are stored in shared memory,
defined with `zone name:size`.
If unused for `timeout` duration (default: 1 hour), the session expires.

By default, Angie refreshes sessions on each use.
To disable this, use `norefresh`.

The session ID from a client request is extracted via `lookup`,
using the first non-empty variable listed.
If none is found, it's a new request.

Use `header` to create the session upon receiving response headers
rather than after full response processing.

Example: session created using `examplecookie`:

```nginx
upstream backend {
    server backend1.example.com:8080;
    server backend2.example.com:8080;

    sticky learn
        create=$upstream_cookie_examplecookie
        lookup=$cookie_examplecookie
        zone=client_sessions:1m;
}
```

`learn` with `remote_action` (PRO 1.8.0+)

Use `remote_action` and `remote_result`
to manage session IDs with an external session store.
The shared memory zone acts as a cache;
the external store is authoritative.
`create` is not compatible with `remote_action`.

Sessions expire after `timeout` (default: 1 hour),
regardless of `remote_action`.

By default, Angie refreshes session TTL on each use.
Use `norefresh` to disable that.

`zone` is optional with `remote_action`.
Without it, Angie always queries the external store.

Basic flow:

- Extract session ID from the first non-empty `lookup` variable.
  If none, fall back to standard load balancing.
- If `zone` is set and session exists, use it and stop.
- If no session or no zone, pick a server and make an HTTP subrequest
  to the `remote_action` endpoint with:
  - session ID ([$sticky_sessid](#v-sticky-sessid));
  - server ID from `sid=` or from [$sticky_sid](#v-sticky-sid).

  Send these as HTTP headers (via [proxy_set_header](https://en.angie.software//angie/docs/configuration/modules/http/http_proxy.md#proxy-set-header)).
- The remote store responds:
  - 200/201/204 confirms the session; cache it if `zone` is set.
  - 409 signals conflict (if `zone` is set) — session linked to another server.
    Use `remote_result` to extract the corrected server ID.
  - Other status codes or missing server ID — fallback to original server.

`remote_result` uses `upstream_http_*` variables
to read headers from the remote store's response.

Example: session ID comes from `$cookie_bar`,
confirmed via `$upstream_http_x_sticky_sid`:

```nginx
http {

    upstream u1 {
        server srv1;
        server srv2;

        sticky learn zone=sz:1m
            lookup=$cookie_bar
            remote_action=/remote_session
            remote_result=$upstream_http_x_sticky_sid;

        zone z 1m;
    }

    server {

        listen localhost;

        location / {
            proxy_pass http://u1/;
        }

        location /remote_session {
            internal;
            proxy_set_header X-Sticky-Sessid $sticky_sessid;
            proxy_set_header X-Sticky-Sid $sticky_sid;
            proxy_set_header X-Sticky-Last $msec;
            proxy_pass http://remote;
        }
    }
}
```

Example response:

```none
HTTP/1.1 200 OK
...
X-Sid: web-server-01
X-Session-Backend: backend-pool-1
```

Resulting Angie variables:

- `$upstream_http_x_sid` → `web-server-01`
- `$upstream_http_x_session_backend` → `backend-pool-1`

`remote_result` will use `web-server-01`
to select the matching `sid`.

The `sticky` directive respects upstream server states:

- Servers marked `down` or failing are excluded.
- Servers over `max_conns` limit are skipped.
- `drain` servers (PRO) may still be selected for new sessions in `sticky` mode when identifiers match.
- Recovered servers are reused automatically.

You can further adjust behavior using
[sticky_secret](#u-sticky-secret) and [sticky_strict](#u-sticky-strict).
If stickiness fails and `sticky_strict` is off,
fallback balancing is used;
if on, the request is rejected.

Each `zone` used in `sticky` must be exclusive to a single `upstream`.
Zones cannot be shared across multiple `upstream` blocks.

<a id="index-17"></a>

<a id="u-sticky-secret"></a>

### sticky_secret

| [Syntax](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)   | `sticky_secret` string;   |
|------------------------------------------------------------------------------------------|---------------------------|
| Default                                                                                  | —                         |
| [Context](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)  | upstream                  |

Adds the string as the salt value to the MD5 hashing function
for the [sticky](#u-sticky) directive in `cookie` and `route` modes.
The string may contain variables, for example, `$remote_addr`:

```nginx
upstream backend {
    server backend1.example.com:8080;
    server backend2.example.com:8080;

    sticky cookie cookie_name;
    sticky_secret my_secret.$remote_addr;
}
```

The salt is appended to the value being hashed;
to verify the hashing mechanism independently:

```console
$ echo -n "<VALUE><SALT>" | md5sum
```

<a id="index-18"></a>

<a id="u-sticky-strict"></a>

### sticky_strict

| [Syntax](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)   | `sticky_strict` `on` | `off`;   |
|------------------------------------------------------------------------------------------|---------------------------------|
| Default                                                                                  | `sticky_strict off;`            |
| [Context](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)  | upstream                        |

When enabled, causes Angie to return an HTTP 502 error to the client
if the desired server is unavailable,
rather than using any other available server
as it would when no servers in the upstream are available.

<a id="index-19"></a>

<a id="u-upstream"></a>

### upstream

| [Syntax](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)   | `upstream` name { ... }   |
|------------------------------------------------------------------------------------------|---------------------------|
| Default                                                                                  | —                         |
| [Context](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)  | http                      |

Defines a group of servers. Servers can listen on different ports. In addition, servers listening on TCP and UNIX domain sockets can be mixed.

Example:

```nginx
upstream backend {
    server backend1.example.com weight=5;
    server 127.0.0.1:8080       max_fails=3 fail_timeout=30s;
    server backup1.example.com  backup;
}
```

<a id="round-robin"></a>

By default, requests are distributed between the servers using a weighted round-robin balancing method. In the above example, each 7 requests will be distributed as follows: 5 requests go to backend1.example.com and one request to each of the second and third servers.

If an error occurs during communication with a server, the request will be passed to the next server, and so on until all of the functioning servers will be tried. If a successful response could not be obtained from any of the servers, the client will receive the result of the communication with the last server.

<a id="index-20"></a>

<a id="u-zone"></a>

### zone

| [Syntax](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)   | `zone` name [size];   |
|------------------------------------------------------------------------------------------|-----------------------|
| Default                                                                                  | —                     |
| [Context](https://en.angie.software//angie/docs/configuration/configfile.md#configfile)  | upstream              |

Defines the name and size of the shared memory zone that keeps the group's configuration and run-time state that are shared between worker processes. Several groups may share the same zone. In this case, it is enough to specify the size only once.

#### NOTE
The zone's content is only preserved on reload when the configured
`size` is unchanged. Any size change — increase or decrease —
causes the zone to be re-created empty.

<a id="built-in-variables-14"></a>

## Built-in Variables

The `http_upstream` module supports the following built-in variables:

<a id="v-sticky-sessid"></a>

### `$sticky_sessid`

Used with `remote_action` in [sticky](#u-sticky);
stores the initial session ID taken from `lookup`.

<a id="v-sticky-sid"></a>

### `$sticky_sid`

Used with `remote_action` in [sticky](#u-sticky);
stores the server ID previously associated with the session.

<a id="v-upstream-addr"></a>

### `$upstream_addr`

stores the IP address and port, or the path to the UNIX domain socket of the upstream server. If several servers were contacted during request processing, their addresses are separated by commas, e.g.:

> 192.168.1.1:80, 192.168.1.2:80, unix:/tmp/sock

If an internal redirect from one server group to another happens, initiated by `X-Accel-Redirect` or [error_page](https://en.angie.software//angie/docs/configuration/modules/http/index.md#error-page), then the server addresses from different groups are separated by colons, e.g.:

> 192.168.1.1:80, 192.168.1.2:80, unix:/tmp/sock : 192.168.10.1:80, 192.168.10.2:80

If a server cannot be selected, the variable keeps the name of the [server group](#u-upstream).

<a id="v-upstream-bytes-received"></a>

### `$upstream_bytes_received`

number of bytes received from an upstream server. Values from several connections are separated by commas and colons like addresses in the [$upstream_addr](#v-upstream-addr) variable.

<a id="v-upstream-bytes-sent"></a>

### `$upstream_bytes_sent`

number of bytes sent to an upstream server. Values from several connections are separated by commas and colons like addresses in the [$upstream_addr](#v-upstream-addr) variable.

<a id="v-upstream-cache-status"></a>

### `$upstream_cache_status`

keeps the status of accessing a response cache. The status can be either
`MISS`, `BYPASS`, `EXPIRED`, `STALE`, `UPDATING`,
`REVALIDATED`, or `HIT`:

- `MISS`: The response is not found in the cache,
  and the request is passed to the upstream server.
- `BYPASS`: The cache is bypassed,
  and the request is passed directly to the upstream server.
- `EXPIRED`: The cached response is stale,
  and a new request is passed to the upstream server to update the content.
- `STALE`: The cached response is stale,
  but is still served to clients
  until the content is eventually updated from the upstream server.
- `UPDATING`: The cached response is stale,
  but is still served to clients
  while the currently ongoing update from the upstream server is in progress.
- `REVALIDATED`: The cached response is stale,
  but was successfully revalidated
  and does not need to be updated from the upstream server.
- `HIT`: The response was taken from the cache.

If the request bypassed the cache without accessing it,
the variable is not set.

<a id="v-upstream-cache-key"></a>

### `$upstream_cache_key`

contains the cache key used for the request.

<a id="v-upstream-connect-time"></a>

### `$upstream_connect_time`

keeps time spent on establishing a connection with the upstream server; the time is kept in seconds with millisecond resolution. In case of SSL, includes time spent on handshake. Times of several connections are separated by commas and colons like addresses in the [$upstream_addr](#v-upstream-addr) variable.

<a id="v-upstream-cookie"></a>

### `$upstream_cookie_<name>`

cookie with the specified name sent by the upstream server in the `Set-Cookie` response header field. Only the cookies from the response of the last server are saved.

<a id="v-upstream-header-time"></a>

### `$upstream_header_time`

keeps time spent on receiving the response header from the upstream server; the time is kept in seconds with millisecond resolution. Times of several responses are separated by commas and colons like addresses in the [$upstream_addr](#v-upstream-addr) variable.

<a id="v-upstream-http"></a>

### `$upstream_http_<name>`

keep server response header fields. For example, the `Server` response header field is available through the `$upstream_http_server` variable. The rules of converting header field names to variable names are the same as for the variables that start with the `$http_` prefix. Only the header fields from the response of the last server are saved.

<a id="v-upstream-request-method"></a>

### `$upstream_request_method`

request method used for the upstream request. It can differ from the client
request method when caching converts `HEAD` to `GET` or when
[proxy_method](https://en.angie.software//angie/docs/configuration/modules/http/http_proxy.md#proxy-method) is set.

<a id="v-upstream-queue-time"></a>

### `$upstream_queue_time`

keeps time the request spent in the [queue](#u-queue)
before the next server selection;
the time is kept in seconds with millisecond resolution.
Times of several attempts are separated by commas and colons
like addresses in the [$upstream_addr](#v-upstream-addr) variable.

<a id="v-upstream-response-length"></a>

### `$upstream_response_length`

keeps the length of the response obtained from the upstream server; the length is kept in bytes. Lengths of several responses are separated by commas and colons like addresses in the [$upstream_addr](#v-upstream-addr) variable.

<a id="v-upstream-response-time"></a>

### `$upstream_response_time`

keeps time spent on receiving the response from the upstream server; the time is kept in seconds with millisecond resolution. Times of several responses are separated by commas and colons like addresses in the [$upstream_addr](#v-upstream-addr) variable.

<a id="v-upstream-status"></a>

### `$upstream_status`

keeps status code of the response obtained from the upstream server. Status codes of several responses are separated by commas and colons like addresses in the [$upstream_addr](#v-upstream-addr) variable. If a server cannot be selected, the variable keeps the 502 (Bad Gateway) status code.

<a id="v-upstream-sticky-status"></a>

### `$upstream_sticky_status`

Status of sticky requests.

| `""`   | Request sent to an upstream where sticky is not enabled.                                       |
|--------|------------------------------------------------------------------------------------------------|
| `NEW`  | Request does not contain sticky information.                                                   |
| `HIT`  | Request with sticky information sent to the desired server.                                    |
| `MISS` | Request with sticky information sent to a server<br/>selected by the load balancing algorithm. |

Statuses from several connections are separated by commas and colons
like addresses in the [$upstream_addr](#v-upstream-addr) variable.

<a id="v-upstream-trailer"></a>

### `$upstream_trailer_<name>`

keeps fields from the end of the response obtained from the upstream server.