Stream Module#
The core stream module implements basic functionality for handling TCP and UDP connections: this includes defining server blocks, traffic routing, configuring proxying, SSL/TLS support, and managing connections for streaming services, such as databases, DNS, and other protocols that operate over TCP and UDP.
The other modules in this section extend this functionality, allowing you to flexibly configure and optimize the stream server for various scenarios and requirements.
When building from the source code, the module isn't built
by default; it must be enabled with the build option
--with-stream.
In packages and images from our repositories, the module is included in the build.
Related guides
Connection Handling: session phases; server by address and SNI.
Config Files: the top-level stream context.
Console Light Web Panel: status_zone fills the TCP/UDP Zones tab.
Configuration Example#
worker_processes auto;
error_log /var/log/angie/error.log info;
events {
worker_connections 1024;
}
stream {
upstream backend {
hash $remote_addr consistent;
server backend1.example.com:12345 weight=5;
server 127.0.0.1:12345 max_fails=3 fail_timeout=30s;
server unix:/tmp/backend3;
}
upstream dns {
server 192.168.0.1:53535;
server dns.example.com:53;
}
server {
listen 12345;
proxy_connect_timeout 1s;
proxy_timeout 3s;
proxy_pass backend;
}
server {
listen 127.0.0.1:53 udp reuseport;
proxy_timeout 20s;
proxy_pass dns;
}
server {
listen [::1]:12345;
proxy_pass unix:/tmp/stream.socket;
}
}
Directives#
error_log_user_tag#
Added in version 1.12.0.
Attaches a tag to the error_log records of every session handled by the
server. The value can contain variables, so the tag
can differ from session to session; a tag whose value is empty is not attached.
The directive can be specified multiple times to add multiple tags. These
directives are inherited from the previous configuration level if and only if
there are no error_log_user_tag directives defined on the current level.
Tags appear, as the tags list, only in error_log destinations
that use format=json; filter=tag: matches them in any format.
listen#
| |
Default | — |
server |
Sets the address and port for the socket on which the server will accept connections. It is possible to specify just the port, so Angie listens on all available IPv4 (and IPv6, if enabled) interfaces. The address can also be a hostname, for example:
listen 127.0.0.1:12345;
listen *:12345;
listen 12345; # same as *:12345
listen localhost:12345;
IPv6 addresses are specified in square brackets:
listen [::1]:12345;
listen [::]:12345;
Port ranges are specified with the first and last port separated by a hyphen:
listen 127.0.0.1:12345-12399;
listen 12345-12399;
UNIX domain sockets are specified with the unix: prefix:
listen unix:/var/run/angie.sock;
Note
Several servers can listen on the same address:port pair; the one that handles a connection is then chosen by server_name, which Angie learns from SNI alone. See Selecting a server for a session for the sequence in full.
| makes this server the default one for the given address:port pair: it handles the connections whose server name matches no other server listening on that pair. If no |
| allows specifying that all connections accepted on this port should work in SSL mode. |
| configures a listening socket for working with datagrams. In order to handle packets from the same address and port in the same session, the reuseport parameter should also be specified. This parameter is incompatible with |
| allows specifying that all connections accepted on this port should use the PROXY protocol. The client must send the complete header within proxy_protocol_timeout, or Angie closes the connection. The values the header carries populate the $proxy_protocol_addr family of variables and, through the RealIP module, can replace the client address. |
The listen directive can have several additional parameters specific to socket-related system calls.
| sets the associated routing table, FIB (the |
| enables "TCP Fast Open" for the listening socket and limits the maximum length for the queue of connections that have not yet completed the three-way handshake. Warning Do not enable this feature unless the server can handle receiving the same SYN packet with data more than once. |
| sets the |
| sets the receive buffer size (the |
| sets the send buffer size (the |
| sets the name of accept filter (the |
| instructs to use a deferred |
| makes a separate |
| this parameter determines (via the |
| this parameter instructs to create an individual listening socket for
each worker process (using the Warning Inappropriate use of this option may have its security implications. |
| enables accepting connections via Multipath TCP (MPTCP) protocol, supported in Linux kernel starting from version 5.6. |
so_keepalive=on | off | [keepidle]:[keepintvl]:[keepcnt]
Configures the "TCP keepalive" behavior for the listening socket.
| if this parameter is omitted then the operating system's settings will be in effect for the socket |
| the SO_KEEPALIVE option is turned on for the socket |
| the SO_KEEPALIVE option is turned off for the socket |
Some operating systems support setting of TCP keepalive parameters on a
per-socket basis using the TCP_KEEPIDLE, TCP_KEEPINTVL, and
TCP_KEEPCNT socket options. On such systems (currently, Linux, NetBSD,
DragonFly, FreeBSD, and macOS), they can be configured using the keepidle,
keepintvl, and keepcnt parameters. One or two parameters may be omitted, in
which case the system default setting for the corresponding socket option will
be in effect. On macOS, the keepidle parameter sets the
TCP_KEEPALIVE socket option instead of TCP_KEEPIDLE.
For example,
so_keepalive=30m::10
will set the idle timeout (TCP_KEEPIDLE) to 30 minutes, leave the probe interval (TCP_KEEPINTVL) at its system default, and set the probes count (TCP_KEEPCNT) to 10 probes.
preread_buffer_size#
Specifies a size of the preread buffer.
preread_timeout#
Specifies a timeout of the preread phase.
proxy_protocol_timeout#
Specifies a timeout for reading the PROXY protocol header to complete. If the entire header is not transmitted within this time, the connection is closed.
resolver#
| |
Default |
|
stream, server, upstream |
Configures name servers used to resolve names of upstream servers into addresses, for example:
resolver 127.0.0.53 [::1]:5353;
The address can be specified as a domain name or IP address, with an optional port. If port is not specified, the port 53 is used. Name servers are queried in a round-robin fashion.
At least one DNS server address or the conf parameter must be specified,
except in the resolver off; form.
Note
Prefer a local trusted resolver such as 127.0.0.53 (systemd-resolved)
over a public one (e.g. 8.8.8.8). Public resolvers expose DNS queries
to third parties and increase susceptibility to cache-poisoning attacks.
Note
The directive value is inherited by nested blocks and can be overridden in them if necessary. Within a single block the directive may be specified only once; repeating it is a configuration error.
Enables DNS resolution for upstream servers referenced through variables or configured with the resolve parameter, and for OCSP stapling.
Besides explicit server addresses, the directive recognizes:
| explicitly disables DNS name resolution |
| reads DNS server addresses from the specified file, or from
|
The file may contain nameserver lines in the same format used by the
system resolver:
nameserver 127.0.1.4
nameserver 127.0.1.5
Addresses read from the file are added to any addresses specified explicitly
in the directive, and all of them are then queried together in a
round-robin fashion. If the file is absent, cannot be read, or lists no
addresses, and none are configured explicitly, Angie falls back to the
default server 127.0.0.1:53.
At runtime, Angie re-reads the file when it changes, updating the set of DNS servers.
By default, Angie caches each answer for the duration of the TTL value in the
DNS response. The optional valid parameter overrides this:
| optional parameter allows overriding the response cache validity |
resolver 127.0.0.53 [::1]:5353 valid=30s;
By default, Angie will look up both IPv4 and IPv6 addresses while resolving.
| disables looking up of IPv4 addresses |
| disables looking up of IPv6 addresses |
| optional parameter; enables the collection of DNS server request and response metrics (/status/resolvers/<zone>) in the specified zone |
Tip
To prevent DNS spoofing, it is recommended to use DNS servers in a properly secured trusted local network.
Tip
When running in Docker, use the corresponding internal DNS server address such as 127.0.0.11.
resolver_timeout#
Sets a timeout for name resolution, for example:
resolver_timeout 5s;
server#
Sets the configuration for a server.
server_name#
Sets names of a virtual server.
Warning
In the stream module, the server_name directive is based on Server Name
Indication (SNI) and only works with TLS connections. To use it,
you must configure the Stream SSL module for TLS termination
or enable TLS preread in the corresponding server block.
Example configuration:
server {
listen 443 ssl;
server_name example.com www.example.com;
ssl_certificate /etc/angie/cert.pem;
ssl_certificate_key /etc/angie/key.pem;
}
The first name becomes the primary server name.
Server names can include an asterisk (*)
to replace the first or last part of a name:
server {
server_name example.com *.example.com www.example.*;
}
These names are called wildcard names.
You can also use regular expressions in server names by preceding the name with
a tilde (~):
server {
server_name www.example.com ~^www\d+\.example\.com$;
}
Regular expressions may include captures that can be used in other directives:
server {
server_name ~^(www\.)?(.+)$;
proxy_pass www.$2:12345;
}
Named captures in regular expressions create variables that can be used in other directives:
server {
server_name ~^(www\.)?(?<domain>.+)$;
proxy_pass www.$domain:12345;
}
If the directive's parameter is set to $hostname, the machine's hostname
is inserted.
When searching for a virtual server by name, if the name matches more than one of the specified variants (e.g., both a wildcard name and a regular expression match), the first matching variant will be chosen in the following order of priority:
The exact name
The longest wildcard name starting with an asterisk, e.g.,
*.example.comThe longest wildcard name ending with an asterisk, e.g.,
mail.*The first matching regular expression (in order of appearance in the configuration file)
server_names_hash_bucket_size#
| |
Default |
|
stream |
Sets the bucket size for the server names hash tables. The default value depends on the size of the processor's cache line.
server_names_hash_max_size#
Sets the maximum size of the server names hash tables.
status_zone#
Allocates a shared memory zone to collect metrics for /status/stream/server_zones/<zone>.
Multiple server contexts can share the same zone for data collection.
All status_zone directives that share a zone
must use the same syntax and, with the alternative syntax,
the same count wherever it is given;
otherwise, the configuration fails to load
because the sizes of the shared memory zone conflict.
The single-value zone syntax aggregates all metrics for the current context in one shared memory zone:
server {
listen 80;
server_name *.example.com;
status_zone single;
# ...
}
The alternative syntax allows specifying the following parameters:
key | A string with variables, whose value determines the grouping of connections in the zone. All connections producing identical values after substitution are grouped together. If substitution yields an empty value, metrics aren't updated. |
zone | The name of the shared memory zone. |
count | The maximum number of separate groups for collecting metrics. If new key values would exceed this limit, they are grouped under zone instead. Required unless another |
In the following example,
the server_zone zone collects metrics
separately for each unique $server_addr value
until there are 10 such groups;
after that, connections with any other $server_addr value
are counted in the group named server_zone:
stream {
upstream backend {
server 192.168.0.1:3306;
server 192.168.0.2:3306;
# ...
}
server {
listen 3306;
proxy_pass backend;
status_zone $server_addr zone=server_zone:10;
}
}
The resulting metrics are split between individual servers in the API output.
Note
These metrics are collected only when status_zone is set. Without it,
the server does not appear in /status/stream/server_zones/<zone>, the
TCP/UDP Zones Widget, or Prometheus output, and no warning
is logged.
stream#
Provides the configuration file context in which the stream server directives are specified.
tcp_nodelay#
Enables or disables the use of the TCP_NODELAY option. The option is enabled for both client connections and connections to proxied servers.
time_format#
Added in version 1.12.0.
Defines a $variable to hold the current local time, rendered with a custom
format string of strftime-like specifiers plus %L for
milliseconds. The directive is useful where $time_local
and $time_iso8601 fall short, for example because they
provide no sub-second precision:
time_format $time_ms "%Y-%m-%dT%H:%M:%S.%L%Z";
log_format tf '$time_ms $remote_addr $status';
With the format above, $time_ms expands to a value such as 2024-01-15T14:34:56.789+03:00.
Redefining a built-in variable, such as $time_local, or
repeating time_format for the same name fails the configuration load
with an error like: "the duplicate "..." variable".
The format string accepts the following specifiers, evaluated in the same local time zone as $time_local:
| 4-digit year |
| 2-digit year |
| month, |
| day of month, |
| day of month, space-padded, |
| hour, |
| hour, |
| minute, |
| second, |
| milliseconds, |
| uppercase |
| lowercase |
| abbreviated weekday name, |
| full weekday name, |
| abbreviated month name, |
| full month name, |
| time zone offset, |
| time zone offset, ISO 8601 style, |
| newline |
| tab |
| literal |
Note
An unrecognized specifier is copied to the output verbatim, including the
leading %; a standalone % at the very end of the format
string is dropped.
variables_hash_bucket_size#
Sets the bucket size for the variables hash table. The details of setting up hash tables are provided in a separate document.
variables_hash_max_size#
Sets the maximum size of the variables hash table. The details of setting up hash tables are provided in a separate document.
Built-in Variables#
The core stream module supports the following built-in variables: Angie version client address in a binary form, value's length is always 4 bytes for IPv4 addresses or 16 bytes for IPv6 addresses number of bytes received from the client number of bytes sent to the client connection serial number host name current time in seconds with milliseconds resolution nginx version PID of the worker process protocol used to communicate with the client: client address from the PROXY protocol header.
The PROXY protocol must be previously enabled by setting the proxy_protocol parameter in the listen directive. client port from the PROXY protocol header.
The PROXY protocol must be previously enabled by setting the proxy_protocol parameter in the listen directive. server address from the PROXY protocol header.
The PROXY protocol must be previously enabled by setting the proxy_protocol parameter in the listen directive. server port from the PROXY protocol header.
The PROXY protocol must be previously enabled by setting the proxy_protocol parameter in the listen directive. TLV obtained from the PROXY protocol header. The name can be a TLV type name or its numeric value. In the latter case, the value is specified in hexadecimal and must start with 0x: SSL TLVs can also be accessed by both TLV type name and its numeric value, both must start with The following TLV type names are supported: The following SSL TLV type names are supported: Also supported is the following special SSL TLV type name: The PROXY protocol must be previously enabled by setting the proxy_protocol parameter in the listen directive. client address client port address of the server which accepted a connection.
Computing a value of this variable usually requires one system call. To avoid a system call, the listen directives must specify addresses and use the port of the server which accepted a connection session duration in seconds with milliseconds resolution session status, can be one of the following: session completed successfully client data could not be parsed, for example, the PROXY protocol header access forbidden, for example, when access is limited for certain client addresses internal server error bad gateway, for example, if an upstream server could not be selected or reached service unavailable, for example, when access is limited by the number of connections local time in the ISO 8601 standard format local time in the Common Log Format$angie_version#$binary_remote_addr#$bytes_received#$bytes_sent#$connection#$hostname#$msec#$nginx_version#$pid#$protocol#TCP or UDP$proxy_protocol_addr#$proxy_protocol_port#$proxy_protocol_server_addr#$proxy_protocol_server_port#$proxy_protocol_tlv_<name>#$proxy_protocol_tlv_alpn
$proxy_protocol_tlv_0x01
ssl_:$proxy_protocol_tlv_ssl_version
$proxy_protocol_tlv_ssl_0x21
alpn (0x01) - upper layer protocol used over the connectionauthority (0x02) - host name value passed by the clientunique_id (0x05) - unique connection identifiernetns (0x30) - namespace namessl (0x20) - SSL TLV structure in binary formatssl_version (0x21) - SSL version used in client connectionssl_cn (0x22) - certificate Common Namessl_cipher (0x23) - name of the used cipherssl_sig_alg (0x24) - algorithm used to sign the certificatessl_key_alg (0x25) - public key algorithmssl_verify - client certificate verification result: 0 if the client presented a certificate and it was successfully verified, or non-zero otherwise$remote_addr#$remote_port#$server_addr#bind parameter.$server_port#$session_time#$status#200400403500502503$time_iso8601#$time_local#
Content