Stream Module#
The core stream module implements basic functionality for handling TCP and UDP connections: this includes defining server blocks, traffic routing, configuring proxying, SSL/TLS support, and managing connections for streaming services, such as databases, DNS, and other protocols that operate over TCP and UDP.
The other modules in this section extend this functionality, allowing you to flexibly configure and optimize the stream server for various scenarios and requirements.
When building from the source code,
this module isn't built by default;
it should be enabled with the
Default — server Sets the address and port for the socket on which the server will accept connections. It is possible to specify just the port, so Angie listens on all available IPv4 (and IPv6, if enabled) interfaces. The address can also be a hostname, for example: IPv6 addresses are specified in square brackets: UNIX domain sockets are specified with the Port ranges are specified with the first and last port separated by a hyphen: Important Different servers must listen on different address:port pairs. allows specifying that all connections accepted on this port should work in SSL mode. configures a listening socket for working with datagrams. In order to handle packets from the same address and port in the same session, the reuseport parameter should also be specified. allows specifying that all connections accepted on this port should use the PROXY protocol. The sets the associated routing table, FIB (the enables "TCP Fast Open" for the listening socket and limits the maximum length for the queue of connections that have not yet completed the three-way handshake. Caution Do not enable this feature unless the server can handle receiving the same SYN packet with data more than once. sets the sets the receive buffer size (the sets the send buffer size (the Sets the name of accept filter (the instructs to use a deferred this parameter instructs to make a separate this parameter determines (via the this parameter instructs to create an individual listening socket for
each worker process (using the Caution Inappropriate use of this option may have its security implications. enables accepting connections via Multipath TCP (MPTCP) protocol,
supported in Linux kernel starting from version 5.6.
This parameter is incompatible with Configures the "TCP keepalive" behavior for the listening socket. if this parameter is omitted then the operating system's settings will be in effect for the socket the SO_KEEPALIVE option is turned on for the socket the SO_KEEPALIVE option is turned off for the socket Some operating systems support setting of TCP keepalive parameters on a
per-socket basis using the For example, will set the idle timeout (TCP_KEEPIDLE) to 30 minutes, leave the probe interval (TCP_KEEPINTVL) at its system default, and set the probes count (TCP_KEEPCNT) to 10 probes. Specifies a size of the preread buffer. Specifies a timeout of the preread phase. Specifies a timeout for reading the PROXY protocol header to complete. If no entire header is transmitted within this time, the connection is closed. Default — stream, server, upstream Configures name servers used to resolve names of upstream servers into addresses, for example: The address can be specified as a domain name or IP address, with an optional port. If port is not specified, the port 53 is used. Name servers are queried in a round-robin fashion. By default, Angie caches answers using the TTL value of a response. The optional valid parameter allows overriding it: optional valid parameter allows overriding cached entry validity By default, Angie will look up both IPv4 and IPv6 addresses while resolving. disables looking up of IPv4 addresses disables looking up of IPv6 addresses optional parameter;
enables the collection of DNS server request and response metrics
(/status/resolvers/<zone>)
in the specified zone. Tip To prevent DNS spoofing, it is recommended to use DNS servers in a properly secured trusted local network. Tip When running in Docker, use the corresponding internal DNS server address such as Sets a timeout for name resolution, for example: Sets the configuration for a server. Sets names of a virtual server. Attention In the Example configuration: The first name becomes the primary server name. Server names can include an asterisk ( These names are called wildcard names. You can also use regular expressions in server names by preceding the name with
a tilde ( Regular expressions may include captures that can be used in other directives: Named captures in regular expressions create variables
that can be used in other directives: If the directive's parameter is set to When searching for a virtual server by name, if the name matches more than one
of the specified variants (e.g., both a wildcard name and a regular expression
match), the first matching variant will be chosen in the following order of
priority: The exact name The longest wildcard name starting with an asterisk, e.g.,
The longest wildcard name ending with an asterisk, e.g., The first matching regular expression (in order of appearance in the
configuration file) Default stream Sets the bucket size for the server names hash tables. The default value depends
on the size of the processor's cache line. Sets the maximum size of the server names hash tables. Allocates a shared memory zone to collect metrics for
/status/stream/server_zones/<zone>. Multiple The single-value zone syntax aggregates all metrics for the current context
in one shared memory zone: The alternative syntax allows specifying the following parameters: key A string with variables,
whose value determines the grouping of connections in the zone.
All connections producing identical values after substitution
are grouped together.
If substitution yields an empty value, metrics aren't updated. zone The name of the shared memory zone. count (optional) The maximum number of separate groups for collecting metrics.
If new key values would exceed this limit,
they are grouped under zone instead. The default value is 1. In the following example,
all connections with the same The resulting metrics are split between individual servers in the API output. Provides the configuration file context in which the stream server directives are specified. Enables or disables the use of the TCP_NODELAY option. The option is enabled for both client connections and connections to proxied servers. Sets the bucket size for the variables hash table. The details of setting up hash tables are provided in a separate document. Sets the maximum size of the variables hash table. The details of setting up hash tables are provided in a separate document. The stream core module supports the following variables: Angie version client address in a binary form, value's length is always 4 bytes for IPv4 addresses or 16 bytes for IPv6 addresses number of bytes received from a client number of bytes sent to a client connection serial number host name current time in seconds with the milliseconds resolution PID of the worker process protocol used to communicate with the client: client address from the PROXY protocol header.
The PROXY protocol must be previously enabled by setting the proxy_protocol parameter in the listen directive. client port from the PROXY protocol header.
The PROXY protocol must be previously enabled by setting the proxy_protocol parameter in the listen directive. server address from the PROXY protocol header.
The PROXY protocol must be previously enabled by setting the proxy_protocol parameter in the listen directive. server port from the PROXY protocol header.
The PROXY protocol must be previously enabled by setting the proxy_protocol parameter in the listen directive. TLV obtained from the PROXY protocol header. The name can be a TLV type name or its numeric value. In the latter case, the value is specified in hexadecimal and must start with 0x: SSL TLVs can also be accessed by both TLV type name and its numeric value, both must start with The following TLV type names are supported: The following SSL TLV type names are supported: Also supported is the following special SSL TLV type name: The PROXY protocol must be previously enabled by setting the proxy_protocol parameter in the listen directive. client address client port address of the server which accepted a connection.
Computing a value of this variable usually requires one system call. To avoid a system call, the listen directives must specify addresses and use the port of the server which accepted a connection session duration in seconds with a milliseconds resolution session status, can be one of the following: session completed successfully client data could not be parsed, for example, the PROXY protocol header access forbidden, for example, when access is limited for certain client addresses internal server error bad gateway, for example, if an upstream server could not be selected or reached service unavailable, for example, when access is limited by the number of connections local time in the ISO 8601 standard format local time in the Common Log Format‑‑with‑stream
build option.
In packages and images from our repos,
the module is included in the build.Configuration Example#
worker_processes auto;
error_log /var/log/angie/error.log info;
events {
worker_connections 1024;
}
stream {
upstream backend {
hash $remote_addr consistent;
server backend1.example.com:12345 weight=5;
server 127.0.0.1:12345 max_fails=3 fail_timeout=30s;
server unix:/tmp/backend3;
}
upstream dns {
server 192.168.0.1:53535;
server dns.example.com:53;
}
server {
listen 12345;
proxy_connect_timeout 1s;
proxy_timeout 3s;
proxy_pass backend;
}
server {
listen 127.0.0.1:53 udp reuseport;
proxy_timeout 20s;
proxy_pass dns;
}
server {
listen [::1]:12345;
proxy_pass unix:/tmp/stream.socket;
}
}
Directives#
listen#
listen
address[:port] [ssl
] [udp
] [proxy_protocol
] [setfib=
number] [fastopen=
number] [backlog=
number] [rcvbuf=
size] [sndbuf=
size] [accept_filter=
filter] [deferred
] [bind
] [ipv6only=
on
| off
] [reuseport
] [so_keepalive=
on|off|[keepidle]:[keepintvl]:[keepcnt]] [multipath
];listen 127.0.0.1:12345;
listen *:12345;
listen 12345; # same as *:12345
listen localhost:12345;
listen [::1]:12345;
listen [::]:12345;
unix:
prefix:listen unix:/var/run/angie.sock;
listen 127.0.0.1:12345-12399;
listen 12345-12399;
ssl
udp
proxy_protocol
listen
directive can have several additional parameters specific to socket-related system calls.setfib=
numberSO_SETFIB
option) for
the listening socket. This currently works only on FreeBSD.fastopen=
numberbacklog=
numberbacklog
parameter in the listen()
call that
limits the maximum length for the queue of pending connections. By
default, backlog
is set to -1
on FreeBSD, DragonFly BSD,
and macOS, and to 511 on other platforms.rcvbuf=
sizeSO_RCVBUF
option) for the listening socket.sndbuf=
sizeSO_SNDBUF
option) for the listening socket.accept_filter=
filterSO_ACCEPTFILTER
option) for
the listening socket that filters incoming connections before passing
them to accept()
. This works only on FreeBSD and NetBSD 5.0+.
Acceptable values are dataready
and httpready
.deferred
accept()
(the
TCP_DEFER_ACCEPT
socket option) on Linux.bind
bind()
call for a
given address:port pair. The fact is that if there are several
listen directives with the same port but different addresses,
and one of the listen
directives listens on all addresses for the
given port (*:port), Angie will bind()
only to *:port. It should be
noted that the getsockname()
system call will be made in this case to
determine the address that accepted the connection. If the setfib
,
fastopen
, backlog
, rcvbuf
, sndbuf
,
accept_filter
, deferred
, ipv6only
,
reuseport
, or so_keepalive
parameters are used then for a
given address:port pair a separate bind()
call will always be
made.ipv6only=on
| off
IPV6_V6ONLY
socket option)
whether an IPv6 socket listening on a wildcard address [::] will accept
only IPv6 connections or both IPv6 and IPv4 connections. This parameter
is turned on by default. It can only be set once on start.reuseport
SO_REUSEPORT
socket option on
Linux 3.9+ and DragonFly BSD, or SO_REUSEPORT_LB
on FreeBSD 12+),
allowing a kernel to distribute incoming connections between worker
processes. This currently works only on Linux 3.9+, DragonFly BSD, and
FreeBSD 12+.multipath
udp
.so_keepalive=on
| off
| [keepidle]:[keepintvl]:[keepcnt]''
on
off
TCP_KEEPIDLE
, TCP_KEEPINTVL
, and
TCP_KEEPCNT
socket options. On such systems (currently, Linux 2.4+,
NetBSD 5+, and FreeBSD 9.0-STABLE), they can be configured using the keepidle,
keepintvl, and keepcnt parameters. One or two parameters may be omitted, in
which case the system default setting for the corresponding socket option will
be in effect.so_keepalive=30m::10
preread_buffer_size#
preread_timeout#
proxy_protocol_timeout#
resolver#
resolver
address ... [valid=
time] [ipv4=
on
| off
] [ipv6=
on
| off
] [status_zone=
zone];resolver 127.0.0.53 [::1]:5353;
valid
resolver 127.0.0.53 [::1]:5353 valid=30s;
ipv4=off
ipv6=off
status_zone
127.0.0.11
.resolver_timeout#
resolver_timeout 5s;
server#
server_name#
stream
module, the server_name
directive is based on Server Name
Indication (SNI) and only works with TLS connections. To use it,
you must configure TLS termination or enable TLS
preread in the corresponding server
block.server {
listen 443 ssl;
server_name example.com www.example.com;
ssl_certificate /etc/angie/cert.pem;
ssl_certificate_key /etc/angie/key.pem;
}
*
)
to replace the first or last part of a name:server {
server_name example.com *.example.com www.example.*;
}
~
):server {
server_name www.example.com ~^www\d+\.example\.com$;
}
server {
server_name ~^(www\.)?(.+)$;
proxy_pass www.$2:12345;
}
server {
server_name ~^(www\.)?(?<domain>.+)$;
proxy_pass www.$domain:12345;
}
$hostname
, the machine's hostname
is inserted.*.example.com
mail.*
server_names_hash_bucket_size#
server_names_hash_bucket_size
size;server_names_hash_bucket_size 32|64|128;
server_names_hash_max_size#
status_zone#
server
contexts can share the same zone for data collection.server {
listen 80;
server_name *.example.com;
status_zone single;
# ...
}
$server_addr
value
are grouped into host_zone
.
Metrics are collected separately for each unique $server_addr
until the number of metric groups reaches 10.
After that, any new $server_addr
values
will be added to the server_zone
group:stream {
upstream backend {
server 192.168.0.1:3306;
server 192.168.0.2:3306;
# ...
}
server {
listen 3306;
proxy_pass backend;
status_zone $server_addr zone=server_zone:10;
}
}
stream#
tcp_nodelay#
variables_hash_bucket_size#
variables_hash_max_size#
Built-in Variables#
$angie_version
#$binary_remote_addr
#$bytes_received
#$bytes_sent
#$connection
#$hostname
#$msec
#$pid
#$protocol
#TCP
or UDP
$proxy_protocol_addr
#$proxy_protocol_port
#$proxy_protocol_server_addr
#$proxy_protocol_server_port
#$proxy_protocol_tlv_<name>
#$proxy_protocol_tlv_alpn
$proxy_protocol_tlv_0x01
ssl_
:$proxy_protocol_tlv_ssl_version
$proxy_protocol_tlv_ssl_0x21
alpn (0x01)
- upper layer protocol used over the connectionauthority (0x02)
- host name value passed by the clientunique_id (0x05)
- unique connection identifiernetns (0x30)
- namespace namessl (0x20)
- SSL TLV structure in binary formatssl_version (0x21)
- SSL version used in client connectionssl_cn (0x22)
- certificate Common Namessl_cipher (0x23)
- name of the used cipherssl_sig_alg (0x24)
- algorithm used to sign the certificatessl_key_alg (0x25)
- public key algorithmssl_verify
- client certificate verification result: 0 if the client presented a certificate and it was successfully verified, or non-zero otherwise$remote_addr
#$remote_port
#$server_addr
#bind
parameter.$server_port
#$session_time
#$status
#200
400
403
500
502
503
$time_iso8601
#$time_local
#