Extended Configuration with Annotations#

This section explains how to enable advanced functionality in ANIC using annotations.

The Ingress resource can utilize basic features of Angie, such as host-based or path-based routing and TLS termination. Advanced features, such as request URI rewriting or inserting additional response headers, can be enabled using annotations.

In addition to advanced features, annotations are necessary for configuring Angie's behavior, such as setting connection timeout values.

Configuration is also available through ConfigMap resources: annotations take precedence.

Using Annotations#

This example uses annotations to configure the Ingress resource:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: cafe-ingress-with-annotations
  annotations:
    ``angie.software/proxy-connect-timeout: "30s"
    ``angie.software/proxy-read-timeout: "20s"
    ``angie.software/client-max-body-size: "4m"
    ``angie.software/server-snippets: |
      location / {
        return 302 /coffee;
      }
spec:
  rules:
  - host: cafe.example.com
    http:
      paths:
      - path: /tea
        pathType: Prefix
        backend:
          service:
            name: tea-svc
            port:
              number: 80
      - path: /coffee
        pathType: Prefix
        backend:
          service:
            name: coffee-svc
            port:
              number: 80

Validation#

ANIC checks the annotations of Ingress resources. If the Ingress is invalid, ANIC will reject it: the Ingress will continue to exist in the cluster, but ANIC will ignore it.

You can check whether ANIC successfully applied the configuration for the Ingress resource. For the Ingress example cafe-ingress-with-annotations, you can run the following command:

$ kubectl describe ing cafe-ingress-with-annotations
...
Events:
  Type     Reason          Age   From                      Message
  ----     ------          ----  ----                      -------
  Normal   AddedOrUpdated  3s    angie-ingress-controller  Configuration for default/cafe-ingress-with-annotations was added or updated

The events section includes a Normal event with the reason AddedOrUpdated, which informs us that the configuration was successfully applied.

If you create an invalid Ingress, ANIC will reject it and generate a Rejected event. For example, if you create the Ingress cafe-ingress-with-annotations with the annotation angie.software/redirect-to-https set to yes please instead of true, you will get:

$ kubectl describe ing cafe-ingress-with-annotations
Events:
  Type     Reason    Age   From                      Message
  ----     ------    ----  ----                      -------
  Warning  Rejected  13s   angie-ingress-controller  annotations.``angie.software/redirect-to-https: Invalid value: "yes please": must be a boolean

Note that the events section includes a Warning event with the reason Rejected.

Note

If you make an existing Ingress invalid, ANIC will reject it and remove the corresponding configuration from ANIC.

Summary of Annotations#

The table below lists the available annotations.

General Configuration#

Annotation

ConfigMap Key

Description

Default Value

Example

angie.software/proxy-connect-timeout

proxy-connect-timeout

Sets the value for the proxy_connect_timeout and grpc_connect_timeout directives.

60s

angie.software/proxy-read-timeout

proxy-read-timeout

Sets the value for the proxy_read_timeout and grpc_read_timeout directives.

60s

angie.software/proxy-send-timeout

proxy-send-timeout

Sets the value for the proxy_send_timeout and grpc_send_timeout directives.

60s

angie.software/client-max-body-size

client-max-body-size

Sets the value for the client_max_body_size directive.

1m

angie.software/proxy-buffering

proxy-buffering

Enables or disables response buffering from the proxied server.

True

angie.software/proxy-buffers

proxy-buffers

Sets the value for the proxy_buffers directive.

Depends on platform.

angie.software/proxy-buffer-size

proxy-buffer-size

Sets the value for the proxy_buffer_size and grpc_buffer_size directives.

Depends on platform.

angie.software/proxy-max-temp-file-size

proxy-max-temp-file-size

Sets the value for the proxy_max_temp_file_size directive.

1024m

angie.software/server-tokens

server-tokens

Enables or disables the server_tokens directive. Additionally, with Angie, a string value can be specified, including an empty string, which disables the "Server" field output.

True

angie.software/path-regex

None

Enables regular expression modifiers for the Ingress path parameter. This corresponds to the Angie location directive. One of the following values can be specified: "case_sensitive", "case_insensitive", or "exact". The annotation applies to the Ingress resource and its paths. When using Master and Minion Ingresses (i.e., Mergeable Ingresses), this annotation can be specified for Minion types. The path-regex annotation specified for Master is ignored and does not affect the paths defined in Minions.

None

angie.software/configmap

None

Allows specifying a specific ConfigMap for configuring the Ingress resource. The specified ConfigMap will take precedence over the global one. If the global and specified ConfigMaps match, the specified one will be applied. Example: angie.software/configmap: "namespace/configmap"

False

See overriding ConfigMap for Ingress resource.

Manipulating URI and Request Headers#

Annotation

ConfigMap Key

Description

Default Value

Example

angie.software/proxy-hide-headers

proxy-hide-headers

Sets the value for one or more proxy_hide_header directives. Example: "``angie.software/proxy-hide-headers": "header-a,header-b"

None

angie.software/proxy-pass-headers

proxy-pass-headers

Sets the value for one or more proxy_pass_header directives. Example: "``angie.software/proxy-pass-headers": "header-a,header-b"

None

angie.software/rewrites

None

Configures URI rewriting using the proxy_pass directive.

None

Authentication and SSL/TLS#

Annotation

ConfigMap Key

Description

Default Value

Example

angie.software/redirect-to-https

redirect-to-https

Sets a 301 redirect rule based on the value of the http_x_forwarded_proto header in the server block, to force incoming traffic to go through HTTPS. Useful for SSL termination at the load balancer before ANIC.

False

ingress.kubernetes.io/ssl-redirect

ssl-redirect

Sets a non-conditional 301 redirect rule for all incoming HTTP traffic, to force incoming traffic to go through HTTPS.

True

angie.software/hsts

hsts

Enables HTTP Strict Transport Security (HSTS) <https://www.nginx.com/blog/http-strict-transport-security-hsts-and-nginx/>: the HSTS header is added to responses from proxied servers. The directive preload is included in the header.

False

angie.software/hsts-max-age

hsts-max-age

Sets the value for the max-age directive of the HSTS header.

2592000 (1 month)

angie.software/hsts-include-subdomains

hsts-include-subdomains

Adds the includeSubDomains directive to the HSTS header.

False

angie.software/hsts-behind-proxy

hsts-behind-proxy

Enables HSTS based on the value of the http_x_forwarded_proto request header. Should be used only when TLS termination is configured at the load balancer (proxy) before ANIC.

Note

To manage the redirect from HTTP to HTTPS, configure the annotation angie.software/redirect-to-https.

False

angie.software/basic-auth-secret

None

Specifies a Secret resource with a list of users for HTTP Basic authentication.

None

angie.software/basic-auth-realm

None

Specifies the realm.

None

Listeners#

Annotation

ConfigMap Key

Description

Default Value

Example

angie.software/listen-ports

None

Configures the HTTP ports on which Angie will listen.

[80]

angie.software/listen-ports-ssl

None

Configures the HTTPS ports on which Angie will listen.

[443]

Backend Services (Upstreams)#

Annotation

ConfigMap Key

Description

Default Value

Example

angie.software/lb-method

lb-method

Sets the load balancing method. To use the round-robin method, specify "round_robin".

"random two least_conn"

angie.software/ssl-services

None

Enables HTTPS or gRPC over SSL when connecting to service endpoints.

None

angie.software/grpc-services

None

Enables gRPC for services.

Note

Requires HTTP/2 (see the http2 key in ConfigMap); works only for Ingress with TLS termination enabled.

None

angie.software/websocket-services

None

Enables WebSocket for services.

None

angie.software/max-fails

max-fails

Sets the value for the max_fails parameter of the server directive.

1

angie.software/max-conns

None

Sets the value for the max_conns parameter of the server directive.

0

angie.software/upstream-zone-size

upstream-zone-size

Sets the size of the shared memory zone for the upstream. For Angie, a special value of 0 disables shared memory zones. For Angie, shared memory zones are required and cannot be disabled. A special value of 0 will be ignored.

256K

angie.software/fail-timeout

fail-timeout

Sets the value for the fail_timeout parameter of the server directive.

10s

angie.software/sticky-cookie-services

None

Configures session persistence.

None

angie.software/sticky-route-services

None

Configures session persistence.

None

angie.software/keepalive

keepalive

Sets the value for the keepalive directive. Note that proxy_set_header Connection ""; is added to the generated configuration when the value > 0.

0

angie.software/health-checks

None

Enables active health checks.

False

angie.software/health-checks-mandatory

None

Configures active health checks as mandatory.

False

angie.software/health-checks-mandatory-queue

None

When active health checks are mandatory, creates a queue where incoming requests are temporarily stored while Angie checks the health of endpoints after configuration reload.

0

angie.software/slow-start

None

Sets the slow start period for the server for the upstream. By default, slow start is activated after the server becomes available or healthy. To enable slow start for newly assigned servers, configure mandatory active health checks.

"0s"

angie.software/use-cluster-ip

None

Enables the use of the service cluster IP and port instead of the default behavior, which uses the pod IP and port. When this field is enabled, fields that configure Angie behavior related to multiple upstreams (such as lb-method and next-upstream) will not take effect, as ANIC will configure Angie with only one upstream that will correspond to the service cluster IP.

False

Rate Limiting#

Annotation

ConfigMap Key

Description

Default Value

Example

angie.software/limit-req-rate

None

Enables request rate limiting for this Ingress by creating a limit_req_zone and applying limit_req for each location. All servers/locations of a single Ingress use one zone. Must have a unit of r/s or r/m.

None

200r/s

angie.software/limit-req-key

None

The key to which the rate limit applies. Can contain text, variables, or their combination. Variables must be enclosed in ${}.

${binary_remote_addr}

${binary_remote_addr}

angie.software/limit-req-zone-size

None

Configures the size of the created limit_req_zone.

10m

20m

angie.software/limit-req-delay

None

Configures the delay parameter of the limit_req directive.

0

100

angie.software/limit-req-no-delay

None

Configures the nodelay parameter of the limit_req directive.

false

true

angie.software/limit-req-burst

None

Configures the burst parameter of the limit_req directive.

None

100

angie.software/limit-req-dry-run

None

Enables dry-run mode. In this mode, rate limiting is not applied, but the number of excess requests is counted as usual in the shared memory zone.

false

true

angie.software/limit-req-log-level

None

Sets the desired logging level for cases when the server refuses to process requests due to rate limits or delays in request processing. Allowed values: info, notice, warn, or error.

error

info

angie.software/limit-req-reject-code

None

Sets the status code returned in response to rejected requests. Must be in the range 400..599.

429

503

angie.software/limit-req-scale

None

Enables constant rate limiting by dividing the configured rate value by the number of Ingress pods currently serving traffic. This adjustment ensures consistent rate limiting even if the number of pods changes due to autoscaling.

Note

This will not work correctly if client requests are not evenly distributed across all Ingress pods (session affinity, long-lived TCP connections with multiple requests, etc.). In such cases, the best results will be achieved by using Angie’s zone synchronization feature.

false

true

Snippets and Custom Templates#

Annotation

ConfigMap Key

Description

Default Value

Example

angie.software/location-snippets

location-snippets

Sets a custom snippet in the context of location.

None

angie.software/server-snippets

server-snippets

Sets a custom snippet in the context of server.

None