Policy#

The Policy resource allows you to configure features such as access control and rate limiting; these can be added to your VirtualServer and VirtualServerRoute resources.

It is implemented as a custom resource.

This is the reference documentation for the Policy resource.

Prerequisites#

Policies work in conjunction with VirtualServer and VirtualServerRoute resources, which must be created separately.

Policy Specification#

Below is an example of a policy that allows access to clients from the subnet 10.0.0.0/8 and denies access to any others:

apiVersion: k8s.angie.software/v1
kind: Policy
metadata:
  name: allow-localhost
spec:
  accessControl:
    allow:
    - 10.0.0.0/8

Field

Description

Type

Required

AccessControl

Access control policy based on the client's IP address.

AccessControl

No

ingressClassName

Specifies which instance of ANIC should handle the Policy resource.

string

No

rateLimit

Rate limiting policy controls the rate of request processing based on a specified key.

RateLimit

No

basicAuth

Basic authentication policy configures Angie to authenticate client requests using HTTP basic authentication credentials.

BasicAuth

No

ingressMTLS

IngressMTLS policy configures client certificate verification.

IngressMTLS

No

egressMTLS

EgressMTLS policy configures authentication and certificate verification for upstream.

EgressMTLS

No

OIDC

OIDC policy configures authentication via an OIDC provider.

OIDC

No

JWT

JWT policy configures Angie to authenticate client requests using JSON Web Tokens.

JWT

No

Note

The policy must include exactly one value.

AccessControl#

The access control policy configures Angie to accept or reject requests from clients with specified IP addresses and subnets.

For example, the following policy allows access to clients from the subnet 10.0.0.0/8 and denies access to any others:

accessControl:
  allow:
  - 10.0.0.0/8

Conversely, the policy below does the opposite: it denies access to clients from 10.0.0.0/8 and allows access to any other clients:

accessControl:
  deny:
  - 10.0.0.0/8

Note

This feature is implemented using the Angie http_access module. The ANIC access control policy supports either allow or deny rules, but not both at the same time (unlike the module).

Field

Description

Type

Required

allow

Allows access for specified networks or addresses. For example, 192.168.1.1 or 10.1.1.0/16.

[]string

No

deny

Denies access for specified networks or addresses. For example, 192.168.1.1 or 10.1.1.0/16.

[]string

No

AccessControl must include either allow or deny.

AccessControl Merge Behavior#

The VirtualServer or VirtualServerRoute resource can reference multiple access control policies. For example, here we reference two policies, each with a configured allow list:

policies:
  - name: allow-policy-one
  - name: allow-policy-two

When referencing multiple access control policies, ANIC will merge their contents into a single allow or deny list.

References to both allow and deny policies, as shown in the example below, are not supported. If both allow and deny lists are specified, ANIC will only use the allow list policies.

policies:
  - name: deny-policy
  - name: allow-policy-one
  - name: allow-policy-two

RateLimit#

The rate limiting policy configures Angie to limit the rate of request processing.

For example, the following policy will limit all subsequent requests coming from a single IP address to a rate of 10 requests per second:

rateLimit:
  rate: 10r/s
  zoneSize: 10M
  key: ${binary_remote_addr}

Note

This feature is implemented using the Angie http_limit_req_module.

Field

Description

Type

Required

rate

Allowed rate of requests. The rate is specified in requests per second (r/s) or requests per minute (r/m).

string

Yes

key

The key to which the rate limit applies. It can contain text, variables, or a combination of both. Variables must be enclosed in ${}. For example: ${binary_remote_addr}. Allowed variables: $binary_remote_addr, $request_uri, $url, $http_, $args, $arg_, $cookie_.

string

Yes

zoneSize

Size of the shared memory zone. Only positive values are allowed. Allowed suffixes are k or m; if no suffix is specified, it defaults to k.

string

Yes

delay

Specifies the threshold at which excess requests become delayed. If this parameter is not specified, all excess requests are delayed.

int

No

noDelay

Disables the delay of excess requests when limiting the number of requests. It takes precedence over delay if both parameters are specified.

bool

No

burst

Excess requests are delayed until their number exceeds the size of burst, after which the request is terminated with an error.

int

No

dryRun

Enables dry run mode. In this mode, rate limiting is not actually applied, but the number of excess requests is counted as usual in the shared memory zone.

bool

No

logLevel

Sets the desired log level for cases when the server refuses to process requests due to rate limits or delays request processing. Allowed values: info, notice, warn or error. The default value is error.

string

No

rejectCode

Specifies the status code returned in response to rejected requests. The value must fall within the range of 400..599. The default value is 503.

int

No

For each policy referenced by a VirtualServer or its VirtualServerRoute, ANIC will generate a single rate limiting zone defined by the http_limit_req directive. If two VirtualServer resources reference the same policy, ANIC will generate two separate rate limiting zones, one for each VirtualServer.

RateLimit Merge Behavior#

The VirtualServer or VirtualServerRoute resource can reference multiple rate limiting policies. For example, here we reference two policies:

policies:
  - name: rate-limit-policy-one
  - name: rate-limit-policy-two

When referencing multiple rate limiting policies, ANIC will configure Angie to use all specified rate limits. If multiple policies are defined, each additional policy inherits the parameters dryRun, logLevel, and rejectCode from the first referenced policy (rate-limit-policy-one in the example above).

BasicAuth#

Configures Angie to authenticate client requests using the basic HTTP authentication scheme.

For example, the following policy will reject all requests that do not contain a valid username and password combination in the HTTP Authentication header:

basicAuth:
  secret: htpasswd-secret
  realm: "My API"

Note

This feature is implemented using the Angie http_auth_basic module.

Field

Description

Type

Required

secret

The name of the Kubernetes secret that contains the Htpasswd configuration. It must be in the same namespace as the Policy resource. The secret must be of type angie.software/htpasswd, and the configuration must be stored in the secret under the key htpasswd; otherwise, the secret will be rejected as invalid.

string

Yes

realm

The realm for basic authentication.

string

No

BasicAuth Merge Behavior#

The VirtualServer or VirtualServerRoute resource can reference multiple basic authentication policies. Only one of them will be applied. All subsequent references will be ignored. For example, here we reference two policies:

policies:
  - name: basic-auth-policy-one
  - name: basic-auth-policy-two

In this example, ANIC will use the configuration from the first reference to the policy "basic-auth-policy-one" and ignore "basic-auth-policy-two".

IngressMTLS#

The IngressMTLS policy configures client certificate verification.

For example, the following policy will verify the client certificate using the Certificate Authority certificate specified in ingress-mtls-secret:

ingressMTLS:
  clientCertSecret: ingress-mtls-secret
  verifyClient: "on"
  verifyDepth: 1

Below is an example of ingress-mtls-secret of type angie.software/ca:

kind: Secret
metadata:
  name: ingress-mtls-secret
apiVersion: v1
type: angie.software/ca
data:
  ca.crt: <base64encoded-certificate>

A VirtualServer resource that references the IngressMTLS policy must have the following settings:

If these conditions are violated, Angie will send a status code of 500 to clients.

You can pass client certificate information, including the certificate itself, to upstream servers. For example:

action:
  proxy:
    upstream: webapp
    requestHeaders:
      set:
        - name: client-cert-subj-dn
          value: ${ssl_client_s_dn} # subject DN
        - name: client-cert
          value: ${ssl_client_escaped_cert} # client certificate in PEM format (urlencoded)

We use the requestHeaders parameter in Action.Proxy to set the values of two headers that Angie will pass to upstream servers. See the list of built-in variables supported by the http_ssl module that you can use to pass client certificate information.

Note

This feature is implemented using the Angie http_ssl module.

Using a Certificate Revocation List#

The IngressMTLS policy supports configuring a CRL for the policy. This can be done in one of two ways.

Note

Only one of these configuration parameters can be used at a time.

  1. Adding a ca.crl field to the angie.software/ca secret type, which contains the certificate revocation list in base64 encoding. Example YAML:

    kind: Secret
metadata:
  name: ingress-mtls-secret
apiVersion: v1
type: angie.software/ca
data:
  ca.crt: <base64encoded-certificate>
  ca.crl: <base64encoded-crl>

  2. Adding a crlFileName field with the name of the CRL file in the IngressMTLS policy specification.

    Note

    This configuration parameter should only be used when the CRL file is larger than 1 MB; otherwise, it is recommended to use the angie.software/ca secret type for managing the CRL.

    Example YAML:

    apiVersion: k8s.angie.software/v1
kind: Policy
metadata:
  name: ingress-mtls-policy
spec:
ingressMTLS:
    clientCertSecret: ingress-mtls-secret
    crlFileName: webapp.crl
    verifyClient: "on"
    verifyDepth: 1

Warning

When configuring the CRL using the ingressMTLS.crlFileName field, additional context should be considered:

  1. ANIC expects the CRL, in this case webapp.crl, to be located in the /etc/angie/secrets directory. To deploy ANIC, a volume mount point will need to be added. Place your CRL in the /etc/angie/secrets directory.

  2. When updating the contents of the CRL (for example, if a new certificate has been revoked), Angie needs to be reloaded to reflect the latest changes. Depending on your environment, this may require updating the name of the CRL file and applying this update to the ingress-mtls.yaml policy so that Angie receives the latest version of the CRL.

Refer to the Kubernetes documentation on volumes for the best implementation for your environment.

Field

Description

Type

Required

clientCertSecret

The name of the Kubernetes secret that contains the certificate of the certificate authority. It must be in the same namespace as the Policy resource. The secret must be of type angie.software/ca, and the configuration must be stored in the secret under the key ca.crt; otherwise, the secret will be rejected as invalid.

string

Yes

verifyClient

Verification for the client. Allowed values: "on", "off", "optional", "optional_no_ca". The default value is "on".

string

No

verifyDepth

Sets the depth of verification in the client certificate chain. The default value is 1.

int

No

crlFileName

The name of the certificate revocation list file. ANIC will look for this file in the /etc/angie/secrets directory.

string

No

IngressMTLS Merge Behavior#

The VirtualServer resource can reference only one IngressMTLS policy. All subsequent references will be ignored. For example, here we reference two policies:

policies:
  - name: ingress-mtls-policy-one
  - name: ingress-mtls-policy-two

In this example, ANIC will use the configuration from the first reference to the policy ingress-mtls-policy-one and ignore ingress-mtls-policy-two.

EgressMTLS#

EgressMTLS configures authentication and certificate verification for upstreams.

For example, the following policy will use egress-mtls-secret for authentication to the upstream application and egress-trusted-ca-secret for verifying the application certificate:

egressMTLS:
  tlsSecret: egress-mtls-secret
  trustedCertSecret: egress-trusted-ca-secret
  verifyServer: on
  verifyDepth: 2

Note

This feature is implemented using the Angie http_proxy module.

Field

Description

Type

Required

tlsSecret

The name of the Kubernetes secret file that contains the TLS certificate and key. It must be in the same namespace as the Policy resource. The secret must be of type kubernetes.io/tls, the certificate must be stored in the secret under the key tls.crt, and the key as tls.key; otherwise, the secret will be rejected as invalid.

string

No

trustedCertSecret

The name of the Kubernetes secret that contains the certificate of the certificate authority. It must be in the same namespace as the Policy resource. The secret must be of type angie.software/ca, and the configuration must be stored in the secret under the key ca.crt; otherwise, the secret will be rejected as invalid.

string

No

verifyServer

Enables verification of the upstream HTTPS server certificate.

bool

No

verifyDepth

Sets the depth of verification in the certificate chain of the proxied HTTPS server. The default value is 1.

int

No

sessionReuse

Allows SSL sessions to be reused with upstreams. The default value is true.

bool

No

serverName

Allows passing the server name via the SNI extension.

bool

No

sslName

Allows overriding the server name used for verifying the upstream HTTPS server certificate.

string

No

ciphers

Specifies the allowed ciphers for requests to the upstream HTTPS server. The default value is DEFAULT.

string

No

protocols

Specifies the protocols for requests to the upstream HTTPS server. The default value is TLSv1, TLSv1.1, TLSv1.2.

string

No

EgressMTLS Merge Behavior#

The VirtualServer or VirtualServerRoute resource can reference multiple EgressMTLS policies. Only one of them will be applied. All subsequent references will be ignored. For example, here we reference two policies:

policies:
  - name: egress-mtls-policy-one
  - name: egress-mtls-policy-two

In this example, ANIC will use the configuration from the first reference to the policy egress-mtls-policy-one and ignore egress-mtls-policy-two.

OIDC#

OIDC (OpenID Connect) provides convenient user authentication through an external provider, using secure tokens to manage access in the system.

Note

This feature is disabled by default. To enable it, set the command-line argument enable-oidc=true.

The OIDC policy configures ANIC as a client (relying party) for authentication via OpenID Connect:

policies:
  - name: oidc-policy

For example, the following configuration uses clientID myclient and clientSecret oidc-secret to authenticate via the OpenID Connect provider https://idp.example.com:

oidc:
 clientID: myclient
 clientSecret: oidc-secret
 authEndpoint: https://idp.example.com/openid-connect/auth
 jwksURI: https://idp.example.com/openid-connect/certs
 tokenEndpoint: https://idp.example.com/openid-connect/token
 scope: openid+profile+email
 accessTokenEnable: true

Mandatory conditions:

  • In the VirtualServer specification, specify the mandatory variables $jwt_claim_iat, $jwt_claim_iss, $jwt_claim_sub, $jwt_claim_aud in maps. These variables ensure token validation during OIDC authentication.

  • Add a secret with the client key. The key must be Base64 encoded:

    apiVersion: v1
kind: Secret
metadata:
  name: oidc-secret
type: angie.software/oidc
data:
  client-secret: <client_secret>

Field

Description

Type

Required

clientID

The client identifier provided by the OIDC provider. Identifies the application that is requesting authorization.

string

Yes

clientSecret

The name of the secret where the client key is stored. It must be in the same namespace as the Policy.

string

Yes

authEndpoint

The URL of the authorization endpoint provided by the OIDC provider. This is the address to which requests for user authentication will be sent.

string

Yes

jwksURI

The URI where the OIDC provider provides certificates (JSON Web Key Set) for verifying the JWT tokens issued by the server (JSON Web Token).

string

Yes

tokenEndpoint

The URL for obtaining authentication and refresh tokens from the OIDC provider.

string

Yes

scope

A list of OIDC scopes to request from the provider. The default value is openid. This value is mandatory. You can also add other scopes using the + sign, for example: openid+profile+email.

string

Yes

accessTokenEnable

Enables the use of Bearer tokens for authorizing access to protected resources on the proxied server.

bool

No

Note

Only one OIDC policy can be referenced at a time in VirtualServer resources. However, one policy can be applied to different routes of VirtualServer resources.

OIDC Merge Behavior#

The VirtualServer resource can reference multiple OIDC policies. Only one of them will be applied. All subsequent references will be ignored. For example, here we reference two policies:

policies:
  - name: oidc-policy-one
  - name: oidc-policy-two

In this example, ANIC will use the configuration from the first reference to the policy oidc-policy-one and ignore oidc-policy-two.

JWT#

The JWT policy configures Angie to authenticate client requests using JSON Web Tokens.

policies:
  - name: jwt-policy

Note

This feature is disabled by default. To enable it, set the command-line argument -enable-jwt=true.

The following policy will reject all requests that do not contain a valid JWT in the HTTP token header:

jwt:
  realm: MyProductAPI
  secret: jwk-secret
  token: $http_token

Mandatory conditions:

  • Add a secret with the client key. The key must be Base64 encoded:

    apiVersion: v1
kind: Secret
metadata:
  name: jwk-secret
type: angie.software/jwk
data:
  jwk: <client_secret>

Field

Description

Type

Required

realm

The realm that the client will see when requesting authentication, for example, a string describing the protected resource.

string

Yes

secret

The name of the Kubernetes secret that stores the JSON Web Key (JWK) used to verify the signatures of JWT tokens. Angie will use the keys from this secret to validate the signatures of JWTs and verify their authenticity. The secret must be in the same namespace as the Policy.

string

Yes

token

Specifies where to extract the JWT from. For example, the variable $http_token refers to the value of the HTTP token header that will be sent by the client.

string

No

JWT Merge Behavior#

The VirtualServer resource can reference multiple JWT policies. Only one of them will be applied. All subsequent references will be ignored. For example, here we reference two policies:

policies:
  - name: jwt-policy-one
  - name: jwt-policy-two

In this example, ANIC will use the configuration from the first reference to the policy jwt-policy-one and ignore jwt-policy-two.

Applying Policies#

Policies can be applied to both VirtualServer and VirtualServerRoute resources. For example:

- VirtualServer:
    apiVersion: k8s.angie.software/v1
    kind: VirtualServer
    metadata:
      name: cafe
      namespace: cafe
    spec:
      host: cafe.example.com
      tls:
        secret: cafe-secret
      policies: # spec policies
        - name: policy1
    upstreams:
      - name: coffee
        service: coffee-svc
        port: 80
    routes:
      - path: /tea
        policies: # route policies
        - name: policy2
          namespace: cafe
          route: tea/tea
      - path: /coffee
        policies: # route policies
          - name: policy3
            namespace: cafe
        action:
          pass: coffee

In the case of VirtualServer, a policy can be applied:

  • to all routes (spec policies)

  • to a specific route (route policies)

    Route policies take precedence over specification policies of the same type. If in the example above the types of policies policy-1 and policy-3 are AccessControl, then for requests to cafe.example.com/coffee, Angie will apply policy-3.

    Overriding is ensured by Angie: specification policies are implemented in the context of the server configuration, while route policies are implemented in the context of location. As a result, route policies have priority within the same type.

  • The VirtualServerRoute resource referenced by the above VirtualServer:

    apiVersion: k8s.angie.software/v1
kind: VirtualServerRoute
metadata:
  name: tea
  namespace: tea
spec:
  host: cafe.example.com
  upstreams:
    - name: tea
      service: tea-svc
      port: 80
  subroutes: # subroute policies
    - path: /tea
      policies:
      - name: policy4
        namespace: tea
      action:
        pass: tea

    In the VirtualServerRoute, a policy can be applied to a nested route (nested route policies).

    Nested route policies take precedence over specification policies of the same type. In the example above, if the type of policies policy-1 (in VirtualServer) and policy-4 is AccessControl, then for requests to cafe.example.com/tea, Angie will apply policy-4. As with VirtualServer, overriding is ensured by Angie.

    Nested route policies always take precedence over route policies regardless of type. For example, the policy policy-2 in the VirtualServer route will be ignored on the nested route /tea, as it has its own policies (in our case, only policy4). If the nested route had no policies, then policy-2 would be applied. This overriding is performed by ANIC - the context of the location for the nested route will contain either route policies or nested route policies, but not both together.

Invalid Policies#

Angie will consider a policy invalid if any of the following conditions are met:

  • The policy fails comprehensive validation.

  • The policy is not present in the cluster.

  • The policy does not meet the requirements for its specific type. For example, the ingressMTLS policy requires TLS termination to be enabled in the VirtualServer.

In the case of an invalid policy, Angie returns a status code of 500 for client requests with the following rules:

  • If a policy is referenced by a VirtualServer route or a VirtualServerRoute nested route, Angie will return a status code of 500 for requests to the URI of that route.

  • If a policy reference is specified in the VirtualServer specification, Angie will return a status code of 500 for requests to all URIs of that VirtualServer.

If a policy is invalid, the VirtualServer or VirtualServerRoute will have a status with a warning about the state and a message explaining why the policy was deemed invalid.

Validation#

There are two types of validation available for the Policy resource:

  • Structural validation using kubectl and the Kubernetes API server.

  • Comprehensive validation using ANIC.

Structural Validation#

The custom resource definition for Policy includes a structural OpenAPI schema that describes the type of each field of the resource.

If you attempt to create (or update) a resource that violates the structural schema (for example, using a string value instead of an array of strings in the allow field), then kubectl and the Kubernetes API server will reject the resource.

  • Example of checking with kubectl:

    kubectl apply -f access-control-policy-allow.yaml

error: error validating "access-control-policy-allow.yaml": error validating data: ValidationError(Policy.spec.accessControl.allow): invalid type for software.angie.k8s.v1.Policy.spec.accessControl.allow: got "string", expected "array"; if you choose to ignore these errors, turn validation off with --validate=false

  • Example of checking with the Kubernetes API server:

    kubectl apply -f access-control-policy-allow.yaml --validate=false

The Policy "webapp-policy" is invalid: spec.accessControl.allow: Invalid value: "string": spec.accessControl.allow in body must be of type array: "string"

If the resource passes structural validation, comprehensive validation by ANIC is performed.

Comprehensive Validation#

ANIC checks the fields of the Policy resource. If the resource is invalid, ANIC will reject it. The resource will remain in the cluster, but ANIC will ignore it.

You can use kubectl to check whether ANIC successfully applied the Policy configuration. For the policy mypolicy, we can run:

kubectl describe pol mypolicy

. . .
Events:
  Type    Reason          Age   From                      Message
-----
Normal  AddedOrUpdated  11s   angie-ingress-controller  Policy default/mypolicy was added or updated

Note that the "Events" section includes a Normal event with the reason AddedOrUpdated, which informs us that the configuration was successfully applied.

If you create an invalid resource, ANIC will reject it and issue a Rejected event. For example, if you create a policy mypolicy with an invalid IP address 10.0.0. in the allow field, you will get:

kubectl describe policy mypolicy
. . .
Events:
  Type     Reason    Age   From                      Message
-----
Warning  Rejected  7s    angie-ingress-controller  Policy default/mypolicy is invalid and was rejected: spec.accessControl.allow[0]: Invalid value: "10.0.0.": must be a CIDR or IP

Note that the "Events" section includes a warning event indicating the reason for the rejection.

Additionally, this information is also available in the status field of the Policy resource. Note the "Status" section of the policy:

kubectl describe pol mypolicy

. . .
Status:
  Message:  Policy default/mypolicy is invalid and was rejected: spec.accessControl.allow[0]: Invalid value: "10.0.0.": must be a CIDR or IP
  Reason:   Rejected
  State:    Invalid

Note

If you make an existing resource invalid, ANIC will reject it.

On this page