TCP Health Checks

Introduction

NGINX and NGINX Plus can continually test your TCP upstream servers, avoid the servers that have failed, and gracefully add the recovered servers into the load‑balanced group.

Prerequisites

• You have configured an upstream group of TCP servers in the stream context, for example:

  stream {
      #...
      upstream stream_backend {
      server backend1.example.com:12345;
      server backend2.example.com:12345;
      server backend3.example.com:12345;
     }
      #...
  }
  

• You have configured a server that passes TCP connections to the server group:

  stream {
      #...
      server {
          listen     12345;
          proxy_pass stream_backend;
      }
      #...
  }
  

Passive TCP Health Checks

If an attempt to connect to an upstream server times out or results in an error, NGINX open source or NGINX Plus can mark the server as unavailable and stop sending requests to it for a defined amount of time. To define the conditions under which NGINX considers an upstream server unavailable, include the following parameters to the server directive

• fail_timeout – The amount of time within which a specified number of connection attempts must fail for the server to be considered unavailable. Also, the amount of time that NGINX considers the server unavailable after marking it so.
• max_fails – The number of failed attempts that happen during the specified time for NGINX to consider the server unavailable.

The default values are 10 seconds and 1 attempt. So if a connection attempt times out or fails at least once in a 10‑second period, NGINX marks the server as unavailable for 10 seconds. The example shows how to set these parameters to 2 failures within 30 seconds:

upstream stream_backend {
    server backend1.example.com:12345 weight=5;
    server backend2.example.com:12345 max_fails=2 fail_timeout=30s;
    server backend3.example.com:12346 max_conns=3;
}

upstream backend {
    server backend1.example.com:12345 slow_start=30s;
    server backend2.example.com;
    server 192.0.0.1 backup;
}

Active TCP Health Checks

Health checks can be configured to test a wide range of failure types. For example, NGINX Plus can continually test upstream servers for responsiveness and avoid servers that have failed.

NGINX Plus sends special health check requests to each upstream server and checks for a response that satisfies certain conditions. If a connection to the server cannot be established, the health check fails, and the server is considered unhealthy. NGINX Plus does not proxy client connections to unhealthy servers. If several health checks are configured for an upstream group, the failure of any check is enough to consider the corresponding server unhealthy.

To enable active health checks:

1. Specify a shared memory zone – a special area where the NGINX Plus worker processes share state information about counters and connections. Add the zone directive to the upstream server group and specify the zone name (here, stream_backend) and the amount of memory (64 KB):

   stream {
       #...
       upstream stream_backend {
           zone   stream_backend 64k;
           server backend1.example.com:12345;
           server backend2.example.com:12345;
           server backend3.example.com:12345;
       }
       #...
   }
   

2. Enable active health checks for the upstream group with the health_check directive:

   stream {
       #...
       server {
           listen        12345;
           proxy_pass    stream_backend;
           health_check;
           #...
       }
   }
   

3. If necessary, reduce a timeout between two consecutive health checks with the health_check_timeout directive. This directive overrides the proxy_timeout value for health checks, as for health checks this timeout needs to be significantly shorter:

   stream {
       #...
       server {
           listen               12345;
           proxy_pass           stream_backend;
           health_check;
           health_check_timeout 5s;
       }
   }
   

4. By default, NGINX Plus sends health check messages to the port specified by the server directive in the upstream block. You can specify another port for health checks, which is particularly helpful when monitoring the health of many services on the same host. To override the port, specify the port parameter of the health_check directive:

   stream {
       #...
       server {
           listen               12345;
           proxy_pass           stream_backend;
           health_check         port=12346;
           health_check_timeout 5s;
       }
   }