gorouter job from routing/0.264.0

Gorouter maintains a dynamic routing table based on updates received from NATS and (when enabled) the Routing API. This routing table maps URLs to backends. The router finds the URL in the routing table that most closely matches the host header of the request and load balances across the associated backends.

Github source: 5283839c or master branch

Properties¶

endpoint_dial_timeout_in_seconds¶

Maximum time in seconds for gorouter to establish a TCP connection with a backend. This timeout comes before tls_handshake_timeout_in_seconds and request_timeout_in_seconds.

Default

5

for_backwards_compatibility_only¶

empty_pool_response_code_503¶

This property is provided for backwards compatibility reasons. We recommend leaving it set to true. When it set to true, the gorouter will return a 503 when a route is known but there is nothing in the pool available to handle the request. When false, the gorouter will return a 404.

Default

true

empty_pool_timeout¶

This property depends on empty_pool_response_code_503. If empty_pool_response_code_503 is set to true, then gorouter will consider empty_pool_timout parameter. If set to 0, gorouter will remove empty pools after pruning their endpoints. The gorouter will respond with 404 code. If set to different value, gorouter will wait until this timeout is reached then will delete the empty pools. Before pool deletion, gorouter will continue to respond with 503 code. The type of empty_pool_timeout is time.Duration. We have to specify the unit of time. For example, a value “10s” means 10 seconds.

Default

0s

healthchecker¶

failure_counter_file¶

File used by the healthchecker to monitor consecutive failures.

Default

/var/vcap/data/gorouter/counters/consecutive_healthchecker_failures.count

metron¶

port¶

The port used to emit dropsonde messages to the Metron agent.

Default

3457

nats¶

ca_certs¶

String of concatenated certificate authorities in PEM format, used to validate certificates presented by NATS.

cert_chain¶

Certificate chain used for client authentication to NATS. In PEM format.

machines¶

IPs of each NATS cluster member

Example

|+
  - 192.168.50.123
  - 192.168.52.123

password¶

Password for NATS authentication

Example

natSpa55w0rd

port¶

TCP port of NATS servers

Example

4222

private_key¶

Private key used for client authentication to NATS. In PEM format.

tls_enabled¶

When enabled, Gorouter will prefer to connect to NATS over TLS

Default

false

user¶

User name for NATS authentication

Example

nats

request_timeout_in_seconds¶

This configures a “request timeout” and a “backend idle timeout”. Requests from router to backend endpoints that are longer than this duration will be canceled and logged as backend-request-timeout errors. In addition, TCP connections between router and backend endpoints that are idle for longer than this duration will be closed. Related properties: router.max_idle_connections and router.keep_alive_probe_interval.

Default

900

router¶

backends¶

ca¶

Certificate authority that was used to sign certificates for TLS-registered backends. In PEM format.

cert_chain¶

Certificate chain used for client authentication to TLS-registered backends. In PEM format.

max_attempts¶

Maximum number of attempts on failing requests against backend routes. This includes CF apps and route-registrar endpoints. A value of 0 implies indefinite retries, i.e. retry until success or endpoint list is exhausted.

Default

3

max_conns¶

Maximum concurrent TCP connections per backend. When set to 0 there is no limit

Default

500

private_key¶

Private key used for client authentication to TLS-registered backends. In PEM format.

balancing_algorithm¶

Algorithm used to distribute requests for a route across backends. Supported values are round-robin and least-connection

Default

round-robin

ca_certs¶

Required. List of Strings of concatenated certificate authorities in PEM format, used to validate server certificates provided by remote systems. Gorouter also trust certificates signed by well-known CAs and by CA certificates installed on the filesystem. These CA certificates are also used to validate client certificates when router.only_trust_client_ca_certs is false.

cipher_suites¶

An ordered, colon-delimited list of golang supported TLS cipher suites in OpenSSL or RFC format. The selected cipher suite will be negotiated according to the order of this list during a TLS handshake. See https://github.com/golang/go/blob/release-branch.go1.9/src/crypto/tls/cipher_suites.go#L369-L390 for golang supported cipher suites. The first four of these are supported for TLSv1.0/1.1 only. See https://www.openssl.org/docs/man1.1.0/apps/ciphers.html for a mapping of OpenSSL and RFC suite names.

Default

ECDHE-RSA-AES128-GCM-SHA256:TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384

client_ca_certs¶

Required. String of concatenated certificate authorities in PEM format, used to validate certificates provided by remote systems. By default, Gorouter will trust certificates signed by well-known CAs and by CA certificates installed on the filesystem.

Default

""

client_cert_validation¶

none - Gorouter will not request client certificates in TLS handshakes, and will ignore them if presented. Incompatible with forwarded_client_cert: forward or sanitize_set. request - Gorouter will request client certificates in TLS handshakes, and will validate them when presented, but will not require them. require - Gorouter will fail a TLS handshake if the client does not provide a certificate signed by a CA it trusts. Select this option if your load balancer terminates TLS and does not require client certificates, and the load balancer provides a compatible client certificate of its own to Gorouter in an independent TLS handshake. This option may also be selected for Isolation Segments when Gorouter is the first point of TLS termination. Many clients of CF platform APIs do not present client certificates in TLS handshakes, so the first point of TLS termination for requests to the system domain must not require them. This option has no effect on the HTTP listener; to disable HTTP support set disable_http: true.

Default

request

debug_address¶

Endpoint for process profiling. For more info see https://github.com/cloudfoundry/debugserver

Default

127.0.0.1:17002

disable_http¶

Disables the http listener on port specified by router.port. This cannot be set to true if enable_ssl is false.

Default

false

disable_log_forwarded_for¶

(optional, boolean) When false, gorouter will include the value of the HTTP header X-Forwarded-For which it sends to a backend. When true, the value of this header will not be logged. This is to comply with EU regulations that do not allow persisting personal data.

Default

false

disable_log_source_ip¶

(optional, boolean) When false, gorouter will include the source IP in access logs. When true, the source IP will not be logged. This is to comply with EU regulations that do not allow persisting personal data. When Gorouter is behind an L3 load balancer that persists the IP of the originating client, set this to true to comply with GDPR.

Default

false

dns_health_check_host¶

Host to ping for confirmation of DNS resolution, only used when Routing API is enabled

Default

uaa.service.cf.internal

drain_timeout¶

Delay in seconds after drain wait has completed to allow existing connections to finish their active requests. The drain will begin when there are no active connections or until the timeout is reached. Any remaining connections that are open when the drain timeout is reached will be closed. To reduce the drain time during deploys, set the value of drain timeout below the request timeout.

Default

900

drain_wait¶

Delay in seconds after shut down is initiated before server stops listening. During this time the server will reject requests to the /health endpoint. This accommodates requests forwarded by a load balancer until it considers the router unhealthy.

Default

20

enable_http2¶

Enables support for HTTP/2 ingress traffic to the Gorouter. Also enables the option to use the HTTP/2 protocol for traffic to specified backends.

Default

true

enable_proxy¶

Enables support for the popular PROXY protocol, allowing downstream load balancers that do not support HTTP to pass along client information.

Default

false

enable_ssl¶

When enabled, Gorouter will listen on port 443 and terminate TLS for requests received on this port.

Default

false

extra_headers_to_log¶

An array of headers that access log events will be annotated with. This only applies to headers on requests.

Default

[]

force_forwarded_proto_https¶

Enables setting X-Forwarded-Proto header if SSL termination happened upstream and incorrectly set the header value. When this property is set to true gorouter sets the header X-Forwarded-Proto to https. When this value set to false, gorouter set the header X-Forwarded-Proto to the protocol of the incoming request

Default

false

forwarded_client_cert¶

How to handle the x-forwarded-client-cert (XFCC) HTTP header. Possible values are: - always_forward: Always forward the XFCC header in the request, regardless of whether the client connection is mTLS. Use this value when your load balancer is forwarding the client certificate and requests are not forwarded to Gorouter over mTLS. In the case where the connection between load balancer and Gorouter is mTLS, the client certificate received by Gorouter in the TLS handshake will not be put in the XFCC header. - forward: Forward the XFCC header received from the client only when the client connection is mTLS. This is a more secure version of always_forward. The client certificate received by Gorouter in the TLS handshake will not be put in the XFCC header. Requires client_cert_validation: request or require. - sanitize_set: Strip any instances of XFCC headers from the client request. When the client connection is mTLS, the client certificate received by Gorouter in the TLS handshake will be put in this header. Values will be base64 encoded PEM. Use this value when Gorouter is the first component to terminate TLS. Requires client_cert_validation: request or require.

Default

always_forward

frontend_idle_timeout¶

(optional, integer) Duration in seconds to maintain an open connection when client supports keep-alive. This property must be configured with regards to how an IaaS-provided load balancer behaves in order to prevent connections from being closed prematurely. Generally, this timeout must be greater than that of the load balancer. As examples, GCP HTTP(S) load balancer has a default timeout of 600 seconds so a value greater than 600 is recommended and AWS ELB has a default timeout of 60 seconds so a value greater than 60 is recommended. However, depending on the IaaS, this timeout may need to be shorter than the load balancer’s timeout, e.g., Azure’s load balancer times out at 240 seconds (by default) and GCP TCP load balancer times out at 600 seconds without sending a TCP RST to clients, so a value lower than this is recommended in order to force it to send the TCP RST.

Default

900

healthcheck_user_agent¶

DEPRECATED. Use /health endpoint on port specified by status.port. User-Agent for the health check agent (usually the Load Balancer).

Default

HTTP-Monitor/1.1

Example

ELB-HealthChecker/1.0

html_error_template¶

(optional) When given a valid Go template, gorouter will generate HTML error pages instead of plain text error pages. The following is an example of a valid HTML template:

Code: {{ .Status }} {{ .StatusText }} Message: {{ .Message }} Cause: {{ .Header.Get “X-Cf-RouterError” }}

Default

""

http_rewrite¶

responses¶

add_headers_if_not_present¶

(optional, array pairs name-value) If set, gorouter will add the given headers into the response if they are not already present.

Example

- name: Strict-Transport-Security
  value: max-age=31536000; includeSubDomains; preload
- name: Cache-Control
  value: no-cache

remove_headers¶

(optional, array pairs name-value) List of HTTP headers that will be stripped from responses.

Example

- name: X-Vcap-Request-Id
- name: Accept-Ranges

ip_local_port_range¶

Ephemeral port range. router.set_kernel_parameters must be enabled.

Default

1024 65535

isolation_segments¶

Routes with these isolation segments will be registered. Used in combination with routing_table_sharding_mode.

Default

[]

keep_alive_probe_interval¶

Interval between TCP keep alive probes. Value is a string (e.g. “10s”)

Default

1s

load_balancer_healthy_threshold¶

Time period in seconds to wait until declaring the router instance started after starting the listener socket. This allows an external load balancer time to register the instance as healthy.

Default

20

logging¶

format¶

timestamp¶

Format for timestamp in component logs. Valid values are ‘rfc3339’, ‘deprecated’, and ‘unix-epoch’.” ‘rfc3339’ is the recommended format. It will result in all timestamps controlled by gorouter to be in RFC3339 format, which is human readable. This includes stdout, pre-start, and post-start logs. This does not include stderr logs from golang libraries. ‘deprecated’ will result in all timestamps being in the format they were before the rfc3339 flag was introduced. This format is different for different logs. We do not recommend using this flag unless you have scripts that expect a particular timestamp format. ‘unix-epoch’ is an old flag that we do not recommend using, but we are keeping for backwards compatibility. It will result in the gorouter logs to be in unix-epoch format. This does not effect pre-start or post-start logs. This does not effect stderr logs from golang libaries.

Default

rfc3339

logging_level¶

Log level for router

Default

info

max_header_kb¶

This value controls the maximum number of bytes (in KB) the gorouter will read parsing the request header’s keys and values, including the request line. It does not limit the size of the request body. Requests with larger headers will result in a 431 status code. Must be between 1 and 1024kb.

Default

1024

max_idle_connections¶

When 0, keepalive connections are disabled. When a number greater than zero, keepalives are enabled, and Gorouter will keep up to this number of idle keepalive connections open across all backends. When keepalive connections are enabled, the maximum number of idle keepalive connections Gorouter will keep open to an individual backend is 100, which cannot be changed.

Default

100

max_open_files¶

The number of file descriptors a router can have open at one time

Default

100000

max_tls_version¶

Maximum accepted version of TLS protocol. Versions below this will, down to the min_tls_version, will also be accepted. Valid values are TLSv1.2 and TLSv1.3. Warning: Setting this to TLSv1.3 will cause things to fail with any Java clients using versions of Java without a fix for the following issue: https://bugs.openjdk.java.net/browse/JDK-8236039

Default

TLSv1.2

min_tls_version¶

Minimum accepted version of TLS protocol. All versions above this, up to the max_tls_version, will also be accepted. Valid values are TLSv1.0, TLSv1.1, TLSv1.2, and TLSv1.3.

Default

TLSv1.2

number_of_cpus¶

Number of CPUs to utilize, the default (-1) will equal the number of available CPUs

Default

-1

offset¶

Default

0

only_trust_client_ca_certs¶

When router.only_trust_client_ca_certs is true, router.client_ca_certs are the only trusted CA certs for client requests. When router.only_trust_client_ca_certs is false, router.client_ca_certs are trusted in addition to router.ca_certs and the CA certificates installed on the filesystem. This will have no affect if the router.client_cert_validation property is set to none.

Default

false

per_app_prometheus_http_metrics_reporting¶

Reports the http latency in a prometheus histogram for each application

Default

false

per_request_metrics_reporting¶

Report the metrics latency, latency. and route_lookup_time for each request

Default

true

port¶

Listening Port for Router.

Default

80

prometheus¶

ca_cert¶

TLS CA cert to verify requests to prometheus endpoint.

Default

""

cert¶

TLS certificate for prometheus server.

Default

""

key¶

TLS private key for prometheus server.

Default

""

port¶

Port for the prometheus endpoint.

server_name¶

The server name used in the certificate for the metrics endpoint.

redact_query_parameters¶

(optional, string) This option can be used to hide potentially sensitive information on GET request query parameters in the access log. none: Log all query parameters on GET requests. E.g. GET /users?name=peter all: Log no query parameters on GET requests. E.g. GET /users hash: Log query parameters in hashed form only. E.g. GET /users?hash=a6909eece4005d85fb1894cc4efa38564b9c30c3

Default

none

requested_route_registration_interval_in_seconds¶

On startup, the router will delay listening for requests by this duration to increase likelihood that it has a complete routing table before serving requests. The router also broadcasts the same duration as a recommended interval to registering clients via NATS. This must be less than 60, otherwise monit will mark the process as failed.

Default

20

route_services¶

cert_chain¶

Certificate chain used for client authentication to TLS-registered route services. In PEM format.

max_attempts¶

Maximum number of attempts on failing requests against route service URLs. The minimum value for this setting is 1. This prevents gorouter from getting blocked by indefinite retries.

Default

3

private_key¶

Private key used for client authentication to TLS-registered route services. In PEM format.

route_services_internal_lookup¶

setting this property to true causes gorouter to bypass another trip through the load balancer when directing traffic to a route service that is a known route by the gorouter.

Default

false

route_services_internal_lookup_allowlist¶

a list of host names for route services that should be resolved internally. Each entry can be a fully qualified domain name or DNS wildcard (i.e. wildcard on 1 segment of a subdomain). If the list is empty, it is not in effect and internal lookup will be attempted for all host names, which can lead to CVE-2019-3789. Please turn on internal lookup only with an allowlist.

Default

[]

route_services_recommend_https¶

Route Services are told where to send requests after processing using the X-CF-Forwarded-Url header. When this property is true, the scheme for this URL is https. When false, the scheme is http. As requests from Route Services to applications on CF transit load balancers and gorouter, disable this property for deployments that have TLS termination disabled.

Default

true

route_services_secret¶

Support for route services is disabled when no value is configured. A robust passphrase is recommended.

Default

""

route_services_secret_decrypt_only¶

To rotate keys, add your new key here and deploy. Then swap this key with the value of route_services_secret and deploy again.

Default

""

route_services_timeout¶

Expiry time of a route service signature in seconds

Default

60

routing_table_sharding_mode¶

all: all routes will be registered. shared-and-segments: both routes for the configured isolation segments and those that do not have an isolation segment specified will be registered. segments: only routes for the configured isolation segments will be registered.

Default

all

sanitize_forwarded_proto¶

When true, Gorouter will strip the X-Forwarded-Proto header when present in client request and set it to the scheme of the request. When false, Gorouter will pass through the value of this header. When force_forwarded_proto_https: true, this property will be ignored. Otherwise, we recommend setting the property to true if Gorouter is the first component to terminate TLS, and set to false when your load balancer is terminating TLS and setting the X-Forwarded-Proto header.

Default

false

secure_cookies¶

Set secure flag on http cookies

Default

false

send_http_start_stop_client_event¶

Send a httpstartstopevent of type client for each request

Default

true

send_http_start_stop_server_event¶

Send a httpstartstopevent of type server for each request

Default

true

set_kernel_parameters¶

Optimize kernel networking settings for GoRouter. If you deploy GoRouter in a container set this to false. This configures the following kernel parameters: * Increases the number of allowed IP local port ranges specified by the router.ip_local_port_range value. * Reserves ports configured for gorouter and other bosh components via /proc/sys/net/ipv4/ip_local_reserved_ports. * Reduces the timeout of TCP_FIN_TIMEOUT, so that TCP/IP can release closed connections faster, making more resources available for new connections. * Disables TCP_TW_RECYCLE to enable fast recycling of TIME_WAIT sockets. * Enables TCP_TW_REUSE to reuse sockets in TIME_WAIT state for new connections when it is safe from protocol viewpoint * Increase the number of max queued connections than are allowed by default.

Default

true

ssl_skip_validation¶

Skip validation of TLS certificates received from route services and UAA.

Default

false

status¶

password¶

Password for HTTP basic auth to the /varz and /routes endpoints.

port¶

Port for the /health, /varz, and /routes endpoints.

Default

8080

user¶

Username for HTTP basic auth to the /varz and /routes endpoints.

Default

router-status

sticky_session_cookie_names¶

The names of the cookies to use for handling sticky sessions

Default

- JSESSIONID

suspend_pruning_if_nats_unavailable¶

Suspend pruning of routes when NATs is unavailable and maintain the current routing table. Note: only non-TLS routes are ever pruned.

Default

true

tls_pem¶

Array of private keys and certificates for serving TLS requests. Each element in the array is an object containing fields ‘private_key’ and ‘cert_chain’, each of which supports a PEM block. Required if router.enable_ssl is true.

Example

|+
  - cert_chain: |
      -----BEGIN CERTIFICATE-----
      -----END CERTIFICATE-----
      -----BEGIN CERTIFICATE-----
      -----END CERTIFICATE-----
    private_key: |
      -----BEGIN RSA PRIVATE KEY-----
      -----END RSA PRIVATE KEY-----

tls_port¶

Listening port for SSL connections to the router, when SSL is enabled.

Default

443

trace_key¶

If the X-Vcap-Trace request header is set and has this value, trace headers are added to the response.

Default

22

tracing¶

enable_w3c¶

Enables the addition of the W3C tracing headers to incoming requests. If the traceparent and tracestate headers exist on the incoming request, they will be updated, if they do not exist they will be created. The W3C tracing specification has more information: https://www.w3.org/TR/trace-context/

Default

false

enable_zipkin¶

Enables the addition of the X-B3-Trace-Id header to incoming requests. If the header already exists on the incoming request, it will not be overwritten.

Default

false

w3c_tenant_id¶

Specifies the tenant ID to use in the W3C tracestate header, only used when W3C tracing headers are enabled. If specified, the tracestate identifier will be “tenant-id@gorouter” where “tenant-id” is the value specified. If not specified, the tracestate identifier will be “gorouter” The W3C tracing specification has more information: https://www.w3.org/TR/trace-context/

Default

""

write_access_logs_locally¶

Enables writing access log to local disk.

Default

true

routing_api¶

auth_disabled¶

When false, Routing API requires OAuth tokens for authentication.

Default

false

ca_certs¶

String of concatenated certificate authorities in PEM format, used to validate certificates provided by the routing API.

cert_chain¶

Certificate chain used for client authentication to the routing API. In PEM format.

enabled¶

When enabled, GoRouter will fetch HTTP routes from the Routing API in addition to routes obtained via NATS.

Default

false

port¶

Port on which Routing API is running.

private_key¶

Private key used for client authentication to the routing API. In PEM format.

uri¶

URL where the routing API can be reached internally

Default

https://routing-api.service.cf.internal

tls_handshake_timeout_in_seconds¶

Maximum time in seconds for gorouter to establish a TLS connection with a backend container. This timeout is for establishing the TLS connection only. Actual HTTP request timeout is defined by router.request_timeout_in_seconds.

Default

10

uaa¶

ca_cert¶

Certificate authority for communication between clients and uaa.

Default

""

clients¶

gorouter¶

secret¶

Password for UAA client for the gorouter.

ssl¶

port¶

Secure Port on which UAA is running.

token_endpoint¶

UAA token endpoint host name. Do not include a scheme in this value; TCP Router will always use TLS to connect to UAA.

Default

uaa.service.cf.internal

websocket_dial_timeout_in_seconds¶

Maximum time in seconds for gorouter to establish a websocket upgrade for the websocket ForwardIO connection with a backend. This timeout comes before tls_handshake_timeout_in_seconds and request_timeout_in_seconds. When not set, defaults to endpoint_dial_timeout_in_seconds.

Default

endpoint_dial_timeout_in_seconds

Templates¶

Templates are rendered and placed onto corresponding instances during the deployment process. This job's templates will be placed into /var/vcap/jobs/gorouter/ directory (learn more).

• bin/bpm-pre-start (from bpm-pre-start.erb)
• bin/dns_health_check (from dns_health_check.erb)
• bin/drain (from drain.erb)
• bin/nats_client (from nats_client.erb)
• bin/post-start (from post-start.erb)
• bin/pre-start (from pre-start.erb)
• bin/retrieve-local-routes (from retrieve-local-routes.erb)
• bin/setup-jq (from setup-jq.erb)
• config/bpm.yml (from bpm.yml.erb)
• config/certs/prometheus/prometheus.crt (from prometheus.crt.erb)
• config/certs/prometheus/prometheus.key (from prometheus.key.erb)
• config/certs/prometheus/prometheus_ca.crt (from prometheus_ca.crt.erb)
• config/certs/uaa/ca.crt (from uaa_ca.crt.erb)
• config/error.html (from error.html.erb)
• config/gorouter.yml (from gorouter.yml.erb)
• config/healthchecker.yml (from healthchecker.yml.erb)
• config/indicators.yml (from indicators.yml.erb)
• config/prom_scraper_config.yml (from prom_scraper_config.yml.erb)

Packages¶

Packages are compiled and placed onto corresponding instances during the deployment process. Packages will be placed into /var/vcap/packages/ directory.

• gorouter
• routing-healthchecker
• routing_utils