diego/0.1488.0

Changes from v0.1487.0 to v0.1488.0

• Verified with garden-runc-release v1.0.2.
• Verified with garden-windows-bosh-release v0.0.9.
• Verified with etcd-release v78.
• Verified with cflinuxfs2-rootfs-release v1.38.0.

IMPORTANT: Diego v0.1488.0 includes configuration to secure the cell rep API via mutual TLS authentication. Please consult “Upgrading to a TLS-Secured Cell Rep API” in the diego-release documentation for detailed steps on how to enable this security without incurring downtime in the Diego deployment. We recommend that all operators enable this security configuration, and doing so on upgrading from v0.1487.0 or earlier requires one fewer deployment step than doing so after upgrading to v0.1488.0 or later.

IMPORTANT: Diego v0.1488.0 requires v1.0.2 or later of garden-runc-release on Linux cells, and garden-linux-release is no longer supported. This change is required because Diego story #133264039 makes use of a new Garden API call, BulkNetOut, that is available only on v1.0.2 of garden-runc-release. Version v0.0.9 of garden-windows-bosh-release is also supported. The manifest generation scripts in diego-release now default to using garden-runc-release as the Linux implementation of the Garden backend instead of garden-linux-release, and the -g flag is now deprecated as a no-op. Please note that changing from garden-linux-release to garden-runc-release requires operators to recreate their Diego cell VMs (either explicitly or as a side-effect of a stemcell deploy), as upgrading from garden-linux to garden-runc in place is not supported. Also, the garden-runc-release repository recently moved to the “cloudfoundry” GitHub organization, so https://bosh.io/releases/github.com/cloudfoundry/garden-runc-release is now the correct source for final BOSH releases.

Significant changes

BBS API

• As a Diego operator, I expect BBS event consumers not to leak connections on failure

BBS Relational Datastore

• Investigate using a less strict isolation level when using mysql as the bbs backend store
• Slow convergence on postgres with many actual lrps

Container Placement

• As a BBS API client, I would like to observe that my LRP instances are not running or my Tasks have failed because there are no cells available matching those placement tags
• As a Diego operator, I expect reps on windows to consume placement-tags configuration
• Fix regression: spaces should be allowed in cell rep AZ

Component Coordination

• As a Diego operator, I expect the default lock TTL to be set to 15 seconds (in flight)

cfdot

• As a cfdot user, I expect to be able to update a DesiredLRP
• As a cfdot user, I expect to be able to retrieve a single Task from the BBS by its guid

SSH

• As a Diego operator, I expect the ssh-proxy to present an endpoint for an HTTP healthcheck
• cloudfoundry/diego-ssh #24: Connection resets

Routing

• cloudfoundry/route-emitter #5: Keep emitting routes when losing the lock and consul is unhealthy

Volume Support (Experimental)

• monit stop/start volume drivers should not lead to bad cell state
• Executor scans and removes unused volume mounts upon startup
• Retry failed voldriver operations
• cancel driver mount when connection drops for mount on localvolume
• use new lager.Context when mounting
• Wire cancellation logic into all voldriver handlers
• clean up and extract voldiver backoff into it’s own package
• Refactor voldriver’s remote client to use the new backoff package

Container Networking Support (Experimental)

• Fix failing vizzini test for ASGs
• cloudfoundry/vizzini #1: ASG tests no longer depend on a listener LRP

Garden-RunC Integration

• As a Diego operator, I expect garden-runc to be the only officially supported Linux garden backend
• As a Diego operator, I expect Diego to use Garden’s new BulkNetOut API to reduce container start-up time on 4.4-kernel systems

Component Logging and Metrics

• As a Diego operator, I expect the BBS to produce an info-level log when starting an ActualLRP only if it makes a real change to the etcd store

Test Suites and Tooling

• cloudfoundry/consuladapter #4: Skip over consul when resetting Consul services

Security

• As a Diego operator, I expect the cell reps also to serve their API on a port that can be configured to accept only TLS connections
• As a Diego operator, I expect the BBS and auctioneer cell clients to prefer communicating to the cells on their TLS-configurable ports if possible
• As a Diego operator, I expect to be able to configure the cell reps to accept only secure communication from remote sources

Documentation

• As an OSS developer, I expect the cloudfoundry/debugserver repository to explain its endpoints via its README
• As a Diego operator, I expect to be able to discover documentation about and examples for the deploying windows cells
• Change developer instructions to use consul 0.7.0

Cleanup

• As a Diego maintainer, I expect not to have the canary job or its dependencies in diego-release

De-Incubation

• Convert cflager package into a subpackage of lager

BOSH job changes

Removed canary job.

BOSH property changes

• Added diego.auctioneer.rep.require_tls: Whether the auctioneer requires communication to the rep via TLS.
• Added diego.auctioneer.rep.ca_cert: CA certificate for communication from the auctioneer to the rep.
• Added diego.auctioneer.rep.client_cert: Client certificate for communication from the auctioneer to the rep.
• Added diego.auctioneer.rep.client_key: Client key for communication from the auctioneer to the rep.
• Added diego.auctioneer.rep.client_session_cache_size: Size of the TLS session cache for the auctioneer to keep for communication to the rep.
• Added diego.bbs.rep.require_tls: Whether the BBS requires communication to the rep via TLS.
• Added diego.bbs.rep.ca_cert: CA certificate for communication from the BBS to the rep.
• Added diego.bbs.rep.client_cert: Client certificate for communication from the BBS to the rep.
• Added diego.bbs.rep.client_key: Client key for communication from the BBS to the rep.
• Added diego.bbs.rep.client_session_cache_size: Size of the TLS session cache for the BBS to keep for communication to the rep.
• Added diego.rep.enable_legacy_api_endpoints: Whether to enable the auction, LRP, and Task endpoints on the legacy rep API server
• Added diego.rep.listen_addr_admin: If legacy endpoints are disabled, the address and port on which to serve the administrative endpoints used to ping and drain the rep.
• Added diego.rep.advertise_domain: Base domain at which the rep should advertise its secure API
• Added diego.rep.listen_addr_securable: Address and port for the TLS-capable server on which the rep serves its workload (Task and LRP) endpoints.
• Added diego.rep.require_tls: Whether to require mutual TLS for communication to the securable rep API server.
• Added diego.rep.ca_cert: CA certificate for the rep to use in its TLS-capable API server.
• Added diego.rep.server_cert: Certificate for the rep to present from its TLS-capable API server.
• Added diego.rep.server_key: Key for the rep to use in its TLS-capable API server.
• Added diego.ssh_proxy.healthcheck_listen_addr: Address and port on which the SSH proxy health-check server listens.
• Removed all diego.canary.* properties.

- name: "diego"
  version: "0.1488.0"
  url: "https://bosh.io/d/github.com/cloudfoundry/diego-release?v=0.1488.0"
  sha1: "aad822353a4753232172896981da1f1f10f45194"

bosh upload-release --sha1 aad822353a4753232172896981da1f1f10f45194 \
  "https://bosh.io/d/github.com/cloudfoundry/diego-release?v=0.1488.0"