cf-mysql-release v34

Hey! We’re thinking of dropping spiff manifest templates in favor of using BOSH’s native features. - Is this OK? Click: :green_heart: - Is disaster? Click: :broken_heart:

To see more, check out our work in progress, cf-mysql-deployment!

Important Changes

• Retroactively remove lock permissions from existing service-broker-created users [#132881499]
  

One major limitation of Galera is that table-level locks are not replicated. We’ve found that many applications which rely on table locking never notice this limitation when moving onto cf-mysql. In order to fail fast, starting in v32, new service bindings are explicitly disallowed from locking tables. Starting with v34, all existing service bindings will no longer have the ability to lock tables. Apps that attempt to lock tables will now see an error of the form:

    MariaDB [cf_eedd5768_9c6c_4388_ae0b_dc64f4022bf4]> LOCK TABLES fruit WRITE;
    ERROR 1044 (42000): Access denied for user 'uoY64cqdw6qyMtNl'@'%' to database 'cf_eedd5768_9c6c_4388_ae0b_dc64f4022bf4'

• Rename broker-deregistrar to deregister-and-purge-instances [#138006305]
  

Too often, Operators have been burned trying out an innocuously-named errand. Renaming the errand, deregister-and-purge-instances makes it clear that this is an errand that should only be run just before bosh delete-deployment.

Dependency Upgrades

cf-mysql relies on a variety of OSS packages. Part of using cf-mysql is that you always get the latest bits! - cf-mysql-release should use MariaDB 10.1.20 [#133906749] - Golang 1.7 upgrade [#133294695] - Bump cf CLI to v6.22.2 or greater [#133940783] - Upgrade xtrabackup [#117744831]

Tuning and Configuration Improvements

• job spec for previous_admin_username should state that it is optional [#133758449]
• Allow monit startup timeout for mariadb_ctl job to be configurable [#135742055]
  • Remove unused database_startup_timeout property from mysql job spec [#133138729]
• As an App Developer, I’d like an exception if trying to store invalid data in the database [#137836651]
• As a Developer, I don’t want DDLs and DMLs to be limited by the number of rows they affect [#137044863]
• [BUG] MySQL jobs fail to start if the mysql port is not 3306 [#136590189]
• [BUG] allow cf-mysql.mysql.binlog_enabled and cf-mysql.mysql.innodb_buffer_pool_instances to be overwritten from property-overrides stub [#133969391]

Switchboard Changes

Similar to CF’s HTTP router, Switchboard can now account for the healthchecks of an upstream load balancer. - Switchboard delays shutdown till external Load Balancer fails over [#137933317] - Switchboard delays startup for a period of time so that a LB can notice and register that it is alive [#137933307] - As a CF-MySQL operator, I expect the switchboard proxies to no longer claim a lock in consul when in consul-enabled mode [#135490687]

Bug Fixes

• [BUG] bootstrap errand should use the same property name for galera_healthcheck as other jobs [#136389887]
• [BUG] Interruptor scaretext directs Operator to do the wrong thing [#137205271]
• [BUG] cf-mysql-broker should always stop when asked to stop [#136631833]
• [BUG] mariadb_ctrl fails to start if mysql takes longer than 60 seconds to start [#137485517]
• [BUG] Link error for rejoin-unsafe errand if no arbitrator job [#138841321]
• [BUG] mariadb control should start if there’s no prestart marker [#138540989]

Miscellaneous Other Improvements

• Reduce the chance of the mysql stalling due to disk filling up from binary logs. [#133888125]
• broker should log when it fails to run the initial mysql script [#133232467]

BOSH 2.0-ification

As mentioned in the cf-mysql-release v32 release notes, we’re gradually improving in our ability to leverage BOSH 2.0 semantics. This work is now being centralized in cf-mysql-deployment. - Ability to remove broker instances (BOSH 2) [#133894727] - Ability to enable cert verification (BOSH 2) [#133894695] - [BUG] Removing broker instances with overrides file should also remove broker related errands [#134320837] - bosh 2 template should default to not requiring CF [#136082119] - cloudfoundry/cf-mysql-release #141: simplify bosh template [#136469529] - As an Operator, I’d like a sample bosh 2.0 override file to help understand how to assign static IPs to the proxy instances [#136539935] - As an Operator, I’d like instructions that help me translate a spiff generated manifest to a BOSH 2.0 manifest [#136290983]

Documentation Updates

• cloudfoundry/cf-mysql-release #139: Update README.md [#135583739]
• [BUG] slow query log is undocumented [#138053415]
• [BUG] Security Groups documentation mistakenly deleted [#138685491]
• Update cf-stub [#137069195] You haven’t needed to use our cf-stub.yml file in some time, so we’ve removed it. Just use a real cf-release deployment manifest.
• Explore if Galera 25.3.19 interferes with bootstrap errand or makes the interruptor irrelevant [#134379769] Galera now includes its own version of the Interruptor. It takes care of some additional corner cases (such as boostrapping from the wrong node), but doesn’t conflict with our own Interruptor. We allow the to coexist. You can disable our Interruptor if it’s too disruptive.

Community Contributions

As an OSS project, we welcome Pull Requests. Sometimes, those pull requests are not directly related to our product direction. So long as they are additive, and don’t introduce conflicts, we’re happy to include them. Including these features in the release does not imply that we routinely test this functionality - these features are considered, “community contributed.” If there are issues with these features, we’re happy to consider additional PRs from the community! - cloudfoundry/cf-mysql-broker #19: Allow any 2.3+ ruby [#136116997] - cloudfoundry/cf-mysql-release #103: Allow the main mysql host to be specified via config. [#118524849] - cloudfoundry/cf-mysql-release #137: smoke_tests should specify domain when pushing cf app [#135204167] - cloudfoundry/cf-mysql-release #108: Default cf_mysql.host to nil [#121247455] - cloudfoundry/cf-mysql-release #142: enable event scheduler [#137246355] - cloudfoundry/cf-mysql-release #144: adding properties override in cf-mysql-template.yml [#137528165] - cloudfoundry/cf-mysql-release #145: Update comments for kill_and_wait() [#138063011] - cloudfoundry/cf-mysql-release #146: Fix quoting in check_mount() [#138063035]

Manifest Changes

For those who manually update your manifests, please complete the survey above by clicking a heart! - cf_mysql.mysql.healthcheck_port has been renamed to cf_mysql.mysql.galera_healthcheck_port - cf_mysql.mysql.galera_healthcheck.endpoint_username, now defaults to “galera-healthcheck” - cf_mysql.broker.auth_username now has a default of admin - cf_mysql.mysql.wsrep_max_ws_rows now defaults to 0, rather than 128K - cf_mysql.mysql.binlog_expire_days defaults to 7 days, not 60 - New: cf_mysql.mysql.advertise_host, optional - New: cf_mysql.mysql.startup_timeout, replaces cf_mysql.mysql.database_startup_timeout Now defaults to 60 seconds (this isn’t 100% accurate as they are used in slightly different places resulting in a better experience when it takes a while to start mysql e.g. continual IST) - New: cf_mysql.mysql.event_scheduler, defaults to “off”

- name: "cf-mysql"
  version: "34"
  url: "https://bosh.io/d/github.com/cloudfoundry/cf-mysql-release?v=34"
  sha1: "87bc9e3fc19030c6feda72e96dd64fc0a2e5d059"

bosh upload-release --sha1 87bc9e3fc19030c6feda72e96dd64fc0a2e5d059 \
  "https://bosh.io/d/github.com/cloudfoundry/cf-mysql-release?v=34"