The themes for this release are cluster stability, security and operability. - cf-mysql v27 should include MariaDB 10.0.24 and Galera 25.3.14 [#114625673]
Deprecation Warning: This will be the last
cf-mysql-release to use MariaDB version 10.0. Subsequent releases will begin to use MariaDB 10.1, which became generally available late last year. Be aware that there may be compatibility differences between 10.0 and 10.1, so take care to test out changes before upgrading past
- Note: cf-mysql now requires stemcells 3125 or more recent. As a security precaution, we recommend you update stemcells frequently.
- Note: If you’ve enabled syslogging, please see the security notice below regarding root credentials.
Last year, we wrote a fussy story: - As an Operator, I’d like configure an overall timeout which will break down into individual timeouts for each of the MySQL job’s startup phases, so that large SST’s can succeed [#100598304]
DatabaseStartupTimeout into simply
mysql_startup_timeout, and tried to intelligently split that up into sane defaults for each of the stages of database startup.
V27 moves much of this into the BOSH pre-start phase of the job lifecycle. This eliminates many timeout-related failures, although the system may still perform lengthy SSTs during
monit start under certain circumstances.
- Leverage BOSH pre-start process [#100444318], [#114259781], [#117744313], [#117583221], [#117549509]
When restarting, a node re-syncs to the cluster using one of two methods, IST or SST. In the case of SST, it throws away its local data, and adopts the dataset of the cluster. We’ve found rare circumstances when that’s not desirable. This release includes an optional Interruptor, which prevents a node from SST when doing so would delete its own database. We’ve also provided a
rejoin-unsafe errand that automates the process of forcing a node to re-join the cluster after an Operator has determined that it’s safe to do so.
- As an Operator, I do not want a possibly-sick node to re-join the cluster if it means there could be data loss. [#118416629], [#118416751], [#118793289]
Galera comes with a few known limitations. In an effort to try to protect users, and enhance stability, we’ve made the following changes. - Ensure MyISAM experimental replication is disabled [#120629227] - As an Operator, I don’t want a Galera-based cluster to accept XA transactions [#121461877] - As an Operator, I’d like my replication sums to be checked [#121008813]
The Operator, or automation, may to want to block all access to the MariaDB cluster, to enable backups, diagnosis, etc. - As an Operator, I can enable and disable all traffic passing through the proxy [#127033173]
Security & Logging
More users are beginning to deploy cf-mysql into production. We’ve prioritized enabling logs and creating diagnostics that are useful to both Operators and Database Administrators to troubleshoot and audit an active cluster. We’ve also made changes to run more securely, in the event that an unauthorized user gains access to the virtual machines. - Mariadb < 10.0.22 expose credentials in mysql.err.log when performing sst [#122147897]
cf-mysql-release v25 and earlier expose the credentials of the mysql
root account. If you’ve enabled syslog, the root password will have been transmitted outside of the cf-mysql deployment. You must change the root password of your root account after upgrading to this version of
- We’ve changed default syslog priority level from
local1.error to the CAPI recommended standard of
- As an Operator, I’d like to be able to see the stdout/err of mariadb_ctl control script [#118612267]
- As a mysql operator, I would like to feel safe knowing that my mariadb process is not running as a privileged user (mysqld now runs as user
- As an operator, I do not want to see my mysql passwords in
ps aux [#120031195]
- As an Operator, I’d like to be able to learn more about slow queries [#113421765]
- As an Operator, I’d like MySQL binlogs enabled on each Master node [#121089515]
- As an Operator, I’d like to enable additional debug logging to understand replication behavior. [#120996959]
- Log diagnostic cluster data to a CSV file [#118415807], [#121713351], [#127034351]
- As an Operator, I don’t want users to be able to access databases whose names start with ‘test’ on a production deployment so I have confidence that my DB is secure by default [#96828790], [#117195649]
- Bug: cloudfoundry/cf-mysql-release #116: Enabling syslog doesn’t appear to send mysql.err.log to syslog [#127031107]
- Bug: Log SST script output to mysql.err.log [#127664015]
- cloudfoundry/cf-mysql-release #89: Enable audit_plugin [#113966825]
- As an Operator, I’d like the Quota Enforcer to connect as a non-root super user so that we can exclude the QE’s activity from the audit logs [#118502911]
- quota-enforcer user should not need ‘GRANT ALL PRIVILEGES’ [#119029525]
- As an operator, I would like to provide a list of users that the quota enforcer will ignore [#119045503]
- As an Operator, I’d like the Quota Enforcer to run less frequently so that it doesn’t fill up debugging log files. [#127033101]
Enhancements and Bug Fixes
- Update cf-mysql-release to Golang 1.7 [#128205943]
- As an operator, I want to configure the number of open files available to mysqld via
- As an Operator, I’d like to have a property to change the name of the MySQL service in the Marketplace [#115805005]
- Enable innodb_large_prefix [#116578809]
- Length of time GRA files are kept around should be increased [#118606579]
- Bug: As an Operator, I want cf-mysql acceptance tests to work with recent versions of cf-release [#117671103]
- Bug: As an Operator, I’d like to change my database pre-seeds without running the bootstrap procedure [#122647541], [#127938125]
- Bug: Fix an out-of control CPU issue on the service broker VMs.
- As an operator, I do not want monit to try and start mariadb_ctrl multiple times at the same time [#119498227]
Community Contributed Enhancements
- cloudfoundry/cf-mysql-release #87: Resolve package name, job name, and port conflicts [#113088085]
- cloudfoundry/cf-mysql-release #100: Add ability to override number of compilation workers [#117224143]
- Change the name of the deployment as it appears in
bosh deployments: cloudfoundry/cf-mysql-release #101: Add ability to override deployment name [#117225385]
- cloudfoundry/cf-mysql-release #97: Trim release size [#116155371]
- cloudfoundry/cf-mysql-release #110:
./scripts/generate-deployment-manifestnow takes a real cf manifest [#121524303]
- Fix arbitrator docs to get rid of 2-step deploy instructions [#115301809]
- As an Operator, I’d like an example for the api_url property [#115819685]
- Seeded database documentation and spec file example [#115451413]
- Simplified the spec files for the service broker so that service plans are only specified where necessary. The broker errands do not need to know about the entire
serviceshash [#116383627], [#116334151]
- README.md should reference specific spiff version [#123626451]
- As an Operator, I’d like to discover documentation that describes how to recover when a node fails due to SST outside of pre-start. [#128550817]
- Bug: README.md still refers to sample_aws_stub.yml file [#121088767]
- Bug: cf_mysql.mysql.galera_healthcheck.db_password does not exist in example stubs file [#128622041]
- As a [service which embeds cf-mysql], I would like to enable internal clients to communicate consistently with a single switchboard proxy in an HA deployment so that they incur fewer deadlock errors [#118507365]
- Acceptance test errand should have a configurable test password [#127359061]
- We’ve updated the version of the
cfcli for use in smoke tests: Operator can see that the smoke test passes when deploying MySQL v.xyz in an environment where cf-release v.abc is running [#123165451]
Note: Since cf-mysql v23, manifest generation requires
spiff version 1.0.7.
MySQL Job Spec Changes
cf_mysql.mysql.max_open_filesto allow you to configure the maximum number of open files. This defaults to
cf_mysql.mysql.galera_portto allow the port for Galera cluster communication to be customized.
cf_mysql.mysql.disable_auto_sst. When this is
true(the default) then nodes will be prevented from performing a SST.
Audit log support
cf_mysql.mysql.server_audit_eventsto configure the types of events that will be logged. If unspecified then logging is disabled.
cf_mysql.mysql.server_audit_file_rotationsto configure the number of audit file rotations to keep. The default is
cf_mysql.mysql.server_audit_file_rotate_size_in_mbwhich controls the size of each audit log file (default `100).
cf_mysql.mysql.server_audit_excluded_usersto exclude database users from the audit log. The following users will always be excluded:
Cluster health logger
cf_mysql.mysql.cluster_health.passwordto set the cluster health logger database password. This must be specified.
cf_mysql.mysql.cluster_health.log_intervalto control the frequency in seconds with which cluster information is logged. This defaults to every
cf_mysql.mysql.galera_healthcheck.db_passwordto set the database password for the galera-healthcheck user. This must be specified.
cf_mysql.mysql.binlog_enabledwhich enables the binary log if
cf_mysql.mysql.binlog_expire_daysto set the number of days to retain binlogs for. This defaults to
Configuring additional debug logging with these properties may result in user data being logged to disk.
cf_mysql.mysql.wsrep_debug. When set to
2, the database server logs additional debug events for Galera replication. The default is
cf_mysql.mysql.log_conflicts. This defaults to
Proxy Job Spec Changes
- The default port for the Proxy API has changed from
cf_mysql.proxy.consul_enabledto enable experimental support for registering the proxy nodes as a service in Consul.
cf_mysql.proxy.consul_service_nameto set the name of the published service in Consul. Defaults to
CF MySQL Broker Job Spec Changes
cf_mysql.broker.portto configure the port the broker will listen on. This defaults to
8081(previously hard-coded to port 80).
cf_mysql.broker.quota_enforcer.passwordto set the database password for the quota-enforcer user. This must be specified.
cf_mysql.broker.quota_enforcer.ignored_usersto specify an array of database users that the quota enforcer will not enforce quotas for.
cf_mysql.broker.quota_enforcer.pauseto configure how frequently quotas should be enforced in seconds (defaults to
1). This was previously hard-coded to 1 second.
Arbitrator Job Spec Changes
Acceptance Tests Errand Spec Changes
cf_mysql.acceptance_tests.passwordto make the smoke test password configurable. A regression in v26 meant that a configured password was not used when specified.
Bootstrap Errand Spec Changes
cf_mysql.mysql.database_startup_timeout. The bootstrap time is now unbounded to allow for possibly lengthy SST.
Upload this release version to the Director:
$ bosh upload-release https://bosh.io/d/github.com/cloudfoundry/cf-mysql-release?v=27 --sha1 40d210801b41fcf805cde3c47d6c762636fc36e0
Modify deployment manifest to use this release in addition to any other used releases:
releases: - name: cf-mysql version: "27"
Finally add needed deployment jobs and specify values for required properties.
Optionally download sha1: 40d210801b41fcf805cde3c47d6c762636fc36e0 release tarball locally:
# ...or download it directly using curl $ curl -L -J -O https://bosh.io/d/github.com/cloudfoundry/cf-mysql-release?v=27 # or with wget... $ wget --content-disposition https://bosh.io/d/github.com/cloudfoundry/cf-mysql-release?v=27