Skip to content

Configuration Format

Example monit Configuration

bpm still sits on top of monit as part of the current BOSH job API. However, the contents of the monit file become simpler and less variable. BOSH would like to remove support for monit eventually and so reducing the exposed feature area will make this easier.

check process server
  with pidfile /var/vcap/sys/run/bpm/server/server.pid
  start program "/var/vcap/jobs/bpm/bin/bpm start server"
  stop program "/var/vcap/jobs/bpm/bin/bpm stop server"
  group vcap

check process worker
  with pidfile /var/vcap/sys/run/bpm/server/worker.pid
  start program "/var/vcap/jobs/bpm/bin/bpm start server -p worker"
  stop program "/var/vcap/jobs/bpm/bin/bpm stop server -p worker"
  group vcap

Job Configuration

Your job configuration must be in a file called bpm.yml in the config directory of your job.

Schema

Property Type Required? Description
processes process[] Yes A top-level listing of all of the processes in your job.

process Schema

Property Type Required? Description
name string Yes The name of this process.
executable string Yes The path to the executable file for this process.
args string[] No The arguments which will be passed to the executable of this process.
env string => string No Any additional environment variables to be included in the environment of this process.
workdir string No The working directory for this process. If not specified this is the value /var/vcap/jobs/JOB.
hooks hooks No The hook configuration for this process (see below).
capabilities string[] No The list of capabilities (without CAP_) which should be granted to this process.
limits limits No The limit configuration for this process (see below).
ephemeral_disk boolean No Whether or not an ephemeral disk should be mounted into the container at /var/vcap/data/JOB.
persistent_disk boolean No Whether or not an persistent disk should be mounted into the container at /var/vcap/store/JOB.
additional_volumes volume[] No A list of additional volumes to mount inside this process. The paths which can be used are restricted (see volume note below).
unsafe unsafe No The unsafe configuration for this process (see below).
shutdown_signal string No The first signal to send to the process when trying to shut it down. Can be either TERM or INT. Defaults to TERM.

hooks Schema

Property Type Required Description
pre_start string No The path to an executable to run before starting the main executable of this process. Should not exceed 30 seconds

limits Schema

Property Type Required Description
memory string No The memory limit to apply to this process. It is formatted as a number and then a single character for units e.g. 1G, 256M.
open_files int No The number of files this process is allowed to have open at any one time.
processes int No The number of processes which this process is allowed to have running at any one moment (inclusive of the main process).

unsafe Schema

Property Type Required Description
privileged boolean No Whether or not this process should execute with increased privileges (see details below).
unrestricted_volumes volume[] No An unrestricted list of additional volumes to mount inside this process (see below).
host_pid_namespace boolean No Use the host's PID namespace inside the container.

volume Schema

Property Type Required Description
path string Yes The absolute path of the volume inside this process.
writable boolean No Whether or not this volume is writable by the process.
allow_executions boolean No Whether or not executable files can be executed from this volume.
mount_only boolean No Whether or not BPM should just mount this directory rather than creating and chowning a backing directory too.
shared boolean No Whether or not BPM should share the mount (internal mountpoints are visible in all namespaces). Not usable in unsafe yet.

*Note: The volumes in additional volumes must have a path inside /var/vcap. If you need to mount a volume outside these paths then you must use the unrestricted_volumes key.

The unrestricted_volumes stanza can include globs in the path attribute. These globs will be evaluated by BPM on startup and each glob match will be created as a new volume with the options specified. Please take care when using this feature not to have your glob match too many different paths. Each mount carries some overhead and it's possible to write a glob which could feasibly recursively mount every single path in a directory as a different mount. We're likely to add some reduction in the future in order to get a minimal equivalent set of mounts but for now the mounting is very naive.

This feature was added to allow jobs which pulled configuration from all jobs on the system to be able to do so without mounting the entire jobs directory (which contains most if not all credentials on the system).

# bad (will match thousands of things, just mount the directory)
unsafe:
  unrestricted_volumes:
  - path: /var/vcap/data/thing/**/*

# good (specific, needed to get around the unknown job paths)
unsafe:
  unrestricted_volumes:
  - path: /var/vcap/jobs/*/config/indicators.yml

Example

# /var/vcap/jobs/server/config/bpm.yml
processes:
- name: server
  executable: /var/vcap/data/packages/server/serve.sh
  args:
  - --port
  - 2424

  env:
    FOO: BAR

  limits:
    processes: 10

  ephemeral_disk: true

  additional_volumes:
  - path: /var/vcap/data/sockets
    writable: true

  capabilities:
  - NET_BIND_SERVICE

- name: worker
  executable: /var/vcap/data/packages/worker/work.sh
  args:
  - --queues
  - 4

  additional_volumes:
  - path: /var/vcap/data/sockets
    writable: true

  hooks:
    pre_start: /var/vcap/jobs/server/bin/worker-setup

Setting Sysctl Kernel Parameters

We recommend setting these parameters in your BOSH pre-start with the following command:

sysctl -e -w net.ipv4.tcp_fin_timeout 10
sysctl -e -w net.ipv4.tcp_tw_reuse 1

You could set these in your bpm pre_start but since these affect the entire host and not just the contained job we like to keep them separate.

Passing Configuration at Runtime

You are also able to pass volumes to mount and environment variables into the process when using bpm run. This is useful to mount volumes and pass configuration which you don't know about until runtime. The syntax for this is:

bpm run -v /var/vcap/data/database:writable,allow_executions -v ... [...]
bpm run -e KEY=value -e ... [...]

Note: The environment variable flag should not be used for secret values as these strings will appear in the process table.

The both flags can be specified multiple times. The volume flag can use the writable, mount_only, or allow_executions options.

The same validations and limitations which apply to the file-based configuration also apply here.

Hooks

Your startup hook must finish with time to spare before the monit start timeout (30s by default). We're looking into ways to make this less vague.

Privileged Jobs

Processes can be marked as privileged by setting the unsafe: {privileged:true} attribute in their configuration. Jobs should almost never use this configuration option as it was only added for jobs which truly need to run as a superuser such as Garden.

Running a privileged job removes some of the safeguards which surround a bpm process. The full list of the effects is as follows:

  • runs as user root and group root
  • grants a larger list of privileges (taken from docker's privileged list)
  • allows new privileges to be gained
  • removes seccomp limitations
  • removes masked and readonly paths (still applies to volumes and /var/vcap/{data,store})
  • all mounts have their nosuid option removed