Here is an overview of the interactions between the CPI and the Agent on CloudStack as an example:

  • The CPI drives the IaaS and the Agent.
  • The Agent is a versatile process which configures the OS to leverage IaaS-provisioned resources (network interfaces, disks, etc.), and perform other BOSH tasks (job compilation, job instantiation, etc.)
  • The CPI asks the IaaS to instantiate VM template, VMs, volumes and possibly other constructs (floating IPs, security groups, connect LBs, etc.)
  • The Agent is initially driven by the CPI through the bosh-registry, and then by the Director through NATS-based messaging. The registry provides Director-side metadata to the Agent.

The following sequence diagram illustrates in more detail the CPI, agent, and registry interactions, in the case of the CloudStack CPI:


Agent config file formats

This section details the configuration and protocols supported by the Agent.

VM Configuration Locations provides a list of Agent configuration files and their roles.


agent.json file

/var/vcap/bosh/agent.json: Start up settings for the Agent that describe how to find bootstrap settings, and disable certain Agent functionality.

The loading agent.json file is described into config_test.go

The Platform part of the file is documented into LinuxOptions

The Infrastructure part of the file is documented into Options and in particular:

Sample agent.json which configures the agent to read from an HTTP metadata service at a custom URL:

{
  "Platform": {
    "Linux": {
      "CreatePartitionIfNoEphemeralDisk": true,
      "DevicePathResolutionType": "virtio"
    }
  },
  "Infrastructure": {
    "NetworkingType": "static",
    "Settings": {
      "Sources": [
        {
          "Type": "HTTP",
          "URI": "http://10.234.228.142"
        }
      ],
      "UseServerName": true,
      "UseRegistry": true
    }
  }
}

Sample agent.json which configures the agent ot read from a config drive. See reference sample config-drive json. See config drive integration test:

{
  "Platform": {
    "Linux": {
      "UseDefaultTmpDir": true,
      "UsePreformattedPersistentDisk": true,
      "BindMountPersistentDisk": false,
      "DevicePathResolutionType": "virtio"
    }
  },
  "Infrastructure": {
    "Settings": {
      "Sources": [
        {
          "Type": "ConfigDrive",
          "DiskPaths": [
            "/dev/disk/by-label/CONFIG-2",
            "/dev/disk/by-label/config-2"
          ],
          "MetaDataPath": "ec2/latest/meta-data.json",
          "UserDataPath": "ec2/latest/user-data.json"
        }
      ],

      "UseServerName": true,
      "UseRegistry": true
    }
  }
}

Metadata server

The metadata server initially serves the registry URL and DNS server list.

Following is a sample content of the user-data part of the HTTP metadata

{
  "server": {"name": "vm-384sd4-r7re9e"},
  "registry": {"endpoint": "http://192.168.0.255:8080/client/api"},
  "dns": {"nameserver": ["10.234.50.180"]}
}

The supported format of the metadata server by the bosh-agent is documented in UserDataContentsType and http_metadata_service_test.go, along with the expected behavior of the bosh agent when reading this config.


Registry

The registry provides bosh-side metadata to the bosh agent.

From the Warden CPI documentation:

  • The registry is used by the CPI to pass data to the Agent. The registry is started on a server specified by registry properties.
  • If SSH tunnel options are provided, a reverse ssh tunnel is created from the MicroBOSH VM to the registry, making the registry available to the agent on remote machine.

Registry HTTP protocol

The Agent expects to communicate with the bosh registry over a REST API documented in api_controller_spec.rb

Reference registry client and servers implementations are available in:

Registry settings format

The JSON payload of the settings stored in the registry, and its format supported by the Agent are documented in settings_test.go

Sample registry content:

{
  "agent_id": "agent-xxxxxx",
  "blobstore": {
    "provider": "local",
    "options": {
      "endpoint": "http://xx.xx.xx.xx:25250",
      "password": "password",
      "blobstore_path": "/var/vcap/micro_bosh/data/cache",
      "user": "agent"
    }
  },
  "disks": {
    "system": "/dev/xvda",
    "ephemeral": "/dev/sdb",
    "persistent": {}
  },
  "env": {},
  "networks": {
    "default": {
      "type": "manual",
      "ip": "10.234.228.158",
      "netmask": "255.255.255.192",
      "cloud_properties": {"name": "3112 - preprod - back"},
      "dns": [
        "10.234.50.180",
        "10.234.71.124"
      ],
      "gateway": "10.234.228.129",
      "mac": null
    }
  },
  "ntp": [],
  "mbus": "nats://nats:nats-password@yy.yy.yyy:4222",
  "vm": {"name": "vm-yyyy"},
  "trusted_certs": null
}

settings.json file

/var/vcap/bosh/settings.json: Local copy of the bootstrap settings used by the Agent to configure network and system properties for the VM. They are refreshed every time Agent is restarted.

The settings.json payload format is the same as the settings format initially returned by the registry.


Next: Building a Stemcell

Previous: CPI API v1


Contribute changes to this page