

Usage

This topic describes cloud properties supported for different resources created by the vSphere CPI. The cloud properties can be specified at the different levels supported in the Cloud Config. Typically, define defaults at the global level and override them at the azs, networks, vm_types vm_extension or disk_types levels.

AZs¶

Schema for cloud_properties section:

datacenters [Array, optional]: Array of datacenters to use for VM placement. Must have only one and it must match datacenter configured in global CPI options.
- name [String, required]: Datacenter name.
- clusters [Array, required]: Array of clusters to use for VM placement.
  - <cluster name> [String, required]: Cluster name.
    - resource_pool [String, optional]: Name of vSphere Resource Pool to use for VM placement.
    - host_group [Dictionary, optional]: Properties of the Host Group to use for VM placement. Available in v52+. (Backwards compatible with old format of specifying Host Group with just a name string of the host group )
      - name [String, required]: Name of the host group in vSphere
      - drs_rule [String, optional]: One of the values from MUST or SHOULD. Is case insensitive. Defaults to SHOULD if not specified or in case of a spelling mistake.
    - drs_rules [Array, optional]: Array of DRS rules applied to constrain VM placement. Must have only one.
      - name [String, required]: Name of a DRS rule that the Director will create.
      - type [String, required]: Type of a DRS rule. Currently only separate_vms is supported.

Example:

azs:
- name: z1
  cloud_properties:
    datacenters:
    - name: my-dc
      clusters:
      - {my-vsphere-cluster: {resource_pool: my-vsphere-res-pool}}

Example with Host Group:

azs:
- name: z1
  cloud_properties:
    datacenters:
    - name: my-dc
      clusters:
      - {my-vsphere-cluster: {host_group: {name: 'my-vsphere-host-group', drs_rule: 'SHOULD'}}}

Networks¶

Schema for cloud_properties section used by manual network subnet:

name [String, required]: Name of the vSphere network. Example: VM Network.

Example of manual network:

networks:
- name: default
  type: manual
  subnets:
  - range: 10.10.0.0/24
    gateway: 10.10.0.1
    cloud_properties:
      name: VM Network

vSphere CPI does not support dynamic and vip networks.

VM Types / VM Extensions¶

Schema for cloud_properties section:

cpu [Integer, required]: Number of CPUs. Example: 1.
ram [Integer, required]: RAM in megabytes. Example: 1024.
disk [Integer, required]: Ephemeral disk size in megabytes. Example: 10240.
cpu_hot_add_enabled [Boolean, optional]: Allows operator to add additional CPU resources while the VM is on. Default: false. Available in v21+.
cpu_reserve_full_mhz [Boolean, optional]: If set true, CPU resource reservation for this virtual machine will always be equal to the max MHz of the ESXi host times the number of CPUs requested for the VM. Default: false. Available in v97.0.15+.
memory_reservation_locked_to_max [Boolean, optional]: If set true, memory resource reservation for this virtual machine will always be equal to the virtual machine's memory size. Default: false. Available in v82+.
memory_hot_add_enabled [Boolean, optional]: Allows operator to add additional memory resources while the VM is on. Default: false. Available in v21+.
upgrade_hw_version [Boolean, optional]: Upgrades the virtual hardware version of a virtual machine to the latest supported version on the ESXi host. Overrides the global upgrade_hw_version. Default: false.
nested_hardware_virtualization [Boolean, optional]: Exposes hardware assisted virtualization to the VM. Default: false.
enable_human_readable_name [Boolean, optional]: Use a name generated by instance group name and deployment name for the VM. Default: false. Available in v53+.
datastores [Array, optional]: Allows operator to specify a list of ephemeral datastores, datastore clusters for the VM. Datastore names are exact datastore names and not regex patterns. At least one of these datastores must be accessible from clusters provided in resource_pools.cloud_properties/azs.cloud_properties or in the global CPI configuration. Available in v23+. Datastore Clusters can be specified as a hash in format of {clusters: [ datastoreCluster1: {}, datastoreCluster2: {}]}. Clusters whose Storage DRS is turned off will be ignored.
datacenters [Array, optional]: Used to override the VM placement specified under azs.cloud_properties. The format is the same as under AZs.
tags [Array, optional]: A list of category and tag name-value pairs used to attach to created VMs. Available in v53+. Available on vCenter 6.5+. The tags to be attached must be exist on vCenter host. For each tag to be attached, both category and tag names should be specified. Note: This tags property is unrelated to the top-level tags block in runtime and deployment configs.
vm_group [String, optional]: Name of VM Group this VM should be part of.
disable_drs [Boolean, Optional]: Disables DRS on this VM type. In short, VM created with this vm_type will NOT be v-motioned by DRS. Available in v53+
nsx [Dictionary, optional]: VMware NSX additions section. Available in CPI v30+ and NSX v6.1+.
- security_groups [Array, optional]: A collection of security group names that the instances should belong to. The CPI will create the security groups if they do not exist. BOSH will also automatically create security groups based on metadata such as deployment name and instance group name. The full list of groups can be seen under create_vm's environment groups. The security groups, if specified under NSX load balancers need not be specified(duplicated) again here. Although CPI makes best effort to de-duplicate them, it is advised not to specify them again.
- lbs [Array, optional]: A collection of NSX Edge Load Balancers (LBs) to which instances should be attached. The LB and Server Pool must exist prior to the deployment.
  - edge_name [String, required]: Name of the NSX Edge.
  - pool_name [String, required]: Name of the Edge's Server Pool.
  - security_group [String, required]: Name of the Pool's target Security Group. The CPI will add the VM to the specified security group (creating the security group if needed), then add the security group to the specified Server Pool.
  - port [Integer, required]: The port that the VM's service is listening on (e.g. 80 for HTTP).
  - monitor_port [Integer, optional]: The healthcheck port that the VM is listening on. Defaults to the value of port.
vmx_options [Dictionary, optional]: Allows operator to specify VM advanced configuration options. All values are subject to YAML's type interpretation, and given that for certain configuration options vSphere will accept only a specific value type please take note of the difference between values with similar appearances such as: true vs "true" and "1234" vs 1234. Refer to the vSphere documentation for more information about what configuration options are accepted. Available in v42+.
storage_policy [Dictionary, optional]: Storage Policy which is applied to a VM and its ephemeral disk. Available in v53+
- name [String, optional]: Name of the storage policy to be applied to the VM and its ephemeral disk. Available in v53+
nsxt [Dictionary, optional]: VMware NSX additions section. Available in CPI v45+.
- ns_groups [Array, optional]: A collection of NS Groups names that the instances should belong to. Available in NSX-T v1.1+.
- vif_type [String, optional]: Supported types: PARENT, null. Overrides the global default_vif_type. Available in NSX-T v2.0+.
- lb [Dictionary, optional]: NSX-T logical Load Balancer. Available in CPI v48+
  - server_pools [Array, optional] Server Pool must exist prior to the deployment. For static server pool, VM is directly added to the server pool. If server pool is dynamic, CPI looks up the NSGroup and adds the VM to the NSGroup.
    - name [String, required]: Name of the Server Pool
    - port [Integer, optional]: The port that the VM's service is listening on (e.g. 80 for HTTP). If port is specified, all connections will be sent to this port on the VM. Only specify a single port (no ranges). If unset, the load balancer will connect the client to the VM using the same port number (e.g. if the client connects to port 443, the load balancer will forward to the VM on port 443).
- tag_nsx_vm_objects [Boolean, optional]: When enabled, tag NSX VM objects with the same set of tags as vsphere VM objects. Available in CPI v97.0.14+.
pci_passthroughs [Array, optional]: Specifies a PCI (Peripheral Component Interconnect) device to attach to VM via vSphere Dynamic DirectPath IO. Requires vSphere 7.0+. Automatically sets the properties memory_reservation_locked_to_max and upgrade_hw_version to true. Each entry requires the PCI card's device_id and vendor_id. Available in v97+.
vgpus [Array, optional]: Specifies an Nvidia GRID vGPU to attach to VM. Automatically sets the properties memory_reservation_locked_to_max and upgrade_hw_version to true. Available in v97+.
root_disk_size_gb [Integer, optional]: Expands the root disk to the size specified (in GiB). Useful when files such as Nvidia CUDA libraries can only be placed on the system disk instead of the ephemeral disk. Note that setting this property uses slightly more datastore space than expected, for non-expanded system disks are vSphere linked clones, which means the datastore footprint is close to zero. Available in v97.0.6+.

Example of a VM asked to be placed into a specific vSphere resource pool with NSX-V and NSX-T integration:

resource_pools:
- name: nsx
  network: default
  stemcell:
    name: bosh-vsphere-esxi-ubuntu-xenial-go_agent
    version: latest
  cloud_properties:
    cpu: 1
    ram: 1_024
    disk: 10_240
    datastores: [prod-ds-1, prod-ds-2, {clusters: [vcpi-sp1: {}  , vcpi-sp2: {}]}]
    datacenters:
    - name: my-dc
      clusters:
      - my-vsphere-cluster: {resource_pool: other-vsphere-res-pool}
    vm_group: cpi-vm-group
    nsx: # NSX-V configuration
      security_groups: [public, dmz]
      lbs:
      - edge_name: my-lb
        pool_name: https-pool
        security_group: https-sg
        port: 443
        monitor_port: 4443 # optional, defaults to `port` value
    storage_policy:
      name: 'vcpi-os-linux'
    vmx_options:
      sched.mem.maxmemctl: "1330"
      disk.enableUUID: "1"
      cpuid.coresPerSocket: "2"
    nsxt: # NSX-T configuration
      ns_groups: [public, dmz]
      vif_type: PARENT
      lb:
        server_pools:
        - name: cpi-pool-1
          port: 80

Example of attaching three tags (from two categories) to created VMs:

vm_types:
- cloud_properties:
    cpu: 2
    datastores: [isc-cl1-ds-0, isc-cl1-ds-1, isc-cl2-ds-0, nfs0-1]
    disk: 30240
    ram: 2048
    tags:
    - category: category-name-a
      tag: tag-name-a-1
    - category: category-name-b
      tag: tag-name-b-1
    - category: category-name-b
      tag: tag-name-b-2
  name: vm_type_name

Example of overriding the default target datastore used by ephemeral storage through a vm_extension specified in the deployment manifest:

vm_extensions:
  - name: ephemeral-to-ds2
    cloud_properties:
      datastores:
        - 'prod-ds-2'
  ```

Example of attaching a PCI card to a VM:

```yaml
vm_extensions:
- name: gpu
  cloud_properties:
    pci_passthroughs:
    - vendor_id: 0x10de # Nvidia
      device_id: 0x1eb8 # Tesla T4

Example of attaching a Virtual GPU to a VM:

vm_extensions:
- name: gpu
  cloud_properties:
    vgpus:
    - grid_t4-16q

Example of expanding the root disk of a VM to 20 GB:

vm_extensions:
- name: 20G_root
  cloud_properties:
    root_disk_size_gb: 20

Disk Types¶

Schema for cloud_properties section:

type [String, optional]: Virtual disk type used for persistent disks: thick, thin, preallocated, eagerZeroedThick. Defaults to preallocated. Available in v12. Overrides the global default_disk_type.
datastores [Array, optional]: List of datastore names, datastore clusters for storing persistent disks. Overrides the global persistent_datastore_pattern. These names are exact datastore names and not regex patterns. Available in v29+. Datastore Clusters can be specified as a hash in format of {clusters: [ datastoreCluster1: {}, datastoreCluster2: {}]}. Clusters whose Storage DRS is turned off will be ignored.

Example of 10GB disk:

- name: default
  disk_size: 10_240

Example of disk with type eagerZeroedThick:

- name: default
  disk_size: 10_240
  cloud_properties:
    type: eagerZeroedThick

Example of persistent disk stored in specific datastores:

- name: default
  disk_size: 10_240
  cloud_properties:
    datastores:
      # datastores to be used as candidates in the vm placement algorithm (see #vm-placement)
      - 'prod-ds-1'
      - 'prod-ds-2'
      - clusters: # storage clusters used if SRDS is turned on
          - vcpi-sp1: {}
          - vcpi-sp2: {}

Global Configuration¶

The CPI can only talk to a single vCenter installation and manage VMs within a single vSphere datacenter.

Schema:

host [String, required]: IP address or hostname of vCenter. Example: 172.16.68.3.
user [String, required]: Username for the API access. Example: root.
password [String, required]: Password for the API access. Example: vmware
connection_options [Object, optional]: Additional connection options
- ca_cert [String, optional]: A list of concatenated CA certificates used to verify the TLS connection to the vCenter server. If no value is provided, the CPI will establish a TLS connection but will not verify the certificate presented by the server.
http_logging [Boolean, optional]: Enables logging all HTTP requests and responses to vSphere API. Default: false. Available in v37+.
default_disk_type [String, optional]: Sets the default disk type. Can be either thin or preallocated, defaults to preallocated. preallocated sets "all space allocated at [VM] creation time and the space is zeroed on demand as the space is used", and thin, "virtual disk is allocated and zeroed on demand as the space is used." Applies to both ephemeral and persistent disks.
ensure_no_ip_conflicts [Boolean, optional]: When creating a VM, ensure that no other VMs exist in the same port group with the same IP address. The CPI queries the vCenter to detect conflict, does not use ping. Default: true. Available in v97.0.5+.
datacenters [Array, optional]: Array of datacenters to use for VM placement. Must have only one.
- name [String, required]: Datacenter name.
- vm_folder [String, optional]: Path to a folder (relative to the datacenter) for storing created VMs. Folder will be automatically created if not found. Defaults to BOSH_VMs.
- template_folder [String, optional]: Path to a folder (relative to the datacenter) for storing uploaded stemcells. Folder will be automatically created if not found. Defaults to BOSH_Templates.
- disk_path [String, optional]: Path to a disk folder for storing persistent disks. Folder will be automatically created in the datastore if not found. Defaults to BOSH_Disks.
- datastore_pattern [String, required if datastore_cluster_pattern is not set or CPI version <= v63]: Pattern for selecting datastores for storing ephemeral disks and stemcells.
- datastore_cluster_pattern [String, required if datastore_pattern is not set]: Pattern for selecting datastore clusters for storing ephemeral disks. Clusters whose Storage DRS is turned off will be ignored. As of v94, datastore clusters in folders can be referenced using the full path with slashes as the delimiter.
- persistent_datastore_pattern [String, required if persistent_datastore_cluster_pattern is not set]: Pattern for selecting datastores for storing persistent disks.
- persistent_datastore_cluster_pattern [String, required if persistent_datastore_pattern is not set]: Pattern for selecting datastore clusters for storing persistent disks. Clusters whose Storage DRS is turned off will be ignored. As of v94, datastore clusters in folders can be referenced using the full path with slashes as the delimiter.
- clusters [Array, required]: Array of clusters to use for VM placement.
  - <cluster name> [String, required]: Cluster name.
    - resource_pool [String, optional]: Specific vSphere resource pool to use within the cluster.
    - host_group [String, optional]: Specific Host Group to use for VM placement. Available in v52+.
nsx [Dictionary, optional]: NSX-V configuration options. This is required if the other NSX features are used below (e.g. 'security_groups' for resource_pools).
- address [String, required]: The NSX server's address. Can be a hostname (e.g. nsx-server.example.com) or an IP address.
- user [String, required]: The login username for the NSX server.
- password [String, required]: The login password for the NSX server.
- ca_cert [String, optional]: A CA certificate that can authenticate the NSX server certificate. Required if the NSX Manager has a self-signed SSL certificate. Must be in PEM format.
enable_auto_anti_affinity_drs_rules [Boolean, optional]: Creates DRS rule to place VMs on separate hosts. DRS Automation Level must be set to "Fully Automated"; does not work when DRS is set to "Partially Automated" or "Manual". May cause VMs to fail to power on if there are more VMs than hosts after initial deployment. Default: false. Available in v33+.
vm_storage_policy_name [Boolean, optional]: Name of the storage Policy which is applied to a VM and its ephemeral disk. Availablein v53+
upgrade_hw_version [Boolean, optional]: Upgrades the virtual hardware version of a virtual machine to the latest supported version on the ESXi host. Default: false.
nsxt [Dictionary, optional]: NSX-T configuration options. Available in v45+.
- use_policy_api [Boolean, optional]: Enabling this feature will use the NSX-T Policy API instead of the Manager API. It affects the attachment of VMs to NS Groups and VM placement in static Load Balancer Pools. This feature requires NSX-T Data Center v3.0 or later. Default: false. Available in v56+. In v58+, the VM's NSX-T segment ports are also tagged. The tags are prepended with bosh/ and include the key/value pairs specified in the tags blocks of the deployment or runtime configurations as well as BOSH default metadata (instance_group, job, index, etc.)
- policy_api_migration_mode [Boolean, optional] This option requires use_policy_api to be set to true. When enabled, the CPI attempts to associate VMs in both the Policy API and the Manager API. The VM is associated with groups and server pools in the Policy API, and with NSGroups and server pools in the Manager API. It will return an error if the Manager API objects do not exist, but not if the Policy API objects do not exist. This option is only intended to be used in conjunction with scripts to help migrate NSX-T entities from the Manager API to the Policy API. Default: false. Available in v74+.
- host [String, required]: The NSX-T server's address. Can be a hostname (e.g. nsx-server.example.com) or an IP address.
- username [String, required]: The login username for the NSX-T server.
- password [String, required]: The login password for the NSX-T server.
- ca_cert [String, optional]: A CA certificate that can authenticate the NSX-T server certificate. Required if the NSX-T Manager has a self-signed SSL certificate. Must be in PEM format.
- default_vif_type [String, optional]: Supported Types: PARENT. Default VIF type attached to logical port. Available in NSX-T v2.0+.
- auth_certificate [String, optional]: Certificate used for certificate-based authentication. Certificate-based authentication takes precedence over username/password if both are specified. Available in v51+.
- auth_private_key [String, optional]: Private key file used for certificate-based authentication. Available in v51+.
- remote_auth [Boolean, optional]: Enables remote authentication for NSX-T via vIDM. Available in v52.1.5+ and v53.0.1+
- allow_overwrite [Boolean, optional]: When enabled, the CPI sets the X-Allow-Overwrite header to 'true' when making NSX-T Management API requests, which allows the Management API to mutate Policy API objects. Default: true for backwards compatibility. Available in v91+.

Note

If the NSX-V or NSX-T Manager has a self-signed certificate, the certificate must be set in the ca_cert property.

Example properties that will place VMs into BOSH_CL cluster within BOSH_DC:

host: 172.16.68.3
user: root
password: vmware
datacenters:
- name: BOSH_DC
  vm_folder: prod-vms
  template_folder: prod-templates
  disk_path: prod-disks
  datastore_pattern: '^prod-ds$'
  persistent_datastore_pattern: '^prod-ds$'
  clusters: [BOSH_CL]

Example that places VMs by default into BOSH_RP vSphere resource pool with NSX integration and enables VM anti-affinity DRS rule:

host: 172.16.68.3
user: root
password: vmware
default_disk_type: thin
enable_auto_anti_affinity_drs_rules: true
vm_storage_policy_name: 'vcpi-mac-policy'
datacenters:
- name: BOSH_DC
  vm_folder: prod-vms
  template_folder: prod-templates
  disk_path: prod-disks
  datastore_pattern: '\Aprod-ds\z'
  persistent_datastore_pattern: '\Aprod-ds\z'
  clusters:
  - BOSH_CL: {resource_pool: BOSH_RP}
nsx:
  address: 172.16.68.4
  user: [email protected]
  password: vmware

Example that places VMs by default into CLUSTER_HOST_GROUP vSphere host group:

host: 172.16.68.3
user: root
password: vmware
default_disk_type: thin
datacenters:
- name: BOSH_DC
  vm_folder: prod-vms
  template_folder: prod-templates
  disk_path: prod-disks
  datastore_pattern: '\Aprod-ds\z'
  persistent_datastore_pattern: '\Aprod-ds\z'
  clusters:
  - BOSH_CL: {host_group: CLUSTER_HOST_GROUP}
nsx:
  address: 172.16.68.4
  user: [email protected]
  password: vmware

Example Cloud Config¶

azs:
- name: z1
  cloud_properties:
    datacenters:
    - clusters: [z1: {}]
- name: z2
  cloud_properties:
    datacenters:
    - clusters: [z2: {}]

vm_types:
- name: default
  cloud_properties:
    cpu: 2
    ram: 1024
    disk: 3240
- name: large
  cloud_properties:
    cpu: 2
    ram: 4096
    disk: 30_240

disk_types:
- name: default
  disk_size: 3000
- name: large
  disk_size: 50_000

networks:
- name: default
  type: manual
  subnets:
  - range: 10.10.0.0/24
    gateway: 10.10.0.1
    az: z1
    dns: [8.8.8.8]
    cloud_properties:
      name: vm-net1
  - range: 10.10.1.0/24
    gateway: 10.10.1.1
    az: z1
    dns: [8.8.8.8]
    cloud_properties:
      name: vm-net2

compilation:
  workers: 5
  reuse_compilation_vms: true
  az: z1
  vm_type: large
  network: default

Notes¶

Assigned VM names (e.g. vm-8dg349-s7cn74-...) should not be manually changed since the CPI uses them to find created VMs. You can use bosh vms --details to find which VM is assigned which job. VMs are also tagged with their assigned job, index and deployment.
Storage DRS and vMotion can be used with vSphere CPI version v18 and above. For additional details see Storage DRS and vMotion Support.
allow_mixed_datastores configuration has been deprecated in favor of setting same datastore pattern for datastore_pattern and persistent_datastore_pattern keys.
The vSphere CPI requires access to port 80/443 for all the ESXi hosts in your vSphere resource pool(s). In order to upload stemcells to vSphere, the vSphere CPI makes use of an API call that returns a URL that the CPI should make a POST request to in order to upload the stemcell. This URL could have a hostname that resolves to any one of the ESXi hosts that are associated with your vSphere resource pool(s).
Setting enable_auto_anti_affinity_drs_rules to true may cause bosh deploy to fail after the initial deployment if there are more VMs than hosts. A workaround is to set enable_auto_anti_affinity_drs_rules to false to perform subsequent deployments.
Support for specifying Datastore Clusters for ephemeral and persistent disks is available with vSphere CPI version v47 and above. For additional detais see Release Notes for v47
Support for specifying Datastore Clusters nested under folders is available with vSphere CPI version v94 and above. For additional detais see Release Notes for v94

VMs¶

VMs have randomly generated cloud identifiers, in the format "vm-#{SecureRandom.uuid}". They are stored on a datacenter as follows:

datacenter.name/vm/vm_folder.name/vm_cid

e.g.

TEST_DATACENTER/vm/58bc710b-aec7-41f6-bb78-7d65f8033e51/vm-f050dbdb-ddcf-4524-b6d8-fad1135c6f7e

Your datacenter will be queried for the path above to try and find a matching VM.

There is nothing in the CPI that prevents using an id generated by the cloud for the cloud id. Indeed, the AWS CPI uses the instance ids generated on AWS for cloud ids. The create_vm() CPI call returns the cid of the created VM, so it's up to the CPI implementor to decide what to use for cloud id.

Although it's technically possible to use the instanceUuid on vSphere (much like how we use AWS instance ids), it's worth noting that this breaks backwards compatibility and would require a fairly hefty migration. This would open up the possibility of allowing an operator to move a VM out of its containing folder on a datacenter, as it would be possible to identify a VM independent of its inventory location.

Networks¶

Networks are uniquely identified by datacenter and network name (which must be unique within the datacenter).

Datastores¶

Datastores are identified by their name and are matched by a regular expression that matches against that name. For example, consider the following datastores in folders:

/foo/bar/datastore1
/foo/bar/anothername
/foo/baz/datastore1

The name of datastore is the last part of each line above, e.g. 'datastore1'. An operator may choose to select both /foo/bar/datastore1 and /foo/baz/datastore1 with the regular expression 'datastore' or even 'datastore1'. They cannot, however, select both /foo/bar/datastore1 and /foo/bar/anothername by using a regular expression that matches against directory structure, e.g. /foo/bar.

Ephemeral and persistent datastores are consumed before shared datastores.

Datastore Paths¶

Persistent disks are stored on datastores in the following paths:

/<datacenter disk path from manifest>/disk-<random disk uuid>.vmdk

Linked Clones¶

The vSphere CPI uses linked clones by default. Linked clones require the clone to be on the same datastore as the source so stemcells are automatically copied to each datastore that their clones will be on. These stemcells look like sc-<uuid> / <datastore managed object id> in the inventory. In the datastore browser the "/" will be quoted to "%2f".

Clusters¶

Each datacenter can have multiple clusters in the cloud properties.

A cluster is identified by its name and its datacenter. Its location within folders in each datacenter does not matter.

In vSphere, cluster names do not need to be unique per datacenter, only their paths needs to be unique. The current vSphere CPI code does not handle this and would only see one cluster if two had the same name.

Clusters do not have any unique ID like a VM's instanceUuid.

Host Group¶

Each cluster definition can have one host group.

For any VM created on this cluster with host group specification , vSphere CPI

Adds the VM to a new vm group (name auto generated from specified host group name)
Creates a VM-Host Affinity rule (name auto generated) to always keep VM on the specified host group.
Deletes the vSphere-CPI generated VM Host Affinity rule and VM Group if no other VM in those rule and groups are remaining.

VM Placement¶

When placing a VM, a cluster is chosen (along with a datastore) based on the VM's memory, ephemeral disk, and persistent disk requirements.

VMs are placed on clusters and datastores based on a weighted random algorithm. The weights are calculated by how many times the requested memory, ephemeral and persistent disk could fit on the cluster.

This means the VMs will be placed according to the following rules.

Clusters are filtered such that the placement only considers clusters that have the available memory and datastore space to host the VM.
If the VM has an existing persistent disk it will be placed on a cluster that has access to that disk.
If the VM only has ephemeral disks or a new persistent disk then BOSH will place the VM on the cluster with access to the most datastore space. The datastore selected based on the weighted random algorithm.
If the VM creation fails on the selected cluster then BOSH will attempt to place the VM on the next cluster that meets the resource requirements of the VM. If the VM cannot be created on any of the clusters then the VM deployment will fail.

During VM placement local datastores and shared datastores are not treated differently. All datastores registered on a cluster are treated the same.

Locality¶

When recreating an existing VM, the CPI tries to create it in a cluster and datastore that are near the largest of its existing persistent disks.

Datacenters¶

The vSphere CPI only supports a single datacenter and errors if more than one is defined in the manifest. It is identified by name.

The current code will not work with a datacenter inside a folder.

Storage Policy¶

The order of precedence for policy and datastore specified for selecting ephemeral datastores is (in order they are written from top to bottom, first being most preferred): - Storage policy is set in vm-type - Datastores in vm-type - Storage policy in Global config - Datastore pattern in global config