Skip to content

Working with Blobs


Examples use CLI v2.

A package may need to reference blobs (binary large objects) in addition to referencing other source files. For example when building a package for PostgreSQL server you may want to include postgresql-9.6.1.tar.gz from Typically it's not recommended to check in blobs directly into a Git repository because Git cannot efficiently track changes to such files. CLI provides a way to manage blobs in a reasonable manner with several commands:

bosh -h|grep blob
  add-blob               Add blob
  blobs                  List blobs
  remove-blob            Remove blob
  sync-blobs             Sync blobs
  upload-blobs           Upload blobs

Adding a blob

Package can reference blobs via files directive in a package spec just like as other source files.

name: cockroachdb
- cockroach-latest.linux-amd64.tgz

Creating a release with above configuration causes following error:

bosh create-release --force
Building a release from directory '/Users/user/workspace/cockroachdb-release':
  - Constructing packages from directory:
      - Reading package from '/Users/user/workspace/cockroachdb-release/packages/cockroachdb':
          Collecting package files:
            Missing files for pattern 'cockroach-latest.linux-amd64.tgz'

CLI expects to find cockroach-latest.linux-amd64.tgz in either blobs or src directory. Since it's a blob it should not be in src directory but rather added with the following command:

bosh add-blob ~/Downloads/cockroach-latest.linux-amd64.tgz cockroach-latest.linux-amd64.tgz

add-blob command:

  • copies file into blobs directory (which should be in .gitignore)
  • updates config/blobs.yml to start tracking blobs

Listing blobs

To list currently tracked blobs use bosh blobs command:

bosh blobs
Path                              Size    Blobstore ID                          SHA1
cockroach-latest.linux-amd64.tgz  15 MiB  (local)                               469004231a9ed1d87de798f12fe2f49cc6ff1d2f
go1.7.4.linux-amd64.tar.gz        80 MiB  7e6431ba-f2c6-4e80-6a16-cd5cd8722b57  2e5baf03d1590e048c84d1d5b4b6f2540efaaea1

2 blobs


Blobs that have not been uploaded to release blobstore will be marked as local until they are uploaded.

Uploading blobs

Blobs should be saved into release blobstore before cutting a new final release so that others can rebuild a release at a future time.

bosh upload-blobs command:

  • uploads all local blobs to release blobstore
  • updates config/blobs.yml with blobstore IDs

config/blobs.yml should be checked into a Git repository.

Removing blobs

Once a blob is no longer needed by a package it can be stopped being tracked.

bosh remove-blob command:

  • removes blob from config/blobs.yml
  • does NOT remove blob from release blobstore so that new releases can be created from older revisions