concourse/0.44.0
You can find the source of this version on GitHub at concourse/concourse. It was created based on the commit 3d5d2eff
.
Release Notes¶
This release is hella backwards incompatible. Read carefully, and ask in IRC (#concourse) if you need help!
We won’t be making such drastic changes after 1.0, but as long as we’re still figuring things out, we don’t want to collect tech debt or land on the wrong set of primitives. - Backwards-incompatible: the progression of artifacts through a build plan has been made more explicit.
Previously there was basically a working directory that would be
streamed from step to step, and aggregate
steps were relied on to
place things under subdirectories, which is how inputs to tasks were
satisfied.
Now, as a plan executes, each step’s produced artifact (for example a
get
step’s fetched bits or the result of a task
’s execution) are
stored in a pool, with the source named after the step.
This change affects many things, but the primary things you’ll notice
are as follows:
- When executing a task
step, its inputs are collected from the
pool, rather than blindly streamed from the previous step. This
means aggregate
is no longer required to satisfy task inputs, and
can now be removed if it’s only wrapping one step.
Tasks are now _required_ to list their set of inputs, otherwise no
inputs will be streamed in. This is backwards-incompatible, but has
many advantages: it’s more explicit, more efficient, and makes it
clearer where the dependent inputs will be placed in a task’s
working directory when it runs.
When a task completes, its resulting working directory is added to
the pool, named after the task itself. This is how you would `put`
using artifacts generated by tasks.
- The
file
attribute of atask
step must now qualify the path with the name of the source providing the file. When executing a
put
step, all sources are fetched from the pool. Later on we may introduce a change so thatput
steps declare their dependencies, but for now streaming everything in is the simplest path forward.The net effect of this is that any params referring to files in
put
steps must now qualify the path with the source name, as they’re all fetched into subdirectories.Now that there’s a flat pool of sources, later steps in a build plan can now refer back to previously fetched (or generated) sources, rather than having to fetch them again.
So, if before you had a plan that looked like this:
plan:
- aggregate:
- get: something
- task: generate-foo
file: build.yml
- put: foo-bucket
params:
from: foo
…it would now look like this:
plan:
- get: something
- task: generate-foo
file: something/build.yml
- put: foo-bucket
params:
from: generate-foo/foo
Notably, the redundant aggregate
is gone, the file
attribute of
the task
step qualifies the filename with the name of the source
containing it, and the put
step qualifies the path to foo
with the
name of the task that it came from.
Also, the something/build.yml
task would now explicitly list its
inputs, if it wasn’t before. So that could mean changing:
platform: linux
image: docker:///busybox
run:
path: something/some-script
…to…
platform: linux
image: docker:///busybox
inputs:
- name: something
run:
path: something/some-script
This has the advantage of making the task config more self-documenting, and removes any doubt as to what inputs will be placed where when the task starts.
Note that listing inputs in the task config is not new, and if you were already listing them before the semantics hasn’t changed. The only difference is that they’re now required. - Backwards-incompatible: worker registration is now done over SSH, using a new component called the TSA.
To upgrade, you’ll have to change your manifest a bit:
- On your workers, replace the gate
job with groundcrew
and remove
the gate
properties.
- The new tsa
job template will have to be added somewhere, and
configured with the atc
credentials (the same way gate
used to
be configured).
Colocating `tsa` with the `atc` works out nicely, so that you can
register its listening port `2222` with your routing layer (e.g.
ELB), which will already be pointing at the ATC.
To compare, see the example AWS VPC manifest.
The main upshot of this change is it’s much easier to securely
register an external worker with Concourse. This new model only needs
the worker to be able to reach the ATC rather than the other way
around.
- Backwards-incompatible: Consul services are now automatically
registered based on the jobs being colocated with the agent. For this
to work, you must edit your deployment manifest and move the
consul-agent
job to the top of each job template list, and remove
your existing Consul services configuration from your manifest.
- The get
and put
steps from a build’s execution can now be hijacked
after they’ve finished or errored. Previously they would be reaped
immediately; now they stick around for 5 minutes afterwards (same
semantics as task
s).
- The S3 resource now
defaults to the us-east-1
region.
- The S3 resource no longer
fails to check when the configured bucket is empty.
- A new BOSH Deployment resource has been introduced. It can be used to
deploy a given set of release/stemcell tarballs with a manifest to a
statically configured BOSH target. The precise versions of the
releases and stemcells are overridden in the manifest before deploying
to ensure it’s not just always rolling forward to latest
.
Usage¶
You can reference this release in your deployment manifest from the releases
section:
- name: "concourse" version: "0.44.0" url: "https://bosh.io/d/github.com/concourse/concourse?v=0.44.0" sha1: "c6acdd181c53e925485a2eb02fa90a028ada354f"
Or upload it to your director with the upload-release
command:
bosh upload-release --sha1 c6acdd181c53e925485a2eb02fa90a028ada354f \ "https://bosh.io/d/github.com/concourse/concourse?v=0.44.0"
Jobs¶
Packages¶
- archive_resource
- atc
- blackbox
- bosh_deployment_resource
- bosh_io_release_resource
- bosh_io_stemcell_resource
- cf_resource
- consul
- docker_image_resource
- fly
- generated_tsa_host_key
- generated_worker_key
- git_resource
- github_release_resource
- golang
- jettison
- pid_utils
- postgresql_9.3
- s3_resource
- semver_resource
- time_resource
- tracker_resource
- tsa
- vagrant_cloud_resource