All Versions
50
Latest Version
Avg Release Cycle
28 days
Latest Release
1292 days ago

Changelog History
Page 5

  • v5.2.4 Changes

    September 27, 2019
    ๐Ÿ”’ ๐Ÿ”— Security
    • ๐Ÿ”’ This is a Security patch using GoLang v1.13.1 that address a recently reported issue with Go net/http (CVE-2019-16276).
      GoLang's net/http (through net/textproto) used to accept and normalize invalid HTTP/1.1 headers with a space before the colon, in violation of RFC 7230. If a Go server is used behind an uncommon reverse proxy that accepts and forwards but doesn't normalize such invalid headers, the reverse proxy and the server can interpret the headers differently. This can lead to filter bypasses or request smuggling.
  • v5.2.3 Changes

    September 03, 2019
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿš€ This release simply involves rebuilding Concourse with GLIBC 2.23. This should maintain backwards compatibility for folks running Concourse on older Linux kernels.
  • v5.2.2 Changes

    September 03, 2019
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿš€ This release simply involves rebuilding Concourse with go 1.12.9 to fix CVE-2019-9512 and CVE-2019-9514. Neither of these is very likely to affect Concourse users in the first place, but we are releasing a patch for good measure.
  • v5.2.1 Changes

    August 20, 2019
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ’ป The web UI used to silently break when your token (which includes a potentially-long JSON-encoded string detailing all the teams you are part of and what roles you have on them) was longer than the size of a single cookie (4096 bytes on most browsers!). This limit has been increased 15-fold, which should unblock most users on clusters with a lot of teams #4280.
  • v5.2.0 Changes

    May 16, 2019
    ๐Ÿฑ ๐Ÿ”— feature, breaking

    ๐Ÿ‘€ @ralekseenkov has implemented generic credential caching for all credential managers!

    This replaces the Vault-only caching functionality. To transition, you'll need to update the following flags.

    --vault-cache is now --secret-cache-enabled

    --vault-max-lease is now --secret-cache-duration

    As part of this change, credential managers now implement a simpler interface that will make it easier to look-up secrets in multiple paths.

    ๐Ÿ‘€ For more information, see Credential Management.

    To follow along with future planned improvements to credential management, check out concourse/rfcs#21.

    ๐Ÿฑ ๐Ÿ”— fix, breaking
    • ๐Ÿ— @stigtermichiel changed the short-flag for fly builds --team from -t from -n to make it consistent across fly. Consistency is key.
    ๐Ÿฑ ๐Ÿ”— feature
    ๐Ÿฑ ๐Ÿ”— feature

    ๐Ÿ”ง The web node can now be configured to enable audit logs, thanks to a PR by @loghen41!

    ๐Ÿ”Š Auditing currently logs API calls to the default logger using flags to enable specific auditing groups.

    ๐Ÿฑ ๐Ÿ”— feature

    ๐Ÿšš Like a phoenix from the ashes, the pipeline navigation sidebar has made its triumphant return. It was initially removed to focus our efforts on the dashboard as a navigation flow. We have concluded that one click is better than two.

    ๐Ÿš€ Expect more design/UX polish in future releases!

    ๐Ÿฑ ๐Ÿ”— feature

    @itsdalmo has introduced a new in_parallel step which can run steps in parallel with more control via additional config: limit which will limit the number of parallel steps, and fail_fast which will interrupt currently running steps and prevent scheduling pending steps.

    ๐Ÿ—„ This sounds a lot like the aggregate step, only better in every way (e.g. it doesn't have a stupid name), so fly set-pipeline will now issue deprecation warnings for aggregate: usage.

    ๐Ÿฑ ๐Ÿ”— feature
    • โž• Added a tooltip to the pause toggle on the dashboard page and the pipeline page explaining why it might be disabled.
    ๐Ÿฑ ๐Ÿ”— feature
    • ๐Ÿ— @hprotzek added the ability to retain build logs for a specific time duration and/or build count. See build_log_retention for more details.
    ๐Ÿ”’ ๐Ÿ”— security
    • 0๏ธโƒฃ We have restricted the SSH MAC algorithms used by the web node to a more secure set, overriding the Go defaults which allow weaker algorithms.
    ๐Ÿฑ ๐Ÿ”— feature
    • ๐Ÿ‘ Concourse is now compatible with Credhub v2.x (except for 2.1 due to a bug)! CredHub v1.9.x is still supported, too.
    ๐Ÿฑ ๐Ÿ”— feature
    • โž• Added ability set a name for the Concourse cluster which will be displayed on the dashboard page by setting cluster-name flag.
    ๐Ÿฑ ๐Ÿ”— feature
    • @cappyzawa added a new get-team subcommand to fly. It allows you to retrieve a single team's config.
    ๐Ÿฑ ๐Ÿ”— feature
    • ๐Ÿ‘ท @rkoster added a new flag --external-garden-url to allow use of a separately-managed Garden server as a worker.
    ๐Ÿฑ ๐Ÿ”— feature
    • @pivotal-kahin-ng added a way of retaining the build history of a job when renaming it, by updating the job name and specifying its old name as old_name. After the pipeline has been configured, the old_name field can be removed.
    ๐Ÿฑ ๐Ÿ”— fix
    • 0๏ธโƒฃ We reduced the default concurrency settings for volume sweeping from 5 to 3 as a way of reducing the stress that volume deletion ends up putting on the system in some cases.
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ›  @edtan fixed a panic caused by running concourse web without a --session-signing-key.
    ๐Ÿฑ ๐Ÿ”— fix

    ๐Ÿ‘ The Concourse API now returns 401 Unauthorized when an expired/invalid token is used to access an endpoint which supports authenticated/unauthenticated views.

    Previously it would just return a 200 response with less data, as if you weren't logged in, which made the behavior somewhat ambiguous and made auto-relogin logic difficult to implement consistently.

    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ›  Fixed a bug with Dex CloudFoundry connector when the user is a member of many teams. Thanks to @daniellavoie!
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ›  Fixed a bug where the user gets a "You are not authorized to view the details of this pipeline" while watching a build.
    ๐Ÿฑ ๐Ÿ”— fix

    ๐Ÿ›  Fixed a bug where aborting a started build prior to a web node re-attaching to it would result in an orphaned, still running, uncompleted build.

    ๐Ÿ”จ Along the way, the general 'aborting' flow has been refactored and should fix up any oddities caused by aborting builds at...inopportune moments.

    ๐Ÿฑ ๐Ÿ”— fix
    • โš  fly prune-worker --all-stalled has been fixed to only return a warning if no stalled workers are found, instead of an error.
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ‘ท concourse quickstart has been fixed to ignore the --worker-tsa-worker-private-key flag.
    ๐Ÿฑ ๐Ÿ”— fix
    ๐Ÿฑ ๐Ÿ”— fix
    • Multiple groups in the same pipeline can no longer use the same name. An error is now raised if attempted.
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ›  Fixed a bug where fly execute --input would hang indefinitely after uploading the input directory as a consequence of the web node stopping.
  • v5.2.0-rc.24

    May 15, 2019
  • v5.1.0 Changes

    April 16, 2019
    ๐Ÿฑ ๐Ÿ”— fix, breaking

    ๐ŸŒ tl;dr: concourse web --peer-url and concourse web --tsa-peer-ip are gone in favor of concourse web --peer-address

    ๐Ÿ”จ We have been doing a lot of internal refactoring and decoupling between various components. One side effect of this is that the web nodes no longer need to stream user artifacts to one another, which was the only reason the concourse web --peer-url flag was needed, so it has been removed.

    ๐Ÿ‘ท However, the SSH gateways (the "TSAs"), which also run on the web nodes, still need their address for the forwarded worker connections advertised to other web nodes. This value used to be inferred by --peer-url, so we've added a new --peer-address flag for it.

    ๐Ÿ”’ ๐Ÿ”— security, breaking

    0๏ธโƒฃ The web node now defaults X-Frame-Options to deny to safeguard against clicjacking attacks. If you run Concourse in an iframe, you'll notice that it doesn't work anymore.

    ๐Ÿ‘€ To configure X-Frame-Options otherwise, see Ingress.

    ๐Ÿฑ ๐Ÿ”— feature
    ๐Ÿฑ ๐Ÿ”— feature
    • ๐Ÿ— Each step in the build log will now show how long it took to initialize and run when hovering over the icon to the right of the header. Thanks for the PR, @mockersf!
    ๐Ÿฑ ๐Ÿ”— feature
    • Resources can now be annotated with icon to put pretty little icons in your pipeline and make different resource types easier to distinguish. This was also a PR by @mockersf - thanks a bunch!
    ๐Ÿ”’ ๐Ÿ”— feature, security

    ๐Ÿ“‡ Resource metadata will no longer be shown by default in exposed pipelines.

    ๐Ÿ“‡ Metadata should never contain credentials or any criticial information, but for some use cases it is not desireable to show e.g. commit messages and authors even though the pipeline is public.

    The resource must now have the public value set in order to show metadata, just like jobs. One caveat is build output: if a job is public, any get step and put steps will still show their metadata.

    ๐Ÿฑ ๐Ÿ”— feature
    • fly execute will now upload inputs and download outputs in parallel.
    ๐Ÿฑ ๐Ÿ”— feature

    ๐Ÿš€ The Concourse BOSH release now packages Ubuntu-flavored images for each core resource type instead of Alpine. This is primarily for compliance reasons. Nothing should really be affected.

    ๐Ÿ‘€ The .tgz distribution continues to use Alpine so the tarball doesn't get even bigger. Once we minimize the amount of resource types we package with Concourse (see (RF)RFC #23) we'll be removing them and standardizing on Ubuntu for simplicity's sake.

    ๐Ÿฑ ๐Ÿ”— feature
    • Generic oAuth can now be configured with different user ID/name keys. They default to user_id and user_name, just as before.
    ๐Ÿฑ ๐Ÿ”— feature
    • ๐Ÿ”ง Generic OIDC auth can now be configured with a different user name key. It defaults to username, just as before.
    ๐Ÿฑ ๐Ÿ”— feature

    ๐Ÿšš Previously, workers would garbage collect containers in volumes sequentially, destroying containers first and then volumes. This meant that if a worker had thousands of volumes to remove, it would go through and destroy them one by one - meanwhile, containers were not being garbage-collected.

    0๏ธโƒฃ Containers and volumes are now garbage-collected in parallel to each other, with a default max-in-flight of 5 containers and 5 volumes at a time. This speeds up garbage-collection overall and prevents an imbalance in volume/container counts from slowing each other down. This is especially important as workers are typically capped at 250 containers, but may have thousands of volumes and may even have a slow disk.

    ๐Ÿฑ ๐Ÿ”— feature
    ๐Ÿฑ ๐Ÿ”— feature
    • Pipelines now have a play/pause button at the top bar, so you don't have to go all the way back to the dashboard and find the pause button there. Thanks for the PR, @robwhitby!
    ๐Ÿฑ ๐Ÿ”— feature
    • ๐Ÿ“‡ URLs in resource metadata are now clickable, thanks to a PR by @Twiknight!
    ๐Ÿ”’ ๐Ÿ”— fix, security

    Fixed a minor information leak that would allow unauthenticated users to fetch the step names and structure for a build whose job is not marked public.

    This only exposed step names, but it was still a little weird to allow it to be fetched. It will now return a 401 Unauthorized instead.

    ๐Ÿฑ ๐Ÿ”— fix

    โฑ Previously, manually-triggered builds would cause resource checking to be performed in the job scheduling loop. This ensured that manually triggered builds ran with the latest versions available, but it also slowed down scheduling for every other job in the pipeline, because they're all scheduled one-by-one.

    ๐Ÿ— In the worst case, this meant a hanging resource check could result in all builds in the pipeline being stuck in the "pending" state for a long period of time (or, "like, forever").

    โฑ So we changed things around a bit: instead, the scheduler just won't start a manually triggered build until the "last checked" timestamp of each of its resources is after the build's "created at" timestamp. And to make that go faster, when a build is manually triggered we'll short-circuit the checking interval for each of its input resources.

    โฑ With this change, if/when a resource check is hanging or slow it at least won't gum up the pipeline scheduling for all the other jobs.

    Expect more improvements in this area in the next few releases! We'll be making jobs schedule in parallel soon so they can't affect each other, and we're working on a new "algorithm" that should scale a lot better with pipelines that have a ton of data or versions.

    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ”จ The above refactoring also fixed a race condition that could result in inputs configured with version: every having versions skipped when a build is manually triggered.
    ๐Ÿฑ ๐Ÿ”— feature
    ๐Ÿฑ ๐Ÿ”— feature
    ๐Ÿฑ ๐Ÿ”— feature
    ๐Ÿฑ ๐Ÿ”— fix
    • version on a get step will now take precedence over versions pinned via the web UI or via version on a resource definition.
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿš€ The HD dashboard view got a little weird in the last couple releases - it's fixed now.
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ›  Fixed the spacing of the pipeline view so super tall pipelines don't get clipped by the top bar.
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ›  Fixed the status:running search functionality on the dashboard view.
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ— When viewing a pipeline build by ID (/builds/123), the top bar will show the breadcrumb for its pipeline and job instead of being empty.
    ๐Ÿฑ ๐Ÿ”— fix
    • The breadcrumb in the top bar now uses actual links, so they can be middle-clicked and right-clicked to your heart's content.
    ๐Ÿฑ ๐Ÿ”— fix
    • The groups bar on the pipeline view now has hover states for each group.
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ›  Fixed a bug that caused credential managers to be instantiated twice, resulting in two auth loops.
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ— When viewing a one-off build in the web UI, the build will now render instead of chucking errors in the browser console.
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ’ป The web UI is now up-to-date with Elm 0.19! You shouldn't really notice anything, but...yay!
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ›  Fixed a crash that would occur when a build finished that produced outputs for a resource that had been un-configured from the pipeline in the meantime.
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ— The web node will now retry on unexpected EOF errors which could occur when a worker was restarted while a build was running a container on it.
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ›  Fixed a bug with the Vault login re-try logic that caused it to go into a fast loop after reaching the maximum interval. Now it'll actually stay at the maximum interval.
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ— When viewing a build for a job that has a ton of builds, only the first batch of builds will be fetched and rendered instead of all of them. Older builds will be automatically loaded if the build being viewed is old, or as the user scrolls to see them.
    ๐Ÿฑ ๐Ÿ”— feature
    • ๐Ÿ’ป We're now consistently using Material Design icons everywhere in our UI - the last of the Font Awesome stragglers have been replaced!
    ๐Ÿฑ ๐Ÿ”— fix

    ๐Ÿ›  Fixed quite a few quirks with the dashboard search:

    Team name autocomplete will now work even if you're not logged in.

    ๐Ÿ›  Fixed the unstyled autosuggest menu in Chrome.

    Hitting the escape key will now un-focus the search field.

    The search autocomplete will now only appear if you press a key with the search field focused.

    Typing ? into the search field will no longer bring up the hotkey help pane.

    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ— fly execute will now print the correct URL for the build when running with -j.
    ๐Ÿฑ ๐Ÿ”— fix
    • fly login will now create ~/.flyrc with stricter permissions (0600).
    ๐Ÿฑ ๐Ÿ”— feature
    • ๐Ÿ— We've added a (hopefully subtle) stripey animation to running builds in the build number list to help differentiate between errored and running builds.
    ๐Ÿฑ ๐Ÿ”— feature
    ๐Ÿฑ ๐Ÿ”— fix

    With v5.0.0 we introduced a bit of a performance regression with loading the versions for a pipeline during scheduling. We've made an incremental change to make it a bit faster.

    ๐Ÿ›  This will also be fixed by the new input candidate algorithm mentioned previously.

    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ”ง The dashboard will no longer crash when a pipeline is configured with a circular dependency.
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ›  Fixed the rendering of many, many pipeline groups.
  • v5.0.1 Changes

    March 25, 2019
    ๐Ÿ”’ ๐Ÿ”— fix, security

    ๐Ÿ›  Fixed a bug when saving wacky versions generated by wacky resource types that let you put wacky arbitrary data in the version.

    โฌ†๏ธ The bug enables limited SQL injection, so we recommend that anyone running 5.0 upgrade to this version as soon as possible. It's a bit concerning that we've ended up with a SQL injection vulnerability in 2019, but this at least appears to be an isolated and easily verifiable case. More on that later.

    Thankfully, this is very difficult and impractical to exploit, and the impact is fairly low despite it being a SQL injection:

    It is only possible to inject a single SELECT query, so there should be no loss of integrity or data.

    The SELECTed value would only be inserted into an internal column which is never exposed to users - it is only used for internal bookkeeping and putting something bogus there will have no effect on the rest of the system.

    This issue only affects resource types that put arbitrary user-specified data into the resource version. This is very unusual - almost all resource types have strict, simple versions (e.g. git refs, version numbers, sha256 digests).

    No core resource types are affected, and most resource types shouldn't be either. The only known resource types that do this are sort of hacky ones that propagate arbitrary data through the pipeline via resource versions.

    How this exploit happened:

    Normally, we use a lightweight framework for constructing queries safely (Masterminds/squirrel), and we always pass all user data as params ($1, $2, etc) so that escaping is never even necessary. In this case however the query was slightly more complicated, so we had to pop open the hood and directly construct a query fragment using sq.Expr.

    Unfortunately the portion that we injected did so by concatenating the resource version JSON into the query fragment. As a result, versions with a single-quote (') in them would break out of the surrounding string and insert their own SQL query. We've changed it to use a param instead, and we've done an audit of all other uses of sq.Expr to verify that they are only ever being given static strings, trivial pre-formatted data, or params.

    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿš€ The BOSH release now sets file permissions for its config values as 0600, which fixes Postgres certificate configuration. Thanks for the PR, @flavorjones!
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿš€ The BOSH release now correctly handles array-values for authorized worker keys. Sorry about that!
  • v5.0.0 Changes

    March 06, 2019

    ๐Ÿš€ This release is a doozy. You should probably read these release notes in full - there are a ton of substantial new features and a good (bad?) amount of breaking changes.

    Sorry this took so long! The holiday season took its toll, but we also got a bit overzealous with piling feature work on master, and well, we restructured the entire project and re-created its pipeline from scratch, so that didn't help.

    ๐Ÿš€ On the plus side, the project restructure is now done, and we'll be implementing a new release process soon that should prevent these kinds of hold-ups from happening again in the future.

    ๐Ÿš€ Special thanks to the many individuals in the community who took part in this release - whether you submitted a PR, helped triage issues, helped people out on the forums or in Discord, or simply cheered us on, every little bit helps keep the project humming along. We deeply appreciate it, and look forward to delivering y'all a better and better CI system - hopefully, more continuously.

    ๐Ÿฑ ๐Ÿ”— feature, breaking

    โฌ†๏ธ We have done a major internal overhaul of how resource versions are stored. As a result, the version history for each resource across your pipelines will be re-set upon upgrading to v5.0.

    โฌ†๏ธ The upgrade does however preserve the state of which versions were disabled, and the data relating versions to builds they were inputs to and outputs of.

    ๐Ÿ”ง In versions prior to v5.0, resource version history was associated to a pipeline resource by name. This meant that if you changed a resource's configuration or type, those old versions would actually stick around, even though they may technically no longer be appropriate.

    With v5.0, resource versions are now tied directly to an anonymous "resource config" - basically the source: and type: for the resource. Pipeline resources instead point to a config, and if their source: or type: changes, they'll point to a new config with its own version history.

    This improves the correctness of the system as a whole, eliminating the need to ever "purge" the history of a resource.

    In addition, now that versions are tied directly to their configs, check containers are also shared across teams, reducing the overall container count. As a result however we limited who can fly intercept check containers.

    ๐Ÿ— Building on this change, we are currently experimenting with improvements that can now be made to reduce the overall checking overhead across a Concourse cluster that has many equivalent resource definitions across pipelines and teams. This is currently off by default while we learn more about the implications - see Global Resources for more information.

    ๐Ÿฑ ๐Ÿ”— fix, breaking

    ๐Ÿ”ง We have removed --allow-all-users as almost every use has been a misuse. You must configure users explicitly now instead. This was done for development environments but even those were trivial to switch to a local user whitelist.

    ๐Ÿ”ง If you were setting this flag before, you probably didn't mean to - setting this with GitHub oAuth configured, for example, would allow literally everyone to be a part of your team and manage your pipelines.

    โฌ†๏ธ After upgrading, any teams that had this configured will preserve the behavior from before - they will continue to allow all users. The next time the teams are configured, however, you will have to specify something else, as the CLI no longer has the flag.

    ๐Ÿฑ ๐Ÿ”— feature, breaking

    The concourse binary distribution has been rejiggered. Rather than a self-contained binary, we now ship it as a .tgz containing the binary and its dependencies pre-extracted. The .tgz should be extracted somewhere like /usr/local, resulting in /usr/local/concourse/bin/....

    ๐Ÿ‘ท The main benefit of this is simplification and faster startup. The concourse worker command no longer needs to extract resource types/etc. on start, so this speeds that up quite a bit.

    ๐Ÿš€ The concourse binary no longer directly embeds Garden-runC code, and instead ships alongside the gdn binary, copied from their releases. This simplifies the interface for configuring Garden and allows us to leverage their build process rather than risking deviation.

    The "breaking" aspect of this is that if you have been passing esoteric flags to Garden you'll have to switch to using a config file via --garden-config instead, or pass them as env vars (e.g. CONCOURSE_GARDEN_FOO_BAR) - flags are no longer supported as those relied on directly embedding their code.

    ๐Ÿฑ ๐Ÿ”— feature, breaking

    ๐Ÿ— Workers can now be configured to periodically rebalance so that they don't end up all forwarding through a single web node. This is done by setting the --rebalance-interval flag on concourse worker. The rebalancing makes sure to drain in-flight connections and should not disrupt any in-flight builds.

    ๐Ÿšš Along the way, we removed support for direct worker registration. The --peer-ip flag is no longer available on concourse worker. To transition to 5.0, just remove the flag - the worker will register via forwarding instead.

    ๐Ÿ‘ท Forwarding is more secure as it doesn't require opening your workers up to inbound traffic. It's easier for us to just focus on one registration method and make sure it works well.

    ๐Ÿ‘ท This also sets us up for enforcing TLS for all traffic to the forwarded workers in the future (#2415).

    ๐Ÿฑ ๐Ÿ”— feature, breaking

    ๐Ÿš€ The Concourse BOSH release has been redesigned and is now centered around the concourse binary.

    โš  > warning: Be sure to recreate your workers after or during the deploy, as the location that the worker stores volumes has changed and the old volume directory will not be cleaned up, effectively leaking disk usage.

    โš  > warning: The additional_resource_types property can no longer be configured. We plan to add another mechanism for co-located resources in future releases.

    ๐Ÿš€ The concourse release no longer needs to be deployed alongside a garden-runc BOSH release, and instead embeds the gdn binary directly.

    ๐Ÿš€ Along the way, we have adopted BPM and now use it for deploying the web node. We also enforce a higher nofile limit which should make large-scale deployments more...scaley.

    ๐Ÿฑ ๐Ÿ”— fix, breaking

    Two flags have been modified to be more consistent with other flag syntax:

    ๐ŸŒ concourse web --vault-auth-param foo=bar should now be specified as concourse web --vault-auth-param foo:bar (note the :).

    ๐ŸŒ concourse web --tsa-team-authorized-keys team=path/to/key should now be specified as concourse web --tsa-team-authorized-keys team:path/to/key (note the :).

    ๐Ÿฑ ๐Ÿ”— feature

    The Concourse GitHub repository has been completely restructured. This isn't really a feature per se, but it should make contributing a lot easier.

    โšก๏ธ More on this on our blog post: The Great Process Update of 2018.

    ๐Ÿฑ ๐Ÿ”— feature

    ๐Ÿ— A new resource, the registry-image resource, has been added to the core. This resource is intended to replace the docker-image resource image for image pulling and pushing (but not building).

    ๐Ÿณ This resource improves on the docker-image resource in a few ways:

    ๐Ÿณ It doesn't run Docker to fetch the image - it's written in pure Go, using the google/go-containerregistry package. This makes the implementation much less error-prone.

    Because it doesn't run Docker, it doesn't need a privileged container. The fewer privileged containers in your cluster, the better - especially in light of [v4.2.3](https://github.com/concourse/recent CVE fixes/releases/tag/v4.2.3).

    By focusing solely on fetching and pushing, the resource is much smaller and simpler. It also has test coverage!

    The output has pretty colors.

    This all results in much faster, more efficient, and resilient image fetching. We recommend everyone to try switching your image_resources and Resource Types over - in most cases this is just a matter of replacing type: docker-image with type: registry-image.

    We intend to deprecate and phase out support for the docker-image resource in favor of the registry-image resource. We can't really do this until there's a solid direction for image building - preferably with a task, not a resource. This is a more natural split, and supports building images without pushing them - a long awaited ask of the docker-image resource.

    ๐Ÿ— An experimental task for this is available at concourse/builder. This is not yet official, but we've using it in our own pipeline and it's been pretty solid. Feel free to give it a try!

    ๐Ÿ— The next step from here is to actually kick off an RFC for reusable tasks - we're still collecting our thoughts for that in (RF)RFC #7. Once this is done we can formalize concourse/builder.

    ๐Ÿฑ ๐Ÿ”— feature

    We have introduced the first phase of role-based access control!

    ๐Ÿ‘€ Right now there are only a few statically defined roles. We started off by supporting the common request of having read-only team members ('team viewer'), and adding a slightly less powerful 'team member' role. See User Roles & Permissions for more information.

    Here's a quick rundown of how things have changed:

    Existing team auth config will be transitioned to the Team Owner role - that is, anyone that can authenticate prior to the upgrade will now be authenticated as an owner of their team. This role is the closest equivalent to what they could do before.

    The main team still has special admin power, with the slight tweak that only users that are an owner of the main team have admin capabilties.

    Before, teams members could rename or destroy their own team. Team owners no longer have this power - only admins can do this.

    The Team Member role is a new role that allows users to have full read and write powers within the team, except for being able to modify the team itself.

    ๐Ÿ— The Team Viewer role is a new role that allows users to browse the team's pipelines, builds, resources, etc. without permitting any sensitive operations (like fly get-pipeline or triggering builds).

    โฌ†๏ธ For a detailed breakdown of each role's capabilties, see the Permission Matrix. To learn how to configure these roles after upgrading, see Setting User Roles.

    If you're curious about the design process for this feature, check out RFC #3 (RBAC)!

    ๐Ÿฑ ๐Ÿ”— feature

    We have replaced resource pausing with resource pinning.

    ๐Ÿ“Œ Resource pausing had the effect of disabling the periodic checking for the paused resource. However we found that in most cases it was being used in combination with disabling versions to effectively pin a resource to the most recent available version.

    However, with global resource versions, each resource actually points to a shared history, so pausing checking wouldn't be enough - if any other pipelines had the same resource, new versions would still arrive!

    So instead, versions can now be pinned individually via the web UI or via the pipeline config (see version). Pinned resources will also skip periodic checking, but now even if the checking still happens (because some other pipeline had it un-pinned), the resource will stay pinned to the desired version.

    ๐Ÿ“Œ A comment can also be left on pinned versions for explaining to your team-mates why you decided to pin the resource.

    โฌ†๏ธ During the 5.0 upgrade, paused resources will be automatically transitioned to their pinned equivalent, by pinning the resource to the most recent available version. A comment will be left on any resources that are migrated so that it's clear to pipeline users.

    ๐Ÿฑ ๐Ÿ”— feature

    ๐Ÿ‘€ Task ((vars)) received a bit of an overhaul, thanks to a PR by @ralekseenkov!

    Values for task ((vars)) can now be provided during fly execute!

    In addition, values may be provided to a task step in a pipeline via vars.

    Tasks can now have ((vars)) pretty much anywhere in their config, not just in image_resource.

    In all cases, vars can also be satisifed via a credential manager, the same as before.

    Admittedly, there is now some cause for confusion with params. This may see clarification with reusable tasks. In addition, pipeline ((params)) will now be referred to as pipeline ((vars)) instead, for consistency.

    ๐Ÿฑ ๐Ÿ”— feature
    ๐Ÿฑ ๐Ÿ”— feature
    • ๐Ÿ‘ท Any volumes or containers that disappeared from their worker (possibly due to a worker being re-created and then coming back under the same name) will now be automatically reaped from the database. This makes it easier for Concourse to recover from this situation rather than erroring with file not found or unknown handle errors.
    ๐Ÿฑ ๐Ÿ”— feature

    ๐Ÿ”Š Logs emitted by Concourse components will now be...slightly prettier? They're still JSON (sorry), but the timestamps and log levels are at least human-readable.

    โšก๏ธ If you've got anything parsing your logs, make sure to update it accordingly!

    ๐Ÿฑ ๐Ÿ”— feature

    ๐Ÿ‘€ Concourse will now automatically retry fetching credentials when the request to the credential manager fails, thanks to a PR by @ralekseenkov!

    0๏ธโƒฃ By default Concourse will retry 5 times, waiting 1 second between each attempt. This can be adjusted with the --secret-retry-attempts and --secret-retry-interval flags on concourse web.

    ๐Ÿฑ ๐Ÿ”— feature

    ๐Ÿ‘ Tasks are now permitted to have inputs, outputs, and caches with overlapping paths. This was a hold-over from older versions of the container runtime that did not support this.

    This means that for simple tasks that e.g. make a commit a git repo, you no longer need to copy the input to the output. Yay!

    ๐Ÿฑ ๐Ÿ”— feature
    • The put step can now be explicitly given a list of inputs to use, rather than using all of them. This can be used to dramatically speed up builds that have a ton of artifacts prior to a put.
    ๐Ÿฑ ๐Ÿ”— feature

    ๐ŸŒฒ The fly login flow has been reworked a bit to better support logging in to a remote session. There's now a prettier landing page that detects when the token transfer fails by allowing you to copy the token to your clipboard instead.

    The auto-login prompt will also no longer ask for the token, because that disrupts the normal flow of the command. Previously it would ask for a token but then eat half of the keystrokes from then on. Now it just won't ask for a token.

    ๐Ÿฑ ๐Ÿ”— feature
    • ๐Ÿ The concourse binary now has a generate-key subcommand to assist with - you guessed it - key generation. This is more portable to other platforms (I'm looking at you, Windows) and is more likely to generate keys that Concourse can actually accept (I'm looking at you, OpenSSH 7.8).
    ๐Ÿฑ ๐Ÿ”— feature
    • ๐Ÿง The concourse worker command can now be given a --garden-use-houdini flag on Linux to use the "no-op" Houdini Garden backend for those odd cases where you don't really want containerization. (Use sparingly.)
    ๐Ÿฑ ๐Ÿ”— feature
    • ๐Ÿ— The timestamps shown in the build header will now transition to absolute instead of relative when the build is over 24 hours old. It wasn't very useful to see things like 128d 15h 30m ago when trying to compare old builds. Thanks for the PR, @Twiknight!
    ๐Ÿฑ ๐Ÿ”— fix

    You may have seen a scary error cropping up around your resources now and then. Something like worker_resource_config_check__resource_config_check_sessio_fkey references unreticulated spline.

    ๐Ÿ›  We fixed it. That thing doesn't even exist anymore. Don't worry about it.

    ๐Ÿฑ ๐Ÿ”— fix

    With Concourse 4.x configured with an oAuth provider such as GitHub, a user could log in via GitHub even if they weren't technically a member of any team. They couldn't do anything, mind you, but it was confusing that they were allowed to log in in the first place.

    This is no longer permitted.

    Similarly, fly login will also check to make sure you've successfully logged in to the target team and return an error if the team isn't in your token.

    ๐Ÿฑ ๐Ÿ”— fix

    ๐Ÿ”ง The AWS SSM credential manager and the AWS SecretsManager credential manager previously had a turf war going on over the AWS_REGION environment variable. They both declared it as their own, meaning if you set it they would both try to be configured, which would fail.

    They now have separately namespaced env vars instead.

    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ— fly intercept will now give a better error when it fails to execute the command (e.g. because bash isn't installed in the image).
    ๐Ÿฑ ๐Ÿ”— feature
    • ๐Ÿ‘ท fly execute can now specify input mappings via -m, which is useful when running with --inputs-from-job when the job renames some inputs.
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿšš fly execute with --include-ignored will no longer blow up when files are removed locally.
    ๐Ÿฑ ๐Ÿ”— feature
    • The error message when a task's file refers to an unknown artifact source (i.e. the foo in foo/ci/task.yml) has been made more descriptive.
    ๐Ÿฑ ๐Ÿ”— feature
    • ๐Ÿ‘ท There's a new fly command for landing workers remotely, called... fly land-worker. This will initiate the landing process via the API and will ultimately result in the worker process exiting. (Which may end up being re-started by whatever process monitor you use, but hey, it landed.)
    ๐Ÿฑ ๐Ÿ”— feature
    • ๐Ÿ‘ท The web UI now explains why some get steps have a yellow icon, via a handy-dandy tooltip. (Spoiler: it means the job has never run with that version before!)
    ๐Ÿฑ ๐Ÿ”— fix
    ๐Ÿฑ ๐Ÿ”— feature
    • ๐Ÿ— fly watch can now be called with --timestamps to show per-line timestamps in the build output. Thanks for the PR, @pivotal-kahin-ng!
    ๐Ÿฑ ๐Ÿ”— fix
    • fly get-pipeline will now throw an error if the specified pipeline does not exist, rather than returning an empty pipeline config.
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ›  Fixed various subtle UI issues with the dashboard page: #2430, #2434, #2435.
    ๐Ÿฑ ๐Ÿ”— fix
    • fly login will no longer prompt for your auth method when a username/password are given via flags. It'll deduce that you're trying to do local auth.
    ๐Ÿฑ ๐Ÿ”— fix
    • Task caches are now supported on Windows!
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ›  Fixed an internal bug that made UNIQUE constraints for resource_configs ineffective (#2509). This was fairly low-impact, but database integrity matters!
    ๐Ÿฑ ๐Ÿ”— feature
    • ๐Ÿ‘ BitBucket auth support has been re-introduced thanks to PRs to Dex and Concourse by @edtan!
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ‘ท The /api/v1/resources and /api/v1/jobs endpoints will now return [] instead of null when there are no resources or jobs, thanks to a PR by @pivotal-kahin-ng.
    ๐Ÿฑ ๐Ÿ”— feature
    • ๐Ÿ‘€ The dashboard page will now indicate whether you are seeing a pipeline because it's exposed by showing an ominous "eye" icon.
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ›  Fixed handling of auth configs set from empty env vars - previously this would result in bogus Dex configuration (e.g. github:, with no org or team) and sometimes cause things to misbehave.
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ’ป The legibility and anti-aliasing of text in the web UI has been improved.
    ๐Ÿฑ ๐Ÿ”— fix

    Cleaned up some dashboard behavior when there are no pipelines:

    ๐Ÿ‘€ you can now see which team you're a member of, rather than one big 'no pipelines set' page

    the bar along the bottom will now show up

    ๐Ÿ’ป there's a fancy ASCII art UI now

    the search function is no longer shown (since there's nothing to search)

    the HD view has been disabled and just redirects to / instead, since there was nothing for it to show

    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ’ป The username part of the top bar will no longer detonate when viewed on a tiny mobile browser.
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ“‡ When a resource's metadata is super wide, it will remain cordoned off to the side rather than uncomfortably squishing the resource's get output. Thanks for the fix, @stigtermichiel!
    ๐Ÿฑ ๐Ÿ”— fix
    • Concourse will now send TCP keepalives for connections to the database. This will allow it to detect when the connection has been interrupted ungracefully. Thanks for the PR, @SimonXming!
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ’ป The manifest.json href in the web UI used to be relative to the URL, meaning it was broken on any page except /. This is now fixed.
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ— The web node used to leak both a connection and a goroutine for each build that completed when configured to drain build logs to syslog. This is now fixed. Sorry about that!
    ๐Ÿฑ ๐Ÿ”— fix
    • The resources and resource types returned by fly get-pipeline will now be in a deterministic order, thanks to a PR by @edtan!
    ๐Ÿฑ ๐Ÿ”— feature
    • fly curl is a new command to assist with (hopefully occasional) manual API requests to Concourse. Thanks for the PR and collaboration, @simonjohansson!
    ๐Ÿฑ ๐Ÿ”— fix
    • The --tsa-authorized-keys flag is now optional, for situations where all authorized keys are associated to teams (via --tsa-team-authorized-keys). Thanks for the fix, @tlwr!
    ๐Ÿฑ ๐Ÿ”— fix
    • The fly status command will now let you know if your token has expired, rather than happily reporting that everything is fine.
    ๐Ÿฑ ๐Ÿ”— feature
    • A fly userinfo command has been added which will let you know which teams you are logged in to and which roles you have in each team.
    ๐Ÿฑ ๐Ÿ”— fix
    • ๐Ÿ›  The positioning of the "no results" text when searching on the dashboard has been fixed.
  • v5.0.0-rc.90

    February 25, 2019