Build Stages Part 2: Stages Order and Conditions

[svenfuchs]

We have shipped iteration 2 for the Build Stages feature.

Along with a lot of bug fixes and small improvements this includes:

• Build Stages section: Define the order of stages, optionally with a condition
• Conditional builds, stages, jobs: Define conditions for accepting/rejecting a given build, stage, or job

Find out more about this on our blog, and documentation here and here.

We would love to hear your feedback. Please leave all thoughts, comments, ideas related to this feature here.

Happy Testing!

FAQ

What feature requests have you received for improving Build Stages so far?

We are adding this list of feature requests (with a rough indication of how likely we are going to prioritize it) so you don't have to ask about these again :)

• Pausing a build after a given stage, and proceeding only after an interactive confirmation. (Yes)
• Add a native build artifacts feature that can be used for sharing artifacts across stages, or another way to more easily share storage or artifacts. (Yes)
• Making the native cache feature more configurable by specifying cache keys per job. This would allow you to reuse caches in more flexible ways in subsequent stages. (Probably yes)
• Allow script etc. on the new stages section, so they can be shared across all jobs in one stage. (Yes, but not before [1])
• Allow specifying allow_failure: true per job on jobs.include. (Probably yes, but not before [1].)
• Allow a stage key on the deploy section. Turn this into a stage and a job. (Not quite sure, not before [1].)
• Add a name/description to jobs, in order to reveal the intend, show them in the UI. (Not quite sure.)
• Consider skip: all, so one does not have to overwrite before_install, install, and script. (Sounds like a good idea? Not before [1])
• Silence skipped commands: no log output. (Not quite sure.)
• Consider "embedded matrix expansion", e.g. jobs.include.env: [FOO=foo, BAR=bar]. (Not quite sure)
• Build config .travis.yml editor/web tool. (Yes, based on the specification produced in [1])
• Allow grouping jobs into arbitrary groups, not depending on each other, like stages, just for visual presentation on the UI. (Probably not)
• Update the GitHub commit status per stage, add with more detailed commit status information. (Unsure, definitely not before [2])
• Consider automatically restarting jobs in later stages that have been canceled because a job on a previous stage failed when this job is restarted. (So far uncertain. There's a lot of complexity involved.)
• In the running tab, consider grouping jobs per build (and possibly stage). (Interesting thought. We are working on improving this UI, and might consider this in a later iteration.)

Other improvements:

• [1] A strict travis-yml parser has been shipped.
• [2] The GitHub commit status API has the known limitation that new updates are being rejected after the 1000th update. They are working on improving this, and providing a way for us to post more updates. Until this is unblocked we are unlikely to make any changes to our commit status updates.

[ljharb]

Heads up, the comment on #11 points to itself instead of to here.

[23Skidoo]

Is there any chance you could implement a more fine-grained job dependency graph feature than just "wait until the previous stage is done"? E.g. in my case I have a bunch of different compiler/OS configurations, and "populate cache" and "build" stages that each have a bunch of jobs corresponding to every configuration. In principle, a "build" job could start as soon as the corresponding "populate cache" job for that configuration was done, but right now the "build" stage doesn't begin until all of the jobs in the "populate cache" stage have finished.

See haskell/cabal#4556 for more details.

[szpak]

Is it possible to skip cache update for a particular stage?

I have a acceptance test job which should leverage cache from the previous stage. However, I don't care about the new artifacts it creates/installed (which would be cached in default configuration) and I don't want to have cache updated at the end of that particular stage (while cache still should be fetched at the beginning of that job to have build faster).

[ujovlado]

Looks like there's a small bug. If I have multiple stages (e.g. 4) and I add conditions to run only 1 stage then the main job will stuck in "created" state.

[ujovlado]

Looks like there's a small bug. If I have multiple stages (e.g. 4) and I add conditions to run only 1 stage then the main job will stuck in "created" state.

Hm, that's also true for multiple jobs (e.g. if conditions remove 2 jobs and 2 remain).

[BanzaiMan]

@ujovlado I think that is travis-ci/travis-ci#8415. We will look into it shortly.

[ujovlado]

@BanzaiMan thanks!