30 Days of Agile Testing! Day thirty.

Day 30:
What action does your team take on a red build?

I had to google the term ‘red build’ - but I can’t find anything relevant!
I guess it means, automated tests have failed (i.e. they’re red, not green).

Every time we release, a bunch of automated tests run. It’s all driven by Slack.
If everything’s good, there’s a whole lot of green ticks for each set of tests, and some really happy green emoji!


Each letter in that block stands for a suite of tests. Some are obvious - UT = Unit Tests, AT = Acceptance tests - et cetera. I’ll let the others remain a mystery :)

If something fails, then you don’t get the green emoji - and you’ll see some red X’s too. Here’s an example:


In this case the ‘B’ suite of tests has failed (a visual regression). 

At this point, here’s what could happen:

A developer will chime in and say “that was me, I expected that to fail!”
Which is great - he can then go and update the test or do whatever he needs to do to make it green again.

If that doesn’t happen, one or more (most likely all) of the developers will look at the results of the particular test that failed.
At this point, it’s usually pretty clear which piece of work made the test in question fail.Once that’s been figured out, the developer can take corrective action.
Either - ship a fix on top of what they’ve already done, to correct the behaviour. Or, revert their changes and go back and fix it before trying again.

The alternative is the test failed for some other reason - resourcing issue, timeout - these things happen sometimes. If that’s determined to be the case, the test can just be restarted.

What does the tester do while this is happening?
Well, from my own experience:

  • sometimes I’m first on the scene. I can investigate the failure first, alert the right people or restart the tests as needed.
  • help assess the risk. Can we ship something quickly to correct this? Or is that too risky, should we pull out?
  • analysis - what was the cause of this failure - why didn’t it get picked up earlier in our pipeline?
  • finally, testing - if we decide to fix it on the spot, I need to make sure - it’s actually fixed, and - we didn’t undo the work we were shipping originally!

And that, I think, is what our team does on a red build.

I hope that’s been useful!

- JE