Quarantine!? Has the zombie apocalypse finally happened!? What’s this new feature with the crazy name?!
Ok, now that you’ve calmed down…
Do you have an annoying test that keeps randomly failing and stopping you from building? Have you ever wanted to hide away tests that you don’t need to worry about for the moment? Then read on!
‘Quarantine’ is a new feature of Bamboo 4 which lets you completely ignore a test and its results. (What? You haven’t heard about the awesomeness that is Bamboo4? Catch up on the kick-ass new features here.)
How can Quarantine help me?
Have you ever wanted to…
- Get rid of those flaky tests that keep randomly failing your build?
Tests can be flaky sometimes. For example, they might rely on an external server, so they just fail at times and it is beyond your control. However, sometimes you can’t just delete these tests to get rid of the problem, because you still need to keep an eye on whatever they are testing. In Bamboo 4, you can quarantine these tests. Your builds will proceed as normal, you will be able to see the result of the test, but those annoying flaky tests won’t affect the outcome of your build.
- Be able to create a build artifact even with failing tests?
Artifacts will only be produced in Bamboo from passing builds. What if it’s crunch time, you need to release your version by tomorrow, and there is only one non-critical test failing? Previously, you would have to dive into the source code and comment out the test. Now, quarantine it, run the build, and Bamboo will create your artifact as normal. Then you can unleash that test and resume normal development.
- Fire off dependant plans from a failing build?
From a post on Atlassian Forums:
My use case is that I have 2 projects to build, one dependent on the other. If the parent project compiles but unit tests fail, I still want to be able to compile the child project and run its unit tests. Currently, if unit tests in the parent project fail, the dependent build will not be started. This is a major problem.
Good news, everyone! Quarantine the relevant tests and your child project will still trigger!
There are some example use cases for this feature here, here,and here.
Hang on, I’m getting complete control over whether my builds pass? Isn’t that dangerous? 😉
Quarantine is an advanced feature, and is for use at your own risk. It is restricted to plan admins, so no one should be quarantining 9001 failing tests just to make your build pass. We believe in giving you control over your build server, and trust that you will use it wisely. In most of the above use cases, quarantining is a temporary solution. If you’re busy now, sure: hide it for a bit and come back to it. It’s much better than deleting your test, because it lets you see that your build is actually green (with a caveat). But it is by no means a permanent measure. Be aware of what tests you have quarantined, and why!
Stop telling me, show me!
Alright! Here is the build results page for a failing plan. Note the ‘Quarantine’ buttons by each test result.
Just click them, and next time your build runs…
Our build passed! Notice that the test results are hidden from this screen, except for a number to tell you how many tests were quarantined. If we click that number we can see more information about these tests…
Finally, you can get an overview of all the quarantined tests from the plan summary level. There’s a nifty little tab that appears when you have tests in quarantine. It looks like this:
From here we can see an overview of all the tests currently in quarantine, their status for the last ten builds while in quarantine, and by whom and when they were placed in quarantine.
Take advantage of this exciting new feature in Bamboo 4!
