This guest blog post is part of an Atlassian blog series raising awareness about testing innovation within the QA community. You can find the other posts in this series under the QA Innovation tag.
At the Atlassian Summit in June this year we announced Bonfire, a tool for rapid bug reporting and session-based testing. In the five months since then we’ve sold over two thousand Bonfire licences, but had it not been for the generous support of my colleagues and my now three year old son, Bonfire wouldn’t exist. My name is Andrew Prentice, I manage the QA team at Atlassian and this is the story of how Bonfire came to be.
The Inspiration
The idea for Bonfire arose from a time tracking study we did in the QA team back in 2009. We looked at four hundred hours of testing and learnt that 16% of our time was spent reporting bugs in Jira. Whilst several factors contributed to this, the biggest culprit was the effort required to create comprehensive bug reports. Often that required multiple round trips between Jira and the software being tested in order to collect and complete the issue meta data, attaching screenshots can be convoluted if you need to annotate them and the context switching involved disrupts testing productivity. It meant a lot of time was spent not testing. If we could halve the time spent creating issues then every QA engineer would gain back a month a year for more testing – the same as hiring an additional person for every twelve already on the team.
The initial requirements were straight forward. We needed a way to:
- Create issues without leaving the context of the application being tested.
- Take screenshots and annotate them without switching apps.
- Populate issue meta-data automatically.
Given Atlassian’s products are web applications, not switching contexts usually meant not leaving the browser tab we were testing in. Extending the browser via an extension would allow this and being browser based would also allow HTML5’s canvas element to be used for drawing annotations and it’s toDataURL() method to save these as images. The privileged access browsers grant to extensions meant we could inspect any page via javascript, making a template system with variable support to automatically populate issue data doable.
The Perspiration
In early 2010, I asked Pete de Zwart, who was developing and maintaining some of our test automation infrastructure at the time, to spike a Firefox extension that could create issues in Jira. I don’t think javascript’s loose typing was quite the holiday from wrangling Maven that Pete was hoping (sorry Pete), but his work showed that this approach was feasible. The question then was how to build the real thing?
Like many companies, Atlassian has a set of values, but unlike any other company I’ve worked for these values really mean something. We question ourselves and each other continually about whether what we’re doing upholds our values and those answers are decisive.
“Be the change you seek” is one of our values and it has created a culture where procrastination and buck passing is rare, fuelled in part by 20% Time, an idea borrowed from Google that encourages everyone to spend a fifth of their time on projects of their choice. Working in an environment where bottom-up innovation is both commonplace and a priority is inspiring. Inspiring enough that it never occurred to me not to try building this tool myself, despite the fact that as a tester, my coding experience was limited to test automation, my javascript knowledge was rudimentary and as a manager I didn’t have time for 20% Time.
Truth be told, the trend towards heavier client-side web apps meant that I’d been wanting to learn javascript for a long time and building a test tool was more than the excuse I needed. Somewhat paradoxically it was my son, who was two years old then, who provided me with the time to work on it. Children change most people’s lives and my wife and I were no different. Friday nights were no longer for going out, but for raising a family and for my wife an opportunity to go to bed early and recover from the joys of parenthood. With everyone else fast asleep and myself on call for any parental emergencies, Friday nights were the perfect opportunity to hack away.
To begin with it was definitely progression via trial and error as I worked on building the screenshot drawing tools. To complicate matters further trigonometry wasn’t my best class at school so I found implementing a tool to draw circles tough. There were many nights of hair-pulling, inexplicable failures and circles rendering in unexpected places all over my screen. Man I hated circles. However, with time and Wikipedia’s trigonometry entries, my understanding of SOH-CAH-TOA grew enough to make it work.
By June 2010 I’d written the screenshot and annotation functionality and integrated it with Pete’s Firefox extension spike. At the Atlassian Summit that year, I showed it to anyone who had sought me out to talk about testing. I wanted validation that this hadn’t turned into a vanity project and that the idea still had merit, but I wasn’t expecting people to ask when it would be available commercially. I came back even more determined to finish the project, but also thinking about ways to make it more useful for any tester, not just Atlassians.
Behold Excalibur!
Reading Douglas Crockford’s Javascript the good parts had a noticeable affect on my javascript and after Summit I started to get into a groove. I created the Safari and Chrome extensions, rewrote the Firefox extension so that it wasn’t written in XUL, added support for multiple Jira servers, full page screenshots, templates and variables. I added more annotation tools, including a magnify tool at the request of Penny Wyatt, a QA engineer who’d worked on the major Jira UI changes that year and had dealt with her fair share of pixel alignment issues.
I released it internally in October 2010. Here’s how it looked:
It’s internal name was Excalibur. I had come to the conclusion that this could become more than a tool for rapid issue reporting. It had the potential to become a swiss army knife for web testing and by all accounts Excalibur was a pretty good knife. It takes more time to explain that name than was put into choosing it (unlike its final name Bonfire – more on that later), but the name served as a reminder to build a tool that testers could wield in their own way, rather than a tool that forced testers to behave in specific ways.
We’ve found that traditional testing approaches aren’t suitable for the rapid pace of Agile development, so the only manual testing we perform is exploratory testing, which traditional test tools don’t support. Customers often ask us to build test case management into Jira, but we don’t want to build something that we wouldn’t use ourselves. We’ve trialled James and Jon Bach’s Session Based Test Management (SBTM) method and although we liked it a lot, it was cumbersome to manage it with Jira and the few SBTM tools around weren’t mature enough for our needs.
Adding support for session based testing in Jira, therefore was at the top of my list for extending Excalibur. What’s more, the idea of providing an atypical test management solution specifically targeted at Agile teams caught the attention of Jira’s product management. In Part II, I’ll explain how their support plus some wheeling and dealing transformed Excalibur from an internal tool into Bonfire the commercial product.
In the meantime if you want to foster innovation in your team or organisation, enable it from the bottom-up by providing people with the time to try, fail and succeed. Watching your peers take on challenges and succeed is incredibly inspiring and creates a culture where everyone’s prepared to help each other out. In addition to Pete’s help, I’m indebted to Craig Sharkie and Ryan Ackley for patiently answered my many javascript and browser extension questions when I was getting started.