At Atlassian, we use test sessions to manually test our products. If you’re a tester and haven’t heard of test sessions before, you might be surprised to learn that you’re probably using them too. That’s because test sessions aren’t new. They’re simply the periods of time, usually an hour or two, sometimes more, devoted to fulfilling specific test objectives. Executing a set of tests with the objective of proving that some piece of functionality works correctly or spending an hour bug hunting with the objective of uncovering unexpected behaviour are both examples of what I mean by test sessions.
Test Sessions > Test Cases
Test sessions differ from test cases in two ways. Firstly, more than one test can be carried out in a single session. Secondly, like test cases, test sessions can tell us who tested what, but they can also tell us how, when and why we tested. That makes test sessions useful for planning, executing and tracking manual testing, because they;
- Require testers to identify test objectives and focus their testing efforts on fulfilling them.
- Encourage testers to modify existing tests and add new tests to meet those objectives as they test.
- Provide a useful way to manage and review exploratory testing.
- Focus attention on the testing that’s actually performed, not just on test case results.
- Give testers the flexibility to respond to changes and re-plan their testing quickly.
To illustrate this, here’s an example using Jira and the test session capabilities of Bonfire.
Creating Sessions
For this example, user authentication is being added to an application and that’s what needs to be tested. The feature is being tracked as a feature request in Jira, but it could have been raised as a story, task or requirement instead.
Immediately, we can begin creating test objectives for this feature by creating some test sessions against this issue and naming those sessions with the test objectives we’ve identified.
There’s no restriction on when test sessions are added. It’s not uncommon for sessions to be created whilst the feature is being planned and developed and tested. Sessions can also be renamed or deleted as needed.
You can see all the test sessions for an issue via the Test Session tab, which provides anyone interested in this feature a nice overview of the planned testing.
If you have scripted manual test cases that you want to run for this feature then you can include those by creating another session to run those as well.
In Sessions
Once development has progressed and it’s time to test, the first thing to do is start a session, either through the Bonfire browser extension like this;
or via the test session’s page in Jira;
Either way displays an orange bar containing the session name to show which session is active. Now it’s time to start testing.
Obviously if you have pre-written tests they can be run during a session, but because test sessions emphasize test objectives over specific test cases, testers are also encouraged to devise and run any additional tests that they feel will help meet the test objective. Creating and executing more tests based on what you discover and learn whilst testing is an approach called exploratory testing. It’s an extremely powerful way of optimising test coverage without incurring the costs associated with writing and maintaining test cases.
Freedom isn’t free
The downside of the freedom that test sessions provide is the difficulty to track exactly what testing has taken place. Without that, it’s hard to grasp the quality of the software and the quality of the testing.
To address this, Bonfire creates an activity stream for each test session that automatically tracks when the session is started, paused and completed as well as all of the issues created in a session. Furthermore testers can add their own notes to the activity stream to record questions and actions to follow up after the session, document what was tested during a session, note which test data or configuration was used during testing or anything else pertinent to the session as they test. This permanent record of test session activity allows testers to refer to previous sessions as needed, and also review sessions with other people.
The Bonfire browser extension (on the right), displays the notes and issues for our current session.
And here’s the same test session in Jira after it’s completed, showing the activity stream and session summary:
When completing a session, Bonfire makes it easy to link the issues found during that session to the issue the session relates to;
And people interested in the feature can track the test progress for it by viewing the status of it’s test sessions;
In Summary
Test sessions aren’t a new way of testing, but a different way to plan and track testing. One that provides a stronger connection between testing and test objectives. They increase the time available for testing, by improving flexibility and responsiveness, whilst reducing the time spent on test planning and documentation. That makes them valuable for both traditional and Agile testing.
They’re great for exploratory testing too, which we find so effective at Atlassian that we don’t perform any other type of manual testing (the only test cases we write are our automated regression tests.) To learn more about exploratory testing and how to take tests sessions even further, I recommend Jonathan and James Bach’s Session Based Test Management method.
To learn more about Bonfire and its test session capabilities try it now for 30 days for free, or check out the many other test management software add-ons for Jira.