Eliga’s favourite tester, Jon Hall makes a compelling case for exploratory testing, sharing his perspective.
What do you think of when you hear ‘testing’ on a project? I am of course referring to software testing; the testing of code; where the primary objective is to ensure the quality of the software under development. I have found many companies sigh at the thought of testing. Project managers often say ‘always the last phase of a project.’ This is often the biggest risk to projects going live on time.
‘My approach to testing is simply to fail fast.’
-Jon Hall (Eliga Services)
You should test at the beginning of any project, with testing input classed as important as any other area in the software delivery lifecycle (SDLC).
Many test managers will have their own interpretation of testing; call me ‘old school’ as this is how I always describe testing to anyone when they ask me what I do for a living. Everyone is a tester at heart, every day you are a tester when using Amazon, BBC News and Candy Crush – all these websites and applications that you use frequently, you are testing as a user.
For this article, I want to share my views and opinions on exploratory testing – my favourite type of software testing, which I learned back in my days as a Junior Tester at Northern Rock. The fact I was given the freedom to start my own path testing a website for unsecured loans, with no one telling me what to do, made me realise the way I approach testing was different to all my colleagues. There really is no right or wrong way to do exploratory testing, and that is what I would tell anyone working with me.
Also, remember that you can do exploratory testing at any time on a project. However, I do recommend that you do it when the product has an end-to-end journey. Even if you think the product is ready to go live, just remember:
‘Just because something works, doesn’t mean it can’t be improved.’
– Shuri (Black Panther)
What is exploratory testing?
Read into it what you wish – exploratory testing is exactly what it says it is. The keyword is ‘explore’ as in ‘go ahead, explore as much as you want.’ Exploratory testing is a specific style/approach in which testers don’t need to document what they will do. That’s the whole point. Exploratory testers don’t know what they are going to do when they start, they think as they go.
It gives testers greater freedom to explore the system in front of them. Potentially, they may find issues they wouldn’t normally find during functional testing or using automation.
What are the most common types of testing in any SDLC?
In addition to exploratory testing, there’s:
Functional testing – Where the primary objective is to ensure the acceptance criteria of a requirement is met. This is the main type of testing in any project.
Destructive testing – Where the primary objective is to perform negative tests around the acceptance criteria to try and break the system.
Non-functional testing – This includes load, performance, security, stress, accessibility and usability testing.
What is the difference between exploratory testing and other types of testing?
Exploratory testing does not focus on requirements. Instead, it focuses your mind, testing what you want on the system/application in the scope of your sprint/project.
My approach to exploratory testing
With functional/destructive tests focusing on meeting the acceptance criteria/user stories, we encourage that functional/destructive testing be automated, if deemed repeatable. If it’s a test that adds value to be run again in the future, we will automate it.
Each requirement in scope will go via an automation feasibility session, where we ask, ‘Is this test case adding value to our existing automation pack?’ or ‘Does this impact existing automation in any way?’ Assuming all requirements have gone through this process, the test team specifies the scenarios for automation. They hand this over to the development team for creation, maintenance and execution. This will then allow the tester(s) to focus on exploratory testing. This has been proven as a very effective way of finding bugs in code as it involves scenarios that haven’t been discussed in sprint planning or with members of your team.
When should we do exploratory testing?
This is a question businesses ask a lot. I strongly recommend planning exploratory testing into every project/sprint. This allows the tester(s) freedom to try ad-hoc scenarios and find undiscovered defects that automation will not come across.
Ideally, the tester should set aside 30 min sessions during a sprint to perform exploratory testing daily (or a few times a week).
You can execute it in parallel with functional and destructive testing, as long as you aren’t changing the system dates in the test environment.
What documentation do you need for exploratory testing?
No, exploratory testing is informal and doesn’t require prep or scripts, but you’ll need to document bugs. As I explained above, you should not know exactly what you are going to do until you start. My approach is to have a blank piece of paper and write down what I’m doing as I go. This helps avoid duplication in the future.
- If testers find bugs they need to know what they did.
- To do this, they can create a script in a test management tool, documenting what they did and what they found.
- If you or your tester finds a bug – make sure it is documented clearly, showing how to recreate it, including evidence of the failure using a screenshot where possible.
I want you to think about the benefits of exploratory testing for your project and business.
I am not saying other types of software testing are not as valuable or important. If anything, I would say exploratory testing will benefit the other phases of testing. It could find something functional testing didn’t find because the tester thought of a different approach to a requirement on an ad-hoc basis.
Why don’t we give the testers 30 mins, a few days a week to enjoy testing an application without following instructions? See what happens! You may save your project a lot of pain in the future.
Testers are hungry to find ‘ghosts in the machine’ – Dr. Lanning (iRobot).
Let them use their minds rather than follow a script all the time.
While you’re at it, check out Jon’s other article on the ‘Three Amigos’ delivery method here.