Exploratory Testing – by Jon Hall, Test Manager at Eliga Services
Eliga's favourite tester, Jon Hall makes a compelling case for exploratory testing, sharing his perspective.
What do you think of when you hear ‘testing’ on a project? I am of course referring to software testing; the testing of code; where the primary objective is to ensure the quality of the software under development. I have found many companies sigh at the thought of testing. Project managers often say ‘always the last phase of a project.' This is often the biggest risk to projects going live on time.
'My approach to testing is simply to fail fast.'
-Jon Hall (Eliga Services)
Testing should be included at the beginning of any project, with testing input classed as important as any other area in the software delivery lifecycle (SDLC).
Many test managers will have their own interpretation of testing; call me ‘old school’ as this is how I always describe testing to anyone when they ask me what I do for a living. Everyone is a tester at heart, every day you are a tester when using Amazon, BBC News and Candy Crush – all these websites and applications that you use frequently, you are testing as a user.
For this article, I want to share my views and opinions on exploratory testing – my favourite type of software testing, which I learned back in my days as a Junior Tester at Northern Rock. The fact I was given the freedom to start my own path testing a website for unsecured loans, with no one telling me what to do, made me realise the way I approach testing was different to all my colleagues. There really is no right or wrong way to do exploratory testing, and that is what I would tell anyone working with me.
Also, remember that you can do exploratory testing any time on a project. However, I do recommend that you do it when the product has an end-to-end journey. Even if you think the product is ready to go live, just remember:
'Just because something works, doesn’t mean it can't be improved.'
– Shuri (Black Panther)
What is exploratory testing?
Read into it what you wish – exploratory testing is exactly what it says it is. The keyword is ‘explore’ as in ‘go ahead, explore as much as you want.' Exploratory testing is a specific style/approach in which testers don’t need to document what they will do. That’s the whole point. Exploratory testers don’t know what they are going to do when they start, they think as they go.
It gives testers greater freedom to explore the system in front of them, and potentially find issues they wouldn’t normally find during functional testing or using automation.
What are the most common types of testing in any SDLC?
In addition to exploratory testing, there's:
- Functional testing – where the primary objective is to ensure the acceptance criteria of a requirement is met. This is the main type of testing in any project.
- Destructive testing – where the primary objective is to perform negative tests around the acceptance criteria to try and break the system.
- Non-functional testing – this includes load, performance, security, stress, accessibility and usability testing. All of which are types of test to meet the acceptance criteria of non-functional requirements.
What is the difference between exploratory testing and other types of testing?
Exploratory testing is not focused on requirements. Instead, it focuses your mind, testing what you want on the system/application in the scope of your sprint/project.
My approach to exploratory testing
With functional/destructive tests focusing on meeting the acceptance criteria/user stories, we encourage that functional/destructive testing be automated, if deemed repeatable. If it’s a test that adds value to be run again in the future, we will automate it.
Each requirement in scope will go via an automation feasibility session, where the key questions will be asked, including ‘Is this test case adding value to our existing automation pack?’ or ‘Does this impact existing automation in any way?’ Assuming all requirements have gone through this process, the test team specifies the scenarios to be automated and hand this over to the development team for creation, maintenance and execution. This will then allow the tester(s) to focus on exploratory testing. This has been proven as a very effective way of finding bugs in code as it involves scenarios that haven’t been discussed in sprint planning or with members of your team.
When should we do exploratory testing?
This is a question I get asked a lot by businesses. I strongly recommend planning exploratory testing into every project/sprint, allowing the tester(s) freedom to try ad-hoc scenarios and find undiscovered defects that automation will not come across.
Ideally, the tester should set aside 30 min sessions during a sprint to perform exploratory testing daily (or a few times a week).
It can also be executed in parallel with functional and destructive testing, as long as you aren’t changing the system dates in the test environment.
Is any documentation needed for exploratory testing?
No, exploratory testing is informal and doesn’t require prep or scripts, but you’ll need to document bugs. As I explained above, you should not know exactly what you are going to do until you start. My approach is to have a blank piece of paper and write down what I’m doing as I go. This helps avoid duplication in the future.
If testers find bugs they need to know what they did.
To do this, they can create a script in a test management tool, documenting what they did and what they found.
If you or your tester finds a bug – make sure it is documented clearly, showing how to recreate it, including evidence of the failure using a screenshot where possible.
I want you to think about the benefits of exploratory testing for your project and business.
I am not saying other types of software testing are not as valuable or important. If anything, I would say exploratory testing will benefit the other phases of testing. It could find something functional testing didn’t find, because the tester thought of a different approach to a requirement on an ad-hoc basis.
Why don’t we give the testers 30 mins, a few days a week to enjoy testing an application without following instructions? See what happens! You may save your project a lot of pain in the future.
'Give testers the freedom to do what they do best!'
-Jon Hall (Eliga Services)
Testers are hungry to find 'ghosts in the machine' - Dr. Lanning (iRobot). Let them use their minds rather than following a script all the time.