I’m going to keep this post quite short. It’s a thought I’ve been having for a while now.
Here goes with the impact statement: Test Techniques are actually just activities that relate to some context of a testing mission.
Think about it…
Take “load testing”, or “accessibility testing”, or “function testing”, or “security testing”, or any of the other 120 different test techniques (which Dan North mentioned in his keynote talk at the Agile Testing Days conference last year). They all relate to a specific purpose of testing. A mission for that test… The “technique” is just a label for the activity.
Initially, when I heard Dan mention about there being over 120 different techniques, I thought “Wow! I’d be pushed to be able to name 50…”. But taking the terminology of each technique out of the equation and thinking about the activity relating to whatever the context of each of your specific testing missions, 120 seems like quite a low number.
So, I would say that at the heart of being a good tester is the ability to take the software that you are testing and think laterally about the many different types of test missions that are relative to the software and that are important to your stakeholders, rather than try and run through a set list of “test techniques”.
Dan Ashby 
Hi Dan,
I think I disagree… at least to some extent…
To me, a test technique is related to “how” you choose to design your tests. What is your angle of attack? Where do you choose to focus? In some/many cases the line might not be crystal clear, while some times its more obvious. Many times it is not simply one technique, but several, or call it a hybrid if you like.
If I tell you to do load testing, is it obvious how you are going to choose how to generate the load? What you are trying to (over)load? The pattern/content within the load?
Same for security testing, is it only one technique? Only one way to trigger or unveil weaknesses?
On a high level I may ask you to test “a product”. Testing is (depending on your favourite definition) uncovering information about the product. Now, applying your knowledge about the field of security issues means “attacking” the product in one or more ways (I’m no sec.test expert but I’ve heard about XSS-vulnerabilities, authentications issues, privilege escalation, SQL-injection, etc)… I think these various technical methods of searching for specific information qualify as techniques. Performing any or all of them constitutes security testing in general.
You can test the workflow in an application, using many or all of its function, focusing on the end to end experience.. or you can go through all the functions/features one by one. Somewhat similar, but different focus, looking for different information.
And in both cases, what technique do you use to choose the input data, how do you choose your oracles?
So… I think, after writing this… that I would suggest that when you refer to the “activity” this is a general, less specific, description of what you do based on the approach(es) and technique(s) you have already decided to apply. If you are e.g. a specialised security tester, then you have already narrowed down the techniques you carry in your toolbox, and the information you are likely to find. And obviously, the activities you are doing are “security testing”.
Did any of that make sense?
Thanks for making me think,
Geir
Ps. Have you had a look at the BBST Test Design course? You might find it interesting, although it might not give you a crystal clear answer either 🙂
http://www.testingeducation.org/BBST/testdesign/
http://www.associationforsoftwaretesting.org/training/courses/
LikeLike
Hi Geir!
Thanks for the comment.
So you use the technique as an instigator for designing your tests? That’s an interesting perspective! I guess we all do that to a degree for some of our tests.
In a way, I think this is how I used to focus all of my testing when I first started out as a tester. I would look at each technique and would plan and design my testing around each technique.
I think this falls down in some areas though. Especially with session based exploratory testing where you have a set charter or mission that might not be focusing on one technique.
Your comment has definitely got me thinking though!! It definitely isn’t as black and white as I was perhaps making out on my post.
And thank YOU for making me think! 🙂
(oh, and for the links too!)
Dan
LikeLike
I think I agree with you, but perhaps not with Dan North’s definition of a test technique. The Oxford English dictionary defines a technique as:
“A way of carrying out a particular task, especially the execution or performance of an artistic work or a scientific procedure: new surgical techniques mean a shorter hospital stay the techniques used by Turner, Rembrandt, and Degas”
So unless you consider your ‘task’ to be testing, and that feels very broad, then I don’t think something like performance testing can be considered a test technique. Test techniques are useful to know because they help us to understand and evaluate a system.
If I have a test charter to test a product and I’m considering using techniques such as accessibility, performance, security, usability etc then I will probably build up a very broad but shallow view of the system. If instead I have a test charter to understand the security of the system then I can turn to my test techniques and apply SQL injection, fuzzing, tampering, etc to try to build up the largest possible picture of the system.
LikeLike
Hi Amy!
Thanks for the comment!
I like the dictionaries definition. It does highlight that techniques are just activities.
And I would agree with you that performance testing shouldn’t be considered a technique, but I think the elements within “performance” testing might be, (as in; load testing, stress testing and speed testing). These are the activities associated with performance, so might merit the label “technique”.
You have got me thinking with your last sentence – should the technique be your mission rather than it being the activity that relates to your mission?
One thing that confuses me regarding this though: are each of your testing charters simply a technique?
I’m not sure this would work so well… Take specifically Security Testing. If you had one charter saying “security testing”, then this seems too high level as there is so much involved in security testing. Certainly the charter should take into consideration the test technique relating to a functional area… That sounds more like a goal of that testing session. 🙂
Thanks!
Dan
LikeLike
Yes I think you’re right that load testing, stress testing and speed testing are all techniques. Although it’s interesting that we have stress testing as a technique to form part of performance testing but then something like fuzzing to form part of security testing. It seems that some types of testing break down far more than others!
I think test charters should specify a goal rather than a task or technique. It doesn’t really matter which techniques you use as long as you achieve the test session goal. Maybe it is the limited time period of a test charter that actually makes you want to define tasks that are more focused than ‘testing’.
Lots to ponder!
LikeLike
Equally I’d agree that Security testing is not a technique in itself. It is a collection of different techniques that form a part of the whole. Amy has already mentioned Fuzzing.
There is an endless list of new techniques you could learn to make you a better tester, they might include aspects of UI Testing, database testing, security testing, automating aspects of these with python or ruby.
Once you tend to learn a set of skills which get grouped under a particular banner, you also tend to get labelled as a specialist of that area of testing. I think this could be potentially misleading…as you then become an “authority” on those skills or techniques.
I don’t yet feel comfortable calling myself a security tester, but I’m getting there. But I incorporate aspects of security testing into my work. Likewise, including other techniques.
Labels are helpful in some ways as they let us define roles, specialisms and ownership of knowledge. But I think it may narrow scope for opportunities when looking for work for example.
LikeLike