System Test Tag

Structuring system test suites – antipatterns and a best practice

Typically, you have your test suite structured in a hierarchic way to keep it organized. The way you structure your system test suite has a considerable impact on how effective and efficient you can use your tests. A good structure of a system test suite supports:

maintaining tests when requirements change
determine which part of your functionality has been tested, and to which degree (coverage)
finding and reusing related tests while creating new tests
selecting a set of test cases to execute (test plan)
finding the root cause of a defect (debugging)

In my opinion, the first two points are the most important ones, as they touch the core of what system tests should do, namely to ensure that the system fulfills its requirements. Of course each project is different and no matter which structure I choose, I always run into the "tyranny of the dominant decomposition" (i.e. there is no such thing as THE best way to build a hierarchy) in the end. Nevertheless, I have seen a couple of anti-patterns in the past, which only very rarely make sense.

Aug

Test Engineering Test Quality Test Smells Ambiguity Natural Language Quality System Test

Conditionals: Why you should avoid these two letters for better test case execution

At Qualicen, it's often my job to check other people's system test cases and tell the team what I think about these tests. So what do I look for? Well, in principle it is simple: After tests are written down, they are "only" executed and maintained. So this is where tests can be bad and I try to spot things that make execution and maintenance harder. For the maintenance, the largest problem here are clones, which we covered in our last blog post. For the test execution, the main problem that you want to avoid is that different testers test different things. This is called ambiguity and comes in many tastes. In this blog post, I want to explain what is structural ambiguity and why it is bad, and this way help you to create better test cases. (Scroll to the summary, if you don't care about the details) ;) Ambiguous test flow The problem for test execution that I want to discuss here, is an ambiguous test flow. This means, that for a single test case, there are multiple paths that a tester can follow when she executes the test. Let’s look at an example. [caption id="attachment_145" align="alignleft" width="810"] A simple, straight-forward natural language system test case.[/caption]