Which defects cannot be detected, taken from H. Femmer, D. Méndez Fernández, S. Wagner, and S. Eder, “Rapid quality assurance with Requirements Smells,” J. Syst. Softw., 2015.

The ugly truth about automatic methods for requirements engineering quality.

Ok, after I publish this blog post, I will probably get some angry calls from my sales department… Well, truth must be told.

There are many crazy defects in requirements, and, as I wrote in my last post, you can detect quite a bunch of them automatically (and you should do so!). When I present our automatic methods for natural language requirement smells or automatic methods for detecting defects in tests to our customers, I’m proud to say that they are usually very excited. Sometimes they are too excited and then this can turn into a problem.

What I mean is that I explain all the amazing things that you can detect with tools and suddenly people think that the tool will solve all the problems that they face. Spoiler alert: It doesn’t. And because we’re a company that is interested in happy customers, I want to briefly summarize all the problems (*that come into my mind) that a tool can’t solve. And because I don’t want to leave you in despair, I will also suggest some solutions, how I personally would suggest to work on that problem. Continue reading The ugly truth about automatic methods for requirements engineering quality.

Structuring system test suites – antipatterns and a best practice

Typically, you have your test suite structured in a hierarchic way to keep it organized. The way you structure your system test suite has a considerable impact on how effective and efficient you can use your tests. A good structure of a system test suite supports:

  • maintaining tests when requirements change
  • determine which part of your functionality has been tested, and to which degree (coverage)
  • finding and reusing related tests while creating new tests
  • selecting a set of test cases to execute (test plan)
  • finding the root cause of a defect (debugging)

In my opinion, the first two points are the most important ones, as they touch the core of what system tests should do, namely to ensure that the system fulfills its requirements.
Of course each project is different and no matter which structure I choose, I always run into the “tyranny of the dominant decomposition” (i.e. there is no such thing as THE best way to build a hierarchy) in the end. Nevertheless, I have seen a couple of anti-patterns in the past, which only very rarely make sense.

Continue reading Structuring system test suites – antipatterns and a best practice