We observe that numerous cyber-physical systems are rapidly gaining functionality and thus development gets more and more complex. Innovations are made possible in many areas by a complex interaction of sensor systems and software. Consider the development of autonomous driving, in which a multitude of different system functions must interact safely with one another in order to make complex decisions with the highest quality in order to transport people safely. In order to master the complexity, the classical, document-centered approaches of system engineering are no longer sufficient and are increasingly being replaced by model-based systems engineering (MBSE) approaches.
The SPES modeling framework provides a comprehensive, tool- and modeling language-independent method for MBSE. It offers a whole range of concrete models, modeling techniques and activities. In this blog post, I will introduce you gently to SPES. I will explain the basic principles of SPES and give some pointers where to find more.
Continue reading Getting Started With the SPES System Modeling Framework
In this blog post I am going to introduce Specmate, the result of a research project I have been involved into. It is an open-source tool to automate test-design, among others. This is the first post of a series in which I am going to show you some of the ideas behind Specmate.
What is test-design and why does it matter?
Test-Design is the activity to come up with the right test-cases for a piece of functionality. But what are the right test-cases? There are many criteria, depending on your focus. For me, there are two main points:
- First, they should test the right content. That means, they relate to the requirements for this functionality and cover every aspect that the requirements talk about. They should hence be able to find faults: deviations of the implementation with respect to the specification.
- Second, they should be feasible. That means, it should be possible to execute the test-cases without wasting resources.
Continue reading Structured Test-Design with Specmate – Part 1: Requirements-based Testing
When we look at requirements documents that are new to us, we often need some help on terms and abbreviations. Creating a glossary to explain these imporant domain terms and abbreviations is a fine idea. It helps new team members to get going, improves the readability of a requirements specification and helps to avoid misunderstandings. The main problem with glossaries is that we create them once and update them only rarely. In consequence, the majority of glossaries are not particulary useful. In this article, Qualicen consultant Maximilian Junker shows how you can get more out of your glossary and keep it always up-to-date.
Continue reading How to Awake Your Glossary From Zombie Mode
Several roles are concerned with requirements quality. Of course, there is the requirements author, writing the requirements. But there is also the reviewer, who proof-reads and validates the requirements. And finally, there is the QA-Engineer, responsible for the overall quality of all artifacts created during the engineering process. Each of these roles needs a different view on requirements and different tools in order to do their work efficiently and achieve a high requirements quality. In this article I am going to show you how the Qualicen products specifically support authors, reviewers and QA-engineers in their work to keep requirements quality high.
Continue reading Three Perspectives on Requirements Quality: Authors, Reviewers, QA-Engineers
Auf dem QS-Tag in Nürnberg habe ich heute unsere Erfahrungen mit der automatisierten Erkennung von Problemen in Anforderungen vorgestellt. Das Feedback war absolut positiv. Hier die Folien des Vortrags. Die Kurzfassung des Vortrags und die Reaktionen der QS-Tag Besucher:
Qualitätsprobleme in Anforderungen führen zu echten Problemen und unnötigen Kosten in der Software-Entwicklung. Manuelle Reviews sind wichtig, aber aufwändig und langwierig. Sind automatisierte Reviews das Allheilmittel? Alleine sicher nicht. Aber mit einer sinnvollen Kombination von automatischen Analsen und manuellen Reviews bekommen wir das beste aus beiden Welten: Schnelles Feedback durch automatische Analysen und tiefes inhaltliches Feedback durch manuelle Reviews.
Erkennen können wir dennoch einige Probleme. Das hat die Erfahrung, auch mit Munich Re, gezeigt. Sprachliche Probleme, den falschen Abstraktionsgrad, strukturelle Inkonsistenten und unnötige Klone sind alles Mängel, die wir automatisch angehen können. Bei der Bewertung der Vollständigkeit von Anforderungen oder dem richten Anforderungsschnitt beißen sich automatische Analysen heute jedoch noch die Zähne aus.
Um die Ergebnise möglicht effektiv aufbereiten zu können brauchen wir zwei Perspektiven: Eine Perspektive für die Anforderungsautorin, die möglichst sofort während des Schreibens mögliche Defekte anzeigt. Und eine andere Perspektive für den Quality-Engineer, der den Überblick über die Qualität vieler verschiedener Dokumente bewahren will.
Spannende Frage aus dem Publikum vor und nach dem Vortrag waren: Wie können wir sicherstellen, dass durch automatische Analyseergebnisse niemand bloßgestellt wird? Und eine zweite spannende Frage: Wie können wir die Perspektive der Tester in die automatisierte Erkennung bringen? Wie können wir sicherstellen, dass Tester und Anforderungsautoren das gleiche Verständnis entwickeln? Brauchen wir dazu spezielle Analysen? Oder eine spezielle Darstellung? Oder beides? Meine Antworten: Sicher Thema der nächsten Blog-Artikel!
Why use Word for Requirements Documentation?
… you might ask. Usually, I strongly encourage our customers to use suitable tools for managing their requirements. There are plenty of reasons why I recommend using professional tools for requirements management, but an important reason is to ensure requirements traceability. For example, of the six sources of RE project failures identified by Michelle Boucher, at least three have to do with traceability.
While we do recommend other tools, Word is available in most companies, documents are easy to exchange, the review mode is pretty good and sometimes using other tools is simply not an option for different reasons. Hence, many people do use Word to create requirements documents. My own experience in using Word for authoring a requirements document is that it seems to work well in the beginning, but as soon as already documented requirements begin to change things start to get nasty. Especially if there is not just one requirements document, but several. Continue reading Requirements Traceability with Microsoft Word
Typically, you have your test suite structured in a hierarchic way to keep it organized. The way you structure your system test suite has a considerable impact on how effective and efficient you can use your tests. A good structure of a system test suite supports:
- maintaining tests when requirements change
- determine which part of your functionality has been tested, and to which degree (coverage)
- finding and reusing related tests while creating new tests
- selecting a set of test cases to execute (test plan)
- finding the root cause of a defect (debugging)
In my opinion, the first two points are the most important ones, as they touch the core of what system tests should do, namely to ensure that the system fulfills its requirements.
Of course each project is different and no matter which structure I choose, I always run into the “tyranny of the dominant decomposition” (i.e. there is no such thing as THE best way to build a hierarchy) in the end. Nevertheless, I have seen a couple of anti-patterns in the past, which only very rarely make sense.
Continue reading Structuring system test suites – antipatterns and a best practice
I recently reviewed a manual test suite of one of our customers. One of the first things I check very early in a review is the number of clones (i.e. duplicated parts of a test suite, usually created by copy and paste). In this recent case, I discovered that nearly 70% of the test suite is duplicated. That means, when I take some arbitrary test step, the chance is 70% that the test step is a 1:1 copy of another step. At the top of the post is a tree map that visualizes the amount of clones I found. Each rectangle represents a test, the more red a rectangle is, the bigger the amount of cloning.
In my experience, cloning in test suites is the biggest problem with regard to maintainability of a test suite. Cloning causes considerable costs as the effectiveness of the test suite decreases and the effort for maintenance rockets. In this post I take a closer look on cloning in test suites. I show you an example to illustrate how clones can look like and explain where clones come from. Later, I give you good reasons why you should care about clones in tests and discusss strategies you can employ to avoid or at least deal with clones.
Continue reading Why test clones mess with your test quality – And how to avoid them