There are many reasons to do automated testing on the GUI level. Automated tests are fast, repeatable, and (hopefully) provide reliable test results. On the long run, they might be even cheaper than manual testing (the only alternative for GUI testing). Done the right way, you can even integrate them in your build system giving you the final verdict that each build is as you expect it to be.
One Can’t Go Wrong With Test Automation, Right?
All that sounds very promising. However, it’s quite easy to waste all these beautiful advantages of test automation by having a test suite of poor quality:
- If your automated test suite is fragile and breaks every second time it is executed, testing becomes annoying quickly. And what is the benefit of a test suite that is unreliable? Would you trust the outcome of such a test suite?
- If your test cases are labor intensive to change, you will slow your development pace. Put in different way: The only way to keep your test suite updated with the same speed as you develop new/changed features is to spend more effort in changing your test suite. And soon, you will ask yourself “Why am I spending so much effort in test automation? What has my automated test suite ever done for me?”
Example of a fragile test suite: Absolute URLs will make your test suite fail as soon as you deploy your application on a different server.
Continue reading Quality Plug-In for Ranorex
When we look at requirements documents that are new to us, we often need some help on terms and abbreviations. Creating a glossary to explain these imporant domain terms and abbreviations is a fine idea. It helps new team members to get going, improves the readability of a requirements specification and helps to avoid misunderstandings. The main problem with glossaries is that we create them once and update them only rarely. In consequence, the majority of glossaries are not particulary useful. In this article, Qualicen consultant Maximilian Junker shows how you can get more out of your glossary and keep it always up-to-date.
Continue reading How to Awake Your Glossary From Zombie Mode
As most of you know, we moved to the GATE in Garching (a university town, 20 minutes out of Munich) recently. So in oder to celebrate our new offices, I wanted to share a few pictures from these days with you.
We moved here in December. Engineers as we are, we loved all the assembling! And not too many things broke, actually 😉
First desks assembled
It’s getting there!
We have the greatest lamps.
Continue reading Moving in at GATE Garching (with Pictures!)
If you haven’t heard from us at Qualicen in while it, it is because we are fortunately(!) very busy right now. Lot’s of cool projects all over Germany and even up in Sweden. Contact us, if you would like to hear more about these projects or get in contact at one of the following venues.
Continue reading Updates from Qualicen
Several roles are concerned with requirements quality. Of course, there is the requirements author, writing the requirements. But there is also the reviewer, who proof-reads and validates the requirements. And finally, there is the QA-Engineer, responsible for the overall quality of all artifacts created during the engineering process. Each of these roles needs a different view on requirements and different tools in order to do their work efficiently and achieve a high requirements quality. In this article I am going to show you how the Qualicen products specifically support authors, reviewers and QA-engineers in their work to keep requirements quality high.
Continue reading Three Perspectives on Requirements Quality: Authors, Reviewers, QA-Engineers
I’ve worked quite some time on understanding and detecting quality defects in requirements documents and requirements quality in general. All the time, I was very dissatisfied with the current state in both research and practice on this topic. I think, the problem behind this is that there is no guidance: In times of rapid change and delivery, where every project looks different, we still have no good rule of what a good requirements document is.
A few years ago, we came up with such a rule, and tried it in various applications. And – so far – it seems to work! We’ve collected this experience and are now ready to tell you about it, because we really believe this should change how you view requirements engineering, and this should change what you consider good requirements documents.
Continue reading Requirements quality is quality-in-use
Team Foundation Server (TFS) and its software-as-a-service counterpart Microsoft Visual Studio Team Services (VSTS) are widely used application lifecycle management (ALM) and test management tools. They offer many great facilities to create tests, manage test plans, and execute them. Consequently, many of our clients as well as prospective customers wanted to use our test improvement software Test Scout along with TFS/VSTS to improve the test case quality. So, here is the question that we always face: How do we get the data from a testing tool into the Test Scout? As always in life, there is a straightforward and a fancy solution. Let me show you what I mean.
First: a simple integration
Test Scout is able to process almost any kind of text format. So, integrating test management tools such as TFS/VSTS is quite straightforward: For each test management tool, we created exports, which we imported into the Scout. For HP ALM, for example, we use a simple script to create a database dump containing all currently existing test cases. We then automatically imported and processed this data in Test Scout to evaluate the test case quality. Since Test Scout keeps versions of each import in its database, the history of all test cases is available in Test Scout. Therefore, all features, such as comparing different versions of test cases and historical development of test cases still works out of the box.
Continue reading Automatic Feedback on TFS/VSTS Test Case Quality in Real Time
For high requirements quality we need quality assurance. In a previous post, I explained why automatic methods cannot replace manual methods. Instead I suggested to combine both worlds. And the ugly truth is, in both system testing and requirements engineering, we need both manual and automatic quality assurance to control requirements quality and test quality. Now you wonder, how? I got you covered. In this brief post, I want to point out how you can combine the two worlds and how you benefit from the combination.
Continue reading Efficiently control requirements quality: the best of two worlds
Auf dem QS-Tag in Nürnberg habe ich heute unsere Erfahrungen mit der automatisierten Erkennung von Problemen in Anforderungen vorgestellt. Das Feedback war absolut positiv. Hier die Folien des Vortrags. Die Kurzfassung des Vortrags und die Reaktionen der QS-Tag Besucher:
Qualitätsprobleme in Anforderungen führen zu echten Problemen und unnötigen Kosten in der Software-Entwicklung. Manuelle Reviews sind wichtig, aber aufwändig und langwierig. Sind automatisierte Reviews das Allheilmittel? Alleine sicher nicht. Aber mit einer sinnvollen Kombination von automatischen Analsen und manuellen Reviews bekommen wir das beste aus beiden Welten: Schnelles Feedback durch automatische Analysen und tiefes inhaltliches Feedback durch manuelle Reviews.
Erkennen können wir dennoch einige Probleme. Das hat die Erfahrung, auch mit Munich Re, gezeigt. Sprachliche Probleme, den falschen Abstraktionsgrad, strukturelle Inkonsistenten und unnötige Klone sind alles Mängel, die wir automatisch angehen können. Bei der Bewertung der Vollständigkeit von Anforderungen oder dem richten Anforderungsschnitt beißen sich automatische Analysen heute jedoch noch die Zähne aus.
Um die Ergebnise möglicht effektiv aufbereiten zu können brauchen wir zwei Perspektiven: Eine Perspektive für die Anforderungsautorin, die möglichst sofort während des Schreibens mögliche Defekte anzeigt. Und eine andere Perspektive für den Quality-Engineer, der den Überblick über die Qualität vieler verschiedener Dokumente bewahren will.
Spannende Frage aus dem Publikum vor und nach dem Vortrag waren: Wie können wir sicherstellen, dass durch automatische Analyseergebnisse niemand bloßgestellt wird? Und eine zweite spannende Frage: Wie können wir die Perspektive der Tester in die automatisierte Erkennung bringen? Wie können wir sicherstellen, dass Tester und Anforderungsautoren das gleiche Verständnis entwickeln? Brauchen wir dazu spezielle Analysen? Oder eine spezielle Darstellung? Oder beides? Meine Antworten: Sicher Thema der nächsten Blog-Artikel!
Why use Word for Requirements Documentation?
… you might ask. Usually, I strongly encourage our customers to use suitable tools for managing their requirements. There are plenty of reasons why I recommend using professional tools for requirements management, but an important reason is to ensure requirements traceability. For example, of the six sources of RE project failures identified by Michelle Boucher, at least three have to do with traceability.
While we do recommend other tools, Word is available in most companies, documents are easy to exchange, the review mode is pretty good and sometimes using other tools is simply not an option for different reasons. Hence, many people do use Word to create requirements documents. My own experience in using Word for authoring a requirements document is that it seems to work well in the beginning, but as soon as already documented requirements begin to change things start to get nasty. Especially if there is not just one requirements document, but several. Continue reading Requirements Traceability with Microsoft Word