There are many reasons to do automated testing on the GUI level. Automated tests are fast, repeatable, and (hopefully) provide reliable test results. On the long run, they might be even cheaper than manual testing (the only alternative for GUI testing). Done the right way, you can even integrate them in your build system giving you the final verdict that each build is as you expect it to be.
One Can’t Go Wrong With Test Automation, Right?
All that sounds very promising. However, it’s quite easy to waste all these beautiful advantages of test automation by having a test suite of poor quality:
- If your automated test suite is fragile and breaks every second time it is executed, testing becomes annoying quickly. And what is the benefit of a test suite that is unreliable? Would you trust the outcome of such a test suite?
- If your test cases are labor intensive to change, you will slow your development pace. Put in different way: The only way to keep your test suite updated with the same speed as you develop new/changed features is to spend more effort in changing your test suite. And soon, you will ask yourself “Why am I spending so much effort in test automation? What has my automated test suite ever done for me?”
Those are all stories we heard not only from our customers, but also experienced painfully ourselves by doing test automation at Qualicen. However, as we are Qualicen and quality analysis is our daily business, we are experimenting for a while now with finding those kinds of quality defects in test scripts automatically. Since we are using Ranorex for GUI testing of our own product Scout, we focused on Ranorex test suites (so far).
Well, the idea of performing static analysis to identify quality defects is not new – at least not for us. Our product Qualicen Scout does exactly that for requirements specifications and manual tests. Scout is a centralized server application that, once connected to your document repository (GIT, SVN, HP ALM/VSTS, Integrity ALM, IBM Doors NG, …), analyzes your documents. It points out quality issues in your requirements/tests and provides smart ways to give you an overview of your document quality (just have a look on our cool dashboards).
Introducing Our Qualicen Ranorex Plug-In
However, for Ranorex, we wanted to try something different. Compared to Scout, we wanted to check Ranorex test cases on the client side so we can deliver instant feedback for test engineers and provide a deep integration of the results into Ranorex Studio (the tool that is used by test engineers to develop test cases). Our vision was to provide feedback to test engineers instantly, as soon as they press a key on their keyboard. And we managed to do that. Well … almost: You have to click on save first ;-).
The outcome is our Qualicen Plugin for Ranorex. It looks quite cool, doesn’t it?
Towards Test Automation Smells:
So what does our Ranorex plug-in actually do? Here is a list of quality issues it is able to detect (or Test Automation Smells as we are calling them):
- Disabled steps – Why do we need these steps at all? They clutter the tests and make them harder to read. Kick them out!
- Super long modules – Why not just break them up in shorter, easier to understand versions? Naming matters as well!
- Unused modules – Keep your test suite clean and remove unused recording modules and module groups.
- Poor UI element identification – Keep your UI repositories in shape. Unstable xpath identifier will make your test execution break often.
- Absolute coordinates for clicks – What will happen if the size of the target widget changes? Clicking on “center” is more likely to hit what you want.
- Recoding modules without validational steps – No validation in testing? What are they doing at all?
- Clones in recording modules and test cases/modules – coming soon 😀
- And many more …
You like what you heard? Why don’t you just give it a try right now? You can download the plugin here:
It is free, however, we are curious what you think about it. So don’t just use our free tools, but help us to improve it, so we can release an even better version soon.
Long story short …
- Test automation is a great tool to improve quality. However, if you do it the wrong way, it will cost you more than you will gain.
- Many typical quality issues can be automatically detected using our Qualicen Plugin for Ranorex.
- You can download this plugin for free from our website.
- Why don’t give it a try right now?