Quality Plug-In for Ranorex

There are many reasons to do automated testing on the GUI level. Automated tests are fast, repeatable, and (hopefully) provide reliable test results. On the long run, they might be even cheaper than manual testing (the only alternative for GUI testing). Done the right way, you can even integrate them in your build system giving you the final verdict that each build is as you expect it to be.

One Can’t Go Wrong With Test Automation, Right?

All that sounds very promising. However, it’s quite easy to waste all these beautiful advantages of test automation by having a test suite of poor quality:

  • If your automated test suite is fragile and breaks every second time it is executed, testing becomes annoying quickly. And what is the benefit of a test suite that is unreliable? Would you trust the outcome of such a test suite?
  • If your test cases are labor intensive to change, you will slow your development pace. Put in different way: The only way to keep your test suite updated with the same speed as you develop new/changed features is to spend more effort in changing your test suite. And soon, you will ask yourself “Why am I spending so much effort in test automation? What has my automated test suite ever done for me?
Example of a fragile test suite

Example of a fragile test suite: Absolute URLs will make your test suite fail as soon as you deploy your application on a different server.

Quality Matters

Those are all stories we heard not only from our customers, but also experienced painfully ourselves by doing test automation at Qualicen. However, as we are Qualicen and quality analysis is our daily business, we are experimenting for a while now with finding those kinds of quality defects in test scripts automatically. Since we are using Ranorex for GUI testing of our own product Scout, we focused on Ranorex test suites (so far).

Well, the idea of performing static analysis to identify quality defects is not new – at least not for us. Our product Qualicen Scout does exactly that for requirements specifications and manual tests. Scout is a centralized server application that, once connected to your document repository (GIT, SVN, HP ALM/VSTS, Integrity ALM, IBM Doors NG, …), analyzes your documents. It points out quality issues in your requirements/tests and provides smart ways to give you an overview of your document quality (just have a look on our cool dashboards).

Introducing Our Qualicen Ranorex Plug-In

However, for Ranorex, we wanted to try something different. Compared to Scout, we wanted to check Ranorex test cases on the client side so we can deliver instant feedback for test engineers and provide a deep integration of the results into Ranorex Studio (the tool that is used by test engineers to develop test cases). Our vision was to provide feedback to test engineers instantly, as soon as they press a key on their keyboard. And we managed to do that. Well … almost: You have to click on save first ;-).

The outcome is our Qualicen Plugin for Ranorex. It looks quite cool, doesn’t it?

The dashboard gives you a nice overview of the state of your test suite.

For each file, the plug-in gives you individual feedback how to make improvements.

Towards Test Automation Smells:

So what does our Ranorex plug-in actually do? Here is a list of quality issues it is able to detect (or Test Automation Smells as we are calling them):

  • Disabled steps – Why do we need these steps at all? They clutter the tests and make them harder to read. Kick them out!
  • Super long modules – Why not just break them up in shorter, easier to understand versions? Naming matters as well!
  • Unused modules – Keep your test suite clean and remove unused recording modules and module groups.
  • Poor UI element identification – Keep your UI repositories in shape. Unstable xpath identifier will make your test execution break often.
  • Absolute coordinates for clicks – What will happen if the size of the target widget changes? Clicking on “center” is more likely to hit what you want.
  • Recoding modules without validational steps – No validation in testing? What are they doing at all?
  • Clones in recording modules and test cases/modules – coming soon 😀
  • And many more …

Sounds Interesting?

You like what you heard? Why don’t you just give it a try right now? You can download the plugin here:

Qualicen Plug-In for Ranorex

It is free, however, we are curious what you think about it. So don’t just use our free tools, but help us to improve it, so we can release an even better version soon.

Long story short …

  • Test automation is a great tool to improve quality. However, if you do it the wrong way, it will cost you more than you will gain.
  • Many typical quality issues can be automatically detected using our Qualicen Plugin for Ranorex.
  • You can download this plugin for free from our website.
  • Why don’t give it a try right now?
4 Comments

    Serge Fournier

    Hello Qualicen,

    I really like your new plugin for Ranorex; I think it’s a very useful tool for improving Ranorex modules.

    Regarding the calculation of the validation ratio, I think it would be relevant to implement an option so that the Ranorex actions “Wait for Exists” or “Wait for Not Exists” are considered validations. Because both actions use validation to verify that a specific repository item exists or not. For us, those validations are very similar to actions “Validate Exist” and “Validate Not Exists”.

      Benedikt Hauptmann

      Hello Serge,

      I’m glad to hear that you like our plugin.

      You are absolutely right: “Wait for Exists” and “Validate Exist” (resp. “… Not Exists”) fulfill a similar purpose. Both can be used to verify the correct behavior of your SUT. We should definitely adapt our smell detection to consider both variants.

      Thank you very much for this valuable feedback. Please keep us updated with further thoughts/ideas. We always like to improve our plugin. 🙂

    Raghu

    Hello Qualicen,

    Thanks for this post and plugin.

    I would like to use your plugin for my test automation project but I have few questions.

    1. As per blog post, it will work for recording modules. Will it also work on Ranorex code modules?
    2. If I use your plugin for my code, will you be keeping my code or code evaluation report as a record on your server?
    3. Last question 🙂 , can I use it for commercial purpose or enterprise level?

    Regards,
    Raghu

      Benedikt Hauptmann

      Hello Raghu,

      Thank you for using our plugin. It’s nice to hear that you like it. I’ll give my best answering your questions:

      1. There are plenty of great source code analysis tools available out there (e.g. https://teamscale.eu). However, test automation is not just source code. So far, there was no similar tool for all the other parts of a test suite: Such as the GUI repository, recording modules, test cases, …. We do not aim at creating yet another source code analysis tool. However, maybe in future, we will integrate our plugin into an existing tool. If you have a favorite, let us know.
      2. Don’t worry, we are sending just telemetry data to our server, such as, your Ranorex version, the version of your plugin and a GUID that is generated once during installation (to avoid counting duplicates). This information is used just for marketing research. For example, it’s important to know which Ranorex versions are used the most in order to keep compatibility. And if you still don’t like it, you can switch off this feature completely.
      3. Go for it! You can use our plugin for commercial purposes and on as many computers as you like. But don’t forget to give us feedback :-).

      Let me know if I missed some parts of your questions.

      Regards,
      Benedikt

Leave a Reply