In my previous blogs and over the years, I already stated how complicated, demanding and challenging is the mobile space, therefore it seems obvious that there needs to be a structured method of building test automation and meeting test coverage goals for mobile apps.
While there are various tools and techniques, in this blog I would like to focus on a methodology that has been around for a while but was never adopted in a serious and scalable way by organizations due to the fact that it is extremely hard to accomplish, there are no sufficient tools out there that support it when it comes to non-proprietary open-source tools and more.
First things first, let’s define what is a Model-Based testing
Model-based testing (MBT) is an application for designing and optionally executing artifacts to perform software testing or system testing. Models can be used to represent the desired behavior of a System Under Test (SUT), or to represent testing strategies and a test environment.
Fig 1: MBT workflow example. Source: Wikipedia
In the context of mobile, if we think about an end-user workflow with an application it will usually start with a Login to the app, performing an action, going back, performing a secondary action and often even a 3rd action based on the previous output of the 2nd. Complex Ha
The importance of modeling a mobile application serves few business goals:
- App release velocity
- App testing coverage (use cases)
- App test automation coverage (%)
- Overall app quality
- Cross-team synchronization (Dev, Test, Business)
As already covered in an old blog I wrote, mobile apps would behave differently, and support different functionality based on the platform they are running (E.g., not every iOS device support both Touch ID and/or 3D Touch gesture). Therefore, being able to not only model the app and generate the right test cases but also to match these tests across the different platforms can be a key to achieving many of the 1-5 goals above.
In the market, today there are various commercial tools that can assist in MBT like CA Agile Requirement Designer, Tricentis Tosca, and others.
Looking at an example provided by one of the commercial vendors in the market (Tricentis), it can show a common workflow around MBT. A team aims to test an application; therefore, they would scan it using the MBT tool to “learn” its use cases, capabilities, and other artifacts so they can stack these into a common repository that can serve the team to build test automation.
Fig 2: Tricentis Tosca MBTA tool
In Fig 2., Tricentis examines a web page to learn its entire options, objects, and other data related items. Once the app is scanned, it can be easily converted into a flow diagram that can serve as the basis for test automation scenarios.
With the above goals in mind, it is important to understand that having an MBT tool that serves the automation team is a good thing, but only if it increases the team efficiency, its release velocity, and the overall test automation coverage. If the output of such tool would be a long list of test cases that either does not cover the most important user flows, or it includes many duplicates than this wouldn’t serve the purpose of MBT but rather will delay existing processes, and add unnecessary work to teams that are already under pressure.
In addition to the above commercial tools, there is an older but free tool that allows Android MBT with robotium called MobiGuitar. This tool not just offers MBT capabilities but also code coverage driven by the generated test scripts.
A best practice in that regards would be to probably use an MBT tool that can generate all the application related artifacts that include the app object repository, the full set of use cases, and allow all of that to be exported to leading open-source test automation frameworks like Selenium, Appium, and others.
Mobile Specific MBT – Best Practices and Examples
Drilling down into a workflow that CA would recommend around MBT would look as follows – In reality, the below is easier said than done for a Mobile App compared to Web and Desktop:
- The business analysts will create the story using tools like CA Agile Requirement Designer or such (see below more examples)
- The story is then passed to an ALM tool (e.g.: CA Agile Central [formerly Rally], Jira, etc.) for project tracking
- Teams use the MBT tools to collaborate
- The automation engineer adds the automation code snippets to the nodes where needed or adds additional nodes for automation.
- The programmer updates the model for technical specs or more technical details.
- The Test Data engineer assigns test data to the model
- Changes to the story are synchronized with the ALM Tool
- Test cases are synchronized with the ALM Tool
- The programmer completes coding
- The code is promoted from Dev to QA
- Testing begins
- The tester uses the test cases with test data from MBT tools for manual test case execution
- The automation scripts with test data are executed for functional and regression testing
To learn more about efficient MBT solutions, practices please refer to these sources:
- How MBT is used at Spotify: http://www.cs.tut.fi/tapahtumat/testaus12/kalvot/Karl_ta.pdf
- fMBT open source: https://github.com/01org/fMBT
- ORBIT project: https://sites.google.com/site/asergrp/projects/orbit
- Experiences of System-Level Model-Based GUI Testing of an Android Application- http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.453.4688&rep=rep1&type=pdf
3 Replies to “Model-Based Testing and Test Impact Analysis”
An another open source tool to add – Graph walker.
Reblogged this on @rm@te The Tester and commented: