Now that we are a few weeks away from Google I/O, and we understand that the complex Android landscape is becoming, even more, complex let’s explore a way Android teams can optimize and plan their test automation across the different platforms and devices.
In the past, I’ve written about the need to connect the 3 layers:
- Application under test
- Test code itself
- Device/OS under test
I related back to my old patent that I jointly submitted years ago in the days of J2ME and also wrote a chapter about it in my newly published book (The Digital Quality Handbook)
Problem Definition
Android OS families support different capabilities and the gap is growing from one Android SDK to the next. As an example, Android devices older than 6.0, cannot support Android Doze for battery usage optimization, or cannot support App Shortcuts (see below example from Google Photos app). These diffs introduce a challenge to Dev and test team that innovate and take advantage of these features since the test code that shall run against these features and devices needs to be turned only towards devices can actually support it.
How can teams sustain a test automation suite that runs specifically on the right devices per supported features?
Proposed Approach
While I don’t have a bulletproof, magic pill to address all challenges that may occur as a result of the above problem, I can surely recommend an approach as described below.
Important to note, that being aware of the problem, is a step toward resolving it 🙂
Assess Your App and DUT:
- Map the different features that your app supports or requires the users to grant permissions for
- Examine your device test lab and filter the devices that support and does not support these specific features
To manage the above, teams can leverage the following:
- A code sample in GIT that runs on a target Android APK file and extracts from its Manifest.MF file the requires permissions, supported features and Android SDK/API’s
- Code Sample: https://github.com/ek121268/AndroidManifestReader
- Use an existing ADB command that extracts from a connected device/s the supported feature
- ADB SHELL
- PM LIST FEATURES
- ADB SHELL
After running the above command, you will get an output that looks like the below …
Compare The Outputs
Once you know your DUT’s capabilities, as well as your App, features to be tested, you can run a simple output comparison and see what can and can’t be tested – From that point, the optimization should be mostly manual – you will setup your test execution and CI in the lab accordingly. While it isn’t simple enough, it still offers a sustainable approach + awareness to both dev and test team that can be useful throughout the development, debugging and testing activities. In the below visual you can see a capabilities diff between a Samsung Note 5/Android 7.0 (left column) and an older Samsung running Android 5.x device capabilities (right column). An immediate diff out of a larger list that I have shows the fingerprint functionality that is supported on the Note 5 but not on the other Samsung device. Such insight should be used when planning the feature testing across these 2 devices (this is just one example).
Bottom Line
As Google continues to innovate and add more features, the existing devices and test framework will find it hard to close the gaps and that’s a challenge that teams need to be aware of, plan for, and optimize so their release vehicles and velocity remain solid.
Happy Optimization!
Continuous Testing and DevOps Blog 