Blog Series (2) Digital Test Coverage Assuring the Right Mobile/Web Testing Mix

It’s an exciting time to be a digital company. Your customers are engaging with your products on various screens, moving between desktop web browsers to apps on mobile devices. But in the effort to guarantee quality web and mobile experiences, organizations are struggling to find the right testing mix.

It’s true that mobile is far more complex and fragmented than the web, but with so many web browser/OS permutations out there (i.e. Chrome OS 47 running on Windows 7, 8.1, 10, Mac Yosemite, etc.) precise testing becomes a challenge.

To help DevTest teams test more precisely, Perfecto recently published the “Digital Test Coverage Index – Edition 3”, a quarterly report that provides a prescriptive way to build a digital test lab that covers 30%, 50%, or 80% of mobile device and web browser markets in various geographies. The report — intended for organizations just starting their digital journey or trying to move to the next stage — is based on market share data and analysis of enterprise customer usage in Perfecto’s cloud testing lab.

Using the above 30%-50%-80% coverage model featured in the Index report, teams can more accurately define their desired testing parameters and allocate the recommended devices and virtual machines running the relevant desktop browsers. Teams that are developing a responsive web application (RWD) can refer to the Index and then test the app in their lab on the recommended smartphones and tablets alongside the recommended desktop browser/OS permutations.

On the subject of web browser/OS mixes: According to our latest Index, of the top 30% of desktop browsers in the U.S. market, Chrome OS 46 (version 47 was just released and is well-adopted already) is by far the leading browser on Windows 7, followed by FireFox OS version 42 on Windows 7, and Safari OS 9 on Mac OS El Capitan. The Index report includes the complete 30%-50%-80% matrix for web/OS and mobile device/OS combinations.

It’s also worth noting that in the browser testing landscape, the Windows 10 platform is gaining momentum and will soon become the second most popular desktop OS in most of the geographies, according to market share numbers.

It will take more than looking at a list of smartphones and web browsers to ensure full digital test coverage for native and hybrid mobile apps, mobile web browsers and RWD. So organizations need to combine their existing customer analytics with a regularly updated test coverage index that reflects market adoption rates in various geographies. Another important metric to monitor is the status of legacy platforms that are still relevant enough to test against. For example, the Samsung Galaxy S3 is a leading legacy smartphone in most markets in the same way that we still see many Windows 7 machines even though Windows 8 and Windows 10 are widely available.

For more details on how to test for the full digital experience, download the free Digital Test Coverage Index.