Category: Digital

Posted on

Mobile, Cross Browser Testing, DevOps and Continuous Testing Trends and Projections for 2018

As we about to wrap out 2017, It’s the right time to get ready to what’s expected next year in the mobile, cross-browser testing and DevOps landscape.

To categorize this post, I will divide the trends into the following buckets (there may be few more points, but I believe the below are the most significant ones)

  • DevOps and Test Automation on Steroids Will Become Key for Digital Winners
  • Artificial Intelligence (AI) and Machine Learning (ML)/ Tools alignment as part of Smarter Testing throughout the pipeline
  • IOT and Digital Transformation Moving to Prime Time

 

DevOps and Automation on Steroids

If in 2017, we’ve seen the tremendous adoption of more agile methods, ATDD, BDD and organizations leaving legacy tools behind in favor of faster and more reliable and agile-ready testing tools, such that can fit the entire continuous testing efforts whether they’re done by Dev, BA, Test or Ops.

In 2018, we will see the above growing to a higher scale, where more manual and legacy tools skills are transforming into more modern ones. The growth in continuous testing (CT), Continuous Integration (CI) and DevOps will also translate into much shorter release cadence as a bridge towards real Continuous Delivery (CD)

 

Related to the above, to be ready for the DevOps and CT trend, engineers need to become more deeply familiar with tools like Espresso, XCUITest, Earl Grey and Appium on the mobile front, and with the open-source web-based framework like the headless google project called Puppeteer, Protractor, and other web driver based framework.

In addition, optimizing the test automation suite to include more API and Non-Functional testing as the UX aspect becomes more and more important.

Shifting as many tests left and right is not a new trend, requirement or buzz – nothing change in my mind around the importance of this practice – the more you can automate and cover earlier, the easier it will be for the entire team to overcome issues, regressions and unexpected events that occur in the project life cycle.

AI, ML, and Smarter Test Automation

While many vendors are seeking for tools that can optimize their test automation suite, and shorten their overall execution time on the “right” platforms, the 2 terms of AI and ML (or Deep learning) are still unclear to many tool vendors, and are being used in varying perspectives that not always mean AI or ML 🙂

The end goal of such solutions is very clear, and the problem it aims to solve is real –> long testing cycles on plenty of mobile devices, desktop browsers, IOT devices and more, generates a lot of data to analyze and as a result, it slows down the DevOps engine. Efficient mechanism and tools that can crawl through the entire test code, understand which tests are the most valuable ones, and which platforms are the most critical to test on due to either customer usage or history of issues etc. can clearly address such pain.

Another angle or goal of such tools is to continuously provide a more reliable and faster test code generation. Coding takes time, requires skills, and varies across platforms. Having a “working” ML/AI tool that can scan through the app under test and generate robust page object model, and functional test code that runs on all platforms, as well as “responds” to changes in the UI, can really speed up TTM for many organization and focus the teams on the important SDLC activities in opposed to forcing Dev and Test to spend precious time on test code maintenance.

IOT and The Digital Transformation

In 2017, Google, Apple, Amazon and other technology giants announced few innovations around digital engagements. To name a few, better digital payments, better digital TV, AR and VR development API and new secure authentication through Face ID. IOT this year, hasn’t shown a huge leap forward, however, what I did notice, was that for specific verticals like Healthcare, and Retail, IOT started serving a key role in their digital user engagements and digital strategy.

In 2018, I believe that the market will see an even more advanced wave in the overall digital landscape where Android and Apple TV, IOT devices, Smart Watches and other digital interfaces becoming more standard in the industry, requiring enterprises to re-think and re-build their entire test lab to fit these new devices.

Such trend will also force the test engineers to adapt to the new platforms and re-architect their test frameworks to support more of these screens either in 1 script of several.

Some insights on testing IOT specifically in the healthcare vertical were recently presented by my colleague Amir Rozenberg – recommend to review the slides below

https://www.slideshare.net/AmirRozenberg/starwest-2017-iot-testing/ 

 

Bottom Line

Do not immediately change whatever you do today, but validate whether what you have right now is future ready and can sustain what’s coming in the near future as mentioned above.

If DevOps is already in practice in your organization, fine – make sure you can scale DevOps, shorten release time, increase test and platform automation coverage, and optimize through smarter techniques your overall pipeline.

AI and ML buzz are really happening, however, the market needs to properly define what it means to introduce these into the SDLC, and what would success look like if they do consider leveraging such. From a landscape perspective, these tools are not yet mature and ready for prime time, so that leaves more time to properly get ready for them.

Happy New 2018 to My Followers.

Posted on

The Role of Artificial Intelligence in E-Commerce Industry

A Guest Blog Post by Ravindra Savaram

When we think about artificial intelligence(AI), the first thing that comes to our mind is a self-driving vehicle or a Terminator-like robot. Both robots and AI are not exactly one and the same. Though often utilized together with bots, artificial intelligence particularly refers to the stimulation of human intelligence processes by machines. AI powers many technologies that we utilize on a daily basis.

Whether AI is something that you have been monitoring for a while or it’s something that you have just come across, it is undeniable that AI is beginning to influence many industries. One place where it is really changing things is e-commerce. From creating personal buying assistants to personalizing the shopping experience, artificial intelligence is something that retailers cannot ignore.

Many areas of e-commerce are ripe for innovation driven by artificial intelligence. Every enhancement to logistics efficiency, recommendations, pricing, or marketing provides retailers an edge over the competition. Retail creates and consumes large volumes of data from various channels. In fact, there is so much data that it’s not possible for a human being to analyze it. These are the ideal conditions for machine learning.

For various data analysis methods, machine learning is the overarching name. In these methods, the computers get insights in data without actually being told where to look for the insights. When exposed a large amount of data, machine learning algorithms can extract patterns and utilize them to generate predictions or insights about the future conditions.

When you upload a cat picture to cat Google Photos, it knows that the object in the picture is a cat. The code that identifies the cat is not written by a human but it is developed as a result of exposing the algorithm to a large number of cat photos(also, the photos of things that are not a cat).

Recommendations

The same principle explained above can be put to use in many e-commerce areas. For instance, the retailers have become really good at recommending products that are related, but the people who do online shopping knows that the recommendation engines get it wrong very frequently. The recommendation engines are quite limited as they can have access to only a small set of data and the ways they can reason about that data are restricted. Machine learning helps merchants find much better ways of modeling the behavior of users so they can make close to exact recommendations about what a customer is interested in buying. With machine learning, the AI can make predictions based on past data. The predictions include what customers will buy next, their typical price threshold, their preferred device and channel, and so on.

Pricing

Today, the online retail industry is constantly presenting new challenges to COOs and CMOs when it comes to pricing. There is a fierce competition among the e-commerce brands of all sizes and guises. Even for an online merchant for a 1000 product list, somewhat tweaking in manual price can become a task that is almost impossible to accomplish. The environment is changing constantly – rival prices, logistics, currency conversions, and delivery rates are just a small sample of numbers or circumstances prone to change continuously.

The tweaking of prices in real time can be accomplished with artificial intelligence depending on multiple data sets including stock levels, resource capacity, internal operations, customer demand and behavior, and market conditions.

High-level of Assistance

The personal shopping assistants were a luxury of the rich or famous once upon a time. Artificial Intelligence has shaken up this scenario and in the process, revolutionized e-commerce. This conversational and intelligent technology has extended to customer service as well. The chatbots and personal digital shopping assistants can suggest the best available products to new visitors in a manner similar to humans, recommend new deals to your returning customers, answer the queries of a customer and provide suggestions, and alert customers when products they may prefer to purchase come into stock or change in price.

Conclusion

By merging intelligent neural networks with massive data sets, the applications of artificial intelligence will help e-commerce companies to build unparalleled competitiveness in the market. The impact of Personalized Merchandising supported by artificial intelligence on the e-commerce industry will continue to rise in the coming years. They not only optimize or automate current processes but also help retailers to avoid common pitfalls of manual approaches, giving customers an enriched experience to maximize profits.

About the Author:

Savaram Ravindra was born and raised in Hyderabad, popularly known as the ‘City of Pearls’. He is presently working as a Content Contributor at Mindmajix.com. His previous professional experience includes Programmer Analyst at Cognizant Technology Solutions. He holds a Masters degree in Nanotechnology from VIT University. He can be contacted atsavaramravindra4@gmail.com. Connect with him also on LinkedIn and Twitter.

Posted on

The Essentials of iOS App Testing For iPhone X

48 hours ago, Apple revealed its new and futuristic iPhone X. Regardless of its design, and debatable price tag, this device also introduced a whole set of functionality, display, and engagement with the end-user.

iOS11 is turning to be quite different from previous releases from both user adoption which is still low (~30%) and also from a quality perspective – 4 patch releases in 1.5 months is a lot.

Most of the changes are already proving to cause issues for existing apps that work fine on iOS11.x and former iPhones like iPhone 8, 7 and others.

In this post, I’d highlight some pitfalls that testers as well as developers ought to be doing immediately if they haven’t done so already to make sure their apps are compliant with the latest Apple mobile portfolio.

The post will be divided into 2 areas: Mobile testing recommendations and App Development recommendations.

Mobile Testing for iPhone X/iOS 11

  • Test across all supported platforms as a general statement. iOS11 isn’t for every device, and apps are stuck on iOS10 that has different functionality than the iOS11. Test your apps across iOS9.3.5, iOS 10.3.3, and the latest iOS11.x
  • iPhone X comes from the factory with iOS11.0.1, requesting for an update to iOS1.1 – that means, this device will never get the intermediate iOS11.0.2/iOS11.0.3 – if customers haven’t yet updated to iOS1.1, you may want to have 1 device like iPhone 8/7 still on iOS11.0.3 so you have coverage for iOS11.Latest-1
  • Display and Screen Size for iPhone X specifically changed, and this device has a 5.8” screen size that is different for all other iPhones. Testing UI elements, Responsive apps layouts and other graphics on this new device is obviously a must (below is an example taken from CVS native app showing UI issues already found by me while playing with the device). This device is also full screen similar to the Samsung S8/Note 8 devices. A lot of tables, text field, and other UI elements need to be iOS11/iPhopne X ready by the developers.

Image title

  • New gestures and engagement flow impact usability as well as test automation scripts. In iPhone X, unlike previous iPhones, the user has no HOME Button to work with. That means that in order for him to launch the task manager  (see below) and switch or kill a background running app, he needs to follow a different flow. What that means is that at first, the app testing teams need to make sure that this new flow is covered in testing, and more important, if these flows are part of a test automation scenario, the code needs to be adapted to match the new flow.

Image title

In addition to the removal of the Home button that causes the new way of engaging with background apps, the way for the user to return to the Home screen has also changed. Getting back to the Home screen is a common step in every test automation, therefore these steps need to account for the changes, and replace the button press with a Swipe Up gesture.

  • Authentication and payment scenarios also changed with the elimination of the Touch ID option, that was replaced with the Face ID. While iPhone X introduced an innovative digital engagement with the Face recognition technology, the de-facto today to log in into apps, make payments and more, is still the Fingerprint authentication. Testing both methods is now a quality and dev requirement. From a scan that I ran through the leading apps in the market (see examples below), there is a clear unpreparedness for iPhone X. Most apps will either show on their UI the option to log in via Touch ID or if they support Face ID, they will allow users to use it, while still showing on the UI and in the app settings the unsupported option.

Image title

  • Testing mobile web and responsive web apps in both landscape and portrait mode with the unique iPhone X display is also a clear and immediate requirement. I also found issues mostly around text truncation and wrong leverage of the entire screen to display the web content.

Image title

 

In addition, trying to work with Hulu.com website proved to also be a challenge. Most menu content is being thrown to the bottom of the screen under the user control, making it simply inaccessible. Obviously, the site is not ready for iPhone X/Safari Browser.

Mobile Apps Development

  • Optimize existing iOS apps from both UI as well as authentication perspective. As spotted above, there are clear compatibility issues around the removal of the Touch ID option, that needs to be modified on the UI side of the apps when launched on iPhone X. In addition, scaling UI elements on the new screen whether for RWD apps or mobile apps needs refactoring as well. Apple is offering app developers a ui guidlines to help make the changes fast.Image title
  • Leverage advanced capabilities in iOS11 that best suit the new chipset (AI11 Bionic) and the camera sensors, to introduce digital engagement capabilities around augmented reality (ARKit API’s) and others. Retail apps and games are surely the 1st most suitable segments to jump on these innovative capabilities and enrich their end users’ experiences.

Image title

Bottom Line

The new iPhone X might be paving the way together with the Android Note 8 for a new era of innovation that offers App developers new opportunities to better engage and increase business values. If quality will not be aligned with these innovative opportunities, as shown above, that transformation will be quite challenging, slow and frustrating for the end users’

It is highly recommended for iOS app vendors across verticals to get hands-on experience with the new platform, assess the gaps in quality and functionality, and make the required changes so they are not “left behind” when the innovative train moves on.

Posted on

Optimizing Mobile Test Automation Across The Pipeline

With the massive innovation that drives the digital market these days, organizations are continuing to develop features, as well as new test code to cover these features.

What I’ve learned is that often, the test code developers would not always stop and look back into their existing test suites and validate whether the new tests that are being developed are somehow a superset to existing ones. In addition, legacy tests are a continuous load and overhead on your SDLC cycles length if they are not being maintained over time.

Oil Transport

Many Owners To The Same Problem

Since we live in an agile/DevQAOps world, test code development is not a QA only problem, but rather everyone3s. Tests are being executed throughout the pipeline from Dev to integration and pre/post production testing.

Use of smart tagging mechanism for your test scenarios (login), suites (App A) and types unit, regression) can be a good step towards gaining control over your tests.

Without some context, discipline, and continuous structured validation of the tests, it will become harder as you progress your SDLC to debug, analyze and solve defects (would be like finding the key in the below visual mess)

Find the Key in the Picture

Recommended Practices

  • Develop the tests with context, tags and proper annotations that would make sense to you and your team even 12 months from the development day. Make sure that in your execution reports you then have a way to filter using these annotations to only get the view of a given functional area, platform etc.
  • Match your device under tests capabilities to the test code and application under test. Make sure that you focus e.g. your fingerprint based tests only on the devices that support it (API XX and above).
  • Perform test code review every agreed upon time – in such review, group your feature specific test suites and try to optimize, merge, eliminate flakiness, identify missing coverage areas etc. It is harder to do it as the time progresses, so depending on your release cadence and test development maturity, set the right goals – more reviews would be better than less – it will also be shorter and more efficient that way since the delta between such review will be smaller.
  • Drive joint Dev, Test, Product, Marketing decisions based on data – When you have the ability to get quality analysis from your entire test suites, it is recommended to gather all counter parts and brainstorm on the findings. Which tests are most effective, can we shrink based on the data the release cycles, are we missing tests for specific areas, are there platforms that are more buggies than others, which tests takes longer than others to finish etc.
  • Optimize your CI and build-acceptance testing – based on the above intelligence, teams can reach data driven decision about what to include in their CI as well. Testing in the build cycle via CI should be fast, reliable with zero false positives. With quality insights on your tests, you can decide and certify the most valuable and fastest tests to get into this CI testing, and by that to shrink the overall process without risking coverage aspect.

CI_Dash1.png

Bottom Line

A test is code, and like you refactor, maintain, retire and improve your code, you should do the same to your tests. Make sure to always be in control over your tests, and by that, gain control over your quality of your app in a continuous manner.

Happy Testing!

Posted on

How the “Digital Quality Handbook” Was Born

Travel back with me… to late September 2016. It’s the Jewish New Year, and I am in Boston, MA. As I celebrate the passing of another great year, I think to myself, “After being in the software quality space for nearly 20 years, isn’t it about time that I reach out to the community of thought leaders and influencers and create an asset that can fill a gap in the market that we can give back to the world?” A book. A practical book. A “how-to” for DevOps practitioners, designed to make them better, faster, and more… perfect(o).

You see, when it comes to assuring the quality of web, mobile and IoT apps, the market is still struggling with key questions around test coverage, automation best practices, optimization of test automation suites, accomplishing more tests within the pipeline of software build cycle, the practice of shifting left and much, much more.

So, while my wife and children continued celebrating in the next room, I immediately (right then and there) started writing the intro to a book that would, eventually, bring together actionable ideas and practices from many of the world’s most recognized experts, thought leaders and influencers in the area of software quality.

To make it easier to both develop and consume the content, the book is set out in four logical sections:

  1. Introduction to continuous quality and the digital space
  2. Advanced test automation practices
  3. Achieving DevOps maturity in the digital era
  4. Expanding quality coverage with UX and non-functional testing

If you’re reading this article close to its post date, I’m currently down in Orlando, participating in a book signing at the StarEast testing conference. Danny McKeown from Paychex, one of the technical reviewers of the book, is with me, both participating in the signing and speaking at the event.

To name the market leaders who took part and contributed to this book:

  1. Microsoft (Donovan Brown)
  2. Applitools (Adam Carmi)
  3. TestFairy (Yair Bar-On)
  4. Applause (Doron Reuveni)
  5. CA & BlazeMeter (Jonathon Wright, Noga Cohen, Jacob Sharir)
  6. InfoStretch (Manish Mathuria)
  7. Rabobank (Wim Selles)
  8. Utopia Solutions (Lee Barnes)
  9. Angie Jones
  10. Jean Ann Harrison
  11. Lior Kinsbruner

And from Perfecto:

  1. Amir Rozenberg
  2. Roy Nuriel
  3. Paul Bruce
  4. Chris Willis
  5. Uzi Eilon
  6. Yoram Mizrachi
  7. Roi Carmel

Without this crew of contributors, the book wouldn’t be what it is today. Some of the contributed content includes:

  • The best way to include visual analysis testing as part of your test code, using any available open-source framework
  • How to develop API tests that complement your mobile UI test automation
  • How to include non-functional performance testing and UX as part of your overall test strategy
  • How to extend open-source tools like Protractor to better test your hybrid app
  • The bible of UX testing
  • What a valid and high-ranked XPATH should look like (with link to an online free tool that provides that rank to you)
  • How to include chatbots testing into your existing mobile testing plans
  • Where in the overall SDLC strategy does crowdsource testing and beta testing fit

Fun fact: We launched the book on Amazon on March 3rd. On March 5th, at approximately 2:51pm Eastern Time, the book had been added to the Hot New Releases in the Software Testing sidebar, and made it to the #1 Bestseller slot in that same category. We took a screen shot. It really happened!

To get your own copy of the book, please refer to this URL – and if you find it valuable, feel free to share your feedback with me.

Happy Reading!

Posted on

Google Mobile Friendly With Perfecto and Quantum

Guest Blog Post by Amir Rozenberg, Senior Director of Product Management, Perfecto

resize

Google recently announced “Mobile First Indexing”, from Google:

To make our results more useful, we’ve begun experiments to make our index mobile-first. Although our search index will continue to be a single index of websites and apps, our algorithms will eventually primarily use the mobile version of a site’s content to rank pages from that site, to understand structured data, and to show snippets from those pages in our results (Source).

screen-shot-2017-02-13-at-5-33-26-pm

More recently they made the Google Mobile-Friendly tool and guidelines available. A very nice interactive version is available here, and images at the bottom of the thread, while there’s also an API (which, thanks to Google, can allow users to exercise first before they code). Google also offers code snippets in several languages.

Notes:

  • Google takes a URL and renders it. If you run multiple executions in parallel there’s no point in sending the same URL from every execution because the result would be the same
  • Google returns basically “MOBILE_FRIENDLY” or not. Suggest to set the assert on that
  • The current API differs from the UI such that it only provides the results for Mobile friendly (and the UI gives also mobile and web page speed). Hopefully, Google adds that to the response 😉
  • This will probably not work for internal pages as Google probably doesn’t have a site-to-site secure connection with your network.

 

For developers and testers who do not have time, testing mobile friendliness repeatedly probably will simply not happen. That’s why I integrated Google Mobile-Friendly API into Quantum:

  • Added 2 Gherkin commands
// If you navigate directly to this page
Then I check mobileFriendly URL "http://www.nfl.com"
// If you got to this page through clicks
Then I check mobileFriendly current URL
  • Added the Gherkin command support (GoogleMobileFriendlyStepsDefs.java)
  • And the script example is pretty simple:
@Web
Feature: NFL validate

  @SimpleValidation
  Scenario: Validate NFL
    Given I open browser to webpage "http://www.nfl.com"
    Then I check mobileFriendly current URL
    Then I check mobileFriendly URL "http://www.nfl.com"
    Then I wait "5" seconds to see the text "video"

 

That’s it. Next steps:

 

Ideas for future improvement:

  • You can automate the validation such that every click would trigger a check with Google behind the scenes.

Just for fun, some more screenshots for detailed analysis for NFL.com:

 

screen-shot-2017-02-13-at-5-33-48-pm

 

screen-shot-2017-02-13-at-5-34-09-pm

screen-shot-2017-02-13-at-5-34-23-pm

 

 

Posted on

3 Motivations That Made Me Switch From iOS to Android

As a mobile evangelist at Perfecto, i foresee the entire mobile and web space for the past 10+ years, following major trends both in the device/hardware front as well as the platform/OS (operating System) front.

I was an Apple user for the past 2 years, using an iPhone 6 Plus device both for my personal as well as my work daily activities. Last month i decided it’s time for a change and i replaced my iPhone with a Google Nexus 6P phablet.

Let me explain some of my reasons to that switch:

  1. Quality and Innovation
  2. Platform Restrictions
  3. Future Looking

vpcqffbw

Quality and Innovation

In the front of quality and/vs. innovation i found out that as a 2 year trend, Apple’s iOS was constantly straggling with quality that mostly came on top of innovative features and end user -experience. For the past 2 years Apple released 10 versions of iOS 8 stopping at a stable GA of iOS 8.4.1, while for iOS 9 Apple released 10+ versions stopping at a recent 9.3.5 GA release that addresses security issues. To compare this trend to Android platform – Android 5.0 Lollipop released in November 2014 and was enhanced till latest version of 5.1.1 (~5 versions in 2 years). Android Marshmallow 6.0 was released in October 2015 and since than only had an additional version of 6.0.1 release. Last month (August 22nd) Google released its new Nougat 7.0 release that is available to users (like me) that hold a Nexus device. iOS 10 is just around the corner with the iPhone 7 devices, but based on the current trend and enormous public Beta versions, it seems like no major changes are expected in the quality/release cadence.

In the Android history we see some major enhancements around sensor based capabilities for payment, logging in as well as UX (user experience) features such as multi window support (see below image), android Doze (battery saving capability). In iOS we also see enhancements around sensors like force-touch, apple pay however these features IMO come in short compared to the platform stability over the past 24 months and the platform constrains which i’ll highlight in the next section.

20160823_142250 Screenshot_20160823-141941

Platform Restrictions

From an ens user perspective, some of the important platform features involves the ability to customize his UX and look and feel of his personal device. Also having the ability to easily manage his media files such as photos and music with a reasonable storage availability. Apple flagship device with massive market share across regions is the iPhone 6/6S with a default storage (un-expandable) of 16GB – I hardly know a person who has this device/storage size that is happy with that, and does not need to constantly delete files, cancel auto savings of WhatsApp media files and alike.  In addition, continuously working with iTunes software as a dependency to media/songs sync is a pain and often i found myself losing my favorite music files or getting them duplicated by simply having to switch from 1 PC to another (people do that, and there are procedures that might have prevented this outcome but still). Compared to the above, most Android devices that are not coming with an external storage option are by default coming with a 64 GB internal memory, and in addition working with music file system is a simple and straight forward task to do.

Switching from my iPhone and iTunes to a Nexus device while having my Gmail account was a very simple thing to do, my music, photos and apps easily “followed” me to the Android device that is already running Android 7 in a stable way.

iOS is not all bad, don’t get me wrong – from an adoption perspective, and device/OS fragmentation this is by far a much better managed platform compared to Android that rolls out its latest GA version in a 4-6 months delay to a non-Nexus device (example: Samsung). In addition the iOS tablets are still a leader in that front with 4-6 years old tablets like iPad Air, iPad 2 that are the most commonly used tablets in the market that can still run iOS 9 OS versions. It is not the case when it comes to Android tablets that tend to be replaced by their end-users in a shorter period of time that iPads.

 

Market_Cal

Future Looking

From a future looking perspective, my opinion is that Google is still going to have a global market share advantage over Apple and will continue to innovate with less frequent releases due to quality than Apple. 2017 is going to show us a continuous battle between Android 7 and iOS 10 in a market that becomes more and more digital and mobile dependent, and with this in mind – the challenge of quality, innovation and less restrictions will be even more critical to independent users as well as large enterprises who are already today fully digital.

As an end-user, i would look at both Google and Apple and examine how their overall digital strategy will transform and enable easier connectivity with smart devices like watches etc., as well as less limited storage and device/OS customization. From a Dev and Test perspective i would assume we will continue to see growing adoption of open-source tools such as Espresso, XCTest UI, Appium etc. as a method of keeping up with the OS platform vendors – Only such open-source frameworks can easily and dynamically grow and support new features and functionalities compared to legacy/commercial tools which are slower to introduce new API’s and new capabilities  into their solutions.

Posted on

Responsive Web: The Importance of Getting Test Coverage Right

When building your test lab as part of a RWD site test plan, it is important to strategically define the right mobile devices and desktop browsers which will be your target for your manual and automated testing.

For mobile device testing you can leverage your own analytics together with market data to complement your coverage and be future ready, or leverage reports such the Digital Test Coverage Index Report.

For web testing you should also look into your web traffic analytics or based on your target markets understand which are the top desktop browsers and OS versions on which you should test against – alternatively, you can also use the digital test coverage index report referenced above.

Related Post: Set Your Digital Test Lab with Mobile and Web Calendars

Coverage is a cross organizational priority where both business, IT, Dev and QA ought to be consistently aligned. You can see a recommended web lab configuration for Q1 2016 below which is taken from the above mentioned Index – Note the inclusion of Beta browser versions in the recommended mix due to the nature silent updates of these versions deployment on end-user browsers.

WCReport
For ongoing RWD projects  – once defining the mobile and web test coverage using the above guidelines, the next steps are of course to try and achieve parallel side by side testing for high efficiency, as well as keep the lab up to date by revising the coverage once a quarter and assure that both the analytics as well as the market trends still matches your existing configuration.

As a best practice and recommendation, please review the below mobile device coverage model which is built out of the 3 layers of Essential, Enhanced and Extended where each of these layers includes a mix of device types such as legacy, new, market leaders and reference devices (like Nexus devices).

MobileCoverageLayers

To learn more, check out our new Responsive Web Testing Guide.

responsive web testing strategy

Posted on

Responsive Web: Test for the Real User Experience

One of the great benefits of building a responsive web site (RWD) is it can give the user a consistent web experience across any digital device, in any location.

Related Post: Responsive Web and Adaptive Web: Pros and Cons

When it comes to RWD testing, it’s important to test the navigation and functionality on desktop web browsers and mobile devices, but that alone is not enough to guarantee a consistent user experience at all times. The end user is constantly moving between environments throughout the day, and these environments have various attributes, including:

  1. Network conditions (Poor, good, no network)
  2. Locations
  3. App context based on platform and location
  4. Background activities (apps running and consuming resources)
  5. Ads and other popups that block your site content (see image below)

IMG_8543

With so many real user environments to consider for both mobile and the desktop web, testing teams should include user conditions in their RWD test plan on top of the traditional testing for UI, navigation, functionality and client-side performance. It will give your DevTest team peace of mind and reduce quality risks significantly.

To learn more, check out our new Responsive Web Testing Guide.

responsive web testing strategy

Posted on

Responsive Web: Five Testing Considerations

With more and more consumers expecting to shop, bank, work and socialize across different devices, organizations are embracing responsive web design (RWD) as a tool to help them deliver a consistent digital experience on every screen.

multiplatform-1024x636

Growth of cross-device transactions (Source: Criteo’s State of Mobile Commerce Report)

But due to the complexity of digital environments and user experiences — responsive web is easier said than done. Organizations that develop RWD sites often face challenges when testing to assure smooth website navigation and a great user experience across multiple devices and platforms.

For more information, read our new Comprehensive Guide to Building a Responsive Web Testing Strategy

To get there, we recommended including the following five building blocks as part of your RWD test plan.
RWDTests-1024x368

Testing for these five areas will help achieve sufficient test coverage, a great user experience and higher traffic to your site.

To download the complete guide for testing RWD Site, go here

responsive-web-testing-strategy-2-600x315