Tag: AI

Posted on

Expediting Web Testing with Puppeteer and Instant Test Automation Technology

The digital landscape has significantly advanced over the past 1-2 years. The new normal in web applications include a shift toward progressive web apps (PWAs), advanced responsive websites, and Flutter apps.

Ensuring both functional as well as API testing of such advance web applications is becoming harder and harder. While there are valid test automation frameworks like Selenium and Cypress, these are covering only a subset of what is needed to complete the entire quality aspects across multiple environments.

New Normal: Instant Test Automation

To address the testing challenge over time, across multiple environments, and with big and dynamic data, tools need to be more intelligent, and utilize AI/ML to really track trends, predict issues prior to them becoming more serious, and guide the developers and testers in real-time.

An instant test automation technology that can leverage a standard test automation framework such as Puppeteer to auto generate both functional as well as API tests on the fly and as things change is a game changer for app developers. This provides alerts and insights in real time that the team can act upon and move much faster.

The way sch solution works is through modeling the website under test, auto generate based on traffic and AI proper tests and runs them to provide the proper insights.

Setting up such solution is easy and needs to be done only once. Also, such tool fits systems that are built atop cloud-native, containers and Amazon ECS.

Benefits of Shifting into Instant and Intelligent Test Automation

With the need to shift left, release faster with confidence, and handle large and complex environment, teams can benefit from a self-generating testing solution that really covers more grounds that a typical code-based framework would. This does not mean that there are no benefits or need for such frameworks like Selenium and others, however, with tools like UP9, teams can complement the code related functional tests with an additional layer of quality validation that touches reliability, APIs, services, security and much more.

Case Study: Quick Tour of Puppeteer & UP9

I took the Google puppeteer framework and connected it with UP9 to try generating a nice model and test automation assets on my local machine.

After following the basic NPM installation instructions, it was easy to launch a local browser and record traffic using the tool.

From my terminal I would then run this command (‘my.new.workspace’ could be any name that the developer selects):

up9 tap:start my.new.workspace

Once I ran that command, using puppeteer, UP9 launches a chrome browser that I can start using to navigate to my website for recording all traffic, actions, API calls and other services. Upon closing the browser, the tool will process all of the flows and generate the model.

Here is the auto generated workspace that was created based on a quick tour I ran with UP9 on the Perforce and Perfecto websites:

The workspace consists of few views:

  1. The generic workspace with all the high-level services model (diagram),
  2. The contracts that were used in my recording,
  3. A detailed traffic analysis with the entire API calls,
  4. Tests that were created and mocks to be used across different environments.

These views were perceived and constructed based on service-traffic, following from the observers of UP9 that were assigned to my designated workspace.

As part of my journey, I landed on the Perfecto free-trial website, and the tool was able to generate the API calls (Request/Response) and allow me to validate that is was working as expected (see below)

If I wanted to also run my auto-generated tests, I have within the UP9 portal (fully SaaS and web-based) a tests area from where I could launch my tests as well.

Bottom Line

As the DevOps market mature towards more intelligent testing using AI and ML, and as the systems under tests (SUTs) are also progressing, there needs to be a realization within the teams that their processes could be improved dramatically using such technologies. Developers productivity together with consistent app quality were a CIO/IT manager top of mind for a long while, and with the recent advancements in the market, teams can really enjoy a new era of smarter ways of working.

Posted on

The Future of Test Automation Must Be Intelligent

A decade of Agile, DevOps and Continuous Testing is behind us. I heard this week in a conference I was presenting the phrase “Agile is Dead” and other claims about testing being inefficient in many ways.

While many tools have evolved over the past years, and test automation practices and lessons learned have been communicated to practitioners, it is IMHO not enough to move the needle on the efficiency of test automation and continuous testing – especially in the demanding and ever changing digital space.

To have a better DevOps pipeline, and to minimize both the waste as well as the level of escaped defects, testing and especially test automation must become smarter.

In this post I would list the opportunities for test automation to be more intelligent.

WHAT?

Test automation intelligence should guide engineers on which test automation scenarios to execute, automate if missing, and continuously maintain. This is one of the most complex debates within each automation team – WHAT TO AUTOMATE?

Answering this question is never easy, requires risk management, tradeoffs, and often guessing. In addition, it does not always align with the sprint time frames, hence, being thrown out of the software iteration.

With more intelligent solutions that are based on production data, business driven decision making, code analysis, teams will be in a better position to make such decisions.

I love the below visual that Ingo Philipp and originally, Michal Bolton drew around testing is always sampling. This shows that test automation usually addresses what we know about risks and the product out of the entire ocean of risks and features that we either don’t know, cannot assess, or things that were addressed in the past and there isn’t sufficient data to advise whether we should continue using.

New tools that are rising such as Launchable, SeaLights and others aim to help answer some of the above questions, and to help form a continuously relevant regression suite that matches the most recent product code changes, history of defects and more.

WHEN?

The second question that needs to be answered in a more intelligent way is – When to automate and execute my test scenarios? Some say – everything , everyday, some say – everything shifts left, and some are breaking the tests based on personas, skills, and naïve objectives.

Scheduling and deciding which tests to run when must be smarter, and needs to be tied to the overall process, the feedback loops, and the ability of the developers to act upon such execution feedback. If testers throws on the developers to late or to early the results, it makes it unrealistic for them to prioritize, resolve and fix.

Intelligent systems, must be able to determine based on the above WHAT section, which test scenarios, types, and cadence, such should be executed. In an ideal world, an intelligent system would be able to split between CI, BAT (build acceptance testing), integration, regression, NFT and production synthetic monitoring the entire test suites.

As the above diagram suggest, the pipeline has a pre-defined set of quality gates, phases, and milestones to manage and mitigate risks. An intelligent testing system should effectively place all the right test scenarios –CONTINUOUSLY– in the right phases. Keep in mind, the right test cases change from one release to the next.

WHICH?

Now that WHAT and WHEN questions are addressed, it is important to have a smarter approach into test automation coverage – on WHICH platforms and environments should my test automation scenarios be running constantly? Which mobile devices, which desktop browsers? which backend configurations are important? which data-driven scenarios should be plugged into the test cases? Which load and KPIs should be applied so I get the proper user experience feedback from my testing? and more.

Determining the WHICH in test automation is an art, and require data analytics, market research, customers validation, production monitoring, AIOps systems and more, but it is an imperative for success.

WHY?

After determining few WHs questions in test automation above, it is time for the analysis part of test automation intelligence – We defined what to automate, when to run, and on which platforms and environments, but once executing the tests, can we efficiently determine WHY tests are failing? What is the root cause per each failure so we lower the time for resolution (MTTR)?

Here, intelligent reporting platforms are critical in minimizing the feedback loop, the future decision making on which tests are of higher value, which platforms are more flaky then others and much more.

Without a solid test automation analysis platform, a great chunk of teams investment in testing can be of waste.

It is not that complicated to realize why tests are constantly failing, and put them into buckets of failure reasons. Perfecto is doing that in an automated fashion through its smart reporting SDK, and allows teams to filter a lot of the test noise post a regression cycle completion.

HOW?

Test creation has always been one of the hardest and most challenging task for SDETs and business testers. It includes great technical skills, meeting short time windows for automation, and requires deep understanding of the product requirements to really automate what’s expected. While many test automation frameworks evolved over the past years, Including Selenium, Appium, Espresso, XCUITest, and Cypress, these are all code based frameworks that even when coming on top of a BDD platform, they are hard to maintain over time, and analyze when they result in a failure.

New and more intelligent test automation frameworks that are codeless or low-code have grown and are becoming a new reality of test engineers. Such tools can really help address few key challenges that are prominent. Time to automate, time to maintain, flakiness and skillset gap. Intelligent test creation that is plugged with a TIA can be the future of smarter test automation, and enable practitioners to create the most relevant test scenarios with minimal maintenance and shorter amount of time. As I am writing this blog, such tools are still growing and have a lot to prove to the world prior to being widely adopted, but this in my mind should be the future of test automation, and once this is in place, connecting these tools to the cloud for parallel and scaled testing will be the ultimate solution for continuous testing.

BOTTOM LINE

A lot of what is written in this blog already exists to some extent, some if being developed these days and will be available in 2021. It is important to start drawing within each DevOps team the future of test automation goals and objectives and learn how such solutions will fit the process from test automation creation, execution, maintenance, analysis, TIA, defect management, quality governance and more.

This year, I launched my 3rd book that focuses on “Accelerating Software Quality: AI and ML in the Age of DevOps“. This book looks at the modern DevOps and explores how using AI and ML and smarter algorithms, teams can maximize their productivity and deliver more quality functionality faster.

There is a lot to get excited for in 2021, and especially around test automation in DevOps – Happy New Year and Good Luck in your Future of Test Automation

Posted on

The Role of Artificial Intelligence in E-Commerce Industry

A Guest Blog Post by Ravindra Savaram

When we think about artificial intelligence(AI), the first thing that comes to our mind is a self-driving vehicle or a Terminator-like robot. Both robots and AI are not exactly one and the same. Though often utilized together with bots, artificial intelligence particularly refers to the stimulation of human intelligence processes by machines. AI powers many technologies that we utilize on a daily basis.

Whether AI is something that you have been monitoring for a while or it’s something that you have just come across, it is undeniable that AI is beginning to influence many industries. One place where it is really changing things is e-commerce. From creating personal buying assistants to personalizing the shopping experience, artificial intelligence is something that retailers cannot ignore.

Many areas of e-commerce are ripe for innovation driven by artificial intelligence. Every enhancement to logistics efficiency, recommendations, pricing, or marketing provides retailers an edge over the competition. Retail creates and consumes large volumes of data from various channels. In fact, there is so much data that it’s not possible for a human being to analyze it. These are the ideal conditions for machine learning.

For various data analysis methods, machine learning is the overarching name. In these methods, the computers get insights in data without actually being told where to look for the insights. When exposed a large amount of data, machine learning algorithms can extract patterns and utilize them to generate predictions or insights about the future conditions.

When you upload a cat picture to cat Google Photos, it knows that the object in the picture is a cat. The code that identifies the cat is not written by a human but it is developed as a result of exposing the algorithm to a large number of cat photos(also, the photos of things that are not a cat).

Recommendations

The same principle explained above can be put to use in many e-commerce areas. For instance, the retailers have become really good at recommending products that are related, but the people who do online shopping knows that the recommendation engines get it wrong very frequently. The recommendation engines are quite limited as they can have access to only a small set of data and the ways they can reason about that data are restricted. Machine learning helps merchants find much better ways of modeling the behavior of users so they can make close to exact recommendations about what a customer is interested in buying. With machine learning, the AI can make predictions based on past data. The predictions include what customers will buy next, their typical price threshold, their preferred device and channel, and so on.

Pricing

Today, the online retail industry is constantly presenting new challenges to COOs and CMOs when it comes to pricing. There is a fierce competition among the e-commerce brands of all sizes and guises. Even for an online merchant for a 1000 product list, somewhat tweaking in manual price can become a task that is almost impossible to accomplish. The environment is changing constantly – rival prices, logistics, currency conversions, and delivery rates are just a small sample of numbers or circumstances prone to change continuously.

The tweaking of prices in real time can be accomplished with artificial intelligence depending on multiple data sets including stock levels, resource capacity, internal operations, customer demand and behavior, and market conditions.

High-level of Assistance

The personal shopping assistants were a luxury of the rich or famous once upon a time. Artificial Intelligence has shaken up this scenario and in the process, revolutionized e-commerce. This conversational and intelligent technology has extended to customer service as well. The chatbots and personal digital shopping assistants can suggest the best available products to new visitors in a manner similar to humans, recommend new deals to your returning customers, answer the queries of a customer and provide suggestions, and alert customers when products they may prefer to purchase come into stock or change in price.

Conclusion

By merging intelligent neural networks with massive data sets, the applications of artificial intelligence will help e-commerce companies to build unparalleled competitiveness in the market. The impact of Personalized Merchandising supported by artificial intelligence on the e-commerce industry will continue to rise in the coming years. They not only optimize or automate current processes but also help retailers to avoid common pitfalls of manual approaches, giving customers an enriched experience to maximize profits.

About the Author:

Savaram Ravindra was born and raised in Hyderabad, popularly known as the ‘City of Pearls’. He is presently working as a Content Contributor at Mindmajix.com. His previous professional experience includes Programmer Analyst at Cognizant Technology Solutions. He holds a Masters degree in Nanotechnology from VIT University. He can be contacted atsavaramravindra4@gmail.com. Connect with him also on LinkedIn and Twitter.