Category: continuous testing

Posted on

The Importance of Software Testing for Businesses

Guest Blog from: Maverick Jones
Maverick Jones is a full-time geek and tech enthusiast. He likes to share his bylines and loves to gain audience attention.

Table of Contents

  • Introduction
  • What is Software Testing?
  • Types of Software Testing
  • 7 Reasons of Software Testing Being Important for Businesses
  • Conclusion

In this digital world, software is the one thing that influences a far bigger audience these days. It enhances the business prospects, contributes to upscale life standards, offers services to large audiences in less time, and evolves a smart world. Besides this, the software offers an industrial impact and plays an important role around all the walkways of people. But for this, it requires a fail-proof and smooth functioning of the software. And the main reason behind it is that software failures can shatter the business. Therefore, testing any software is a must.

Software testing ensures that the software performs adequately and offers user satisfaction. Testers make sure that the performance of the software is not sacrificed and for that, they run various tests using both manual and automated testing approaches.

In this blog, we will learn more about software testing and see how this concept enables business software to work smoothly.

What is Software Testing?

Software testing is a very critical process that offers an analysis that enables the testers to identify and evaluate the application. It determines whether the newly developed app meets the Business Requirement Specifications. It is a continuous process and is popularly known as Software Testing Life Cycle. This process works along with the life cycle of software development.

Software testing verifies each functionality at every stage and validates the app performance with the need. Besides, when it comes to checking software perfectly and delivering bug-free software, the approach of software testing can help in improvising the functionalities and usability of apps. With different types of testing approaches, software development companies test applications and make them safer for their clients.

Types of Software Testing

Software applications can be validated and verified in two ways. By applying two different types of testing – Automation and Manual Testing.

Automation testing is an approach where the testers write different scripts and then they rely on external software to perform software tests. Besides, when the testers want to double-check the product’s quality, even the manually tested software has to go through the automation tools. And this is the type of testing that assures great accuracy level and cost-efficiency. Besides, it also saves a lot of time.

Besides, manual testing is quite different. And as the name suggests, it is a very simple process of testing the software manually. It doesn’t require any type of automated testing tool that enables the testers to check the software automatically. Instead, the tester can work on the software and perform the testing process. The manual tester can easily detect errors and bugs by following through with different levels.

7 Reasons of Software Testing Being Important for Businesses

Here are some of the top reasons that show the importance of software testing –

1.   Security

Security is one of the most important reasons for software testing. And to consider it can be the smartest choice of it all. Security is considered a very crucial, sensitive, and vulnerable part of software testing. Besides, there are situations in which all the important details and information of the company gets stolen away. Therefore, security is considered the most reliable reason to make people believe in the quality of the software. Security is one thing that ensures that the product is safer to use.

2.   Customer Satisfaction

The main objective of the business owners of the products is to provide the best services and better customer satisfaction. In order to offer quality and customer satisfaction, adopting a software testing process is essential as it provides the perfect user experience. As the software development company tries to develop the best software for their clients with a testing approach, they will earn a reputation for reliable clients.

Therefore, to get praise from the clients and offer them a system that enables long-term benefits, having a perfect software testing approach is very important. A properly tested and secured system earns the client’s trust easily and also becomes safer to make transactions.

3.   Product Quality

To ensure that some of the products come to life in a secured manner, following the requirements of the product is necessary. The product you offer to the clients must be such that it serves them in one or another way. Besides, it must always bring out value as promised. And to make sure that it functions correctly by offering an effective customer experience, testing the products before their launch is necessary.

Every testing team must check the compatibility of the device before launching the product or application. For example, when you are planning to launch a software, checking that it is compatible with different devices and operating systems is a must. This shows the quality of the software.

4.   Saves Money

Software testing comes with various benefits and one of them is cost-effectiveness. This is the main reason why organizations go for software testing services. The software testing comprises a bunch of projects. When the testers find bugs in the early stage, fixing those bugs doesn’t cost much. Therefore, it has become a prerequisite.

Every business owner hires a tester or quality analyst who has great experience and thorough knowledge of the projects.

5.   Determines Software Performance

When an application or software has low performance, it brings down the app’s reputation in the market. The users no more prefer to use that application and even stop trusting the company that offers such products. And eventually, the company’s reputation also suffers. Therefore, every application or software must never be introduced in the market without software testing.

Once the software is tested and its performance is perfect, it must be launched in the market. And even after the launch, to keep the reputation of the product intact, continuous testing becomes necessary. Basically, if the performance of the software goes down, everything goes to waste. Therefore, considering software testing is the best option in determining the software performance is a must.

6.   Easy to Add New Features

When the code is old and interconnected, it becomes more difficult to update. And this is why tests are performed that counteract this calcification tendency and enable the developers to add new features to the software. But when the software developer is new, changing older parts of the code becomes horrifying for him. And this is where test results come in to offer every detailed information about the broken parts of the code.

This process enables the business owners to make their products stand out in the market.

7.   Enhances Development Process

With the help of Quality Assurance, the software testing services providers can find an array of errors and scenarios. And once they get a hold of the information about bugs, the developers can easily help in fixing them in very little time. the same in no time. To have such a smooth development process, the software testers should work with the development team parallelly in order to accelerate the procedure.

Conclusion

As seen in this blog, the importance of software testing is of high value in the world of software development. Software testing is a fine whole process that ensures the perfect quality of the product. Besides, testing also offers better usability, enhanced functionality, and reduces maintenance costs. Therefore, whenever you hire a software development company for your project, make sure to hire a team that has expert software testers who can use the latest tools and techniques to secure your product.

This Blog was provided by: Maverick Jones
Maverick Jones is a full-time geek and tech enthusiast. He likes to share his bylines and loves to gain audience attention.

Posted on

Code-Based Test Automation vs. Codeless Automation

 

As more advanced technologies are entering the continuous testing landscape powered by AI/ML, organizations and especially, practitioners, are debating which is better, and why if any should they adopt codeless test authoring solutions?

In this blog, I will be providing the various considerations to switch, and/or combine between the 2 test automation methods.

TL,DR –> There isn’t a magic answer to this debate, and there isn’t a method that is good vs. bad.

Top Considerations

To better address the question on when and why to use either methods, here are the top items to consider, not listed by importance, since each team might relate to different objectives and priorities:

  • What are the application use cases and flows to automate?
  • Which persona is going to create and maintain these scenarios?
  • What are the skill-set within the team/individuals for the job?
  • Is the app under test Mobile/web/responsive/desktop?
  • What are the time constrains for the project, and what are the release cadence going forward (weekly/monthly)?
  • Is the test suite meant to be integrated to other tools (CI/CD/Frameworks)?
  • Are there any advanced automation scenarios (chatbots, IOT, location, audio, etc.)?
  • What are the cost boundaries (tools, project, lab, etc.)
  • Is the test suite meant to be executed at large scale?
  • Is the project a new one, or an additional layer of coverage on top of existing code-based suite? (Selenium/Appium, etc.)

Diving Deeper

Now that we’ve listed some important consideration, let’s explain them a bit deeper.

For teams that are already working on a project (web/Mobile), and have implemented a good working amount of code-based test scenario that are embedded into processes, CI/CD, and other triggers, such consideration should be heavily considered – what is the incentive to change? are there coverage gap in the code-based suite? is there too much noise and flakiness attached to the existing test code? etc. Only if there is a good incentive like the ones mentioned, teams should consider adding codeless test scenarios into their pipeline.

On the other hand, for teams that are just starting a new project, that’s a perfect timing to look into the entire team skills, and decide based on the technology the project is built on, what tools to use – if might make sense for example, for a newly created website to combine a Selenium framework that SDETs that are Java/JavaScript developers would lead together with business testers that can remove some of the load from them via ML-Driven codeless selenium tools.

After covering use cases, quality of existing test suite, new vs. existing projects, lets also consider the time-frames and budget allocated to the project. It is clear that recording codeless scripts takes on average 6-10 times faster than coding the same scenario in Java or other development language. It involves setting up the platform and test environment, coding, debugging, executing at scale, assertions, and more. This obviously also translate into $$ savings. On the other hand, not all test scenarios can be easily recorded, since, for some advanced flows, coding might be a better approach and an easier one to maintain over time. This is why, sometimes it is better to look at the job-to-be-done, prior to rushing into scripting.

Code and Codeless Creation is Key for High Test Automation Coverage

Next in the overall debate is the Eco-system and tools landscape within the organization. Including a new technology is not easy, often not well-accepted, and also not always justified. In today’s reality when squad teams are working together and consists of a variety of resources with varying skills, objectives, and preferences, integrating a new technology should be done with proper consideration, and with proper validation that it “plays” nice within the other existing tools. Codeless tools in that context, should fill in an important gap within the team, integrate well into existing CI/CD and other processes, and not cause duplication of effort, or additional overhead.

Lastly for this blog (not for the entire debate), I would touch the cost of maintenance of test automation scripts. This is perhaps one of the problematic items for any test automation team. Writing a script once, making it run across platforms over time is an easier said than done task. Applications are constantly changing, and so does the platforms under test (mobile devices/OS versions, Browsers), therefore, scripts needs to be properly maintained to ensure clean and noise-free pipeline. Codeless in many ways addresses such challenge through self-healing of elements, test steps, and more. It can be also achieved within code-based projects through advanced reporting and analysis, with automated root-cause analysis and other methods, but codeless does shine the most in such cases.

Perfecto/TestCraft Object Self-Healing and Weight Mechanism (ML)

Bottom Line

I tried to keep this blog short and to the point, and leave the practitioners the decision-making across the 2 methods. As written in this post, there are plenty of questions to address prior to adopting a codeless tool, and how to combine it within existing code-based suites. Combining both methods in my honest opinion is the way to go in the future, and the way to maximize the overall test automation coverage with greater efficiency across the teams. Make the right decision that fits the project now, and also in the long-run.

I will be running a live webinar covering the exact blog topic hosted by Shani Shoham from 21. Feel free to register here

Posted on

Most Recommended Software Testing Books to Read in 2020 and Beyond

As we kick of a new decade of software development and testing, and as digital continuous to challenge test automation engineers that are trying to fit the testing within shorter then ever cycles, here are the top recommended software testing books to consider – the order isn’t the priority, they are all awesome books and equally recommended:

#1 Agile Testing – Lisa Crispin and Janet Gregory

Readers will come away from this book understanding

  • How to get testers engaged in agile development
  • Where testers and QA managers fit on an agile team
  • What to look for when hiring an agile tester
  • How to transition from a traditional cycle to agile development
  • How to complete testing activities in short iterations
  • How to use tests to successfully guide development
  • How to overcome barriers to test automation

#2 More Agile Testing – Lisa Crispin and Janet Gregory

Main learning objectives from the book:

  • How to clarify testing activities within the team
  • Ways to collaborate with business experts to identify valuable features and deliver the right capabilities
  • How to design automated tests for superior reliability and easier maintenance
  • How agile team members can improve and expand their testing skills
  • How to plan “just enough,” balancing small increments with larger feature sets and the entire system
  • How to use testing to identify and mitigate risks associated with your current agile processes and to prevent defects
  • How to address challenges within your product or organizational context
  • How to perform exploratory testing using “personas” and “tours”
  • Exploratory testing approaches that engage the whole team, using test charters with session- and thread-based techniques
  • How to bring new agile testers up to speed quickly–without overwhelming them

#3 Hands on Mobile App Testing – Daniel Knott

Readers will learn how to

  • Establish your optimal mobile test and launch strategy
  • Create tests that reflect your customers, data networks, devices, and business models
  • Choose and implement the best Android and iOS testing tools
  • Automate testing while ensuring comprehensive coverage
  • Master both functional and nonfunctional approaches to testing
  • Address mobile’s rapid release cycles
  • Test on emulators, simulators, and actual devices
  • Test native, hybrid, and Web mobile apps
  • Gain value from crowd and cloud testing (and understand their limitations)
  • Test database access and local storage
  • Drive value from testing throughout your app life-cycle
  • Start testing wearables, connected homes/cars, and Internet of Things devices

#4 Enterprise Continuous Testing – Wolfgang Platz

Enterprise Continuous Testing: Transforming Testing for Agile and DevOps introduces a Continuous Testing strategy that helps enterprises accelerate and prioritize testing to meet the needs of fast-paced Agile and DevOps initiatives. Software testing has traditionally been the enemy of speed and innovation—a slow, costly process that delays releases while delivering questionable business value. This new strategy helps you test smarter, so testing provides rapid insight into what matters most to the business.

#5 Continuous Testing for DevOps Professionals – Eran Kinsbruner and Leading Market Experts

Continuous Testing for DevOps Professionals is the definitive guide for DevOps teams and covers the best practices required to excel at Continuous Testing (CT) at each step of the DevOps pipeline. It was developed in collaboration with top industry experts from across the DevOps domain from leading companies such as CloudBees, Tricentis, Testim.io, Test.ai, Perfecto, and many more.The book is aimed at all DevOps practitioners, including software developers, testers, operations managers, and IT/business executives

#6 Accelerate –The Science of Lean Software and DevOps, Nicole Forsgren

How can we apply technology to drive business value? For years, we’ve been told that the performance of software delivery teams doesn’t matter―that it can’t provide a competitive advantage to our companies. Through four years of groundbreaking research to include data collected from the State of DevOps reports conducted with Puppet, Dr. Nicole Forsgren, Jez Humble, and Gene Kim set out to find a way to measure software delivery performance―and what drives it―using rigorous statistical methods. This book presents both the findings and the science behind that research, making the information accessible for readers to apply in their own organizations.

#7 Complete Guide to Test Automation, Arnon Axelrod

  • Know the real value to be expected from test automation
  • Discover the key traits that will make your test automation project succeed
  • Be aware of the different considerations to take into account when planning automated tests vs. manual tests
  • Determine who should implement the tests and the implications of this decision
  • Architect the test project and fit it to the architecture of the tested application
  • Design and implement highly reliable automated tests
  • Begin gaining value from test automation earlier
  • Integrate test automation into the business processes of the development team
  • Leverage test automation to improve your organization’s performance and quality, even without formal authority
  • Understand how different types of automated tests will fit into your testing strategy, including unit testing, load and performance testing, visual testing, and more

#8 The DevOps Handbook, Gene Kim, Jez Humble, Patrick Debois, John Willis

More than ever, the effective management of technology is critical for business competitiveness. For decades, technology leaders have struggled to balance agility, reliability, and security. The consequences of failure have never been greater―whether it’s the healthcare.gov debacle, cardholder data breaches, or missing the boat with Big Data in the cloud. And yet, high performers using DevOps principles, such as Google, Amazon, Facebook, Etsy, and Netflix, are routinely and reliably deploying code into production hundreds, or even thousands, of times per day. Following in the footsteps of The Phoenix Project, The DevOps Handbook shows leaders how to replicate these incredible outcomes, by showing how to integrate Product Management, Development, QA, IT Operations, and Information Security to elevate your company and win in the marketplace

#9 The Digital Quality Handbook – Eran Kinsbruner and Industry Experts

As mobile and web technologies continue to expand and basically drives large organizational business in virtually every vertical or industry, it is critical to understand how to take existing release practices for mobile and web apps to the next level, including software development life cycle (SDLC), tools, quality, etc. Organizations which are already enjoying the power of digital are still struggling with various challenges that can be related to many factors, such as:

  • SDLC and processes maturity
  • Expanding test coverage to include more non-functional testing, user condition testing, etc.
  • Coping with existing limitations of open source tools and frameworks
  • Sustaining correctly sized and up-to-date mobile test labs
  • Getting proper quality insights upon each test cycle prior and post production
  • Branching wisely cross-platform and cross-feature test suites

#10 Specification by Example , How Successful Teams Deliver the Right Software, Gojko Adzic

Gojko has a list of great books, this is one of the great ones, but check out other of his books as well.

Specification by Example is an emerging practice for creating software based on realistic examples, bridging the communication gap between business stakeholders and the dev teams building the software. In this book, author Gojko Adzic distills interviews with successful teams worldwide, sharing how they specify, develop, and deliver software, without defects, in short iterative delivery cycles.

#11 Clean Code,  A Handbook of Agile Software Craftsmanship, Robert C. Martin

Readers will come away from this book understanding

  • How to tell the difference between good and bad code
  • How to write good code and how to transform bad code into good code
  • How to create good names, good functions, good objects, and good classes
  • How to format code for maximum readability
  • How to implement complete error handling without obscuring code logic
  • How to unit test and practice test-driven development

#12 Continuous Delivery,  Reliable Software Releases through Build, Test, and Deployment Automation, Jez Humble and David Farley

The authors introduce state-of-the-art techniques, including automated infrastructure management and data migration, and the use of virtualization. For each, they review key issues, identify best practices, and demonstrate how to mitigate risks. Coverage includes

  • Automating all facets of building, integrating, testing, and deploying software
  • Implementing deployment pipelines at team and organizational levels
  • Improving collaboration between developers, testers, and operations
  • Developing features incrementally on large and distributed teams
  • Implementing an effective configuration management strategy
  • Automating acceptance testing, from analysis to implementation
  • Testing capacity and other non-functional requirements
  • Implementing continuous deployment and zero-downtime releases
  • Managing infrastructure, data, components and dependencies
  • Navigating risk management, compliance, and auditing

#13 The Guide to Software Testability, Ash Winter and Rob Meaney

Learn practical insights on how testability can help bring teams together to observe, control and understand the systems they build. Enabling them to better meet customer needs, achieve a transparent level of quality and predictability of delivery.

#14 A Practical Guide to Testing in DevOps, Katrina Clokie

A Practical Guide to Testing in DevOps offers direction and advice to anyone involved in testing in a DevOps environment

#15 Test Automation University, Angie Jones and Applitools

While not a book, a great shout out to my colleague, friend and co-author in one of my above mentioned books Angie Jones for launching, leading and building the Test Automation University. This is obviously an additional online resource that complements books and other written material.

Summary

I am sure that there are plenty of other great and practical books, but i went with this list as a start. If you feel that there must be an additional up to date book, reach out to me, and I will be more than happy to add to this list.

Happy Reading!

 

 

Posted on

Core Pillars of Stable Test Automation Suites

For those who follow me, you already know that to build a continuous testing suites that scales, and that lasts for more than few software iterations, teams must follow rigorous processes.

When i say teams – I mean developers, test engineers and business testers.

 

 

This post isn’t about balancing the work load between personas, but rather on continuously sharpening the test suites value to the entire team.

As we close a decade and enter a new one, and with some cool advancements in the technologies that include AI/ML but also challenging platforms like PWAs, Foldables, 5G and IOT – it’s the perfect time to re-evaluate our testing management and maintenance processes.

What Causes Test Instabilities?

Especially across the digital landscape, there are quite a few recurring patterns that causes test instabilities or flakiness.

  • There are the scripts and frameworks that we use – in this category we can list object locators strategy, popups (security, system), coding skills, timing and steps synchronizations and more.
  • There are backend issues around services, networking and up to date test data and environments.
  • Lab related issues that includes platforms being unavailable or poorly setup for testing (screen is locked, network is disconnected, app isn’t installed, device isn’t in ready state mode)
  • Lastly, Orchestration – This is a tricky root cause since there are not that many early warnings to these kind of issues up until you execute and scale up. Not having sufficient platform to test against, or knowing that a platform is in use is often hard to find (but – not impossible :))

In addition to the above 4 major categories, there are also of course the software development dynamics that includes advancements in the product, market changes that impact your test scripts and infrastructure, and also the external dependencies that you cannot always control (network carriers, 3rd party services that you depend on, etc.).

Test Automation Certification for Value vs. ROI

While there is no 100% safety net for the above mentioned RCAs (root cause analysis), but there are few steps that teams can take and enforce to minimize the risks.

My few recommended tips that should be examined every 2-3 software releases/iterations are:

  1. Think with the end in mind when you write a test automation script. Will the test provide value more than just once? Which test suites/types such test will fit (regression, functional, unit, etc.)
  2. Consider the maintainability of such test script over time, how difficult will that be? and will it be worth the hassle?
  3. Certify the test script prior to integrating it into any pipeline. Certification should address the stability of the test across multiple platforms (only works on my machine is not an option), test execution duration (Less than 5 minutes please), If it is a test that detected defects it needs to be scored somehow as a higher value test that others.
  4. Monitor and measure the impact of your test suites and correct not longer than once a quarter!

The above image is an example of an external dependency that can impact the performance of your app. Assuming you test your mobile app against real cellular networks, you should acknowledge that not every carrier behave and performs the same, and it may impact your overall results. Knowing, measuring and taking this into account can save false negatives and other wrong speculations.

Lastly, and as this section implies – it is time we stop measuring test automation ROI and $ but clear value and risk mitigation to the business. Test automation value is by far more wide than the legacy ROI metrics that were simply covering for cost of test creation and execution regardless of what the test executions actually results in.

I have recently delivered a session about test automation stability for the OnlineTestConf. You can view the recording and slides here

Power Of Analytics

While a lot of the above might be clear to many practitioners, what i often see in the market is that the time element prevents teams from actually seeing what’s wring with their testing infrastructures. Teams invest a lot of money, resources and of course time into building a cool set of testing suites, however once their done, the auto pilot steps in, and until something is broken, these suites remains blind spots. This is again, a legacy practice, and in the modern DevOps/Agile/Continuous Testing era, teams ought to monitor and gain maximum visibility into their test code continuously!

As the above visual suggest – only when you look into the reports in a smart way, you can improve quality, deliver fast and up to date feedback and Yes – get some ROI (in a form of value) from your investment.

Imagine you can upon demand answer questions like –

  • what are my most problematic test scenarios?
  • what are my most flaky or buggy platforms (mobile/web)?
  • which job within my CI is the longest?
  • Which test cases must be retired or be placed in a “blocked” bucket for maintenance (due to popups, objects, other)?

Knowing the above answers upon and perhaps prior to clicking on the RUN button could be a major productivity boost to your entire cycle, wouldn’t it?

The below are 2 real life example from the Perfecto tool, but you can think about similar implementation as well – In the below, based on AI and smart algorithms the tool scans the test reports for pre-defined patterns like popups, elements not found, platforms in use and others, but also allows teams to customize these and add unique and relevant RCAs on their own to better slice and dice the test data.

Bottom Line

As we wrap a decade of software development and testing, it’s perhaps time that we take our test automation a level up and inject smart algorithms, processes and more so we are always on top of what we build, and improve whenever there is a need.

Happy Testing!