A featured contribution from Leadership Perspectives: a curated forum reserved for leaders nominated by our subscribers and vetted by the Construction Tech Review Advisory Board.
Sauce Labs
5 Transformative Insights in Software Testing
1. Automated testing pipelines are an essential enabler of great software development organizations. Automation of tests is a huge step in mitigating the rising cost in testing software that comes with the accelerated pace of Agile and DevOps release cycles. The more tests you automate, the smaller the investment you have to make in manual testing resources, and the more you can rely on testing for rapid feedback during development. The standardization of open source frameworks like Selenium and Appium makes it easy and cost effective to automate your tests across a broad spectrum of operating system, browser, and device combinations. You write your tests once and all of those platforms are automatically supported. And the use of cloud services like Sauce Labs makes it even more affordable as you no longer have to invest in building and maintaining the infrastructure of an internal testing grid.
2. Testing belongs in the cloud. This is because testing is exactly the kind of problem that cloud computing excels at: It's typically highly parallelizable, and the need for quick testing feedback cycles also makes for a "spiky" load pattern. By using cloud computing, software testers can devote significant computing resources to testing just when they need them, without bearing the cost of those resources when they don't need them. Because they're provisioned on demand, cloud resources also have the opportunity to provide a consistent testing environment, increasing test stability by eliminating problems related to persistent state.
Why not just spread out those load spikes by using fewer computing resources over a longer period of time? The answer is that the benefits of rapid testing feedback to a software development cycle are too important to compromise.
Software testing sits at the center of a feedback loop that is critical to innovation.
Software testing sits at the center of a feedback loop that is critical to innovation
3. Mine your testing data for insights. Organizations have a wealth of automated testing data that can be used to discover bottlenecks and problematic tests. If used successfully this data can dramatically speed development by making tests more reliable, and accelerating the overall testing process. Unfortunately, organizations are often overwhelmed by the sheer volume of data, as well as the development effort of operationalizing the analysis and dissemination of the information to their development and QA organizations. However, recently several open source efforts have emerged to ease analysis of testing data and the creation of dashboards. Two notable ones are Capital One's Hygieia project and Walmart's TestArmada Admiral tool.
Taking the next logical step, big data and machine learning technologies are already in use, and more and more enterprises will come to rely on the benefits of adopting them. The advancements in these technologies make data powerful in helping with strategic and tactical business decisions. For example, by analyzing usage patterns, Sauce has come up with a way to identify customers that are struggling in fully utilizing our service even before they reach out to us. This allows us to be proactive in helping them make the most of their investment in Sauce.
4. The future of automated testing depends on good standards. As browser vendors have added support for the increasingly standardized WebDriver protocol, automated testing for the web has become dramatically simpler and more effective. The need for test automation standards in mobile testing is a part of what has driven the adoption of Appium (also based on WebDriver). Looking to the future, I think the same trend will continue. Even more than we saw with the adoption of mobile, IoT software testing promises to be challenging. Because the variety of devices and device capabilities is so much greater than even in mobile testing, balancing adequate device coverage against testing costs will be a complex problem. Testing framework standards and device virtualization technology may be helpful and necessary in bringing software development and testing costs under control. There's an opportunity here to bring some of the lessons learned from web and mobile testing to bear on IoT, so that as an industry we can get a running start on a difficult problem.
5. Focus is key. The lesson I've learned that has had the biggest impact on my organization is to focus as much as possible on our core competency. In build vs. buy situations regarding non-core technologies, lean toward "buy" as much as possible. We've seen the benefits of this choice every time we've made it. If you don't need to build infrastructure and can instead use cloud solutions, do not hesitate to go with the cloud. This allows you to focus on putting business solutions together rather than worrying about building and maintaining infrastructure. This is why Sauce is so valuable to our customers: if they don't have to worry about building and maintaining their internal testing grid (PC servers, Mac servers, Linux servers, Android Emulators, iOS Simulators, Android devices, iOS devices, OS images, browser images, and how they all work together), they can focus on building a world class test suite that will insure high quality services. This, in turn, allows the CIO to spend much more time and energy on business strategy rather than non-core technology tactics.
