Foraging for Quality

Dileep Marway, Head of QA, The Economist

Dileep Marway, Head of QA, The Economist

My name is Dileep Marway and I work for The Economist. The Economist offers authoritative insight and opinion on international news, politics, business, finance, science, technology and the connections between them.

Having worked in the engineering community for over 10 years I am extremely passionate about quality assurance. Quality assurance (QA) has moved forwards at rapid pace in the last few years, it is an area which is thriving and I am an advocate of ‘QA Engineers’ to advance themselves to strive to learn new skills, keeping themselves current in a booming industry.

You may have noticed that I have not mentioned the manual tester or automation tester in the first paragraph, there is some brains behind this madness, believe me!

To me, quality assurance is more than just making sure your application meets its requirements. It also functions as a form of communication across a team and application, meaning another engineer can confidently begin contributing to an application, using the tests as a guide.

Greater complexity of software applications and their increasing competitiveness have created the need for more efficient and exhaustive testing processes. When you have a complex piece of software the importance of functional testing is key, it de-risks going live with a poor product which does not meet customer expectations.

Synergy of Engineers and Quality Assurance Engineers

Functional testing (FT) is the empowerment of QA and engineers to guarantee that the highlights and functionalities of an application are in a state of harmony. It answers questions like ‘can the user do this’ or ‘does the particular feature work’. Without these a user could be impacted in a huge way.

Other functional tests look at its overall performance (e.g. testing scalability, reliability, security and compatibility). I’ve read many academics argue that these are non-functional aspects, though in my view anything that impacts the user experience or output is functional.

The synergy between engineering and QA is accomplished via the title ‘QA Engineers’ now being given to “modern” testing professionals. I champion that we should specialise in QA, but can cut code and contribute to the squad. The advantage of being able to cut code is that we can ‘shift testing left’ and can critique code quality.

Functional testing can involve the following testing phases:

1) Unit testing - focuses on a single “unit of code”, such as a class method.
2) Integration testing - builds on the unit test by testing multiple pieces of your application together.
3) Smoke testing – addresses the most basic functionalities crucial to the working of the product.
4) Regression testing – ensures that previously working functionality is still functional.
5) User acceptance testing – end users review the functioning of the product.

A key concept is to have a solid set of unit tests, issues can be found early and refactoring can take place with confidence. FTs there as a safety net rather than the first line of defence. Unit tests should be the car engine and FT’s should be the paintwork. Our QA Engineers are involved in helping setting unit testing standards, namely pushing TDD is key (explained below).

From experience tests should not be driven by the software development processes, but should be driven by what the user expects expects the function of the software to actually be.

Conditions for a great functional test

1) Derived from the product requirements.
2) Visible to stakeholders
3) Gives actionable information to developers
4) Audited frequently to ensure they are updated with product requirements
5) Correct set of data.

From experience tests should not be driven by the software development processes, but should be driven by what the user expects the function of the software to actually be.

6) Test cases cover all possible scenarios.

TDD / Test-Driven Development approach to FT

This refers to a process by which you develop and test your code. Under a TDD paradigm, your development process looks something like this:

1. Before coding, you write a test
2. Run the tests along with any other tests
3. Write the minimum amount of code necessary to make the tests pass
4. Refactor
5. Repeat the process

With TDD, you can get broad test coverage of your application. This gives you the confidence to make changes to your application as it grows, as you can see older tests passing and trust that your changes have not regressed.

BDD / Behaviour-Driven Development approach to FT

Behaviour Driven Development is a process by which you can approach development and testing. BDD is more focused on how you test than when you test. With BDD, you focus your tests on behaviour, rather than implementation, ideally starting from your customer’s/user’s expected experience. In my view leads itself are more so to functional testing.

Advancements in testing processes have now meant that more efficient ways of working are now available. For instance, the use of the Cucumber tool (BDD) for automating browser tests and Gherkin for human readable syntax allows you to provide a business user with detailed scenarios which outline how someone uses the system.

Also, the use of user stories early in the life cycle gives clear product expectations, making it easier to test.

e.g. User story – Companies are using the job search application and they should be able to create a job application.

FT:
1) Open application
2) Log in as the company
3) Create a job posting

Based on my experience, a user story should have explicit acceptance criteria. Good acceptance criteria are ones that “must” resolve to either a “Pass” or “Fail” after observations are made.

Automation vs Manual testing

Automation is more about checking or confirming that something is true than it is about testing or recognising unanticipated problems.

The Automated ‘product’ has helped to make processes more efficient, especially tasks such as regression testing. These processes have meant that functional testing can be done in a quicker and efficient way, leaving manual testers to work on edge cases.

Terming the output as an automation product is key, it is essential that the code is maintained and created in such a way that an engineer cares about it, but is also willing to contribute to it.

Not all tests can be automated; we still need manual testing in a variety of cases. It makes sense to take a manual testing approach when you’re dealing with legacy systems that don’t easily support automation, when you need to adhere to strict regulations and require documentation as a result.

Agile and continuous testing

API testing creates more reliable code. Historically, testing would take place at the graphical user interface (GUI) level. When a developer would finish their work, they would hand it off to the QA engineer. The engineers had limited time so they would test the code at the highest level - the GUI. This would cover both the frontend and the backend development.

This worked for manual testing and for the beginning of automation testing, but isn’t right for the age of agile and continuous testing. GUI testing is brittle and automated scripts break easily. In addition, teams can’t wait for the entire system to be updated and the GUI to be ready before testing occurs.

In the age of agile, testing must take place at a lower level of abstraction (API level). API tests can be created before development is complete. This means developers can validate their code based on pre-written tests (Test Driven Development).

Machine Learning

In the future machine learning driven test automation will be the key driver. In most companies, user interface (UI) elements are always changing, which means that test scripts fail as the UI has changed and tests are brittle. Artificial intelligence and machine learning mean that test adaptation can be done easily, this provides a workaround and the user is asked to ‘accept’ or ‘deny’ the workaround. To which the piece of test will adapt accordingly.

Summary

Today’s users are tech savvy, they use software in nearly every aspect of their daily lives and they have high expectations for how this software should work in order to help them go about their everyday needs better, smarter and faster. As a result, software testing has become increasingly important to and intertwined with the entire software development process.

Whether your team is using hard core test driven development (TDD) or simply wants to automate otherwise manual QA, FT benefits from clear, explicit user stories that have little room for ambiguity.