There’s a point in time after you get involved in anything to do with software testing where you’ll stumble across the testing pyramid, which was first put forward in Mike Cohn’s 2009 seminal book Succeeding with Agile.
The testing pyramid effectively creates a hierarchy of testing over 3 key levels, where the higher up the pyramid and those 3 levels you go, the inference is you’ll be spending more time and money, while the width of each segment denotes the volume of tests normally occurring at each stage: lots at the base, reducing to very few at the top. So, what are these 3 levels and how do they work?
The first level – the base – is Unit testing – usually performed by developers to test each component at the code level. Unit testing is the base of the pyramid because it’s typically the first testing done on any software development, but it’s also the largest volume of testing – there are lots of small components to be tested, so there are lots of unit tests. They are quick and relatively easy to do.
Next comes the middle segment of the pyramid, which is Integration testing. This is where integrations between components are tested to see how those components work together in relation to the technical specification & requirements. These are in the middle of the pyramid because they come after the unit tests, but the volume here decreases – there are typically less integrations than components, so you can do more within each integration test, but also they are more complex and take more time.
Lastly, at the top of the pyramid comes the UI tests. These are tests performed in the user interface. These would be toward the last tests to be performed – code and integrations typically come before a functioning UI – but they are also usually the lowest volume of tests to occur, largely because the UI and workflows that go across it encompass many functions, requiring a larger degree of complexity and subjectivity – they involve the user after all!
And once you understand this concept of the testing pyramid, you also start to understand why developers want to get as close to the code as possible with their tests – unit & integration testing – because it can be quicker to perform. But also why UI testing (and any mandate to optimize it) comes with a strong focus on reducing cost, time and effort – you need to test the UI, but you also have an eye on the clock, budgets and available testing manpower. No one ever has enough time (or money) to test everything!
So leaving aside unit testing – developers will tell you they’ll take care of this – if we go from the top down and start at the UI testing, while test automation tooling is empowering testers (and even non testers alike) to go faster, be more efficient, more productive and more cost effective when testing in the UI, there’s always going to be a level of optimality in what automated technology can offer at any point in time, even with AI/ML.
Which means the old adage of “work smarter, not harder” comes into play. Because actually when you look at a lot of UI testing, the majority – arguably – is not testing at all. It’s test data management – TDM. Creating, updating and cleaning down data to provide the environment in which the actual test can occur.
Think about testing a CRM. The test says “validate a user can update the stage of an opportunity”. Seems simple. But to test an opportunity status change, you need an opportunity. And the opportunity needs to sit within an account. But if the data is unknown, not present or not valid, UI testing must involve setting up that data to then test with it. So you end up with long UI tests for which 50%+ of the test steps can simply be creating the right state of data – the account & opportunity in the CRM case – to eventually perform the test against.
Therefore a smarter approach would be beneficial. Brains to go with the automated tech. The smart play here is to travel down the testing pyramid and combine the UI tests with integrations. Basically, use integrations- whether APIs or database connectivity- together with test data repositories or generators to remove those 50%+ data management “test” steps and in one fell swoop, make a few API or DB calls that take milliseconds, chop out those data creation test steps in the UI and create the same data via integrations. Then you can focus the UI testing purely on performing the objective of the test against.
The result? When using automation that incorporates UI with APIs (or other integrations), tests become 50%+ more efficient in test creation and maintenance, while being 75% faster to run. So by drawing a circle around the UI and integration segments of the testing pyramid to combine them together, time and cost goes down when compared to testing in the UI alone. Which then creates the environment to either scale up testing- increase the % of code tested at all three levels of the pyramid because you have more time & capacity- or release sooner with quality still assured. Or combine a bit of both.
Not a bad place to be, so long as you adopt the right strategy and automation to get you there.
