Continuous testing is a key element in successfully implementation of DevOps pipeline. Continuous testing, which is often called shift-left testing, is an approach to software and system testing in which testing is performed earlier in the software lifecycle, with the goal of increasing quality, shortening long test cycles and reducing the possibility of software defects making their way into production code. This concept of continuous testing can be achieved through progressive test automation as well.
One way that DevOps teams do Continuous Testing is by shifting integration testing to the left of its usual position in the delivery pipeline so that it occurs as close as possible to the build process and this shift left happens through progressive test automation scripts, which means your testing stages start in parallel to development stage. TDD and BDD are good example of doing progressing testing.
The goal of most DevOps projects is to automate as many manual processes in the software development process as possible. Some roadblocks in a DevOps pipeline that lead to slow deployment include error-prone manual processes such as handoffs from a development group to a QA group, including ones that require signatures or bureaucratic approval. These kinds of handoffs mean there is a lack of shared ownership of the end product, which is contrary to the basic Agile testing and development methodology that says all members of a cross-functional Agile team are equally responsible for the quality of the product or the success of the project. Because of this, testing on an agile project is done by the whole team, not just designated testers or quality assurance professionals, including team members whose primary expertise may be in programming, business analysis, database or system operation.
Continuous Testing is one of the several continuous activities that should take place simultaneously in a DevOps pipeline, including
- Continuous Build.
- Continuous Integration (CI)
- Continuous Delivery (CD)
- Continuous Deployment
Continuous build or build automation is the first stage in implementing a Continuous Delivery DevOps pipeline. If your developers are practicing test-driven development (TDD), they’ll write unit tests for each piece of code they write, even before the code itself is written. An important part of the agile methodology, TDD helps developers think through the desired behaviour of each unit of software they’re building, including inputs, outputs, and error conditions. New features implemented by developers are then checked into a central code base prior to the software build, which compiles the source code into binary code.
Continuous Integration is a practice where members of a software development team use a version control system and integrate their work frequently to the same location, such as a master branch. Each change is built and verified by means of tests and other verifications in order to detect any integration errors as quickly as possible; which may include your BDD scripts. With build automation, the software build happens automatically, using tools such as Gradle or Maven, rather than when a developer manually invokes the complier.
In the last stage of a Continuous Delivery pipeline, once an application passes all the required tests, it’s then released into production. For all intents and purposes, this means releasing every good build to users. A fully automated CD pipeline requires the ability to deploy and release any version of a software application to any environment.
DevTestOps = DevOps + Continuous Testing
Another aspect which can be added in Continuous Testing is to pull Non-Functional testing within your CI/CD pipeline scope. This practice can help achieve proper Shift-Left. i.e., performing performance, security, accessibility testing within your pipeline scope.
DevSecOps = DevOps + Continuous Security Testing
Patrick Dubois, who coined the term DevOps, said in an interview that he was inspired to promote increased collaboration between developers and operations because he felt, as a developer, “if you’re really passionate about what you’re building…you do feel responsible about how your application is doing in production.” In the past decade, DevOps has led to a cultural shift in the way many organizations deliver software applications.
These days, however, most organizations have realized that DevOps without continuous testing leads to the quicker deployment of buggy software into the production environment, which is why the term DevSecOps & DevTestOps are gaining currency as a way to describe the culture of collaboration needed among all of the teams involved in software development (developers, operations, quality assurance, business analysts, management, etc.) in order to speed up the delivery of high-quality software.
Building a successful Continuous Delivery pipeline means reducing the cost, time, and risk of delivering software changes by allowing for more incremental updates to applications in production. As your CD pipeline becomes automated, the obvious next step is to incorporate more quality control steps into the pipeline process, either through test-first approaches — such as test-driven development (TDD), acceptance test-driven development (ATDD), and behaviour-driven development (BDD) — or by continually running automated scripts(functional + non-functional) to test the new features. All these activities coincide with modern shift-left & in-sprint testing principles.
As your software releases increase in frequency, the suite of automated test scripts — including those that do the security, performance, load, spike and soak tests described above — can be stored and re-used in future release cycles. These scripts can also be incorporated into test automation frameworks and used to reassure project stakeholders that thorough and adequate testing has been successfully completed.
Security testing categories
- Vulnerability scanning (SAST & DAST)
- Security scanning
- Penetration testing
- Risk assessment
- Security auditing
- Ethical hacking
- Posture assessment
Application Security Testing Tools Pyramid
Tools & Frameworks which help in Continuous Security Testing
- OWASP ZAP
- Burp
- Snyk
- JFrog Xray
- Checkmarks
- Klocwork
- Aquascan
- Veracode
- eschecker
- Appknox
- Checkmark
Tools & Frameworks which help in Continuous Functional Testing
- Cucumber
- JBehave
- FirNesse
- TestLeft
- SpecFlow
- BeanSpec
- Concordion
- JDave
References:
https://medium.com/@hemanshu-chauhan/devtestops-bac3215eba25
https://continuousdelivery.com/foundations/test-automation/
Author: Hemanshu Chauhan
Hemanshu Chauhan is a thought leader in Quality Engineering domain with a blend of DevOps and Development experience. He has been in industry for over 12 years and has helped numerous clients in achieving Quality Engineering enabled solutions in DevOps & Agile work environment. As a result of his exceptional contributions in IT industry across the globe, Hemanshu has also been endorsed as Global Talent by Tech Nation UK society. He is a famous tech coach& keynote speaker at different international conferences and IT platforms like Medium, Meetup, Academia etc.
Contact Hemanshu:
https://www.linkedin.com/in/hemanshu-chauhan/
https://www.hemanshuchauhan.com
