In an initial post on architectural quality factors, we describe the...

availability, modifiability, performance, security, testability, and usability

... of software intensive systems [Bass et al, 2013].

The goal of Continuous Delivery is for systems to be able to 'continuously demonstrate their state of production readiness' [Ford, 2015]. In order to do this, system components must carry low specification debt, and we require high testability quality factor be exhibited by all components within the CD pipeline.

[IEEE 90] defines testability as -

1. The degree to which a system or component facilitates the establishment of test criteria and the performance of tests to determine whether those criteria have been met
2. The degree to which a requirement is stated in terms that permit establishment of test criteria and performance of tests to determine whether those criteria have been met

To achieve high levels of testability, we require a holistic design focus - the 'entire system' must be designed, covering both functional and specification concerns -

• Repeatability and reliability of acceptance tests are critical quality factors
• Establishment of test criteria implies a design focus on the management of executable specification
• In addition to automated testing, a key testability tactic is to apply executable assertions to fast-fail and quickly reject invalid state as system boundaries; this is a key reason for DevOps users to acquire enhanced, usable views of production state via measurement and metrics

In design, a key heuristic is that we should focus on test in isolation in order to achieve high component level quality (build quality in). We are then able to assemble high quality systems from high quality components.

We discuss architectural evaluation of system testability in Architecting Continuously Testable Systems.

Materials

[Bass et al, 2013] Software Architecture in Practice (3rd Edition) (SEI Series in Software Engineering), Len Bass, Paul Clements, Rick Kazman, Addison-Wesley, 2013.
[Ford, 2015] Engineering Practices for Continuous Delivery, Neal Ford [pdf]
[IEEE 1999] IEEE Computer Society. IEEE Standard Computer Dictionary: A
Compilation of IEEE Standard Computer Glossaries, 610. New York,
N.Y.: Institute of Electrical and Electronics Engineers, 1990.