Requirements traceability forms the foundation for thorough software testing

January 24, 2017

Blog

The ability to define and bi-directionally trace requirements throughout the software development lifecycle and throughout all the development artifac...

The ability to define and bi-directionally trace requirements throughout the software development lifecycle and throughout all the development artifacts is an indispensable aspect of developing high-assurance software. In many cases, it’s also part of having the code certified by regulatory bodies. And it’s a critical element to ensuring that today’s critical software is reliable, safe, and secure. All three of these aspects, but especially security, which is a foundation for the other two, must be built into code from the ground up. It has simply become too complex to tack on like a bandage.

To accomplish all three of these elements, developers must be able to clearly state the functional, safety, and security requirements of the system and then have the ability to verify that the software and system meets those requirements. They need requirements traceability, which includes the ability to trace a requirement from its document to the code that carries it out, as well as to trace from that code back to the requirement.

There was a time when traceability could be done with informal methods like walk-throughs, but today’s code has become so large and complex—not to mention vital—that such methods can simply no longer be trusted. Automated tools can test both to see that the code exists to carry out the established requirements and also to make sure that such code isn’t compromised by subtle coding errors. The first step is to establish bi-directional requirements traceability that can follow code from each requirement to the source code that implements that function, and similarly be able to trace from that source code back to the requirement.

Among the tools needed to accomplish the goals of software quality through verification are static and dynamic analysis. Static analysis tools work with the uncompiled source code to analyze it for various quality aspects such as clarity, consistency, and complexity, as well as check it against coding rules to comply with specific safety- or security-related coding standards. Furthermore, static analysis can be used to establish the foundational knowledge of data and control flow, which are vital to understanding the potential weaknesses and vulnerabilities in the code. That is, who, or what persons or software entities have access to which data and how can they affect control? This helps determine if a given required function is gaining the needed access or if unauthorized entities may also be doing so. Static analysis also enables automatic test-case generation by determining the proper stimuli for the software components in the application.

Dynamic analysis, on the other hand, runs on the compiled and executing code. Dynamic analysis uses the information generated during static analysis to test the compiled code and coordinate it with the source code to automatically generate a suite of tests. These tests can then be supplemented by manually created tests derived from the requirements documents. These can include functional security tests, such as simulated attempts to gain control of a device by feeding it incorrect data.

Coverage analysis provides a measure for how much of the code has been exercised, but more specifically through graphical visualization provides branch/decision coverage and procedure/function call coverage, providing more insight into the code’s response to the given stimuli. This tells you not only if the requirements have been addressed, but also if they’ve been correctly executed without hidden hazards. The combination of requirements traceability with coverage analysis can also turn up areas of “dead code,” or code that’s never executed. This code can mostly be an inconvenience, but it can also be a security threat if a hacker can gain access and from there gain control. It’s code that can’t be traced and should therefore be eliminated.

The ability to define and trace requirements throughout the lifecycle and throughout the development artifacts including the executing code is an indispensable aspect to being able to verify code, and in many cases to having code certified by regulatory bodies. In today’s world of increasingly complex software, it must be accomplished with automated tools. Requirements tracing is both supported by and supports other testing activities to make up the whole picture, but it represents a foundation on which to pursue the other activities, such as standards compliance, correct coding, unit system and integration testing, and more.

Jay Thomas, a Technical Development Manager for LDRA Technology in San Bruno, Calif., and has worked on embedded controls simulation, processor simulation, mission- and safety-critical flight software, and communications applications in the aerospace industry.

Jay Thomas, LDRA Technology
Categories
Debug & Test