Automated Methods for Cyber Test and Evaluation

Abstract

Cyber security of mission-critical software is a relatively new concern that is difficult to measure and hence difficult to incorporate effectively in software development contracts. The DoD has typically relied on black-box approaches to software testing. However, cyber vulnerabilities, particularly those deliberately injected into systems, are often statistically invisible with respect to affordable levels of black-box testing, which implies that they cannot be effectively detected using conventional testing techniques. This motivates augmenting traditional testing approaches with additional types of test and analysis procedures. This paper explores application of automated testing and other automated analysis methods to reduce cyber risks. We analyze several types of undesirable software behaviors and identify automated methods that could detect them within practical limits on time and computational resources.