Tips for CDRLs/Requirements when Acquiring/Developing AI-Enabled Systems

Abstract

The Department of Defense (DoD) is challenging Acquisition professionals to manage the development of systems incorporating AI functions either as upgrades or new programs of record. But, AI functions present unique challenges associated with requirements and subsequently when creating a suitable Contract Data Requirements List (CDRL). The problem stems from the ability to ensure the quality and quantity of training data sets which can limit the reliability of AI performance. Currently, there is limited guidance regarding topics for discussion during an AI requirements review or as to what AI related information should be required in CDRLs. However, a recent investigation into the lack of AI development guidelines prompted a NOSSA-funded project. Using an AI “sandbox” approach, a DoD representative program, involving AI/ML algorithms supporting a mission planner with autonomous vehicle selection and navigation, was used to determine realistic requirements specific to systems incorporating one or more AI functions. As a result of their analysis, this paper presents contents for an AI Development Plan (AIDP) to be part of a CDRL. Within the AIDP, measurements and new evaluation methods are also offered, as well as questions and considerations to support quality AI development.