Managing the Development and Integration of AI-Based Mission Autonomy in Unmanned Systems: A Programmatic and System Engineering Approach

Abstract

The Department of War (DoW) is aggressively integrating artificial intelligence (AI) into unmanned systems across all domains. However, as autonomy increases, program managers and system engineers face new challenges in verification and trust assurance. The lack of standardized processes for validating AI behaviors within legacy command-and-control architectures complicates risk management and compliance. This project seeks to define a structured framework that enables effective planning, testing, and deployment of AI-enabled autonomy while maintaining system reliability and operator confidence. Traditional requirement frameworks rely heavily on positive requirements that define what the system must do. However, AI-based mission autonomy introduces behaviors that may be emergent, probabilistic, or non-deterministic, requiring a complementary emphasis on negative requirements that define what the system must not do under any conditions. DoW acquisition and verification processes do not yet provide structured methods for developing, managing, and testing these behavior constraints. This gap inhibits safety certification, complicates mission risk assessments, and limits operator trust in autonomous systems.