Planning for AI Sustainment: A Methodology for Maintenance and Cost Management

Abstract

The sustainment requirements of Artificial Intelligence (AI)–enabled systems are largely unexplored within the Department of Defense’s programs of record (POR). Many programs often overlook maintenance needs for AI systems, extending beyond base hardware or software upkeep. However, prior research indicates a distinctive maintenance requirement for the machine learning models that power AI-enabled systems, and outlines strategies for planning and integrating AI maintenance into product support (Cruickshank & Kohtz, 2023). Notably, the adoption of industry best practices, program maintenance considerations, and machine learning operations (MLOps) are crucial for crafting an AI system’s sustainment strategy. This research builds upon the existing framework to further comprehend the extent of preventative and routine maintenance required by an AI-enabled system. We specifically investigate the degree of maintenance or “touch-time” needed to sustain a system’s machine learning model(s). By examining a typical year of operations and sustainment for an AI-enabled computer vision system, we highlight primary maintenance considerations (i.e., maintenance tasks, task difficulty, and task frequency) and propose a method to estimate these factors. We then apply varying levels of maintenance based on organic, hybrid, or contractor logistics support to fully comprehend the sustainment costs. Our research offers a robust framework for program offices to more accurately predict initial and ongoing operation and sustainment costs when conducting a business case analysis. This will enable the selection of the most cost-effective sustainment strategy for a POR that intends to use any AI-enabled system.