Using Natural-Language Processing and Large Language Models to Restructure DoD Comptroller Budget Materials into Portfolio Views

Abstract

When the yearly President’s Budget (PB) is submitted to Congress by the Department of Defense (DoD) Comptroller Budget Documents is complemented by “Justification Books” or “J-Books.” These detailed documents provide budgetary details by individual programs, projects, and activities within individual military departments or defense agencies rather than from an integrated portfolio or mission perspective. This disjointed structure makes it difficult for non-DoD insiders (and likely congressional staff for whom these materials are intended) to understand the net operational effect of the requested investments let alone their constituent program elements. Given that final reports Section 809 Panel and subsequent statutes provide that the DoD should use data-driven portfolio management for acquisition and capability investments, we asked how the existing J-Book documents could be restructured to facilitate a portfolio view. Our paper first provides the results from our exploratory use of natural language processing (NLP) techniques to perform a key word search across multiple J-Books to extract and subsequently process the content associated with a key word. For the purposes of demonstration, we focused on using these techniques to identify disparate elements of Joint All Domain Command and Control (JADC2) in these J-Books. JADC2 was chosen as this DoD strategy spans multiple service’s Research, Development, Test & Evaluation (RDT&E) J-Book volumes. This research also explored whether emerging large language models (LLMs) could be used to answer different types of portfolio or other questions about DoD spending without changing the existing layout and document delivery approach. We provide the results for our implementation of a dashboard proof-of-concept with an LLM interface from refactoring these budget materials including a temporal analysis of the J-Books content spanning multiple years. The final demonstration’s use case is from the perspective of a new congressional staffer trying to understand the differences between these budget materials across the years.