Using Natural Language Processing, Sentiment Analysis, and Text Mining to Determine if Text in Selected Acquisition Report Executive Summaries Are Highly Correlated with Major Defense Acquisition Program (MDAP) Unit Costs and Can Be Used as a Variable to Predict Future MDAP Costs

Abstract

Major Defense Acquisition Programs (MDAPs) are required to report cost, performance, and schedule information updates to Congress annually via a Selected Acquisition Report (SAR). One of the components of the SAR is its executive summary, which provides an updated outlook of the health of the MDAP as well as what direction performance metrics may be trending. The executive summary is entirely textual. Traditional MDAP analysis is conducted using structured, continuous, and categorical data attributes. However, analysis of text to predict program metrics has rarely been used. This research conducts sentiment analysis of SAR executive summaries to determine whether their average emotional valence sentiment is highly correlated with MDAP unit cost metrics. Negative correlation depicts that, as average emotional valence sentiment increases, unit cost decreases, and positive correlation depicts that as average sentiment increases, so does its unit cost. If the results show high correlation, then average sentiment in the SAR executive summary may possibly be used as a primary or proxy variable in models that predict future MDAP costs. The results of our study found that, at most, only 12% of MDAP SAR executive summaries produce strong correlations (|r|>=0.70) to possibly predict future MDAP costs.