Keeping Humans in Command: Preserving Agency in AI-Driven Federal Acquisition

Abstract

As artificial intelligence (AI) rapidly transforms government operations and private industry, the federal acquisition process faces a new paradox: the increasing use of AI to generate, evaluate, and award contract proposals. This convergence creates an “elephant in the room” scenario, where AI-enabled procurement systems in government may ultimately review proposals drafted by AI tools in industry, potentially diminishing the role of human judgment in critical decision-making processes. While automation offers efficiency, consistency, and scalability, the absence of human oversight risks eroding accountability, fairness, ethical discernment, and institutional knowledge in the acquisition process. This paper examines the dual-use evolution of AI in federal procurement, specifically how government acquisition offices are adopting AI-driven tools for market research, requirements generation, and proposal evaluation, while industry leverages similar technologies to optimize proposal writing and cost modeling. Drawing on guidance from America’s AI Action Plan and other federal AI policy frameworks, this research identifies the core risks of an AI-to-AI procurement ecosystem: diminished transparency, amplified algorithmic bias, and the potential loss of critical human judgment in evaluating qualitative and context-sensitive factors. The expected outcome of this research is a framework for preserving meaningful human oversight within AI-augmented acquisition environments. This includes recommendations for: (1) defining human-in-the-loop thresholds across acquisition phases; (2) establishing verification and audit mechanisms for AI-generated content; and (3) integrating AI literacy training for acquisition professionals. By confronting the AI-to-AI dilemma directly, this study seeks to confirm that automation enhances, rather than replaces, the human expertise, ethical reasoning, and strategic judgment that underpin the integrity of federal acquisitions. The goal is a balanced approach where AI supports data-driven decision-making, while humans remain the final arbiters of fairness, accountability, and mission alignment, preserving the essential role of human agency at the core of accelerating warfighting capabilities. This research directly aligns with the theme of this year’s Naval Postgraduate School Acquisition Research Symposium, “Accelerating Warfighting Capabilities,” by advancing an ethical and operational framework that confirms AI adoption strengthens, rather than undermines, the speed, trust, and integrity of defense acquisition.