Avoiding the “I’ll know it when I see it” Pitfall: Furthering A Choice-Based Conjoint (CBC) Model for Government Source Selections

Abstract

The Department of Defense (DOD) current source selection methods are at an increased risk of experiencing sustained bid protests. During source selections, the government frequently contradicts itself between its advertised stated order of importance for acquisition evaluation criteria (pre-award) and its actual choice behavior during source selections (Butler, 2014). This paper provides a summation of research, conducted from 2021 to 2022, that explored the following research objectives: 1) Determine the degree of disconnect between stated preferences during pre-award acquisition phase and actual choice behavior in defense acquisition source selections, 2) develop a deep understanding of quality attributes in evaluating logistics-based service acquisitions, 3) provide a Choice-Based Conjoint (CBC) framework that the DOD could utilize to enhance source selection criteria development in both logistics and further categories of government spending. The research utilized methods such as interviews and spend analysis techniques to identify quality attributes of logistics-based acquisitions that would best discriminate as evaluation factors for award. Later, these attributes were used to develop a CBC exercise that enabled us to calculate attribute utilities and relative importance for each attribute. The summarized research in this paper provides a way forward to empirically deduce the relative importance for source selection evaluation factors, potentially reducing bid protest occurrences in future source selections.