Please use this identifier to cite or link to this item:
https://dair.nps.edu/handle/123456789/4393
Title: | Increasing Confidence in Machine Learned (ML) Functional Behavior during Artificial Intelligence (AI) Development using Training Data Set Measurements |
Authors: | Bruce Nagy |
Keywords: | Machine Learned ML Artificial Intelligence AI Training Data Set Measurements |
Issue Date: | 10-May-2021 |
Publisher: | Acquisition Research Program |
Citation: | Published--Unlimited Distribution |
Series/Report no.: | Acquisition Management;SYM-AM-21-086 |
Abstract: | Both the commercial world and Department of Defense (DoD) are challenged with system safety issues when dealing with Machine Learned (ML)/Artificial Intelligence (AI) deployed products. DoD has a more severe issue when deploying weapons that could unintentionally harm groups of people and property. Commercial manufacturers are motivated by profit, while DoD is motivated by defense readiness. Both are in a race and can suffer the consequences from focusing too much on the finish line. Establishing formal oversight ensures safe algorithm performance. This paper presents a measurement approach that scrutinizes the quality and quantity of training data used when developing ML/AI algorithms. Measuring quality and quantity of training data increases confidence in how the algorithm will perform in a “realistic” operational environment. Combining modality with measurements determines: (1) how to curate data to support a realistic deployed environment; (2) what attributes take priority during training to ensure robust composition of the data; and (3) how attribute prioritization is reflected in size of the training set. The measurements provide a greater understanding of the operational environment, taking into account issues that result when missing and/or sparse data occur, as well as how data sources supply input to the algorithm during deployment. |
Description: | Acquisition Management / Defense Acquisition Community Contributor |
URI: | https://dair.nps.edu/handle/123456789/4393 |
Appears in Collections: | Annual Acquisition Research Symposium Proceedings & Presentations |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
SYM-AM-21-086.pdf | 2.24 MB | Adobe PDF | View/Open |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.