Risk Management and Information Assurance Decision Support

Abstract

Like any organization, the DoD still relies on security analysts who can ensure that security requirements are satisfied. Relying on one expert's opinion can be risky, because the degree of uncertainty involved in a single person's decision could increase with time, memory failure, or inexperience. In previous work, we introduced the multifactor quality measurement method (MQM) where we reduce this risk by collecting security ratings from multiple experts with documented expertise in specific technical areas of cybersecurity. The next step is to automate the scenario generation where less experienced IT personnel can create scenarios that correspond to their own system architecture using our tool. The automation allows one to crowdsource security assessments from experts. The tool will collect and analyze the expert ratings and return the results to the original requestor. In this paper, we propose our designed prototype for the tool and we share the results of evaluating the prototype on 30 students who are completing a master's degree in cybersecurity at Carnegie Mellon University. Based on the qualitative and usability analysis of responses, our proposed method is shown effective in systematic scenario elicitation. Participants had a 100% task completion rate with 57% of participants achieving complete task-success, and the remaining 43% of participants achieving partial task-success. Finally, we discuss our findings and future directions for this research in systematic scenario elicitation.