Experience Searching for Causal Factors in Personal Process Student Data

Abstract

The objective of this study is to apply recently developed techniques to infer causality from observational software engineering data. Determining causation rather than just correlation is fundamental to selecting factors that control outcomes such as cost, schedule, and quality. The Tetrad tool's PC and FGES causal search algorithms were applied to software engineering data from 4940 programs written in the C programming language collected during Personal Software Process (PSP) training. PSP programs have previously been used in empirical research quantitative relationships between developer and project factors. Both algorithms successfully identified the expected relationships and did not find contradictory or implausible associations. Many of the available causal inference search algorithms require Gaussian distributional families with linear effects. The linear relationship may be especially important for software engineering research and may require prior knowledge and data transformation. Because software engineering has depended on small-scale, low-power experiments, often using non-representative students, inferring causal relationships would expand the insight available to researchers. Inferring causation from observational software engineering data shows much promise, but is currently limited by researcher understanding of the capability and limits of causal inference, the quality of the underlying data, and the general requirement for linear effects.