Notes on “Development and evaluation of a model of programming errors”

June 24, 2009

Development and evaluation of a model of programming errors: Ko and Myers.

The authors created a model of programming errors based on J. Reason’s general model of human errors.

In the model, breakdowns can occur in specification activities, implementation activities, or debugging activities. These breakdowns cascade into further breakdowns resulting in the introduction of errors in the program. A breakdown is caused by a knowledge, attentional or strategic problem when performing one of the subactivities.

Thus, we can categorize an error by the chain of breakdowns that led to it. Each breakdown is composed of a broken artifact, the activity that was being performed on the artifact at the time of error introduction, and whether the underlying problem was knowledge, attentional or strategic.

The authors validate the model using two observational studies using the Contextual Inquiry method. This means the experimenters tracked the subjects’ goals as they worked in addition to recording their activities. They fit the data using the model and show that the model has good descriptive and explanatory power.

This is among the nicest theory-driven ESE papers I’ve read. I think it’s a great approach. I suspect the authors of the “theory use in software engineering” paper would approve.

Now that they’ve planted this model:

  • We can ask of future studies of errors in software engineering, “Why didn’t you relate this work to the model?”
  • We can measure the reliability of the model by checking whether different observers of programming activity fit errors and breakdowns to the model in the same way.
  • We can merge in future refinements of the underlying cognitive psychology model.
  • We can develop baselines of prevalence of the various breakdown possibilities, and show how these change as programmers gain experience.
  • We can begin to measure quantitatively how the various constructs interact, then design experiments that control for these interactions.
  • Etc.

All this considered, it’s too bad Ko has moved on.

%d bloggers like this: