Notes on “Debugging: The Good, The Bad, and the Quirky–A qualitative analysis of novices’ strategies”

June 24, 2009

Debugging: the good, the bad, and the quirky–a qualitative analysis of novices’ strategies: L. Murphy et al.

Programming novices warm up by solving a programming problem. They are then given an semantically incorrect syntactically correct solution to the same problem and asked to debug it. The researchers logged the activities during the debugging tasks, interviewed the subjects, and surveyed the subjects on debugging matters.

The researchers categorized the strategies/behaviours they witnessed. The 13* strategy categories match with at least one other study:

  1. Gain domain knowledge e.g. reread problem specification, examine sample output
  2. Tracing the program i.e. mentally or on paper, print statements
  3. Testing i.e. running the program
  4. Understanding code i.e. reading the code
  5. Using resources i.e. JavaDoc, Java tutorial
  6. Using tools i.e. a debugger
  7. Isolating the problem e.g. commenting out code, forcing certain control flow
  8. Pattern matching e.g. recognizing a cliched error
  9. Consider alternatives i.e. devising hypotheses
  10. Environmental e.g. documenting their work, using editor features such as undo
  11. Work around problem e.g. rewriting code, or introducing new code to work around errors in other code.
  12. Just in case e.g. Making unnecessary cleanup changes
  13. Tinkering i.e. random, unproductive changes

*The paper says 12 strategies, I think this was a typo since they list 13.

Many of the strategies were used ineffectively: Useless tracing, failing to note a failing test, shallow code reading, using unfamiliar tools, commenting out or modifying correct code, completely rewriting code. Some of the strategies are inherently ineffective, especially tinkering.

The papers also notes several amusing novice behaviours.

%d bloggers like this: