Notes On Some Cognitive Models of Debugging

Toward a theory of computer program bugs: an empirical test—Iris Vessey

Does serial position of a bug in a program affect debugging?

Does height of the bug in the AST affect debugging?

Experiment: Programmers of varying experience debugging programs {low,high} in the {serial program, AST}.

Results:

There is a significant effect of expertise on debugging.

Anecdotally, they observed vastly different debugging strategies.

Claim: Supported claim debugging is fundamentally about program comprehension.

Towards a model of programmers’ cognitive processes in software maintenance: a structural learning theory approach for debugging—Hale, D.P. and Haworth, D.A.

Mental model for debugging/maintenance based on Structural Learning Theory.

Declarative models describe structure of knowledge:

• Syntactic knowledge: Syntax of the language used.
• Semantic models: Language-independent data types and algorithms

Semantic models have multiple levels. Semantics of language all the way up to domain knowledge.

Procedural models describe programmer activity during task.

Debugging strategies:

• Direct diagnosis
• Control flow: Follow the control flow of the program beginning to end
• Data flow: Work backward from output
• Slicing: Identify all statements that influence a suspicious variable

Debugging performance not significantly correlated with lines of code or two complexity metrics.

Model for debugging: Search problem.

Problem : w.r.t. Output, Actual != Required

Goal: w.r.t. Output, Actual == Required

Find a rule s.t. goal is a possible output of the rule and the current situation is in the domain of the rule.

If no rule available, switch goals s.t. solving new goal will change the situation. Otherwise apply the rule and check if the goal is satisfied.

Direct diagnosis is a rule.

Situational variables may influence choice of strategy or hypothesis.

Unfortunately, the paper lacks examples of how to use the model in practice. It also lacks examples of rules.

An evaluation of the cognitive processes of programmers engaged in software debugging—Hale, J.E. and Sharpe, S. and Hale, D.P.

A protocol analysis of 20 professional programmers performing a simple, small-program debugging task. Supports the Hale and Haworth model. Specifically: Hale and Haworth have the right process steps. The order of the steps is partially supported. Process steps (not in order):

1. Evaluate Program (32% of debugging time)
2. Update representation (30% of debugging time)
3. Generate hypothesis
4. Goal/Rule Match
5. Switch Goal (21.5% of debugging time)
6. Implement Correction
7. Verify Correction

Some empirical differences from the model:

• Programmers do not switch goals after attempting a goal/rule match
• Programmers do not implement corrections after attempting a goal/rule match
• Programmers do not evaluate the program again after verifying a correction