Archive for April, 2009

Notes on “Why Programs Fail”

April 30, 2009

Greg lent me Why Programs Fail so I could read about program slicing. What a great book! I wish I had it when I was working on DB2. It details many tools and techniques that would have been tremendously useful for me, for my team and in fact for anyone working on a large code base. In particular my mind was blown by the example of 30-minute automatic debugging of a gcc optimizer crashing bug. I think the book would make a phenomenal textbook for an undergraduate class.

From a research perspective it wasn’t too interesting, though. The information on slicing didn’t go beyond what I already wrote about.

There’s a second edition due in May. I’m looking forward to it.

Amazon links below for more information.

Why Programs Fail: A Guide to Systematic Debugging

Why Programs Fail: A Guide to Systematic Debugging (Second Edition)

Software Reuse in the Large

April 29, 2009

One more thesis idea to close out the day. I’ve done no prior art research on this one and I’m almost certain much of this ground has been covered. I welcome links and pointers.

Now that there are vast libraries of available code online, much of programming is about selective reuse.

How do programmers reuse code? How do they find candidate (external and internal) modules to solve goals? How do they evaluate whether a piece of software meets their needs, both technically and legally? How do novices and experts differ in this process? What do programmer reuse searches tell us about the language of intentions? What questions do programmers ask during these tasks? Among some sample of open-source projects, how much code is internal and how much is external? How do small and large projects differ? Do these reuse ratios correlate with anything interesting?

Summary of Potential Thesis Directions, April 29 2009

April 29, 2009
  • Personality studies of programmers. It turns out personality effects are pretty weak, and the results of personality studies of programmers are probably not all that surprising.
  • Program comprehension/drills for program comprehension. Research in program comprehension was very popular in the eighties and early nineties. It’s died down since then. I’d be trying to make a mark in a very crowded space.
  • Technique drills for programmers. It’s too hard to measure improvement.
  • Version Control. I’m not sure if there’s a whole thesis here, but I might be able to get a paper out of it, e.g. A Reference Architecture for Version Control Systems
  • Citations Comparison. Another one that seems more paper-worthy than thesis-worthy. Not really Computer Science specific, either. [Blog post]
  • By the way, did anyone study any real programs? I could go looking and do a review of research that was done on toy/research programs that could be repeated on real programs. Also not clear if it’s large enough. See Would Dicing Help.
  • Would Dicing Help? A lot of ground has been covered but I think there’s room for more. [Blog post]
  • Motivation and Programming. It wouldn’t meet some of my goals discussed below. [Blog post]
  • Static checker info in an IDE. I’m not familiar with the existing work in this area yet. [Blog post]
  • Software reuse in the large. I’m really not familiar with the existing work in this area yet. [Blog post]

I need to be studying for an exam, dammit, not dealing with trivialities like the next nine months of my life!

Here are the goals I’m trying to satisfy:

  • Research people care about. I won’t be able to sustain my interest in a subject unless important and powerful others care about it. If I’m going to spend the better part of a year on something, it had better matter.
  • Startup-preparation. My current post-graduation goal is to start a company. I’d like my thesis work to align with that goal as much as possible. Ideally my thesis involves building something I could later commercialize. Failing that I’d like to learn skills that will significantly help me commercialize something. Failing that I want my thesis to be easy so I can spend time on better things.
  • Scope. I’d like to err on the smaller side. I have lots I want to learn that isn’t thesis related.
  • Risk-mitigation. I have near-zero tolerance for risk in this endeavor. I’m just not willing to entertain that famous “you don’t know if it’ll work” research mentality when I’ve only got nine months. I want (and believe I can have) a plan with clear effort-reward correlation at the end.

Next steps:

I’m dangerously uninformed about the static checker/IDE idea. For all I know it’s already out there since it’s an obvious idea. I need to look for it. I also need to look into the software reuse area.

ICSE is the leading target for people in our area. I need to skim ICSE proceedings to get a sense of what interests people today. I still need to skim through Jorge’s thesis, too.

Rory et. al estimate it’ll cost me a week solid to read everything from Steve Easterbrook’s Empirical Studies course. If I’m going to design an empirical study, I need to find that week.

I’ve got an exam on the fourth and leave for ICSE ’09 on the fifteenth!

Type inference, static checkers and IDEs

April 29, 2009

In Haskell every function has a type. If you don’t tell it the type of your function, it’ll infer it for you.

Prelude> let plus a b = a + b
Prelude> :t plus
plus :: (Num a) => a -> a -> a

Some Haskell programmers write out the type of every function they write. It’s a tradeoff. The function becomes a little more resistant to change, because you might need to change the type signature if you change the function. In return, you get compiler-checked documentation for your function, right there in the code.

Of course, data in other languages have types too, whether or not the language enforces them. It would be perfectly possible for a JavaScript static checker to figure out there’s a problem if you subtract a string from an integer.

var i = 7, s = "asdf";
print(i - s);

The types of i and s might change later, but at the print statement, they’re a number and a string.

I’ve discussed this with Patrick Dubroy and Zak Kincaid, and we agree: A better way would be to have your IDE perform the analysis and make the information available live. I’m not sure how the information would be ideally presented (maybe as a tooltip, or as a specially marked insertion in the editing window) but it would be great to explore.

Motivation and Programming

April 29, 2009

We know that motivation is the single most important factor in the acquisition of expertise. It is surprising, then, how little is known about the motivation of programmers and programming students.

Some sociologists and economists have looked into it, curious about motivations of open-source developers (e.g. a survey of Linux kernel developers). Educators have an obvious interest (e.g. A survey of student motivation at two schools in the UK). There’s also some research by corporations and governments, some research into the use of games for motivation, and an investigation or two of cultural differences.

To me this looks like a ripe area for further research. Someone needs to study student motivations in way that controls for response bias. We could go looking for gender or cultural differences. We could compare students in different schools.

We could compare professionals of different kinds. Big company vs. small company. Maintainer vs. commando. Tester vs. front-end engineer.

We could do an experiment on the students of the summer programming course. We could compare the use of toy assignment problems to beautiful elegant mathematical problems to seemingly-applied business and science problems.

Slicing, Dicing and Chopping: Little-Known Advanced Tools for Programmers

April 29, 2009

Slicing, Dicing, Chopping and other Mysteries

Most programmers have never heard of slicing, dicing or chopping applied to computer programs. They are simple ideas, and most advanced programmers have probably at least used slicing without knowing it. Unfortunately, the more powerful applications seemingly never made it out of academia.

In 1981 Mark Weiser published “Program Slicing.” Weiser was a phenomenal researcher—he was a chief scientist at the renowned Xerox Palo Alto Research Center and he is the academic father of ubiquitous computing. It’s a shame his other accomplishments overshadow this one.

Slicing is simple. Given a program behaviour, we can construct a new, smaller, simpler program that has the same behaviour: We just delete the irrelevant parts of the program that didn’t contribute. Given a bug, slicing removes the code that could never have contributed to the problem.

Most programmers do this mentally to some extent, and occasionally some will actually delete code during debugging. A few tools to do automatic slicing exist, but they’re rarely used.

Slicing: Show only code that contributed to bad behaviour.

Dicing extends the concept. If we have examples of code that runs correctly, under certain conditions we can assume that the code that contributes to the correct behaviour is correct. So we remove it, too. That leaves a much smaller program to examine.

Dicing: Show only code that contributed to bad behaviour and didn’t contribute to good behaviour.

Chopping is another related idea. If we insert some new code, and it reveals an incorrect behaviour later in the program, chopping shows us the code between the new code and bad behaviour that was affected by the inserted code.

Chopping: Show only code that contributes to bad behaviour and was affected by some given piece of code.

Static slicing does not use the values of program inputs.

Dynamic slicing works with specific values of program inputs.

Conditioned slicing works assuming a given logical condition.

Amorphous slicing works using semantics-preserving source transformations other than code deletion. All of the other techniques above only use deletion to simplify a program. Amorphous slicing permits more powerful techniques.

Harman and Herions have a good review of slicing literature as of 2001.

Tools

What would it take to use a dicing tool?

To find a bug, you gather a trace of all the code executed until the observable bug behaviour. That’s your slice.

Your slice could be large. Now you want to dice it, that is, eliminate known-good code. If only you had a large number of known-good executions of your program! You could use them to hone in on bugs, fast.

In the heyday of slicing research, the eighties and early nineties, it might have been hard to find a large library of known-good executions. These days, it would be easy. Automatic test suites are cool now. Everybody’s got one.

In fact, this is exactly the insight that Hsin Pan had at Purdue University. Pan invented a set of heuristics to combine test data with dicing. In fact, Pan wrote a PhD thesis on the subject and even built a whole debugging infrastructure called Spyder. Here’s a technical report and the dissertation.

Had you heard of Spyder before today?

Me neither.

Directions

In the first Empirical Studies of Programmers workshop, Bill Curtis published “By the way, did anyone study any real programmers?”, decrying the glut of empirical research on novices and lack thereof on professionals.

I propose a modern variant: By the way, did anyone study any real programs?

We have at our disposal the source code and version history of thousands of software projects. Some of them also track bugs thoroughly and link these bugs to test cases. Using such a project, it should be possible to retroactively test a dicing tool.

We have a version of a project that contains a bug. We eventually have a test case which exposes the bug. We use these with a dicing tool to produce suspect lines of code. To derive a perfectly quantitative measure of dicing tool quality, we compare these suspect lines of code with the code affected by the eventual patch for the bug.

I see another direction for dicing research: To my knowledge, no popular IDE supports dicing. We ought to be able to devise a good user interface for dicing. We wouldn’t even need to build a dicing-tool: We could make paper prototypes. I’d like to explore a dicing interface that highlights suspect code the same way Shark Code Browser highlights slow code in yellow.

Debugging and maintenance is a major part of software development. One might assume that as an industry so concerned with efficiency we would use every tool available to us. It’s clear though that opportunities abound for some innovation.

Michael Lee-Chin’s Tips for Success

April 28, 2009

Michael Lee-Chin is the billionaire entrepreneur behind the very successful Canadian investment company AIC. (That’s C, not G like the disaster-ridden American insurance company.) He’s incredibly inspirational—a true rags to riches self-made man. Here are some notes from a talk he gave today at the University Club in Toronto.

  1. Principles. Have principles and stick to them.
  2. Passion. Be passionate about what you do. With passion, you can outwork anyone who isn’t as passionate. Having strong principles makes it easier to be passionate.
  3. Patience. Success comes slowly. Five years of hard work is a bare minimum. Having passion makes it easier to be patient.
  4. Perseverance/Perspiration. Expect to work hard. All of the above make it possible to work harder.

Every successful businessperson does the following five things:

  1. Owns just a few businesses.
  2. Understands these businesses thoroughly.
  3. These businesses are in industries that are positioned for long-term growth and profit.
  4. Holds these businesses for the long term.
  5. Use other people’s money to enhance the rate of return.

On a personal level:

  1. Build a good reputation and let nothing touch it.
  2. Differentiate yourself. Do not become a commodity.

He believes in “Do well, do good.” That is, it’s not enough just to be personally successful. One must also give back to society.

Another big lesson is to ask for what you want, no matter how unreasonable it might seem.

Some key quotes:

“Danger = Crisis + Opportunity” (from Chinese)

“Risk more than others think is safe, care more than others think is wise, dream more than others think is practical, expect more than others think is possible.” (of unknown origin)

“To invest successfully over a lifetime does not require a stratospheric IQ, unusual business insights, or inside information. What’s needed is a sound intellectual framework for making decisions and the ability to keep emotions from corroding that framework.” (from Warren Buffet)

Notes on the Empirical Studies of Programmers, Seventh Workshop

April 26, 2009

An Experimental Evaluation of Quality Guidelines on the Maintainability of Object-Oriented Design Documents

Lionel C. Briand, Christian Bunse, John W. Daly

A system designed with proper coupling, cohesion, clarity, generalisation-specialisation depth and simplicity is easier to maintain.

I liked the explicit presentation of the experiment in this paper.

A Bug’s Eye View of Immediate Visual Feedback in Direct-Manipulation Programming Systems

Curtis Cook, Margaret Burnett, Derrick Boom

Live presentation of results helps with debugging some bug types but not all in a direct-manipulation visual programming environment.

Programming Complex Objects in Spreadsheets: An Empirical Study Comparing Textual Formula Entry with Direct Manipulation and Gestures

Herkimer J. Gottfried, Margaret M. Burnett

A graphical representation of new data type creation from a template was superior to a textual representation in speed, correctness and user comfort.

Using a Fine-Grained Comparative Evaluation Technique to Understand and Design Software Visualization Tools

Paul Mulholland

We should carefully evaluate why a tool works or doesn’t, and use this detailed information to inform the design of new tools. The new tools will be superior.

A Glimpse of Expert Programmers’ Mental Imagery

Marian Petre, Alan F. Blackwell

There are some commonalities in the way a few expert programmers describe their mental imagery of a program.

An Empirical Study of Novice Program Comprehension in the Imperative and Object-Oriented Styles

Vennila Ramalingam, Susan Wiedenbeck

Given comparable imperative and object-oriented programs, novices understand the imperative program better. However, this understanding is at the program level. They have a better understanding of the domain from the object-oriented program.

A Study of End-User Programming for Geographic Information Systems

Carol Traynor and Marian G. Williams

Programming by demonstration can work for GIS software.

Program Understanding Behavior During Debugging of Large-Scale Software

Anneliese von Mayrhauser, A. Marie Vans

A lack of either domain experience or familiarity with software will push down the level of abstraction for program comprehension. n=4

Visual Programming: The Outlook from Academia and Industry

K. N. Whitley and Alan F. Blackwell

At the time of publication, researchers were speculative and optimistic about visual programming. Professional programmers (non-users of VP) were sceptical of visual programming. Users of one VP environment were convinced of the benefits.

A Study of Program Entry Time Predictions for Application-Specific Visual and Textual Languages

Marian G. Williams, J. Nicholas Buehler

Programming time can be split into thinking time and program entry time. The authors created an accurate model of program entry time for both a graphical programming environment and a textual one. This model can be used to protect future research from the confounding effects of program entry time differences.

Notes on the Empirical Studies of Programmers, Fifth Workshop

April 26, 2009

Beyond Program Understanding: A Look at Programming Expertise in Industry

Lucy M. Berlin

In industry, programmers learn via apprenticeship.

I include the below because I feel it supports my belief that CS education needs to move as close as possible to an apprenticeship model.

The study proposes a view of programming expertise that ascribes their effectiveness not only to their knowledge of generic programming concepts and strategies, but to detailed knowledge of how these are expressed in their programming languages; the principles, design rationale, history and the quirks of the software framework they work within; proficiency in the use of broad range of ancillary tools; and knowledge of and appropriate use of human and written information resources.

This more encompassing view of programming expertise may help explain the great individual differences in programming productivity. It suggests that use of expert mentors may substantially reduce the time and effort required to retool oneself from one environment to another, and that the experts’ mentorship role is an extremely important one in an organization.

Externalising information during coding activities: Effects of expertise, environment and task

Simon P. Davies

When the display of code is restricted, experts write down more low-level details such as code snippets, whereas novices write down more high-level details such as architecture diagrams or control-flow diagrams. Novices rely on their diagrams to perform transformations in the problem space. Experts rely on their notes to mentally simulate code.

Tales of Debugging from The Front Lines

Marc Eisenstadt

An informal survey of debugging anecdotes revealed:

  1. Top reasons why bugs are difficult to find
    1. Large temporal or spatial gap between bug cause and bug symptom
    2. Breaking debugging tools
    3. Programmer has faulty perception, e.g. misleading syntax
    4. Programmer has broken semantic model
    5. Excessively ugly code
  2. Top methods for finding them
    1. Data-gathering
      1. Inserting print statements or other debugging code
      2. Step through in a debugger
      3. Comparing good and bad core dumps
      4. Debugger breakpoints
      5. Specialist tools
    2. Hand simulation of code or other thinking procedures
    3. Recognition of bug symptoms from memory
  3. Top root causes
    1. Memory overwrites
    2. Vendor hardware/software faults
    3. Incorrect logic
    4. Incorrect initializations
    5. Syntax problem

Learning Computer Programming: A Route to General Reasoning Skills?

Adrienne Y. Lee, Nancy Pennington

Advanced programming skills don’t transfer to other domains. Debugging skills transfer as general diagnostic skills.

Comparing the Comprehensibility of Textual and Graphical Programs: The Case of Petri Nets

Thomas G. Moher, David C. Mak, Brad Blumenthal, Laura M. Leventhal

This reproduction of an earlier study agreed that graphical program representations are no better and sometimes worse than textual.

A juicy morsel:

Not only is no single representation best for all kinds of programs, no single representation is even best for all tasks involving the same program.

Positive Test Bias in Software Testing by Professionals: What’s Right and What’s Wrong

Positive test bias is confirmation bias applied to computer programming: It’s the creation of too many tests that the program operates correctly on correct input and insufficient tests that it operates properly on incorrect input.

Better specifications and a lack of feedback from errors mitigate positive test bias.

Incidentally, I happen to believe that security bugs are instances of positive test bias.

On Claims and Citations

April 26, 2009

Help Needed: Where can I download a machine-readable database of publications in computer science or especially empirical software engineering? Or is this a job for Amazon Mechanical Turk?

At a small gathering of junior computer scientists today we had an involved discussion about the reputability of claims in scientific publishing.

A few key agreements from my perspective:

  • Sometimes a lemma is more interesting than the main theorem in a publication. Then future citations cite the paper for the lemma though not for its main result. More generally, sometimes we cite tangential results, not the main result of papers.
  • Sometimes a paper’s research does not truly support its claims. Future citations of the paper lose this information.
  • Sometimes a paper’s research does not support its claims as strongly or as generally as citations of it seem to imply.
  • Sometimes researchers will repeat a citation of a paper without reading the paper. Thus imperfect paraphrasings propagate unchecked.
  • Sometimes good research is built on foundations of less rigorous research.

It would be nice to build a database of publications with metadata for supported claims. It should include the strength and method of the support of the claim. Then we compare citations of that paper against the claims the paper actually supports.

This method would apply even in mathematical fields where the purity of knowledge is higher. For example, we could imagine a proof P that relies on theorem T. A paper presenting P would cite a paper proving T. If P no longer holds if T is false, then it is interesting to know how thoroughly the proof of T has been checked. Has it been glanced over casually by two overworked reviewers, accepted on trust and reputation? Has it been checked formally by computer? Our database would include this kind of information in its entry for the paper proving T.

We should also consider the importance of a given citation to the citing paper.

Some citations are critical. In this situation a paper would lose its force if the cited paper were incorrect.

Some citations are informational. In this situation a paper is cited because it is considered interesting and relevant. But a later debunking would not invalidate the citing paper.

Some citations are methodological. In this situation a paper is cited because its method has been copied or tweaked. This leads us to another important consideration I will not discuss today: The (lack of) standardization of methodology in empirical software engineering.

An near-term achievable goal would be to gather a large number of machine-readable papers, build a dependency graph of citations, and compare the citing sentences. It is common practice to tag a citing sentence with a reference[1].  These sentences we could extract programmatically. Thus we could gather a large number of citing sentences and compare them to each other and to the actual claims in the paper. I suspect we would not like the smell of what we found.

Help Needed (repeat): Where can I download a machine-readable database of publications in computer science or especially empirical software engineering? Or is this a job for Amazon Mechanical Turk?

[1] Not a real citation.

Notes on the Empirical Studies of Programmers, Fourth Workshop

April 26, 2009

The preface to the proceedings of the fourth workshop explained why I couldn’t find the third workshop: It was held jointly with CHI ’89 and no bound proceedings were published. There wasn’t even a true call for papers.

A Feasibility Study of Early Class Hierarchy Construction in Object-Oriented Development

Joseph L. Dvorak and Thomas G. Moher

It is difficult to design class hierarchies given just requirements if domain experience is lacking.

A semantic net is a (conceptually) better model for classes than an inheritance hierarchy. Semantic net links would be relationships such as inclusion, deletion, similarity and causality.

An Investigation of Procedure and Variable Names as Beacons During Program Comprehension

Edward M. Gellenbeck and Curtis R. Cook

Function and variable names are beacons, but strong code beacons such as the swap in a sort overwhelm the power of names.

Beacons as a theory do not rest on a solid foundation: Most beacon research has centred on the swap in the sort.

How Do We Read Algorithms? A Case Study

Crosby, M.E., Stelovsky, J.

According to an eye-tracking experiment, experts’ code-reading strategies differ from novices’. Both experts and novices use a variety of strategies.

Does Signaling Help Professional Programmers Read and Understand Computer Programs?

Edward M. Gillenbeck, Curtis R. Cook

Typographic signaling about important program sections, header comments before modules and mnemonic module names boost program comprehensibility.

Use of Structure Editing Tools by Novice Programmers

Dennis R. Goldenson, Bing Jyun Wang

GENIE environments disallow syntax errors by construction. They provide a rich editing environment for programs that supports direct manipulation of program constructs. If taught properly, even novices are able to take advantage of their power.

Comprehensibility of Visual and Textual Programs: A Test of Superlativism Against the ‘Match-Mismatch’ Conjecture

T. R. G. Green, M. Petre, R. K. E. Bellamy

Visual programs take longer to understand than textual. Programmer’s comprehension is mostly about training. Visual programming languages only have advantages when the structures match the tasks and the users know it.

How Experts Infer Novice Programmer Expertise: A Protocol Analysis of LISP Code Evaluation

Thomas Mastaglio, John Rieman

Experts use a limited set of cues to determine the expertise of another programmer.

Transfer of Training Between Programming Subtasks: Is Knowledge Really Use Specific?

Nancy Pennington, Robert Nicolich

There is substantial transfer between programming subskills, such as writing and comprehending computer programs.

What Do Expert Commenters Communicate By Means of Descriptive Commenting?

R. Douglas Riecken

Expert programmers mostly try to communicate semantic domain knowledge in comments. They also add spacing to make code more clear.

Novices actually need help with complex syntactic constructs before they can absorb the semantic knowledge.

Psychological Design Rationale for an Intelligent Tutoring System for Smalltalk

Mark K. Singley, John M. Carroll, Sherman R. Alpert

Great progress has been made in the psychology of programming, but this research has not affected the design of tools much. A tool is presented with a design informed by empirical results.

An Empirical Study of the Effects of Design/Documentation Formats on Expert System Modifiability

Kathleen M. Swigger, Robert P. Brazile

In a large expert system, a Petri net for control flow was superior to an ER diagram for comprehensibility.

Strategy Selection and Change in PASCAL Programming

Quanfeng Wu, John. R. Anderson

Programmers are tenacious in using the most appropriate looping strategy for problems. Programming languages ought to support cognitively natural looping constructs for loops where termination conditions are naturally found at the beginning, in the middle, or at the end of an iteration body.

Notes on the Empirical Studies of Programmers, Sixth Workshop

April 25, 2009

Techniques for Exploring Program Comprehension

Deborah A. Boehm-Davis, Jean E. Fox, Brian H. Philips

Program comprehension is driven by top-down and bottom-up processes to construct a goal hierarchy for a program. Programmers rely on more than just the goal hierarchy, however.

Evaluating the Effect of Inheritance on the Maintainability of Object-Oriented Software

John Daly, Andrew Brooks, James Miller, Marc Roper, Murray Wood

3 levels of inheritance helped maintainability vs. a non-inheritance control. At five levels of inheritance, the advantage is lost and may even be reversed.

Conceptual Entropy and its Effect on Class Hierarchies

J. Dvorak

Conceptual entropy is manifested by increasing conceptual inconsistency as we travel down the hierarchy. That is, the deeper the level of the hierarchy, the greater the probability that a subclass will not consistently extend and/or specialize the concept of its superclass.

When, How and Why do Novice Programmers Reuse Code?

Christopher M. Hoadley, Marcia C. Linn, Lydia M. Mann, Michael J. Clancy

An abstract understanding of a function and a belief in the benefits of code reuse are major contributors to code reuse among novices.

Evaluating Program Representation in a Demonstrational Visual Shell

Francesmary Modugno, Albert T. Corbett, Brad A. Myers

A graphical programming-by-example system allows users to create programs with variables, loops and conditionals better than a comparable textual system.

System Structure and Design

Robert S. Rist

An object-oriented system designer must consider control flow, data flow, and class design. Design is a process of choosing a part of the system and pursuing links through one of these dimensions to generate a system structure.

Learning About the Algebraic Specification of Abstract Data Types

Judith Segal

Formal methods are hard to teach.

Adaptation of Programming Plans in Transfer Between Programming Languages: A Developmental Approach

Susan Wiedenbeck, Jean Scholtz

When learning a new programming language, programmers bring their old paradigm with them. It can take months to learn a new paradigm. Programmers are likely to hit a plateau and fail to reach the most advanced features of the new environment.

Cover Flow as History Visualization

April 25, 2009

Apple has made a big deal of “Cover Flow”, a way of visualization and manipulating a collection of images.

It started in iTunes as a way of visualizing album covers.

Cover Flow in iTunes

Cover Flow in iTunes

It’s pretty but not useful. I don’t mentally sort music by album and it’s a clumsy way to cue memories.

Then, they introduced Cover Flow as a way of browsing files in Finder.

Cover Flow in Mac OS X Finder

Cover Flow in Mac OS X Finder

Again, it’s nifty but not useful. Large images of folders and icons of file types don’t help much.

Now, they have Cover Flow for browsing my internet history in Safari, and it’s brilliant.

Cover Flow Safari 4 History

Cover Flow Safari 4 History

I remembered reading something about making my own bagels. The history search made it easy search the text of pages that I had visited. The large images made them easy to remember. This combination made it possible to find the bagel recipe I want to try. To top it all off, the whole thing is gorgeous. Safari 4’s history visualization with cover flow is phenomenal.

Vista.

April 24, 2009
  1. Download Microsoft Windows Vista Business, for free, from MSDNAA.
  2. Burn it to DVD+R. Learn that DVD-R is evil.
  3. Partition drive using Boot Camp. Boot Camp partitions live without a reboot! Partitioning fails due to a disk integrity error.
  4. Reboot computer using Mac OS X Leopard installation CD. Run Disk Utility. Repair disk.
  5. Reboot normally. Partition drive using Boot Camp. Leave 50GB for Vista.
  6. Reboot using Vista CD.  Answer a few setup questions.
  7. Vista installation zooms by!
  8. Enter a password for Vista.
  9. Wait for self-configuration to finish.
  10. Insert Mac OS X Leopard installation CD. Install all sorts of drivers and things.
  11. Reboot into Mac OS X. Retrieve important passwords from Keychain. Store them and some files to USB stick.
  12. Reboot into Windows. Connect to internet!
  13. Download 51 updates, mostly critical security patches. Reboot.
  14. Between 10 and step 13, confirm your intentions to Vista 3,042 times. Yes, I really want to do the thing I just asked you to do, Vista.
  15. Download Project, Visio, Visual Studio from MSDNAA. Copy .iso files to USB stick for later burning. Toys to play with and learn!
  16. Fiddle with various .iso mounting tools for an hour. Settle on one. Install free expensive Microsoft Software. Installations (seem to) go much faster from USB drive than from hard drive.
  17. Download and install Expression.
  18. Spend hour or more trying to get Dell 2405FPW monitor to work properly. Its maximum resolution is 1024×768 in Windows Vista even though in XP or Mac OS X or Ubuntu it works at 1920×1200. No driver for it.
  19. Download 24 new critical updates made necessary by new software installations.
  20. Look for toys to play with. Fail. Turn on Aero and play with Windows-Tab for fourteen minutes and thirty-two seconds. Discover and get mind blown by Windows Live Writer’s sheer awesome. Laugh yet be intrigued at IE8’s suggested sites. People who like Windows Live Search also like MySpace. People who visit Microsoft Update also visit Firefox Download Page.
  21. Bring laptop to office. Forget power cable.
  22. Watch in terror as battery meter drops precipitously. Scramble to find power settings. Find them. Turn on maximum power saver. Forty-five minutes of charge remaining at 3/4 full battery. Scramble to find option to turn off Bluetooth and WiFi. Fail. There is no easy way.
  23. Reboot into Mac OS X. Miss Windows Live Writer. Make note to learn Visual Studio, F#, C#, Project, Visio, Expression, Power Shell.
  24. Look forward to Julie Larson-Green saving the Windows UI.

Notes on the Empirical Studies of Programmers, First Workshop (and some papers cited therein)

April 24, 2009

Human Detection and Diagnosis of System Failures

J. Rasmussen

Symptomatic troubleshooting matches symptoms of a problem to memories of symptoms due to past problems.

Topographic troubleshooting narrows bug locations in the source code. Strategies include working backward from erroneous results, elimination of known good code and mentally simulating candidate bug locations.

Comprehension Differences in Debugging by Skilled and Novice Programmers

Experts debug better than novices because they form better hypotheses about bug possibilities.

Experts and novices perform qualitatively similar activities during debugging.

Troubleshooting by comprehension means mentally simulating the program and comparing actual results to desired.

Plans in Programming: Definition, Demonstration and Development

Robert S. Rist

Programs are plan trees designed to accomplish hierarchical goals.

Novices use plans less as difficulty goes up. Specific plans are learned through experience.

Towards a Theory of the Comprehension of Computer Programs

R. Brooks

Program comprehension is largely a search for beacons in the code confirming a hypotheses about program function.

Processes in Computer Program Comprehension

Susan Wiedenbeck

Experts recall beacons better than novices. Beacons are focal points for program comprehension by experts.

Cognitive Processes in Program Comprehension

Stanley Letovsky

Programmers form a mental model of a program by “combining information gathered through reading the code and documentation with knowledge from a knowledge base of programming expertise.”

During a program comprehension task, subjects make inquiries, that is, form conjectures and ask questions.

Questions are divided into why, how, what, whether and discrepancy questions.

Why questions ask about the purpose of some code or design. (Or, what is the goal?)

How questions ask how known goals are accomplished by the program.

What questions are about names. (What is foo?)

Whether questions match hypotheses against the program. Does the program behave in a certain way?

Discrepancy questions reflect inconsistencies in the mental model of the program.

Conjectures are hypothesized answers to the questions.

A mental model of a program includes the goals, functions and data structures of the program. It also includes how they relate to each other.

Programmers build the mental model both top-down and bottom-up.

Mental Models and Software Maintenance

David C. Littman, Jeannine Pinto, Stanley Letovsky and Elliot Soloway

There are two main strategies for learning about a program for a change task. A systematic strategy aims to study the whole program and understand it before making the change. An as-needed strategy aims to target the learning to the bare minimum needed to make a change.

On a program change task for a small program, experts were systematic and novices were not. Those who followed the systematic strategy were much more successful than those that didn’t.

The Effects of Program-Dependent and Program-Independent Deletions on Software Cloze Tests

Mark Thomas, Stuart Zweben

Cloze test is the fancy name for fill in the blanks. Cloze tests are an accepted measure in the psychology of reading.

If a blank can be completed regardless of the understanding of the problem in question, it is a program-independent term.

If a blank can only be completed with a good understanding of how the program works, it is a program-dependent-term.

Software engineering researchers creating cloze tests must keep this distinction in mind and control for it.

An Empirical Study of the Effects of Modularity on Program Modifiability

Timothy D. Korson, Vijay K. Vaishnavi

…a modular program is faster to modify than a non-modular, but otherwise equivalent version of the same program, when one of the following conditions hold:

  1. Modularity has been used to implement “information hiding” which localizes changes required by a modification.
  2. Existing modules in a program perform useful generic operations, some of which can be used in implementing a modification.
  3. A significant understanding of, and changes to, the existing code are required for performing a modification.

Without one of the above conditions, increased modularity doesn’t help.

This study had an experimental design which ruthlessly controlled for programmer ability.

Experiments on Slicing-based Debugging Aids

A program slice for a given value is the subset of the larger program that affects the value.

Automatic program slicing does not help in debugging, surprisingly.

Program dicing is program slicing, but with code that contributes to known correct values also removed.

Automatic program dicing does help in debugging.

Analyzing the High Frequency Bugs in Novice Programs

A few types of bugs account for most of the errors new programmers make; however, they rarely reflect misconceptions about programming language constructs.

Ideas Generated

Do a variety of cloze tests: Delete all variables. Delete all keywords. Delete all expressions. And so on.

Construct a modern program dicing tool that integrates with a modern unit test framework.