Archive for July, 2009

Peer Evaluation Dysfunctions

July 30, 2009

Every business is a system, and the performance of individuals is largely the result of the way the system operates… the system causes 80% of the problems in a business, and the system is management’s responsibility.

Ranking people for merit raises pits individual employees against each other and strongly discourages collaboration.

There is no greater demotivator than a reward system that is perceived to be unfair. It doesn’t matter whether the system is fair or not. If there is a perception of unfairness, then those who think that they have been treated unfairly will rapidly lose their motivation.

When we optimize a part of a chain, we invariably suboptimize overall performance.

—Mary Poppendieck, The Best Software Writing I, 2005.


Business Process Design and Software Design

July 30, 2009

In fact, I think that more attention needs to be drawn to the parallels between business process design and software architecture. I think that modern businesses are so complex that you basically have to have the skills of a software architect to get them right.

—Joel Spolsky, The Best Software Writing I, 2005.

Game: Make statements that are true of software architecture and business design.

My turn: “Modularization promotes correctness and maintainability through coherency and information-hiding, but sacrifices cross-cutting performance optimizations. ”

Your turn:

Inheritance Is Over-Taught (and taught too early)

July 21, 2009

Software Carpentry is gnashing through object-oriented programming.

It seems like the de facto way of teaching object-oriented programming involves diving into inheritance and polymorphism. These tend to take up a large proportion of the time because inheritance and polymorphism are hard.

I think the amount of time spent on these concepts is disproportionate to the amount they are used and encourages the overuse of these tools.

Composition, delegation and other relationships can all be modeled using references to other objects. Inheritance, however, relies on modeling a domain using only specialization relationships.

I think very few real domains actually are modeled really nicely using inheritance hierarchies.  I can only think of a few examples: Actors in simulations such as games, data structure libraries, and graphical user interfaces.

Where there is no way to model a domain using specialization, you’re actually lucky. In some cases, there is more than one way to model a domain using specialization. For example, you might be tempted to specialize objects that have physical properties. Some of them have behaviours related to mass, some related to energetic activity. Do you impose a partial ordering on the dimensions? Which partial ordering? Do you teach mixins? Multiple inheritance? Or do you try to teach why you should discard inheritance in this case?

Thankfully for the real world, the kind of domain modeled by inheritance tends to be the kind of domain where a few experts write a library that everyone then uses. This further reduces the argument for teaching inheritance.

Nearly all other relationship types degenerate into composition, delegation and reference.

Of the holy triumvirate of object-oriented principles (Inheritance, Polymorphism and Encapsulation) I think Encapsulation deserves the centre stage and the bulk of the attention, but it doesn’t get it. I like to ask “What information does this class protect from others?” It’s a much more productive question than devising inheritance relationships, which can be viewed as the task of sorting out a partial ordering of dimensions of classification of objects i.e. complicated.

Inheritance should be viewed simply as just one method of code reuse, and a particularly hard-to-use one at that. As such it is appropriate for programmers learning about refactoring and advanced code quality practices (i.e. advanced programmers only). At this time programmers will have a much stronger foundation for understanding this concept. Proper design of inheritance hierarchies requires thinking cleanly about TWO external interfaces for a class: The typical object and the interface to subclasses.

To make matters worse, we’ve only been discussing traditional inheritance. Prototype inheritance is another paradigm entirely. Many smart people think prototype inheritance is a better system overall, but the world is stuck with these 1970s concepts.

So to sum up this rant:

  • Inheritance is hard to use right in the real world.
  • Compared to other object relationship types, inheritance isn’t used that much in the real world.
  • Inheritance is just one of many ways of reusing code.
  • Inheritance is taught too early to novice programmers.
  • Inheritance consumes too much of the object-oriented education curriculum.

In my view, instead of the status quo we should:

  • Teach objects only in the context of information-hiding. Treat classes as simple, standalone templates for objects.
  • Teach delegation as the primary method of code reuse.
  • Omit discussions of inheritance until as late as possible.

Theory product

July 9, 2009

For each item on the left what does the theory on the right predict about DSLs as a consequence?

Things I can’t do with my iPhone*

July 8, 2009

Things I can’t do with my iPhone*

*that I should be able to and can be implemented in software.**

** that I may simply not have learned how to do yet.

  • Download a picture from picasaweb
  • Sync over bluetooth or wifi. I could do this with my several-years-old motorola phone and Apple’s own iSync. Two steps forward, one step back.
  • Sync saved Safari passwords, Airport networks, or VPN settings from Keychain.
  • Manage iCal to-do lists
  • Record calls
  • Get the app store to remember my password

Fido + iPhone Fail.

July 7, 2009

A while back I compared the big smartphones available in Canada. I decided I wanted the iPhone 3GS. Then I compared Fido to Rogers for my needs and found a Fido plan that I liked. This seemed convenient because I’m already a Fido customer, which I thought would make things easier.

On June 19 I went to the biggest Fido store in Toronto to buy one, but they were out of stock. I called around and all the other Fido stores were also out of stock. I thought this was silly, because they had been advertising the phone heavily and this was the third iPhone—they should have anticipated the demand better.

The next day I called Fido to order one over the phone. I waited 25 minutes on hold before my phone went out of service and the call dropped.

The next day I called Fido to order one over the phone. The woman on the phone told me it would probably be shipped to me in the next week or so, but in the worst case it would arrive July 7th. She wanted to activate my new plan right away, but we agreed that it would be silly for me to pay for data and visual voicemail and other features until I had a phone to use them with. So, I would receive my new phone, call Fido, and activate my new plan to give Fido lots more money.

About two weeks later I called to check on the order. I was assured the July 7th deadline was still good.

Today is July 7th. I called to find out what had happened.

The gentleman on the phone told me the phones were out of stock and backordered, but there was a waiting list.

I told him that I had already been promised a July 7th delivery, and asked him why I didn’t have my phone.

He told me he would check to see if it had been shipped yet. It hadn’t. They had no shipping tracking number.

I asked him when I could expect a new phone.

He looked into my file and told me that as a matter of policy they could not give me a new phone at agreement price and that the back office had blocked my order.


I asked him to clarify since I didn’t understand.

He said that as a result of my phone plan change in November which included a system access fee I could not order a new phone.

What just happened?

I asked him what happened on June 21 when I ordered my phone and it was to be delivered July 7th at the latest.

He said the woman I spoke with had made a mistake.

I asked him why I had been allowed to order a phone then. I also suggested to him that it was strange that the back office would block my order due to a policy I had never heard of without notifying me.

He left to check with his supervisor. When he came back, he said that my plan change in November made me ineligible for a new phone, even though I didn’t get a new phone at that time.

(In November, I changed plans. I was already a Fido customer, but on an old plan. I switched to a new, better plan and paid a $35 activation fee. I’ve always used my own unlocked phone and I’ve never signed a long-term contract.)

I told him that this didn’t make sense, because if I wasn’t already a customer, I could walk into a Fido store, give them money for a new phone, and sign up for an agreement. I asked him how we could solve this problem.

He said he couldn’t do anything about the policy.

I asked who I could talk to who could solve this problem and clarify what the hell had happened here.

A few minutes later, I had a supervisor named Oliver on the line.

We talked through the situation. Oliver made some exceptions that I don’t fully understand to work around the Fido system. As of now, I’m on a different (worse) plan, but, if I can find an iPhone in a store, I’ll be able to buy it without the system throwing up some error. I’m also on a waiting list if the central Fido gets some new stock.

I called the biggest Fido store in Toronto. They are out of stock. I called around and all the other Fido stores were also out of stock. I thought this was silly, because they had been advertising the phone heavily and this was the third iPhone—they should have anticipated the demand better.

Journal Publication and Subscription Fees

July 4, 2009

Greg shares

Journal publishers worry that university libraries will stop paying for journals. This would take away the funding that currently supports (a) profits and (b) a quality control process for papers via peer review.

The current unofficial system differs from the official system. In the official system, an interested reader must be affiliated with a paying institution to get access to papers, and thus institutions have an incentive to pay. In the unofficial system, everyone has access to published papers, and university libraries pay to maintain the status quo.

People generally agree that it would be nice to bring the official and unofficial systems together, but nobody knows exactly how the new system will work.

Reformulated, Well-Formed Research Questions

July 3, 2009

According to the Case Study Research book, my research questions should be “well-formed.” This means they are of the form “Who, What, When, Where, Why, How, How Much.”

I have translated my research questions into this form below.

It is clear that many of my questions are not best answered with a case study. It is also clear that some of my questions can only be answered in a case study.

I think someone could spend a whole career finding quantitative answers to quantitative questions below, but I’m not sure it would be worth it. These questions ask only for credible answers, not conclusive answers.

  • What are the differences in maintenance between DSL programs and GPL programs?
    • How much maintenance effort is spent on DSL programs?
      • How much effort is spent on debugging DSL programs?
        • Why does the amount of debugging effort for DSL programs differ from the amount of debugging effort for similar-sized GPL programs?
        • What are the differences in the nature of bugs between DSL programs and GPL programs?
          • How do programmers form mental models of DSL program execution?
          • What is the accuracy of these mental models?
            • How much difficulty do programmers have forming cognitive models of alternative execution models? Why?
            • How are programmers trained in specific DSLs?
          • What are the differences in difficulty caused by tool differences?
          • How much do GPL tools help with DSL maintenance?
          • What are the differences in difficulty caused by differences in size between DSL programs and GPL programs?
            • How do we compare the size of a DSL program to the size of a GPL program?
            • Why should or shouldn’t we compare by functionality?
            • Why should or shouldn’t we compare by effort invested in creation?
            • Why should or shouldn’t we compare by physical size, i.e. bytes/tokens/lines?
        • How do programmers debug DSL programs?
        • How much do programmers follow a known model of GPL debugging, e.g. Hale and Haworth 1991?
        • How much do programmers use known GPL debugging tactics and strategies on DSL programs?
    • How much effort is spent on change of DSL programs to meet new requirements?

Outline of a Literature Review

July 3, 2009
  1. DSLs are important
    1. Definitive ANTLR reference positioned as book teaching how to make DSLs
    2. Sprinkle et al. Guest Editor introduction in latest IEEE software on DSLs
    3. S. Dmitriev “Language Oriented Programming: The Next Programming Paradigm”
  2. DSLs
    1. What is a DSL
      1. Definitions (Variety of sources)
      2. Bentley 1986 Little Languages
    2. How DSLs are implemented
      1. Supporting the DSL Spectrum (review of DSL approaches)
      2. When and how to develop domain-specific languages
      3. Notable design patterns for DSLs
    3. DSL benefits
      1. Supporting the DSL Spectrum (claimed 50:1 improvement in program length)
    4. Why DSLs work
      1. SWYN: Notation has a large effect on performance. Mental models of evaluators.
      2. When and how to develop domain-specific languages
      3. Prechelt 2000/2003: Scripting langs. vs. stodgier langs empirical comparison
    5. Misc. Surveys of other DSL literature
      1. Domain-specific languages in software development
      2. Domain-specific languages: An annotated bibliography
      3. Landin 1966: The next 700 programming languages
      4. Beyond LEX and YACC: Introduction to a number of DSLs to simplify compiler construction
      5. Fowler 2005 on language workbenches
  3. Debugging
    1. Maintenance is expensive and important
      1. Measuring and managing software maintenance
    2. There is a variety of data on debugging strategies and tactics and tools to support these
      1. Weiser 1982: Programmers use slices when debugging (just a tactic)
      2. Gugerty and Olson 1986 Debugging by Skilled and Novice Programmers (not much theory)
      3. Eisenstadt 1993: Tales of debugging from the front lines
      4. Murphy 2008 Debugging–a qualitative analysis of novices’ strategies
      5. Ko 2005, Ko 2006: Detailed study of corrective and perfective maintenance tasks, how it informs Whyline/IDE design
      6. Ko 2006: Debugging by asking q. about program output (Introduction of the Whyline)
      7. Liu 2006: Statistical Debugging: A hypothesis testing based approach
      8. Ruthruff 2005: Empirical study of fault localization for end-user programmers
      9. Ducasse and Emde 1988 A review of automated debugging systems (some debugging strategies and types of knowledge)
    3. There are a variety of models of bugs and debugging
      1. Vessey 1989 Toward a theory of computer program bugs (debugging is about program comprehension)
      2. Hale 1991 Towards a model of programmers’ cognitive processes during software maintenance, Hale 1999 evaluation of the model (Structural Learning Theory)
      3. Eisenstadt 1993: Tales of debugging from the front lines
      4. Von Mayrhauser/Vans 1997/1999: Program understanding behavior during maintenance (Integrated Comprehension Model)
      5. Xu 2004: Cognitive processes during program debugging (Bloom’s Taxonomy in the Cognitive Domain)
      6. Ramanujan 2000: An experimental investigation of the impact of individual, program, and organizational characteristics on software maintenance effort (Human Information Processing)
    4. Misc survey
      1. Debugging: A review of the literature (McCauley 2008)
  4. DSLs + Debugging: Nearly green-field
    1. There is a need for DSL debuggers
      1. Weaving a debugging aspect into DSL etc.
      2. KHEPERA
      3. Editorial Review of above: Cost of maintenance + proliferation of DSL usage = DSL maintenance
    2. Survey of literature looking at DSL maintenance
      1. Little Languages: Little maintenance?
      2. DSL maintenance vs. UML maintenance in IEEE software
      3. A software engineering experiment in software component generation: Experiment on DSL vs. GPL tasks
    3. Editorial Review: Most published research on debugging in general does not apply to DSLs in general. Not much literature on DSL maintenance
    4. Need for theory-based approach
      1. Hannay 2007 A Systematic Review of Theory Use in Software Engineering Experiments
      2. Ko 2005: Framework and methodology for studying the causes of errors in programming systems (theory of bug introduction)

This justifies and leads into my thesis, which, in the current plan:

  • Extends Ko’s 2005 theory to make predictions about DSL bugs (by incorporating knowledge of what DSLs are, when they are used and why they work)
  • Applies the various theories of debugging to DSLs or shows why they are difficult to apply
  • Claims that there is virtually no existing research on costs of DSL maintenance
  • Claims that there is very little understanding of DSL maintenance issues
  • Predicts that DSL maintenance is an important consideration
  • Examines the above predictions via interviews of DSL-using practitioners in Toronto.

Notes on “An experimental investigation of the impact of individual, program, and organizational characteristics on software maintenance effort”

July 2, 2009

An experimental investigation of the impact of individual, program, and organizational characteristics on software maintenance effort: Ramanujan, S. and Scamell, R.W. and Shah, J.R.

Maintenance effort is expensive.

The Human Information Processing model is a model of how cognition works. Information travels back and forth from our extremely short-term sensory buffers, to short-term memory, to buffer memory, to long-term memory. Short-term memory has a small capacity. Information moves from short-term to long-term memory via rehearsal. Long-term memory is subdivided into episodic and non-episodic memory. Episodic memory includes time and place information (what we think of as “memories”). Non-episodic memory is what we would typically call “knowledge.” We subdivide non-episodic memory into semantic and syntactic knowledge. Semantic knowledge is abstract knowledge independent of context. Syntactic knowledge is specific, contextual, rote-memorization knowledge. The buffer is essentially the focus of our attention, including the active subset of long-term memory.

Maintenance tasks are subdivided into program comprehension, program modification and program composition.

Program comprehension -> Creation in the buffer of a semantic understanding of the program using syntactic and semantic knowledge from LTM.

Program modification -> Manipulations on the buffer structure. Correction of the buffer structure based on discrepancies from output.

Program composition -> Problem analyzed in buffer into ‘given state’ and ‘desired state.’ To achieve desired state, knowledge is transferred from LTM into buffer. Problem solution conceived as plan for program. Stepwise refinement on this plan.

43 predictions from HIP supported in Ramanujan and Cooper 1994.

There is little agreement in the literature about how to operationalize software maintenance effort.

Some options: Number of repair requests, number of repairs per production run. Assumption: All repairs of equal magnitude. Total cost of maintenance in terms of labor hours. Assumption: Constant quality of labor.

Proposal: Labor hours weighted by quality of labor. (Unclear how to operationalize quality of labor)

For this study: Time required to successfully implement changes.

The study tested seven hypotheses and found support for most. Conclusions include: Keep control-flow complexity low, name variables well, don’t use novices for maintenance on large software.

Constructs in the hypotheses include: Program control-flow complexity, program size, time to maintain, variable-name mnemonicity, programmer semantic knowledge, and time pressure.

For my purposes, control-flow complexity and variable-name mnemonicity may not be relevant for many DSLs.