Captain Obvious and the Thesis Hunt

May 7, 2009

New experimental blog post format today. Lots of little items.

I skimmed about half the ICSE influential papers. Topics: Software architectures, modularity, distributed computing, software processes, requirements, assertions in code, program slicing.

There were more than fifty papers published in ICSE’08. A small fraction were ESE. As I browse the archives, I see no common themes. It seems to me that ICSE’s scope is basically anything interesting in software engineering. And here I was thinking it was an Empirical Software Engineering conference. Whoops!

Barry Boehm has A View of 20th and 21st Century Software Engineering: A high level overview of trends in SE.

Sudden realization: Jorge’s gold standard thesis had an obvious result. It’s a nice paper because of the crisp demonstration of that obvious result. I need to turn off my “I don’t want to do it because it’s obvious” filter.

Some researchers at the University of Calgary outline the place of ESE in ICSE: On the Success of Empirical Studies in the International Conference on Software Engineering.

DSLs suck as a research target for at least four reasons: It is impossible to define a representative sample of DSLs, it is impossible to generate an “equivalent” non-DSL program, DSL creation requires expertise that isn’t commonly taught, DSL usage is subject to extreme practice effects.

No matter what DSLs are chosen, a critic could argue that they are bad examples and not representative of the results of Language-Oriented Programming in general.

Some DSLs don’t have clear non-DSL equivalents. What would be the equivalent non-DSL representation for an HTML + CSS page? For DSLs for which we can specify equivalent behaviour in a general programming language, if we find the DSL superior, a critic could argue that our “equivalent” program was just not a good program. If we fail to find a superiority, a critic could argue that our “equivalent” program was in fact a DSL.

Even DSL supporters admit that it isn’t easy to make new DSLs. One requires a good grounding in language implementation (rare) and it helps to have a helping of good taste (rarer). The trouble is that every practioner has extensive training on LOP’s main competitor, that is, proper library design using language-provided abstractions as designed.

Greg’s idea of demonstrating the inferiority of DSLs in debugging tasks is a good one but the factors above mean the experimental design will require some ingenuity. If you’re an ingenious experiment designer, call me.

A Loose Upper Bound on Intra-Programmer Skill Variability.

Counterbalanced one-factor within-subjects study. Experiments were separated by two weeks.

Condition one: Participants were encouraged to get a good night’s rest. One hour before the task, they were fed a light meal of  healthy juices, coffee and foods. They were presented with exercises and reading designed to enhance a sense of self-efficacy and boost self-confidence. They were led through a mild yoga routine designed to increase concentration and mental acuity.

Condition two: Participants were invited to a late-night sexy party the night before the task. One hour before the task they were fed a heavy meal of pizza, candy and decaffeinated high-fructose carbonated beverages, carefully timed to cause low blood sugar at the time of the task. In the hour leading up to the task they were presented with unrelated hard problems designed to mentally exhaust them and crush their puny sense of self-worth.

Task: Participants completed three archived ACM-ICPC programming problems at each session and were allowed to work on them until a white-box test suite completely passed. We timed the results.

Results: The mean performance ratio between conditions was 6:1. Null hypothesis of a 1:1 ratio rejected, (Χ t, p < 0.01)

Discussion: It was a really fun sexy party. Also we faked our Ethics Review Board approvals. Also the authors are like, so drunk right now.

Oh, it’s nice to dream.

Andreas Zeller and Microsoft Research applied principal components analysis to several code complexity measures to better predict defect locations. Cool.

Microsoft Research straightened out some myths on how developers spend their time. Maintaining mental models. (Juicy tidbit: No, IMs and email don’t help productivity by preventing face-to-face interactions.) I especially like their idea of IDE hyperlinks to design documents or whiteboard snapshots. Somebody has to have made an Eclipse plugin that does this already.

Zak informs me that the Scala Eclipse plugin has inferred type hovers. Go Scala. May you change the world of IDEs forever in ways I’ve predicted.


3 Responses to “Captain Obvious and the Thesis Hunt”

  1. George Says:

    Oh man I really think Scala is cool. Maybe I should learn it?

  2. Jorge Says:

    I WANT TO READ THE PARTY PAPER! And so would everybody else I think 🙂

  3. RH Says:

    With respect to the “Maintaining Mental Models: A Study of Developer Work Habits”, it might be interesting to look at how those work habits vary for software developers who work remotely or telecommute (writing as someone who has telecommuted for a number of years).

Comments are closed.

%d bloggers like this: