Reformulated, Well-Formed Research Questions

July 3, 2009

According to the Case Study Research book, my research questions should be “well-formed.” This means they are of the form “Who, What, When, Where, Why, How, How Much.”

I have translated my research questions into this form below.

It is clear that many of my questions are not best answered with a case study. It is also clear that some of my questions can only be answered in a case study.

I think someone could spend a whole career finding quantitative answers to quantitative questions below, but I’m not sure it would be worth it. These questions ask only for credible answers, not conclusive answers.

  • What are the differences in maintenance between DSL programs and GPL programs?
    • How much maintenance effort is spent on DSL programs?
      • How much effort is spent on debugging DSL programs?
        • Why does the amount of debugging effort for DSL programs differ from the amount of debugging effort for similar-sized GPL programs?
        • What are the differences in the nature of bugs between DSL programs and GPL programs?
          • How do programmers form mental models of DSL program execution?
          • What is the accuracy of these mental models?
            • How much difficulty do programmers have forming cognitive models of alternative execution models? Why?
            • How are programmers trained in specific DSLs?
          • What are the differences in difficulty caused by tool differences?
          • How much do GPL tools help with DSL maintenance?
          • What are the differences in difficulty caused by differences in size between DSL programs and GPL programs?
            • How do we compare the size of a DSL program to the size of a GPL program?
            • Why should or shouldn’t we compare by functionality?
            • Why should or shouldn’t we compare by effort invested in creation?
            • Why should or shouldn’t we compare by physical size, i.e. bytes/tokens/lines?
        • How do programmers debug DSL programs?
        • How much do programmers follow a known model of GPL debugging, e.g. Hale and Haworth 1991?
        • How much do programmers use known GPL debugging tactics and strategies on DSL programs?
    • How much effort is spent on change of DSL programs to meet new requirements?

3 Responses to “Reformulated, Well-Formed Research Questions”

  1. Neil Says:

    Hi Aran, are you factoring in the different contexts of use for DSLs and GPLs? I’m thinking that part of the maintenance advantage might be that DSLs are applied to more specific areas, that are less cognitively demanding, less prone to change or scope creep, etc. Also, do you think that you can easily compare these things across organizations? My feeling is you would have to compare within organizations to account for that variable.

  2. aran Says:

    @Neil: I’ve been thinking about ways to deal with exactly that. I suspect that DSLs are really only used in areas where the requirements are particularly clean.

  3. Ian Says:

    I had also been wondering if you need to qualify the developer’s experience level (or maybe you only intend to interview experienced developers).

    After reading the questions above, a number of other questions occurred to me when I was skimming a couple of DSL articles/blog entries I came across this afternoon. In the developer’s opinion could a GPL have been used with appropriate functions? Is the DSL intermixed with GPL code (kind of like embedded SQL)? If so, were do most problems occur – in the DSL code, GPL code or the interface? Are they using a generally available product/implementation of the/a DSL, or is it specific to their company? If so, do they also maintain the DSL, what is it written in, and do they find more bugs in the implementation or the code written to use it? Do they use multiple DSLs?

    And I know … you’re *just* writing a Master’s Thesis … not a PhD book – you don’t really want more questions :).

Comments are closed.

%d bloggers like this: