The State of ESE

May 10, 2009

Let me get this straight:

  1. We are not able to systematically get requirements right.
  2. We are not able to systematically get estimation right.
  3. We have completely standardized on an diagramming notation that doesn’t address the real needs of diagram users. Accordingly no one uses it.
  4. A large proportion of software engineering research is poured into the general area of formal methods. Yet, the ROI on formal methods is not known to be positive in general or in any specific case.
  5. Debugging is the greatest time sink in any programming activity and yet nearly no curricula contain a systematic debugging course.
  6. We have no agreed notion of the populations of
    1. software engineers,
    2. software engineering students,
    3. software engineering problems,
    4. or of software engineering environments. Therefore we cannot draw representative samples of any of these.
  7. We have no standardized (valid, reliable) constructs for measuring software engineer skill on any dimension whatsoever.
  8. The greatest general productivity improvements in the real world have been attributed to successively higher-level programming languages, yet programming language theory had nearly no influence on the most important programming language of all time (C). Modern programming language theory has had next-to-no influence on the most influential programming languages of this generation (Ruby, Python, PHP).
  9. In sum, we have no knowledge whatsoever of transferability of software engineering research into practice.

And yet, despite these failings, software “out there” has managed to take over the world. Without “us.”



9 Responses to “The State of ESE”

  1. aran Says:

    Bonus for reading the comments: Motivation is well known to be the single most important driver of learning and expertise, and yet the vast majority of post-secondary educators receive no training in motivation and make no systematic effort to motivate their students.

  2. Greg Wilson Says:

    All true — now, what are you going to do about it? In particular, can you develop training to address #5 that draws on pedagogical research in teaching medical diagnosis, forensics, the scientific method, and/or literary criticism, and demonstrate that it has an impact on productivity? (And can you substantiate the claim in #8 that PP&R are the “most influential languages” of our generation?)

  3. Neil Says:

    I’m waiting on part two, “Why I’m in Grad School studying software engineering”

  4. aran Says:

    @Greg: TIOBE backs me up on #8. As for #5 a case study might be able to demonstrate value in such a course, but I don’t think an experiment is plausible.

    @Neil: Not quite sure what you mean.

  5. Ian Says:

    Sounds like you are at stage 2 in the 5 stages of a project:

    1. Euphoria and Excitement
    2. Disenchantment
    3. Search for the Guilty
    4. Punishment of the Innocent
    5. Reward for the Uninvolved

    On the bright side, it sounds like there is lots of choice – the field is wide open 🙂

  6. Jorge Says:

    Yes, it’s all true. And just wait until ICSE to see the priorities of our research community.

    Software development research is still at a pre-scientific stage; yet we have thousands of researchers pretending they are the ones saving the industry from total collapse.

  7. Neil Says:

    I guess as opposed to being in industry or a start-up where the innovation seems to happen, viz. Facebook, Twitter, Rails, Scrum, Kanban, etc. etc.

    Although wasn’t Smalltalk somewhat academically created?

  8. George Says:

    Time to switch to machine learning? 😛

  9. Excellent analysis. Now you know what everyone else is doing wrong, your way is clear for doing SE research that *will* make an impact.

    BTW this is *research*. Most of it will fail. If success were guaranteed, then it’s not research. What matters is the knowledge gained in the process. So whatever you choose to focus on, make sure it will have good learning outcomes, whatever the fate of the technology that you invent. Oh, and technology diffusion studies suggest we should expect about a 15 year delay from innovation to adoption. So don’t expect to see SE research having any impact on practice in the near term.

    Having said all that, it is correct to conclude that there’s lots of crap research going on in SE. Rise above it.

Comments are closed.

%d bloggers like this: