More Notes on the Case Study Research Method

June 21, 2009

After I posted my notes on Robert Yin’s “Case Study Research”, Greg replied with no fewer than ten comments. Once I got over my initial !@#$ reaction, I was glad because the questions made me think more carefully about the material.

What (in practice) is the difference between Exploratory, Descriptive or Explanatory purposes?

Exploratory: The goal is to develop propositions for further study.

Descriptive: The goal is to describe something unknown.

Explanatory: The goal is to make causal conclusions about why a decision or event happened as it did. There is an intention that the results generalize to other situations.

Thus the difference between the purposes is a difference in goals and research questions.

Do you have to have a theory before you can start a case study? Or is the purpose of an exploratory case study to (help) develop a theory?

In descriptive and explanatory studies, one starts with propositions. In an exploratory study, we may not start with propositions. Instead, we state the purpose and the success/exit criteria of the exploration.

How do you tell if you have the characteristics of a good case study investigator & are applying them?  Most people think they’re good listeners, but are wrong.

There is no test. An investigator must be honest with himself/herself and work to remedy any weaknesses.

You quote “Nor, in many instances, do case study reports end up in journals” 😦  Does the book explain why not?  And is that actually the case in empirical software engineering?

Why aren’t case study reports always published in journals? What happens to case studies in Empirical Software Engineering?

There is no commonly accepted outline format for a case study reports. Many case studies are published as books, or they are commissioned reports. They are much larger than typical experiments.

Any suggestions on the design of a case study database?  Templates you could clone?

A case study database is not like a computer database. Instead, it is simply the name for the idea of separating the raw data from the presentation of the data in the final report. Within reason, a reader ought to be able to go back and read the documents, interviews and other information that informed the report.

Do you need to have a theory to test? What do you do in the Exploratory case?

Exploratory case studies replace theory propositions with goals and criteria for success.

In your summary, what do you mean by a “whether” question?

This is me getting into trouble with badly-formed questions. I don’t trust case studies to tell me “whether” an intervention had a desired effect (well-formed: “What is the effect of the intervention?”).

In your summary, you say, “Case studies seem to be used in policy  to evaluate programs and decisions. I don’t think this is valid.”  Yin and others clearly think it *is* valid — why have they reached a different conclusion from you?

I think Yin would agree that “What”, “How much” and other questions are best addressed with other research methods. I’m not sure what Yin would say on the use of case studies for evaluation. I think case studies can provide great insight into why a program or decision works, but I would never trust a case study investigator who tells me a program is working because he or she interviewed some beneficiaries.

On matters of causality and bias, it is Yin’s opinion that, with sufficient rigour, case studies can be as reliable as other methods such as experiments, especially since experiments are also subject to bias. I just don’t agree. It would take a rather large experiment to determine whether decisions made informed by case studies were better than decisions made informed by quantitative methods.

In your summary, you say, “Several of my questions seem better addressed by surveys.”  Doesn’t a survey pre-suppose a strong theory?

Not necessarily. For example, surveys can ask “How Much” questions. I want to know “how much” maintenance is done on DSL programs. I can get at this by asking some companies about it in interviews, but an answer from a survey of many companies would be stronger evidence in my opinion.

Advertisements

3 Responses to “More Notes on the Case Study Research Method”

  1. Jorge Says:

    Good answers, just one comment:

    “It would take a rather large experiment to determine whether decisions made informed by case studies were better than decisions made informed by quantitative methods.”

    Or maybe it would take a rather large case study? With so many variables and confounding factors affecting the outcome, wouldn’t a case study be a better method to address this question?

  2. aran Says:

    @Jorge: Ah, yes, or perhaps a survey of many decision-makers? 🙂

  3. Wogari Says:

    Is there such an exploratory case study? Why not experimental study if so?

    I need clarity in this regard.


Comments are closed.

%d bloggers like this: