Views on Internal and External Validity in Empirical Software Engineering

Views on Internal and External Validity in Empirical Software Engineering



Empirical methods have grown common in software engineering, but there is no consensus on how to carry it out. Is practical relevance key? Do internally valid studies have any value? Should we replicate more, which is common in other sciences to address the trade off between internal and external validity? We asked the community how empirical research should take place in software engineering, complemented with a literature survey about the status of empirical research.

Literature Review

We analyzed the recent years of ESEC/FSE, ICSE, and EMSE regarding five objectives:
  1. Was an empirical method applied?
  2. Were the experimental subjects human or non-human?
  3. Were the human experimental subjects professionals or students?
  4. Was an internal or external replication reported?
  5. How are threats to validity described?
Overview of results:

Download csv file with detailed results

Questionnaire

We designed numerous questions to assess participants' opinion. All questions were optional.

Download questionnaire (pdf)

Responses

Download responses and categorization (csv)
Question 23: Factors for balancing internal vs. external validity (csv)

Contact

Responsible for this project:
Janet Siegmund (siegmunj@fim.uni-passau.de)
Norbert Siegmund (siegmunn@fim.uni-passau.de)
Sven Apel (apel@uni-passau.de)