Empirical Software Engineering Issues. Critical Assessment and Future Directions

Victor R. Basili, Dieter Rombach, and Kurt Schneider Introduction In 1992, a Dagstuhl seminar was held on Experimental Software Engineering Issues (seminar no. 9238). Its goal was to discuss the state of the art of empirical software engineering (ESE) by assessing past accomplishments, raising open questions, and proposing a future research agenda. Since 1992, the topic of ESE has been adopted more widely by academia as an interesting and promising research topic, and in industrial practice as a necessary infrastructure technology for goal-oriented, sustained process improvement. At the same time, the spectrum of methods applied in ESE has broadened. For example, in 1992, the empirical methods applied in software engineering were basically restricted to quantitative studies (mostly controlled experiments), whereas since then, a range of qualitative methods have been introduced, from observational to ethnographical studies. Thus, the field can be said to have moved from experimental to empirical software engineering. We believe that it is now time to again bring together practitioners and researchers to identify both the progress made since 1992 and the most important challenges for the next five to ten years.

Autorentext
Dr. Kurt Schneider leitet das Fachgebiet Software Engineering an der Leibniz Universität Hannover. Er hat in Erlangen Informatik studiert und anschließend an der Universität Stuttgart promoviert. Bei einem Forschungsaufenthalt an der University of Colorado at Boulder beschäftigte er sich mit Techniken zum systematischen Lernen aus Erfahrung im Software Engineering. Er war sieben Jahre bei der DaimlerChrysler AG am Forschungszentrum Ulm tätig. In Projekten mit verschiedenen Unternehmensbereichen spielten Softwarequalität, Prozessgestaltung und wiederum die Erfahrungsnutzung eine wichtige Rolle. Zu seinen Forschungsthemen gehören daneben Softwareanforderungen und agile Methoden, um Informationsflüsse und Dokumentationen in Softwareprojekten zu optimieren. Kurt Schneider legt viel Wert darauf, diese Themen praxisnah zu bearbeiten und vermitteln. Prof. Dr. Dieter Rombach - und Prof. Dr. Peter Liggesmeyer - sind Lehrstuhlinhaber für Software Engineering an der TU Kaiserslautern. Sie leiten gemeinsam das Fraunhofer Institut für Experimentelles Software Engineering.

Inhalt
Session 1 The Empirical Paradigm.- The Empirical Paradigm Introduction .- Approaches for Empirical Validation.- Techniques for Empirical Validation.- Status of Empirical Research in Software Engineering.- Position Papers.- Aggregation of Empirical Evidence.- Empirical Evaluation in Software Engineering: Role, Strategy, and Limitations.- New Opportunities for Empirical Research.- Empirical Paradigm: Position Paper.- The Value of Empirical Evidence for Practitioners and Researchers.- Exploration Versus Confirmation.- Empirical Paradigm The Role of Experiments.- The Role of Controlled Experiments in Software Engineering Research.- Position Papers.- Creating Real Value in Software Engineering Experiments.- From Silver Bullets to Philosophers' Stones: Who Wants to Be Just an Empiricist?.- Social and Human Aspects of Software Engineering.- Longitudinal Studies in Evidence-Based Software Engineering.- The Use of Grounded Theory in Empirical Software Engineering.- Historical Review.- Exploration and Confirmation: An Historical Perspective.- Working Group Results.- Combining Study Designs and Techniques Working Group Results.- Optimizing Return-On-Investment (ROI) for Empirical Software Engineering Studies Working Group Results.- The Role of Controlled Experiments Working Group Results.- Discussion and Summary.- The Empirical Paradigm Discussion and Summary.- Session 2 Measurement and Model Building.- Measurement and Model Building Introduction.- Data Sharing.- Data Collection, Analysis, and Sharing Strategies for Enabling Software Measurement and Model Building.- Knowledge Acquisition in Software Engineering Requires Sharing of Data and Artifacts.- Effective Data Interpretation.- Effective Data Interpretation.- Software Support Tools and Experimental Work.- PositionPapers.- Measurement and Interpretation of Productivity and Functional Correctness.- Synthesising Research Results.- On the Quality of Data.- Working Group Results.- Potential of Open Source Systems as Project Repositories for Empirical Studies Working Group Results.- Data Sharing Enabling Technologies Working Group Results.- Documenting Theories Working Group Results.- Discussion and Summary.- Measurement and Model Building Discussion and Summary.- Session 3 Technology Transfer and Education.- Technology Transfer and Education Introduction.- Technology Transfer.- Empirical Studies as a Basis for Technology Transfer.- Position Papers.- Relationships and Responsibilities of Software Experimentation.- The (Practical) Importance of SE Experiments.- How to Improve the Use of Controlled Experiments as a Means for Early Technology Transfer.- Extending Empirical Studies to Cover More Realistic Industrial Development and Project Management Issues.- Empirical Case Studies in Industry: Some Thoughts.- Software Process Simulation Frameworks in Support of Packaging and Transferring Empirical Evidence.- Structuring Families of Industrial Case Studies.- Education.- Empirical Software Engineering: Teaching Methods and Conducting Studies.- Educational Objectives for Empirical Methods.- Position Papers.- On Landscaping and Influence of Empirical Studies.- Involving Industry Professionals in Empirical Studies with Students.- Working Group Results.- Industry-Research Collaboration Working Group Results .- Teaching Empirical Methods to Undergraduate Students Working Group Results.- Discussion and Summary.- Technology Transfer and Education Discussion and Summary.- Roadmapping.- Empirical Software Engineering Research Roadmap Introduction.- Working Group Results.- Roadmapping WorkingGroup 1 Results.- Roadmapping Working Group 2 Results.- Roadmapping Working Group 3 Results.- Roadmapping Working Group 4 Results.- Discussion and Summary.- Empirical Software Engineering Research Roadmap Discussion and Summary.