Designing User Studies in Informatics

Computer software is becoming increasingly pervasive in all aspects of everyday life and frequently plays a decisive factor in people's decisions. User studies of such software and its components will provide developers with essential data for improving designs and products while allowing causal conclusion to be drawn about effectiveness and usefulness.

Designing User Studies in Informatics is intended as a practical guide for academics and business professionals wanting to evaluate user interactions with software. It provides a succinct overview of the theoretical background and includes the user study designs most suited for evaluating software and its components combined with a practical, hands-on approach to executing them. While applicable to all domains, the book concentrates on examples taken primarily from medicine and healthcare.

"This is an excellent, practical handbook that is unique in its highlighting of critical study design flaws, such as comparisons of new software with demo software or default values, and the remedies against them." Carol Friedman, Professor, Department of Biomedical Informatics, Columbia University

"From variable selection through the process of identification of the appropriate type of experimental design for a particular user study, Dr Leroy's book aids the informatician in the scientific process. This book fill an important niche in reducing time to discovery and should be required reading by anyone conducting user studies in informatics." Kristin M. Tolle, Ph.D., Director of Natural User Interfaces, Microsoft Research Connections

"I've wished for a book like this since I started grad school. So many resources either belabor 'why', neglecting 'how-to', or focus on minute details, losing sight of the overall task. The real studies described in this book illuminate how various research concepts and techniques apply to informatics and provide concrete ideas on how to conductvalid, informative, and publishable studies." Byron Marshall, Assistant Professor, College of Business, Oregon State University

Zusammenfassung
This book provides a practical, hands-on guide to conducting user studies in informatics. Its purpose is to explain the foundations of different experimental designs together with the appropriate statistical analyses for studies most often conducted in computing. Common mistakes are highlighted together with guidelines on how they should be avoided. The book is intended for advanced undergraduate students, beginning graduate students and as a refresher for any researcher evaluating the usefulness of informatics for people by doing user studies. With clear, non-technical language, fundamental concepts are explained and illustrated using diverse examples. In addition to the foundations, practical tips to starting, acquiring permission, recruiting participants, conducting and publishing studies are included. A how-to guide, in the form of a cookbook, is also included. The cookbook recipes can be followed step-by-step or adjusted as necessary for different studies. Each recipe contains step-by-step instructions and concrete advice.

Inhalt
SECTION 1 - DESIGNING THE USER STUDY.- Chapter 1: Overview.- Chapter 2: Variables.- Chapter 3: Design Equation and Statistics.- Chapter 4: Between-Subjects Design.- Chapter 5: Within-Subject Designs.- Chapter 6: Advanced Designs.- SECTION 2 - PRACTICAL TIPS
Chapter 7: Understanding Main and Interaction Effects.- Chapter 8: Conducting Multiple Comparisons.- Chapter 9: Gold Standard and User Evaluations.- Chapter 10: Recruiting and Motivating Study Participants.- Chapter 11: Institutional Review Board (IRB) Approval.- Chapter 12: Resources.- SECTION 3 - COMMON MISTAKES TO AVOID
Chapter 13: Avoid Bias.- Chapter 14: Avoid Missing the Effect.- Chapter 15: Avoid Missing Variables or Conditions.- Chapter 16: Other Errors to Avoid.- SECTION 4 - Appendix: COOKBOOK for Designing User Studies in Informatics.- Recipe1: Evaluating Standalone Algorithms using Artifacts.- Recipe 2: Evaluating Standalone Algorithms using Subjects.- Recipe 3: Comparing Algorithms Using Artifacts.- Recipe 4: Comparing Algorithms using Subjects.- Recipe 5: Evaluating Standalone Systems using Subjects.- Recipe 6: Comparing Systems Using Subjects.