Designing Educational Project and Program Evaluations

Drawing upon experiences at state and local level project evaluation, and based on current research in the professional literature, Payne presents a practical, systematic, and flexible approach to educational evaluations. Evaluators at all levels -- state, local and classroom -- will find ideas useful in conducting, managing, and using evaluations. Special user targets identified are state department of education personnel and local school system administrative personnel. The volume can be used by those doing evaluation projects `in the field', or as a text for graduate courses at an introductory level.
The book begins with an overview of the generic evaluation process. Chapter Two is devoted to the criteria for judging the effectiveness of evaluation practice. Chapter Three addresses the all important topic of evaluation goals and objectives. Chapters Four, Five and Six basically are concerned with the approach, framework, or design of an evaluation study. Chapter Four contains a discussion of four major philosophical frameworks or metaphors and the implications of these frameworks for conducting an evaluation. Chapters Five and Six describe predominantly quantitative and qualitative designs, respectively. Design, implementation and operational issues related to instrumentation (Chapter Seven), management and decision making (Chapter Eight), and reporting and utilization of results (Chapter Nine) are next addressed. The final chapter of the book (Chapter Ten) considers the evaluation of educational products and materials.

Inhalt

1. Educational Evaluation: Definitions, Purposes, and Processes.- The Nature of Project and Program Evaluation.- The Roles of Evaluation.- Evaluation Is Not Research.- Activities in the Evaluation Process.- Evaluation Truisms.- Cogitations.- Suggested Readings.- 2. Criteria for Effective and Ethical Evaluation Practice.- The Place of Values.- The Importance of Ethics.- Professional Standards and Metaevaluation.- An Example Lesson from the Standards.- Cogitations.- Suggested Readings.- 3. Evaluation Goals, Objectives, and Questions.- Identifying and Involving Stakeholders.- Kinds of Evaluation Questions.- Levels of Specificity in Educational Outcomes.- Program Development and Evaluation Questions.- Developing Evaluation Questions.- Standard Setting.- Evaluation Questions and the Utilization of Results.- Cogitations.- Suggested Readings.- 4. Evaluation Metaphors.- The Nature of Metaphors in Education and Evaluation.- The Management Evaluation Metaphor.- An Illustration of a CIPP Evaluation: Evaluating a Gifted and Talented Program.- The Judicial Metaphor.- The Anthropological Metaphor.- An Attempt at Responsive Evaluation: Evaluating a Teacher Evaluation System.- The Consumer Metaphor.- Metaphor Selection: In Praise of Eclecticism.- Cogitations.- Suggested Readings.- 5. Quantitatively Oriented Data Collection Designs.- Factors Affecting Evaluation Design Decisions.- Elements of a Data Collection Design.- The Validity of Data Collection Designs.- Data Collection Designs.- Cogitations.- Suggested Readings.- 6. Qualitative and Ethnographic Evaluation.- A Rationale for Interpretive Inquiry.- Qualitative Design Issues.- Data Collection Methods.- Data Analysis.- Cogitations.- Suggested Readings.- 7. Creating and Selecting Instrumentation.- Characteristics of High-Quality Data.-Types of Measures.- Opinionnaire and Free-Response Methods.- Observation Methods.- Approaches to the Assessment of Affective Variables.- Authentic Assessment.- Locating Information About Measuring Devices.- Cogitations.- Suggested Readings.- 8. Managing Evaluations.- The Role of the Evaluator.- Planning Considerations.- Legal Considerations.- Cost Considerations.- Decision Making.- Cogitations.- Suggested Readings.- 9. Communicating and Using Evaluation Results.- A Rationale for Reporting.- Report Preparation.- Factors Related to Utilization of Evaluation Results.- Cogitations.- Suggested Readings.- 10. Evaluating Educational Materials.- Criteria for Evaluating Educational Materials.- Evaluating Instructional Text.- Evaluating Computer Educational Software.- Cogitations.- Suggested Readings.- References.- Appendices.- A. Multiple Criterion Measures for Evaluation of School Programs.- B. Contract for Professional Services.- Contract for Educational Program Audit.- Name Index.