The ABCs of Evaluation

PREFACE Demonstrating Success Overview Using This Book THE AUTHORS What Is Evaluation?Common Denominators Two Definitions of Evaluation Evaluating Efficiency, Effectiveness, and Impact Evaluating Alternatives Identifying Areas to Improve Other Levels Formal Reasons to Evaluate Practical Application Formative and Summative Evaluations Putting It All Together Why Evaluate?Grants and Grantsmanship Benefits Benefits to Sponsors and Staff Opportunities Understanding of Outcomes Limitations Considerations Social and Political Situations Decision-Making Needs Research for New Solutions Putting It All Together Key Words and Concepts Decision Making: Whom to Involve, How, and Why?Program and Project Cycles Overlooked Stakeholders Program Phases Evaluation and Monitoring Sample Data Sources for Monitoring a Job-Training Program Sample Data Types for Monitoring a Job-Training Program Program Planning Cycle and Evaluation Needs Analysis Program Planning Program Implementation and Formative Evaluation Final Evaluation: Summative Types of Evaluation for Decision Making Putting It All Together Key Words and Concepts Starting Point: The Evaluator's Program Description What an Evaluator's Program Description Can Do Developing an Evaluator's Program Description Questions Based on the EPD Questions Not Specifically Based on the EPD Putting It All Together Key Words and Concepts Choosing an Evaluation Model More Than One Way to Evaluate Four Evaluation Models Discrepancy Model Goal-Free Model Transaction Model Decision-Making Model Other Models Goal-Based Method Choosing the Right Model Evaluation Design Format Other Considerations Putting It All Together Key Words and Concepts Data Sources Starting Point Qualitative and Quantitative Data Sources Four Levels of Data Nominal Data Ordinal Data Interval Data Ratio Data Nature of Data Collection Existing Data Newly Collected Data Interview Scales Sentence Completion Tests Observational Analysis Putting It All Together Key Words and Concepts Data Analysis Data Analysis Data Terms Statistics Measures of Central Tendency Measures of Variability Levels of Data Quality of Work: The Extent to Which the Employee Actually Completes a Job Assignment Correctly Putting It All Together Key Words and Concepts Is It Evaluation or Is It Research?Determining Evaluation or Research from Intent Determining Evaluation or Research from Design Qualitative and Quantitative Characteristics Sampling Probability Sampling Methods Nonprobability Sampling Methods Sampling and Surveys Putting It All Together Key Words and Concepts Writing the Evaluation Report Communicating Your Findings Evaluation Reports The Report Outline Section One: Summary Section Two: Statement of the Evaluation's Purpose Section Three: Background Information Section Four: Description of the Evaluation Study Section Five: Results Section Six: Discussion of the Program and Its Results Section Seven: Conclusions and Recommendations Putting It All Together Key Words and Concepts Evaluation as a Business Is There a Market for Program Evaluation?Public Sector Private Sector How Do You Find the Business?Proactive Resources Public Sector Private Sector Reactive Resources Emergency Resources How Do You Negotiate a Contract?Negotiating Responsibilities Pricing Your Services Preparing a Contract Keeping the Business Alive Maintaining Relationships with Funding Organizations Long-Term Contracts Putting It All Together Key Words and Concepts More On Data Analysis Sample Evaluation Report Sample Evaluation Contract