Types of Program Evaluation
Program evaluation can be a needs assessment, a formative or process evaluation, a summative or outcomes evaluation, or an impact evaluation.
Needs assessment: Program evaluation can be conducted to assess a need and provide a community foundation to a future program or project. This form of evaluation occurs before starting a new program and is called a needs assessment. Needs assessments are conducted to answer questions, such as:
BOX 19.1 ABOUT THE RESEARCH PROCESS
WHAT IS SCIENTIFIC RESEARCH?
Scientific research is a systematic process of investigation about a natural or a social phenomenon or problem in order to find an explanation or a solution.
OBSERVATION
A problem is observed or identified.
Example: "Many women are paid less than men for the same position, qualifications, and types of work."
VARIABLE
A property of people or objects that takes on two or more values.
Must include categories that are both exhaustive and mutually exclusive. Example: Age or income (can take values such as 7, 2, 3, 4, 5, 6, 7, etc.).
HYPOTHESIS OR RESEARCH QUESTION Research Question
- A research question is a question about the relationship between at least two variables.
- Example: Is there a correlation between student employment status and academic performance?
Hypothesis
- A hypothesis is a tentative answer to a research question that must be empirically verified.
- Example: Students who are employed part-time have higher grade point averages (GPAs) than students who are employed full time.
- Note: This suggests a relationship between employment status and academic performance (GPA).
VERIFICATION OR EXPERIMENTATION
Data Collection
Population: The population is the total set of individuals, objects, groups, or events in which the researcher is interested.
Sample: A sample is a relatively small subset selected from a population.
Data Analysis
- Descriptive statistics: Procedures that help us organize and describe data collected from either a sample or a population.
Example: Total participants = 25; men = 12; women = 13
- Inferential statistics: The logic and procedures concerned with making predictions or inferences about a population from observations and analyses of a sample.
Example: There is a positive correlation between academic performance and the employment status of a student. Students who work part-time have higher GPAs than students who work full-time.
Generalization
- Predictions or inferences about a population from observations and analyses of a sample.
Example: There is a positive correlation between academic performance and the employment status of a student. Students who work part time have a higher GPA than students who work full time.
Research Process
- Step 1: Problem statement
- Step 2: Research plan
- Step 3: Data collection
- Step 4: Data analysis
- Step 5: Presentation of findings
- What are the characteristics, needs, and priorities of the target population?
- What are potential barriers/facilitators?
- What is the most appropriate response?
Formative or process evaluation: Formative or process evaluation aims to measure the outputs of a project and ensure that it is on track to meeting its goals. Formative evaluation assesses whether activities are being completed as planned and scheduled. It is designed to answer key questions, such as:
- Who are the beneficiaries or clients?
- What activities or services are provided to them?
- Where are the services being provided?
- When and how long will the services be provided?
Summative or outcome evaluation: As the name suggests, summative evaluation aims to measure the short-term and medium-term outcomes of a project. Summative or outcomes evaluation assesses whether a project changes a problematic situation into a new desired situation. Summative or outcomes evaluation helps answer questions like:
- To what extent did the program produce the intended results or outcomes?
- How effective was the program?
- Did the program achieve its goals?
- What was the level of quality of the services provided?
Impact evaluation: Impact evaluation is designed to measure the long-term outcomes or the impact of the program on the target population. Impact evaluation will attempt to answer questions such as:
- To what extent can changes be attributed to the program?
- What are the net effects?
- What are final consequences?
- Is the program worth the resources it costs?
