Evaluation is assessing the success of a project in meeting its goals. It involves measuring the outcomes of the project and reflecting on what has been learnt as result of the activity. Good quality evaluation is essential to understanding what was done, how well it was done and whether it made a difference to the people involved.
Thinking about evaluation throughout all stages of the project (including design, implementation and reporting) can help project staff, community partners and stakeholders make effective decisions, build evidence to support their programs and continuously strengthen and improve the project.
The four main steps to evaluation are:
- collecting data – quantitative and qualitative
- collating the findings
- sharing learnings and making recommendations.
Reasons to evaluate
- Monitor progress towards identified goals.
- Assess if a project has achieved its intended goals and purpose.
- Allow for continual improvement.
- Add to the evidence base for quality integrated health promotion.
- Hold key partners and funding bodies accountable.
- Assess how sustainable and meaningful the project was for participants.
- Justify the need for further funding and support.
- Help to understand if our projects are working.
- Help to understand and improve practice.
Types of evaluation
Project evaluation has been separated into three main categories based on when the evaluation is being conducted and the main type of information collected. See the types of evaluation in the table below.
To clarify need for the project.
To clarify the theory of change.
To develop a logic model and evaluation plan.
To ensure the project is being delivered effectively and efficiently.
To measure short-term results.
To help identify key learnings.
To collect information and data, to learn and continuously improve.
Looks at the broader outcomes and impact of the project, that is, the effectiveness of an intervention on the target group. It helps to assess progress on the project outcomes (what the project or project has achieved). This is usually the last stage when you produce your post-project or project evaluation report. Hopefully you will share your learnings to help inform and improve future projects.
To consider impact of the project.
To assess progress on project outcomes.
To produce project evaluation report.
Planning is important. Drafting an evaluation plan can be time-consuming, but it’s best to do it as early as possible before you put your plan in to action. The reasons are:
- to identify the goals and objectives of the evaluation
- to provide a transparent guide for how the evaluation will be conducted
- it helps you to decide what sort of information you need to collect and the best possible methods
- it helps design the most appropriate evaluation questions
- it clarifies roles and responsibilities of those involved in the evaluation
- it determines the most appropriate strategy/design and clarifies assumptions/evidence on which project design and implementation are based
- outline how a project intends to produce results
- determine the most appropriate data collection methods
- outline how the evaluation results will be disseminated
- assess what resources are needed to do the evaluation.
The four main steps to develop an evaluation plan are:
- set project objectives and goals
- develop evaluation questions
- develop evaluation methods
- set up a timeline for evaluation activities.
Indicators of evaluation
An indicator is a marker of change or progress that shows how your project plan is progressing. Indicators for your evaluation should be:
- developed together with stakeholders
- developed in relation to evaluation questions
- SMART (specific, measurable, achievable, realistic and time-related) Indicators help determine what data needs to be collected, to help you measure the progress of your project and whether it is on track to achieve its objectives.
The table below highlights the types of indicators that can be used in project evaluation.
|Project inputs||The resources needed to operate the project.||
|Process||Monitor how well the project is implemented, if it is reaching the intended target and if it is of an acceptable quality.||
Project advisory group/steering group established.
Number of sessions delivered by peer educators in community. Community assessment conducted and reported.
Reach of social marketing campaigns recorded / media coverage achieved.
Number of event attendees.
Number of GP requests for information during the campaign period.
Number of service visits by peer educators.
Number of prompt cards handed to patients.
Proportion of target audience that participated in the project.
|Impact||Monitor the progress of achieving the project objectives, which usually relate to some type of short-term changes. Impact indicators will usually relate to changes in knowledge, attitudes and intended behaviour.||
Changes in awareness, knowledge and skills (e.g. increased confidence of peer educators to share cancer prevention messages; improved awareness of breast screening).
Changes in intended behaviour (e.g. intention to screen).
Improved system for screening reminder and recall.
Changes in screening service usage (e.g. increase in the number of women from each Under Screened Recruitment Program target population attending cervical cancer screening tests and BreastScreen clinics for mammograms). Increased partnership networks.
|Outcome||Used to assess if the project goal has been achieved and are more likely to include actual behaviours, health status and quality of life (longer term changes or changes sustained over time).||
Strengthening community participation.
Increased community awareness / motivation to participate in the National Bowel Cancer Screening Program.
Improved timely access for Aboriginal people to cervical screening services.
Increased cervical (or bowel, breast) screening rates.
Key evaluation questions
Key evaluation questions are high-level questions that an evaluation is designed to answer – they are not questions that are asked on a survey. They should cover key areas (e.g. reach, appropriateness, implementation, effectiveness, efficiency and/or maintenance).
Your choice of key evaluation questions should be developed with consideration of the type of evaluation you do, its intended users, what purpose the evaluation has, and the evaluation criteria you use. Having 5-7 questions for your key evaluation questions is often enough. Read the Centre for Disease Control’s short guide to developing evaluation questions.
These questions were used for the Under Screened Recruitment Program evaluation report as an example.
- How effective was the program design?
- What was the impact of the program?
- What was the outcome of the program?
- What elements could be sustained?
When seeking community evaluation
Always keep your community in mind when designing evaluation tools that measure their awareness or knowledge. For instance, if you presented cancer screening messages to a group of people that was part of a larger community event, you probably should not expect the audience to fill out a two-page survey. For evaluation that measures community awareness or knowledge, think about the most important data you want to collect, the easiest way to collect it and always keeping your evaluation short.