Cookies on Knowhow Nonprofit

We use cookies in order for parts of Knowhow Nonprofit to work properly, and also to collect information about how you use the site. We use this information to improve the site and tailor our services to you. For more, see our page on privacy and data protection.

OK

Skip to content. | Skip to navigation

Community-made content which you can improve Case study from our community

How to Evaluate your Local Area

All Fair Share Trust (FST) Local Agents are required to undertake evaluations as part of the programme. The purpose of FST local evaluation is threefold:

1. for funded groups – to celebrate and raise profile;

2. for Local Agents – to assess work done and measure impact;

3. for CFN – to share learning & highlight recommendations for future funding.

Around the network, a variety of methods have been used and lessons learned which will be helpful for Local Agents and any funders or groups looking to evaluate the work they have done. This document looks at some of this learning from around the network.

1

Ongoing Evaluation

As the end of the programme draws near, how do you retain the interest of Panel members as their primary role and function in endorsing recommendations for funding come to an end? Scottish Community Foundation suggests giving the Panels an ongoing project monitoring and evaluation role to provide them with an ongoing remit and ownership of the funded projects throughout the life of the programme

2

Refreshing Priorities

Community Foundation for Merseyside (CFM) noticed that at one point grant making stopped in the Liscard Ward. The reason for this was a low number of applications received together with poorer quality applications that did not demonstrate that projects would tackle the locally identified priorities. To improve this situation, CFM conducted a needs analysis in the area based on research and community consultation. Panel membership was also refreshed and numbers increased. The aim was to review the needs in the area to ensure that the priorities were still relevant. Changing the priorities (i.e. ensuring they are still in line with the Indices of Multiple Deprivation) allowed groups to apply with new, innovative project ideas.

The lesson learned is that grant making and area need is ever changing and we must be flexible and adaptable to be proactive to such changes rather than reactive. Carrying out ongoing research and consultations helps us keep abreast of changes.

3

Baseline Data

In order to effectively evaluate the impact of the work that has been done in an area, reliable baseline data is needed for comparison. Across the FST network, we have seen examples of some imaginative ways of gathering this information. For example, if work has been done on improving the facilities in a local playground, it may be possible to gather photos showing the area prior to work being undertaken, to compare with ‘after’ photos. There may be police records relating to graffiti or anti-social behaviour, or council records showing playground user numbers. Were there press cuttings or minutes of local residents’ meetings raising concerns about safety, prior to the funded project’s intervention? And how about your original Neighbourhood Assessment documents?

All of these and more may provide evidence against which up-to-date information can be aligned.

4

Methods of gathering data

Good qualitative data can be gathered through a range of methods, including ‘vox pop’ interviews in the neighbourhood; video booths; electronic voting; text or Twitter; mystery shopping; role play; photographs or visual audit; interactive exercises such as drawing maps or timelines of a project, or drawing pictures to illustrate impacts; local media reports/coverage; local blogs or community websites; surveys to gather information about numbers of beneficiaries and how they benefited from the work. A good evaluation will use a broad and creative approach to getting the best out of research participants. A good external evaluator will have thought this through very carefully, and will be using tailored and wide-ranging methods for evidence gathering.

5

Giving clear guidance to local evaluators

When drawing up briefs, it is helpful to give clear guidance to bidders about your expectations of the final reports’ format and presentation. In 2009, a local panel, with support from the Scottish Community Foundation (SCF) commissioned an evaluation of the Fair Share Trust programme in the Rural Area of South Lanarkshire. Following an open tender process, consultants were commissioned to undertake the research and a draft report was submitted to the local panel and SCF. All comments and amendments, including grammatical errors and formatting suggestions, were forwarded to the consultants to enable them to produce the final evaluation report. While the content of the final report was very good, the style and presentation made it difficult to read and resulted in the panel Chair re-formatting the entire document to improve layout and general legibility. With hindsight, SCF say that the tender brief should have contained style guidance for the presentation of the final report. They have since commissioned other external evaluations and now include a section on the expected format and presentation of the final report in every tender brief.

6

Independent mid-term evaluations

Birmingham Community Foundation (BCF) note that an evaluation carried out after the first five years of FST has given them an excellent insight into how effective FST has been in meeting the objectives of the programme. While the findings identified a number of areas where the programme had been most successful, there were other areas that needed improvement: for example recognition of the FST name and better communication and co-operation between residents living on individual estates. While most of the findings were already known and documented, independent evaluation of the challenges facing the delivery of the programme have helped BCF to develop its overall sustainability strategy for the area and given them the evidence to take this work forward.

7

Open monitoring processes

Some groups been proactive in ensuring that they have appropriate systems in place for evaluating and monitoring the impact of their projects. With support from Scottish Community Foundation (SCF), one group devised a system that captured relevant data to evidence how they were meeting the defined outcomes of the project. As a result, SCF receives an additional spreadsheet with each monitoring report that records whether the group has met each of the identified outcomes. The group has then used this to re-negotiate targets for the 2nd year of the project in light of their experience. For example, in Year 1 their target was to support 300 people. They achieved 553. For year 2, the target has been adjusted to 150, to take into account that the project will still have to service some of the existing clients. While resource-intensive, the time spent meeting with the group to define monitoring systems has been worthwhile as the funder ends up with quality monitoring data which can be better used for evaluation purposes.

Further information

 

Key Recomendations

Below are some of the key recommendations made by Local Agents on working with independent evaluators:

  • in the most successful evaluations, consultants had prior knowledge of the sector and the FST area
  • evaluators need to factor in community consultation
  • evaluators spent time with the funded projects and other stakeholders in the community, gathering views to ensure that they were clear about strengths and weaknesses of the programme
  • consultants with more creative and qualitative approaches may be more attuned to the spirit of FST.

Contributors

Page last edited May 26, 2017 History

Help us to improve this page – give us feedback.

1 star 2 stars 3 stars 4 stars 5 stars 3/5 from 1264 ratings