What are participatory methods?
We discuss three methods here.
- Visual methods, such as participants’ drawings, collages, photos and video footage.
- Interactive verbal feedback, such as participants’ diaries and storytelling.
- Diagrams and other creative tools.
Why use participatory methods?
Participatory methods offer a less time consuming way to gather data than individual interviews or questionnaires.
However, more importantly, they are seen as a way of passing some of the power in an evaluation to the participant. As a result the participant is more likely to feel engaged and own the results.
Creative and participatory methods allow different views and perspectives to be captured in specific locations and at specific times.
Different creative tools can be used for people who may not respond well to interview situations or to a questionnaire, or who may not have narrative skills.
For children, particularly, creative methods may help them to recall events and feelings and talk about their behaviour.
Visual methods can offer a valid method of self-reporting which is not reliant on words or numbers.
They can also capture information which can be accessed later, for example, digitally without having to rely on memory.
Participatory methods can be experienced as fun as compared with responding to a questionnaire or other more formal tools.
Participants can be more independent and less restricted by criteria set by the evaluator.
With some methods, for example, visual methods, or storytelling, stakeholders can become partners in the evaluation process.
Visual methods of capturing outcomes information can be used to draw out perspectives and feelings that may not have been articulated through words.
They can provide a way to capture events in a person’s life. The data generated can be used to illustrate change over time.
It is important to discuss the photographs or artwork, framing discussion around themes or indicators, and allowing new themes to emerge.
For example, photo diaries and Photovoice include discussions about the individual photographs taken by participants.
When using photographs, you need to take into account confidentiality issues, and also seek consent from the individuals being photographed.
Interactive verbal feedback
Verbal methods can also provide interactive feedback.
For example, participants can write on a ‘graffiti wall’ at events and workshops. They can keep a daily or weekly diary, or use social media.
You can get quantitative information in a number of ways in a workshop or other group situation. You can ask for a show of hands or hold a secret ballot. Or you could ask people to move around and group themselves to demonstrate agreement with certain categories.
You can collect information on the usefulness of your internet-based services by tracking the number of hits and visitors to your site and inviting users to email you with feedback.
Hosting an interactive website can be a quick and relatively easy way to consult users, share information and get feedback.
Participants can be asked to collect stories to demonstrate change. This has the particular strength that participants in the evaluation are directly involved in deciding the sorts of change to be recorded.
Most significant change
The most significant change (MSC) is a technique that is used mainly for monitoring and evaluating in an international development context. After stories are collected the most significant stories are selected to demonstrate outcomes and impact.
Stakeholders work as a group to discuss the value of the reported change. In a large programme, there may be several layers or levels at which these stories are shared and the most significant ones selected.
The method can be expanded by further group work on the stories, to analyse how different stories overlap and the causal connections between different groups of stories.
Stories and case studies present specific cases, and if you are trying to come to any generalised findings, great care must be taken, analysing against a framework of criteria and indicators.
Using mobile phones
The use of mobile phones to collect outcomes data in real time (for example, behaviour change) is another approach currently used more in international development evaluation than in the UK voluntary sector.
Data can be sent directly to a web based management information system, with results updated as records are collected. For example, Magpi (developed by DataDyne, Inc, based in Kenya) is a mobile data collection programme that has been used in 170 countries worldwide, largely for public health data collection.
Studies have been done about using mobile phones to collect data about the use of drugs and alcohol by young people, and their related behaviours. Participants answer questions about daily activities, alcohol use, stressors and negative moods. It has also been used to collect data on sexual activity in young people.
Using the internet
Online polls are used to quickly assess people’s opinion on a specific subject. They feature one question only, and results (or at least the number of responses) are fed back to the website in real time.
Polls can be a useful way to gather data quickly and cheaply. However, they are sometimes considered to be an irritation by website users and, as those who respond are self-selecting, considerable sample bias is likely. Online polls can work well on news websites or on popular forums as long as the result of the poll matters to users and it fits well within the context of the website.
You can also set up a short online survey for your website users, for example to provide information on how they have used your publications and other resources. This could provide you with outcome information that might otherwise be unavailable.
However, again, you will be unable to control who answers the survey. Some simple questions that help to identify the respondents’ profile will be useful.
Discussion forums for data analysis
Website discussion forums can be a great way for users to get information, share experiences and meet like-minded people. They can also be a fantastic source of data for evaluation purposes.
As long as confidentiality is not breached (that is, the identity of the users is not revealed), discussion forum data can be mined for evidence of the difference a service has made or to identify areas of improvement.
As users can stay anonymous, discussion on online forums is often more honest, revealing and detailed than data gathered by other means. The analysis of users’ discussions can also uncover unexpected outcomes and new areas of need.
You may be able to set up an online site where participants can share their diaries or their case stories to provide evidence of progress and change.
Your project or programme participants may also provide feedback through blogs, podcasts and video storytelling, using social network sites, such as Facebook, YouTube and Twitter.
There is online monitoring and reporting software that can allow media files to be uploaded and accessed as part of a report. This can bring your project and its outcomes to life for a wider audience, as well as to your existing or potential funders.
One disadvantage of using social media for monitoring purposes is that it may generate a large amount of unstructured data, so you need to plan carefully how it will be captured and analysed around specific indicators.
Using diagrams and other creative tools
Many of the ideas for participatory information collection are influenced by an evaluation approach called Participatory Learning and Action (PLA).
PLA emerged from various traditions including Participatory Rural Appraisal and participatory education approaches used mainly in developing countries in the global south as well as in urban areas in the global north where it is known as Participatory Appraisal.
PLA tools include the following.
These can be used to show a participant’s journey over time, noting important changes.
Daily activity charts
Activities recorded can illustrate the extent to which an individual is making progress against specific indicators.
This can be done by simply listing preferences in a priority order, by ‘pairwise ranking’, asking participants to choose between pairs of options, or by using a matrix, showing options on one axis and criteria on the other.
Project outcomes can be written on cards, and participants (individually or in groups) can be asked to sort the cards into piles to indicate their rating.
One option for this exercise is to divide a circle into segments, with each segment representing an outcome or outcome indicator. Participants can then tick the segment if they feel that this outcome has been achieved. It can also be used to show responses to activities or processes.
Causal impact diagram
These diagrams show how different variables relate to each other, with links showing positive or negative relations between them. They can help to interpret what happened.
Geographical mapping is used particularly in international development evaluation to illustrate how land is used, and access to resources, such as water and irrigation. Repeated maps can show change over time.
Social maps can provide a visual and spatial illustration of social information, for example, on demographics, health, and infrastructure.
You can use large sheets of paper and draw the outline of a person’s body. Participants are then asked to respond to questions, such as:
- what are you most worried about?
- what are you most hopeful about?
Participants draw and write their answer on or around their body outline.
Different sorts of diagrams (for example, a triangle or star) can be used to map indicators of mental health, physical activity, social well-being and healthy eating.
Other types of diagram
- Flow diagrams, webs and networks show interrelationships between different issues.
- Venn diagrams are overlapping circles which show distinct and common features. They can be useful in analysing relationships between different stakeholders and institutions. Diagrams can be redrawn and compared over time to assess and analyse change.
- Road journeys can be used to chart an individual or group’s journey and change over time.
- Matrices show the relationship between two variables as a table with quantified values. For example, they can be used to show outcomes by gender or ethnic group or to tabulate material from other types of diagram. Wheels or pie charts show the relative proportions of different elements of an item. They can show, for example, how people spend time during the day or week.
Challenges of participatory methods
There are a number of challenges to consider. For many participatory methods, there is an element of self-selection in who participates. This may bias the findings.
Some participants may lack confidence in the method used, for example drawing, photography or storytelling, and may as a result leave out details.
A high level of skill may be needed to interpret diagrammatic or other visual representations, for example of wellbeing.
Aggregating data from participatory methods can be difficult. Use of scoring systems may give a false illusion of objectivity.
A judgement may have to be made about how representative the participants in a participatory activity were.
Care needs to be taken when interpreting the data, as it may only represent a partial construct, and there may be a danger of over-emphasising data, analysing data out of context or distorting findings.
Tips for choosing a collection method
As with other information collection methods, time and care should be taken to make sure that you obtain valid and credible evidence.
- Spend time in advance to understand the context and to identify appropriate tools.
- Understand whether you need to make adaptations before you apply a tool.
- Accompany participatory tools with a semi-structured interview, observation, active listening, probing and note taking as appropriate. Record diagrams with digital photography.
- Seek out voices that are not being heard.
- Meet people when and where it suits them (not just you).
- Allow enough time for interaction.
- Record who attends and identify different stakeholder groups among participants.
- Prepare a list of key questions with which to interrogate the information collected.
- Have a continuous process of review in which you set aside time to make sense of collected information regularly rather than leaving it until all information has been collected.