Cookies on Knowhow Nonprofit

We use cookies in order for parts of Knowhow Nonprofit to work properly, and also to collect information about how you use the site. We use this information to improve the site and tailor our services to you. For more, see our page on privacy and data protection.

OK

Skip to content. | Skip to navigation

Community-made content which you can improve Case study from our community

How to analyse qualitative data for evaluation

Qualitative data is data that is not numerical. It may include open-ended responses to questionnaires, data from interviews or focus groups, or creative responses such as photographs, pictures or videos.

Analysing qualitative data will help you produce findings on the nature of change that individuals or organisations you work with have experienced. You may also be able to look at what aspects of the way you work have led to change.

Analysis involves finding patterns and themes in the data you have collected for your evaluation. Analysing your data will help you report on it effectively and use it to make decisions.

You may have started your evaluation with questions you wanted to answer – for example, have we achieved our intended outcomes, or have we reached the individuals and organisations that we expected to? Analysis will help you to answer these questions.  

Things you'll need

  • Your qualitative data
  • Your evaluation framework or theory of change
  • Your evaluation questions
  • The software you have chosen to analyse your data (see step 3 in this How To)
1

Decide on your approach

There are two main approaches to analysing qualitative data.                  

‘Code and count’

This approach is helpful when you want to understand how many people gave a particular response. It involves ‘coding’ your data into different categories and counting how many responses are in each category. You can then use quantitative methods to analyse the data.

Code and count is good for larger sample sizes. It is often used to analyse data from open-ended questions in surveys or when you have data that can easily be separated into distinct categories. 

Be cautious about how you use the counts from your data. Depending on your sample size, you may not be able to generalise from the data that you have collected. In addition, how often something is mentioned may not be an indicator of its importance.

‘Theme and explore’

Using this approach, you review your data to see what main themes emerge and then explore how different people have responded around each of these. You can select key quotations from each respondent to illustrate the themes you have found. This is more like a literary analysis.

Theme and explore is good for smaller sample sizes and more complex subjects. It is particularly helpful when your respondents have different understandings of the same issue and you want to compare them. It can also help develop evaluation findings around how the way in which you work has contributed to changes that individuals or organisations you work with have experienced.

2

Create your codes or themes

Once you have decided which approach you are taking, you can generate the codes or themes you will use to analyse your data.

Code and count

A ‘code’ is made up of three parts:

  1. The code itself – a number or letter that represents the code.
  2. The category it represents.
  3. What is included or excluded.

The following example is a set of codes that was used to analyse data from interviews about why people moved into a care home.

Code

Category

Inclusion/exclusion

I

Ill health

Include general references to illness as well as mentions of specific illnesses.

Do not include statements such as ‘the doctor said I should’ if they do not also include a mention of illness.

F

Family

Include:

  • Family unable to look after
  • Family didn’t want to
  • Family too far away

Do not include references to respondent not wanting to be a burden on their family.

P

Professional recommendation

Include references to doctor, nurse, social worker, other health or care professionals.

Include ‘the doctor said I should’ and ‘the doctor forced me to’.

Your codes should be clear and unambiguous. Ideally two people would code the same responses in the same ways.

You can decide your codes in advance (pre-coding), or decide on them once you have looked at your data (emergent coding), or use a combination of the two.

  • Pre-coding is helpful if you know which answers you are likely to get. The outcomes and indicators in your evaluation framework may provide you with some pre-made codes.
  • Emergent coding is helpful if you are unsure of what your data will say. Start by reading through 10-15% of your responses and deciding on a draft set of codes. Use these to code your responses and check if you need to change your codes after looking at each further 10-15% of responses. Remember to re-code the data you’ve already looked at if you subsequently change your codes.

Codes can be descriptive, analytical or both. The example codes for the care home interviews are descriptive. They categorise what people say, without reading between the lines. Analytical codes allow you to categorise how people say things. They might be as simple as coding ‘positive’ and ‘negative’ statements about an issue or event. Alternatively, they might be more complex interpretations.

Theme and explore

For this approach, you need themes rather than codes. A theme is also a category but may not have such rigid inclusion and exclusion criteria. Themes are usually decided on after having read most or all responses. Then you can decide which themes best fit the data and what you want to understand from it.

For example, if you had interviewed people about their attitudes to food and read through your data, you might find the following themes emerging:

  • Food as fuel
  • Food as pleasure
  • Moral aspects to food (‘good’ vs ‘naughty’)
  • Emotional aspects such as comfort eating or expressing love through food
3

Categorise your data (‘code’ or ‘theme’)

Now that you have your codes or themes, you can use them to sort your data before summarising what it says. You can categorise data in various ways, depending on how much data you have and what software is available to you.

By hand

With a small amount of paper-based data and a small number of codes or themes, you can categorise by hand. Read through your data and make a note of the codes or themes in the margin. You can then cut up the transcripts and paste them onto larger sheets of paper, one for each code or theme. You may also want to highlight data in different colours.

Using MS Word or equivalent

If your data is in MS Word or Google Docs you can use a similar approach to paper-based data. Use the comments feature to make notes in the margin, or copy and paste sections of your transcripts into a new document under each code or theme.

Using a spreadsheet

If you have a survey dataset that you can export to a spreadsheet, you can use this to categorise your responses. If you are using code and count, create a column for each code and put a ‘1’ in the column if that code is mentioned in the survey response. You can then use the ‘sum’ formula to count how many times the code is mentioned, and the ‘filter’ function to view all the responses for a particular code.

Using specialist data analysis software

You can use a specialist software package to analyse qualitative data. Quirkos is an affordable option if you are working with text. Atlas.ti enables you to work with text, images, audio and video data. MAXQDA and NVivo are the market leaders for working with both qualitative and quantitative data. These packages allow you to code data more quickly, search for codes or groups of codes, and visualise your data in graphs or charts. They are more expensive but if you analyse qualitative data regularly then you may wish to invest in them.

Tips for categorising data

However you categorise your data, there are some key things to remember:

  • Data can be categorised into more than one code or theme, but try not to do this too often.
  • If you are using code and count you will need to make notes of how often each code appears. You may want to create a table or tally chart to do this.
  • Whether you are using code and count, or theme and explore you will also need a category for ‘don’t know,’ ‘no answer’ or ‘other’ responses. If ‘other’ responses make up more than 5% of your total, consider looking at the data again to identify possible additional codes or themes – this helps make sure you’re not missing something important.
  • It can be very helpful to write notes to yourself of any key points as you go through your data, and make a note of any particularly interesting quotes from respondents.
4

Think critically about your data

Once you have categorised your data, look at it again to draw out key findings – don’t assume the data speaks for itself!

Questions that you might want to ask of your categorised data include:

  • Are there any links between codes? Are some things mentioned together frequently? If so, what does this mean?
  • Are there any other patterns, themes or trends?
  • Are there any deviations from these patterns? How can these be explained?
  • Are outcomes different for different groups of people?
  • Why were some outcomes achieved, and others not achieved? How does this link to the outputs?
  • How do people understand their own journey or story? What do they think has caused or affected the outcomes that they have experienced? How does this relate to your theory of change?
  • What has surprised you about the data?
  • What has disturbed you or challenged your assumptions?
  • What do you need to find out more about?

You can write short notes or memos about each of these which will help you to construct your evaluation report.

5

Check your analysis

Make sure that your analysis can be verified and that you can justify the claims that you make based on your analysis.

  • Keep a paper trail including copies of your notes, your coded data, any memos that you write to yourself. If someone asks how you reached a conclusion, you should be able to talk them through the process.
  • Check your analysis with others. It can be helpful to have two people code some of the data and check whether their coding matches. You can also invite others to give feedback on your interpretations. You may also wish to check your analysis with your evaluation respondents to check that you are representing them accurately and to see if you have missed anything.
  • Wherever possible, check data from different sources to see if the results are the same or different (this is called ‘triangulating’). Discrepancies indicate something interesting that’s worth exploring further.
  • Check your own biases. Human beings are prone to looking for evidence that confirms what we already think. Write down your initial views on the data and deliberately look for evidence to dis-confirm your views.
  • Coding your data can result in looking at statements out of context. Check back against the rest of the data provided by a respondent (for example the whole transcript of their interview) to make sure you haven’t misinterpreted data. This is easier to do with the theme and explore approach as you usually have less data to work with.
6

Use your analysis

Now you’re ready to bring together your data analysis into a report or other presentation format. Read our guides on writing an evaluation report and using creative reporting formats for evaluation.

Further information

This how-to was contributed by NCVO Charities Evaluation Services.

Contributors

Page last edited Jul 31, 2017 History

Help us to improve this page – give us feedback.

1 star 2 stars 3 stars 4 stars 5 stars 2.7/5 from 13 ratings