Measuring and Commissioning Outcomes and Social Value wiki

Explaining the definitions, practice and frameworks for commissioning outcomes and social value in public services delivered by voluntary and community organisations.

What is social value?

'We mean “value” not in its narrow sense but in its true sense—recognising the importance of social, environmental and economic well-being across our communities and in our lives' - Chris White, sponsor of the Public Services (Social Value) Bill.

Demos describe social value as the following:

'‘Social value’ refers to wider non-financial impacts of programmes, organisations and interventions, including the wellbeing of individuals and communities, social capital and the environment. These are typically described as ‘soft’ outcomes, mainly because they are difficult to quantify and measure. This in turn poses a problem for those seeking to measure the effectiveness of a particular intervention or activity with soft outcomes – be they the providers of that activity, the commissioners of that activity, funders, users, and so on. Outcomes that cannot be quantified cannot be counted, evaluated or compared. It is understandable, therefore, that the measurement of social value by ascribing quantifiable values to these soft outcomes preoccupies policy makers in this field.' (From Measuring Social Value: the gap between policy and practice (PDF 879KB). Demos, 2010)

Social Return on Investment (SROI)

SROI is a very specific measurement of value - based on the financial 'return' of a service.

The London Business School defines it thus:

'The SROI ratio shows the value of the social and environmental impact that has been created in financial terms. This makes it possible to weigh social benefit against the cost of investment. SROI also offers a framework for exploring how change is happening as a result of an intervention, showing ways in which this can be improved upon. Through the SROI process we learn how value is created by an organisation, and this is just as important as what the ratio tells us'. (From Measuring Social Value: the gap between policy and practice (PDF 879KB). Demos, 2010).

What is outcomes-based commissioning?

Ordinarily, the public sector purchases 'outputs' from providers. 'Outputs' are activity - say, running a lunch club 5 days a week for older people. This will be specified in the initial tender documents and subsequently in the contract between commissioner and provider.

Commissioning 'outcomes' requires very different tender specifications. Because an outcome isn't the activity itself, but the impact it has. So, rather than asking for 20 older people to be served lunch, the commissioner will ask for the impact they wish this service to have on those older people. That will be more than just their lunch - it might be that they feel less isolated, more mobile, and able to have time away from their carers and families.

1. There are three types of outcomes that can be measured:    Individual outcomes - e.g. Keith now gets up in the mornings without staff support;
2. Service level outcomes - e.g. the service supported 10 service users to access the correct benefits;
3. Strategic outcomes - e.g. more people will be helped to live at home.

We explore some of the challenges of outcomes-based commissioing below. While it isn't widespread - and contracts and tenders based on outputs is still the norm - there is widespread agreement that this will achieve better results by having wide indicators of success, and by encouraging providers to be more innovative and focused on their final impact.

The shift to outcomes requires a cultural change that, though challenging, is likely to enhance the quality of services and generate the best value for money in public services, by focusing on the difference services actually make to their users and not just on the outputs produced.

The Challenge of Social Value

It is generally considered that voluntary sector organisations are often well placed to deliver public services with wider benefits. It is said that VCOs tend to have better knowledge of and contact with the customer base, the ability to serve hard to reach groups, the capacity to listen to users' needs and generate innovative solutions to meet them, and a reputation of trustworthiness earned from being value-driven and working for years in a specialist service area.

The Public Administration Select Committee (PASC) report Public Services and the Third Sector: Rethoric and Reality, published in July 2008, says The central claim made by the Government, and by advocates of a greater role for the sector in service delivery, is that third sector organisations can deliver services in distinctive ways which will improve outcomes for service users. We were unable to corroborate that claim. 'Too much of the discussion is still hypothetical or anecdotal'.

This is a strong statement, and one that the sector cannot ignore. Existing evidence of the impact produced by the sector for service users does not yet have the breadth and depth needed to corroborate claims that the sector has distinctive ways of delivering better outcomes. The sector, with help from government, must invest in making sure that this body of evidence is collected going forward. This entails knowing what to measure and how to measure it.

Public Service Delivery Network Member, Frontline service provider:

'What we measure counts: measuring outcomes is important both in its own right, to demonstrate the effectiveness of what we are doing, and in creating the right incentives to improve performance'.

Public Services (Social Value) Bill

In April 2008, David Cameron declared: ‘The next Conservative  government will attempt to establish a measure of social value that will inform our policy-making when in power… When making decisions, ministers will take account not just of economic efficiency, but also social efficiency'.

Subsequently, 2011-12 have seen the Public Services (Social Value) Bill, introduced by MP Chris White, progress through Parliament with cross-party support. Progress and Parliamentary records of the Bill and its readings can be tracked on the UK Parliament website.

What does the Bill do?

In an article for NCVO, Chris White explained;

'The Public Services (Social Enterprise and Social Value) Bill, aims to reform the commissioning process by asking public sector bodies to consider how they can improve the “social, economic and environmental” well-being of their areas, through public sector contracts.

'This will mean that other factors than pure cost will be considered, and weigh the important contributions that our charities and voluntary organisations make across our communities, when they bid for public sector contracts.

'To take one example – imagine a local health authority is seeking to engage new providers to take on some adult social care services. It could merely decide to contract it out to the cheapest provider based on pure financial cost.

'But with social value as part of the contract, it would also be asked to look at those providers who have a track record of improving the well-being of residents of the local area, whether that is through mentoring schemes, engaging local volunteers, sourcing food from environmentally sustainable local providers or providing training to young people.

'Moreover, the local health authority would also have to engage with voluntary organisations and charities, so that a sensible consensus could be agreed to how social value would be relevant in this contract. This dialogue will give a chance for new innovations to occur within public sector commissioning and drive up standards.'

Content of the Bill

Article 3 of the Bill states that:

'The authority must consider—
(a) how what is proposed to be procured might improve the economic, social and environmental well-being of the relevant area, and
(b) how, in conducting the process of procurement, it might act with a view to securing that improvement'.

Article 6 of the Bill state that

'The authority must consider under subsection (3)(b) only matters that are relevant to what is proposed to be procured and, in doing so, must consider the extent to which it is proportionate in all the circumstances to take those matters into account'.

Parliamentary Debate

Parliamentary debate has raised a number of questions on implementation and reach of the Bill:

• 'The Bill places a duty on all commissioners in contracting authorities to consider how to improve social, economic and environmental well-being in services-only contracts that have a value above the EU procurement threshold and are at their pre-procurement stage', said Labour MP Gareth Thomas;
• Hazel Blears MP commented that, 'in a year’s time, more commissioning will have social value at its heart. What work are we doing to hone in on how we measure social value so that we get a grip on this, because otherwise it will remain a fairly nebulous concept that is very easy for people who do not share the values to wriggle out of, and I am sure that that is not what he wants?'
• 'Will the local government finance regulations on best value be amended to allow social enterprises that over-bid a private sector organisation to have that contract awarded? Currently, it is very difficult for local authorities to award contracts to a higher bidder without, in some cases, a Secretary of State’s approval—as the right hon. Member for Salford and Eccles will know, as many such cases probably came across her desk'. Mike Freer, MP.
• Nick Hurd MP responded, 'We believe that the Bill will complement that useful guidance by sending an additional signal to commissioners outside local authorities that, where it is relevant and proportionate, they should consider social value at the pre-procurement phase when they are considering commissioning services. That is how we can balance out the areas in which we think the biggest impact can be made now and our desire not to impose too many disproportionate duties and burdens on people who are doing already very difficult jobs. That is where we are comfortable with pushing the agenda forward'.

You can read House of Commons debate on the Bill in Hansard.

Measuring Social Value

Measuring Social Value is widely recognised as a challenging task. Demos say (Measuring Social Value: the gap between policy and practice (PDF 879KB), 2010) three things must be remembered:

1. 'we should not forget our ultimate goal is social impact – measurement is merely a tool to help us maximise this';
2. 'the move to better measures of social return on investment is a journey. The important thing is to embark on that journey, get the benefit from the first steps, and not worry too much that nirvana is a long way off';
3. 'the sector is going to need a lot of help'.

Recommended Tools

There are a number of tools available (full list here). They're suitability varies depending on the organisation or project you're looking to monitor.

A number of tools can be found listed on our impact pages and Community Matters launched the Your Value tool in 2011. This latter tool is supported by Department of Communities and Local Government.

Reading through the Demos report, Measuring Social Value: the gap between policy and practice, also gives a useful insight into the skills and processes that organisations will need in place in order to measure their social value.

Procuring Social Value

Taken from Office for Government Commerce (OGC) guidance, ‘Buy and Make a Difference’ (2008), explaining the correct process throughout the procurement cycle.

Pre-Procurement

Before procurement, OGC ecommend that the voluntary and community sector - as well as communities themselves - be involved in to help form the spec etc of the contract.

Service Specifications

‘Social considerations can be included in specifications where they are directly relevant to the subject matter of the contract. Core requirements are essential parts of a contract, releffcyed in both the specifications and in the conditions of the contract. A social issue can be a core requirement and reflected in the specifications provided it is central to the subject of the procurement and consitustant with the public procurement Regularions’. ‘Buy and Make a Difference’ OGC (2008)

At selection stage

OGC state that,

'The procurement Regulations contain an exhaustive list of references or evidence that potential suppliers can be required to provide in order to demonstrate their technical capability in relation to the nature, quantity and purpose of the contract. If a contract requires specific know-how in the “social” field, specific experience may be used as a criterion to prove the suitability of potential suppliers in regard to technical and/or professional ability. Contracting authorities can ask potential suppliers for relevant evidence of technical and/or professional ability, for example, language skills or cultural awareness'.

Award Stage

OGC write:

'A contract should be awarded to the tenderer offering the best value for money – that is, the optimum combination of whole-life costs and quality to meet the authority’s requirements. Value for money in this context equates to ‘most economically advantageous’ for the contracting authority (not wider), in the parlance of the UK Regulations implementing the EU Procurement Directives. The Regulations list a number of criteria, by way of example, that contracting authorities can use to identify which tender would be the most economically advantageous. These award criteria include price, delivery or performance dates, running costs, cost-effectiveness, quality, aesthetic and functional characteristics, after-sales service and technical assistance. Criteria involving social considerations may be used to determine the most economically advantageous tender where they provide an economic advantage for the contracting authority which is linked to the product or service which is the subject matter of the contract.Where there are two or more bids which are equal on value for money grounds, it is possible to use ‘additional social award criteria’ to determine between them; legal advice should be sought first, as it is very rare for bids to be equal in this way'.

Contract Conditions and Social Clauses

OGC continue,

'The term social clauses refers to special conditions relating to the performance of a contract, which address social issues. Contracting authorities may lay down special conditions relating to the performance of a contract, provided that these are compatible with European Community law and are indicated in the contract notice or specifications. Contract conditions must relate to the performance of the individual contract in question. They should be relevant and able to be met by whoever wins the tender from the time at which the contract starts.

'They should not be disguised technical specifications or selection or award criteria. If, for example, particular skills or qualifications are needed for a contract, these should be considered at the selection stage; additional training, not essential for a contract, by suppliers can be agreed on a voluntary basis
once the contract has been awarded.

'Contract conditions must not discriminate directly or indirectly against national or non-national tenderers. Contract conditions that require changes to the organisation, structure or policy of a supplier established in another Member State might be considered discriminatory or a barrier to free trade. Value for money should be maintained; contract conditions should be supported by the benefits they accrue set against the cost of achieving them. Care should be taken to avoid the imposition of blanket clauses on suppliers, which could be regarded as burdensome and might deter suppliers from competing for government work.

'Contracting authorities have a wide range of possibilities for determining the contractual clauses on social issues. They may in particular be intended to favour on-site vocational training, the employment of people experiencing difficulty in achieving integration or the fight against unemployment'.

Relationship Management

OGC says.

'There may be opportunities post-award for contracting authorities to work outside the formal procurement process, on a voluntary basis, to promote the importance of social issues such as equality and adult skills to their suppliers and supply chain. This can be an effective means of influencing suppliers’ culture, and helping to ensure that it fits with the contracting authority’s own set of values and needs'.

Outcomes Based Commissioning

A cultural shift

The move to outcomes is a cultural shift, requiring policy makers, commissioners and providers to define the outcomes that need to be achieved through public services and devise ways of measuring their achievement.

To commission for outcomes, agencies generally need to resist the temptation of being overly prescriptive in their service specification, focusing instead on the ultimate desired result and leaving it to providers to work out the details. They also need to be absolutely clear - and realistic - about the impact they want services to achieve for users. They need to think of the best way of allocating the risk of non-achievement, and of how to introduce flexibility in contracts to respond to change. Finally, commissioners must be clear (and open-minded) about what evidence will need to be collected to demonstrate achievement of the outcomes.

Public Service Delivery Network Member, Commissioner:

'the outcomes based commissioning model will only work if outcome focused / user friendly contract / performance management systems exist'.

Public Service Delivery Network Member, Frontline service provider:

'Commissioners often look down their noses at soft' outcomes. Give us the hard facts they say.  But there is nothing soft or woolly about measuring the internal changes that service users experience. Those are what really count when it comes to bringing about and sustaining lasting change'.

Providers also need to gear up for outcomes. They need to be clear on changes they want to bring about with their actions and to use the most appropriate tools to monitor and document these. Outcome monitoring can be daunting for providers, because while delivering activities is straight forward, outcomes may sometimes not be achieved. Providers need to be ready to deal with the notion of failure when outcomes are not met, and to understand the reason behind this failure, which may be due to many factors, for example an overambitious or unachievable outcome or a problem with the service design.

Making the cultural shift towards outcome-focused public services poses challenges to commissioners and providers, and it requires buy in and investment from both sides, an investment that should be ultimately rewarded by the achievement of more effective services.

Public Service Delivery Network Member, Commissioner:

'The understanding and use of outcome measurement, capture and reporting is still very under-developed across all the sectors. A culture change is needed.'