ePapers Repository

Impact measurement practice in the UK third sector: a review of emerging evidence

Harlock, Jenny (2013) Impact measurement practice in the UK third sector: a review of emerging evidence. Working Paper. University of Birmingham, Birmingham.

[img]
Preview
PDF
WP106_Impact_measurement_practice_in_the_UK_third_sector_%2D_Harlock,_July_2013.pdf
570Kb

URL of Published Version: http://www.tsrc.ac.uk/Research/EconomicandSocialImpact/ImpactmeasurementpracticeintheUKthirdsector/tabid/1027/Default.aspx

Abstract

This paper reviews emerging evidence about how the UK third sector is measuring its performance and impact. Impact measurement has become an increasingly important activity for third sector organisations in recent years, yet impact – and how to measure it – remain contested issues in policy, research and practice. By examining what we think we know about how third sector organisations undertake impact measurement and identifying areas for further research, this paper therefore aims to contribute to the ongoing discussion and debate about whether, and how, organisations should measure their impact.
This review examines five key questions: how is impact and its measurement understood in existing research, policy and practice? what do we know about who is undertaking impact measurement in the sector? why are organisations measuring their impact? what practices and approaches organisations are using to measure impact? and key challenges and implications of impact measurement for the sector. Research on impact measurement is at an early stage, and has thus far tended to be boosterist in nature; nevertheless this review reveals a growing concern that requirements and demands for impact data from third sector funders are taking precedence over the requirements and needs of beneficiaries and service users. In this context, approaches which focus on organisational learning are being developed from within the third sector.

Type of Work:Monograph (Working Paper)
School/Faculty:Colleges (2008 onwards) > College of Social Sciences
Department:Third Sector Research Centre
References:

ACEVO (Association of Chief Executives of Voluntary Organisations) (2003) Replacing the State? The Case for Third Sector Public Service Delivery, London: ACEVO.
ACEVO, CfG, Institute of Fundraising, NCVO, NPC, Small Charities Coalition, Social Enterprise UK and SROI Network (2012) Principles of Good Impact Reporting, accessed 12/03/13, available at: http://www.cfg.org.uk/resources/Publications/~/media/Files/Resources/CFDG%20Publications/SME330%20CFDG%20Impact%20Report_v6.ashx.
Alcock, P. (2010) ‘A Strategic Unity: defining the third sector in the UK’, Voluntary Sector Review 1 (1): 5-24.
Alcock, P., Brannelly, T. and Ross, L. (2004) Formality or Flexibility? Voluntary Sector Contracting in Social Care and Health, Birmingham: University of Birmingham.
Arvidson, M. (2009) ‘Impact and evaluation in the UK third sector: reviewing literature and exploring ideas’, Third Sector Research Centre Working Paper 27, University of Birmingham.
Arvidson, M. and Lyon, F. (2013) ‘Social Impact Measurement and Non-profit Organisations: Compliance, Resistance and Promotion’, published online, Voluntas, 30 April 2013, DOI 10.1007/s11266-013-9373-6.
Arvidson, M., Lyon, F., McKay, S. and Moro, D. (2010) ‘The ambitions and challenges of SROI’, Third Sector Research Centre Working Paper 40, University of Birmingham.
Arvidson, M., Lyon, F., McKay, S. and Moro, D. (2013) ‘Valuing the Social? The nature and controversies of measuring Social Return on Investment (SROI)’, Voluntary Sector Review 4 (1): 3-18.
Audit Commission (2007) Hearts and Minds: Commissioning from the voluntary sector, London: Audit Commission
Bagnoli, L. and Megali, C. (2011) ‘Measuring Performance in Social Enterprises’, Nonprofit and Voluntary Sector Quarterly, 40 (1): 149-165.
Barman, E. (2007) ‘What is the Bottom Line for Nonprofit Organisations? A History of Measurement in the British Voluntary Sector’, Voluntas 18: 101-115.
Bhutta, M. (2005) Shared Aspirations: the Role of the Voluntary and Community Sector in Improving the Funding Relationship with Government, London: National Council for Voluntary Organisations (NCVO).
Blackmore, A. (2006) How Voluntary and Community Organisations Can Help Transform Public Services, London: NCVO.
Breckell, P., Harrison, K., Robert, N. (2010) Impact Reporting in the UK Charity Sector, CFDG/Cass Business School, accessed 15/02/13, available from:
http://www.cfg.org.uk/resources/~/media/Files/Resources/Impact%20Reporting%20in%20the%20UK%20Charity%20Sector.ashx.
Bull, M. (2007) ‘Balance: The Development of a Social Enterprise Business Performance Tool’, Social Enterprise Journal 3 (1): 49-66.
Cabinet Office (2006) A Summary Guide: Guidance to Funders, London: HM Treasury.
Cabinet Office (2007) The Future Role of the Third Sector in Social and Economic Regeneration: Final Report, London: HM Treasury.
Cabinet Office (2010a) Modernising Commissioning, Increasing the role of charities, social enterprises, mutuals and cooperatives in public service delivery, London: The Stationery Office.
Cabinet Office (2010b) Exposure of the voluntary, community and social enterprise sector to cuts in public funding, London: Cabinet Office.
Cabinet Office (2010c) Better Together: preparing for local spending cuts to the voluntary, community and social enterprise sector, London: Cabinet Office.
Cairns, B., Harris, M. and Hutchinson, R. (2006) Servants of the Community or Agents of Government? The Role of Community-based Organisations and their Contribution to Public Services Delivery and Civil Renewal, London: Institute for Voluntary Action Research/BASAAC.
Carmel, E. and Harlock, J. (2008) ‘Instituting the Third Sector as a Governable Terrain: Partnership, Procurement and Performance in the UK’, Policy and Politics 36 (2): 152-71.
CfE (Coalition for Efficiency) (2012) The Practical Approach: A Handbook on How Skilled Volunteers Can Help Charities Measure Their Impact, London: CfE.
CFG (Charity Finance Group), ACEVO and NPC (2012) Principles into Practice: How charities and social enterprise communicate impact, accessed 13/03/13, available at:
http://www.cfg.org.uk/resources/Publications/~/media/Files/Resources/CFDG%20Publications/CFG%20Impact%20Report%20hi-res%20final.ashx.
Chapman, T., Robinson, F., Brown, J., Crow, R., Bell, V. and Bailey, E. (2010a) What makes third sector organisations tick? Interactions of foresight, enterprise, capability and impact, Newcastle: Northern Rock Foundation.
Chapman, T., van der Graaf, P., Bell, V., Robinson, F. and Crow, R. (2010b) Keeping the Show on the Road: a survey of dynamics and change amongst third sector organisations in North East England and Cumbria, Newcastle: Northern Rock Foundation.
Chapman, T., Bell, V. and Robinson F. (2012) Measuring Impact: easy to say, hard to do, Newcastle-upon-Tyne, Northern Rock Foundation.
Clarke, J., Kane, D., Wilding, K. and Bass, P. (2012) The UK Civil Society Almanac 2012, London: NCVO.
Dacombe, R. (2011) ‘Can we argue against it? Performance management and state funding of voluntary organizations in the UK’, Public Money and Management 31 (3): 159-66.
Department of Health (2010) Measuring Social Value – How Five Social Enterprises Did It, London: the Stationery Office.
Ebrahim, A. (2002) ‘The role of information in the reproduction of NGO-funder relationships’, Nonprofit and Voluntary Sector Quarterly 31 (1): 84-114.
Ellis, J. (2009) ‘Monitoring and Evaluation in the third sector: meeting accountability and learning needs’, Paper for the 15th NCVO/VSSN Researching the Voluntary Sector Conference 2009.
Ellis, J. and Gregory, T. (2008) Developing monitoring and evaluation in the third sector: Research report, London: Charities Evaluation Service (CES).
Fitzhugh, H. and Stevenson, N. (2009) Making a difference: an introduction to social impact measurement, Social Enterprise East of England, accessed 13/03/13, available from:
http://the-guild.co.uk/newsite/content/2009_Measuring_making_a_difference_SEEE.pdf.
Flockhart, A. (2005) ‘The Use of Social Return on Investment (SROI) and Investment Ready Tools (IRT) to Bridge the Financial Credibility Gap’, Social Enterprise Journal 1 (1): 29-42.
Francis, J. and Byford, S. (2011) SCIE’s approach to economic evaluation in social care, London: SCIE.
Gibbon, J. and Dey, C. (2011) ‘Developments in Social Impact Measurement in the Third Sector: Scaling Up or Dumbing Down?’, Social and Environmental Accountability Journal 31 (1): 63-72.
Hall, M. (2012) ‘Evaluation logics in the third sector’, Voluntas: international journal of voluntary and nonprofit organisations, published online April 2013, doi:10.1007/s11266-012-9339-0.
Heady, L. and Keen, S. (2010) Social Return on Investment for funders, London: New Philanthropy Capital, accessed 12/03/13, http://www.thinknpc.org/publications/social-return-on-investment/.
Heady, L. and Rowley, S. (2008) Turning the tables: Putting Scottish charities in control of reporting, London: NPC.
Hind, A. (2011) ‘New development: Increasing public trust and confidence in charities: on the side of angels’, Public Money and Management 31 (3): 201-205.
H.M. Government (2011) Open Public Services White Paper, London: The Stationery Office.
Hyndman, N. and Jones, R. (2011) ‘Editorial: Good governance in charities – some key issues’ in Public Money and Management 31 (1): 151-155.
Kendall, J. and Knapp, M. (1996) The Voluntary Sector in the UK, Manchester: Manchester University Press.
Kendall, J. and Knapp, M. (2000) ‘Measuring the performance of voluntary organizations’, Public Management: an international journal of research and theory 2 (1): 105-132.
Leat, D. (2006) 'Grantmaking foundations and performance measurement: Playing pool?', Public Policy and Administration 21: 25–37.
Lumley, T., Rickey, B. and Pike, M. (2011) Inspiring impact: Working together for a bigger impact in the UK social sector, London: NPC.
Lyon, F. (2010) ‘Measuring the value of social and community impact : the role of social enterprises in public services’ in P. Hunter (ed.) Social enterprise for public service: how does the third sector deliver? London: The Smith Institute.
Lyon, F. and Arvidson, M. (2011) ‘Social impact measurement as an entrepreneurial process’, Third Sector Research Centre Briefing Paper 66, University of Birmingham.
Lyon, F., Arvidson, M., Etherington, D., and Vickers, I. (2010) Social impact measurement (SIM) experiences and future directions for third sector organisations in the East of England, available at:
http://the-guild.co.uk/newsite/content/2010_Joining_the_Dots_Strand_2_Long_Report_EEDA.pdf.
Macmillan, R. (2010) ‘The third sector delivering public services: an evidence review’, Third Sector Research Centre Working Paper 20, University of Birmingham.
Macmillan, R. (2012) ‘Distinction’ in the third sector’, Third Sector Research Centre Working Paper 89, University of Birmingham.
Martin, S., Downe, J., Grace, C., Nutley, S. (2010) ‘Validity, Utilization and Evidence-Based Policy: The Development and Impact of Performance Improvement Regimes in Local Public Services’ Evaluation, 16 (1): 31-42.
Millar, R. and Hall, K. (2012) ‘Social Return on Investment (SROI) and Performance Measurement: The opportunities and barriers for social enterprises in health and social care’, Public Management Review: 1-19. Published online, June 2012, DOI:10.1080/14719037.2012.698857.
Moxham, C. and Boaden, R. (2007) ‘The impact of performance measurement in the voluntary sector’, International Journal of Operations and Production Management, 27 (8): 826-845.
Mullins, D., van Bortel, G., Ham, Y.-J. and Watson, C. (2010) Social investment performance
management toolkit for housing organisations. Birmingham: University of Birmingham.
National Audit Office (NAO) (2009) Intelligent Monitoring: An element of Financial Relationships with Third Sector Organisations, London: NAO.
National Council for Voluntary Organisations (NCVO) (2013) Inspiring Impact: The Code of Good Impact Practice, Draft for consultation, London: NCVO.
New Economics Foundation (Nef) (2007) Measuring Real Value, A DIY guide to social return on investment, London: Nef.
Nicholls, A. (2009) ‘‘We do good things don’t we?’: Blended Value Accounting in Social Entrepreneurship’, Accounting, Organizations and Society, 34 (6-7): 755-769.
Nicholls, J., Lawlor, E., Neitzert, E. and Goodspeed, T. (2009) A guide to social return on investment, London: The Cabinet Office.
NPC (New Philanthropy Capital) (2010) Talking about Results, London: NPC.
Ogain, E. N., Lumley, T. and Pritchard, D. (2012) Making an Impact, London: NPC.
Polonsky, M. and Grau, S. L. (2010) ‘Assessing the social impact of charitable organizations – four alternative approaches’, International Journal of Nonprofit and Voluntary Sector Marketing 16: 195-211.
Rickey, B., Lumley, T. and Ogain, E. (2011) A Journey to Greater Impact. Six charities that learned to measure better, London: New Philanthropy Capital.
Rinaldo, H. (2010) Getting Started in Social Impact Measurement: A guide to choosing how to measure social impact, Norwich: The Guild.
Ryan, P. and Lyne, I. (2008) 'Social enterprise and the measurement of social value: methodological issues with the calculation and application of the social return on investment', Education, Knowledge and Economy 2 (3): 223–37.
Sanderson, I. (2002) ‘Evaluation, Policy Learning and Evidence-Based Policy Making’, Public Administration, 80 (1): 1-22.
Sheridan, K. (2011) ‘Measuring the impact of social enterprises’, British Journal of Healthcare Management, 17 (4): 152-156.
Stevenson, N., Taylor, M., Lyon, F., Rigby, M. (2010) Joining the Dots, Social Impact Measurement, commissioning from the Third Sector and supporting Social Enterprise development, Norwich: The Guild.

Substance (2012) List of measurement tools and systems, accessed 23/04/13, available at:
http://inspiringimpact.org/our-plan/data-tools-and-systems/.
Teasdale, S. (2010) ‘What's in a name? The construction of social enterprise’, Third Sector Research Centre Working Paper 46, University of Birmingham.
Teasdale, S., Alcock, P. and Smith, G. (2012) ‘Legislating for the big society? The case of the Public Services (Social Value) Bill’, Public Money and Management, 32 (3): 201-208.
Wadia, A. and Parkinson, D. (2011) Outcome and Outcome Indicator Banks, Availability and Use, London: Charities Evaluation Service.
Westall, A. (2009) ‘Value and the third sector: Working Paper on ideas for future research’, Third Sector Research Centre Working Paper 25, University of Birmingham.
Wilkes, V. and Mullins, D. (2012) ‘Community investment by social housing organisations: measuring the impact’, Third Sector Research Centre Survey Report for HACT, University of Birmingham.
Wimbush, E. (2011) ‘Implementing an outcomes approach to public management and accountability in the UK – are we learning the lessons?’ Public Money and Management, 31 (3): 211-218.
Wood, C. and Leighton, D. 2010 Measuring Social Value: The gap between policy and practice London: Demos.

Date:01 July 2013
Series/Collection Name:TSRC Working Paper Series
Keywords:Impact, performance, social impact, social value, third sector.
Subjects:H Social Sciences > H Social Sciences (General)
Related URLs:
URLURL Type
http://tsrc.ac.ukOrganisation
Funders:Economic and Social Research Council, Office for Civil Society, Barrow Cadbury Trust
Copyright Holders:Third Sector Research Centre
ID Code:1800

Export Reference As : ASCII + BibTeX + Dublin Core + EndNote + HTML + METS + MODS + OpenURL Object + Reference Manager + Refer + RefWorks
Share this item :
QR Code for this page

Repository Staff Only: item control page