Benchmarking

Identifying the performance indicators

In public sector benchmarking, the performance indicators used tend to focus

on cost and efficiency or differentiation. Cost variables might include items

such as labour efficiency, or total costs for a particular function as a percentage of income.

 

Regarding differentiation, many of the metrics used would be of a qualitative

nature, such as client satisfaction or quality of service. It is difficult to measure

these directly due to their subjectivity. One approach is to use customer

surveys for these.

 

In attempting to find relevant metrics, benchmarking exercises carried out in

the past by similar organisations can be a useful source. Much information

about these is available in accounting and business journals, and online, or by

contacting organisations that have already performed a benchmarking exercise. This is easier in the public sector, as the government – as overseer

and beneficiary of the benchmarking – can often force other organisations to

disclose information. This would not be the case in the private sector.

 

Selecting the benchmark

When choosing the benchmark, we can talk about different types of

benchmarking:

·           Internal benchmarking uses another organisation within the same

organisation. For example, a comparison of the performance of the

procurement department of one hospital with the procurement

department of another hospital.

·           External functional benchmarking is where a particular function is

compared with that function for the organisation that performs that

function best, regardless of which industry they are in.

·           Competitive benchmarking is where a competitor is used as the

·           benchmark. This may not be so common in the public sector.

 

When using external functional benchmarking, using a similar organisation in

terms of objectives and size can make the process easier. It is not necessarythen to take into account differences between the two organisations when comparing their performance, and it should be easier to adopt the practices of the benchmark if they are similar. For example, if a school uses a similar school in a different area as a comparator.

 

External functional benchmarking can also be performed successfully using out of category organisations – ie organisations that may be have totally different objectives and even different primary activities. Using such benchmarks will make the process more complex, but may provide the opportunity for an organisation to overtake, rather than simply to catch up with comparable organisations. For example, a public sector logistics department could use one of the private sector international courier companies as a benchmark for its logistics.

 

The most important factor when selecting the benchmark is to identify the ‘best in class’ for the activity or business process being benchmarked.

 

Measuring the performance of the benchmark

Prior to starting to perform measurement, the organisation will have identified what it wants to measure (in step two), so it should already be clear what information is required. The question now is how to obtain it.

 

Much information is already in the public domain. Many organisations publish information about best practice benchmarks for particular industries. Financial reports may provide information about cost efficiency. There may be newspaper reports or analysts’ reports available for larger listed companies.

Publicly available information is a good place to start, but it is unlikely to provide all the information required for a successful benchmarking exercise.

 

Another source of information is ‘data sharing’ where other organisations are contacted, either directly and formally, or through professional conferences.

This can be supplemented by interviewing of staff at the benchmark.

 

Factors that influence the effectiveness of benchmarking

Based on some empirical research, Sandra Tillema tried to identify what

factors determine whether or not a benchmarking exercise actually leads to improved performance. A study of benchmarking carried by four Dutch water boards concluded that the performance of those water boards had not improved after the benchmarking.

 

Tillema claims that one reason for the lack of success of many public sector

benchmarking exercises is that they focus only on measuring performance

against the benchmark. They do not attempt to learn from, and adopt the practices of, the benchmark. Thus, the benchmarking is often a measuring exercise, not a learning exercise.

 

A second factor is the influence of stakeholders. A benchmarking exercise will

only lead to improvements if pressure is put on the organisation from its stakeholders to narrow the gap identified between the organisation and the

benchmark. This pressure can come from internal stakeholders, such as supervisory boards, or external stakeholders, such as users of the service. In the case of the Dutch water boards, no pressure had been exerted on the managements of the board, which is why no improvement was experienced.

 

In some situations, benchmarking may lead to economic pressure on an

organisation to improve. If poor benchmarking results are published, users of

the service may switch to alternative providers. So benchmarking can lead to

economic pressure on poorer organisations. In the case of state universities,

for example, students have a choice which university to go to, and their choice

may be influenced by published results of benchmarking.

 

Such economic pressure only works in situations where users can switch service provider. This is not always the case, and monopoly providers of services will not feel this economic pressure. Another issue is that users may not always understand the results of benchmarking, and their decisions may be based on factors other than the benchmarking exercise – for example, the student that chooses a university based on the better nightlife in the city where it is located, rather than the publicly available results of benchmarking.

 

Dysfunctional effects of benchmarking

Tillema notes that in common with all performance measurement, benchmarking may lead to dysfunctional behaviour. Management may take actions to improve their measured scores without improving underlying performance. A criticism by some of UK schools is that management focus on improving their performance in the government league tables, not on providing a good education for pupils. This is part of a general problem in performance management, which is the ‘what gets measured gets done’ concept.

 

Benchmarking can also be used to defend rather than improve poor performance. In such cases, management focus on explaining why their organisation performed poorly, citing factors that make their organisation different from its peers. In such cases, the benchmarking leads to little or no improvement in performance.

 

One factor that leads to greater levels of dysfunctional behaviour is where stakeholders misinterpret the results. For example, they do not take account of different operating environments, or different objectives of the organisation and benchmark, and this leads to unrealistic pressure to close the performance gaps.

 

Misinterpretation is less common where the results are only made available to expert stakeholders. Because the expert stakeholders have a better understanding of the performance, and the factors that may differentiate the organisation from its peers, they will be more realistic in their assessment of the results. This reduces the incentive for managers to manipulate the results.

 

League tables

League tables are commonly used in the public sector to present the results of benchmarking. Under league tables, the various metrics are summarised into a weighted average overall score. A league table is then prepared, showing all

organisations ranked according to their overall score – for example, UK schools are ranked by exam results.

 

The benefit of league tables is that many different areas of performance are

summarised into one final score, showing how well the organisation has performed overall. League tables are also designed to improve competition among the organisations, giving an incentive to the poorer performers to improve so that they move up the table.

 

Common criticisms of league tables are that they apply arbitrary weighting to

the various factors that are used in the calculation of the final score. It is also argued that they do not take into account differences between the organisations being measured. In schools, for example, one factor that affects

the performance of schools quite strongly is the demographics of the area where the schools are located, with schools in poorer areas typically appearing towards the bottom of the table.

 

Conclusion

There has been a move towards making public sector bodies more efficient and effective, using more targets and benchmarking. It is not clear whether targets have improved the performance of public sector bodies or not. Benchmarking can lead to improved performance in some public organisations. However, it is most successful where stakeholders have the ability to apply pressure to the organisations to narrow the gap between their actual performance and that of the benchmark. If not, then the benchmarking may not lead to improved performance.

 

Nick Ryan is a freelance lecturer and writer

 

References

1.    Bruder Jr, Kenneth A Gray, Edward M, ‘Public sector benchmarking: a practical approach’, published in Public Management, September 1994

2.    Sandra Tillema, ‘Public sector benchmarking and performance improvement: what is the link and can it be improved?’, published in Public Money and Management, January 2010.

3.    Noel Hyndman and Francis McGeough, ‘NPM and performance measurement: a comparative study of the public sectors in Ireland and the UK’, published in Irish Accounting Review, December 2008

4.  NHS Staff overview, published on the website, www.ic.nhs.uk

5.  Graham Prentice, Simon Burgess and Carol Propper, ‘Performance pay in the public sector: A review of the issues and evidence’. Published for the UK Office of Manpower Economics – www.ome.uk.com

6. Benchmarking Process at

 www.kaiserassociates.com/capabilities/benchmarking

                                     Page：1 2>>      See the original>>