Benchmarking at the University of Adelaide
What is Benchmarking?
Benchmarking is a means of comparing the University's performance or standards, or both, with those of its peers. It is a means by which the University can monitor its relative performance, identify gaps, seek fresh approaches to bring about improvements, set goals, establish priorities for change and resource allocation, and follow through with change processes based on empirical evidence. It can be about broad University-wide issues or specific matters affecting only one area; it can be strategic (addressing priority issues) or cyclical (addressing a number of areas on a regular basis) or ad hoc (taking advantage of an opportunity). Benchmarking projects can be as simple as doing a desktop survey of relevant websites or may involve a formal request for information and/or an agreement with another institution.
Whatever its scope or subject matter, benchmarking is an important element of the University's quality assurance cycle.
Essentially, benchmarking attempts to answer the following questions:
- How do the standards we have set ourselves compare to our peers?
- How does our performance measure against the outcomes of national and comparator institutions?
- How can we adapt good practice examples from other institutions to our own organisation?
Benchmarking allows the University to:
- identify and monitor standards and performance in order to improve University outcomes, processes and practices
- discover new ideas for achieving the University's 'core objectives' as outlined in its Strategic Plan
- provide an evidence-based framework for change and improvement.
- inform planning and goal setting
- improve decision-making through referencing comparative data
- bring an external focus to internal activities
The University's Learning Committee has endorsed the following priority topics for University-wide benchmarking:
- Cohort reporting
- Assessment monitoring
- Graduate attributes
The University encourages benchmarking with comparator institutions within Australia and overseas as a method of improving performance and assuring standards. While there is no prescribed methodology for conducting benchmarking exercises, the University expects staff to comply with the following benchmarking principles and code of conduct.
Benchmarking projects undertaken by Faculties and Divisions of the University will:
- support the University's mission, values and strategic priorities
- be characterised by a commitment to: learning from best practice; implementation of potential improvements arising from the findings of benchmarking projects; and sharing of good practices once projects are completed
- be balanced in terms of the value received compared to costs involved in undertaking the projects
- have the approval of the relevant manager or unit head.
Code of Conduct
The following should be taken into account when undertaking benchmarking projects where a request for information is involved:
- Confidentiality: All benchmarking exchanges should be treated as confidential. Publication and external communication of findings should not proceed without the permission of all partners involved in the project.
- Use: Benchmarking information should not be used for other than the express purpose for which it was obtained without prior consent of all participating partners.
- Exchange: The type or level of information exchanged should be comparable between the benchmarking partners.
- Agreement: If a benchmarking agreement is entered into, issues about confidentiality, use and the type and level of information to be exchanged should be included in the agreement.
Authorities and Responsibilities
Approvals and management responsibilities should be assigned in accordance with the normal University organisation structure and reporting relationships, and routine management practices must apply to any significant project undertaken.
- The Benchmarking Reports Repository (staff only access) must be checked prior to commencing a project to ascertain whether similar projects have or are already being undertaken.
- Benchmarking projects involving a formal request for information from another institution must be approved by the relevant manager or unit head. This can be a Head of School, a General Manager/Director, an Executive Dean, a Deputy Vice-Chancellor or Vice-President, or, in the case of any large-scale University-wide benchmarking, may be the Vice-Chancellor.
- If the scope of the project affects more than one area, then consultation and agreement between the areas impacted are essential prior to the project’s commencement.
- Special care must be undertaken when projects require that the University's corporate data be shared with other institutions: in this case the relevant data custodian must be contacted and it will be his/her responsibility to ensure that appropriate approvals for the data transfer are obtained from senior management.
- The responsibility for running the project will be with the unit manager under whose authority the project was approved or his/her delegate.
- Contact with partner organisations will normally be through the unit manager taking responsibility for the project, unless delegated by him/her. When institutional support is needed for a project to proceed with particular partners, then the responsible manager must contact the relevant Deputy Vice-Chancellor or Vice-President.
- Written agreements with other institutions and organisations with which projects are undertaken must be entered into in line with the Contract Management framework and signed in accordance with the formal delegations of the University.
- The University expects that benchmarking projects will be funded by the area that initiates, manages and accepts responsibility for the project. If central funds are required such as might be the case for large projects, then a submission must be made through the planning and budgeting process. If a case for special funding is appropriate and the timeframe does not permit submissions through the planning and budgeting process, then the unit manager must approach the relevant Deputy Vice-Chancellor or Vice-President in the first instance.
- When confidentiality considerations allow, benchmarking project reports should be lodged with the Learning and Quality Support Unit as soon as possible after the completion of the project so that they can be recorded in the Benchmarking Reports Repository (staff only access). The repository has Adelaide-only access.
- Each year, a summary report on the benchmarking projects undertaken by the Faculties must be submitted to the Deputy Vice-Chancellor and Vice-President (Academic) as part of the Faculty Performance Report.
Integration with QA Systems
Benchmarking projects are most successful when they are integrated with other initiatives and processes designed to improve outcomes within the University. To this end, the University expects that projects, findings and implementation plans will be embedded into Divisional and Faculty operational and business plans.
Overall authority for the maintenance of the Benchmarking Framework rests with the Deputy Vice-Chancellor and Vice-President (Academic). However management accountability for the framework is with the Pro-Vice Chancellor (Student Learning). Responsibility for all operational tasks related to the framework rests with Learning and Quality Support.
The University encourages benchmarking not only with comparator institutions within Australia but also with institutions overseas.
Selection Criteria for International Benchmarking Partners
The current criteria for selection of the University's international benchmarking partners are as follows:
- Universities with which the University of Adelaide has a Memorandum of Understanding (MOU) or other agreement, preferably including a reference to benchmarking
- Universities which have English as a primary language
- Universities which are research intensive
- Universities which are comprehensive, preferably including medicine
- Universities which are of comparable size to the University of Adelaide
International Benchmarking Partners
In July 2008, the University of Adelaide and the University of Canterbury (New Zealand) entered into a Memorandum of Understanding to facilitate collaboration between the two universities. This included agreement to engage in an ongoing benchmarking relationship and an initial series of benchmarking projects were agreed.
The University sees the relationship that has been developed with the University of Canterbury as a pilot of an international benchmarking approach for the University, with the emphasis being a qualitative one. A series of projects have been initiated around:
- student experience,
- performance reporting,
- sustainability, and
- Faculty of Sciences assessment practices.
Regular management meetings are held via videoconferencing. The Vice-Chancellor, the then Pro Vice-Chancellor (Learning & Quality) and Director, Strategy and Planning visited the University of Canterbury from 6-10 July 2009 to further progress benchmarking activities. This was followed by a site visit to Adelaide in November.
These projects are seen as pilots, providing a template for application to collaborations with other disciplines and other universities and are all ongoing, with continuing communications between relevant staff.
During 2011, the Quality Enhancement Committee agreed to a broad approach which includes active initiation of informal benchmarking opportunities and reporting best practice, and the development of specific benchmarking guidelines to facilitate an increase in Faculty and School benchmarking initiatives. It was also agreed that a second benchmarking partnership be initiated with one of the Universities identified by the Quality Enhancement Committee for this purpose.
International Benchmarking Measures
The tables below show the indicators, measures and associated data definitions for research, learning and teaching, and financial performance.
|Research||Refereed journal articles per Academic Staff FTE per broad discipline||The number of refereed journal articles per Academic Staff full-time equivalent, broken down by the following fields (these fields are used in the Shanghai Jiao Tong University (SJTU) rankings):
|Research Activity||Ratio of research revenue to total||Income from research activities as a % of total activities (excluding controlled entities).|
|Research Training Activity||Ratio of the number of higher degree by research students to total||Number of higher degree by research (PhD and Masters) enrolments as a % of total higher education level student enrolment numbers.|
Learning and Teaching
|Graduate Satisfaction||% of graduates satisfied with their course experience||Percentage of graduates satisfied with their course experience overall (equivalent for each partner of the Graduate Careers Australia Course Experience Questionnaire).|
|Employer Satisfaction||% of graduates employed||Percentage of graduates employed of those available for work in the year after graduation (equivalent for each partner of the Graduate Careers Australia Graduate Destination Survey).|
|Learning Performance||Undergraduate retention rates||The number of undergraduate students retained as a % of the total from the previous year less the students that have completed.|
|Learning Performance||Undergraduate progress rates||The number of undergraduate course enrolments with a grade of pass or more as a % of the total undergraduate course enrolment.|
|Research and Teaching Support||% of expenditure on the library collection||Expenditure on the library collection (excluding salaries and infrastructure) as a percentage of total expenditure, excluding controlled entities.|
|Salary Expenditure||% of expenditure on all salaries||Expenditure on all salaries as a percentage of total expenditure, excluding controlled entities.|
|Government Support||% of revenue from Government sources||Income from all Government sources as a percentage of total expenditure, excluding controlled entities.|
|Financial Health||Operating Margin||Operating result as a percentage of total operating revenue, excluding controlled entities.|
|Financial Health||Current ratio||Ratio of the value of current assets to current liabilities, excluding controlled entities.|
|Knowledge Transfer||% of revenue from Intellectual Property (IP)||The value of revenue generated from IP (including licences and patents), including controlled entities.|
Although the University does not prescribe any particular approach to benchmarking, Faculties and Divisions must be aware of the management arrangements that apply as described in Managing Benchmarking.
The benchmarking process suggested by the University follows the University's Quality Assurance Planning and Budgeting, Implementation, Review and Improvement (PIRI) system.
Types of Benchmarking
The University employs a number of different types of benchmarking to support its goal of continuous improvement:
- Strategic Benchmarking
Used to improve overall performance by examining the long-term strategies and general approaches of institutions that have succeeded in areas of strategic priority for the University.
- Performance Benchmarking
Used to compare and monitor the performance of the University with its peers using a range of metrics including financial, research, and learning and teaching performance indicators.
- Functional Benchmarking
Used to compare and improve functional areas in the organisation such as Human Resources or Finance.
- Process Benchmarking
Used when the focus is on improving specific critical processes and operations. Benchmarking partners are sought from best practice organisations that perform similar work or deliver similar service.
- Internal Benchmarking
Involves comparing practices and processes with other units in the University. The advantage of internal benchmarking is that access to sensitive data and information is easier; standardised data is often readily available; and usually less time and resources are needed. There may be relatively few barriers to implementation as practices may be relatively easy to transfer across the same organisation.
- International Benchmarking
Involves strategic, performance, functional and process benchmarking with comparator institutions overseas. International benchmarking widens the University's focus and helps to ensure international competitiveness.
- Quantitative Benchmarking
Looks at quantifiable outputs of an operation. The benchmarks are hard measures. Measurement is critical to help the University monitor its current performance relative to that of best practice institutions.
- Qualitative Benchmarking
Looks at the systems and processes that deliver the results. The qualitative benchmarks are generally attributes of best practices in a functional area and these benchmarks could be simply a checklist of essential attributes constituting best practice.
Selecting Benchmarking Partners
Selecting appropriate benchmarking partners is essential for successful strategic and performance benchmarking. The University usually undertakes this type of benchmarking with other Go8 universities. A benchmarking partner should:
- have a compatible mission, values and objectives
- be of comparable size
- be a research intensive university
- have a similar discipline mix
- have superior performance in the areas to be benchmarked, and
- have a commitment to quality management and a 'willingness to share'.
In addition to the above, international benchmarking partners should:
- have a Memorandum of Understanding (MOU) with the University or other agreement, which includes reference to benchmarking, and
- have English as the primary language.
For functional and process benchmarking, choose partners with best practice in the areas or processes to be compared.
Membership of benchmarking consortia is another avenue for comparing performance and outcomes with other institutions.
The area that initiates and manages the project will be responsible for providing the resources for the project. The following resources are normally needed for a benchmarking project:
- Staff time
An investment of staff time is required in collecting, analysing and reporting benchmarking data.
- Logistical Costs
Some benchmarking projects may require site visits or participation in benchmarking partners’ forums.
- Implementation Costs
The implementation of recommendations arising in the final stage of the benchmarking project may incur costs.
The value of benchmarking is considerably enhanced if the findings are shared with similar or related units within the University. The University encourages the production of benchmarking reports that can be included in the Benchmarking Reports Repository (staff only access) for the benefit of other sections of the University. This includes externally produced consortia reports.
Benchmarking reports will vary depending on the size and complexity of the exercise. They may include:
- a gap analysis
- a discussion of best practice examples
- recommendations for the adaptation of initiatives to the University of Adelaide context
- a cost/benefit analysis.
Progress towards implementing improvements based on benchmarking projects, and their effect on outcomes, should also be shared and areas are encouraged to submit and discuss progress with similar units, and with University committees and bodies, such as the University Research Committee, the University Learning Committee and the Internationalisation Strategy Committee.
Ideally, implementation plans should be prepared to operationalise recommendations arising from benchmarking reports. Progress against these plans should be regularly monitored.
Benchmarking reports should be lodged with Learning and Quality Support by the areas undertaking benchmarking projects.
The Checklist covers the key activities of benchmarking:
|Key Activities||Tasks Completed (✓)|
|Project Selection (Identify what is to be benchmarked)|
|Form an Internal Benchmarking Team|
|Select the Benchmarking Partners (Consider the necessary protocols required such as confidentiality arrangement, agreements, code of practice, etc.)|
|Finalise Benchmarks (Measures & indicators)|
|Analyse Data (Determine performance gaps, reasons for gaps, cost/adaptation benefit analysis)|
|Communicate Findings (Gain acceptance from management and area staff)|
|Set Functional Targets (Implement specific improvement actions)|
|Prepare a Monitor Progress Plan (Include responsibilities and deadlines)|
Useful Data Sources & References
|Type of Data||Source|
|Group of Eight|
||Go8 Executive Dashboard|
|Department of Education|
||Selected Higher Education Statistics Series|
|Allocation of Government Funds
||Office for Learning & Teaching|
|Council of Australian University Librarians (CAUL)|
Online Resources and University of Adelaide Library Collection:
- Alstete, JW 1996, Benchmarking in higher education: adapting best practices to improve quality, ASHE-ERIC Higher Education Report, No 1995.5, Washington, USA.
- American Productivity and Quality Centre 2014, The Benchmarking Code of Conduct, viewed 4 July 2014, <http://www.apqc.org/knowledge-base/documents/benchmarking-code-conduct>
- Australasian Council on Open Distance and E-Learning 2014, ACODE - Leading in Technology Enhanced Learning and Teaching - Benchmarks for Technology Enhanced Learning, viewed 4 July 2014, <http://www.acode.edu.au/pluginfile.php/579/mod_resource/content/3/TEL_Benchmarks.pdf>
- Australian Universities Quality Agency 2014, AUQA Good Practice Database, viewed 4 July 2014, <http://www.auqa.edu.au/gp/about/>
- Bender, BE & Schuh JH c2002, ‘Using benchmarking to inform practice in higher education’, New Directions for Higher Education, No 118, Jossey-Bass, San Francisco.
- CHEMS 1998, Benchmarking in Higher Education: An International Review, The Association of Commonwealth Universities, United Kingdom viewed 4 July 2014, <https://www.acu.ac.uk/publications/?pageID=48>
- Department of Education Science & Training 2002, Best Practice Processes for University Research Commercialisation Final Report, viewed 4 July 2014, <http://www.howardpartners.com.au/publications/best-practice-processes.pdf>
- Dudden, RF c2007, Using benchmarking, needs assessment, quality improvement, outcome measurement, and library standards: a how-to-do-it manual with CD-ROM, Neal-Schuman Publishers, New York, NY.
- Evans, A c1994, Benchmarking: taking your organisation towards best practice, Business Library, Melbourne.
- Fielden, J 1997, Benchmarking University Performance, CHEMS Paper No 19, The Association of Commonwealth Universities, United Kingdom, viewed 4 July 2014, <https://www.acu.ac.uk/publications/?pageID=48>
- Jackson, N & Lund, HS 2000, Benchmarking for Higher Education, Society for Research into Higher Education & Open University Press, Buckingham.
- Lidbury, C 1997, Benchmarking, Evaluation and Strategic Management in the Public Sector, OECD Working Papers No 5:67, Organisation for Economic Co-operation and Development, Paris, France.
- Matters, M & Evans, A 1996, The Nuts and Bolts of Benchmarking, Benchmarking Link-Up Australia.
- McGuire, L 2002, ‘Counting performance or performance that counts?: benchmarking government services in Australia’, Working Paper Series (Monash University, Department of Management 5/02), Monash University, Caulfield East, Victoria, viewed 4 July 2014 <http://arrow.monash.edu.au/vital/access/manager/Repository/monash:62915?sort=bs_metadata.peerreviewed%2F>
- McKinnon, KR 2000, Benchmarking: a manual for Australian universities, Department of Education, Training and Youth Affairs, Higher Education Division, Canberra.
- Meade, PH 1997, A Guide to Benchmarking, University of Otago, Dunedin.
- NIES 1995, Benchmarking self help manual: your organisation’s guide to achieving best practices (2nd Ed), Australian Government Publishing Service, Canberra.
- Raa, TT 2009, The Economics of Benchmarking: measuring performance for competitive advantage, Palgrave Macmillan, New York.
- Schofield, A 1998, Benchmarking in higher education: an international review, Commonwealth Higher Education Management Service, UNESCO, London.
- Secolsky, C & Denison, DB 2012, Handbook on measurement, assessment, and evaluation in higher education, Routledge, New York.
- Stella, A & Woodhouse, D 2007, Benchmarking in Australian Higher Education: a thematic analysis of AUQA Audit Reports, Australian Universities Quality Agency.
- Zairi, M 1996, Benchmarking for Best Practice: continuous learning through sustainable innovation, Butterworth Heinemann, Oxford, Boston.