Working paper

Managing for Outcomes in the New Zealand Public Management System (WP 04/15)

Author: Anna-Luis Cook

Abstract#

Managing for Outcomes is the process by which central agencies and Ministers are seeking to promote a results focus within the New Zealand public management system. This paper argues that if a results focus is to truly be introduced to the New Zealand public management system then all aspects of the wider system will need to be amended, in order to support a general cultural change. In promoting this view the paper provides a summary of the system as it currently stands and considers how the structural, financial, strategic and performance management elements of the wider public management system could be amended to support an outcomes focus within the core New Zealand public service.

Acknowledgements#

This paper summarises a thesis prepared in partial fulfilment of the requirements for the Masters of Public Management degree from Victoria University of Wellington.

In completing that thesis I need to acknowledge the support and assistance I received from my colleagues in the New Zealand Treasury. I also wish to thank Charles Brougham, Peter Hughes, Chris Martin, Alex Matheson, Brett Mudgway, Richard Mulgan, John Raine, Derek Russell, Geoff Sadler, Jonathan Slater and Judith Smith for willingly sharing their thoughts and experiences with me.

Thanks also to Bill Ryan, Graham Scott and Tom Berthold for their comments on the initial thesis.

Disclaimer#

The views, opinions, findings, and conclusions or recommendations expressed in this Working Paper are strictly those of the author(s). They do not necessarily reflect the views of the New Zealand Treasury.  The Treasury takes no responsibility for any errors or omissions in, or for the correctness of, the information contained in these working papers. The paper is presented not as policy, but with a view to inform and stimulate wider debate.

1  Introduction#

Building a picture of an effective public management system is akin to developing and then building a jigsaw. A number of disparate and ill-shaped component parts need to be placed in appropriate settings to create a unified and recognisable image.

Developing an effective results-oriented public management system is even more difficult than the analogy suggests, because the image has to be more than an illusion. It has to be a workable model that merges history, goals and culture together to create an environment within which the needs and desires of a wide range of stakeholders are efficiently and effectively realised.

In seeking to develop an effective results-oriented public management system it is important therefore to not only paint the perfect picture, the hearts and minds of the systems’ designers and managers, in addition to those who develop and implement policy, also need to be captured or at least engaged. For it is only by changing the culture of the environment that the orientation of the public service will truly be aligned with any image created by the architects of that system.

This paper argues that an outcomes-oriented culture change across the New Zealand public management system has to be supported by structural and systemic change, in some instances moving away from “old” responses to the challenges posed by new institutional economic theories. It posits that whilst the challenges identified by these theories are still relevant, an effective outcomes-oriented public management system will respond to them through the use of a variety of levers. In some instances these will reflect market or contractual mechanisms, but in other instances softer – community-focused – mechanisms could prove to be more effective.

In other words, in shifting the New Zealand public management system towards an outcomes-oriented environment there would appear to be a need to refresh the current tenets of the system, whilst retaining those aspects of the system that lend it strength. The process of managing this change will require some level of understanding about what that environment will look like in the future, and preferably a common theoretical framework against which system changes can be tested.

The paper is based on a premise that the New Zealand public management system, and its component parts, is an institution that must be accountable to the people, through Parliament, and the services delivered through government agents[1] should be in line with achieving the objectives of the Government of the day. At the same time, it is recognised that the systems that provide for accountability must be sufficiently flexible to allow for innovation in the way agencies seek to deliver specified services.

Finally, the paper attempts to paint a picture of what the New Zealand model might look like in a managing for outcomes environment. This picture is not meant to be a prescriptive image, rather it has been provided with the goal of promoting debate around one concrete description of a potential future model. It suggests that shifting the culture of the New Zealand public management system to incorporate an increased focus on outcomes should be supported by both sending consistent messages about the need for an outcomes focus and mechanistic changes such as those identified below.

  • The structures of the system need to support an outcomes focus in strategic management and coordination of policies and service delivery. This could be achieved through reassessment of ministerial structures and agency relationships.
  • Regular strategic reviews should provide for the articulation of outcomes an agency or a sector will be working towards; identification of the capability required to deliver specified interventions; and the resourcing required to support the agency or sector in seeking to achieve the articulated outcomes.
  • Two related, but separate, feedback loops are required to assess performance. The first should support the accountability requirements of the system and the second should support the development of a learning environment across the public sector. The two loops should inform, but not drive, each other.
  • Accountabilities within the system should reflect the decision-rights provided to individual agents, and be supported by a culture that promotes responsibility for areas which agents are able to influence.
  • Financial management systems should provide managers with the flexibility to deliver interventions identified as most appropriate in relation to specific outcomes, whilst retaining sufficient transparency of activities for effective Parliamentary oversight.
  • Leadership, by Ministers, senior public servants and those in influencing roles, should model, promote and support the behavioural changes required to give these mechanistic changes life.

Notes

  • [1]By which I mean core public service departments, agencies within the ambit of the wider State sector, and those private sector or non-governmental agencies contracted to deliver services.

2  The New Zealand model#

The New Zealand public management system, as we currently know it, was introduced in the late 1980s, with the passing of the State Sector Act 1988 and the Public Finance Act 1989. Based upon theories of new public management prevalent at the time the “New Zealand model” has been studied by a number of commentators and other jurisdictions.

Overall commentators and reviewers have concluded that the New Zealand public management system provides a sound basis for the efficient management of resources, and that the reform process resolved many of the issues identified at the time. As with any change process, however, there are always aspects of reform that are not fully implemented and, over time, other (sometimes related) issues are identified as requiring resolution. In the New Zealand context, these aspects of the State sector reforms are consistently identified as needing to provide an increased focus on outcomes and consideration of capability issues.

2.1  Where have we come from?#

The mid to late 1980s in New Zealand can best be described as a period of change – primarily driven by the perceived need to reshape the economy, but also driven by shifts in ideological perceptions about how society should be structured and New Zealand’s role in the world (James, 1992). Similarly, the economic imperatives of the time have been identified as a key driver for change of the public management system. However, a shift in perspectives about the role of Government in the economy and a desire to make the State sector more responsive to Ministerial demands also appear to have been important drivers for change (Schick, 1996; and Scott, 2001).

These latter two points largely mirrored international trends and thinking about the structures and systems of public management – with the development of new public management theories in many “like” countries (for example, the United Kingdom, the United States, Canada and Australia). In essence, new public management theories sought to develop systems that would be more efficient and effective in the way they delivered services for citizens.

Largely based on economics, the theoretical basis for the reforms identified various “challenges” that needed to be addressed in order for public management systems to be efficient and effective. As Boston et al (1996) outline, these theories provided suggestions about ways in which the risks posed by self-maximising behaviour on the part of stakeholders within the system could best be addressed. Possible responses explored in New Zealand included: separation of functions between different agencies; the use of formal “contracts” to govern relationships between different stakeholders; and delegation of managerial responsibilities to those best placed to respond to individual issues.

In New Zealand, underpinning the responses to the challenges posited by institutional economic theories were five key principles first expounded in Government Management (Treasury, 1987) - see Figure 1. The key shifts in the public management system were signalled by the implementation of two Acts: the State Sector Act 1988 and the Public Finance Act 1989. These legislative changes provided for the development and implementation of a system based on performance management (clear specification of objectives, freedom to manage and accountability), supported by institutional design issues focused on the separation of activities.

Figure 1 – Core principles underpinning the New Zealand public management system

  • Clarity of objectives – The initial element of a management process must be as clear a specification as possible of the objectives which managers are responsible for achieving. This implies both a clear identification of individual objectives – objective performance targets in areas where that is possible, for example – and the avoidance of multiple, conflicting objectives.
  • Freedom to manage – Once objectives are clearly stated, managers must be given the power to make their achievement possible. Managers should, for example, have freedom to make resource allocation decisions on a basis which enables the most efficient attainment of objectives. Controls over inputs are in general likely to prevent such a process.
  • Accountability – Freedom to manage is not itself a sufficient precondition for good management. Incentives and sanctions must be in place to modify the behaviour of managers to ensure that they do act to meet established objectives rather than pursuing independent goals of their own. For this to be achieved, managers must be accountable for the decisions they make, and those on whose behalf they act must have the means to make that accountability “stick’.
  • Effective assessment of performance – If managers are to be accountable for their performance, those to whom they are accountable must have the means to establish the quality of that performance. How well have managers met stated objectives? Are any deficiencies the result of poor management or of external factors over which the managers concerned had no control?
  • Adequate Information Flows – If emphasis is to be placed on performance assessment, a sufficient quantity and quality of information concerning performance will be required.  If managers are to be given a goal of efficient resource use, for example, accounting systems must provide the information to enable an adequate assessment of the quality of resource decisions.

More specifically, the new model provided for an emphasis on setting clear, non-conflicting goals; giving authority to managers and boards to manage their businesses; and simplifying accountability arrangements (Scott, 2001). Implementation of the principles expounded provided for: devolution of control over inputs, leading to significant change in the human resource and financial management systems; the introduction of a performance management system designed to hold chief executives and departments accountable for specified activities and behaviours; and structural change to support clear specification of objectives and to allow for competition where appropriate, in most instances this led to the structural division of policy development and operational delivery[2].

Notes

  • [2]Commercialisation of some Government services, through the establishment of State Owned Enterprises, had started in the mid-1980s.

2.2  An effective and efficient system?#

Over the last fifteen years five key reviews of the New Zealand Public Management System have been undertaken, in addition to the work of a number of commentators who have analysed various aspects of the reforms[3]. The various reviews, and general commentary, have generally been positive about the core of the New Zealand model. The conclusions reached by both reviewers and commentators do not suggest that the clock should be turned back or that the core of the system should be significantly altered[4]. In coming to a common conclusion that the system is relatively sound both reviewers and commentators have tended to point to the following as positive aspects of the reforms: clearer accountabilities; greater transparency of Government activity; clarity of roles and responsibilities; devolution of resource management; better information about the activities of departments and the outputs they deliver; and the philosophical connectedness of the reforms[5].

In large part these strengths tend to focus on the efficient delivery of services over the 1990s. As a result, in identifying weaknesses within, or challenges for, the New Zealand model the commentators tend to point to the need for a greater focus on the effective delivery of services and maintaining and developing the capability of the public service.

2.2.1  A lack of focus on outcomes?#

Provision for formal strategic management processes in the New Zealand model was not made until the mid 1990s, with the development of the Strategic Results Areas (SRA) process. Until that time outcomes tended to be identified through the strategic policy development processes – making the identification of outcomes dependant upon the quality of policy advice provided by officials and informal dissemination of information.

Many reviewers and commentators have suggested, however, that the provision of strategic policy analysis is insufficient in providing for an effective identification of outcomes sought by government. Instead, they tend to call for a strengthened strategic management system – one where outcomes can be specifically referenced and articulated by both Ministers and senior managers within the government. This recommendation seems to arise from a perspective that improved articulation of desired outcomes may provide for increased strategic engagement, alignment, planning and evaluation.

More recently the Managing for Outcomes process has provided for some strengthening of the current strategic management components of the New Zealand public management system. Initially heralded by the development of Statements of Intent – accountability documents designed to provide for the articulation of outcomes that an agency is seeking to achieve – the process now also emphasises a need for agencies to consider other aspects of the planning cycle, particularly evaluation of interventions, and the capability the agency will require if it is to effectively work towards the achievement of identified outcomes.

The risk with the current Managing for Outcomes process, however, is that with little or no reference to how the strategic management system sits within the wider public management system there seems to be a presumption that by purely focusing on improved strategic management the government will be better positioned to achieve articulated outcomes.

2.2.2  Poor focus on ownership / capability?#

The architects of the New Zealand system provided for a split between two concepts associated with the delivery of services by the public service – the purchase and ownership interests [see Scott, Bushnell and Sallee (1990); and Scott, Ball and Dale (1997)]. The former concept recognises that Ministers (particularly Vote Ministers) would probably be most interested in purchasing services at least cost, and that in focusing on this it would be possible to drive efficiency gains through the system.

The ownership interest (to be overseen by Responsible Ministers) is a concept that reflects the need for chief executives and Ministers, as stewards of the public interest, to ensure that the ongoing capability of the public service to deliver services would be maintained and developed over time. The ability of chief executives to effectively manage and develop future capability requirements over the 1990s could be seen as limited, however, because of the constraining pressures exerted on baselines over the 1990s. A point that can be supported by increased calls for capability funding over recent budgets, and by a number of statements that suggest the capability of the public service has been reduced over the last ten to fifteen years, and a renewed focus on finding ways to address those areas where capability issues have been identified.

This environmental shift has become particularly apparent since the election of the Government in 1999. The current Minister of State Services has been clear that capability is an issue that needs to be addressed across the State sector in a variety of speeches [see Mallard (2000:1); (2001), (2002:1), (2000:2); and (2000:3);] and in setting the terms of reference for the Review of the Centre in 2001 (Advisory Group, 2001).

In order to effectively manage for outcomes it is important that resources – both financial and physical – be aligned in appropriate ways. The general perception of commentators is that the systems within the New Zealand model do not provide for this – in part through poor coordination of policy development, service delivery and use of resources; and in part through the operation of the financial management system.

The points made above should not be read as suggesting that the picture is completely bleak. There are a number of examples where outcomes have been built into policy development, resources are marshalled in pursuit of outcomes, performance is assessed against outcome measures, evaluation of policies is undertaken, chief executives have effectively managed their departments, and agencies have operated effectively with others to address cross-cutting issues. It is perceived that a managing for outcomes environment would, however, be characterised by such behaviours being seen as the norm rather than exceptions.

2.2.3  Conclusion#

On balance, commentators suggest that whilst the core of the model is sound and no major reforms are required the New Zealand model needs to be strengthened so that “outcomes” and “ownership” considerations become an integral part of the wider system, supported by a culture that provides for an increased focus on the achievement of results. These perceptions have been reinforced by an expressed desire, on the part of key Ministers, for change towards a managing for outcomes focus in the New Zealand public management system.

In part, these demands may reflect a perception that having developed a sound efficient base for the New Zealand public management system, it is now time to focus on finding a way to address the “wicked problems” that face governments – i.e. those defined by Harmon and Meyer (1986) as: problems with multiple, conflicting definitions and no clear solutions, and whose necessarily imperfect solutions beget further problems (p. 391).

Accepting that the core of the system is basically sound does not mean, however, that we should not revisit the core principles upon which the New Zealand public management model sits. Nor does it mean that the ways in which these core principles have been implemented are those that will best support a shift in focus towards outcomes focused management.

Notes

  • [3]The Logan Report (1991), reviews by the State Services Commission (1993), (1998:1) and (1998:2), the Schick Review (1996) and the Review of the Centre (2001), Boston et al. (1996), Petrie and Webber (2001), Scott (2001) and Norman (2003),
  • [4]A point reinforced by Norman’s (2003) research, where a number of those engaged within the system were interviewed. The interviewees are reported as being supportive of the general base of the reforms of the 1980s.
  • [5]Cangiano (1996) suggests that whilst ex-post, all the reforms within the public sector, appear as a well conceived and mutually reinforcing package in terms of sequencing; in practice, however, the end points became clearer as the reforms proceeded so that most of them were a program-in-the-making.

2.3  What issues need to be considered moving forward?#

New Zealand is not alone in finding that efficiency has been easier to promote than effectiveness within the new public management model. In a recent paper on the new agenda of public sector modernisation, the OECD (2002) suggests that internationally the first-generation reforms have been efficient but insufficient – i.e. that whilst the reforms have provided for more efficient government many of the reforms have created unintended perverse effects on service delivery and governance.

Related issues to be considered in determining how to provide for an increased focus on outcomes within the public management system include: the contractual culture that developed over the 1990s; a weakened understanding of the collective interest; the strategic environment within which public service managers operate; and the public’s expectations of the public service. An underlying theme related to each of these issues is the extent to which public servants should and can be held to account for their activities.

2.3.1  A contractualist culture?#

In 1996 Schick suggested that the reliance on contracts in the New Zealand model was leading to a checklist mentality, where managers only delivered those things that were specified in the formal contracts that supported the performance management system. This concern was highlighted again in the terms of reference for the Review of the Centre (2001) where the review team was asked to consider: whether the division of the State sector into a large number of departments and agencies, including the division between policy and delivery, is leading to an excessively narrow focus by managers and a loss of co-ordination across the public sector (p.39).

The focus on the use of performance indicators as an accountability tool – as they tend to become in a contractualist culture – and the ability to use them to shape the culture of organisations and clearly specify expectations of public servants, potentially creates difficulties as the public management system moves to provide for the specification of outcomes. These difficulties arise for two reasons.

Firstly, because outcomes are not easily specified, nor is it simple to attribute “successful” achievement of outcomes to any particular intervention, the ability to hold individuals to account for the achievement of specified outcomes is reduced. In turn, this potentially undermines current processes utilised to support performance management components of the wider system.

Secondly, the focus on performance indicators supporting accountability systems can limit an organisation’s ability to provide for a learning environment – where indicators can be used to evaluate the effectiveness of programmes with the goal of identifying whether it is appropriate to change the mix of interventions used in pursuit of those outcomes.

2.3.2  The collective interest#

The collective interest can best be described as an understanding that the agencies within which public servants work are not separate entities in and of themselves. Rather, government can be perceived as a corporation where the agencies that make up the corporation need to work together – whether in the development of policy or the delivery of services – to efficiently and effectively achieve the goals of the government.

The “silo” effect on the culture of the New Zealand public management system, which arguably resulted from the contractual nature of the State sector reforms and the implementation of machinery of government principles focused on the separation of activities, may not have been anticipated by the architects of the reforms. Nevertheless, it is clear that implementation of systems and processes based on these principles did have a fragmentation effect on the collective, or whole of government, perspectives of the public sector.

The nature of outcomes means that more often than not the key goals of government will only be achieved if agencies coordinate their activities. This suggests that a managing for outcomes environment will need to be supported by a culture that allows for improved alignment of resources and “joined-up” delivery of services. One obvious response to this need could be to provide for some agglomeration of the structures of the public management system. However, structural change is often insufficient in itself for promoting cultural change.

It could be argued, therefore, that the cultural change sought will only be developed if the component systems of the public management model all support: joint specification of the outcomes sought by government; flexible use of resources across structural divides (whether organisational or less concrete structures such as the use of Votes and portfolios); and unified delivery of services.

2.3.3  The strategic environment#

The strategic environment within which the public service operates will have an important bearing upon the institutional design decisions that can be made. Most often discussed in terms of differences between the public and private sectors, the strategic environment of the public service has been characterised by theorists (Yates, 1991; Alford, 1993; Ring and Perry, 1985; Johnson and Scholes, 2002; Kelly and Muers, 2002; and Norman, 2003) as needing to take into account different concepts of value; competition for resources; different strategic timeframes; exposure to public scrutiny; different legislative frameworks; and managerial constraints.

In essence, the “political” nature of the public sector provides an environment where a number of goals need to be worked towards at any one time, and the public expectations of accountability can be strict. In considering how the New Zealand public management system can be reshaped to support a managing for outcomes environment all of these environmental issues will need to be taken into account.

2.3.4  Public expectations - Participatory governance and direct accountability#

Internationally public expectations of governments, and the ways in which they consult with their constituencies have tended to undergo a paradigm shift which tends to be expressed as a desire for greater citizen involvement in decision-making, and an interest in greater direct accountability to citizens, i.e. without Parliament acting as an intermediary. This is most obviously evidenced by growing media interest in the performance of agencies delivery services for the government, and calls for representative groups to have greater levels of participation in policy formulation and implementation [Wyman, (2001)]. This creates some tensions around ensuring that appropriate targets are utilised.

Difficulties tend to arise, however, when it comes to the targets that will be utilised to provide a framework within which this accountability will be provided – largely because of the difficulties associated with setting outcome focussed targets and assessing performance against them.

Looking towards a managing for outcomes environment it could be expected that calls for direct accountability will continue to increase and that the systems and processes developed will need to be able to respond to increasing public demands.

2.4  Conclusion#

The challenges facing the New Zealand public management system have developed for a variety of reasons. Many appear to be relatively long-standing, reflecting the need to finalise the reforms of the late 1980s, whilst others seem to reflect environmental shifts – either as a result of the system degrading over time, or as a result of responses to ongoing tensions between the principles of “freedom to manage” and “accountability’. Responding to the identified challenges will require some (re)balancing of the various tensions that exist within a public management system.

As calls for an increased focus on outcomes in the New Zealand model become more demanding it is clear that new ways of identifying and supporting the values to be given precedence are required. Mechanistic systems have been, and will be, identified but these will only be effective if the hearts and minds of the public managers are brought to an implicit and explicit understanding of why the systems have been introduced and what they are trying to achieve. The following chapters canvass the how, what and why of potential changes to the system.

Initially, however, chapter three provides a discussion of how the principles and values that underlie the current system may be reassessed to support these changes.

Responding to the challenges posited in the preceding chapter could be as simple as making pragmatic decisions as the Managing for Outcomes process evolves. However, shifting the New Zealand public management model towards an outcomes focus could potentially lead to significant changes that may not be consistent with the principles upon which the system was originally designed.

Before considering whether the principles that support the New Zealand public management system are appropriate it is important to understand the “nature” of outcomes, and how managing for outcomes might be defined. These understandings are important because they help to identify some of the issues that need to be resolved in determining what values should be given precedence in setting a principle based framework.

3.1  The nature of outcomes#

Structuring a system focused on the achievement of outcomes with performance assessment and accountability principles at its core can create some difficulties. These tend to arise because:

  • outcomes can be specified at different levels of generality;
  • the myriad of relationships that exist between outcomes and outputs can create difficulties in determining what intervention is responsible for the realisation of a specific outcome;
  • the timeframes within which the achievement (or not) of specified outcomes can be assessed do not lend themselves easily to annual processes associated with the budgetary or performance management systems; and
  • it is difficult to specify performance indicators that adequately capture the expectations of Ministers without also providing for perverse incentives.

All of these issues can be addressed at some level when we look towards a managing for outcomes environment. In combination, however, it would appear that there may be some constraints around moving all parts of the New Zealand public management system to a pure outcomes focus (i.e. a system focused on outcomes to the exclusion of outputs), particularly if accountability is to be retained as a core principle.

3.2  What is managing for outcomes?#

The current guidance for departments (Steering Group for the Managing for Outcomes Roll-out 2004/05, 2003) states that Managing for Outcomes: will lead to a more responsive Public Service that is better able to identify and deliver the interventions that best contribute to the outcomes Government is seeking. This may mean doing things differently, having the confidence to better align resources to the most effective and efficient outputs (p. 3).

With this vision in mind, and looking to the core principles expounded by the Treasury in 1987 (see Figure 1), it is possible to suggest that within a managing for outcomes environment the New Zealand public management model should be underpinned by:

  • clear specification of the results (outcomes) sought by the Government of the day;
  • management systems that engender and reward an outcomes focus where appropriate, through strategic planning; financial management systems; the incorporation of good evaluation processes that ensure lessons are built into strategy and policy development; and a performance management system that provides for a balance between accountability and responsibility;
  • appropriate management of present and future capability requirements; and
  • a beliefs system that provides a focus on achieving results for citizens.

These points are largely consistent with those that have been identified by other jurisdictions seeking to provide for a greater focus on outcomes.

For example, at a detailed level the United Kingdom has implemented processes to support joined-up government. In reviewing the pilot processes the British Comptroller and Auditor-General (2001) identified the following criteria as prerequisites for successful results management: working towards clearly defined, mutually valued, shared goals; evaluating progress towards achieving the desired goal and taking remedial action when necessary; ensuring that sufficient and appropriate resources are available; directing the team and the initiative towards the goal; and working well together to achieve a shared responsibility (p. 9).

Work by the OECD in the same area – Governing for Results (2002) – suggests that there is a need to go beyond the rhetoric of reform to look at the practicalities of implementing a results focus (p. 3). The implementation issues identified by the OECD include: how to provide for cascading targets where establishing a “clear read” is challenging; aligning results and financial information; engaging Parliaments at an outcomes level; determining whether outcomes or outputs should be used as a proxy for results where increased attention to outcomes budgeting and management more explicitly move from the realm of management to the intersection between management and politics (p. 5); the role of audit, evaluation and results; and motivation of personnel (pp. 3 – 6).

The following section examines the key principles that provide a basis for the New Zealand public management system, and whether or how they may be amended to better support an outcomes framework. As noted in the introduction to this paper the constitutional convention of the public management system existing to support the government of the day in achieving its identified goals, and in the process of doing so being accountable to the people (through Parliament) for the efficient and effective use of resources generated through taxation has been taken as a given. This shapes some of the assumptions that follow.

3.3  Key principles#

The key principles underlying the New Zealand public management system, as outlined in Government Management (Treasury, 1987) are: clarity of objectives; freedom to manage; accountability; effective assessment of performance; and adequate information flows [see Figure 1]. Norman (2003) has suggested that the core principles of the New Zealand model provides for a distinctively cybernetic or thermostatic-like view of organisations (p. 43). In other words, the system provides for a cyclical flow of information about the activities of government, within a closed environment.

The principles as originally expounded are outlined below, with some consideration of how they might be reshaped looking towards a managing for outcomes environment. In attempting to undertake the latter I have been cognisant of the paradoxes that can be created within a cybernetic system, such as those highlighted by Norman (2003).

3.3.1  Clear specification of objectives#

The clarity of objectives principle responds to a perception that agents need to be aware of what they are expected to achieve before they can effectively deliver against specified expectations.

What does this mean in a managing for outcomes environment? Firstly, the imperative that objectives actually be specified must stand – people need to know what they are seeking to achieve if they are to effectively work towards that goal.

Secondly, it needs to be recognised that “objectives” will be specified for different reasons and in different ways. In this case “clear specification of objectives” will need to allow for both the explicit specification of outputs with performance indicators that will assist in implementation of accountability systems; and clear articulation of outcomes to provide a framework within which policy can be developed and with performance indicators that will assist in learning and evaluation.

Thirdly, the system needs to have a realistic expectation of how outcomes will be expressed. Articulation of outcomes is not a simple process: it is often difficult to untangle individual objectives from the whole, and it is nearly impossible to develop outcome statements that do not reflect or lead to multiple and sometimes conflicting objectives. Further, the multi-layered nature of outcomes means that attribution of interventions to the achievement of outcomes can be difficult. In a public management system based on performance management and allowing for accountability of activities this potentially creates real difficulties in moving towards a managing for outcomes environment.

These difficulties can, partially, be resolved by allowing for the differentiation of outcomes into different strata, but it also means that the issues associated with providing for some understanding about the level at which outcome objectives can support an accountability system need to be addressed. Issues which become more complicated when it is acknowledged that using performance measures associated with outcomes can also provide for the development of a learning environment, where the effectiveness of interventions can be assessed as an input for the policy development process.

Finally, specification of outputs and anticipated links between outputs and outcomes will continue to be important as this should provide for a way for management of some of the tensions between accountability and evaluation for learning. This will be of particular moment if specification of the two allows for a focus on different aspects of the wider system.

Clarity of expectations is important for all involved in a system – if only because it provides a level of transparency about what the different actors are expected to deliver. How those expectations are expressed and with what purpose would appear, however, to be an issue that needs to be resolved in looking towards a managing for outcomes environment. If, as the discussion around the other principles will suggest, outputs still need to have a clear role it would appear that a revised principle (at the least) needs to be more specific about what is meant by “objectives’.

3.3.2  Managerial discretion#

The “freedom to manage” principle is based on the theory that those closest to the decision point will generally be best placed to make relevant management choices. Implementation of the principle has tended to be restricted by risk management processes built into the New Zealand public management system – for example, through the promulgation of central agency guidelines relating to financial delegations, bargaining parameters, and the monitoring of major information technology projects.

The impact of rhetoric on the implementation of the New Zealand model can, perhaps, be most clearly identified in relation to this principle. The quid pro quo for freedom to manage in all of the papers supporting the reforms was the introduction and maintenance of an effective accountability system – managers would have the freedom to manage inputs, but they would still have to account and be responsible for performance against specified objectives.

It may be that this rhetoric helped to drive the culture that has seen chief executives resist central agency engagement in assessing and assisting with the effective and efficient operation of government agencies (no doubt assisted by central agencies taking some time to determine what their role should be in an environment not characterised by command and control systems). It could also explain why increased financial flexibility – in order to support a managing for outcomes focus – is now sought by some chief executives and others charged with implementing a managing for outcomes focus. Conversely, within the current environment there are also demands for greater centralisation of some activities and decisions, particularly those activities or systems focused on human resource management.

Both issues seem to come back to a discussion about the level at which decision-rights should be set – i.e. should chief executives or Ministers make decisions about the inputs to be utilised, the outputs to be delivered or the outcomes sought – in order to provide for managerial freedom whilst also managing risks (political, policy and operational) to the Government.

Determining where these decision-rights best sit is not a simple process – as Brady (2002) illustrates in her paper on the decentralisation of decision-making. No common risk management framework exists, and often where decision-rights sit in various circumstances will depend upon the “political” nature of those decisions. Nevertheless, it might be possible to look at a matrix that relates “political / strategic” risk management by Ministers to managerial ability to make the most informed decisions.

Figure 2 – Decision rights

  Political / Strategic risk management needs
High Low

Managerial ability to make decisions

High

Outputs

(key interventions utilised to support Government priorities)

Inputs

Low

 

Outcomes

Outputs

(business-as-usual services, often mandated by legislation)

Such a matrix would suggest that Ministers will have the most interest when the political / strategic nature of the decision is high, and managers’ ability to make an informed decision is low, and that this is most likely to require a focus on outcomes. The converse will be true when the political / strategic nature tends to be low, and managerial ability to make an informed decision is high, and that this will generally occur around inputs. Finally, depending upon the nature of specific outputs, both Ministers and managers will have clear roles to play.

3.3.3  Accountability, Performance Management and Adequate Information Flows#

A core aspect of the New Zealand model is the strong focus on performance management, and holding managers to account for the delivery of specified services.

The three principles work from an assumption that it is possible, within a public sector environment, to hold agents to account for specified objectives. This has proven to be significantly difficult over the last fifteen years, primarily because:

  • specification of objectives has not been sufficiently robust (particularly around outcomes or behavioural expectations of chief executives);
  • the State Services Commissioner has been faced with institutional constraints in the effective management of chief executives (in part, because of the trilateral employment relationship between the Commissioner, Ministers and chief executives);
  • information asymmetries have often been acute; and
  • the accountability systems do not provide for the assessment of departmental, as compared to chief executive, performance.

These issues are likely to be exacerbated within an environment focused on the achievement of outcomes where: specification and attribution of activities become more difficult; the cultural behaviours and norms expected of agents are more intrinsic; and external factors have a greater impact on performance.

All of which suggests that accountability for chief executives should continue to sit against those aspects of service delivery where it is possible to specify clear performance indicators that are unlikely to suffer from attribution difficulties – i.e. where external factors are unlikely to impact. It may, however, also be possible to provide for some form of accountability for the more intrinsic behaviours and norms that support a managing for outcomes environment. Some reconsideration of the current principle will be required, therefore, if the latter is to be provided for.

Performance indicators can be developed to provide for these accountability requirements. They can also have a wider role, albeit one which will need to be expressed and managed carefully in an environment where the public is calling for increased levels of direct accountability. Performance indicators can be developed to indicate whether the combined activities of government are having some level of success in providing for the achievement of outcomes. The principle, as it is currently expressed, does not provide for this role and should be rewritten to provide recognition of the importance of evaluation and learning.

The architects of the system also identified that appropriate information flows would be necessary to support the accountability / performance management aspects of the New Zealand model. This is a principle that arguably has increased resonance moving forward, as a managing for outcomes system will be dependant upon good information flows between and within agencies as officials seek to address the multiple needs of citizens – whether in seeking to ensure that compliance costs in the private sector are reduced, or in seeking to ensure sound social outcomes for those that need assistance.

At the same time, the system will continue to need effective information flows to support both the accountability and evaluation principles. This suggests that the principle needs to be reshaped, providing for a variety of information needs.

3.3.4  Revisited principles#

The preceding discussion, coupled with a perception that the core of the New Zealand public management system is sound, suggests that the core headlines of the principles upon which the system is built should be retained, but that some realignment of the principles might be required to support the cultural change needed if a managing for outcomes environment is to effectively be introduced. Such a reshaping might allow for the core principles of the system to be defined as follows:

Clarity of objectives – The initial element of a management process must be as clear a specification as possible of the objectives which managers are responsible for achieving. This implies that the objectives managers are accountable for achieving, the behaviours expected of managers, and the objectives managers are expected to work towards should all be clearly expressed.

Managerial discretion – Managers must be given the power to make the achievement of specified objectives possible, within limits determined by constitutional, risk management and accountability systems. Managers should have the freedom to make resource allocation decisions on a basis that enables the most efficient delivery of outputs, and the effective management of capability, but they should gain Ministerial agreement to the most effective mix of interventions to be used in seeking to achieve outcomes.

Accountability – Freedom to manage is not a sufficient precondition for good management. Intrinsic and extrinsic incentives and sanctions must be in place to modify the behaviour of managers to ensure that they do act to meet established objectives rather than pursuing independent goals of their own. For this to be achieved, managers must be accountable for the decisions they make, and those on whose behalf they act must have the means to make that accountability “stick’.

Effective assessment of performance – If managers are to effectively manage for outcomes, and be held accountable for the delivery of outputs, sound systems that provide for both the evaluation and assessment of performance are required. These systems should inform each other, but not constrain the ability of managers to learn from mistakes in identifying the most appropriate mix of interventions to assist in the achievement of outcomes.

Adequate information flows – Adequate information flows between and within agencies are an imperative component in a system seeking to provide for whole-of-government coordination, performance management and assessment of accountabilities. Information processes that support all three aspects will be required in a public management system seeking to manage for outcomes.

3.3.5  Conclusion#

The base principles of the New Zealand public management model need to be revisited if the rituals, routines and stories that support the model are to provide for a greater focus on outcomes.

Given that a complete shift to an outcomes-oriented system is likely to be constrained by the constitutional convention for accountability in how taxpayers’ funds are expended, and the nature of outcomes (which tends to preclude simple attribution of the effectiveness of interventions in a timely manner), it is probable that chief executives should be held to account for managing for outcomes, rather than management of outcomes. This distinction, coupled with the suggested changes to the principles, will prove to be important in the (re)design of both financial and performance management systems.

Amending the principles that underlie the system in isolation of other institutional change will be insufficient to promote effective culture change however. The following section looks at cultural change and tools that might prove useful in shifting the focus of the New Zealand public management sector.

3.4  Culture change#

In looking towards a managing for outcomes environment it is clear that the public service will need to have a culture that is also concerned about achieving results for citizens – rather than simply achieving specified output targets. The system will only achieve a shift towards a results orientation if those in the front-line are focused on coordinating services and meeting the needs of citizens, in addition to delivering the services they are “contracted” to deliver.

This could be achieved by continuing to embed a performance culture across the sector, albeit one focused on outcomes, through the auspices of a performance management system focused on setting incentives for chief executives to deliver a results culture within their organisations. It may, however, prove more effective to tap into the personal goals of public servants, using personal and professional drivers to further engender a focus on the achievement of outcomes across the public sector.

Johnson and Scholes (2002) outline the need for all parts of a system to be addressed in considering the culture of an organisation – or in this case the culture of an institution. The cultural web identifies six interacting areas that combine to create the cultural paradigm an agency works within: stories; symbols; power structures; organisational structures; control systems; and rituals and routines.

This suggests that in attempting to shift the New Zealand public management system towards an outcomes focus we need to consider:

  • how stories about “best practice and positive results” (for example, improved collaboration and more flexible use of resources) can be disseminated across the public sector as they occur;
  • how key symbols (particularly the rhetoric of the system) can be utilised to support an increased focus on outcomes;
  • how the power structures that sit around Ministers, and in Ministerial relationships with departments could be amended;
  • whether the current organisational structures do hamper the effective achievement of outcomes, and if other forms might be more effective;
  • how the primary control systems within the system (i.e. the financial and performance management systems) can be shaped to better support managing for outcomes; and
  • whether the ways in which agencies engage with each other (i.e. the rituals and routines) need to be amended (for example, how can we provide for better collaboration in the setting of strategic directions in sectors; real consultation in the development of policy; and more integrated service delivery).

The paper seeks to answer these questions in subsequent chapters. Initially, however, it is important to understand the tools and frameworks that might help in the development of change processes.

3.5  How do we effect change?#

Knowing what aspects of a system need to be focused on is one thing, but it is also important to understand how change can be effected, whether within an organisation or across an institution.

The change process associated with the State sector reforms can be seen as having addressed all of the aspects identified by Johnson and Scholes (2002), in addition to having clear political leadership and amending the rhetoric of the system. These tools, along with concepts of “change levers’, provide some insight into how the structural and systemic changes to be identified might be achieved.

3.5.1  Political leadership

Schick (1996) has noted that in New Zealand Ministers had a key role to play in progressing the State sector reforms, driving the changes through legislation and as a component of the more wide reaching changes to the economy. More recently, much of the impetus around providing for an increased focus on outcomes has been provided by comments ascribed to the Minister of State Services [see Mallard (2000:1); (2000:2); (2000:3); (2001): and (2002)]. The importance of political leadership in public sector reforms is reinforced by the example of the United Kingdom, where the leadership of the Blair government has been paramount in shifting the culture of the United Kingdom’s public management system towards a results orientation.

This suggests that Ministerial pressure and support for change will continue to be an imperative aspect of the jigsaw if New Zealand is going to continue down the road towards managing for outcomes. The role of Ministers within the system, and how they exercise their responsibilities will, therefore, become an important question to be considered in subsequent chapters.

3.5.2  Rhetoric

Rhetoric, the use of language to shape culture, has also proven to be an important change tool in the past. In discussing ways in which the system can be amended to provide for an increased focus on outcomes it is important to be clear about the perceptions of the current system and the ways in which language has helped to develop these perceptions. Over the years language within the New Zealand public management system has become value-laden and whilst the definitions may or may not be ones everyone would concur with, some words have taken on particular meaning.

The words (and associated concepts of) efficiency and effectiveness provide a good example of this. In common lexicon efficiency has taken on the meaning of least cost, whilst effectiveness is generally understood as pointing towards the achievement of goals. Efficiency and effectiveness are seen as equally important goals for a public management system, but how they are traded off against each other appears to be identified as an issue that needs to be resolved. Perhaps, instead, it is important to determine how the impact they have on systems and processes can be balanced.

Similarly concepts such as purchase and ownership, and outputs and outcomes, have also become value-laden over time. Like efficiency and effectiveness there seems to be a perception that they need to be traded against each other in the development of the public management system. As this paper explores this is not necessarily the case – a well-functioning public management system should, in fact, balance consideration of efficiency, effectiveness, ownership, outputs and outcomes in the management of government agencies and the government as an institution.

3.5.3  Levers of control

Simons (1995), Etzioni (1961), Ouchi (1980), and Dalton (1971) identify ways in which power and control can be considered within organisations. In doing so, they provide an indication of the different levers that can be utilised in looking to shape a managing for outcomes environment. A common theme throughout these theories is that control can be expressed and / or change effected through the use of both “hard” and “soft” management tools. In other words, we can look to coerce and restrain actors by using performance management and boundary systems, or we can try to influence the behaviour of actors through utilising existing beliefs systems and clan (peer) pressure.

It could be argued that over the last fifteen years, in focusing on efficiency and rules, the New Zealand model has tended to rely upon controls at the “harder” end of the spectrum. For example, the significant focus on the coercive and remunerative forms of power over the last fifteen years through the use of individual contracts for chief executives, the use of bonuses to support good performance, and strict performance systems based on explicit sanctions and incentives have supported these two forms of control.

Looking forward to a managing for outcomes environment, an agglomeration of the theories outlined above suggests that the system needs to ensure that the levers of control utilised are sufficiently flexible to conform to their individual roles, whilst also allowing for practitioners to apply rules in innovative ways that assist in the achievement of results.

4  Structures#

Effective management systems require an institutional design that provides for both sound systems and organisational structures. Structures are usually seen purely as the organisational forms utilised to deliver services. In thinking about the New Zealand public management system it is also important to recognise that such a definition is too narrow and needs to include less tangible structures, including: the Cabinet; Ministerial relations – as evidenced by the use of portfolios; vote structures; and the role Parliament and its officers play in the wider system.

4.1  Roles and Responsibilities#

The New Zealand public management system incorporates a number of different actors with different roles and responsibilities. The relationships between these different levels of government has tended to be discussed in terms of hierarchical and accountability relationships. This will largely be appropriate, after all the Executive and the State sector do need to be accountable to citizens (through Parliament) for the use of taxpayers’ funds. However, in looking towards a managing for outcomes environment it may be that the ways in which accountability is provided, or the relationships between the Executive and agencies will need to be amended to assist in creating a learning environment.

4.2  Machinery of Government principles#

The State Services Commission (1999) has identified three machinery of government principles for organisational design. These focus on effectiveness and efficiency, risk management, and constitutional conventions.

A potential failing of these principles is that they do not provide for consideration of how less tangible structural arrangements (for example, vote structures) should be constructed. Boston et al (1996) suggest that decisions about machinery of government design in the early period of the State sector reforms were guided by eight objectives which may assist in determining a set of base principles that can support more than organisational design choices. These are:

  • Maximise allocative and productive efficiency;
  • Reduce the range of state functions under direct ministerial control;
  • Ensure clear lines of managerial and political accountability;
  • Ensure that all organisations have a clear mission and that inconsistent objectives are transparent, and ideally, the responsibility of separate organisations;
  • Ensure high-quality, contestable policy advice across all sectors;
  • Minimise the risk of bureaucratic, provider or regulator capture;
  • Ensure good horizontal coordination; and
  • Improve the bureaucratic representation of disadvantaged or poorly represented groups (p. 81).

A common theme of the objectives identified by Boston et al (1996) is the need to allow for transparency of activity and separation of policy and service delivery activities. This transparency was largely perceived as a benefit because it would assist in specifying outputs (in part for accountability purposes), but also because it would support a clear focus on individual interventions. It may be, however, that this focus on transparency and separation of activities has contributed to what is currently perceived as “fragmentation” in the New Zealand public management system.

If this is the case (and the following section looks at issues that impact upon coordination of resources in the New Zealand model), then it may be appropriate to revisit the core machinery of government principles so that they have a greater focus on coordination of and collaboration in policy formation and service delivery. Reshaping the principles identified by Boston et al to allow for this focus might result in principles such as:

  • ensuring that all organisations are working towards common, and non-conflicting, outcomes;
  • providing for good coordination of policy advice and delivery of interventions;
  • ensuring high quality policy advice; and
  • maximising effective and efficient service delivery.

4.3  Coordination of resources#

The Review of the Centre (2001) suggested that resources within the public management system are not utilised effectively, and that the fragmented nature of the state sector has created a situation where a lack of coordinated activities hampers the management of resources in way that supports managing for outcomes.

Specific issues identified as arising from the perceived fragmentation of services include: coordination problems because of the number of agencies and the overlapping areas of delegation and boundaries used by government agencies and local government; difficulties in interaction with government for citizens; financial resources being split into a number of small pools creating overlaps and gaps in funding; and inadequate systems to allow service delivery to inform policy development.

There is no optimal structure for a state sector when considering how cross-cutting issues can be addressed. In fact, coordination issues are as likely to occur within large conglomerate agencies as they are in a more fragmented system. The British Comptroller and Auditor-General (2001) suggests that coordination issues need to be addressed by:

bringing together of a number of public, private and voluntary sector bodies to work across organisational boundaries towards a common goal [through]:

  • Realigning organisational boundaries – bringing together the whole or parts of two or more organisations to create a new organisation;
  • Formal partnerships – working together by contract, protocol or framework agreement;
  • Informal partnerships – working together by liaison, consultation or unwritten mutual agreement (p.1).

The United Kingdom has piloted a range of structural mechanisms for addressing cross-cutting issues. Reviews to date suggest that some progress has been made in the different pilot areas, but both Flinders (2002) and Clark (2002) sound some notes of caution:

The current reforms are not substituting old structures with new, but, on the contrary, are imposing new layers and mechanisms upon the existing framework. It is therefore possible to identify the evolution of a complex matrix within central government. A programme intended to increase efficiency and effectiveness may therefore deliver the unintended consequences of increased costs, more bureaucracy, reduced flexibility and pluralistic stagnation (Flinders, 2002, p. 67).

Current initiatives seem to be framed within a managerial preoccupation that relies on inter-agency structures while neglecting other, vital, aspects such as operating culture, central to the delivery of any joined-up policy. This means that while managerial objectives may be achieved, policy outcome goals may not (Clark, 2002, pp. 115 – 116).

These comments reinforce the point that all levers of control, systems and processes need to balance and support each other if a managing for outcomes environment is to be successfully achieved. In other words, whilst structural change could prove to be a useful response in some instances, it should only utilised if it appropriately supports other initiatives.

The diversification of votes, portfolios, departments and other agencies in the New Zealand model can also create impediments for the strategic coordination of policy and service delivery as it can:

  • increase the number of people that need to be engaged in strategy setting;
  • lead to the development of a culture where agencies compete against each other;
  • create systemic barriers to the effective use of resources in seeking to achieve identified goals;
  • make it more difficult for Ministers to engage in strategy development; and
  • reduce the ability of Ministers to focus on ownership issues.

So, what does this mean for a managing for outcomes environment?

4.3.1  Votes and portfolios#

The power structures of an organisation (or in this case the institution) need to support any proposed change process. It could be argued that the current structures tend to reinforce the fragmentation of the State by allowing for agencies to have multiple requirements placed upon them, and thereby reducing the avenues for providing a common direction. Streamlining of votes and portfolios could allow for an increased level of engagement and coordination between Ministers and agencies on sector outcomes, thereby supporting an increased focus on outcomes.

Discussions about streamlining of Ministerial relationships can be difficult, in part because Ministers tend to see direct control over relevant resources as providing them with the ability to effect change. Streamlining of votes and portfolios would not necessarily require a reduction of the number of Ministers within the New Zealand public management system, or the creation of a formal inner Cabinet (although the existence of Ministers outside Cabinet essentially provides the latter already).

Rather, senior Ministers could be allocated portfolios for major agencies or policy areas with less experienced Ministers allocated sensible groupings of portfolios within those sectors. The letters of delegation, from the Prime Minister, could then identify the role of the more senior Ministers as having responsibility for strategic development of policy and oversight of ownership issues (where appropriate), whilst the other Ministers would be more focussed on operational issues. The difference between this proposal and current structures would simply be some rationalisation of the number of associate Ministers and the ways in which they are spread across a range of sectors. Providing for senior Ministers to take a lead role in budget-setting processes could also strengthen these proposed structural relationships.

Obviously, decisions about the structure of Cabinet, the number of portfolios and Votes is a matter for the Government of the day. Some streamlining of these could, however, assist Ministers to better achieve the outcomes they seek.

4.3.2  Organisational structures#

The current organisational structures within the New Zealand model should be revisited, over time, to determine whether they do indeed reinforce an environment that does not support the alignment or coordination of resources. Appropriate responses to situations where this is deemed to be the case should then be developed.

In some instances this might result in structural change. For example, moving some of the smaller departments, or Crown entities, into larger agencies may well assist in providing for higher quality policy advice and better coordination in some sectors. In other instances more informal mechanisms for supporting better coordination might be appropriate and should be taken into account. For example, the ability to provide for the development of a lead agency, service delivery coordination processes such as memorandums of understanding, or other less formal systems should be considered. In undertaking any machinery of government analysis the impact of any decision on the wider sector should also be taken into account.

4.3.3  Roles of central and lead agencies#

An analogy that could assist in developing an understanding about the roles of central and lead agencies is that of a large corporate with a number of business units where the role of Cabinet would be one of a direction setting and governance body, overseeing the activities of the various departments (business units), with the assistance of the central agencies (the corporate office).

Within such a structure central agencies would have a clear mandate to provide advice and support for Ministers, by operationalising oversight of departments and agencies within the State sector through flexible and empowering management systems.

The role of lead agencies could be to oversee areas of strategy development across unified sectors (for example, the justice sector), encourage ongoing evaluation of policies and programmes, and contribute to oversight and performance assessment of agencies within that sector. This should provide for more coordination across sectors, whilst retaining some physical separation of agencies where it is considered appropriate for this to occur.

4.3.4  Structures for the future#

Streamlining of Ministerial relationships and provision for more coordination between agencies at a “strategic” policy-setting level could result in less fragmentation. It could also lead to the establishment of stronger silos between different sectors if systems are not sufficiently robust to provide for dialogue on cross-cutting issues.

Figure 3 – Possible structural hierarchy

 

Figure 3 – Possible structural hierarchy.

Figure 3 illustrates one way the structures of the New Zealand public management system could support a managing for outcomes environment. In this scenario senior Ministers and the lead agency would be responsible for setting strategy across the sector (in consultation with junior Ministers and the other agencies), whilst all relevant Ministers and agencies would be responsible for delivering interventions consistent with the sectoral strategy.

The efficacy of such a model would rest in its ability to promote collaborative behaviour within sectors or networks. Achievement of this would be dependant upon leaders within those sectors providing best practice examples of articulation of outcomes, equitable processes for the reallocation of resources and rewarding of evaluative activity.

4.4  Conclusion#

More coordination of activity – whether in the administrative aspects of the system or in the delivery of services – will be an important foundation for the development of a managing for outcomes environment. In considering how this might be provided for it is important to acknowledge that the principles of machinery of government will need to be refocused on this goal; and that both formal and informal responses should be considered when looking at how different coordination issues should be resolved.

More specifically, the base principles underlying structural decisions in the New Zealand public management system could be revisited to:

  • ensure that all organisations work towards common, and non-conflicting, outcomes;
  • provide for structures that support good coordination of policy advice and delivery of interventions;
  • provide for structures that ensure high quality policy advice; and
  • maximise effective and efficient service delivery.

Having identified such principles it is also important to consider how both the power structures and organisational forms within the New Zealand public management system can support an increased focus on outcomes. This chapter has suggested that structures impacting on relationships between Ministers and agencies could be streamlined to provide for increased coordination; lead agencies should be used more creatively; and different avenues for coordinating the delivery of services should also be considered. These changes will be insufficient, however, if those who oversee and guide the structures within the institution do not model the collaborative behaviours that will be required to effect change. The following chapters also indicate some of the roles and responsibilities Ministers and chief executives will need to assume in managing for outcomes through revisited strategic and financial management systems.

5  Strategic management#

Strategic management is a phrase that means many different things to different people. Poister and Streib (1996) suggest that:

strategic management is concerned with strengthening the long-term viability and effectiveness of public sector organisations in terms of both substantive policy and management capacity. It integrates all other management processes to provide a systemic, coherent and effective approach to establishing, monitoring, and updating an agency’s strategic objectives (Poister and Streib, 1999, p. 308).

This chapter outlines some of the constraints that may have affected the development of strategic management systems over the 1990s, and which continue to constrain the development of strategic management into the future; and ways in which these constraints may be addressed. The chapter also outlines the current Managing for Outcomes process, and considers whether it has sufficiently robust processes in place to assist agencies meet the identified constraints.

5.1  Articulation of outcomes#

As noted earlier one of the key principles upon which the New Zealand public management system rests is the clear specification of objectives. Within a managing for outcomes environment realisation of this principle will require clear articulation of the outcomes sought by Government.

Over the past fifteen years we have seen a clear focus on specification of outputs within the New Zealand public management system[6]. Articulation of outcomes is, however, something that the New Zealand model has struggled with. This would appear to be due to the following.

A lack of Ministerial engagement and to some extent a lack of willingness to engage – Ministers are clearly willing to engage in policy-making processes, but seem to have been less interested in participating in formal strategy setting processes. Ryan (2003) suggests that this may, in part, arise because: where policy problems and solutions are complex and uncertain, the adversarial character of Westminster-derived politics particularly when combined with strong party discipline, provides little motivation for the executive to be precise about its goals and objectives (p. 27).

Lack of advice and support for Ministers – Gorringe (Grimes et al., 2001) suggests that Ministers have been hampered in fulfilling their role by information asymmetries, resource constraints (particularly on their time), the complexity of the New Zealand model, and a lack of advice and support as a counter to these issues. It was assumed that Ministers would purchase any advice and assistance they required, however this has largely not occurred and there is little evidence of departments or central agencies providing assistance, possibly because agency responsibilities in this area are unclear and unspecified.

Implementation of the State sector reforms did not take into account the implications of a strong performance culture on the identification of outcomes – the focus on performance through assessment against performance targets through the early and mid 1990s has potentially created perverse incentives whereby chief executives (like Ministers) appear to have been unwilling to clearly specify outcomes, against the risk that they could be held to account for non-achievement.

Lack of systemic support – arguably, attempts to provide for articulation of outcomes during the 1990s was not supported by other institutional changes. For example: financial management systems focused on outputs, and incremental budgetary processes did not easily allow for consideration of “alignment” of new spending with governmental goals; performance management systems were not designed in a way to encourage taking responsibility for the achievement of outcomes; reporting requirements to Parliament encouraged an outputs focus; and the continued policy of structural separation of agencies on operational / policy lines underlined the expectation that coordination was second to efficiency as a system goal.

Fragmentation of the public service – the fragmented nature of the public service can work against effective articulation of outcomes, if only because of the number of different agencies engaged in specifying outcomes in particular areas. The bottom-up approach currently being utilised to specify outcomes could underscore this risk.

5.1.1  Current processes – lessons learnt

During the 1990s attempts were made to provide for more systematic identification of the outcomes government sought from the public service. More recently the Managing for Outcomes process has been developed and implemented across the core public sector[7]. The process, as designed, seems to have learned from many of the past attempts. It encourages departments to take ownership of the process, and to develop agency specific outcomes in consultation with relevant Ministers, and it has been accepted that an agency should be focusing on an “important few” outcomes rather than trying to push all activities into the framework. Further, guidance provided by the State Services Commission and the Treasury puts the specification of outcomes into a framework of continuous improvement – where planning, evaluation and review are set alongside direction setting.

A recent evaluation of the process to date (Economics and Strategy Group, 2003) suggests that articulation of outcomes has successfully been introduced. However, the authors of the report also suggest that what has been accomplished to date is simply the first step in embedding an outcomes-focused management system within agencies.

More specifically the review identified that: the process has provided a focus for discussions about an agency’s purpose and focus; identification of outcomes has provided for improved expression of tangible departmental contributions to outcomes; and Statements of Intent have proved to be useful communication tools. As a counter-point to these success factors the review also found that some constraints on the process do exist. These are identified as:

  • High-level outcome frameworks may have constrained the development of realistic and effective operational strategies and business plans. The more ambitious “reach” outcomes tend to be presented as more important, despite being less within the control or influence of the department.
  • The task of identifying, developing and evaluating effective and useful performance measures is an onerous one and seems likely to exceed available skills and resources; it may even distort future workloads in some departments.
  • As a result, easy-to-collect (mostly high level) performance indicators in some sectors (e.g. RS&T, health and economic performance) may be substituted for more operationally relevant measures. Also, departments may simply measure what they can afford to, omitting potentially critical interventions.
  • Inconsistencies in the MfO approach of departments, including lack of precision in outcome description, could easily impede (or over-state in some cases) the opportunities for cross-agency collaboration (p. 6).

Two key issues appear to underlie these constraints: how accountability will be provided for; and the provision of support to agencies.

5.1.2  Possible responses to identified challenges

It is clear from the above that agencies would appreciate clarity about how evaluation of outcomes will affect accountability, and the provision of more tools, frameworks and examples of best practice. In addition it would appear that an increased level of Ministerial engagement would be welcomed, along with some clarification about who is responsible for the articulation of outcomes.

Increased dialogue with Ministers about the benefits of explicitly specifying the outcomes they wish agencies to focus on might assist in addressing the former issue, and should be supported by clarification of the “accountabilities” associated with specification of outcomes. This process would most usefully be undertaken in conjunction with the Audit Office, to ensure that Parliament is both cognisant and supportive of the paradigm to be established.

A sense of chief executive responsibility should continue to be encouraged by holding chief executives to account for their exhibited behaviours in this area; whether through supporting their Ministers in articulating outcomes, or leading the sector(s) within which they work. Concerns about how performance management systems will be coupled with an outcomes focused strategic management system will only be managed over time as Ministers and central agencies model behaviours which support chief executives in taking responsibility for, but not necessarily being held accountable for, the achievement of outcomes.

Further consideration should be given to how outcomes can best be aligned across and through the different strata of agencies that make up the State sector. This could result in departments identifying outcomes that then need to be picked up by relevant Crown entities, or the development of joint outcomes across departments with a reflection of the dual responsibilities placed upon chief executives as a result. There is some limited evidence of the development of joint outcomes across the public service, and this could best be built upon through disseminating knowledge to chief executives and their senior managers.

Finally, continued dissemination of tools, frameworks and examples of best practice should assist agencies to continually improve the level at which they are able to articulate outcomes, the links between outcomes and outputs, and the capability required to effectively and efficiently deliver services.

Notes

  • [6]Which is not to suggest that outputs are perfectly specified across the public service
  • [7]All departments will have current statements of intent for the 2003/04 year.

5.2  Specification of outputs#

Having identified the outcomes agencies will work towards, it is also important to specify the outputs that will be most likely to achieve those goals. This specification tends to take place during policy development processes. Generally undertaken in “bite-sized chunks” strategic policy reviews will revisit an articulated policy issue and identify ways in which a problem may best be resolved. Usually, these processes will allow for a reassessment of current policy settings, but they do not always allow for an evaluation of current interventions. The limited nature of reviews (in order to keep them at a manageable scope) generally means that the range of activities undertaken by an agency will not be reviewed however.

The State Services Commission’s model for a managing for outcomes process provides for a planning process where the question of “what is the best way to achieve our goals” may be posed. In answering this question agencies should be able to move towards the specification of the outputs most appropriate for the achievement of articulated outcomes. Processes to support such a reassessment are not well-developed however. The New Zealand process of a baseline review is possibly the closest tool available, but this can be a negative and lengthy process for all agencies involved. In looking to provide for a complete reassessment of the outputs an agency delivers new and more effective tools may be required.

5.3  Identification of capability requirements#

Identifying the outputs to be delivered will be insufficient if an agency does not also have the capability to deliver specified activities. A key aspect of a managing for outcomes environment will, therefore, be the development and maintenance of the capability required by an agency, or agencies, for the effective delivery of services. Capability is most often thought of in terms of people. However, whilst the human resources of an agency are an important component of its capability, so are the information systems and capital assets required by the agency to efficiently and effectively deliver the services identified.

The Managing for Outcomes process has suggested that identification of the capability each agency requires to deliver specified outcomes is something agencies should start to focus on over the 2003/04 financial year and beyond. As agencies are supposed to consult with central agencies as they work through the Managing for Outcomes process this could allow for some conversations about the issues facing individual agencies.

5.4  Implementation and Review#

Finding ways of providing for coordinated delivery of services will be a core element of achieving results across some areas of government. Promotion of behaviours to support this collaboration will occur in a variety of ways, but if senior public servants take a lead by promoting better collaboration and coordination by designing and supporting more “joined-up” implementation systems (where appropriate) the required cultural changes may come about in a shorter time-frame.

As an integral component of effective strategic management, the review process does create some difficulties within a public management system predicated upon the assessment of performance for accountability reasons. These difficulties tend to arise because of the disincentives that may be placed upon reviews designed to identify both successes and the weaknesses associated with different programmes (particularly since the latter information is important for strengthening policy settings).

5.5  Conclusion#

Strategic management at a whole of government level will only be effective if it helps to shape and inform organisational strategies. The two need to be aligned if outcomes are to be successfully worked towards. It is for this reason that, to the extent possible, departments must continue to “own” their statements of intent and planning processes, even if they incorporate more actors into those processes.

The strategic management system needs to be focussed on outcomes, and identification of the links between outcomes and outputs, identification of the capability an agency requires; and providing for ongoing review of the activities that support the strategic direction of both agencies and the government.

Ministers taking more ownership over the specification and coordination of outcomes; chief executives taking responsibility for working more effectively across sectors; and the development of tools and frameworks to assist agencies in identifying their intervention logic and capability requirements; and central agencies playing a greater role in supporting the development of joint outcomes are different interventions that may assist agencies as they move towards a managing for outcomes environment. This leadership should also help to build and disseminate the stories, rituals and routines identified as assisting in the promotion of cultural change.

The last fifteen years have seen the performance management systems within the New Zealand public management model focused on supporting the principle of accountability. In doing so it could be argued that the performance management systems have operated in a manner that is counter to the generation of a learning environment and the evaluation of the effectiveness of policies – why would Ministers and chief executives provide information where they might be held to account for “failed” policies?

A managing for outcomes environment requires the evaluation of policies and the impact interventions have on outcomes. Without this it will be impossible to determine not only whether the public service is “doing the right” things, but also whether it is “doing them right”. It is important, therefore, that a way be found to encourage evaluation whilst ensuring that “negative” findings will not necessarily impact in terms of accountability.

6.1  Accountability#

The nature of accountability relationships will be dependant upon the different levers and forms of control available to different actors within the New Zealand public sector management system, and the culture that exists within the current environment.

6.1.1  The concept of “Accountability”

“Accountability” as a concept has a number of different nuances. This is particularly true when looking towards a managing for outcomes environment where it is likely that questions such as: whether people should be held to account, or have to give account; the extent to which straight-line accountability is sensible when duality and strategic coordination across agencies is being encouraged; and the extent to which the system can provide for the intrinsic incentives and sanctions on performance that are not obvious in a strongly contractual system need to be resolved. It is possible to summarise these different questions in terms of whether to provide for accountability or to encourage responsibility.

Mulgan (2000) suggests that accountability and responsibility need to be seen as separate, but related, concepts, where the intrinsic difference sits in whether the control factor is imposed by external parties or through the individual’s own sense of self. Having made this distinction, Mulgan suggests that the core concept of accountability has developed over time to cover four areas: “professional” and “personal” accountability; accountability as “control”; accountability as “responsiveness’” and accountability as “dialogue”.

The contractualist nature of the New Zealand public management system tends to reflect the concept of accountability as control, which Mulgan (2000) suggests is a feature of the various institutional checks and balances by which democracies seek to control the action of the governments(p. 556). He suggests that control is not only exercised by external agencies, but also through the interweaving of mechanisms to ensure accountability through the institutions that make up a public management system.

The concept of “accountability as control” is closely related to what Day and Klein (1987) define as managerial accountability; i.e. accountability focused on making those with delegated authority answerable for carrying out agreed tasks according to agreed criteria of performance. The New Zealand public management system would appear to have good systems in place to provide for managerial accountability around the tasks Day and Klein (1987) describe as fiscal / regularity tasks (inputs) and process / efficiency tasks (outputs). However, a key question arises in considering whether the systems adequately provide for accountability for programme / effectiveness tasks (outcomes) or, in fact, the extent to which the system should provide for accountability for outcomes.

It could be that these aspects of accountability can be provided through levers such as those identified by strengthening concepts of personal and professional accountability. This could be promoted within the public management system through the establishment of clear expectations on chief executives and / or through using clan controls to promote the self-controls identified by Dalton (1971) – where morality, professional behaviour and a core understanding of the ethos of the public service will drive individuals to perform to the best of their ability.

6.1.2  Accountability in the New Zealand context

If Ministers are to take a lead on the specification of outcomes, systems to support Ministerial accountability for outcomes might be most appropriate. On the other hand, it could be argued that the parliamentary cycle, parliamentary scrutiny and the general media coverage of governmental activities already provide a significant level of accountability for Ministers – which may in fact create disincentives for the articulation of outcomes.

The decision rights matrix indicated by Figure 2also suggests that chief executives should take a lead in relation to the delivery of outputs and management of inputs. It went on to suggest that chief executives should also be accountable for outputs and the intrinsic behaviours and norms that support a managing for outcomes environment. At these levels it is easier to implement accountability as control systems – in large part because the information flows associated with a closed performance management system have fewer external influences.

At an outcomes level it might be more appropriate, however, to engender a culture where chief executives take “responsibility’, rather than being held to account for the achievement of outcomes. Engendering such a culture would require a clear understanding on the part of “oversight” agencies, Ministers and Parliament about the distinction being made.

6.1.3  Conclusion

In looking towards a managing for outcomes environment, accepting that some part of the jigsaw needs to provide for accountability of executive decisions and public service action, it may prove necessary to provide for a range of different accountability systems that support a range of incentives and sanctions.

A key lesson from evaluations of the current public management system lies in the need for a learning environment to sit alongside these accountability systems. How this might be provided for is discussed in the next section.

6.2  Evaluation#

Good information about the effectiveness of policy and the management challenges facing agencies is important in any public management system – particularly those looking to assess whether interventions have had an anticipated impact on outcomes. However, the New Zealand public management model has poor systems for evaluating and reporting the effectiveness of policy implementation (State Services Commission and Treasury, 2003). Arguably, until this is addressed it is unlikely that the model will not effectively support a managing for outcomes environment – primarily because without learning from effective evaluations agencies will not be in a position to appropriately shift resources from poorly performing programmes and policies to those that are more effective.

6.2.1  Evaluation in the New Zealand public management system#

A recent report on evaluative activity in the New Zealand public management sector, undertaken by representatives of the State Services Commission and Treasury (2003), found that there is: considerable variation in the extent and quality of planning and use of evaluative activity across the New Zealand State sector [p. 14].

Examples of good practice were identified in terms of both deciding what evaluative activity to undertake, and the use of evaluative activity. The authors also suggested that three broad areas need to be addressed if the quality of evaluation across the public sector is to improve. These areas are: a culture of inquiry; poor coordination and prioritisation; and capability.

It could be argued that issues in these areas have arisen because no clear expectations around capability were expressed in the implementation of the New Zealand model – in part because there was a view that evaluation was so obviously important that agencies would build the capability they require in-house[8].

At the same time, central agencies have tended to distance themselves from any overarching analysis of strategic (and cross-sectoral) policy areas, and in some cases anecdotal evidence suggests that Treasury actively discouraged evaluation in the early to mid 1990s by supporting lower cost policy initiatives. This has shifted over time, and new budget bids must now have an evaluation plan associated with them. The fact remains, however, that no central capacity for evaluating policies deemed to be significant to the government exists, and this can create a disconnect between policy, resourcing and accountability decisions. For, how can a system effectively manage for outcomes when it does not evaluate the effectiveness of policies and programmes?

Central agencies have suggested that the issues identified be addressed by central agencies: building a culture of inquiry, initially by setting expectations of agencies and chief executives; promoting more coordinated evaluation activity; and supporting agencies to build the capability they require to undertake evaluation (pp. 32 – 36).

6.2.2  How can evaluative activity be promoted?#

How evaluative activity can be built into a public management system is an issue a number of jurisdictions have attempted, and are attempting, to resolve. Boyle and Lemaire (1999) have identified three “waves” of governments seeking to introduce evaluation to their public management systems (drawing off work by Derlien):

  • first wave – countries sought to institutionalise evaluation to improve effectiveness;
  • second wave – evaluation was seen as a tool of public accountability via the budgetary process;
  • third wave – countries are looking to both the first and second waves with the hope that evaluation will provide for good governance as well as accountability (pp. 1 – 3).

In comparing these three waves Boyle and Lemaire (1999) identified that four basic issues can influence how evaluative activity is built into a public management system. They suggest that these include the availability of sound data systems, a history of social science, the existence of appropriate capability and capacity, and the timeframe available for institutionalisation of evaluative activity. They also suggest that in developing a public management system that provides for evaluation it is important to take wider environmental issues into account, in order to create a balance between supply of and demand for evaluation. This links to the notion that all aspects of a public management system need to align if outcomes are to be efficiently and effectively achieved.

Bringing all of the points made above together, Boyle and Lemaire (1999) have identified a range of criteria to assist in determining how an evaluation regime can be provided for within a public management system.

As the appendix suggests, there would appear to be a need to provide for more demand and supply of evaluative activity across the New Zealand public management system. Demand needs to be driven by both Ministers and the Parliament, with both allowing for evaluation to be undertaken within a learning environment. Agencies that could assist in building this demand may include the Audit Office and central and lead agencies.

It would also appear that all policies and programmes should be subject to evaluative activity of some kind. This may simply involve monitoring of activity levels and simple performance indicators. In other instances, generally where a programme has been funded in a particular way, or where the policy or programme is likely to have a significant impact upon New Zealanders, then a full evaluation of those policies and programmes should be undertaken.

Finally, evaluation should be liked into a range of other public management systems, including the budget process, to allow for the information generated to influence subsequent decisions.

6.2.3  Conclusion#

Evaluative activity – assessing whether interventions have had the anticipated impact – is a key component in a results-oriented system. The New Zealand public management system does not have a strong history of evidence-based decision-making. As noted, recent work by the central agencies has suggested that before evaluative activity can be embedded in the public management system a culture of inquiry needs to be generated. Officials have proposed that, initially, this be generated by central agencies setting expectations and using persuasion.

This section supports the need for generating a change in culture. It also suggests, however, that evaluative activity needs to be anchored in both the legislative and executive branches of government, and that some consideration needs to be given to how cross-cutting issues can be evaluated. It may be that central agencies will need to take a lead role in the latter area. Finally, this section also suggests that evaluative activity should be linked into other aspects of the public management system. In part, this should support the wider processes but it should also ensure that the findings of individual evaluations can be acted upon.

6.3  Performance indicators#

“Good” performance indicators will be integral to effective accountability and evaluation systems. There are, however, inherent difficulties associated with developing “good” indicators. For example, Smith (1995) has suggested that unintended behavioural responses of performance indicators will include:

  • tunnel vision – an emphasis by management on phenomena that are quantified in the performance measure scheme, at the expense of unquantified aspects of performance;
  • suboptimisation – the pursuit of narrow local objectives by managers, at the expense of the objectives of the organisation as a whole;
  • myopia – the pursuit of short term targets at the expense of legitimate long term objectives;
  • measure fixation – an emphasis on measures of success rather than the underlying objectives (e.g. the waiting list);
  • misrepresentation – the deliberate manipulation of data so that reported behaviour differs from actual behaviour;
  • misinterpretation – although in possession of all the facts, bounded rationality might cause the controller systematically to misinterpret them, and to send the wrong policy signals to the agent;
  • gaming – the deliberative manipulation of behaviour to secure strategic advantage; and
  • ossification – organisational paralysis brought about by an excessively rigid system of performance evaluation.

Within a public management system where “accountability” is given a high weighting as a principle these perverse incentives can be strengthened, particularly in terms of seeking to reduce the level of specification used. It may be, however, that the issues could be addressed by providing for the use of “appropriate” indicators (i.e. those defined by the Auditor-General (2002)as those which are relevant to stakeholders; understandable, transparent and traceable; and complete), placing short-term expectations in a longer-term context, encouraging staff to take ownership of data coupled with increased audit, and assessment processes that take into account the nature of the organisation.

6.3.1  Different indicators for different organisations?#

It has been suggested that different forms of organisation will need to consider different performance indicators, and that control levers will often differ depending on the nature of services delivered by an agency [see Controller and Auditor-General, (2002); and Cunningham and Harris, (2001)].

Focusing on results and action controls (i.e. outcomes and outputs respectively), it is possible to see that there could well be some organisations where it might be possible to focus on outcome assessment rather than output assessment, if some of the issues identified about attribution can also be addressed. On the other hand, it could be that the links between output quality indicators and immediate outcome indicators will be so closely linked that they could act as proxies for each other.

6.3.2  Performance indicators for outputs#

Services delivered in the public service vary greatly. They range from the development of policy advice through to assessing biosecurity risks at the border or case managing beneficiaries. It is not, therefore, possible to identify common performance indicators across these outputs.

The OECD (2000) has identified a matrix that could assist in identifying what indications are most appropriate in looking to develop output performance indicators for different types of activity. It identifies possible indicators agencies should focus on when assessing output delivery: quality, quantity, timeliness and cost. In doing so, the matrix suggests that not all indicators will be applicable for all outputs.

For example, where outputs are relatively standardised and production processes are well-articulated an organisation should be able to focus on quantity, cost and timeliness indicators rather than quality indicators. Conversely, where outputs are not able to be standardised, and production processes tend to be less well articulated an organisation is more likely to focus on quality and timeliness indicators.

A good example of the latter would be the provision of policy advice, whilst an example of the former might be the processing of residence applications. The latter example creates some difficulties when we look towards a managing for outcomes environment: in part, because it leads to a question of how a results focus can be engendered if the quality of decisions are not used as an indicator of performance.

In other words, how can those organisations that have become dependant upon the use of performance indicators to drive cultural change now promote a managing for outcomes environment if they are not also coupling performance measures (i.e. for accountability reasons) with outcomes? There is no easy answer here, except possibly to look back to the cultural web outlined in chapter three and seeking change tools within that framework.

Notes

  • [8]In the Australian context evaluation has also been devolved to departments, however it is now suggested that some level of central evaluation may need to be reintroduced.

6.4  Towards managing for outcomes#

The proposed principle for performance measurement suggests that if managers are to effectively manage for outcomes, and be held accountable for the delivery of outputs, sound systems that provide for both the evaluation and assessment of performance are required. These systems should inform each other, but not constrain the ability of managers to learn from mistakes in identifying the most appropriate mix of interventions to assist in the achievement of outcomes.

Previous sections of this chapter have identified different ways in which accountability and evaluation of interventions could be provided for. In essence, both suggest that cybernetic systems allowing for specification of objectives, implementation of programmes, and assessment against appropriate indicators will be appropriate. It is also likely that the two systems should inform each other – particularly if chief executives are to be accountable for undertaking and acting upon evaluations.

These interlinked systems will, however, continue to create tensions for those trying to determine what it is they are and are not accountable for. These tensions might possibly be resolved by being clear about the different expectations on actors, and through the use of performance indicators throughout the system that are designed to meet the different needs associated with measurement of output delivery and achievement of outcomes.

Performance management in a managing for outcomes environment will require provision for both accountability and a learning environment within the one public management system. Providing for this dichotomy will require a significant cultural change within the New Zealand model.

In some areas agencies (and chief executives as a proxy for the agency) will need to be given room to make mistakes, whilst still being held accountability for the decision-making processes they make as a result. In other words, we need to develop a culture that holds Ministers and chief executives responsible for the articulation of outcomes; and accountable for the identification of outputs most likely to achieve those outcomes, efficient delivery of those outputs, evaluation of whether anticipated results have been achieved, and amendment of activities if evaluation shows that results have not been achieved.

Performance indicators for outcomes become important tools assisting in evaluation processes, but indicators of outputs will remain equally important for driving accountability systems within this framework. Looking towards the next chapter, performance indicators for outputs will also be important in providing the information needed to assist in identifying the level of resource required by an agency if it is to effectively and efficiently deliver those outputs.

Development of a culture that provides for both accountability and a learning environment will need to be promoted by the use of “best practice” examples, systems that support evaluation as strongly as those systems that support accountability, and continual reinforcement of the need for both in the public statements of Ministers and senior public servants.

7  Financial management#

In any effective public management system the financial management systems utilised need to easily provide for resources to be aligned with the strategic direction sought, whilst also ensuring that appropriate accountability processes are in place.

The tension between the freedom to manage and accountability principles tends to come into stark relief at this point. Particularly since the tension between the two arises from the core question of “where should decision rights sit”? Other issues also arise, however, particularly once the control systems and rituals and routines of the New Zealand public management system are tested against an agreed position on “decision-rights’.

Specifically, in looking to support a managing for outcomes environment it is important to ask whether: the financial flexibility currently available to chief executives is appropriate; whether the budget process could be amended to provide for more strategic engagement on the part of Ministers; and if the budgetary system can provide for the reallocation of resources to support outcomes.

7.1  Decision rights#

Appropriations systems can be shaped in a variety of different ways to support the level of financial flexibility Ministers and Parliament determine is appropriate. The question that has prompted most debate over the last eighteen months is whether, in looking towards a managing for outcomes environment, the financial management system should now provide for outcomes-based appropriations.

Appropriations can be made at three levels – inputs, outputs and outcomes[9]. Recent work within Treasury has concluded that whilst outcomes-based appropriations would provide a significant level of management freedom, it would be difficult to provide for the other principles that underpin the New Zealand public management system – particularly clear specification of objectives and accountability (Treasury, 2003). Most importantly, it was considered that outcome based approaches would render appropriations ineffective in their primary role of acting as a constraint.

Further, officials considered that outcomes-based appropriations would be unreliable both as a control or an accounting mechanism, and would not be relevant in a performance-based accountability system. A pure outcomes-based appropriations model was perceived as less likely to generate the information critical for outcomes-focused management, particularly the information required to support reallocation of resources within and between Votes and portfolios.

This conclusion is consistent with the proposal in chapter three that decision rights for chief executives should sit at an inputs level, with some flexibility between Ministers and chief executives over how outputs are determined.

7.2  Can an outputs-based focus support managing for outcomes?#

Some issues with the current outputs system do exist however. In moving to a managing for outcomes environment it is important that these issues be addressed so that chief executives and Ministers are aware about what mix of interventions will be most effective and efficient.

7.2.1  Appropriate cost allocation systems

Robust cost allocation systems are integral to the identification of an appropriate cost for outputs. In turn, transparency about the cost implications of different interventions will be necessary management information for those determining what intervention mix is most appropriate in seeking to achieve different outcomes.

The ability of the cost allocation systems within each agency will need to be determined over time, as managers look to identify the costs associated with different interventions. The Managing for Outcomes process potentially provides an avenue for agencies to consider whether they have appropriate cost allocation systems, for they will be unable to properly cost shifts in interventions or capability without adequate information. The Treasury would appear to have a role to play here also, in part by asking agencies to provide robust costings, but possibly more importantly by providing examples of good practice in different areas.

7.2.2  Output class aggregation and appropriations

Outputs are funded through appropriations targeted at specific output classes, organised and aggregated on the basis of a Cabinet minute. Proponents for increased financial flexibility argue that these principles need to be revisited and broadened, and that the principles are now generally honoured in the breach, rather than in reality.

Over 2002 and 2003 the Treasury considered whether more flexibility in the design of outputs classes, and/or the links between output classes and appropriations could be provided. The Public Finance (State Sector Management) Amendment Bill reflects a recommendation that output classes be retained, but that more financial flexibility could be provided by allowing appropriations to incorporate one or more output class – depending upon appropriations structures agreed with Ministers. As a result, chief executives may be able to move resources between output classes covered by one appropriation, and face very few constraints in doing so. This proposed change should provide significant additional flexibility for chief executives seeking to manage for outcomes.

Notes

  • [9]An input appropriation is one where the dollar constraint is expressed in terms of the inputs used in the processes supporting delivery of services. An output appropriation is one where the dollar constraint is expressed in terms of the intended service(s) to be delivered. An outcome appropriation is one where the dollar constraint is expressed in terms of the intended outcome.

7.3  Review of baselines#

During the 1990s there was a clear focus on using baseline management to foster efficiency in the New Zealand public management system – primarily through the policy of a fixed nominal baseline[10], but also through setting expectations that agencies would not seek additional funding to meet capability requirements.

Schick (1996) refers to this process in The Spirit of Reform. At that point practitioners were suggesting that the process had been effective in helping to identify cost-savings within baselines. They were, however, starting to suggest that continued downward pressure on baselines might result in a reduction of capability within agencies. Anecdotal evidence suggests that the majority of savings within votes have now been identified and that is timely to find other mechanisms by which baselines can be managed in a way that minimises costs to the Crown whilst also ensuring that agencies are appropriately resourced.

The Managing for Outcomes process provides an avenue by which regular reviews of baselines could be undertaken, within the context of examining the needs of an agency in working towards the strategic direction identified at the beginning of the process. Such a review should focus on ensuring that the agency will be both effective and efficient moving into the future.

7.4  Reallocation of resources#

The processes that underlie New Zealand’s budgetary system mean that increased spending at the margins is considered at each budget round, rather than a comprehensive revisiting of baselines. Some attempts have been made to provide for value-for-money exercises across the public sector [see OECD, 2003:1].

The most recent attempt to provide for a comprehensive value-for-money review was undertaken in 2002. The results of this process have been identified by officials as mixed – essentially some reallocation of resources was apparent within votes, but there was no significant cross-vote reallocation.

In a report to the OECD (2003) officials identified the key lessons of the 2002 exercise as: the need for strong support of Ministers and departments; a lack of a fiscal imperative will reduce the incentives on actors to assist in finding “savings’; and the lack of good information about the efficiency and effectiveness of interventions will hamper the ability of Ministers and chief executives to make reallocation decisions.

Assuming that a managing for outcomes environment will be characterised by the shifting of resources to support interventions that have been proven to better support the achievement of outcomes it would appear that all of these lessons need to be addressed – potentially be creating incentives for key players.

7.4.1  The creation of incentives

Currently Ministers are unable to retain any savings they identify within their votes. One incentive for reallocation may be created if Ministers are allowed to retain all, or a component, of the savings they identify. In the first instance this would encourage a culture of reallocation within votes. Coupled with more strategic groupings of Ministers it may be that the ability to retain savings could also lead to reallocation of savings within portfolios.

Incentives could also be provided by reassessing the role of multi-year appropriations. The recent central agency review of Managing for Outcomes project support found that chief executives are finding that the annual budget process can be difficult to reconcile with longer-term strategies (Managing for Outcomes Formative Evaluation Team, 2003). If chief executives were provided with more flexibility in this area they might be more comfortable with the concept of looking at different ways in which resources can be allocated to support the achievement of outcomes.

Finally, greater use of multi-year appropriations might assist chief executives in allocating resources ways that support the achievement of articulated outcomes.

Notes

  • [10]Where baseline appropriations are only increased if a Minister successfully bids for additional resources, most often in pursuit of promoting a new policy but occasionally to support input cost increases or capital purchases, and otherwise chief executives are expected to fund increased costs (such as wage increases) through efficiency savings.

7.5  Conclusion#

Looking towards a managing for outcomes environment it would appear that the financial management system needs to be designed in a way that supports both a results-focus and clear accountability. It needs, therefore, to have a greater emphasis on outputs than the strategic management system, whilst providing for the allocation of resources within an outcomes framework. This would suggest that the current system needs to be amended to provide for:

  • proactive baseline reviews – to help move away from incrementalism and to allow for better assessment and management of capability / ownership issues;
  • reallocation of resources in a more responsive manner; and
  • wider appropriations to provide managers with greater flexibility, whilst also retaining sufficient transparency.

The system will need to be supported by good information about the various costs associated with the delivery of services – if only to ensure that the right “price” is paid for different outputs.

Amending the system in this way should provide both Ministers and chief executives with more opportunities to align how they expend resources with the outcomes they pursue. This flexibility will be constrained, however, if Ministers and chief executives do not have sufficiently robust information about the outputs they need to deliver, or the capability required to do so.

Their flexibility will also be constrained if some of the rituals around the budget process, and perceptions of the quality assessment role undertaken by central and lead agencies are not amended or better understood. For example, my recent experience suggests that current budget processes allow for a reasonable amount of gaming as Ministers seek to gain the resources they have identified as necessary for promoting particular initiatives, or for maintaining current capability. The short timeframes available for assessment of budget initiatives, and the information asymmetries that characterise the current process will, arguably, always lead to this tension. Changing the routines, allowing for more engagement early in a process (whether policy development or strategic planning processes) may provide for increased trust between agencies, better information for central and lead agencies required to assess budget initiatives, and more informed allocation of resources.

8  A managing for outcomes model#

The preceding chapters have identified that in developing a managing for outcomes model the New Zealand public management system should:

  • be underpinned by structures that provide for coordination of activity;
  • be based upon a performance management culture that provides for both accountability for outputs and evaluation of impacts on outcomes;
  • be supported by a financial management system that provides managers with financial flexibility and for the reallocation of resources; and
  • be placed within a strategic framework that clearly specifies all objectives sought by Ministers.

The paper suggests that the principles upon which the system is based need to be revisited, and that this could result in the identification of principles such as:

  • Clarity of objectives – The initial element of a management process must be as clear a specification as possible of the objectives which managers are responsible for achieving. This implies that the objectives managers are accountable for achieving, the behaviours expected of managers, and the objectives managers are expected to work towards should all be clearly expressed.
  • Managerial discretion – Managers must be given the power to make the achievement of specified objectives possible, within limits determined by constitutional, risk management and accountability systems. Managers should have the freedom to make resource allocation decisions on a basis that enables the most efficient delivery of outputs, and the effective management of capability, but they should gain Ministerial agreement to the most effective mix of interventions to be used in seeking to achieve outcomes.
  • Accountability – Freedom to manage is not a sufficient precondition for good management. Intrinsic and extrinsic incentives and sanctions must be in place to modify the behaviour of managers to ensure that they do act to meet established objectives rather than pursuing independent goals of their own. For this to be achieved, managers must be accountable for the decisions they make, and those on whose behalf they act must have the means to make that accountability “stick’.
  • Effective assessment of performance – If managers are to effectively manage for outcomes, and be held accountable for the delivery of outputs, sound systems that provide for both the evaluation and assessment of performance are required. These systems should inform each other, but not constrain the ability of managers to learn from mistakes in identifying the most appropriate mix of interventions to assist in the achievement of outcomes.
  • Adequate information flows – Adequate information flows between and within agencies are an imperative component in a system seeking to provide for whole-of-government coordination, performance management and assessment of accountabilities. Information processes that support all three aspects will be required in a public management system seeking to manage for outcomes.

Further, the paper reflects a view that in providing for this model outputs and outcomes will need to play differing roles in the different parts of the system. Specifically, the strategic management system should be focused on outcomes, with clear reflections of the links between the outcomes and the interventions to be used and the capability required by the agency in seeking to achieve the specified outcomes. The financial management system should be based upon outputs, whilst supporting (re)allocation of resources in order to support outcomes. Finally, the performance management system should provide for the assessment, monitoring and evaluation of performance against both outcomes and outputs, albeit with different foci for each.

What does this mean for the institutional design of the New Zealand public management system? The following sections outline one possible picture of how the system might operate.

8.1  Ministerial leadership#

Ministers would need to promote change by modelling the behaviours they expect of the public service. This could extend to:

  • structuring the ways in which Ministers work together so that senior Ministers have clear responsibilities for setting strategies and budget allocations in sectoral areas;
  • providing for active promotion of reallocation of resources both within votes and portfolios, but also across votes if strategic processes indicate that there may be a more effective way of achieving outcomes;
  • promoting the evaluation of interventions, with the goal of providing for improved policy advice on how outcomes may best be achieved;
  • engaging more proactively on the establishment of output performance indicators and the decisions involved in moving resources between appropriations (i.e. more engagement in the purchase role); and
  • engaging more proactively in ensuring that agencies have appropriate capability mixes to support the achievement of outcomes (i.e. more engagement in the ownership role).

Ministers will need to be supported in undertaking some of these roles, possibly through central agencies providing more direct advice and support where requested.

8.2  Structures#

The structures that support the New Zealand public sector model should be sufficiently flexible to promote improved coordination and alignment of activity across the public service. This could be supported through some revisiting of the machinery of government principles, and be provided for through: restructuring of Ministerial relationships, as referred to above; increased use of the “lead” agency concept; and continued innovation around ways to support “joined-up” government at the front-line.

8.3  Strategic management#

This paper has suggested that the strategic management system needs to be focussed on outcomes, and identification of the links between outcomes and outputs, identification of the capability an agency requires; and providing for ongoing review of the activities that support the strategic direction of both agencies and the government.

The Managing for Outcomes process appears to provide a sound basis for the ongoing identification of these factors. It could also prove to be a sound core for other component parts of the wider system to be wrapped around.

More specifically, the Managing for Outcomes process could allow for regular strategic reviews of agencies or sectors (possibly every three to five years) with the aim of:

  • specifying outcomes for the agency or sector for the coming three to five years;
  • identifying capability and policy issues that need to be addressed in working towards those outcomes;
  • considering possible financial management changes to support an outcomes framework within the agency or sector, and ongoing funding arrangements for the period;
  • identifying appropriate performance indicators (to be used in holding the chief executive to account for service delivery and wider organisational or individual performance areas);
  • identifying wider policy areas to be evaluated over the period leading into the next review; and
  • identification of respective roles and responsibilities over the period leading into the next review.

Such a process should provide for close dialogue between departments, central agencies and Ministers. It should also provide for an identification of the ways in which ongoing risks will be managed over the period, and the role central agencies will play in relation to the department or sector. This could result in the identification of areas where central agencies will be obliged to undertake work for, or interact with, the agency or sector – thereby underlining the relational aspects that will have to support a managing for outcomes environment.

8.4  Financial management#

In addition to considering how financial decisions can be made in ways that support managing for outcomes, it may be that the financial management system should be amended to provide for:

  • proactive baseline reviews – to help move away from incrementalism and to allow for better assessment and management of capability and ownership issues;
  • reallocation of resources in a more responsive manner; and
  • wider appropriations to provide managers with greater flexibility, whilst also retaining sufficient transparency.

Where the strategic management system will primarily be focused on outcomes the financial management system needs to be based on outputs. This will occur not only because of the attribution issues associated with needing to report to Parliament on the ways in which resources have been used, but also because robust information about the cost of outputs is required to assist in reallocation and setting baselines that allow for both service delivery and ongoing development and maintenance of capability.

8.5  Performance management#

Performance management in a managing for outcomes environment will require provision for both accountability and a learning environment within the one public management system. More specifically, it will be important to develop a culture that holds Ministers and chief executives responsible for the articulation of outcomes; and accountable for the identification of outputs most likely to achieve those outcomes, efficient delivery of those outputs, evaluation of whether anticipated results have been achieved, and amendment of activities if evaluation shows that results have not been achieved.

9  Conclusion#

As Johnson and Scholes’ (2001) cultural web shows, cultural change is best effected by looking at a range of different interventions and ensuring that they support each other. The recommendations made above fulfil many of the aspects identified in the cultural web. Specifically, they provide for the power structures, organisational structures, control systems and rituals and routines in the New Zealand public management system to be amended to allow for a greater focus on outcomes.

It will be important that the stories and symbols of the system also support this cultural shift. The latter is already occurring – in part through Ministers reinforcing the cultural shift in public statements (although greater buy-in from Ministers would promote the change further), and through the change in rhetoric and behaviour on the part of central agencies. The latter will, possibly, be most important however – the stories that surround the system will need to provide evidence that an outcomes-focus on the part of agencies will be supported.

Shifting the culture of the New Zealand public management system towards one focused on achievement of outcomes will be a slow process. It is not a process that can rely purely on shifts within the strategic management process – as the cultural web suggests there are a number of other aspects that also need to be addressed if the overall paradigm is to be successfully altered. This paper has identified one possible picture of a managing for outcomes system in the New Zealand context, with the hope that it will lead to further debate and the development of an image that all actors within the system can easily work towards.

References#

Advisory Group (2001) “Review of the Centre – Presented to the Ministers of State Services and Finance.” Wellington, State Services Commission, November.

Alford, John (1993) “Towards a new public management model: beyond ‘managerialism’ and its critics.” Australian Journal of Public Administration 52(2): 135–148.

Boston, Jonathan, John Martin, June Pallot and Pat Walsh eds. (1996) Public Management: The New Zealand Model (Auckland: Oxford University Press).

Boyle, Richard and Donald Lemaire (1999) Building Effective Evaluation Capacity: Lessons from Practice (New Brunswick: Transaction Publishers).

Brady, Natalie (2002) “Striking a Balance: Centralised and Decentralised Decisions in Government.” Wellington, New Zealand Treasury, Working Paper 02/15, September. www.treasury.govt.nz

Cangiano, Marco (1996) “Accountability and Transparency in the Public Sector: The New Zealand Experience.” International Monetary Fund, IMF Working Paper No. 96/122, November.

Comptroller and Auditor-General (2001) “Joining Up to Improve Public Services.” London, National Audit Office, November.

Controller and Auditor-General (2002) “Reporting Public Sector Performance: 2nd edition.” Wellington, Controller and Auditor-General, January.

Clark, Tim (2002) “New Labour’s Big Idea: Joined-up Government.” Social Policy and Society 1(2): 107-117.

Cunningham, Gary M. and Jean E. Harris (2001) “A Heuristic Framework for Accountability of Governmental Subunits.” Public Management Review 3(2) 145 – 165.

Dalton, Gene W. (1971) “Motivation and Control in Organisations.” in Dalton and Laurence (eds) Motivations and Control in Organisations (Homewood, Ill:Irwin): 1 – 35.

Day, Patricia and Rudolf Klein (1987) Accountabilities: Five public services (London: Tavistock Publishers).

Economics and Strategy Group (2003) “Departmental Uptake of the Managing for Outcomes Initiative: An independent evaluation by Economics & Strategy Group for the interdepartmental Managing for Outcomes Steering Group.” Wellington, State Services Commission, August. www.ssc.govt.nz

Etzioni, Amitai (1961) A Comparative Analysis of Complex Organisations (New York: Free Press).

Flinders, Matthew (2002) “Governance in Whitehall.” Public Administration 80(1): 51 – 75.

Grimes, Arthur, Alan Jones, Roger Procter and Grant Scobie (2001) Economics for Policy: Expanding the Boundaries: Essays by Peter Gorringe (Wellington, Institute of Policy Studies).

Harmon, M.M. and R.T Meyer (1986) Organisation Theory for Public Administration (Glenview, Ill: Scott, Foresman and Co).

James, Colin (1992) New Territory: The transformation of New Zealand 1984 – 92 (Wellington, Bridget Williams Books Ltd).

Johnson, Gerry and Kevan Scholes (2002) Exploring Corporate Strategy: Sixth edition (Edinburgh, Pearson Education Limited)

Kelly, Gavin and Stephen Muers (2002) “Creating Public Value: An analytical framework for public service reform.” London, Strategy Unit, Cabinet Office, www.strategy.gov.uk

Mallard, Hon. Trevor (2000:1) “Speech to the Public Service Senior Managers’ Forum.” 8 March. www.beehive.govt.nz

- (2000:2) “Launch of the Price Waterhouse Cooper Public Sector Leadership Best Practice Survey Report.” 14 June. www.beehive.govt.nz

- (2000:3) “Chen and Palmer Seminar.” 4 July. www.beehive.govt.nz

- (2001) “Speech to Public Service Senior Managers’ Forum.” 16 August. www.beehive.govt.nz

- (2002) “Speech to Public Service Senior Managers’ Forum.” 24 September. www.ssc.govt.nz

Mulgan, Richard (2000) “Accountability: An ever-expanding concept?” Public Administration 78(3): 555 – 573.

Norman, Richard (2003) Obedient Servants?: Management freedoms and accountabilities in the New Zealand public sector (Wellington: Victoria University Press)

OECD (2000) “The OECD Outputs Manual.” Paper presented at the 21st Annual Meeting of Senior Budget Officials, Paris, France, 29-30 May.

- (2002:1) “Governing for Results.” Paper presented to the 25th session of the Public Management Committee, Paris, 21–22 March.

- (2002:2) “Public Sector Modernisation: A New Agenda.” Paper presented to the 26th session of the Public Management Committee, Paris, 30–31 October.

- (2003) “Reallocations Survey: Response from New Zealand.” Paper presented to the 1st meeting of the Expert Group looking at Budgetary Institutions and Reallocation, Paris, 18–19 February.

Ouchi, William G. (1980) “Markets, Bureaucracies, and Clans.” Administrative Science Quarterly 25: 129 – 141.

Petrie, Murray and David Webber (2001) “Review of Evidence on Broad Outcome of Public Sector Management Regime.” Wellington, New Zealand Treasury, Working Paper 01/06, www.treasury.govt.nz

Poister, H. and G. Streib (1999) “Strategic management in the public sector: Concepts, models and processes.” Public Productivity and Management Review, 22(3): 308 – 325.

Ring, Peter Smith and James L. Perry (1985) “Strategic Management in Public and Private Organisations: Implications of Distinctive Contexts and Constraints.” Academy of Management Review 10(2): 276 – 286.

Ryan, Bill (2003) “Learning MFO: Developments in managing for outcomes: A Queensland case study.” Brisbane, Institute of Public Administration Queensland, Pre-publication version.

Schick, Allen (1996) “The Spirit of Reform: Managing the New Zealand State Sector in a Time of Change.” Wellington, State Services Commission and the Treasury, August. www.ssc.govt.nz

Scott, Graham (2001) Public Management in New Zealand: Lessons and Challenges (Wellington: New Zealand Business Roundtable).

Scott, Graham, Peter Bushnell and Nikitin Sallee (1990) “Reform of the Core Public Sector: New Zealand Experience.” Governance: An International Journal of Policy and Administration 3(2): 138 – 167.

Scott, Graham, Ian Ball and Tony Dale (1997) “New Zealand’s Public Sector Management Reform: Implications for the United States.” Journal of Policy Analysis and Management 16(3): 357 – 381.

Simons, Robert (1995) “Control in an Age of Empowerment.” Harvard Business Review March-April.

Smith, Peter (1995) “On the Unintended Consequences of publishing performance Data in the Public Sector.” International Journal of Public Administration 18 (2 & 3): 277 – 310.

State Services Commission (1993) “New Zealand Public Sector Reform.” Wellington, State Services Commission.

- (1998:1) “Assessment of the State of the New Zealand Public Service.” Wellington, State Services Commission, Occasional Paper No. 1, October, www.ssc.govt.nz

- (1998:2) “The State Ten Years on from the Reforms.” Wellington, State Services Commission, Occasional Paper No. 6, October, www.ssc.govt.nz

- (1999) Crown Entities: Organisational Design.” Wellington, State Services Commission, Occasional Paper No. 21, September, www.ssc.govt.nz

State Services Commission and The Treasury (2003) Doing the Right Things and Doing Them Right: Improving Evaluative Activity in the New Zealand State Sector (Wellington, State Services Commission), September, www.ssc.govt.nz

Steering Group for the Managing for Outcomes Roll-out 2004/05 (2003) “Managing for Outcomes: Guidance for Departments.” Wellington, State Services Commission, www.ssc.govt.nz

Steering Group Review of State Sector Reforms (1991) “Review of State Sector Reforms.” (the “Logan Report”) Wellington, State Services Commission, November.

The Treasury (1987) Government Management, Brief to the Incoming Government, Volume 1 (Wellington, The Treasury).

- (2003) “Appropriations and the New Zealand Public Management System.” Unpublished internal Treasury paper.

Upton, Simon (1998) “Opening Address.” Speech to Public Sector Senior Management Conference, Wellington, 15 October.

Wyman, Miriam (2001) “Thinking about Governance: A Draft Discussion Paper.” The Commonwealth Foundation Citizens and Governance Programme, www.commonwealthfoundation.com

Yates, Douglas (1985) “Management in Public and Private Organisations: Similarities and Differences.” in Ott, Hyde and Shafritz (eds) Public Management: The essential readings (Chicago: Lyceum Books):39 – 58.

Appendix - Analysis of evaluation in the New Zealand public management system#

Appendix Table - Analysis of evaluation in the New Zealand public management system

Anchoring the evaluation regime

Should the evaluation regime be anchored in the legislative or executive branch, or both, and where within those branches?

Evaluative activity should be anchored in both the legislative and executive branches, primarily because the two branches of government will be looking to evaluate government activity from different perspectives – the legislature will essentially look to develop some understanding of the effectiveness of government policies and programmes from an accountability perspective, whilst the executive should be looking be looking to build any learnings into strategic management and policy development processes. 

Anchoring evaluation capacity within organisations

What are the advantages and disadvantages of (a) centralised versus decentralised evaluation capacity and (b) using internal versus external evaluators?

The agencies within which evaluative activity could be anchored will differ depending upon the nature of the activity:

  • the Office of the Auditor-General and Comptroller could be best placed to undertake, or commission, evaluation of programmes for Parliament;
  • central and lead agencies are likely to be best placed to commission evaluation of cross-sectoral policies and programmes; and
  • departments will be best placed to commission evaluation of agency specific programmes.

Evaluation coverage

What kind of activities should be covered by evaluation?

All government policies and programmes should be subject to evaluation.  This is not to suggest that they should all be evaluated however.  Those programmes most likely to be subject to evaluation should be selected according to a mix of criteria that could include consideration of: the basis on which the programme has been funded – if on a pilot basis only then continued funding should be dependant upon a positive evaluation; the significance of the policy or programme to wider governmental policies; and whether public comment suggests that there are significant issues associated with implementation of policy in the area (for example, current public concerns about the impacts of migration in the Auckland area could lead to some evaluation of immigration and settlement policies).

Using evaluation in decision-making

What are the key domains of use, and how can evaluation utilisation be improved?

As the criteria outlined above suggest evaluation should be linked to a range of other public management systems – including the budget process (where reallocation processes could be supported by the information generated); audit (where Parliament has significant concerns about the performance of a particular policy); strategic management systems (so that strategies can be amended where appropriate); and the policy development process.

Fostering demand

What are the most effective ways of fostering demand for evaluation activity?

Demand for evaluation will increase if the end-users – policy analysts, managers, Ministers and parliamentarians – can see how they can use the information generated to support decision- making.  In some cases this can be prompted through the encouragement of central agencies, and setting expectations on chief executives.  A related issue is that of fostering supply – which will in turn have an impact on demand (particularly by chief executives as both suppliers and end-users).  If evaluative activity is only utilised to support an accountability regime it is difficult to envision any burgeoning of evaluative activity within agencies in the public sector.