2 Obtaining Quality Assurance (QA)

Independent quality assurance must be undertaken on all RISs. The criteria for assessing quality are the same regardless of whether the RIS is assessed by the authoring agency or by RIAT.

2.1  Independent quality assurance

If the quality assurance is undertaken by the agency, it must be done by a person or group not directly involved in preparing the RIS and nominated by the agency's Chief Executive. A statement on the quality of the impact analysis will be provided in the Cabinet paper (see below).

The reviewer (whether RIAT or the agency) will distinguish between the RIS (and the analysis it summarises) and the actual regulatory proposal. The role of the reviewer is not to provide advice on the merit of the regulatory proposals, but on the quality of the RIS. The quality assurance should be undertaken before final advice is provided to the portfolio Minister.

2.2  Early warning

Ministers have expressed a strong preference for early warning where a significant RIS or discussion document is unlikely to meet the RIA requirements and where a RIS is required but will not be prepared.

Early warning is the primary responsibility of the agency responsible for preparing the RIS or discussion document, and needs to be given sufficient priority by agency officials. Further, for any significant RIS or discussion document that has not met, or in the view of the RIA team is unlikely to meet the RIA requirements, Treasury may advise the Minister of Finance and the Minister for Regulatory Reform, including whether these Ministers could usefully bring any issues to the attention of the portfolio Minister or other colleagues.

2.3  QA criteria

The QA criteria (see Part 5) should be used as a basis for the formal QA assessment. The first three criteria are the most important in terms of the substance of the analysis, and more weight should be placed on these aspects:

• Complete - Ensure that all the required information (see Annex 4.1) is provided in the RIS.
• Convincing - This criterion relates to the analytical framework that has been employed, and the level and type of analysis that has been undertaken. The Undertaking RIA (see Part 2) section of the Handbook should be used as a guide to assessment against this dimension of quality.
• Consulted - The Effective Consultation section (see Part 3) of the Handbook sets out the requirements for consultation. It is important that the RIS does not just state what consultation has been undertaken, but also explains the nature of any issues raised or views expressed by stakeholders, and how these have been taken into account in the development of the final proposal.

The final criterion - clear and concise - relates to the presentation of material in the RIS. Information should be succinct and in plain English, to enable decision-makers to easily understand the issues and trade-offs associated with the choices they are making. The RIS should also be sufficiently clear so the general public can understand the basis on which government decisions have been taken. It may be more helpful to present some information in tabular or diagrammatic form, and flexibility of presentation is permitted.

More guidance on applying the QA criteria can be found on in the section Providing QA. They should be used in conjunction with the overview of required information (see Annex 4.1) for the RIS and the guidance on impact analysis (see Part 2) provided in this handbook, including consultation (see Part 3) requirements.

2.4  Features of a robust quality assurance process

The process for achieving robust quality assurance is not prescribed, as agencies will need to tailor processes according to their own structures, policy processes and available resources. However, the following characteristics should be considered:

• The reviewer is nominated by the agency's Chief Executive and provides the opinion on quality of the impact analysis in the Cabinet paper. This person should therefore have sign-out authority and have suitable capability - including a thorough understanding of the RIA regime, and sufficient experience and expertise in policy analysis.
• The reviewer should be provided with early warning and have sufficient time to undertake quality assurance (ideally 5-10 working days).
• Time should be allowed for iteration with the reviewer, so that comments and queries can be addressed.

The reviewer should be provided with the RIS, including the completed disclosure statement. They may ask for material to test statements made in the RIS, eg, evidence that has been cited or referenced, assumptions and calculations underlying the cost benefit analysis, or the summary of stakeholder submissions. This material should be provided, so that the reviewer can be assured that the analysis is correct and robust.

When the agency is responsible for providing the quality assurance, it can be acquired in different ways:

• Some agencies have internal RIS review panels, comprising people from different policy teams.
• A permanent panel may not be possible in smaller agencies. Another option is to identify a pool of experienced people who can be drawn on, on an ad hoc basis. This pool could be comprised of people from other agencies (ie, not just internally sourced).
• For some large or complex pieces of work, or for small agencies where conflicts of interest are difficult to avoid, it may be appropriate to outsource independent quality assurance such as from a private sector consultant or subject matter expert (eg, academic). In these circumstances, it is important that the reviewer is familiar with the government's RIA requirements and the quality assurance criteria.

In addition to the formal quality assurance, a further test of whether the RIS is clear and well-communicated is to have someone completely uninvolved with the subject matter review the RIS. This can help ensure that the RIS be will easily understood by audiences with perhaps little or no prior history of the issues, including Ministers (hence assisting decision-making), and also the general public when it is published (thus meeting the transparency and accountability functions of the RIS).

2.5  Regulatory proposals that do not meet the RIA requirements

For any regulatory proposal that does not meet the RIA requirements, Treasury may advise the Minister of Finance and the Minister for Regulatory Reform. This includes regulatory proposals:

• for which a RIS was required but not prepared, or
• for which the RIA (as summarised in the RIS) is deficient.

For proposals that do not meet the criteria for RIAT involvement, this advice may be provided by the relevant Treasury policy team.

For proposals that only partially meet the RIA requirements, reasons should be given in the Cabinet paper to explain the key deficiencies and risks for Cabinet's decision.

2.6  Significant proposals that do not meet the RIA requirements

If a regulatory proposal meets the criteria for RIAT involvement, but does not meet the Government's RIA requirements and is ultimately agreed to by Cabinet, then it will be subject to a post-implementation review. The nature and timing of this review are to be:

• agreed by the lead agency in consultation with Treasury, and
• signed off by the responsible Minister, in consultation with the Minister of Finance and the Minister for Regulatory Reform.

2.7  Further guidance

More detailed advice on undertaking independent quality assurance is provided in Part 5.