6 Obtaining quality assurance
6.1 Independent quality assurance
Independent quality assurance must be undertaken on all RISs. The criteria for assessing quality are the same regardless of whether the RIS is assessed by the authoring agency or by RIAT. If the quality assurance is undertaken by the agency, it must be done by a person or group not directly involved in preparing the RIS and nominated by the agency's Chief Executive. A statement on the quality of the impact analysis will be provided in the Cabinet paper (see Section 7.2).
The reviewer (whether RIAT or the agency) will distinguish between the RIS (and the analysis it summarises) and the actual regulatory proposal. The role of the reviewer is not to provide advice on the merit of the regulatory proposals. So, whatever the reviewer's assessment of the quality of the RIS, it does not represent a view on the merits or otherwise of the options proposed.
Ideally, the quality assurance should be undertaken before final advice is provided to the portfolio Minister.
6.2 Quality assurance criteria
The dimensions of quality assurance against which RISs are to be assessed are set out in Annex 3. They should be used in conjunction with the overview of required information for the RIS and the guidance on impact analysis provided in this handbook, including consultation requirements.
6.3 Features of a robust quality assurance process
The process for achieving robust quality assurance is not prescribed, as agencies will need to tailor processes according to their own structures, policy processes and available resources. However, the following characteristics should be considered:
- The reviewer is nominated by the agency's Chief Executive and provides the opinion on quality of the impact analysis in the Cabinet paper. This person should therefore have sign-out authority and have suitable capability - including a thorough understanding of the RIA regime, and sufficient experience and expertise in policy analysis.
- A certain level of independence is required. This means that the person responsible for the preparation of the RIS should not undertake the quality assurance.
- The reviewer should be provided with early warning and have sufficient time to undertake quality assurance (ideally 5-10 working days).
- Time should be allowed for iteration with the reviewer, so that comments and queries can be addressed.
The reviewer should be provided with the RIS, including the completed disclosure statement. They may ask for material to test statements made in the RIS, eg, evidence that has been cited or referenced, assumptions and calculations underlying the cost benefit analysis, or the summary of stakeholder submissions. This material should be provided, so that the reviewer can be assured that the analysis is correct and robust.
When the agency is responsible for providing the quality assurance, it can be acquired in different ways:
- Some agencies have internal RIS review panels, comprising people from different policy teams.
- A permanent panel may not be possible in smaller agencies. Another option is to identify a pool of experienced people who can be drawn on, on an ad hoc basis. This pool could be comprised of people from other agencies (ie, not just internally sourced).
- For some large or complex pieces of work, or for small agencies where conflicts of interest are difficult to avoid, it may be appropriate to outsource independent quality assurance such as from a private sector consultant or subject matter expert (eg, academic). In these circumstances, it is important that the reviewer is familiar with the government's RIA requirements and the quality assurance criteria.
In addition to the formal quality assurance, a further test of whether the RIS is clear and well-communicated is to have someone completely uninvolved with the subject matter review the RIS. This can help ensure that the RIS be will easily understood by audiences with perhaps little or no prior history of the issues, including Ministers (hence assisting decision-making), and also the general public when it is published (thus meeting the transparency and accountability functions of the RIS).
