Providing final quality assurance

Material required

The reviewer should be provided with the RIS, including the completed disclosure statement. They may ask for material to test statements made in the RIS, e.g. evidence that has been cited or referenced, assumptions and calculations underlying the cost benefit analysis, or the summary of stakeholder submissions. This material should be provided, so that the reviewer can be assured that the analysis is correct and robust.

Applying the QA criteria

The QA criteria should be used as a basis for the formal QA assessment. The first three criteria are the most important in terms of the substance of the analysis, and more weight should be placed on these aspects:

• Complete - ensure that all the required information is provided in the RIS.
• Convincing - this criterion relates to the analytical framework that has been employed, and the level and type of analysis that has been undertaken. Chapter 2 of the Regulatory Impact Analysis handbook should be used as a guide to assessment against this dimension of quality.
• Consulted - Chapter 4 of the Regulatory Impact Analysis handbook sets out the requirements for efficient and effective consultation. It is important that the RIS does not just state what consultation has been undertaken, but also explains the nature of any issues raised or views expressed by stakeholders, and how these have been taken into account in the development of the final proposal.

The final criterion - clear and concise - relates to the presentation of material in the RIS. Information should be succinct and in plain English, to enable decision-makers to easily understand the issues and trade-offs associated with the choices they are making. The RIS should also be sufficiently clear so the general public can understand the basis on which government decisions have been taken. It may be more helpful to present some information in tabular or diagrammatic form, and flexibility of presentation is permitted.

The disclosure statement

The purpose of the agency disclosure statement is to provide agency accountability for the quality of their policy advice and to allow the person responsible for preparing the RIS to explain any constraints they faced in undertaking this analysis (e.g. key gaps, assumptions, dependencies, caveats or uncertainties).

The reviewer should take the information in the disclosure statement into account when forming a QA opinion. The main issue this raises is to what extent any constraints identified should be considered a mitigating factor with respect to the quality of the analysis. Judgement will be required on a case-by-case basis, but in general, reviewers should consider whether the constraint is a genuine analytical constraint and whether it was reasonably possible to overcome it.

For instance, a genuine analytical constraint may exist when there are no existing data e.g. on the scale of the policy problem (and it is simply not possible to obtain or gather such data). In this case, the RIS should note the uncertainty and risks this raises, but the QA opinion can be subject to the constraint. Alternatively, the QA opinion may determine that the RIS does not meet the “convincing” criterion, but note that these deficiencies have been identified.

Another example is when the portfolio Minister has directed that analysis be undertaken only on particular policy options (and other feasible options are taken off the table prior to the preparation of the RIA/RIS). In this case, the reviewer may state whether the analysis is as good as could be expected in light of these constraints, but nonetheless only partially meets the quality assurance criteria. In such a situation, the agency's disclosure statement should also identify the alternative options that they would have analysed, had they been able to consider the full set of feasible options.

Preparing a QA statement

The QA statement for the Cabinet paper needs to:

• State whether the RIA requirements have been met/partially met/not met; and
• Identify any issues raised by the reviewer in relation to any of the dimensions of quality specified in the QA criteria and guidance.

The purpose of this statement is to provide decision-makers with advice on the quality of the information in the RIS and the reliance they should place on the underlying analysis. It is not a comment on the efforts of the authoring agency.

In practice, judgement is required in deciding which category a RIS falls into (particularly when choosing between “meets” and “partially meets” and between “partially meets” and “does not meet”). The reviewer needs to consider the context of the decisions being taken (e.g. whether they are in principle or final policy decisions) and any constraints that have been identified in the Agency Disclosure Statement that may compromise the quality of the analysis.

In general, we recommend that “does not meet” is used when RIS falls short of the standards on more than one aspect (e.g. several components of the required information are absent or of inadequate quality). “Partially meets” may be appropriate when the RIS meets the quality standards on most dimensions, but there is one particular area of deficiency that should be highlighted. Some illustrative examples are provided in Annex D: Illustrative QA statements.

The QA statement must use the term “meets”, “partially meets” or “does not meet” the RIA requirements, because Cabinet Office will reflect this in the top sheet they prepare for the Cabinet paper.

There is no set format for the information in the second bullet point, as this will depend on the particular circumstances of the individual RIS. However, the statement should:

• Be succinct;
• Provide an indication as to the reliance that can be placed on the RIS, as a basis for informed decision-making;
• Relate the issues raised to the relevant QA criterion; and
• Explain any gaps between the analysis in the RIS and what they would have expected to see, and the implications or risks this poses. That is, what further analysis could or should have been undertaken, and/or what risk mitigation can be done (e.g. additional, targeted consultation).

Template statement

The overall opinion is to be included in the Cabinet paper under the heading Quality of the Impact Analysis. The statement will include the following:

“[Name of team or position of person completing opinion - either from authoring agency or RIAT] has reviewed the Regulatory Impact Statement (RIS) prepared by [name of agency] and associated supporting material, and

[Statement on whether the reviewer considers that the information and analysis summarised in the RIS meets/does not meet/partially meets the quality assurance criteria]

[Comment on any issues that have been identified in relation to any of the dimensions of quality specified in the quality assurance guidance].”