The Treasury

Global Navigation

Personal tools

Treasury
Publication

Guide to Cabinet's Impact Analysis Requirements

11.4. What Quality Assurance involves

There are two aspects to Quality Assurance: assessing and assisting. Formal assessment of the final Regulatory Impact Assessment is a mandatory requirement, and represents the core role of assessors. This assessment is made using the Quality Assurance criteria. But assessors can also provide assistance to the writer, to help lift the quality of the final product. There are choices around the degree to which assessors get involved in the earlier stages of the policy development process, illustrated in the box below.

Degree of Quality Assurance involvement

Degree of Quality Assurance involvement.

Formal assessment (required)

The core role involves assessing the final Regulatory Impact Assessment. Based on our experience, we strongly recommend that you plan for at least one iteration of the Regulatory Impact Assessment. This means the Quality Assurance assessors would provide comments on at least one draft of the Regulatory Impact Assessment.

Formal assessment is required for Regulatory Impact Assessments provided for final policy decisions, as well as those that are to be submitted to Cabinet to support any in principle or intermediate policy decisions.

However, the Quality Assurance for interim Regulatory Impact Assessments will need to be tailored to the circumstances, taking into account the stage of policy development, the nature of the decision being sought, and the level of analysis possible. At early stages of the policy process, it may not be feasible to prepare a comprehensive Regulatory Impact Assessment, so the Quality Assurance will need to reflect these constraints.

Both the Quality Assurance assessors and the people responsible for the preparation of the Regulatory Impact Assessment should be clear that the assessors are concerned solely with the quality of the underlying analysis and its presentation in the Regulatory Impact Assessment. The role of assessors is not to assess the merits of any policy optionsconsidered in the Regulatory Impact Assessment — the assessors do not provide a view on whether theproposal is a good idea. In practice it can sometimes be hard to draw a firm distinction between the quality of the Regulatory Impact Assessment and the quality of the proposal. But essentially the assessors need to determine whether Ministers have enough information, of sufficient quality, to make an informed decision.

Material required

As well as the final Regulatory Impact Assessment, the Quality Assurance assessors may want to ask for material to test statements made about the Impact Analysis. For example, you may wish to view evidence that has been cited or referenced, assumptions and calculations underlying the cost benefit analysis, or the summary of stakeholder submissions.

Applying the Quality Assurance criteria

The criteria for assessing the Regulatory Impact Assessment are the same regardless of who undertakes Quality Assurance. All four dimensions must be assessed for each element of the Impact Analysis framework. The associated questions, however, are indicative and do not purport to be exhaustive.

The first three criteria are the most important in terms of the substance of the analysis, and more weight should be placed on these aspects:

Complete:

  • Is all the necessary information in the Regulatory Impact Assessment, as set out in the relevant template?

Convincing:

  • Is the analysis accurate, robust and balanced?
  • Are the analysis and conclusions supported by the analytical framework, and a commensurate assessment of costs and benefits and supporting evidence?
  • Do assumptions make sense?

Consulted:

  • Does the Regulatory Impact Assessment show evidence of efficient and effective consultation with stakeholders, key affected parties and relevant experts?
  • Does it show how any issues raised have been addressed or dealt with?

Clear and concise:

  • Is the material communicated in plain English?
  • Is the Regulatory Impact Assessment of an appropriate length?

An important issue for consideration relates to how far any constraints identified - such as a lack of time for consultation, or gaps in the available data - should be considered a mitigating factor with respect to the quality of the Impact Analysis. Judgement will be required on a case-by-case basis. In general, Quality Assurance assessors should consider whether the significance of the constraint is such that it impairs the ability of Cabinet to fully rely on the analysis in the Regulatory Impact Assessment for its decision making.

For instance, a genuine analytical constraint may exist when there are no existing data. For example, on the scale of the policy problem and it is simply not possible to obtain or gather such data.

There are two possible ways in which this situation can be handled:

  • the Regulatory Impact Assessment would note the uncertainty and risks this raises, and the Quality Assurance assessment could be subject to the constraint
  • the Quality Assurance assessment might determine that the Regulatory Impact Assessment does not meet the “convincing” criterion, but note that these deficiencies have been identified.

There is a “line” between these two forms of Quality Assurance statement and it is a matter of judgement on a case-by-case basis to discern where that line is.

Another example is when the portfolio Minister has directed that analysis be undertaken only on particular policy options (and other feasible options are taken off the table prior to the preparation of the Regulatory Impact Assessment). In this case, the Quality Assurance assessors may state whether the analysis is as good as could be expected in light of these constraints, but nonetheless only partially meets the Quality Assurance criteria. In such a situation, the Regulatory Impact Assessment should also identify the alternative options that they would have analysed, had they been able to consider the full set of feasible options.

Page top