Laser Audit of the Recipient Audit Framework

Laser Audit of the Recipient Audit Framework PDF Version (313 KB)

Table of Contents

• Executive Summary
• 1. Introduction
  • 1.1 Background
  • 1.2 Audit Objective and Scope
  • 1.3 Methodology and Audit Approach
  • 1.4 Conformance with Professional Standards
• 2. Findings and Recommendations
  • 2.1 Finding 1
  • 2.2 Finding 2
  • 2.3 Finding 3
• 3. Conclusion
• 4. Management Action Plan

June 2021

© Her Majesty the Queen in Right of Canada, 2021
Cat. No.: PS4-282/2021E-PDF
ISBN: 978-0-660-39487-9

This material may be freely reproduced for non-commercial purposes provided that the source is acknowledged.

Executive Summary

The objective of this audit was to assess the design and implementation of Public Safety’s Recipient Audit Framework to support: the assessment of recipient compliance with funding agreements; cost-effective oversight and control of grants and contributions program expenditures; and, risk based administration requirements on recipients.

Why this is important

In 2019-20, grants and contributions represented 81.5% of Public Safety’s total expenses of $1.053 billion.^Footnote1 Oversight of the use of these public funds is important to support sound management, control and accountability. One source of such oversight is a recipient audit, which is an independent assessment to provide assurance on a recipient's compliance with a funding agreement. However, due to finite resources, it is not possible nor efficient to audit all recipients and related funds. The design and implementation of a risk-based recipient audit framework can help support the efficient and effective management and oversight of public funds transferred to recipients. A framework which includes a risk-based plan of audits to be conducted should be supported by tools (e.g., policy, directives, guidelines) to ensure key stakeholders are aware of their responsibilities and are able to effectively discharge them.

Key Findings

The audit found that:

• Elements of the Recipient Audit Framework are in place but their requirements have not been fully or consistently met; other elements of the Framework are in development.
• There have been challenges incorporating risk into recipient audit plans.
• Resource constraints have affected the ability to complete all recipient audits and related activities.

Recommendations

Recommendation 1:

The Assistant Deputy Minister, Corporate Management Branch, should update and finalize elements of the Recipient Audit Framework, including the Recipient Audit Policy, Recipient Audit Directive, and other tools and templates in development; and then implement requirements noted therein.

Recommendation 2:

The Assistant Deputy Minister, Corporate Management Branch, should continue the design and implementation of its revised Risk Management Framework to support:

• Incorporating program risk into risk management tools;
• Aligning recipient audit plans with generated risk assessment results, and documenting/challenging rationale in cases which they are not; and,
• Providing training on risk management tools to program managers, officers and other users as needed.

Recommendation 3:

The Assistant Deputy Minister, Corporate Management Branch and the Assistant Deputy Minister, Emergency Management and Programs Branch, should collaborate to:

• Clearly define and communicate stakeholders’ responsibilities related to recipient audit activities; and,
• Review resource allocations within their respective teams (i.e., Centre of Expertise and Program Audit Unit) in relation to their associated responsibilities and aim to optimize resources or seek ways to supplement them.

Conclusion

Improvements are required to the design and implementation of Public Safety’s Recipient Audit Framework.

More specifically, the Department has experienced challenges in implementing the Treasury Board policy requirement of ‘developing and executing a risk-based plan for recipient audits’. The selection of recipient audits on the Department’s Recipient Audit Plan is not supported by robust methodology and tools needed to achieve risk-based results. Despite this, efforts are underway within the Department to address this shortcoming.

Further, while elements of the Recipient Audit Framework are in place, including the Recipient Audit Policy and Directive, their requirements have not been fully or consistently met. Resource constraints have played a factor in the Department’s inability to meet the requirements and to complete all recipient audits on the Plan in a timely manner.

1. Introduction

1.1 Background

Transfer payments are one of the government’s key instruments in furthering its policy objectives and priorities. The payments can be administered as either grants, which are transfers of funds to recipients based on pre-established eligibility and other entitlement criteria; or as contributions, which are subject to performance conditions specified in a funding agreement.

A recipient audit is an independent assessment to provide assurance on a recipient's compliance with a funding agreement. The scope of a recipient audit may address any or all financial and non-financial aspects of the funding agreement. These audits can strengthen the recipients’: accountability; risk and resource management; and, good governance in support of the delivery of programs and services.

In 2017, Public Safety completed the Internal Audit of Grants and Contributions that found improvement was required to establish a more effective methodology for the development and oversight of recipient audits. The audit recommended a risk-based approach be used to identify recipient audits in order to comply with the requirements outlined in Treasury Board's Policy on Transfer Payments.

At Public Safety, the Grants and Contributions Centre of Expertise within the Portfolio Financial Affairs Division of the Corporate Management Branch is responsible for the development and maintenance of the departmental Recipient Audit Plan as well as the related policy, directive, guidance and tools that support the Plan. The Centre of Expertise is also responsible for providing oversight of the recipient audit process and reporting to senior management on the Plan and its results. From a programs perspective, program managers are responsible for identifying potential recipient audits using supporting guidance and tools, assisting auditors as needed during the audits (e.g., facilitating information requests), addressing any audit findings in consultation with recipients, and reporting results (i.e., audit reports and management action plans) to the Centre of Expertise.

In December 2018, in response to the 2017 Internal Audit findings, the Centre of Expertise launched a planning exercise to coordinate the development of the 3-Year Recipient Audit Plan. The execution of this ongoing plan has been underway since this time within Public Safety’s programs.

1.2 Audit Objective and Scope

The objective of this laser audit was to assess the design and implementation of Public Safety’s Recipient Audit Framework to support: the assessment of recipient compliance with funding agreements; cost-effective oversight and control of grants and contributions program expenditures; and, risk based administration requirements on recipients.

The scope of this audit included the following elements of the Department’s Recipient Audit Framework:

• Three-Year Recipient Audit Plan;
• Methodology for selecting recipients for audit;
• Policies, directives and other guidance available related to recipient audits; and,
• Monitoring and reporting structures in place to advise senior management of the Recipient Audit Plan and related activities.

The scope included the activities of the Corporate Management Branch where the Grants and Contributions Centre of Expertise resides; as well as the Emergency Management and Programs Branch, where the majority of the Department’s grants and contributions programs reside. These two branches represent the main focus of recipient audit activities within the Department, in terms of both planning and conducting recipient audits.

The scope included a three-year period of recipient audit activities, beginning in fiscal year 2018-19 and ending at the conclusion of fiscal year 2020-21.

1.3 Methodology and Audit Approach

The audit relied on the following to reach the findings and conclusions noted in this report:

• Document review;
• Interviews with key stakeholders; and,
• A risk assessment based on an analysis of documents reviewed and interviews conducted.

1.4 Conformance with Professional Standards

This audit conforms to the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing and the Government of Canada’s Policy on Internal Audit, as supported by the results of the Quality Assurance and Improvement Program.

2. Findings and Recommendations

2.1 Finding 1: Elements of the Recipient Audit Framework are in place but their requirements have not been fully or consistently met; other elements of the Framework are in development.

The core elements of Public Safety’s Recipient Audit Framework include the departmental Recipient Audit Policy, Recipient Audit Directive, and the annually updated 3-Year Recipient Audit Plan. Other tools to supplement the Recipient Audit Framework are under development by the Centre of Expertise including a recipient audit user guide, a checklist of information required to perform a recipient audit, and a template for management response and action plans.

The audit found that while the above-noted Framework has been in place, there have remained challenges meeting the requirements of the Recipient Audit Policy and Directive. For example, the governance process for reviewing and approving the 2019-20 to 2021-22 Recipient Audit Plan was not implemented as prescribed in the departmental Policy and Directive – i.e., it was not approved by the Directors General Grants and Contributions Committee (or any other committee) and was neither brought forward to the Departmental Management Committee nor the Departmental Audit Committee. Shifting priorities resulting from the COVID-19 pandemic and changes to departmental governance were noted as factors for why these oversight activities did not take place as planned.

Specifically, with respect to governance, the Directors General Grants and Contributions Committee was dissolved in February 2020 due to a lack of activity; and the Departmental Management Committee’s oversight responsibility for recipient audit activities was transferred to the Resource Management Committee in April 2020. In practice, the Resource Management Committee has replaced both these committees in their respective oversight roles. This new governance structure has not yet been reflected in the Policy and Directive. As well, COVID-19 delayed the review and approval of the 3-Year Recipient Audit Plan under this new structure.

As a consequence, the Policy and Directive requirement for the development of a departmental trends report and departmental management action plan has not been completed. These products would be beneficial for the identification, tracking and resolution of systemic issues found during recipient audits, and would assist senior management within the Department stay informed of the results of the Recipient Audit Plan.

Recommendation 1: The Assistant Deputy Minister, Corporate Management Branch, should update and finalize elements of the Recipient Audit Framework, including the Recipient Audit Policy, Recipient Audit Directive, and other tools and templates in development; and then implement requirements noted therein.

2.2 Finding 2: There have been challenges incorporating risk into recipient audit plans.

In line with the Treasury Board Directive on Transfer Payments, the objective of Public Safety’s Recipient Audit Directive is “to achieve effective, risk based approaches in conducting recipient audits”. However, we found risk has not been the primary factor in the selection and conduct of recipient audits. Similar observations were made during the 2017 Internal Audit of Grants and Contributions.

The audit found the majority of funding agreements initially chosen for recipient audits from 2018-19 to 2020-21 were assessed as low or medium risk (29 of 34 agreements).

The few high risk agreements initially proposed for audit during this timeframe were deferred to a future year, or removed from the Plan altogether in one case. While there are plans to audit these deferrals and other high risk agreements in future years of the Plan (beyond 2020-21), none were conducted during the three-year period (2018-2021) of recipient audit activities examined during this audit. Further, we did not find evidence to suggest that rationale for decisions affecting high risk agreements on the Plan was requested, documented, and challenged by the Centre of Expertise as part of its responsibilities in developing the Plan and providing oversight of the process. The inactivity of the now dissolved Directors General Grants and Contributions Committee further contributed to the lack of oversight. It is important that decisions to include, exclude, defer or accelerate audits on the Plan are supported by documented rationale and challenged accordingly, particularly as they are related to risk overrides.

Challenges incorporating risk into the recipient audit planning process may be caused with how risk is measured.

Currently, the process for assessing the risk ranking for funding agreements is not supported by robust tools and methodology. The overwhelming majority of active funding agreements (83%) in the Public Safety Information Management System (PSIMS) are rated as low risk. Interviewees suggested this may be the result of the calibration of the PSIMS risk assessment tool and/or the degree of subjectivity with which risk is scored. Interviewees also questioned whether program managers and officers who are responsible for assessing risk have been appropriately trained to do so. It was noted that the Centre of Expertise conducted training for Public Safety’s community of practice in 2018 to support risk assessments. While the Centre of Expertise noted improvement in the quantity of risk assessments conducted, the overall quality of assessments remained unchanged due to the rudimentary nature of current tools available.

The Centre of Expertise has been working on a new Risk Management Framework to improve the risk functionalities in PSIMS and to update the Department’s existing Project Level Risk Management Directive. According to the Centre of Expertise, this new Risk Management Framework represents a significant change in the way Public Safety manages risk – “it is an integrated suite of policies, risk assessment tools, mitigation strategies, templates and documents supporting program officers and the community of practice through the lifespan of funding agreements.” Until this new Framework is in place, the existing tools upon which this audit was based contain standardized risk assessment criteria and corresponding mitigation strategies that do not fully meet the Department’s needs in order to support compliance with Treasury Board Policy.

As the new Risk Management Framework is still in development, it is not clear at this time if or how the Framework plans to incorporate program-level risk into its tools or if the focus will remain at the project level which factors in the value of funding and the risk profile of recipients. Program risk, along with the value of funding and the risk profile of recipients, are three key elements deputy heads are responsible for reflecting in oversight activities such as auditing according to the Treasury Board Policy on Transfer Payments.

Recommendation 2: The Assistant Deputy Minister, Corporate Management Branch, should continue the design and implementation of its new Risk Management Framework to support:

• Incorporating program risk into risk management tools;
• Aligning recipient audit plans with generated risk assessment results, and documenting/challenging rationale in cases which they are not; and,
• Providing training on risk management tools to program managers, officers and other users as needed.

2.3 Finding 3: Resource constraints have affected the ability to complete all recipient audits and related activities.

From the perspective of the Corporate Management Branch, the audit found that there is currently one resource in the Centre of Expertise who dedicates approximately 20 percent of their workload to recipient audit activities including the coordination of the annual Recipient Audit Plan. As a result, this has limited the Centre of Expertise’s ability to consistently and comprehensively fulfill the requirements and its responsibilities outlined in the Recipient Audit Directive, including the development of key reports to highlight trends and track management action plans.

From a program perspective, as of April 2020, support for recipient audit activities is being provided by the Program Audit Unit which resides in the Program Integration Directorate of the Emergency Management and Program Branch. This unit provides the Centre of Expertise with input for the Recipient Audit Plan by coordinating with program staff to update the list of recipient audits they wish to undertake. This unit is also involved in the conduct of recipient audits, both performing them in-house and contracting them out to a third party. Given this unit’s involvement in recipient audit activities is fairly new, the definition of its roles and responsibilities in relation to that of other stakeholders (e.g., Centre of Expertise, program managers) would benefit from greater clarity and communication.

Interviews with the Program Audit Unit revealed that resource constraints have limited its ability to complete all recipient audits on the Plan. For example, of the 15 recipient audits initially planned for fiscal year 2020-21, eight were deferred, partially due to resource constraints. Specifically, it was noted there is no budget for recipient audits to be completed by an external contractor; instead, funding is sought each year for this purpose. In the past, programs would be asked to pay for their audits but O&M shortfalls made it difficult to proceed, and as a result the Program Audit Unit began conducting some audits in-house. Further, this unit is also responsible for mandatory audits under the Disaster Financial Assistance Arrangements, which was raised as contributing to the challenges in executing the Plan as intended.

Recommendation 3: The Assistant Deputy Minister, Corporate Management Branch and the Assistant Deputy Minister, Emergency Management and Programs Branch, should collaborate to:

• clearly define and communicate stakeholders’ responsibilities related to recipient audit activities; and,
• review resource allocations within their respective teams (i.e., Centre of Expertise and Program Audit Unit) in relation to their associated responsibilities and aim to optimize resources or seek ways to supplement them.

3. Conclusion

Improvements are required to the design and implementation of Public Safety’s Recipient Audit Framework.

More specifically, the Department has experienced challenges in implementing the Treasury Board policy requirement of ‘developing and executing a risk-based plan for recipient audits’. The selection of recipient audits on the Department’s Recipient Audit Plan is not supported by robust methodology and tools needed to achieve risk-based results. Despite this, efforts are underway within the Department to address this shortcoming.

While elements of the Recipient Audit Framework are in place, including the Recipient Audit Policy and Directive, their requirements have not been fully or consistently met. Resource constraints have contributed to the Centre of Expertise’s inability to meet their reporting and oversight requirements as well as the Program Audit Unit’s ability to complete all recipient audits on the Plan in a timely manner.

4. Management Action Plan

Acknowledgements

The Internal Audit and Evaluation Directorate would like to thank all those who provided advice and assistance during the audit.

Date modified:

2022-11-10