ARCHIVE - 2008-2009 Formative Evaluation of the Intergovernmental Affairs Division

Archived Content

Information identified as archived is provided for reference, research or record-keeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

Final Report

Adobe Acrobat version (PDF 277KB)

Table of Contents

[ * ] - In accordance with the Privacy and Access to Information Acts, some information may have been severed from the original reports.

Executive Summary

This report presents the findings of the evaluation of the Federal Provincial Territorial (F/P/T) related activities of the Intergovernmental Affairs Division (IGA) within Public Safety Canada (PS).

Funded through the Public Safety and Anti-Terrorism (PSAT) envelope announced in the 2001 Federal Budget, an F/P/T forum on Emergency Management (EM) was established to enable F/P/T Ministers and Deputy Ministers responsible for EM the opportunity to deliberate and provide advice and guidance on EM issues of national concern. Some activities related to this forum have been transferred to the PS Emergency Management and National Security Branch (EMNS). IGA retained responsibilities for monitoring, analyzing and sharing information about policy and/or changes at the P/T level that may have an impact on or be of interest to the Department. Within this context, IGA provides support to its clients (Minister, Deputy Minister and senior departmental officials) with regard to the F/P/T Forum on Emergencies, F/P/T Justice meetings, and other F/P/T bilateral activities. IGA also collects and shares information with its partners who are considered to be PS, portfolio and external personnel who deal directly or indirectly with F/P/T relations, information and advice on public safety matters.

The objective of this mid-term, formative evaluation is to assess the degree to which IGA is making progress in the achievement of its objectives. The evaluation will be used to gain an understanding of the current state of IGA and to inform actions for future adjustment and improvements. The following evaluation questions have been assessed through consultations and document review.

Success

Program Design and Implementation

The evaluation draws the following conclusions:

Recommendations and Management Action Plan

Based on the findings and conclusions of this report, it is recommended that the Assistant Deputy Minister, Strategic Policy Branch, ensure that IGA take action to address the following recommendations. Management has responded to each with an Action Plan as outlined below.

Recommendation 1: Complete its business plan and fill organizational design gaps. IGA should review and document all elements of high value service to clients and interaction with partners, the processes and activities required for their provision and delivery, and necessary resource expertise and tools. The review should result in a description of the gap between its present performance and expected performance, and an action plan designed to fill that gap, integrated into its comprehensive business plan.  

Recommendation 2: Develop and implement a strategy to build awareness of the IGA mandate with partners. IGA should develop a strategy to build awareness with its internal and external partners on its mandate, roles and responsibilities. Awareness activities should set clear expectations of IGA responsibilities.

Recommendation 3: Take measures to protect and share corporate knowledge by detailing and documenting key business processes. IGA should identify and document its key business processes to assure proper ongoing management and dissemination of information within the Division and the Department.

1. Introduction

This report presents the findings of the evaluation of the Federal Provincial Territorial (F/P/T) related activities of the Intergovernmental Affairs Division (IGA) within Public Safety Canada (PS). This evaluation was conducted in response to Treasury Board Secretariat requirements with regard to National Security Policy reporting requirements.

2. Profile

2.1 Overview

At a meeting in January 2004, Canadian First Ministers recognized the need for governments to work collaboratively on key emergency management issues. Subsequently, the National Security Policy that was announced in April 2004 called for the establishment of a permanent high-level F/P/T forum on Emergency Management (EM). The need for such a forum was reiterated in early 2005 when Ministers responsible for EM met for the first time since 1993. The decision was taken to create a high-level, permanent, standing forum on EM to provide a governance structure that would enable F/P/T Ministers and Deputy Ministers responsible for EM the opportunity to deliberate and provide advice and guidance on EM issues of national concern.

This initiative was funded through the Public Safety and Anti-Terrorism (PSAT) envelope announced in the 2001 Federal Budget. The funding provided was $1.1 million per year starting in 2005-2006 through 2009-2010 and on-going, and was allocated so that the Initiative could fulfill five functions:

2.2 Evolution of the Initiative

In 2006-2007, the Initiative became the responsibility of the Strategic Policy Branch. The Branch was created after a reorganization of its predecessor, the Public Relations and Portfolio Affaires Branch. The Intergovernmental Affairs Division (IGA) was established and assumed responsibility for the Branch’s activities related to F/P/T relations, including the Initiative. To date, all IGA funding has been from the PSAT envelope.

In 2007, through a Memorandum of Understanding, several functions were transferred from IGA to the Emergency Management and National Security Branch (EMNS) along with $85,000 per year for Operations and Maintenance. Functions transferred included:

IGA remained responsible for the activities described below with regards to monitoring, analyzing and sharing information about policy and/or changes at the P/T level that may impact or be of interest to the Department. IGA has defined its clients as being senior departmental officials (e.g. Minister, Deputy Minister and Associate Deputy Minister). IGA considers PS, portfolio and external personnel who deal directly or indirectly with F/P/T relations, information and advice on public safety matters as partners.

The services that IGA provides to its primary clients include:

IGA interacts with its partners through the following activities:

2.3 Reporting Structure

The IGA Director reports to the Director General, Intergovernmental Affairs and Citizen Engagement Directorate who reports directly to the Assistant Deputy Minister of the Strategic Policy Branch of PS.

2.4 Resources

IGA’s resources are presented in Exhibit A.

Exhibit A: IGA Resources

Full Time Equivalents

Planned Annual Funding

9

Salaries

$517,000

Benefits

$103,000

O&M

$412,400*

Accommodation

$67,200

Total

$ 1,099,600

*In 2007-2008, $85,000 was transferred from this total to the Emergency Management and National Security Branch.

2.5 Logic Model

A logic model is an essential tool in conducting an evaluation study. It is a visual representation that links a program’s activities, outputs and outcomes, provides a systematic and visual method of illustrating the program theory and shows the logic of how a program, policy or initiative is expected to achieve its objectives. It also provides the basis for developing the performance measurement and evaluation strategies. A logic model for IGA is presented in Exhibit B.

Exhibit B – Logic Model for the Intergovernmental Affaires Division

Exhibit B – Logic Model for the Intergovernmental Affaires Division

3. Focus of the Evaluation

3.1 Objective

The objective of this mid-term, formative evaluation is to assess the degree to which IGA is making progress in the achievement of its objectives.

3.2 Scope

The scope of the evaluation includes the activities of IGA related to the F/P/T forum on emergencies, F/P/T Justice meetings, other F/P/T bilateral activities and information provided to partners. The evaluation will be used to gain an understanding of the current state of the IGA and to inform actions for future adjustment and improvements.

Activities of the Coordination Directorate in Emergency Management and National Security (EMNS) in support of the F/P/T Forum on Emergencies are not included in this evaluation. PSAT is referenced solely to provide background and set context, but is not included as an object of this evaluation.

3.3 Methodology

The evaluation was informed by the Guide for the Review of Evaluation (2004) and Preparing and Using Results-based Management and Accountability Frameworks (2005) prepared by the Treasury Board of Canada Secretariat Centre of Excellence for Evaluation.

The following data and methods were used to conduct the evaluation:

The Evaluation was limited by the following considerations:

3.4 Evaluation Questions

Success

Program Design and Implementation

4. Findings

4.1 Success

4.1.1 Client Satisfaction

Feedback from participants of F/P/T meetings of Ministers, or Deputy Ministers responsible for justice and public safety was positive. There was an appreciation of the complexity and challenges of planning, coordinating and supporting these events. Some of the challenges noted included those related to timeliness and accuracy.

4.1.2 Interactions with partners

Some IGA partners in the Department expressed the observation that generally speaking, their understanding of IGA’s role was incomplete, and that they were unclear on what its responsibilities were.

The perceived value of environmental scan reports is mixed.Some readers said they were a good source of information while others suggested they offered little value. They did appear to provide value to personnel in policy areas. IGA has recently surveyed its clients and preliminary analysis shows that most respondents find the reports useful. Interviewees stated that ad hoc reports on significant current potential changes within provincial or territorial governments from a personnel or policy perspective are valuable.

4.1.3 Fulfilling Roles and Responsibilities

IGA is making some progress towards fulfilling its roles and responsibilities that are partially defined in its current business plan (still under development). Exhibit B presents the status of IGA activities.  

Exhibit C: IGA Activities Status and Comments

Services to clients
Activity Status Comments
Supporting the Minister and Deputy Minister as necessary with regard to the Forum of F/P/T Ministers responsible for EM and Justice Established  Reported very well performed.  
Developing priorities for the PS federal perspective in setting the agenda for F/P/T Justice meetings Developing Improvements could be made and under consideration.
Coordinating departmental input to confirmed agenda items supporting the Minister and Deputy Minister at the F/P/T Justice meetings Established

Reported very well performed.

Preparing materials for other bilateral meetings with P/T counterparts for the Minister, Deputy Minister and senior officials Refining/Implementing Protocols being refined (Reported to be time consuming.)
Producing periodic reports that provide strategic intergovernmental analysis and advice to ensure the intergovernmental agenda is consistent with PS priorities Developing Ad hoc information updates being sent to branches. Preliminary analyses being developed.
Providing input to corporate documents such as memorandums to Cabinet, Treasury Board Submissions, access to information requests, planning and reporting documents. Developing Some support provided.

Interactions with Partners
Activity Status Comments
Twice-monthly environmental scans that result in the publication of reports that are available to PS personnel and that include P/T items of interest categorized by subject Established List of users expanding. User survey completed and modifications made (Production reported to be time consuming.)
Producing and distributing ad hoc reports on issues or items of high enough importance to warrant immediate communication Developing Production reported to be time consuming.
Liaising with central agencies and acting as the central communications contact for internal or external queries about the position of the department on P/T policy issues or concerns including advice on correspondence with provinces and territories Developing Minimal time devoted to it (done on an “as needed” basis).
Liaising with Privy Council Office by facilitating requests from PCO and conducting research to identify an individual within the department who can satisfy the request and monitoring the department’s and PCO’s position on P/T policy and issues for consistency Established Contact is maintained.
Monitoring/tracking activity within F/P/T working groups Refining/Implementing Minimal time devoted to it.

Internet site on EM fora.    

Developing and maintaining an Intranet site on F/P/T activities.

Developed and maintained by EMNS
 
Developing  


Working with Communications group to implement.

4.1.4 Improving Efficiency, Effectiveness and Quality

There is evidence to suggest that there are areas where IGA has the opportunity to contribute to improvements.

Some interviewees suggested that certain aspects of the F/T/P meetings are not sufficiently strategic in focus and do not result in significant accomplishment. We note that IGA cannot be held accountable for the outcomes of these meetings, however, some interviewees did indicate that IGA does have an opportunity to provide added value by addressing or assisting in the resolution of perceived issues such as:

Justice Canada considers that it has a strong working relationship with IGA. There appears to exist opportunities to improve the relationship by sharing information and increasing communications concerning the Memorandum of Understanding between the two organizations, the development of meeting agendas, and the timely completion of tasks.

The relationship with the PS Emergency Management and National Security (EMNS) Branch could be strengthened. Consultations revealed that this group has limited knowledge of IGA roles and responsibilities.

Interviewees indicated that PS branches and directorates need to be informed and understand the F/P/T activities in other parts of the department and within portfolio agencies to ensure compatibility of approach and avoid inconsistent messaging. This is beyond the scope of IGA’s current responsibilities; however IGA does fulfill this role to a certain extent for the F/P/T fora. A suggestion was made that IGA could act as a centralized focal point for rationalizing, integrating and disseminating departmental and portfolio information about P/T relations and activities. Support for IGA in this role was mixed. However, all participants expressed a willingness to share F/T/P fora information.

4.2 Program Design and Implementation

4.2.1 Definition and Alignment of Expected Results

Statements related to the IGA mandate, objectives and delivery strategy are currently found in various documents, but the results and outcomes that IGA intends to achieve are not comprehensively defined.IGA has a reasonable understanding of its expected outputs and has identified activities and tasks that it expects to complete in a given time period. For example, the number of meetings is known and environmental scans are distributed twice per month. However, the partially completed business plan does not fully link these outputs to the achievement of outcomes and departmental objectives. 

Consultation participants confirmed the contribution that IGA is making to the achievement of several strategic outcomes including:

IGA has conducted a comparative study with other federal organizations which identifies roles and responsibilities of similar organizations. An analysis of this document reveals that IGA plays a similar role and function as other IGA groups across the Government of Canada. Best practices are being examined at this time for possible application to IGA’s context.

4.2.2 Definition, Communication and Implementation of Procedures, Processes, Roles and Responsibilities

It has been observed that corporate knowledge gaps have been created at IGA as employees leave or change positions. Attempts have been made to better define some processes e.g., those associated with the development and production of environmental scan reports. However, more details on roles and responsibilities and service standards could be included.

IGA should improve the documentation of its business processes. Processes that have not been documented include those associated with:

4.2.3 Governance and Risk Mitigation

An F/T/P intradepartmental working group has been established to explore F/P/T and departmental responsibilities regarding Justice and emergency management. On a regular basis, the Director participates in an IGA Directors’ Network which brings together colleagues and associates who work in the federal government in similar intergovernmental organizations dealing with F/P/T relations to share ideas and best practices.

Some planning documents have identified potential risks to organizations working in the Emergency Management domain. It was observed that IGA does not appear to be employing a risk-based approach to managing the organization, i.e., a comprehensive risk assessment of IGA that identifies possible threats, impacts and mitigation strategies has not been conducted.

4.2.4 Performance Measurement

IGA does not have a comprehensive, ongoing performance measurement strategy in place. There is evidence that some performance information is being collected. The data collected includes numbers of F/P/T meetings supported, environmental scans and other ad hoc F/P/T information products. This type of quantitative performance data provides limited insight on the quality, efficiency and effectiveness of IGA activities.

Data is primarily used to contribute to periodic departmental reports. There exists an opportunity to improve and use data to measure accomplishment of goals, services provided, report performance trends, make adjustments in operations or procedures, etc. Application of useful performance indicators is expected to accompany the full development of IGA’s business plan.

5. Conclusions

The evaluation draws the following conclusions:

6. Recommendations and Management Action Plan

Based on the findings and conclusions of this report, it is recommended that the Assistant Deputy Minister, Strategic Policy Branch, ensure that IGA take action to address the following recommendations. Management has responded to each with an Action Plan as outlined below.

Recommendation 1: Complete its business plan and fill organizational design gaps. IGA should review and document all elements of high value service to clients and interaction with partners, the processes and activities required for their provision and delivery, and necessary resource expertise and tools. The review should result in a description of the gap between its present performance and expected performance, and an action plan designed to fill that gap, integrated into its comprehensive business plan.  

Recommendation 2: Develop and implement a strategy to build awareness of the IGA mandate with partners. IGA should develop a strategy to build awareness with its internal and external partners on its mandate, roles and responsibilities. Awareness activities should set clear expectations of IGA responsibilities.

Recommendation 3: Take measures to protect and share corporate knowledge by detailing and documenting key business processes. IGA should identify and document its key business processes to assure proper ongoing management and dissemination of information within the Division and the Department.

Appendix A: Documents Reviewed

[ * ]

Appendix B: Consultations

Date modified: