Results-Based Management and Accountability Framework
First Nations Policing Policy and Program

Adobe Acrobat version (PDF 85.2 KB)

Table of Contents

• 1.0 Introduction
• 2.0 Profile
  • 2.1 Background
  • 2.2 Delivery Approach
  • 2.3 Resources
  • 2.4 Primary Reach
  • 2.5 Planned Results
  • 2.6 Governance Structure for the First Nations Policing Policy and Program
    • 2.6.1 Departmental Structure
    • 2.6.2 Partners in the Negotiation and Implementation of Tripartite Agreements
• 3.0 Logic Model
• 4.0 Ongoing Performance Measurement
  • 4.1 Performance Measurement
  • 4.2 Ongoing Performance Measurement Strategy
• 5.0 FNPP Evaluation Strategy
  • 5.1 Evaluating a Community Policing Program
  • 5.2 Methods of Data Collections
    • 5.2.1 Approach
    • 5.2.2 Methods of Data Collections
    • 5.2.3 Measures of Outcomes (Indicators)
    • 5.2.4 Resource Considerations
  • 5.3 Measuring Performance and Results - Summary
• 6.0 Reporting Strategy

1.0 Introduction

The Result-Based Management and Accountability Framework (RMAF) for the First Nations Policing Policy and Program (FNPP), formalizes the Department of the Solicitor General's commitment to ensure that the FNPP, a Business Line of the Department, is set up logically, has an accountability framework as well as a strong commitment to results. This initial RMAF is a starting point and will continue to be improved according to identified needs and emerging issues. As the Department's experience with and understanding of performance and results measurement grows the context, approach and techniques will be refined.  This RMAF includes:

• Profile - a concise description of the FNPP, including a discussion of the background, need, target population, delivery approach, resources, governance structure and planned results;

• Logic Model- an illustration of the results chain or how the activities of the FNPP, are expected to contribute to the achievement of the ultimate outcome - safer and more sustainable First Nations communities;

• Ongoing Performance Measurement Strategy - a plan for the ongoing measurement of performance, including the identification of indicators for the outputs and outcomes in the logic model and a measurement strategy describing how these indicators will be collected, and how often;

• Evaluation Strategy - a plan for the evaluation of the FNPP, including an outline of the intended approach, methods for data collection, considerations concerning the selection of indicators, and resource estimates.  The identification of evaluation issues and questions, the identification of associated data requirements, and a data collection strategy which will serve as the foundation for subsequent evaluation activities;

• Reporting Strategy - a plan to ensure the systematic reporting on the results of ongoing performance measurement as well as evaluation, to ensure that all reporting requirements are met.

2.0 Profile

2.1 Background

The First Nations Policing Policy and Program (FNPP), a non-statutory program, was first introduced by the federal government in June 1991, to address the recognized need of expanding and improving the level and quality of First Nations policing services.  At its inception in 1991, the FNPP was the responsibility of the Department of Indian Affairs and Northern Development (DIAND).  In April 1992, the program was transferred from DIAND to the Department.

Two separate levels of government - the federal and the provincial, govern policing in Canada.  Most policing responsibility is under provincial jurisdiction.  Provincial legislatures are empowered by virtue of s. 92(14) of the Constitution Act to make laws in relation to the "administration of justice" and are granted jurisdiction over civil and criminal justice of which policing is a part.  The Task Force Report on the Indian Policing Policy Review (January 1990) opined that jurisdiction over on reserve policing is shared and that both levels of government have a measure of responsibility and a role to play.

In order to achieve its main objective of helping to improve social order, public security and personal safety within First Nations communities, the FNPP establishes and funds police services that are professional, effective and responsive to the needs of the First Nations and Inuit communities they serve.  As such, federal and provincial governments are pursuing three key objectives namely:

• To ensure that First Nations people have access to policing services that are responsive to their needs and that meet acceptable standards with respect to the quality and level of service;
• To support First Nations communities to establish structures for the management, administration and accountability of their police services; and
• To implement and administer the FNPP in partnership with First Nations communities.

The FNPP has become the basis for enhanced public safety and stability for First Nations communities.  This has been achieved thought the partnering of Canada, provinces and territories, and First Nations and Inuit communities, as outlined in Section 2.2.  These tripartite agreements provide dedicated community policing services to reduce crime and solve problems of disorder and instability.

Similarly, the FNPP contributes to the enhancement of community support for other parts of the justice system as well as the overall improvement of the quality of life experienced by First Nations communities, by "providing a stable platform from which other goals can be achieved, such as keeping nurses in remote communities, and facilitating economic development to help address social gaps ^[1] ".

2.2 Delivery Approach

The FNPP operates on the principle of tripartite partnership with communities, provinces, and territories.  Under the current FNPP, there are three types of agreements:

1. Self-Administered (SA) Agreements, which are negotiated among the federal government, provincial or territorial governments and First Nations communities.  Under such an agreement the First Nation is responsible for managing its own police service and is staffed primarily by officers of First Nations descent.
2. First Nations Community Policing Service (FNCPS) Framework Agreements are bilateral agreements between Canada and a province or territory which allow for the future signing of individual Community Tripartite Agreements that will provide for policing services within the province or territory.
3. Community Tripartite Agreements (CTA) are a direct result of the signing of an FNCPS Framework Agreement.  Like SA agreements, CTAs are negotiated among Canada, the partnering province or territory, and the First Nations community.  Under a CTA arrangement, the First Nations community has dedicated officers from an existing police service, typically the Royal Canadian Mounted Police (RCMP), who spend a minimum of 80% of their time on reserve.

Under the FNPP, the federal government pays 52% and the provincial or territorial government pays 48% of the government contribution towards the cost of the First Nations policing service.

In terms of assessing funding requirements under the FNPP, the three parties negotiate to determine the level of service and officer and civilian complement, for which federal and provincial / territorial governments agree to cost-share.  There is no specific formula to determine this, as there are many individual factors that enter into the negotiation of level of service and of policing budgets.  These include but are not limited to:

• the level of service previously provided by the provincial police service to the First Nations;
• the comparative level of service provided to neighboring communities;
• the appropriate officer to population ratio - the per unit cost;
• the demographic characteristics of the population to be served;
• the size and nature of the geographic area to be covered;
• the nature of activities, including positive cultural and traditionally activities as well as negative activities that occur within the territory and;
• the police workload in the community based on crime statistics and crime prevention activities.

Furthermore, the FNPP outlines the following criteria, which must be met in order for a First Nations community to qualify to receive funding to support a police service ^[2]:

• All police officers in First Nations-administered policing services must be First Nations people, except where the First Nation agrees to and participates in the staffing of a non-First Nations police officer. While the positions are full time, the duration of such staffing will be negotiated by the federal government, the relevant province / territory and the First Nation;
• Professional standards (e.g. training) of the police service must meet the standards of the province or territory in which it operates;
• Police officers must be properly appointed by the appropriate authorities, and empowered to enforce all applicable laws; and
• The police service must consult with and be accountable to the community it serves through a police board, commission or advisory body.

While not all policing costs are covered by the FNPP, eligible costs do include: program administration, including governance mechanisms; recruiting, training and education (including formal in-service training); salaries and benefits for police officers and civilian staff; and expenditure, such as operating, maintenance and minor capital costs.  In the event that activities such as an analysis of policing needs, research and development activities, tripartite negotiations, evaluations and public education are undertaken, funding may also be provided to the First Nations community under the FNPP.

2.3 Resources

According to the 2003-2004 Estimates, Part III - Report on Plans and Priorities, funding for the FNPP is broken down as follows:

Breakdown of FNPP Funding
Aboriginal Policing Directorate[3]	$90,447,000
Policing and Law Enforcement Branch[4]	$1,000,000
Total	$91,447,000

The Report on Plans and Priorities also notes that 36 full-time equivalents (FTEs) are devoted to managing and administering the FNPP.

2.4 Primary Reach

The FNPP applies to all First Nations reserves and settlement lands, to certain other First Nations communities located on federal crown land, and to Inuit communities.  The main participants in negotiating the tripartite agreements are those provincial/territorial, First Nations and federal authorities with responsibility for public safety.  However, given the goals of the FNPP, there are a number of other stakeholders who, through dedicated community policy arrangements in First Nations communities, are directly affected.  These stakeholders include:

• police community (e.g., RCMP, provincial and territorial police services);
• other essential service providers (nurses, teachers, court workers, social workers);
• community consultative groups;
• police commissions;
• public security committees;
• DIAND
• Department of Justice; and
• Health Canada

2.5 Planned Results

In the short-term, the results most closely associated with the FNPP's activities, and relevant to public safety are the following:

• Consultations and relationships with partners;
• Establishment and maintenance of culturally appropriate police services in First Nation communities;
• Increased awareness of all partners of responsibilities and obligations vis à vis community policing;
• Identification of emerging issues and the interdependency of Aboriginal issues; and
• Training and development in support of community governance.

The anticipated results in the medium-term resulting from the short-term outcomes include the following:

• First Nations, access to culturally appropriate policing services;
• Effective and efficient policing services;
• Compliance with terms and conditions of the contribution agreements;
• Consideration of policing and public safety in broader Aboriginal initiatives; and
• Appropriate community capacity of police commissions, public security committees, consultative groups, and police management boards.

The ultimate desired outcome of the FNPP is to contribute to safer and more sustainable First Nation communities.  In examining the long-term results of the FNPP, the identification and description of key contextual factors external to the program are an important consideration particularly since the desired ultimate outcome is shared by other interests.

2.6 Governance Structure for the First Nations Policing Policy and Program

2.6.1 Departmental Structure

Within the federal government, the Solicitor General of Canada is the lead Minister responsible for law enforcement, corrections, and security.  In fulfilling this accountability to Parliament, the Minister provides national leadership with respect to public safety and security.

The Department is charged with the responsibility of implementation, administration, and management of the FNPP.  Moreover, the Deputy Minister (DM) is responsible for the overall leadership and direction of the FNPP and is advised by the Assistant Deputy Minister, Corrections and Aboriginal Policing Branch.

Aboriginal Policing Directorate

The Aboriginal Policing Directorate (APD) is responsible for the implementation and administration of the FNPP. Specifically, APD

• promotes more effective police services for First Nations and Inuit people;
• develops and coordinates federal responses to policing, and policing related, policy issues that impact First Nations communities; and
• performs related program development and research activities.

The APD works in partnership with First Nations communities and their leaders, and provinces and territories to negotiate tripartite agreements in order to provide professional, effective, and responsive police services in First Nations communities.  The APD's main responsibilities include:

• implementing the FNPP through negotiating and administering tripartite policing agreements, and providing support (training and development) to communities to establish and maintain an arms-length oversight capacity; and
• supporting program development through pilot projects, research and evaluation, and involvement in other Aboriginal initiatives and strategies related to justice, community development and stability, combating substance abuse, reducing family violence, and creating safe and healthy communities.

The Director General (DG), who is responsible for implementing the FNPP in accordance with the policy objectives and program parameters also heads the APD.  The DG is also responsible for providing leadership and guidance for future policy and program direction.  APD is comprised of three Divisions:

• Aboriginal Affairs Divisions

Under the leadership of a Deputy Director General, this Division is responsible for policy development and analysis, including research in support of the FNPP and participation in horizontal federal initiatives.  This Division is also responsible for the planning, management, and administrations of the FNPP's multi-year budget, the majority of which is disbursed through transfer payments (or contributions) to eligible recipients.  Furthermore, the Deputy Director General supports the DG in providing corporate direction and oversight of the APD in addition to being the chief point of contact for the First Nations Chiefs of Police Association, a key client group.

• Operations Division

Under the leadership of a Senior Director, this Division manages and directs the development and negotiation of tripartite policing agreements with First Nations communities and provincial / territorial governments, while also providing support to the overall negotiations process and implementation of these tripartite policing agreements.  Seven regional offices report directly to this Senior Director.  Regional Managers are responsible for facilitating, negotiating, monitoring, liaising and providing effective communications and program support for the FNPP.  The Senior Director ensures that these responsibilities are carried out in a team approach and provides guidance, direction and overall leadership in order to reach the objectives and responsibilities of the APD.  This Division is also responsible for monitoring the tripartite agreements in order to ensure compliance and effectiveness.

Furthermore, the Senior Director of this Division is responsible for developing and maintaining effective working relations between Aboriginal clients and federal and provincial / territorial officials in support of improved First Nations policing programs and services.

• Strategic Initiatives Division

The Director of Strategic Initiatives is responsible for addressing cross-cutting policy issues that are central to the evolution of the FNPP.  Related initiatives are usually time-limited, and require an interdisciplinary approach for successful resolution.

2.6.2 Partners in the Negotiation and Implementation of Tripartite Agreements

Provinces and Territories

Provinces and territories have the primary responsibility for the administration of justice, including policing, within their jurisdiction.  Under the FNPP, they agree to cost-share dedicated policing in order to enhance the level and quality of policing in First Nations communities.  While Canada's funding is tied to a multi-year funding envelope that is subject to annual Parliamentary approval of appropriations, provinces and territories seek their required funding via annual submissions to their Treasury and Management Boards.  In addition, provinces and territories establish and monitor operational policing standards and are responsible for appointing officers and empowering them to enforce all applicable laws.

First Nations

First Nations policing is a working partnership between First Nations communities and the police to prevent crime, find solutions to recurring problems, and enhance the quality of life of the communities being served.  Consequently, community leaders play a significant role in developing policing services for their communities that are responsive to its needs. Through the tripartite negotiation process, communities are involved in the determination of an appropriate service delivery model, the actual level of service to be delivered, and the associated resources.  First Nations police services must meet the policing standard of the province in which it operates and must remain independent from political interference and be accountable to the community it serves.  Self-administered police services must establish an independent police governing authority to oversee and manage all aspects of its police service.  Communities receiving services through an RCMP Community Tripartite Agreement must also create community consultative groups or police management boards that will be responsible for identifying and communicating community priorities and needs.

Service Providers

Service providers can include First Nations-administered police services or the RCMP.  Regardless, they are responsible for the level of services as specified in the negotiated tripartite agreements.  As such, all officers must be properly trained and appointed in the province or territory in which they work.

3.0 Logic Model

Introduction

The Government's introduction of the First Nations Policing Policy (FNPP) in 1991 must be set within a broader public safety context.  In the 1980s and early 1990s, several key Aboriginal Justice Inquiries noted the difficulties Aboriginal people face in the justice system such as access to fair, just, and equitable policing services.  From its inception, the goal of the FNPP has been to provide dedicated policing services to First Nations communities across Canada.  The level of services was and is to be comparable to that provided to non-First Nations communities with similar geographic, population, and criminality and community justice characteristics.  The introduction of the FNPP was also set against the backdrop of the complex issue of the shared jurisdiction of federal and provincial government for policing in First Nations communities.

The Model

The model, on page 12, presents the major recurring activities and outputs that produce the results intended by APD and its partners and desired of the FNPP namely providing First Nations across Canada with access to police services that are professional, effective, and culturally appropriate, and accountable to the communities they serve.  Solicitor General Canada, in partnership with provinces and First Nations, invests staff, time and money to implement, administer, and manage the FNPP.

The core activity of APD is to negotiate, implement, and monitor tripartite agreements with provinces and First Nations communities to provide funding for dedicated community policing.  Community based policing is a partnership between police and community to identify, prioritize, and solve local crime and disorder problems, thereby improving the quality of life. Since the FNPP is a non-statutory program, limits on resources in turn limit the number of agreements and beneficiaries.

Ancillary activities undertaken by APD that support the establishment and maintenance of culturally appropriate police services include:

• Funding of monitoring activities

APD funds monitoring activities (i.e. financial audits, management action plans, operational audits^[5] etc.) to assess the extent to which First Nations policing governing authorities and police services have complied with the legal and financial requirements set-out in their FNPP policing agreement.  The results of these monitoring activities also indicate to funders whether police governing authorities have assumed an effective and appropriate role in directing and managing their police services.

• Building and maintaining relationship with partners

APD also enters into collaborative or partnership arrangements to work cooperatively on issues, special projects, and information exchange to improve program delivery and performance.  Productive partnerships, vis à vis the identified needs of partners, are important to the evolution of the FNPP, given the tripartite and cost-shared nature of the program in an area of shared federal-provincial jurisdiction.

• Policy development and analysis

In fulfilling the department's mandate for the overall leadership and direction of the FNPP, APD is responsible for responding to and influencing policy issues that are core to the goals and objectives of the FNPP - namely public safety in First Nations, and advocates on those issues which enhance the delivery and evolution of the FNPP and the safety and stability of First Nations communities.  The following are some of the issues that are important to the future of the FNPP: self-government; community safety and sustainability; governance; national law enforcement trends; and Aboriginal community justice. The future direction of these issues will have a significant impact of the level, quality, and sustainability of dedicated policing in First Nations communities.

• Support for community oversight capacity

Provincial laws and government directives regarding policing set out the specific responsibilities of civilian oversight bodies charged with overseeing their police services, including developing budgets for policing operations, establishing policies for efficient and effective policing, and providing policy direction to their police chiefs.  The FNPP places emphasis on building the knowledge and capacity of community members and leaders to provide this oversight, particularly in a manner that assures police independence from partisan and inappropriate influences.  Community capacity in this regard depends on participation in training and development activities in the roles, responsibilities, and functions of governance bodies.  APD is committed to working in partnership to invest in training opportunities for community-led police governance.

FNPP Logic Model

Activities and Outputs	Reach	Immediate Outcomes	Intermediate Outcomes	Ultimate Outcomes
Building/Maintaining relationships	First Nations and Inuit communities Provinces/Territories Services providers	Consultations and relationships established and maintained with partners	Improved communication Improved funding alignment	Contribute to safer and more sustainable First Nations communities
Negotiate and renew agreements	First Nations and Inuit communities Provinces / Territories Service providers	Establishment and maintenance of culturally appropriate police services	First Nations have access to culturally appropriate policing servicesEffective and efficient policing services
Monitoring	Band council / Tribal councils Community police services Police Governing Authorities Provinces / Territories	Increased awareness of responsibilities and obligations	Compliance with terms and conditions of the contribution agreements
Policy development and analysis	First Nations and Inuit communities Provinces / Territories Key Federal Departments Other essential service providers (e.g., nurses, teachers, court workers, social workers)	Identification of emerging issues Identification of the interdependency of Aboriginal issues	Anticipation of and response to policy issues affecting the program Consideration for policing and public safety in broader Aboriginal initiative
Support for community police governance/advisory bodies	First Nations an Inuit communities:PGA/CCG/ PMB/PSCPolice community (e.g., RCMP provincial, territorial, police services and First Nation)	Training and development, (including PGA workshops, First Nations Policing Summits)information sharing and networking	Appropriate community capacity of police commissions, public security committees, community consultative groups, police management boards

4.0 Ongoing Performance Measurement

4.1 Performance Measurement

Performance measurement forms the nucleus for managing for results.  The relationship between performance measurement and program evaluation is mutually supportive and an important element of the APD's strategic management process.  Data and information collected from a variety of sources is used to:

• Strengthen accountability, internally and externally;
• Enhance policy and program decision-making;
• Improve program implementation and administration;
• Assist APD in determining effective resource use; and
• Support strategic planning and the setting of priorities.

4.2 Ongoing Performance Measurement Strategy

Ongoing performance measurement will rely extensively on existing documents and databases.  This will provide management with the required information on the use of resources, outputs, as well as immediate outcomes.

A database will be used to capture ongoing performance information.  This database will integrate some of the information that currently needs to be extracted from documents and will therefore make performance reporting more efficient and less resource intensive.

5.0 FNPP Evaluation Strategy

5.1 Evaluating a Community Policing Program

The evaluation of a community policing program is complex in that the concept of community policing has been implemented in a variety of ways depending on the vision of the community, its demands and characteristics, resource availability, and an array of other factors that influence community-based initiatives such as the role of other levels of government.  Moreover, community policing is a philosophy of service delivery predicated on two key goals: police-community partnership and cooperative problem solving.  The policing practices in support of these goals, while common in intent, are often modified to respond to the specific needs, expectations, and capacity of the communities being policed.

The design of a community policing program evaluation is, therefore, challenging on two fronts: defining the parameters within which to assess the effectiveness of service delivery, particularly across multiple communities, and determining the appropriate metrics by which to interpret results and from which to arrive at concrete conclusions.

The use of traditional performance measures and indicators (i.e. number of calls answered, response time, reports written, clearance rates etc) doesn't capture the many other important contributions that police make to the quality of life in a community and its members.  As such, multiple measures are called for to better illustrate the cause and effect relation between public safety and community quality of life.  This calls for the use of qualitative measures that describe, rather than count, outcomes.  Therefore, the discovery, testing, and confirmation of suitable indicators usually with longitudinal data via evaluation research are typically longer-term, heavily reliant on community participation, and can be costly.

5.2 Evaluating the FNPP

5.2.1 Approach

An outcome-based evaluation of the FNPP is planned for 2005-06 to advance the overall understanding and verification of the impact of dedicated community policing in First Nation communities.  An important consideration in designing and planning the evaluation is the issue of attribution for sequential outcomes over time, particularly the long-term. The ultimate, desired outcome is safer and more sustainable First Nations communities.  Since community safety and stability is a collaborative effort involving other essential service providers and community interests, it is not feasible to measure and attribute progress solely to the FNPP.  Rather, the FNPP's relevance to the desired end result, via the application of its short and medium term outcomes, is the focus.

In completing the framework, the focus of the evaluation will be reviewed and refined based on: the resource available for the evaluation including data collection; the consideration of any new pertinent issues; how best to integrate the different aspect of the two policing models (self-administered agreements and RCMP Community Tripartite Agreements); and the availability and completeness of data.  Participation of partners in the collection of data, assessment of finding, and development of conclusions is important given that the FNPP is a tripartite program in an area of shared jurisdiction.  Provinces and First Nations will be consulted on what information is needed; its availability; how it will be collected and how the results will be used.

In general, the evaluation will examine the impact of FNPP policing from three perspectives:

• Relevance: Does the program continue to be consistent with departmental and government-wide priorities and does it realistically address an actual need?

• Success: Is the program effective in meeting its objectives, within budget and without unwanted outcomes?

• Cost-effectiveness: Are the most appropriate and efficient means being used to achieve objectives, relative to alternative design and delivery approaches?

5.2.2 Methods of Data Collections

The data / information collection methods to collect useful information about program outputs, outcomes, and impacts could include:

• Official Records

Documents reviewed at the time of the evaluation could include, for example, audit reports, financial statements, files, management action plans and follow-up audit reports. Ongoing performance information will also be reviewed, assessed and analyzed to draw conclusion re: relevance and success of the FNPP.  In addition to what has been done on an ongoing basis, existing databases will need to be analysed in depth, or new databases may be created to help address specific evaluation issues.  This could include data from partners and other federal government departments (i.e. Indian and Northern Affairs Canada, Health Canada, Aboriginal Business Canada etc.)

• Focus Group Interviews

Focus group interviews of departmental staff, representatives of First Nations communities, provincial and / or territorial governments, academia, service providers, and other identified stakeholders could be used to record the perception of opinions of individuals vis à vis the success of the FNPP.  Depending on the depth and breadth of these consultations, as well as the time and budget available, the consultation could take the form of interviews or questionnaires.

• Survey Research

Survey research will be used to determine attitudes, values, perceptions, feelings, and beliefs of community members and in turn to address questions of community satisfaction with their police service.  For example, case studies an opportunity to take an in-depth look at particular aspects of an issue and get a qualitative and quantitative assessment of a limited number of issues; while very valuable, case studies are very resource intensive.  In the case of the FNPP evaluation, case studies will provide a qualitative and quantitative assessment of the cumulative impact of the FNPP to date in selected communities, with respect to public safety, social stability and economic development.  Similarly, benchmark surveys are a practical analytical tool to judge progress and track change over time.  An in-depth review of FNPP against other agencies and programs of a similar nature (or benchmark study) could be useful.

Is important to note that not all methods may be employed at the time of an evaluation primarily due to the availability and completeness of performance information.  However, to the extent feasible, reliance on multiple lines of evidence (e.g. facts and figures, examples, narratives, testimony, benchmarking against standard police practices) is planned.  This principle will be respected when specific issues, data / information collection methods and overall budget are determined.

5.2.3 Measures of Outcomes (Indicators)

As indicators are measures or pointers that help to quantify or describe achievement of outcomes they do not tell the whole story.  For example, indicators do not explain why progress did or did not occur - indicators only indicate.  Selecting appropriate indicators is, therefore, critical particularly in a program area where direct attribution, in the long-term, is not easily achieved as dramatic outcomes are not expected to occur within short periods of time.  The process of developing appropriate and practical outcome measures that are practical and accurately reflect the progress towards the desired outcomes will be an evolutionary one with two notable features:

1. reliance on indirect measures in cases where no direct measure is available; and
2. use of both qualitative and quantitative indicators to more fully capture the contribution of the FNPP over time (for example, the perceptions of community members are best understood within the qualitative mode of inquiry).

5.2.4 Resource Considerations

The evaluation, for completion on 2005-06, will be carried out via an independent consultant.  The consultant will be responsible for reviewing and confirming the evaluation framework and the appropriate methodologies (according to factors identified above, i.e. issues and resources); assembling the appropriate data and information, undertaking the analysis, consulting with APD and key stakeholders on the findings and observations, and preparing the final report by March 2006.  The cost of this work is estimated at $250,000.

In addition, it is estimated that APD will spend $500,000 to complete research projects and fill data gaps, consult with stakeholders on the evaluation framework, support the work of an FNPP evaluation steering committee, comprised of key partners and stakeholders, and re-design its data collection processes.  The estimated total cost of $750,000 will be absorbed by APD as part of the new funds approved for FNPP program expansion in the 2003 Budget.

5.3 Measuring Performance and Results - Summary

Evaluation Issues	Indicators	Source of Information	Responsibility for Data Collection	Timing / Frequency
				Ongoing Monitoring	Summative Evaluation
1.Program Relevance
a) Does the FNPP continue to reflect federal government and departmental priorities?	congruence with federal policy priorities	Throne Speech budgets economic updates departmental strategic plans / initiatives SGC Report on Plans and Priorities other federal initiatives	Aboriginal Policing Directorate (APD)- Policy	X
b) Does the FNPP continue to reflect the needs of First Nations communities?	level of community satisfaction measured by:	Aboriginal Policing Tracking System (APATS)	APD - Policy
	# enhanced policing agreements			X
	# renewed policing agreements			X
- identification of emerging issues	# policy issues and priorities identified	HQ	APD - Policy	X
- identification of interdependency of Aboriginal issues	# inputs into OGD and provincial initiatives	HQ and Regions	APD - Policy	X
	# initiatives led by APD	HQ	APD - Policy	X
	# policy analysts	HQ	APD - Policy	X
	% program $ to policy	HQ	APD - Policy	X
- training and development workshops (i.e. PGA workshops, First Nations policing summits)	# people participating in training workshops	HQ	APD - Policy
	# training days	HQ	APD - Policy	X
	# training workshops	HQ	APD - Policy	X
	evaluation of capacity and effectiveness of governing bodies	expert opinions, community surveys, consultation with PGAs and FNCPA	APD - Policy		X
2. Program Successa) To what extent has the FNPP been successful in achieving its intermediate objectives?
-improved communication	# and type of consultation with partners	HQ and Regions	APD - Operations and Policy	X
	analysis of the quality of relationships	Interview of partners	APD - Operations and Policy		X
	# and types of agreements reached	APATS	APD - Operations	X
-improved funding alignment	analysis of funding congruence	Regions	APD - Operations	X	X
-First Nations have access to culturally appropriate policing services	# and % Aboriginal police chiefs (self-admin agmts only)	APATS	APD - Operations	X
	# and % Aboriginal police officers	APATS	APD - Operations	X
-effective and efficient policing services	# reported cases	CCJS/regions	APD - Operations	X
	case load per officers	CCJS/regions	APD - Operations	X
	# and type of case clearance	CCJS/regions	APD - Operations	X
	# incidents of assistance to general public	CCJS/regions	APD - Operations	X
	Analysis of community satisfaction re: cultural appropriateness and effectiveness	Survey of QFNCPA, RCMP surveys, case studies, annual RCMP reports to communities	APD - Policy		X
	Analysis of police: population ratio	Review of FNPP agreements	APD - Operations		X
	Analysis of police per capita costs	Review of FNPP agreements	APD - Operations		X
-compliance with terms and conditions of contribution agreements	Analysis of cost per officer	Review of FNPP agreements	APD - Operations		X
-anticipation of and response to policy issues affecting the program	# and % of audits with no or minor issues	Review of CAC audit findings	APD - Operations	X
	# policy issues resolved	Survey (Policy)		X
	analysis of policy-making effectiveness	Interview of central agencies, OGDs, departmental staff			X
-consideration of policing and public safety in broader Aboriginal initiatives	# initiatives that seek, in part, to improve public safety	analysis of new initiatives (principles and priorities)	APD Policy	X
appropriate community capacity of police commissions, public security committees, community consultative groups, police management boards	% management boards set-up	survey of community oversight bodies	APD-Operations	X
	analysis of training efforts conducted by 3rd parties	participant surveys as part of funding agreement	Police colleges and other organizations leading training activities	X
	-# training sessions
	-% CCG/PMB participating in training
c) To what extent has the FNPP contributed to safer and more sustainable First Nations communities (ultimate outcome)?	community wellness / stability measures, over time, that may demonstrate casual relationships (negative or positive) as a result of an on-sight police service:	surveys, case studies, anecdotal evidance	APD - Policy (in partnership with other federal and provincial agencies)		X
	examples: crime, health, economic development, education, employment, statistics, community housing and infrastructure conditions.
d) Did the FNPP result in any unintended outcomes, either positive or negative?	positive and / or negative outcomes examples: increased volunteerism OGD programs work better (medical services, economic development) better coordination between local service providers withdrawal of provincial police presence raised expectations of First Nations - not realized hiring of security guards raised expectations e.g. INAC v-a-v FNGA and self-govt overlaps and gaps in service (e.g. emergency services)	anecdotal and survey information	APD-Policy		X
3. Cost- Effectiveness
Is the FNPP a cost effective way of achieving stated (logic model) outcomes?	case study of a community that has recently transitioned to a new type of service (before and after analysis using cost comparisons and testimonials, based on socio-economic examples)	A First Nation community recently undertaking transition from provincial policing to FNPP	APD - Policy, province, First Nation community		X
4. Program Administration
a) Is a contribution program the most effective delivery mechanism?	examination of other federal programs and services and how they are delivered - analyze in terms of pros / cons and resource implications	consultations and file \ document review	APD - Policy		X
b) Is APD's decentralized delivery model effective for achieving the stated outcomes?	opinions v-à-v efficiency and effectiveness assessment of partner satisfaction	survey of First Nations, provinces and territories	APD - Policy		X
c) Does APD have an effective performance measurement strategy?	extent to which performance indicators currently available are used by management	APATSmonitoring the execution of management action plansqualitative information from interviews with management,records of decisions from retreats, management meeting vis-à-vis program priorities, implementation or administration	ADP - Policy		X

6.0 Reporting Strategy

Progress will also be assessed against the targets identified in the Department's Report on Plans and Priorities (RPP).  Additionally, the Department's 2003-2006, Sustainable Development Strategy sets out specific targets related to research that will directly support the FNPP evaluation.  Yearly progress vis à vis these targets will also reported in the Departmental Performance Report (DPR). The results of the summative evaluation will be provided in the form of an in-depth evaluation report, submitted to the Department's Management Review Committee (MRC) for approval.  This report, once approved, will become a public document and, as such, accessible to others upon request.  The evaluation strategy will be reviewed on an annual basis and adjustments will be made, as required, to ensure that the performance indicators, research results and information are appropriate and useful for ongoing management requirements.

Ongoing assessment of the results achieved and the resources involved during the previous fiscal year will be contained in the Departmental Performance Report.  This strategy is expected to lead to better provincial and national reporting of program activity and short term outcomes by way of the indicators identified in Section 4 to monitor ongoing performance.  The results of ongoing performance measurement will be used to make the necessary adjustments to the program and new knowledge gained through this process will be integrated into program policy decisions as and when appropriate.

Footnotes

^[1] Solicitor General Canada, Performance Report, for the period ending March 31, 2002.

^[2] Solicitor General Canada, First Nations Policing Policy, revised 1996.

^[3] Planned spending in fiscal year 2003-2004 and 2004-05 includes $18M and $24M, respectively, for the FNPP as announced in the February Budget.

^[4] The Policing and Law Enforcement Branch provides funding under the Measures to Combat Organized Crime Initiative and the Tobacco Control Strategy for the Akwesasne Mohawk Police Service to participate in joint force operations to combat cross-border criminal activities.

^[5] Operational audits deal with broad performance issues, focusing on the extent to which police service operations have been effective including the adequacy of management systems, controls, and practices, including those intended to control and safeguard assets to ensure due economy, efficiency, and effectiveness.  Operational audits are the purview of provincial partners and may be cost-shared with the federal government.