2013-2014 Evaluation of the Aboriginal Community Safety Development Contribution Program

PDF (524 KB)

Table of Contents

EXECUTIVE SUMMARY

Evaluation supports accountability to Parliament and Canadians by helping the Government of Canada to credibly report on the results achieved with resources invested in programs. Evaluation supports deputy heads in managing for results by informing them about whether their programs are producing the outcomes that they were designed to achieve, at an affordable cost; and supports policy and program improvements by helping to identify lessons learned and best practices.

What we examined

The Aboriginal Community Safety Development Contribution Program (or the “Program”) was developed as part of the government's concrete action on missing and murdered Aboriginal women to help communities respond holistically to community factors. The objective of the Contribution Program is to support communities to develop an efficient, integrated approach to maximize government investments by supporting three broad activities: developing community capacity (knowledge building/sharing); supporting communities to develop community safety plans; and/or supporting community based pilot projects designed to explore and implement holistic, Aboriginal healing models responding to the safety needs of Aboriginal women and girls.

Why it is important

Aboriginal women and girls face continuing high risk factors on and off reserve. Integrated, comprehensive approaches move communities from fixing problems to building communities into a sustainable society. Supporting communities to more strategically tailor their responses to community safety allow programs to build on existing resources while optimizing communities' ability to access existing and new sources of funding.

What we found

Relevance

There is ample evidence in available documents and relevant literature, and from interviews with both internal government and external key informants, that the Aboriginal Community Safety Development Contribution Program remains relevant. The need for improved safety in Aboriginal communities especially for women and girls remains, and the Program is well aligned with federal government and Public Safety Canada priorities in its approach to the issue of safety in Aboriginal communities. It appears to be working in concert or in parallel with a number of similar initiatives but there is no evidence of undue duplication of effort.

There is, however, an important opportunity for greater coordination among federal departments and agencies and among different levels of government in working to address community safety and the root causes of violence in Aboriginal communities. The Program offers a method for encouraging this kind of coordination.

Within Public Safety Canada, there is the potential for closer integration of this Program with the crime prevention and aboriginal policing units in encouraging and supporting community-led planning and priority setting in relation to community safety.

Performance

It is evident that the Program provided Aboriginal communities with tools and opportunities to identify their key safety issues and their collective resources that could be used to work towards community safety and healing, but it is unclear from the available information what the views of community participants are about the quality or effectiveness of those tools. There appears to be a long-term benefit to the tool development process in that once they are in place and used by communities, the information is readily available for other initiatives.

Despite some early challenges there appears to have been some success in training trainers for the community workshops, and success as well in conducting the workshops, with some evidence of improved community motivation to address issues related to community safety. The idea of having Aboriginal facilitators with local knowledge was viewed positively, but there was reportedly some variation in skills and experience of facilitators. Training of facilitators reportedly did not sufficiently meet the need, especially in high-risk communities. Program managers are aware of the need for improvement and have planned changes in recruitment and training strategies.

Program documents show that by September 2012 the program had delivered workshops in 25 communities. These communities were more aware of their issues and there was a rise in community motivation to deal with the issues. To date Public Safety Canada has received nine safety plans. The modest numbers of workshops and community safety plans are plausibly the result of the nature of the program design, which acknowledges the long-term nature of the challenge especially in high-risk communities, and recognizes that progress is very likely to be slow and incremental. The large majority of participating communities progressed in ways in keeping with program objectives, with some having produced community safety plans and others still working on earlier stages of readiness.

The Program has successfully initiated community engagement, but sustaining the engagement has proved challenging. Most observers believe that ongoing support is required for this sort of community development process to succeed, especially given the fact that Aboriginal community leadership is typically extremely busy with day-to-day operations.

Because of the community-driven nature of the Program there appears to be considerable variation in the issues addressed in community workshops and in the community safety plans developed to date. Not all of the work has centred on community safety directly. However, it is evident that the community workshop and planning processes are engaging participants in discussions about the root causes of violence in their communities, which is viewed in the literature and among experienced Aboriginal community developers as an essential early step in improving the health of high-risk communities.

It is premature at this point to draw conclusions about the impacts of the Program on levels of safety in participating communities as compared to before the initiative or as compared to other similar communities. At this point there does not appear to be agreement about what constitutes a safe community and how that would be measured. It will be worthwhile going forward to consider establishing targets for progress that reflect realistic expectations.

In terms of Program efficiency, the average program administration ratio was 93%. This result is expected since this is a unique program that requires a high level of support from program staff. It required gradual adaptation of approaches and tools as experience was gained in the field.

There appears to be considerable opportunity for greater efficiency on the part of the federal government as a whole through greater coordination and collaboration in working with Aboriginal communities. The Program offers a basis for such improved synergy through its support of community planning and its reported establishment of trusting working relations in communities. At its best, the Program offers a platform to transform the way departments and agencies operate in addressing complex, interdependent issues such as those experienced in many Aboriginal communities.

There is near-unanimity among internal and external observers and in the literature that the community-led approach adopted by the Program is the required path to eventual health and safety in Aboriginal communities, and that pursuing that path to good effect will require patience and continuous effort and support to communities over the long-term.

Recommendations

The Internal Audit and Evaluation Directorate recommends that the Assistant Deputy Minister, Community Safety and Countering Crime Branch of Public Safety Canada, implement the following:

In collaboration with the Assistant Deputy Minister Emergency Management and Programs Branch:

The ongoing collaboration would allow the department to leverage results across programs and maximize departmental investments.

Management Response and Action Plan

Management accepts all recommendations and will implement an action plan.

1. INTRODUCTION

In the government of Canada, evaluation is the systematic collection and analysis of evidence on the outcomes of programs to make judgments about their relevance, performance and alternative delivery methods to achieve the same results. To that end, the Treasury Board Policy on Evaluation requires that all ongoing programs of grants and contributions are evaluated every five years to support policy and program improvement, expenditure management, Cabinet decision making, and public reporting.

This report presents the results of the Public Safety Canada (PS) 2013-2014 Evaluation of the Aboriginal Community Safety Development Contribution Program (ACSDCP). It was created in 2010 to support communities in developing efficient, integrated approaches to community safety to enable safer, more secure Aboriginal communities. This evaluation provides Canadians, parliamentarians, Ministers, central agencies, and the Deputy Minister of Public Safety with an evidence-based, neutral assessment of the relevance and performance (effectiveness, efficiency and economy) of this federal government program. 

2. PROFILE

2.1 Background

The ACSDCP was developed as part of the government's concrete action on missing and murdered Aboriginal women to help communities respond holistically to community factors. Integrated, comprehensive approaches move communities from fixing problems to building communities into a sustainable society. This translates into healthier communities for all of the community's residents, directly through integrated programming, and indirectly by being responsive to individual's needs and aspirations.

Various funding programs currently exist to respond to violence against women, however, in many cases, these programs respond to specific issues only, and do not allow communities to co-ordinate or combine their responses. Supporting communities to more strategically tailor their responses to community safety allow programs to build on existing resources while optimizing communities' ability to access existing and new sources of funding.

By playing a leadership role, through transfer payments, the initiative is structured to link community solutions with policy development. The starting points are community-based projects taking into account the safety needs of Aboriginal women and girls. The end points are increased knowledge within communities and government as well as effective and appropriate government policies.

2.2 Program Objectives

The objective of the ACSDCP is to support communities to develop efficient, integrated approaches to maximize government investments. This will be accomplished by supporting three broad activities:

Only the pilot projects will be directly responding to violence against women and girls. The capacity building and community safety plans are designed to help communities generally and as such will not be measured for its direct influence or impact on Aboriginal women and girls.

2.3 Resources

Table 1 illustrates the initial funding allocations for the ACSDCP.

Table 1 – Funding allocations ($ in dollars)
2010-11 2011-12 2012-13 2013-14
Salary 77,592 258,649 258,649 258,649
O&M 76,803 175,997 365,997 365,997
EBP 15,518 51,730 51,730 51,730
Total Vote 1 169,913 486,376 676,376 676,376
G&C 320,000 480,000 690,000 690,000
Total Vote 5 320,000 480,000 690,000 690,000
Accommodations 10,087 33,624 33,624 33,624
Total 500,000 1,000,000 1,400,000 1,400,000

2.4 Logic Model

The logic model is a visual representation that links what the program is funded to do (activities) with what it produces (outputs) and what it intends to achieve (outcomes). It also provides the basis for developing the evaluation matrix, which gave the evaluation team a roadmap for conducting this evaluation.

Figure 1 – Logic Model for the Aboriginal Community Safety Development Contribution Program

Image Description

The following are the five activities of the program: tool development/dissemination; training/networking; development of community plans; pilot projects; strategic linkages.

Activity 1: Tool development/dissemination lead to the following outputs: identify knowledge gaps, develop knowledge base, and share knowledge. These outputs lead to the following immediate outcomes: increase in knowledge base; increase in applicability of tools/knowledge; broader range of tools/knowledge.

Activity 2: Training/networking lead to the following outputs: develop training teams; community exchange; develop networks; improve accessibility of resources. These outputs lead to the following immediate outcomes: improved community capacity; improved information and knowledge; improved readiness.

Activity 3: Development of Community Plans lead to the following outputs: undertake community consultations or needs assessments; define priorities and strategies; access expertise. These outputs lead to the following immediate outcomes: communities able to set priorities and develop capacity; service providers work collaboratively to respond in an integrated manner; funders target their programs more effectively.

Activity 4: Pilot projects lead to the following output: contribution agreement to support projects with selected communities to test an innovative response to violence against women and girls. This output leads to the following immediate outcome: implementation of promising model.

Activity 5: Strategic Linkages lead to the following output: develop strategic partnerships with other federal organizations and stakeholders. This output leads to the following immediate outcome: other government programs are leveraged to enhance ACSD effectiveness

Immediate outcomes for activities 1, 2, 3 and 4 lead to the following intermediate outcomes: communities develop and/or implement efficient, comprehensive and integrated approaches in response to safety issues; improved federal policy based on knowledge of community needs and approaches that work; resources are optimized; the root causes of victimization are systematically identified and/or addressed.

Intermediate outcome for activity 5 is: contribution to the alignment of government programing and policy decisions.

All intermediate outcomes lead to the following final outcome: Safer, more secure Aboriginal people & communities.

Note: Strategic Linkages were not part of the initial logic model developed at the onset of the Program. Therefore, objectives for these activities were not defined and tracked by the Program. However, it was assessed in this evaluation.

3. About The Evaluation

3.1 Objective

The objective of the evaluation is to provide Canadians, parliamentarians, Ministers, central agencies, and the Deputy Minister of Public Safety with an evidence-based, neutral assessment of the relevance and performance (effectiveness, efficiency and economy) of this federal government program.

More specifically, this evaluation supports:

3.2 Scope

The evaluation assessed the outputs and outcomes of the ACSD Contribution Program starting in 2010-2011. The administration and oversight of the contribution agreement by PS was also assessed. 

3.3 Methodology

This evaluation was conducted in accordance with the Treasury Board of Canada Policy on Evaluation, the Standard on Evaluation for the Government of Canada and the PS Evaluation Policy. The evaluators used a goal-based, implicit design approach.

3.3.1 Evaluation Core Issues and Questions

As required by the Directive on the Evaluation Function, the following issue areas and evaluation questions were addressed in the evaluation:

Relevance
Performance-Effectiveness
Performance-Program Administration/Efficiency and Economy

3.3.2 Lines of Evidence

The methodology for the evaluation included the following lines of evidence:

Literature Review

Canadian and international academic research and Non-Government Organization documents focusing on Aboriginal crime prevention approaches and community development were reviewed.

Document and Administrative Data Review

The document review included the following types of documents: reviews, corporate documents, accountability and policy documents, inception documents, reports on plans and priorities, speeches from the Throne, government reports on issues relevant to the ACSDCP as well as other independent reports. A list of documents reviewed is presented at Annex A.

Key Informant Interviews

Interviews were conducted with twelve key informants including ACSDCP officers and managers, PS senior managers, officials in other federal government departments and agencies with an interest in Aboriginal community safety, and independent experts with insights into community development and the safety of women and girls in an Aboriginal context. The interviews focused mostly on relevance. The evaluation also relied on interviews conducted as part of the Program Review, which focused on effectiveness, as described below.

Analysis of Program Review Findings

Because the program recently conducted an internal program review that covered many of the same issues as this evaluation, findings from the review have been used as a line of evidence for the evaluation. Those findings focused mainly on issues of effectiveness using key informant interviews and on-site community case studies. The coordination of findings from the two studies avoided duplication of effort and allowed for cost savings in the conduct of the evaluation.

3.4 Limitations

The following section describes data limitations.  

3.5 Protocols

This report was submitted to program managers and to the responsible Assistant Deputy Minister for review and acceptance. A Management Response and Action Plan was prepared in response to the evaluation recommendations. These documents were presented to the PS Departmental Evaluation Committee for consideration and for final approval by the Deputy Minister of Public Safety.

4. Findings

4.1 Relevance

4.1.1 Continuing Need

In order to establish if there is continuing need for the PS Aboriginal community safety and community development activities, the evaluation examined available literature on violence against Aboriginal women and on Aboriginal community development, and PS and other federal documents on the issue. It also interviewed external experts, federal government officials working on Aboriginal issues and PS officials. These sources demonstrated convincing evidence of a continuing need.

Violence against women, which remains a problem in Canada and throughout the world, is compounded for Aboriginal women. The literature reviewed for the evaluation highlights the continuing high risk factors that Aboriginal women and girls face on and off reserve. There are over 430,000 First Nation women aged 15 and older on and off-reserve, and more than 150,000 Metis and Inuit women. They face three times the risk of violence that non-Aboriginal women do, in particular from family members and acquaintances.1 Aboriginal women are particularly vulnerable to spousal violence. Nearly one-quarter (24%) of Aboriginal women reported some form of spousal violence. This is about three and a half times greater than among non-Aboriginal women (7%).2 

Aboriginal women represent about 4% of Canada's female population and about 16% of all female homicides, according to a recent RCMP study.3 Numerous articles point to contributing factors such as intergenerational cycles of abuse and violence that are community-wide, linked to historic trauma such as residential schools, have resulted in situations where victims are sometimes actively discouraged from speaking out, particularly in smaller and more remote communities, where police may be some distance away. Clusters of other risk factors, such as high levels of poverty, over-crowding, substance abuse, family breakdown, etc. increase women's vulnerability both with Aboriginal communities and in the larger society. Aboriginal women face a higher rate of violence at the hands of strangers, acquaintances and from family members and a higher rate of homicide. Because the factors underlying the disproportionate levels of violence are complex and interrelated, there is a need for longer-term sustained and coordinated measures to effect significant change.4

Some authors5 and several expert informants interviewed for this evaluation suggest that the road to resolving this problem lies in community-led processes designed to acknowledge the past and its consequences, and develop solutions that are based on traditional local values. Program documents cite the active and enthusiastic participation of communities (i.e., the demand for the ACSDCP) as an indicator that there is still a need. Reports of community workshops held under the auspices of the ACSDCP document discussions among community members that emphasize the existence and intransigence of the problems the program is designed to address.

One external expert key informant pointed out that in her work in Aboriginal communities they rely primarily on commonly-used sources of statistics on violence against Aboriginal women as a “break the silence” strategy, and that the communities they work with often provide their own local statistics to further emphasize the nature and extent of the problem. Several federal government key informants pointed out that communities participating in the ACSDCP targeted issues of particular local concern, and that once trust was established in the community development process these issues could be in the nature of sexual exploitation and other highly sensitive issues.

4.1.2 Alignment with Federal and Departmental Priorities

A review of federal government documents and interviews with federal key informants indicated that the ACSDCP is well-aligned with federal and PS priorities.

Both the document review and the interviews pointed to government priorities aligned closely with the objectives of the ACSDCP. Key points raised included references in the 2013 Speech from the Throne noting that “Aboriginal women are disproportionately the victims of violent crime. Our Government will renew its efforts to address the issue of missing and murdered Aboriginal women." It is also aligned with the PS strategic outcome of community safety and resilience.

Documents noted that both Justice Canada and PS received funds from 2010 to 2015 to improve law enforcement response and reduce violence against Aboriginal women and girls, with a clear mandate to carrying out this reduction in violence. Interview respondents were unanimous that the program is closely aligned with both governmental and PS priorities. At the federal level, interviewees said that the program is aligned with government priorities on building safe communities, the safety of Aboriginal women and girls, and Aboriginal economic participation. 

4.1.3 Alignment with Federal Roles and Responsibilities

Aboriginal Community Safety falls within the mandate of PS as stated in the Public Safety and Emergency Preparedness Act in Section 4. (2): “The Minister is responsible for exercising leadership at the national level relating to public safety and emergency preparedness” and Section 6. (1): “In exercising his or her powers and in performing his or her duties and functions and with due regard to the powers conferred on the provinces and territories, the Minister may: … b. cooperate with any province, foreign state, international organization or any other entity; c. make grants or contributions; d. facilitate the sharing of information, where authorized, to promote public safety objectives”.

The program supports the Minister's public policy leadership role in corrections and criminal justice. The Department works to increase the efficiency and effectiveness of the criminal justice system, including a focus on mental health issues in the correctional system and at-risk or marginalized populations in communities. Through the ACSDCP PS plays a key role to ensure that communities have the knowledge and ability to improve community safety and to assume responsibility for corrections and healing.

A large majority of internal government key informants believe that the program is in keeping with federal roles and responsibilities, although the cross-jurisdictional nature of the issues the program is designed to address are well-acknowledged. The view is that many Aboriginal communities are not capable by themselves of emerging from poverty, poor health and other associated challenges, and it is fitting that the federal government take a leadership role and work directly with communities. The ACSDCP is viewed by those who know about the program as a first step in establishing a framework that will enable the participating communities and all levels of government to work more effectively together to bring about positive change.

Several key informants pointed out that the ACSDCP resembles a previous program at Indian and Northern Affairs Canada6 in the 1990s which recognized an important federal role. It was called the Comprehensive Community Planning Program, which was based on the notion that communities need to take charge of themselves with the government as a supporter, not a driver. However, at the time many communities did not have a sufficient level of readiness or community cohesion to build a meaningful plan, and the program was not renewed. The ACSDCP is viewed by key informants as working to address this readiness issue in its design.

External experts agreed that there is a federal role, but cautioned that the initiative must come from within the communities, and that there is a perceived risk that federal initiatives often assert undue control.

4.1.4 Complementarity with Other Related Government Initiatives

Documents reviewed for the evaluation note that in 2010 Canada announced $10 million allocated to address violence against Indigenous women in Canada, and that the ACSDCP is part of this overall initiative. The documents also note that PS goals of a safe and resilient Canada and community safety overlap with this program, and that Aboriginal Affairs and Northern Development Canada, Justice Canada, Health Canada, the Royal Canadian Mounted Police (RCMP) and other initiatives at PS are all working in this direction from various program and mandate perspectives. These programs are seen to complement each other rather than duplicating effort.  

Within PS, documents and key informants point to the clear and close relationship between the ACSDCP, the National Crime Prevention Strategy and the First Nations Policing Program, in that all three seek to improve Aboriginal community safety recognizing the importance of community-based planning and priority setting. Key informants suggest that the ACSDCP presents an opportunity to streamline efforts to work with Aboriginal communities in meeting the objectives of all three programs, and that the recent reorganization within PS could facilitate this streamlining and improved coordination among these three program areas.

Government key informants also pointed to non-governmental organizations initiatives with related objectives in relation to violence and crime prevention and suicide prevention, including Red Cross programs “Respect Ed” and “Walk the Circle”. Again, these kinds of programs are referenced as being complementary and not duplicative. One external expert referred to a B.C. initiative through the Legal Services Society and the Ending Violence Association of B.C., which was described as similar to the ACSDCP, but with a starting point of workshops to deliver information on the law and on violence against women. Ultimately the approach seeks to assist communities to develop action plans and safety plans.

According to the program's internal review, the program is seen as complementary to a variety of other government initiatives that adopt a horizontal approach in working with Aboriginal communities. These include the Ring of Fire initiative in Northern Ontario (participation in mining development), RCMP efforts to mobilize Aboriginal communities to respond to gang violence, Saskatchewan crime prevention strategies, Health Canada's First Nations and Inuit Health Branch interest in responding to violence in Aboriginal communities, and the Urban Aboriginal Strategy led by Aboriginal Affairs and Northern Development Canada.

4.2 Performance-Effectiveness

Performance was assessed primarily by drawing on the findings of a recent ACSDCP internal review that included on-site case study analysis. In addition, the evaluation reviewed available program documents and data and included questions on performance in the key informant interviews.

4.2.1 Tool Development and Dissemination

The program review noted that ACSDCP provided Aboriginal communities with tools and opportunities to identify their key safety issues and their collective resources that could be used to work towards community safety and healing, but it did not comment on the quality or effectiveness of those tools. Program officers report that there has been considerable success with the tools that have been developed, and that facilitators working on behalf of the program in communities often bring other tools as well to share with community members. They also describe tool development as a gradual process of improvement to find suitable tools for particular community circumstances. Those officers say that they have received positive feedback, especially as the tools have been improved over time.

There is a reported long-term benefit to the tool development process in that once they are in place and used by communities, the information is readily available for other initiatives, so other government programs, for example, can benefit from the same tools.

4.2.2 Training and Networking

Documents on earlier phases of the initiative show successful training the trainer sessions, including 18 trainers in 2011. One document noted that 60% of community workshop participants were seeing improvement in community motivation to address issues related to community safety, and attributing that in part to the sessions.

Program officers reported that 26 facilitators are now trained. They said the program had hoped for a strong facilitators' network, but that only two of the original 22 facilitators applied under the program's recruiting process in 2012, and four additional facilitators came on board later, so they now have a complement of six, (with four working at the moment). The program is starting a new recruitment process and working with facilitators (current and past) to help them apply, since the previous process was found to be quite onerous for individual applicants.

The program review found that:

4.2.3 Community Safety Plans

Program documents show that by June 2013 the program had delivered workshops in 25 communities out of 50 initially contacted. Out of the 25:

Positive feedback from communities was received and the following key outcomes were achieved: more awareness of community issues and a rise in community motivation to deal with the issues.

Currently, nine safety plan proposals have been funded. These plans reflect the safety issues that the community has prioritized and is prepared to take action on. A few reflect comprehensive, integrated approaches to addressing safety issues that will require additional funding or supports from external agencies. Most plans need to be refined to be useful in guiding actions on the issues identified.

Interview respondents identified challenges in developing the plans. Some are not complete because of changes of leadership at the community level or changes in circumstances among leading participants in the planning process. As well, there was found to be a limited capacity in some communities, where planning leaders were managing the process on the side of their desks while maintaining responsibility for a host of other functions in the community. Program officers and external experts noted that for this initiative to be successful, on-going support to communities is often important.

Key informants found that there have been some modest gains in engaging service providers working in the communities, but that this was a step requiring contributions from well beyond the program itself. One barrier noted was that delivery agencies sometimes showed a reluctance to engage actively in the communities' plans and in integrating with other service providers due to perceived restrictions in mandates and the fear of breaches of confidentiality, especially in the health and social services areas.

The program review found that:

4.2.4 Pilot Projects

Two pilot projects are currently underway. They are designed to test alternative approaches to community development relating to the safety of Aboriginal women and children, and the root causes of violence in Aboriginal communities. One is in Peepeekisis Saskatchewan and the other is in an urban setting-Thompson Manitoba. In the next phase of the ACSDCP, there are plans to conduct three more pilots focused more directly in violence against women.

The Peepeekisis project, focused on spousal abuse, is scheduled to be completed in March 2015 and is currently being evaluated independently of this current ACSDCP evaluation. The project is viewed by respondents as being highly successful, having gone well beyond its initial scope. It included a 52-week training program for community leaders, and has a powerful and active advisory body of elders. The support group for the project meets on a regular basis and includes couples from a range of age groups. There is a connection with the courts, since some people are allowed to participate in this as a parole condition. It is viewed at this stage as a really good model.

The Thompson project is also viewed positively by respondents. It is still in operation, and is aligned with the Urban Aboriginal Strategy in Thompson, and therefore was able to link with an established steering committee of Aboriginal and non-Aboriginal community leaders and service providers, and government representatives. It has been comprised of a series of smaller projects that have resulted, according to one respondent, in a dilution of the project's objectives. However, there have been positives in terms of being able to link the project to existing capacity building initiatives in the community.

4.2.5 Development of Integrated Approaches

Internal and external key informants pointed to the need for ongoing connections and supports once plans are in place, to help ensure that implementation takes place and that communities are able to connect with available programs and services that could help them make progress. PS senior managers pointed out that PS is in a unique position to provide this kind of post-plan support because of the positive working relationships staff have developed with the communities. These relationships present an opportunity to leverage improved overall coordination of federal programming in Aboriginal communities, and to provide guidance to community leaders in their dealings with federal departments and agencies. Health Canada recognized this opportunity and provided the ACSDCP with funds to engage some particularly challenged communities using the program's engagement and planning model.

In terms of the impacts of the ACSDCP on community engagement, internal key informants believe that more communities have an active approach to community safety in place than before the ASCDCP process, which is leading to a more integrated approach with the support of other government agencies. However, they cautioned that when community members/staff go back to their day jobs it is hard to maintain the integrated approach set out in the plans. The hope is that in the next phase of the program, they will strengthen the engagement of communities with other agencies that have people and funds to help with implementation, and in that way those who wrote the plan will keep on top of it and build on early success.

Key informants associated with the ACSDCP and those working in other federal departments and agencies all made references to the fact that an initiative such as the ACSDCP is by its nature long-term, requiring ongoing support, and that expectations of results at the community level should be modest, especially in the early years. Partly this is because of the capacity building elements of the initiative, and the fact that many of the participating communities fall into the “most in need” category by design. As well, respondents said, the program seeks to identify root causes of violence in the communities, and to start the process of addressing those root causes, while at the same time developing an immediate approach to limiting risk to individuals in the communities. A large majority of key informants hold the program in very high regard and consider that the gains that been achieved to date, while modest in relation to long-term objectives, are substantial and meaningful.

One respondent associated with the ACSDCP pointed out that there are different levels of success. For some a proposal, actually done by the community, to do a community safety plan is a great step forward. For others, the process of doing a plan as a community is the biggest step. For still others, the focus is on what comes after the plan is in place. “Governments have been so negative regarding Aboriginal communities for so long. Characterizations are usually about what is lacking, what is wrong, what capacity is missing. But there is huge resiliency there-they are strong people and we are trying to work with that. Success is reaching that key turning point where the community is empowered and optimistic about the future.”

External experts consulted for the evaluation were not sufficiently aware of the ACSDCP to be able to comment on its effectiveness, except to say that the overall approach of a community-led planning initiative with support from government, and an acknowledgement of the necessarily holistic nature of the endeavour, were solid starting points.

4.2.6 Knowledge Base for Federal Government Policy Making

Documents show that PS, in response to feedback, developed a resource guide for community development – Moving Towards a Stronger Future: An Aboriginal Resource Guide for Community Development (MTSF). This guide is available for communities, and also for government agencies working in Aboriginal communities.

Internal government key informants felt that it is too early to suggest that the ACSDCP has led to specific policy changes. However, they point out that partner federal departments and agencies working together to renew the overall federal response to the issue of missing and murdered Aboriginal women appear to agree that the process has to be centred on listening to communities, and that change has to start on the government side. In that cross-departmental renewal context, ACSDCP is viewed as quietly changing the way government does business with communities. Part of that change, they say, is improving how government agencies work together.

4.2.7 Identifying Root Causes

The founding documents of the ACSDCP demonstrate recognition that issues of violence in Aboriginal communities are a result of a complex range of root causes. Some of those causes relate to historical wrongs against Canada's Aboriginal people and damages from more modern treatment such as those associated with Residential Schools and the paternalistic nature of the Indian Act, and some likely flow from the conditions of poverty and hopelessness found on many First Nation reserves and other Aboriginal communities.

Comments from key informants focused on the diversity among participating communities in how they dealt with root causes in the course of the ACSDCP workshops and safety planning. Most said that root causes have been identified, but in an ad hoc way rather than systematically. Some communities reportedly went deep into addressing sexual violence, while others were not there yet in being able to tackle an issue of this sensitivity. The sense among respondents was that all communities engaged in the ASCDCP have addressed some root causes. This outcome reflects the complexity of the issues and the degrees of readiness of communities.

One example given was that governance can be a barrier in some communities. If community leaders are not supportive of delving into sensitive issues such as the abuse of women, for example, the community is likely not able to tackle this barrier. They instead are willing to tackle some issues that are not as sensitive/personally risky to bring up.

Even when the root causes are identified, they aren't always addressed. Several respondents said that communities know the root causes, but it is very difficult to tackle. It takes a lot of courage. One respondent said that the discussion of root causes came out a lot sooner than she expected. “The experience in the past has been that root causes are identified 10+yrs down the road. But some of these communities are dealing with root causes (like sexual abuse) in a 6-8 month period. Discussions on residential schools may have helped push this forward.” Another key informant pointed out that the program was able to ask federal partners for help in certain circumstances, such as when sexual abuse issues emerged, and the facilitators weren't prepared to address this issue themselves. Having these partnerships was found to be a great benefit.

One community tackled the issue of substance abuse. It created its own detox centre and have put 100+ people through the detox process. The resources provided through the ACSDCP weren't the most important part, but the push came in part from the workshops and planning process.

The program review found that:

4.2.8 Safer, More Secure Aboriginal People and Communities

The internal program review noted the following limited set of findings related to longer-term outcomes:

Program key informants and other PS officials interviewed took the view that expectations at this early stage have to be limited, and that the workshop and planning processes all represented incremental progress in a population of largely “high risk” communities for which the prospects for becoming healthy, safe communities may be generational in scope. Examined from this perspective, the great majority of key informants including external experts believe that the approach adopted by the ACSDCP holds promise and is making progress.

4.3 Performance-Efficiency and Economy

4.3.1 Program Efficiency and Administration Ratio

The efficiency review consisted of analysis of program documents regarding factors that influenced program results, analysis of key informant interviews, and analysis of financial data to determine the program administration ratio.

Financial information combined with program estimates approximates the four-year average program administration ratio7 at 93% (see Annex B). This result is expected since this is a unique program that requires a high level of support from program staff. This was a new initiative within aboriginal communities; program staff noted some trial-and-error in the first few years, particularly in the planning and implementation of facilitator training and community workshops. As well, key informants, particularly those directly involved in the program, point to the fact that the methods being used for the program are not suited to a highly efficient, predictable process. Communities move at the speed they are able to, and success can only come as communities gradually make progress at their own pace. In many cases communities and participants are accustomed to “being told what to do”, so it represents a significant adaptation to get used to a community-driven process. The main constraint, they say, has been that governments are accustomed to working in specific, short time frames, which are artificial and counter-productive in the context of a program like the ACSDCP. 

Most key informants also noted that the limited amount of time and resources available to the program for each community has meant that the required on-going support and follow-up from workshops and planning processes has been lacking. This has created inefficiencies in that a period of progress is often followed by a period with no progress (and often changes in Band personnel that disrupt progress). Additional resources to maintain contact with communities and provide some degree of ongoing support would enhance progress and represent greater efficiency in terms of the benefits accruing from program efforts.

Documents reviewed for the evaluation note that a Standing Offer Contract process has been implemented to identify and recruit additional community facilitators. This is expected to be a more efficient and effective approach to matching these resource people with communities compared to the contracting process used for the previous recruitment exercise, in which considerable effort was expended by the program with few applicants participating in the process. A “circuit training” method for trainers is being used in BC to address the high costs associated with travel to remote locations. This method is being tested and evaluated.

4.3.2 Program Theory and Design

The evaluation of the program's theory and design centered on an examination of external literature on Aboriginal community engagement and engaging communities on issues of safety, especially safety for women and girls, including articles relating to domestic violence, as well as external and internal PS key informant interviews.

Key informants at PS, including senior managers interviewed for the evaluation, consider the ACSDCP to be a tremendously innovative approach with the potential to be transformative in enabling Aboriginal communities to become safe, healthy and self-sufficient. The program, they say, has the potential to play a critical role, with community safety plans as the basis for all or most federal government policies and program responses to issues in Aboriginal communities. It can be transformative for communities because they emerge from the process with the will and capacity to lead change for themselves, and it can be transformative for governments because they learn to listen and respond to the needs identified by communities, and to adapt policies and programs to those needs, rather than having government policies and programs drive what takes place in communities.

Key informants also pointed to the opportunity to develop more integrated approaches across levels of government and among different agencies working in Aboriginal communities. Progress in this area has reportedly been slow, but integrated approaches offer the potential for greater effectiveness and more economical use of available resources.

Features of the program design identified by all key informants as positive and critical to success include:

Several external key informants and a number of academics in the literature reviewed for the evaluation, while not referring to the ACSDCP itself, pointed to the importance of enabling community participants to delve into fundamental issues of colonialism and government mistreatment and a rebuilding of individual and community strength through re-establishing identity and the value of local traditions and relationships within the communities. On that basis of cultural strength, they say, it is possible for participants to safely address issues of violence, sexual abuse and other highly sensitive issues, along with poverty, substance abuse and other issues that are at the root of violence against women in Aboriginal communities.

Both internal and external key informants said that for a process such as the ACSDCP to be successful, it is critical to have the active, formal support of community leaders including Chief and Counsel. This requirement applies both to the early stages of the process so that key issues in the community can be raised and discussed openly, and to later stages where community support is critical to developing and implementing a community safety plan. Program managers agree that this is important, although the internal program review pointed to instances where success of the ACSDCP was undermined by a lack of sufficient Chief and Council support.

The Program Review also found that:

5. CONCLUSIONS

5.1 Relevance

There is ample evidence in available documents and relevant literature, and from interviews with both internal government and external key informants, that the ACSDCP remains relevant. The need for improved safety in Aboriginal communities especially for women and girls remains, and the ACSDCP is well aligned with federal government and PS priorities in its approach to the issue of safety in Aboriginal communities. It appears to be working in concert or in parallel with a number of similar initiatives but there is no evidence of undue duplication of effort.

There is, however, an important opportunity for greater coordination among federal departments and agencies and among different levels of government in working to address community safety and the root causes of violence in Aboriginal communities. The ACSDCP offers a method for encouraging this kind of coordination.

Within PS, there is the potential for closer integration of the ACSDCP with the crime prevention and aboriginal policing units in encouraging and supporting community-led planning and priority setting in relation to community safety.

5.2 Performance-Effectiveness

It is evident that the program provided Aboriginal communities with tools and opportunities to identify their key safety issues and their collective resources that could be used to work towards community safety and healing, but it is unclear from the available information what the views of community participants are about the quality or effectiveness of those tools. There appears to be a long-term benefit to the tool development process in that once they are in place and used by communities, the information is readily available for other initiatives.

Despite some early challenges there appears to have been some success in training trainers for the community workshops, and success as well in conducting the workshops, with some evidence of improved community motivation to address issues related to community safety. The idea of having Aboriginal facilitators with local knowledge was viewed positively, but there was reportedly some variation in skills and experience of facilitators. Training of facilitators reportedly did not sufficiently meet the need, especially in high-risk communities. Program managers are aware of the need for improvement and have planned changes in recruitment and training strategies.

Program documents show that by September 2012 the program had delivered workshops in 25 communities. These communities were more aware of their issues and there was a rise in community motivation to deal with the issues. To date PS has received 9 safety plans. The modest numbers of workshops and community safety plans are plausibly the result of the nature of the program design, which acknowledges the long-term nature of the challenge especially in high-risk communities, and recognizes that progress is very likely to be slow and incremental. The large majority of participating communities progressed in ways in keeping with program objectives, with some having produced community safety plans and others still working on earlier stages of readiness.

The program has successfully initiated community engagement, but sustaining the engagement has proved challenging. Most observers believe that ongoing support is required for this sort of community development process to succeed, especially given the fact that Aboriginal community leadership is typically extremely busy with day-to-day operations.

Because of the community-driven nature of the initiative there appears to be considerable variation in the issues addressed in community workshops and in the community safety plans developed to date. Not all of the work has centred on community safety directly. However, it is evident that the community workshop and planning processes are engaging participants in discussions about the root causes of violence in their communities, which is viewed in the literature and among experienced Aboriginal community developers as an essential early step in improving the health of high-risk communities.

It is premature at this point to draw conclusions about the impacts of the ACSDCP on levels of safety in participating communities as compared to before the initiative or as compared to other similar communities. At this point there does not appear to be agreement about what constitutes a safe community and how that would be measured. It will be worthwhile going forward to consider establishing targets for progress that reflect realistic expectations.

5.3 Performance-Efficiency and Economy

The program administration ratio was 93%. This is mainly due to the fact that this was a new initiative for aboriginal communities which required gradual adaptation of approaches and tools as experience was gained in the field.

There appears to be considerable opportunity for greater efficiency on the part of the federal government as a whole through greater coordination and collaboration in working with Aboriginal communities. The ACSDCP offers a basis for such improved synergy through its support of community planning and its reported establishment of trusting working relations in communities. At its best, the ACSDCP offers a platform to transform the way departments and agencies operate in addressing complex, interdependent issues such as those experienced in many Aboriginal communities.

There is near-unanimity among internal and external observers and in the literature that the community-led approach adopted by the ACSDCP is the required path to eventual health and safety in Aboriginal communities, and that pursuing that path to good effect will require patience and continuous effort and support to communities over the long-term.

6. RECOMMENDATIONS

The Internal Audit and Evaluation Directorate recommends that the Assistant Deputy Minister, Community Safety and Countering Crime Branch of Public Safety Canada, implement the following:

In collaboration with the Assistant Deputy Minister Emergency Management and Programs Branch:

The ongoing collaboration would allow the department to leverage results across programs and maximize departmental investments.

7. MANAGEMENT RESPONSE AND ACTION PLAN

We have reviewed and agree with the recommendations. While respecting the communities' readiness and capacity to engage and sustain a community-driven development process, the Crime Prevention and Aboriginal Community Safety Division (CPACSD) will develop, prior to April1, 2015, supporting material that will help guide the communities through the process. This material, while not being prescriptive, will provide more direction for communities to navigate the community safety development process and successfully identify and determine responses to their community-identified needs or issues. In developing the material, based on lessons learned to date, the CPACSD will identify common milestones to community development and sustainability of communities engaged in the community safety development process. These milestones can then be translated into indicators that will allow the CPACSD to monitor more consistently progress at the community level. These indicators will be identified and their applicability to various community situations will be vetted by March 31, 2016. The results and adaptations will then form the basis of monitoring and assessing the level of community capacity and sustainability of community efforts and will be included in a revised Performance Measurement Strategy (PMS).

Also based on lessons learned, and the level of Public Safety (PS) staff support needed by communities, new targets have been set for the next five years. Assuming that only PS funds are available, the CPACSD has identified $385k/per year to support 2-3 call-ups against the standing offer per year, which would result in 6-10 communities supported to engage in the safety planning process each year. The contribution funds will now be dedicated to support implementation of innovative responses to violence against women. Assuming an average project length of 3 years, nine 3-year projects, and 3-5 shorter-term projects could be supported between 2015/2016 and 2019/2020. The PMS will be amended to reflect these targets.

Experience to date shows that the ACSDCP can be a great vehicle for multi-partner collaboration. The community safety planning process equips communities to identify their needs, build on existing strengths and develop responses to issues that resonates with the community and can be implemented at a pace that allows communities to take ownership of the issues and its solutions. The premise of ACSDCP is based on the fact that unless and until a community takes ownership, no solutions can be sustainable. In this context, partnerships created are done so in response to needs identified by the community, and are therefore more likely to be sustainable over the long term. PS offers three separate potential partners that can be beneficial to communities, and the CPACSD will create an internal working group comprised of both program/operations and policy/research representatives of Programs Directorate, Emergency Management and Programs Branch, along with the Research, Aboriginal Policing Policy and CPACS Divisions of Community Safety and Countering Crime Branch. This working group would help identify which communities are already engaged with PS, and it would allow for PS responses to community needs to be more targeted and timely. The working group will be created prior to April 1, 2015 and would be operational thereafter.

Recommendation Management Response Action Planned Planned Completion Date
1. Over the course of the first year of the renewed program, develop means to monitor and assess the level of community capacity and sustainability of PS program outcomes in the communities that have completed Community Safety Plans. Accept Determine, in implementing the renewed approach, milestones that would help better gauge community commitment to the process.
Consistently include milestones in delivering the process
March 31, 2016
2. Revise the program's Performance Measurement Strategy and establish targets for activities leading to the development of Community Safety Plans to better reflect the reality of achievable accomplishments. Accept The Performance Measurement Strategy will be revised to reflect the Evaluation findings and recommendations and to incorporate the feedback received from the Program Review Results received in July 2014. March 31, 2016

In collaboration with the Assistant Deputy Minister Emergency Management and Programs Branch:

3. Ensure that program managers, policy officers, and researchers of the departmental programs that serve Aboriginal communities within the Department (FNPP, NCPS, ACSDCP) collaborate on an ongoing basis to better integrate:

  • program delivery at the community level, and
  • research and policy activities at the division, directorate, branch and departmental level.

The ongoing collaboration would allow the department to leverage results across programs and maximize departmental investments.

Accept Invite representatives to participate in a working group with participants from all three programs (including policy, research and programs/operations) whose mandate would be to create synergies, when feasible, to integrate departmental activities and program delivery practices in response to community expressed needs through the community safety planning process.

March 31, 2016

ANNEX A: DOCUMENTS REVIEWED

AMR Planning and Consulting (2011). Collaboration to End Violence: National Aboriginal Women's Forum. Report on Outcomes and Recommendations from Working Sessions and Conference Report.

Assembly of First Nations (2012). Demanding Justice and Fulfilling Rights: A Strategy to End Violence against Indigenous Women and Girls.

Bauman, T. (2006). Final Report of the Indigenous Facilitation and Mediation Project July 2003-June 2006: research findings, recommendations and implementation.

Byrne, Tracy et al. (2011). Stopping Violence Against Aboriginal Women: A Summary of Root Causes, Vulnerabilities and Recommendations from Key Literature. Intergovernmental and Community Relations and Ministry of Aboriginal Relations and Reconciliation.

Chartrand, L., and C. McKay. (2006). A Review of Research on Criminal Victimization and First Nations, Metis and Inuit Peoples 1990 to 2001.

Day, A., Francisco, A. and R. Jones (2013). Programs to improve interpersonal safety in Indigenous communities: evidence and issues. Closing the Gap Clearinghouse. Issue paper no. 4.

Department of Justice Canada. Department of Public Safety and Emergency Preparedness Act (S.C. 2005 c.10)

Department of Justice Canada. Royal Canadian Mounted Police Act (R.S.C., 1985, c. R-10).

Department of Justice Canada (2010). Missing and Murdered Aboriginal Women Initiative.

Department of Justice Canada (2013). Compendium of Promising Practices to Reduce Violence and Increase of Aboriginal Women in Canada.

Department of Justice Canada. 2010-2011 Report on Plans and Priorities

Department of Justice Canada. 2011-2012 Report on Plans and Priorities

Department of Justice Canada. 2012-2013 Report on Plans and Priorities

Federal Provincial Territorial Ministers Responsible for Justice and Public Safety (2013). Draft Justice Framework to Address Violence Against Aboriginal Women and Girls.

Fletcher, F., McKennitt, D., and Baydala, L. (2008). Community Capacity Building: An Aboriginal Exploratory Case Study. Pimatisiwin: A Journal of Aboriginal and Indigenous Community Health. 5:9-32.

Government of Canada. Speech from the Throne 2010.

Government of Canada. Speech from the Throne 2013.

Harder, L. (2013). Evaluation of Aboriginal Collaborative Decision-Making Projects.

Mathyssen, I. (2011). Ending Violence Against Aboriginal Women and Girls: Empowerment – A New Beginning. Report to the Standing Committee on the Status of Women. 41st Parliament, 1st Session.

Moving Towards a Stronger Future: An Aboriginal Resource Guide for Community Development.

Native Women's Association of Canada. (2009). Voices of Our Sisters in Spirit: A Report to Families and Communities.

Native Women's Association of Canada (2010). What Their Stories Tell: Research Findings from the Sisters in Spirit.

Poonwassie, A., & Charter, A. (2001). An Aboriginal Worldview of Helping: Empowering Approaches.

Prime Minister of Canada (2013). Seizing Canada's Moment: Prosperity and Opportunity in an Uncertain World.

Public Safety Canada (2010). Performance Measurement Strategy: Aboriginal Community Safety Development Contribution Program.

Public Safety Canada (2012). Internal Audit of Policy, Priority-setting and Planning Public Safety.

Public Safety Canada. 2010-2011 Report on Plans and Priorities.

Public Safety Canada. 2011-2012 Report on Plans and Priorities.

Public Safety Canada. 2012-2013 Report on Plans and Priorities.

Public Safety Canada. 2013-2014 Report on Plans and Priorities.

Royal Canadian Mounted Police:
http://www.rcmp-grc.gc.ca/pubs/mmaw-faapd-eng.pdf

Royal Canadian Mounted Police. 2010-2011 Report on Plans and Priorities.

Royal Canadian Mounted Police. 2011-2012 Report on Plans and Priorities.

Royal Canadian Mounted Police. 2012-2013 Report on Plans and Priorities.

Royal Canadian Mounted Police. 2013-2014 Report on Plans and Priorities.

Statistics Canada (2004). General Social Survey: Victimization.

Statistics Canada (2011). Violent victimization of Aboriginal women in the Canadian provinces, 2009.

Statistics Canada (2012). Homicide in Canada, 2011.

Swanson, J (2013). A Model for Aboriginal Facilitation: an Open, Empowering Way to Get Consensus and Action.

Totten, M. (2009). Aboriginal Youth and Violent Gang Involvement in Canada. IPC Review, 3: 135-136

Wesley-Esquimaux, C., & Calliou, B. (2010). Best practices in Aboriginal Community Development: A Literature Review and Wise Practices Approach.

ANNEX B: FINANCIAL INFORMATION

The amounts below represent the total estimated cost to the federal government.8 The values are in dollars and have been rounded to the nearest hundred.

PROGRAM ADMINISTRATION COSTS 2010-11 2011-12 2012-13 2013-14
Public Safety Program Staff at HQ (up to Director level)
Salaries 153,100 205,400 207,100 368,000
Spent Operations and Maintenance (O&M) 59,800 62,200 111,000 78,800
Subtotal 212,900 267,600 318,100 446,800
Director General's Office        
Salaries 1,500 1,500 1,500 1,300
Operations and Maintenance (O&M) 0 0 0 0
Subtotal 1,500 1,500 1,500 1,300
TOTAL PROGRAM 214,400 269,100 319,600 448,100
Internal Services        
Salaries 61,900 82,800 83,400 147,700
Operations and Maintenance (O&M) 23,900 24,900 44,400 31,500
Subtotal 85,800 107,700 127,800 179,200
Employee Benefits Plan
(20% of Salary Expenditures)
43,300 57,900 58,400 103,400
PWGSC Accommodation Allowance
(13% of Salary Expenditures)
28,100 37,700 38,000 67,200
TOTAL PROGRAM ADMINISTRATION COST TO GOVERNMENT 371,600 472,400 543,800 797,900
TRANSFER PAYMENTS        
Budget 320,000 480,000 690,000 690,000
Transfers 105,500 91,000 - -
Contributions paid 425,500 571,000 666,600 684,500
Budget minus Contributions paid 0 0 23,400 5,500
PROGRAM ADMINISTRATION RATIO        
Annual 87% 83% 82% 117%
Four Year Average       93%

Notes:



  1. 1

    Statistics Canada (2011). Violent victimization of Aboriginal women in the Canadian provinces, 2009.

  2. 2

    Statistics Canada (2004). General Social Survey: Victimization.

  3. 3

    Royal Canadian Mounted Police: http://www.rcmp-grc.gc.ca/pubs/mmaw-faapd-eng.pdf.

  4. 4

    Public Safety Canada (2010). Performance Measurement Strategy: Aboriginal Community Safety Development Contribution Program; Byrne, Tracy et al. (2011). Stopping Violence Against Aboriginal Women: A Summary of Root Causes, Vulnerabilities and Recommendations from Key Literature. Intergovernmental and Community Relations and Ministry of Aboriginal Relations and Reconciliation; Statistics Canada (2012). Homicide in Canada, 2011.

  5. 5

    See for example: Swanson, J (n.d.). A model for aboriginal facilitation: an Open, Empowering Way to Get Consensus and Action; Poonwassie, A., & Charter, A. (2001). An Aboriginal Worldview of Helping: Empowering Approaches; Wesley-Esquimaux, C., & Calliou, B. (2010). Best practices in Aboriginal Community Development: A Literature Review and Wise Practices Approach.

  6. 6

    Now called Aboriginal Affairs and Northern Development Canada

  7. 7

    The program administration ratio refers to the total program administration cost as a percentage of the contribution paid in a given year.

  8. 8

    The program administration ratio refers to the total program administration cost as a percentage of the contribution paid in a given year.

Date modified: