ARCHIVE - 2008-2009 Targeted Evaluation of the Cross-Cultural Roundtable on Security

Adobe Acrobat version (PDF 384KB)

Table of Contents

• Executive Summary
• 1. Background and Context
  • 1.1 Description of the Cross-Cultural Roundtable on Security
  • 1.2 Evaluation Methodology
    • 1.2.1 Evaluation Issues and Questions
    • 1.2.2 Data Collection Methodologies
• 2. Key Findings
  • 2.1 Relevance
    • 2.1.1 Question 1: Continued Need, Scope and Objectives
    • 2.1.2 Question 2: Activity Alignment
  • 2.2 Design and Delivery
    • 2.2.3 Question 5: Outreach Events
    • 2.2.4 Question 6: Roles and Responsibilities
    • 2.2.5 Question 7: Existing Processes
    • 2.2.6 Question 8: Performance Measurement
  • 2.3 Impact
    • 2.3.1 Question 9: Increased GoC Understanding and Sensitivity
    • 2.3.2 Question 10: Increased Community Understanding
    • 2.3.3 Question 11: Community/GoC Contac
  • 2.4 Cost Effectiveness
    • 2.4.1 Question 12: CCRS Cost-Effectiveness
• 3. Overall Conclusion and Recommendations
• Appendix A: CCRS Logic Model
• Appendix B: Evaluation Matrix
• Appendix C: List of Documents Reviewed
• Appendix D: List of Interviewees
• Appendix E: Expenditures for 2006-07

List of Tables

Table 1.     Summary of CCRS Evaluation Questions
Table 2.     Number of Interviews Conducted, by Interview Group

[ * ] - In accordance with the Privacy and Access to Information Acts, some information may have been severed from the original reports.

Executive Summary

This report presents the evaluation findings of a targeted evaluation of the Cross-Cultural Roundtable on Security (CCRS), conducted between October 2007 and March 2008. The evaluation was based on a framework developed by the Citizen Engagement Directorate, Public Safety Canada (PS), in consultation with the Department’s Management Assurance Division. The evaluation addressed four key issues: program design and delivery, Impact, relevance, and program effectiveness and efficiency. A total of twelve evaluation questions were explored.

Methodologies used during the evaluation were document review and key informant interviews. Evaluations are generally designed to provide multiple lines of evidence in support of evaluation findings. However, given the scope, budget, and perceived low risk of the CCRS program, a targeted evaluation using only two lines of evidence was undertaken.

Relevance:

There continues to be a need for the CCRS. National security remains a key priority of the Government of Canada (GoC) and there are no other programs that provide a similar opportunity for dialogue between the GoC and Canada’s multicultural communities on issues of national security. Outreach activities sponsored by other federal Government departments and agencies do take place; however, for the most part these are complementary to, as opposed to duplicative of, CCRS activities. It was noted, though, that there is a lack of communication or coordination with regards to outreach activities undertaken by Government departments or agencies, by the CCRS Secretariat, or by its individual members. There is no comprehensive list maintained of outreach activities taking place. This could be useful for the purpose of helping to identify pertinent events for CCRS member participation; particularly given limits on the outreach that can be undertaken by the CCRS.

The current objectives and scope of the CCRS seem to be appropriate, although objectives have not been formally articulated and communicated to CCRS stakeholders. As well, according to interviewees, there is some confusion relating to roles and responsibilities of the various stakeholders. It is unclear from the information gathered whether confusion relating to roles and responsibilities stems from a need to more clearly define the roles and responsibilities or whether the roles and responsibilities need to be more effectively communicated.

CCRS activities are aligned with the CCRS objective to have ongoing dialogue between GoC and communities on national security issues.

Design and Delivery:

The CCRS Secretariat has undertaken and completed start-up activities including the development of a formal nomination process for CCRS membership, the creation of guidance documents, the convening of CCRS meetings and the creation of the Interdepartmental Committee (IDC). However, there is no formalized, documented process for setting CCRS annual objectives, and interviewees are unclear on the process being used. Similarly, there is no formal, documented process for setting agendas; although interviewees were generally in agreement on the process being used. As well, with respect to outreach activities, there is no formal process in place for selecting and prioritizing these events, although the CCRS Secretariat has recognized the need for a formalized process and is working to establish this.

CCRS members noted that there is no systematic feedback to CCRS members on actions taken on, or responses to, their recommendations; although this has happened on an ad hoc basis. As well, there are no defined performance measures in place for the CCRS.

Impact:

The CCRS has had an impact on the Government’s understanding and sensitivity to community needs. As well, some concrete examples were noted of CCRS impacts on GoC policies, directives and communications materials. However, there is limited evidence that the CCRS has contributed to better community understanding of national security policies, practices and challenges. This may be due to the fact that there are no formal mechanisms in place for disseminating information to communities, and there has been limited formal CCRS member-organized outreach conducted to date. As well, some interviewees noted a lack of clarity as to whether or not CCRS is mandated to do outreach.

Cost Effectiveness:

In the absence of cost information and similar comparison programs, Government Consulting Services (GCS) was not able to determine whether the operation of the CCRS was cost-effective. However, stakeholders feel that not enough resources have been allocated to the program and that there are possible improvements that could be made to improve effectiveness. In particular, stakeholders identified a need to stabilize the staff of the Secretariat, as well as a need for additional funding for outreach activities.

Recommendations:

Given these findings, GCS made a number of recommendations as follows:

1. That the CCRS Secretariat attempts to establish and maintain a calendar of outreach events relevant to the CCRS, and provides CCRS members with regular information on upcoming outreach events that may be of interest to them. In addition, in order to facilitate this calendar of events, CCRS members should provide information to the CCRS Secretariat about events going on in their communities, and any outreach activities they may participate in.
2. That the CCRS Secretariat clarifies the objectives of the initiative and clearly communicate these objectives, and roles and responsibilities, to CCRS stakeholders
3. That the CCRS Secretariat formalizes and clarifies the process for setting annual objectives.
4. That the CCRS Secretariat considers formalizing the process for setting meeting agendas.
5. That the CCRS Secretariat communicates these processes to CCRS members.
6. That the CCRS Secretariat establishes performance measurement indicators, including ‘actions in response to CCRS discussions or recommendations’, and begins collecting the data necessary to report on these indicators.
7. That the CCRS Secretariat clarifies the mandate of the CCRS to undertake outreach and dissemination of information to communities.
8. That the CCRS Secretariat establishes mechanisms for disseminating information to communities, and either directly disseminates information to communities or supports CCRS members or other stakeholders, as appropriate, in using these mechanisms.
9. That the CCRS Secretariat explores possible ways to stabilize the CCRS Secretariat staff and, if possible, dedicates staff solely to the support of the CCRS. In addition, the CCRS Secretariat should seek additional CCRS funding, to allow for outreach activities.

1. Background and Context

This report presents the evaluation findings of the Cross-Cultural Roundtable on Security (CCRS). The main objective of the evaluation was to assess the extent to which the CCRS is on track to meet its intermediate and long-term objectives, as per the existing CCRS Logic Model (see Appendix A) as well as the effectiveness and efficiency of program design, delivery and management. Given the limited budget and perceived low risk of the CCRS program, a targeted evaluation was undertaken.

1.1 Description of the Cross-Cultural Roundtable on Security

Over the past few decades, but particularly in recent years, a significant migration of diverse ethno-cultural groups has been observed around the world. Many countries, including Canada, have become mosaics of multicultural communities. This trend is expected to continue as Canada moves forward into the 21st century.

Today’s threat environment is increasingly international and trans-national in nature. The new and more complex security environment requires Canada to deal with the reality that in an open society, serious tensions and extremism can develop among communities, even in the most tolerant of settings. Consequently, Canadians and the Government of Canada (GoC) must stand together in the reaffirmation that the use of violence to pursue political, religious or ideological goals must and will be met with a determined response.

Within this context, to make its approach to security more effective and to fulfill its commitment to inclusion and mutual respect, the GoC created the CCRS as a key commitment of the National Security Policy (NSP).

The mandate of the CCRS is to engage Canadians and the GoC in a long-term dialogue on national security issues in a diverse and pluralistic society. In this respect, the CCRS provides a critical and innovative forum to discuss emerging trends and developments emanating from national security matters in order to better inform policy makers.

Through their meetings with Government officials and interactions with Canadian communities, Roundtable members:

• Provide insights to the GoC on how national security measures do or may potentially impact Canada’s diverse communities; and
• Facilitate a broad exchange of information between the Government and diverse communities on the impact of national security issues consistent with Canadian rights and responsibilities.

The Citizen Engagement Directorate at Public Safety Canada (PS) houses a Secretariat for the CCRS, which provides administrative (e.g. , meeting logistics) and strategic (e.g. , agenda setting) support for the Roundtable. An interdepartmental committee (IDC) has been created to allow relevant Departments and Agencies to provide input into the CCRS (e.g. , input into agendas, deliver presentations at CCRS meetings). IDC members are invited to attend all formal CCRS meetings and receive the reports of these meetings. There are currently eight Departments and Agencies that participate on the IDC.

The CCRS has a membership of 15 men and women from across Canada, appointed jointly by the Minister of Public Safety and the Minister of Justice and Attorney General of Canada, for terms of one year, renewable for a maximum of three years. The Minister of Public Safety and the Minister of Justice and Attorney General are also responsible for appointing a Chair and Vice-Chair of the CCRS. 1

Formal meetings of the CCRS provide an opportunity for both policy-makers, program managers, and senior executives to obtain feedback from CCRS members on the policies, programs, and other issues brought before CCRS members. The reports from the meetings are also disseminated to the relevant policy-makers and senior executives directly or indirectly impacted by the issues on the CCRS agenda. All meeting reports are also sent to the Ministers of Public Safety and of Justice from the Chair of the CCRS.

1.2 Evaluation Methodology

This evaluation was based on the Evaluation Framework developed by the Citizen Engagement Directorate, PS, in consultation with the Department’s Management Assurance Division. The evaluation was conducted between October 2007 and March 2008. At the outset of the evaluation, GCS reviewed the evaluation plan and, in consultation with the Citizen Engagement Directorate, made some modifications to reflect key questions and indicators to be addressed through the evaluation.

1.2.2 Evaluation Issues and Questions

The evaluation was designed to address four key issues: program design and delivery, impact, relevance, and program effectiveness and efficiency. A total of twelve evaluation questions were explored, as shown in Table 1. The full evaluation matrix, with indicators and lines of evidence, is presented in Appendix B.

Table 1. Summary of CCRS Evaluation Questions.

1.2.1 Data Collection Methodologies

The following methodologies were used as sources of information for the targeted evaluation:

• Document review, and
• Key informant interviews.

These methods are described in more detail in the following sections.

Document Review

A review of relevant documentation served as one of two primary lines of evidence for the evaluation. Background documents collected and reviewed include:

• Inception documents and planning documents for PS that detail the intended objectives and outcomes of the CCRS (e.g. , [   *   ], Terms of Reference, Program Guidelines, Logic Model);
• Budget guidelines; and
• CCRS expenditures, meeting and outreach reports, communication products, speeches, and work plans.

In addition to informing the development of instruments for the key informant interviews, the review of relevant documentation was used to assess the implementation of activities and the achievement of outputs, the extent to which systems and procedures are in place for the future achievement and reporting of outcomes, and early indications of program outcomes. A full list of documents reviewed is included in Appendix C.

Key Informant Interviews

Key informant interviews served as an important source of information for the evaluation, by providing information on stakeholder views in response to all of the evaluation questions. A total of 29 interviews were conducted with stakeholders. Table 2 outlines the categories of individuals who were interviewed as well as the number of interviews conducted. A list of interviewees is provided in Appendix D.

Table 2. Number of Interviews Conducted, by Interview Group.

* Note: CCRS Secretariat representatives included one former and one current CCRS Secretariat staff member.

With the exception of Public Safety Executives, the CCRS Secretariat, and a couple of other interviewees who preferred to be interviewed in-person, interviews were conducted via telephone. Interview guides were developed and provided to all interviewees in advance of the interviews. The information gathered through the interviews was summarized in a Microsoft Excel template and organized by interview question.

Limitations of the Evaluation Methodology

Evaluations are generally designed to provide multiple lines of evidence in support of evaluation findings. Given the scope, budget, and perceived low risk of the CCRS program, a targeted evaluation was undertaken, which included only two lines of evidence. However, it was expected that these two lines of evidence would be sufficient to draw conclusions, given the large number of relevant stakeholders who were interviewed, and the high proportion of all relevant stakeholders this group represents.

2. Key Findings

This section of the report presents a summary of the evaluation findings, which are organized into the issue areas of relevance, design and delivery, impact, and cost-effectiveness, and are presented by evaluation question, as found in the evaluation matrix (see Appendix B). Within the findings noted below, figures may be presented identifying the number of interviewees who gave a particular response to an interview question. For example “(20 of 29)” indicates that 20 individuals, out of a total of 29 interviewees, gave a particular response. The total number of interviewees (i.e. , ‘n’) may not be consistent from one question to the next because not all interview questions were asked of all interviewees and some interviewees may not have responded to every question. In addition, when referring to stakeholders throughout this report, stakeholders include members of the Roundtable as well as individual departments or agencies participating in and benefiting from the work of the Roundtable.

2.1 Relevance

2.1.1 Question 1: Continued Need, Scope and Objectives

Q1: Is there a continued need for the CCRS and are the current objectives and scope appropriate?

Conclusion: There is a continued need for the CCRS because national security remains a key priority of the GoC and there are no other programs that provide the opportunity for dialogue between the GoC and Canada’s multicultural communities on issues of national security, and particularly with a group where knowledge about national security issues is fostered, and an ongoing and sustained dialogue contributes to informed decisions. Based on comments by representatives of the IDC, Government departments seem to still need the advice that the CCRS provides. The current objectives and scope appear appropriate, although there may be opportunities to more clearly define the objectives.

GoC Priority for National Security

The creation of Public Safety Canada (PS) in 2003 demonstrated that national security was a growing priority for the GoC. In addition, in the Speech from the Throne to open the first session of the thirty-eighth Parliament in October 2004, the Government acknowledged national security as a priority with its introduction of Canada’s National Security Policy Statement. This policy was created to ensure that Canada is prepared for and can respond to current and future threats and focuses on addressing three core national security interests:

• Protecting Canada and Canadians at home and abroad;
• Ensuring Canada is not a base for threats to our allies; and
• Contributing to international security.

The creation of an advisory Cross-Cultural Roundtable on Security, composed of members of Canada's ethno-cultural and religious communities, was one measure identified as key to an integrated system designed to achieve the goals of the policy. 3

Most interviewees (24 of 27) also agreed that the CCRS serves a continued need. Only one person interviewed did not believe this is the case (two others were not sure). When expanding on their conviction, some (11 of 27) indicated that the CCRS provides advice or input that is still needed by federal departments and agencies. This is particularly noteworthy when examining the responses across the various groups of stakeholders included in the interviews. Two-thirds of those interviewed from the interdepartmental committee (6 of 9) raised this point. This would suggest that federal departments continue to need the advice, provided by the CCRS, into national security issues or policies. Similarly, almost half of the IDC interviewees (4 of 9), and one-third of all interviewees, indicated that the CCRS addresses a need for continued dialogue between the Government and public. About one-quarter of interviewees (7 of 27) also noted terrorism or other threats to society that continue to exist. The majority of respondents (18 of 26) indicated that the reasons for the existence of the CCRS have not changed since its inception. This was particularly the case for CCRS members (12 of 14 indicated the reasons have not changed) and IDC representatives (5 of 8 indicated the reasons have not changed).

Duplicative and Complimentary Programs

Members of the IDC and PS were asked, in interviews, if they were aware of any other citizen engagement initiatives or programs that may overlap with, or complement, the CCRS. According to interviewees there are no other initiatives that could be viewed as meeting the same needs or objectives of the CCRS. All interviewees noted that there are other citizen engagement initiatives or programs that exist at departments such as Canadian Heritage, Foreign Affairs and International Trade Canada, Transport Canada, the Canada Border Services Agency, Justice Canada and Citizenship and Immigration Canada. However, most of these interviewees (13 of 15) viewed these programs as complementary to the CCRS. Many (9 of 15) noted that these programs are more focused on outreach to specific communities or groups, or on specific issues, as opposed to more general and ongoing dialogue with various groups. Those interviewees who did observe some overlap noted that there may, on occasion, be overlap in terms of topics of discussion, and in some cases members of the CCRS may also participate in these other outreach activities. However, a couple of interviewees also indicated that they would like to see more coordination between the CCRS and other relevant outreach events held by other departments and agencies.

Interviewees identified the Advisory Council on National Security as another source of advice and input to Government on topics relating to national security; however, the Advisory Council was viewed as complementary to the CCRS, as opposed to duplicative. The main difference between the CCRS and the Advisory Council is in the makeup of its membership. The CCRS consists of individuals from various religious and ethno-cultural communities, whereas the Advisory Council is made up of members with expertise in security. “Unlike the Advisory Council on National Security, whose mandate is to provide expert advice on national security, the CCRS is meant to provide community advice related to how security measures might impact those communities.”4

CCRS Mandate, Objectives and Scope

The CCRS Terms of Reference indicate that “the mandate of the Roundtable is to engage Canadians and the GoC in an ongoing dialogue on national security in a diverse and pluralistic society.”5 The Terms of Reference also indicate that the scope of the CCRS includes:

• Providing insights on how national security measures may impact Canada's diverse communities;
• Promoting the protection of civil order, mutual respect and common understanding; and
• Facilitating a broad exchange of information between the Government and communities on the impact of national security issues consistent with Canadian rights and responsibilities.

Many interviewees (17 of 27) generally agreed that the objectives, scope and national security focus of the CCRS are appropriate. This was particularly the case for IDC interviewees (7 of 9) and CCRS members (10 of 14). Note that approximately one quarter of all interviewees (7 of 27) suggested that there is a need to clarify the mandate or objectives of the CCRS. These individuals represented all interviewee groups: CCRS members, IDC members, and PS staff and executives. It is worth noting that while the CCRS has a well-defined mandate and scope; it does not have formally articulated objectives and interviewees seem to be using these terms interchangeably. For example, interviewees suggested that the objectives of the CCRS are: to provide community insight into the GoC (21 of 29); provide GoC insight into communities (13 of 29); and to create dialogue between the GoC and communities (13 of 29). These responses are in-line with the defined scope and mandate of the CCRS, as listed above. (It should be noted, however, that interviewees may have been confused by the term “objectives” used in the interviews. PS staff has indicated that, in discussions with CCRS members, it has become apparent to them that members are unclear on what is meant by “objectives”, whereas the term “priorities” seems more clear to them. )

In addition, only about half of all interviewees (14 of 29) believe that CCRS members and other stakeholders have a good understanding of CCRS overall objectives and their role in meeting those overall objectives. Almost as many interviewees (11 of 29) stated that they do not believe CCRS members and other stakeholders have a good understanding of the objectives and their role in meeting them. CCRS members were more likely than the other groups of interviewees to indicate that they and other stakeholders have a good understanding of the objectives of the CCRS and their role in meeting them (8 of 14 CCRS members, versus 4 of 9 IDC members and 2 of 6 PS interviewees).

2.1.2 Question 2: Activity Alignment

Q2: Are CCRS activities aligned to overall objectives?

Conclusion: CCRS activities are aligned with the CCRS objective to have ongoing dialogue between GoC and communities on national security issues.

CCRS Meetings

As stated previously, the objective (i.e. , mandate) of the CCRS is to engage Canadians and the GoC in an ongoing dialogue on national security in a diverse and pluralistic society. The CCRS meetings have presented opportunities for dialog between Government and CCRS members, particularly relating to the role of Government and Government policies. Almost three quarters of interviewees (20 of 27) believe that CCRS meetings align to the objectives of the CCRS. This was a consistent belief across all three interviewee groups.

A review of program documentation revealed that issues related to national security were addressed in the formal CCRS meetings. The following priorities, as well as others, were reflected in CCRS discussions: the federal Government’s Anti-Terrorism Act; programs and measures in place to protect Canadians from threats to marine security; accountability and oversight of Agencies with national security responsibilities; specific approaches Government might pursue to enhance interactions with members of diverse communities; and border and transportation security issues via Canada's Smart Borders initiative. This was supported by interviewees, with all interviewees noting, when asked about the alignment of CCRS meeting discussions with national security priorities, that CCRS meetings sufficiently address Government of Canada national security priorities. (Note however that only 6 of 12 interviewees commented on this issue. )

CCRS Outreach Activities

An assessment of the documentation available about community events in which CCRS members have participated (both CCRS outreach events with the public and other events attended by CCRS members such as conferences), shows that for the most part these events have been aligned with CCRS objectives. CCRS planned events (e.g. , Regional symposia) aligned well to the objectives of CCRS since they were sponsored, planned, and implemented by PS through the CCRS Secretariat. Therefore, these events focused on national security-related topics, and have provided an opportunity for dialogue between Government of Canada representatives and various communities.

There was not a lot of information available with regards to other events attended by CCRS members; however, where these events were reported or shared with the group, they seemed to align well to the objectives of the CCRS. For example, CCRS participation in the Second Public-Private Sector Summit on National Security (Ottawa, May 9-10, 2006) aligns with the objectives of the CCRS. The Chair of the CCRS acted as a panellist for the Plenary Session "New Ideas and Innovations in National Security” at this event. As well, two CCRS member participated in the Ethnocultural Roundtable in August 2005, which again aligned to the objectives of the CCRS as the goal of that roundtable was to discuss the topics related to: combating racism, hate speech and hate crimes; protection of the most vulnerable amongst us; and security and human rights.

Information from interviews was slightly inconsistent with document findings. Just over half of all interviewees (15 of 27) indicated that outreach activities align with CCRS objectives, while just under one-third did not agree (8 of 27). This disagreement appears to stem from the fact that some interviewees (6 of 27) believe that some events have been poorly planned. Specifically, several CCRS members indicated that outreach events were not organized, articulated or focused enough; or that if CCRS members had been allowed to play a greater role in planning and participating at the events they would have better met the objectives. A couple of IDC members, on the other hand, indicated that some events may not have unfolded as originally planned, perhaps due to the selection of contentious speakers or the inclusion of particularly vocal interest groups.

IDC members were more inclined to feel that outreach activities did align with CCRS objectives (6 of 9 agreed, with the remaining three interviewees either feeling they somewhat aligned or not commenting). Senior executives at PS were least likely to feel that outreach events aligned with CCRS objectives, with 3 of 4 interviewees noting that there could have been better alignment in the case of some past outreach events.

2.2 Design and Delivery

Q3: Has the CCRS Secretariat successfully implemented planned start-up activities?

Conclusion: A lack of documentation regarding planned start-up activities for the CCRS makes it difficult to draw a conclusion on their successful implementation. However, information collected throughout the evaluation shows that certain start-up activities for the CCRS were completed, including a formal nomination process for membership, the creation of guidance documents, the convening of members for CCRS formal meetings and the creation of an Interdepartmental Committee.

CCRS Nomination Process

Program documentation shows that a process was established and followed for the nomination and selection of the current 15 CCRS members. A call for nominations was made by the Minister of Public Safety and the Minister of Justice and Attorney General in the summer of 2004. The eligibility criterion for membership was formulated in order to ensure a diverse representation of Canadians from various regions and backgrounds. To be eligible for nomination, candidates had to demonstrate linkages to community through membership, advocacy, and volunteer work; and understanding of security-related matters. An inventory of potential nominees was established following a screening process using the criteria listed above. For the appointment of the initial members of the CCRS, in March 2005, the Minister of Public Safety Canada and the Minister of Justice and Attorney General of Canada were jointly responsible for appointing new members and for selecting members for re-appointment to a second term. The Chair and Vice-Chair were selected from among the fifteen members.

While rotation of initial members had not occurred prior to submission of this report, short and long-term strategies for membership renewal were approved by the Minister in January 2008 in the Interim Management Framework for The Cross-Cultural Roundtable on Security. 6  New appointments would be for terms of two years with the possibility of an extension of one year. As well, with staggered membership renewal, the new rotation schedule means the CCRS will eventually see a turnover of five members each year without the need to replace all members in one exercise. This approach was proposed in order to ensure a degree of continuity of membership over the years, and to avoid the loss of all context and knowledge that would take place if all members’ terms ended simultaneously. This new framework will address the issue of the need for a formal structure for membership renewal, as identified by interviewees.

Guidance Documents

Various guidance documents were created to direct the CCRS, for both initial and ongoing operations. These documents include: 

• Cross-Cultural Roundtable on Security Terms of Reference (March 7, 2005): provides information on the mandate and scope of the CCRS, and outlines the functions of the Chair, Vice-Chair and Roundtable Secretariat.
• Developing an Outreach Strategy for the Cross-Cultural Roundtable on Security (April 2005): outlines various outreach strategies for the CCRS.
• Principles and Procedural Rules Guiding the Dialogue of the Cross-Cultural Roundtable on Security (Roundtable) (March 7, 2005): guiding document for CCRS dialogue, as well as describes the role of the Chair, Members and CCRS Secretariat. This document also defines how communications of the CCRS will be conducted.

CCRS Meetings

The initial meeting of the CCRS convened on March 7-8, 2005. The meeting brought together 14 of the 15 members appointed to the Roundtable; the Deputy Prime Minister and Minister of Public Safety and Emergency Preparedness; the Attorney General of Canada and Minister of Justice; the Minister of State for Multiculturalism; the Deputy Ministers of Public Safety and Emergency Preparedness Canada, Department of Justice, and Canadian Heritage; the National Security Advisor to the Prime Minister and Assistant Secretary of the Privy Council Office; as well as the Agency Heads for the Canadian Security Intelligence Service (CSIS) and the Royal Canadian Mounted Police (RCMP). The Executive Vice-President of the Canada Border Services Agency (CBSA) also participated, as did a number of senior Government officials from departments and agencies responsible for national security policy and programs. A senior representative from Canadian Heritage was also present. The fifteenth member of the Roundtable was out of the country and unable to attend the meeting. This first meeting laid the groundwork for how the Roundtable would function, outlined issues that were top of mind for Government officials and the members, clarified what needed to be addressed in the Forward Agenda and outlined the interactions the members wished to pursue with Government and communities of interest. Between the first meeting in 2005 and March 31, 2008, there have been nine full meetings of the CCRS and two regional meetings.

Interdepartmental Committee for CCRS

The Interdepartmental Committee was created and held its first meeting in February, 2005. The committee currently includes representation from: Department of Foreign Affairs and International Trade (DFAIT), the Royal Canadian Mounted Police, the Canadian Security Intelligence Service, Public Safety Canada, the Canada Border Services Agency, the Department of Justice (DoJ), Canadian Heritage (PCH), Transport Canada (TC), and the Privy Council Office (PCO).

Most interviewees (13 of 15) indicated that the Interdepartmental Committee includes all relevant departments. The two interviewees who indicated that it did not include all relevant departments were from the IDC. (Note: only IDC members and PS employees were asked this question. ) Suggestions for potential additions to the Interdepartmental Committee include the Department of National Defence (DND) and the SIRC (Security Intelligence Review Committee).

Based on the number of meeting reports, there have been eleven meetings of the IDC since its creation in 2005, and March 31, 2008.

2.2.2 Question 4:  Priority Setting

Q4: How are priorities (i.e. annual objectives, agenda items) for the CCRS set and is this appropriate?

Conclusion: There is no formalized, documented process for setting CCRS annual objectives. As well, based on responses by interviewees, it seems that whatever process is used, is not clear to all stakeholders. Similarly, there is no formal, documented process for setting agendas; although interviewees were generally in agreement on the process being used. While interviewees believe that there is opportunity to provide input into the agenda-setting process, some felt more lead time is required for this to happen.

Appropriateness of Objective Setting

A review of documentation showed that there are no formally documented processes for setting priorities for the CCRS. This was confirmed by interviewees, who indicated that there is no clear process for objective setting. While some (8 of 29) indicated that annual objectives are set at the formal meetings, an equal number indicated that objectives are set on an ad-hoc basis. A few (6 of 29) believe that the objectives are set by the Secretariat which could include consultation with the Chair (although none of these respondents were PS representatives), and a few (4 of 29) suggested that there are no annual objectives set for the CCRS. This indicates a need for clarity regarding the process for setting annual objectives.

Regardless of how objectives are set, just over half of the interviewees indicated that CCRS members are given sufficient opportunity for input. CCRS members, in particular, felt that the CCRS is given sufficient opportunity for input, with over three quarters of them agreeing (11 of 14). In terms of seeking additional input into CCRS priorities, many (5 of 13) indicated that input should be sought from other stakeholders, while some (4 of 13) did not think other input was needed. (Note:  CCRS representatives were not asked this question. ) All of PS staff who were asked this question agreed that additional input should be sought from other sources. On the other hand, only one IDC member indicated additional input should be sought. Almost half of the IDC interviewees (4 of 9) indicated additional input was not required, and an equal number were unsure or did not comment. Suggestions for who could provide additional input include the senior management teams from Public Safety Canada and the Department of Justice, local police forces, CSIS, PCO, RCMP, and the broad intelligence and security community.

Suggested improvements relating to the objective setting process include: that the mandate of the CCRS be clarified (as stated by 10 of 29 interviewees); and that a formal process for objective setting is needed (as stated by 5 of 29 interviewees).

There were mixed opinions by CCRS members on whether CCRS annual objectives sufficiently addressed community issues, in line with national security objectives. (Note: only CCRS members were asked this question. ) Some CCRS members (6 of 14) indicated that CCRS annual objectives do sufficiently address these issues, while some others (4 of 14) felt that the objectives did not. A few (3 of 14) noted that the lack of clarity of objectives prevented them from being able to answer the question. (Note: as discussed in question 1 above, there may have been some confusion amongst interviewees as a result of the use of the term “objectives” as opposed to “priorities”. )

Appropriateness of Agenda Setting

While there is no formally documented process for agenda setting, this process seems clearer than that of objective setting. Interviewees are generally in agreement about the process for agenda-setting for the CCRS meetings. Most interviewees (26 of 27) stated that agendas for the CCRS are set by the Secretariat with many indicating that the Secretariat consults with either the Chair (12 of 27) or CCRS members (6 of 27) in this process. In fact, when looking at only the responses from CCRS members and PS staff, percentages are even higher of those who agreed that the CCRS Chair provides input into the agenda (10 of 14 CCRS members, and 2 of 4 PS interviewees). Almost all IDC members (7 of 9) indicated that the IDC provides input into the process, which was also stated by three quarters of PS representatives (3 of 4) who were asked this question.

PS and IDC interviewees were asked whether input should be sought from any other relevant stakeholders. Only two interviewees who responded to this question indicated that other input is required from stakeholders. Both of these interviewees were Secretariat or PS representatives who stated that additional input should be sought from CCRS members. One of these also noted that the IDC should have additional input.

While some of the interviewees (9 of 27) indicated that the current process for agenda-setting is fine, some others (11 of 27) suggested better planning around the agenda setting process. IDC members were most likely to feel the process relating to agenda setting is fine (5 of 9). Of those who felt there was a need for better planning, five CCRS members indicated that CCRS members should have more lead time to provide input into the agenda, and three (two IDC members and one representative from PS) indicated that the IDC should be provided lead time to provide input into the agenda.

There were mixed opinions expressed by CCRS members on whether CCRS meetings (i.e. , via the agenda) addressed community issues in line with national security objectives. Some CCRS members (4 of 14) indicated they believe that CCRS meeting agendas sufficiently address community issues; some (5 of 14) did not believe so, and the remainder felt the agendas addressed community issues to some extent but could do more. There was also some disagreement regarding whether the agendas should include community versus national security issues. Some (4 of 14) commented that the focus of the agendas could be more on regional / local community issues and a few (2 of 14) commented that the focus needs to be more on national security issues, rather than those that are specific to one community. This may be the result of a lack of clear understanding of the mandate of the CCRS, as discussed previously in question 1.

2.2.3 Question 5: Outreach Events

Q5: How are outreach events for the CCRS set and is this appropriate?

Conclusion: Although documentation suggests there is a process in place for identifying outreach events it is uncertain to what extent this process is applied. There is no formal process in place for selecting and prioritizing these events. A more formal process around outreach is required, as recognized by the recent submission to the Minister of PS of the Interim Management Framework for the Cross-Cultural Roundtable on Security which included the establishment of a more formalized process for outreach.

Identification, Selection and Prioritization of Outreach Events

In a question and answer document dated June 8, 2006, reference is made to an “annual outreach program.”  In addition, according to this document, community dialog for 2006/2007 would focus on three themes. This suggests that there was some outreach planning done, on an annual basis. It also seems that potential outreach events are discussed at meetings of the CCRS. Finally, included in the document review for this evaluation was a form that had been completed by CCRS members to propose an outreach event. This form contained information about:

• the location or venue of the event;
• the type of event;
• the date and time of the event;
• who was expected to participate (speakers, moderator, target audience, Government representatives);
• the objectives of the event;
• key topics to be discussed;
• any support required (e.g. , communications or financial);
• how the event would be evaluated, and
• how reporting would take place after the event, to PS and to participants.

Based on these items, it seems that there is some planning for CCRS outreach activities. However, the extent to which the process for identifying events is formalized is not evident. As well, there was no evidence of a formal process for prioritizing or selecting outreach events that have been proposed. CCRS members and PS interviewees were asked about the process for identifying, prioritizing and selecting outreach events. Almost half of all interviewees (8 of 18) indicated that there is no formal process for selecting outreach events. PS staff in particular noted the lack of a formal process, with 3 of 4 interviewees commenting on this absence. All of the PS interviewees and one half of the CCRS members (7 of 14) indicated that CCRS members initiate outreach events (11 of 18). As well, a few interviewees (4 of 18) noted that the Secretariat works with the members to hold the events.

Information provided by the CCRS Secretariat indicates that it is aware of a need for a more formalized process relating to outreach events. In the Interim Management Framework for the Cross-Cultural Roundtable on Security recently prepared and presented to the Minister of PS, an outreach and engagement strategy for the CCRS was proposed. The aim of the strategy is to provide CCRS members with a greater role in recommending outreach activities, and to address issues of accountability and reporting on outreach. One of the components of the new outreach strategy is the potential creation of a working group on outreach and engagement, which, it is proposed, would be chaired by the Vice-Chair of the CCRS and include three to four other CCRS members and a member of the CCRS Secretariat. Formal outreach proposals would be submitted for review to this working group, in order for the working group to recommend outreach projects. In addition, one of the responsibilities of the Vice-Chair would be to provide a report on all outreach activities undertaken on an annual basis.

2.2.4 Question 6:  Roles and Responsibilities

Q6: Are roles and responsibilities of those involved in the CCRS clearly defined and understood?

Conclusion: Roles and responsibilities are defined in the Terms of Reference of the CCRS; however, not all stakeholders are clear on the roles and responsibilities of the various parties involved.

Definition of Roles and Responsibilities

The Terms of Reference document of the CCRS outlines the mandate and scope of the CCRS, membership eligibility criteria, includes a conflict of interest clause, and articulates the functions of the Roundtable (including the Chair and Vice-Chair) and of the Secretariat. 7 Principles and procedural rules have also been developed for guiding the dialogue of the CCRS. This document outlines the role of the Chair, the role of members of the CCRS, the role of the Secretariat, and other roles relating to communications for the CCRS. The roles and responsibilities of the members of the CCRS and of the Secretariat are also articulated in Program founding documents.

Clarity of Roles and Responsibilities

Despite the existence of these documents, which describe roles and responsibilities, interviewees gave a mixed response when asked whether roles and responsibilities were understood by all parties. Almost one quarter of all interviewees (7 of 29) indicated that roles and responsibilities are not understood by all parties. This was particularly true according to IDC members with over half of the IDC members interviewed (5 of 9) indicating that roles and responsibilities of stakeholders are not well understood. Just under half of the interviewees (13 of 29) indicated that roles and responsibilities of one party may be understood, but not the roles and responsibilities of all parties. CCRS members, particularly, gave mixed responses regarding the extent to which the roles and responsibilities of the various stakeholder groups are understood. Almost two thirds of CCRS members (9 of 14) suggested that stakeholders may understand their roles, or the roles of one of the other stakeholder groups, but not understand the roles of all the stakeholders. As well, over one third of CCRS members (5 of 14) admitted that they are not aware of the role of the IDC. Approximately one-quarter of interviewees stated that roles and responsibilities are generally understood.

Almost one-third of interviewees (9 of 29), including half of the CCRS members (7 of 14) noted the high turnover in the Secretariat as a contributing factor in the lack of clarity in roles and responsibilities. A few others (5 of 29) suggested that the Secretariat needs to better understand its own role. Again, this comment came most prominently from CCRS members (4 of the 5). Finally, a few (6 of 29) also commented that roles and responsibilities are not clearly defined and articulated. (This comment came from all three groups of interviewees. )

It is unclear from the information gathered whether this issue stems from the need to more clearly define the roles and responsibilities in the guidance documents or whether the roles and responsibilities need to be more effectively communicated.

2.2.5 Question 7: Existing Processes

Q7: Are ongoing processes in place to allow the CCRS to function effectively?

Conclusion: Administrative processes are in place to support the administrative functioning of the CCRS. The Secretariat is generally considered effective, although interviewees noted the high turnover of staff at the Secretariat, which may be impacting on its ability to provide effective support to the CCRS. Summary reports of CCRS and IDC meetings are reportedly useful, although the timeliness of the distribution of meeting summaries is less obvious, at least when it comes to CCRS meeting summaries. There is no systematic feedback to CCRS members on actions taken on, or responses to, their recommendations; although this has happened in the past on an informal and ad hoc basis.

Effectiveness of the CCRS Secretariat

Based on the CCRS founding documents, the CCRS Secretariat was established to provide support to the Roundtable. In general, according to the documentation, the Secretariat:

• Manages the nomination and selection process of CCRS membership;
• Undertakes strategic planning related to Roundtable meetings and activities;
• Monitors national security issues and other relevant emerging issues;
• Manages the IDC;
• Prepares and coordinates the distribution of all required materials and correspondence to CCRS members and Government participants prior to and following Roundtable meetings;
• Manages all administrative details of the Roundtable meetings (e.g. , contracts, travel/accommodation);
• Drafts summary reports of each Roundtable meeting in consultation with the Roundtable's Chair;
• Engages federal Government organizations responsible for national security in the delivery of responses and concrete actions resulting from issues raised within the Roundtable exchanges; and
• Prepares the Deputy Prime Minister and Minister of PS's progress reports to the Cabinet Committee on Security, Public Health and Emergencies. 8

Many interviewees (15 of 27) indicated that the Secretariat has generally been effective in coordinating and implementing the administrative details of the CCRS. Some others (10 of 27) agreed that the Secretariat has been effective in some areas (e.g. , logistics), but less so in others (e.g. , agenda setting). Only two interviewees stated that the Secretariat has not been effective. A few interviewees (5 of 27) also noted that the support of the Secretariat has diminished over time, with this comment coming most often from CCRS members (4 of the 5). These comments may be attributable to staffing issues within the Secretariat, as approximately three-quarters of all interviewees (20 of 27) commented on the high turnover of staff within the Secretariat, and indicated that this has had an impact on the CCRS.

Interdepartmental Committee Meetings

With regards to the IDC, program documentation provided for the evaluation indicates that two committee meetings were held during fiscal year 2004-05, four in fiscal year 2005-06, three in fiscal year 2006-07, and two have taken place in fiscal year 2007-8. IDC members and two Secretariat staff members were asked, in interviews, whether or not the IDC met regularly and varied responses were received. Approximately one quarter of those who were asked this question (4 of 11 interviewees) indicated that the IDC met regularly (at least once per month, or every six weeks). Other responses varied greatly (e.g. , every 2 months, 2-3 times per year, 3-4 times per year).

Based on the distribution lists for the summaries of two IDC meetings, which were provided to the GCS consultants, IDC meeting summaries are distributed only to the members of the IDC and PS representatives. With regards to the timeliness of IDC meeting summaries, just under half of the interviewees who were asked about this (7 of 15), indicated that there were no summaries of IDC meetings. Surprisingly, two thirds of IDC members who were interviewed (6 of 9) indicated they did not receive summaries of IDC meetings. The fact that IDC members interviewed for this evaluation responded that they do not receive IDC meeting summaries may be due to the fact that they are not necessarily the ones who attend the IDC meetings. If an IDC member sends a member of his staff to the IDC meeting, in his place, that staff member receives the IDC meeting summary.

Of those interviewees who did receive meeting summaries, almost all (7 of 8) indicated they were timely (the one other interviewee could not say) and almost all (6 of 7) suggested these summaries were valuable as a source of information or for tracking action items (one interviewee was not asked this question). Evidence, provided to the evaluators, of the distribution of the IDC summary reports confirms the timeliness of the IDC meeting summary reports. Email distribution notes for the summaries of two meetings verified that these reports were provided within one to two weeks following meetings. However, email evidence was not provided for all IDC meetings so it is not possible to say if this has always been the case.

CCRS Meetings

Based on the information provided by the CCRS Secretariat, the inaugural meeting of the CCRS took place in fiscal year 2004-05, three CCRS meetings were held during fiscal year 2005-06, two in fiscal year 2006-07 and three in fiscal year 2007-8. 9  Interviewees gave differing accounts of how many meetings of the CCRS are held each year, with the majority reporting between two and four meetings per year (15 of 29 interviewees). All CCRS members who commented on the number of times the CCRS meets (11 of the 14 CCRS members provided a number) agreed that there are two to four meetings per year. Many interviewees (19 of 29) felt the number of CCRS meeting held was appropriate. A few (3 of 29) felt there should be more meetings, with the remaining answering that the number of meetings depends on other factors such as the agendas, or issues that need to be discussed. When looking only at CCRS member responses, most (11 of 14) indicated the number of meetings was sufficient, a couple (2 of 14) indicated it was insufficient, and one did not comment. None of the PS staff felt there were insufficient meetings.

According to CCRS founding documentation, CCRS summary reports are shared with the Roundtable members; the IDC members; the Ministers and Deputy Ministers of PS, DOJ and PCH (Multiculturalism); as well as with Heads of federal agencies involved in the ongoing development, monitoring and evaluation of the National Security Policy. However, documentation was not available for this evaluation to confirm this distribution.

With regards to CCRS meeting summaries, some interviewees (13 of 29) reported that they received the summaries in a timely manner, while others (9 of 29) reported they do not. Responses were fairly consistent across the stakeholder groups interviewed, with the exception of IDC members, with over half of this group (5 of 9) indicating they do not receive summaries of CCRS meetings. However, as mentioned previously, IDC members interviewed for this evaluation may not personally attend IDC meetings, but may instead send a staff representative. Of those who reported that they did receive CCRS meeting summaries, almost all (22 of 23) noted they were valuable or somewhat valuable.

GoC Feedback on CCRS Discussions or Recommendations

Finally, on the topic of feedback provided to CCRS members on their discussions or recommendations, most interviewees (21 of 26) pointed out that summaries of actions, or responses to CCRS recommendations, are not provided by Government of Canada departments or agencies. In fact, IDC members were the most likely to respond that Government departments do not provide summaries of their actions to the CCRS (8 of 9 IDC interviewees). Some interviewees (8 of 29, representing all three groups of interviewees) commented that these would be useful, as they would indicate to CCRS members the impact that CCRS recommendations and dialogues have on Government. However, the extent to which Government departments and agencies are able to report on their actions, following CCRS discussions or recommendations, will vary depending on the topic of discussion or the recommendation. Policy-making, with the federal context, can take years. As well, it is usually difficult to attribute changes made to policies to any one single influence. Therefore, it may be difficult for departments and agencies to report back, in a timely manner, to the CCRS providing specific, concrete examples of responses to what they heard during CCRS discussions or to recommendations the CCRS may have made. A few interviewees (6 of 29, all of which were CCRS members) noted that on occasion some Government agencies have provided responses to CCRS recommendations; with the RCMP and CSIS, in particular, having been noted as the sources of feedback / responses. However, these responses were not typical, and no formal process exists for tracking or reporting on CCRS discussions or recommendations.

2.2.6 Question 8:  Performance Measurement

Q8: Are performance measures in place to assess the ongoing and future success of the CCRS?

Conclusion: There are limited defined measures of success at this time, although interviewees had a few suggestions on potential performance indicators.

Performance Measurement

To date, only limited performance measures have been identified for the CCRS initiative. According to PS staff, there is very little or no performance data at all being collected for the CCRS program. PS staff have indicated, however, that a Results Based Management Accountability Framework (RMAF) will be developed in the coming months which will provide guidance in performance measures that could and should be collected to measure the performance of the CCRS.

When asked about performance measures that could be tracked, just over half of all interviewees (15 of 27) made suggestions (although not all were specific measures, but rather methods of measurement). Performance measures suggested included:

• Number of policies impacted (changed) by CCRS discussions or recommendations;
• Extent to which CCRS is alleviating the fears and negative attitudes, within communities, of Government policies;
• Community groups’ understanding of national security policy, levels of comfort at airports, perceptions of racial profiling (through polling of groups over time);
• Number of CCRS meetings held;
• Turnout at meetings;
• Number of newsletters distributed;
• Number of outreach events held / number of attendees at events;
• Participation / representation of departments or agencies at CCRS meetings;
• Number of topics discussed at CCRS meetings;
• Stakeholder views on the achievement of objectives; and
• Departments / agencies views on value of input received.

Just under half of the interviewees (12 of 17) did not have any suggestions for performance measures.

2.3 Impact

2.3.1 Question 9: Increased GoC Understanding and Sensitivity

Q9: To what extent has the CCRS allowed the Government of Canada to better understand, and be sensitive to, community needs?

Conclusion: The CCRS has had an impact on the Government’s understanding and sensitivity to community needs. As well, although it is always difficult to show attribution for activities affecting policy development (as a result of the many factors that influence policy, and the timeframes for policy development), some concrete examples were noted of CCRS impacts on policies, directives and communications materials.

Mechanisms for Relaying Issues to the GoC

The key mechanism in place to allow CCRS members to convey community national security related issues to the Government of Canada are the formal meetings, as cited by many of the interviewees (20 of 27). A few interviewees also suggested other means of relaying community concerns to the GoC such as:

• through direct access to Government officials (reported by 3 of 27 interviewees, but two were PS interviewees),
• at outreach events (2 of 27, but both were CCRS members),
• through the Secretariat (2 of 27, but both were IDC members), or
• through special Ministerial events to which members are invited (2 of 27, but neither were CCRS members).

Impact on GoC Understanding of Community Needs

Based on several documents reviewed for the evaluation, the CCRS is impacting Government departments and agencies in terms of their understanding of, and sensitivity to, community needs. According to a document entitled “Questions Raised by the CCRS at the May 2005 Meeting and Responses by the Agencies”, the RCMP, CSIS, and the CBSA have all found the CCRS meetings to be useful. In response to the question “What have the agencies gained from interactions/discussions with roundtable members so far?”, RCMP, CSIS and CBSA responses said:

RCMP: “The value we gained was the interaction with the group to understand their issues and concerns. The true value of the interaction was the follow-up questions and the opportunity to discuss the country's threats and risks and how they can be raised to the Government… This interaction challenged some of our assumptions in a number of areas and that is always healthy.”

CSIS: “The Roundtable is proving to be an extremely valuable resource for CSIS as it seeks to improve its understanding of Canada's different ethnic communities and its outreach to those communities.”

CBSA: The CCRS is “an excellent forum to discuss the concerns of the community when it comes to its work and to better explain what we do. Listening to the concerns and thoughts of the communities will help us to better understand our limitations and will assist us in better explaining and communicating our programs to the public.”

As well, a presentation made by the Chair of the CCRS at the Second Public-Private Sector Summit on National Security, in May 2006, states that “(b)ased on our (CCRS) feedback, security agencies are re-assessing how they should interact with diverse communities with a long-term view to establishing sustained and effective relationships. More broadly, federal Government departments, including DoJ and PS, are also re-examining how they engage in outreach activities to Canadians on national security issues. Through our outreach, we have recognized the need to build trust and understanding among our diverse communities and the security agencies, toward the common goal of enhanced security.”

Most interviewees (22 of 29) agreed that the CCRS is having an impact on the Government’s understanding of, and sensitivity to, community needs. Interviewees were limited in providing specific examples, but indicating that, as a result of the CCRS:

• the Government is more sensitive to community needs (10 of 29 interviewees, including 5 of 6 PS representatives);
• the Government has changed the language it uses (9 of 29 interviewees); and
• the members bring community concerns to the Government (7 of 29 interviewees).

Impact of CCRS on GoC Policies, Directives and Communication Materials

When looking at whether Government policies, directives or communications materials have been impacted by the work of the CCRS, numerous examples were noted in the document review. One example is regarding the input the Roundtable members had in developing the RCMP's Bias-Free Policing Policy, a policy designed to build a more sustained and effective relationship with diverse communities. Also the CCRS' dialogue with Government officials resulted in the development, by PS and its Portfolio agencies, of a booklet on national security aimed at building awareness and understanding about the roles and responsibilities of the security agencies and the complaint mechanisms in place. The booklet was aimed at all Canadians, but was thought particularly useful to those new to Canada. One of the meeting reports of the IDC also suggested that CCRS input would be sought on a document by Canadian Heritage on best practices for outreach, although it is not evident that this input took place.

According to the [   *   ], the following specific outcomes can be attributed to the CCRS:

• “More effective and culturally sensitive approach to border security, law enforcement and security intelligence activities (e.g. new approach to preparations to press release in arrests of 17 alleged terrorists in Toronto in June 2006 that involved pre-briefings of select community leaders, and a new lexicon for these communications that was more specific to the threat).
• The CBSA used the Roundtable as a focus group to further refine their policy on the fair treatment of travellers. Transport Canada similarly consulted the Roundtable regarding the Passenger Protect Program, and their feedback served to inform policy decisions.
• The RCMP, CSIS and CBSA all met with a Sub-Committee of the Roundtable to look at training and recruitment issues for security enforcement and security intelligence to discuss how relationships with ethno-cultural communities could be enhanced. New programs or new approaches were explored. These agencies have now all launched outreach activities. PS was convinced, via the Roundtable dialogue, of the need to adopt a citizen engagement strategy to engage Canadians more broadly on the issue of national security and the role that citizens may play in addressing terrorism and how communities can prepare themselves for terrorist incidents and mitigate repercussions of hate and violence.” 

Interviewees also noted some of these impacts. PS staff and IDC members were asked in interviews whether Government policies, directives or communications materials have been impacted by the work of the CCRS. Almost all PS staff (5 of 6) and almost half of the IDC members (4 of 9) suggested CCRS has shaped/influenced policy. Almost as many PS staff (4 of 6) and IDC members (3 of 9) indicated that the Government has changed the language it uses as a result of the CCRS. Finally, one third of interviewees (2 of 9 IDC members, and 3 of 6 PS representatives) mentioned the impact the CCRS had in the creation of a security booklet10.

2.3.2 Question 10: Increased Community Understanding

Q10: To what extent has the CCRS contributed to a better community understanding of National Security policies, practices and challenges?

Conclusion: There is limited evidence that the CCRS has contributed to better community understanding. This may be due to the fact that there are no formal mechanisms in place for disseminating information to communities and there has been limited formal CCRS member-organized outreach conducted to date.

CCRS Impact on Communities

Some interviewees (8 of 21) believe that the CCRS is having an impact on community understanding of security issues; while a few (5 of 21) felt that to date they have had only a partial impact. Only a few interviewees said there has been no impact to date, however these were all CCRS members (3 of 14). Over one quarter of interviewees (6 of 21) did not know, or could not comment.

It is possible that the limited impact may be related to the fact that there are no formal mechanisms in place to disseminate information to communities. Some interviewees (7 of 16) suggested that there was no structured way for CCRS members to disseminate information to their communities. Interviewees suggested, though, that some dissemination has taken place through outreach events, or other ad hoc activities to engage and inform communities. As well, some interviewees (7 of 16) noted that to a certain extent dissemination of information to communities depends on the individual CCRS members, since they may distribute information through their own interactions in their communities and through their existing networks. One quarter of interviewees (4 of 16) suggested that information dissemination to communities is not a role of CCRS members.

This limited impact may also be related to the fact that there has been little formal CCRS member-organized outreach conducted to date, which might be related to a lack of clarity as to whether CCRS is mandated or resourced to do outreach (noted by 5 of 21 interviewees, with 4 of them being CCRS members). Additional information about outreach undertaken by the CCRS is provided in section 2.3.3.

2.3.3 Question 11: Community/GoC Contact

Q11: To what extent has the CCRS led to an increased capability or opportunity for direct contact between communities and Government of Canada personnel?

Conclusion: The CCRS has led to an increased capability or opportunity for direct contact between communities and Government of Canada personnel, through the CCRS meetings themselves, and through CCRS outreach events. However, limited CCRS member-organized outreach has taken place, to date. According to interviewees, additional funds are required to be more active in this area.

Direct Contact Between Communities and the GoC

The existence of the CCRS has led to increased opportunity for direct contact between communities and Government of Canada personnel through the simple fact that CCRS is made up of representatives of various communities and Government of Canada representatives participate in the CCRS meetings. Therefore, the conduct of the meetings offers opportunities for direct contact between communities and Government personnel.

Outside of the CCRS meetings, however, there are also opportunities for direct contact between communities and Government personnel through CCRS outreach events. There has been little formal tracking of outreach events that have taken place, therefore the GCS consultants were unable to ascertain, for certain, how many outreach events have taken place. However, based on documents reviewed, outreach activities of the CCRS have included:

1. Four regional (day-long) symposiums held in Montreal, Calgary, Vancouver and Toronto, with individuals interested in national security issues
2. Other symposiums as follows:
   • An Atlantic regional symposium, "Engaging Canadian Society in Keeping Canada Safe", in collaboration with the New Brunswick Multicultural Council Inc (NBMC). The symposium was held on February 5, 2006 in Fredericton, NB. Participants included members of various ethno-cultural communities, youth, academics and federal officials from PS, RCMP, CSIS, CBSA and Canadian Heritage
   • A local symposium held in Calgary, "National Security is Everyone's Concern" on March 18, 2006. The symposium brought together Calgarians of different backgrounds with members of the security agencies like RCMP, CSIS and CBSA to engage in an open and frank dialogue as to how to best implement new security measures.
   • A local symposium  held in Edmonton, Alberta on May 28, 2006, entitled "Symposium on Security and Civil Liberties"
3. An “Ethnocultural Roundtable”, held Saturday, August 13, 2005 in Burnaby, BC. Attended by 11 community participants, 3 departmental participants and 2 minister's office participants, and
4. A youth forum to discuss issues of concern with religious leaders of the Islamic and Anglican faiths in December 2005.

In addition, documentation received noted that two CCRS members attended a British Columbia conference entitled “Balance Between Security, Human Rights and Accountability” put on by BC’s Provincial Committee on Diversity and Policing, at the Justice Institute of BC, from November 30 to December 2, 2007. In addition, a PS senior executive participated in a panel at this conference, at the behest of a CCRS member, and PS staff also attended.

Other outreach events were noted in the documentation, however it is unclear to what extent these events were CCRS events, or whether they were other outreach undertaken by Government agencies who also participate in the CCRS. According to a presentation by the Chair of the CCRS at the Second Public-Private Sector Summit on National Security, “The CCRS has facilitated numerous meetings between Government and local community groups to ensure that a two-way dialogue takes place - explaining what security measures have been enacted, why, and what is being done to protect all Canadians. Equally important has been the information exchange, the listening to those who may feel there is a disproportionate impact on them from some of these measures.”  However, interviewees for this evaluation reported very few CCRS member-organized outreach events, and some interviewees (11 of 27, from all stakeholder groups) noted a lack of funding for outreach events.

Most interviewees (23 of 29) agreed that the CCRS has contributed to increased capability or opportunities for direct contact between communities and Government of Canada officials involved in national security. Many (15 of 29) cited the outreach events, while others (10 of 29) referred to the fact that the establishment of the CCRS has created a link or mechanism for the Government and communities to connect. Several interviewees referred to relationships which have been established through the CCRS, or networks that have been opened to Government, that have enabled further contact between Government and communities.

On a related note, throughout the evaluation, information from interviewees and document reviews highlighted a general lack of communication or coordination with regards to outreach activities. As indicated on page 6 there are a number of Government departments and agencies undertaking citizen engagement activities that may complement CCRS activities. However, currently there is no way for the CCRS, or PS, to have a comprehensive list of the activities that are taking place. Additionally, as previously mentioned, there is little tracking of CCRS outreach events -- either those originated by the CCRS or those that CCRS members may attend but that are sponsored by other organizations. As a result the CCRS Secretariat has limited ability or capacity to identify events that are planned and that are relevant to the CCRS, or to identify events that could, or should, have CCRS participation.

2.4 Cost Effectiveness

2.4.1 Question 12: CCRS Cost-Effectiveness

Q12: Is the operation of the CCRS cost-effective?

Conclusion: In the absence of cost information and similar comparison programs, GCS was not able to determine whether the operation of the CCRS was cost-effective. Information gathered related to cost-effectiveness shows that stakeholders feel that not enough resources have been allocated to the program and that there are possible improvements that could be made to improve effectiveness.

Sufficiency of Resources for CCRS

With regards to whether or not there are sufficient resources available to achieve the objectives of the CCRS, almost three quarters of interviewees (20 of 27) felt there are not sufficient financial and/or human resources to achieve the objectives of the CCRS. All other interviewees were not sure or could not comment. Almost half of all interviewees (13 of 27) commented regarding issues of staffing at the Secretariat – either a lack of staff, or unfilled positions due to turnover. Some others (11 of 27) noted that the CCRS does not have funding for outreach activities. On a related note, half of the CCRS members interviewed for the evaluation (7 of 14) complained that the CCRS does not have its own dedicated funds for outreach activities, or that the Secretariat has not told the CCRS what funds are available to them for these types of activities.

When asked what impact the lack of human and/or financial resources was having on the CCRS, over half of the respondents (15 of 26) stated that the CCRS is not able to conduct outreach activities. A few others noted that without additional funds communications around CCRS could not be done to raise the profile of the CCRS (3 of 26). The areas of greatest need identified by interviewees were outreach (14 of 26 respondents), and resourcing for the Secretariat (9 of 26). Both of these items were noted by 3 of 4 PS Senior Executives.

The [   *   ] noted pressures on the CCRS. According to this document, expectations of the CCRS members include funding beyond what is currently available. There is a call to increase the size of the CCRS and to increase the frequency of meetings, as well as for support around communication activities. Funding is not currently adequate to meet these member requests. As well, this document points out that supporting the CCRS and its Chair is labour-intensive, and a number of staffing issues have compounded the Secretariat’s capacity issue. Further, there is a need to stabilize staffing levels within the Secretariat. Finally, although salary funding was not maximized in 2006-07 due to a number of staff departures in the fiscal year, O&M required for funding CCRS meetings and related events, and regional symposia, etc. , was higher than originally allocated to the CCRS. Therefore, it was necessary to transfer funding from other sources internal to PS to fund CCRS-related activities. In the year 2006-07, $388,000 was transferred from other PS funded initiatives, to support the series of regional symposia and other CCRS activities. This was in addition to the CCRS’ annual operating budget of $175,000. The expenditures chart from 2006-07 and 2007-08 is attached at Appendix E.

When asked whether funds are appropriately allocated to CCRS priorities, the majority of interviewees (14 of 23) were not able to comment. Approximately one quarter of interviewees (6 of 23) did not think resources were appropriately allocated to CCRS priorities. All other interviewees did not respond to this question.

Alternative and Improvements to CCRS

The majority of interviewees either did not believe there were more cost-effective alternatives to the CCRS (14 of 29) or could not identify other more cost-effective alternatives (9 of 29). Alternative approaches were put forward by some interviewees; however, interviewees typically felt that these alternatives would not be as effective as the CCRS.

When asked about improvements to the CCRS, some interviewees (13 of 29) noted a need to stabilize the staff at the Secretariat. Over half of the CCRS members (9 of 14) made this suggestion. A few others (6 of 29) suggested that CCRS needs to do more outreach or allocate more funds for outreach. Clearly defining CCRS objectives was mentioned by a few interviewees (4 of 29) and a few (4 of 29, of which 3 were CCRS members) stated that CCRS needs to be informed of the funds available to it. There were a few other suggestions put forward by interviewees (3 of 29 interviewees for each point below), as follows:

• Changes to the terms and rotations of members of the CCRS;
• Use of alternative communication means (e.g. , video-conferencing);
• Creating an executive group within the CCRS;
• Better feedback on CCRS impacts;
• Members need to be more reflective of Canadians; and
• Better selection of CCRS members to ensure they are knowledgeable and have the necessary time available to commit to the CCRS.

3. Overall Conclusion and Recommendations

This section of the report provides an overall conclusion and, where appropriate, recommendations for the CCRS.

Relevance:

It is clear that there continues to be a need for the CCRS. National security remains a key priority of the GoC and there are no other programs that provide a similar opportunity for dialogue between the GoC and Canada’s multicultural communities on issues of national security. The Advisory Council on National Security was noted by interviewees, however the membership and role of this committee differs from that of the CCRS. Some other GoC initiatives were identified that touch on similar topics as the CCRS, or may target similar communities, however for the most part these initiatives are more focused on a particular issue or group, and often are one-time, ad hoc events. A couple of interviewees indicated they would like to see more coordination between the CCRS and these other outreach events.

On a related note, it is important for the CCRS to be able to identify those events most pertinent to the mandate of the CCRS, and to participate in these events. The extent to which CCRS members can participate in outreach events is limited by funding and time. In general there is a lack of communication or coordination with regards to outreach activities that do take place. There is no comprehensive list maintained of outreach activities taking place – either those originated by the CCRS or by individual members, or those originated by others. The fact that it is impossible to say, at any given time, what events are planned makes it difficult to identify and coordinate CCRS member participation. In order to improve the outreach function of the CCRS, it would be important to improve coordination and information-sharing.  Therefore, GCS recommends that:

…the CCRS Secretariat attempts to coordinate, where possible, its activities with outreach or citizen engagement activities of other Government of Canada departments and agencies, and

…the CCRS Secretariat establish and maintain a calendar of outreach events relevant to the CCRS, and to provide CCRS members with regular information on upcoming outreach events that may be of interest to them. In addition, in order to facilitate this calendar of events, CCRS members, as well as relevant Departments and agencies, should provide information to the CCRS Secretariat about events going on in their communities, and any outreach activities they may participate in.

The current objectives and scope of the CCRS seem to be appropriate, although while the CCRS has a well-defined mandate and scope, objectives have not been formally articulated and communicated to CCRS stakeholders. CCRS stakeholders interviewed were not clear on what the CCRS objectives were, and often used the terms “mandate”, “scope” and “objectives” interchangeably. As well, many of the interviewees indicated that they do not believe CCRS members and other stakeholders have a good understanding of the objectives and their role in meeting them. It is unclear from the information gathered whether confusion relating to roles and responsibilities stems from a need to more clearly define the roles and responsibilities or whether the roles and responsibilities need to be more effectively communicated. Therefore, GCS recommends that:

…the CCRS Secretariat clarifies the objectives of the initiative and clearly communicates these objectives, and roles and responsibilities, to CCRS stakeholders (i.e. , CCRS members and IDC members).

CCRS activities are aligned with the CCRS objective to have ongoing dialogue between GoC and communities on national security issues.

Design and Delivery:

The CCRS has undertaken start-up activities including the development of a formal nomination process for CCRS membership, the creation of guidance documents, the convening of members for the CCRS formal meetings and the creation of the IDC. However, there is no formalized, documented process for setting CCRS annual objectives, and the process is not clear to all stakeholders. Clarification around the setting of CCRS objectives would be beneficial to CCRS members.

Similarly, there is no formal, documented process for setting agendas; although interviewees were generally in agreement on the process being used. While interviewees believe that there is opportunity to provide input into the agenda-setting process, some felt more lead time is required for this to happen. In addition, there seems to be some confusion relating to whether agenda items should focus more on regional or local issues, versus national issues. Formalizing the process used for setting CCRS agendas could benefit the CCRS.

Therefore, GCS recommends that:

…the CCRS Secretariat formalizes and clarifies the process for setting annual objectives, and

…the CCRS Secretariat considers formalizing the process for setting meeting agendas, and

…the CCRS Secretariat communicates these processes to CCRS members.

With respect to outreach activities, there is no formal process in place for selecting and prioritizing these events. A more formal process around outreach is required, as recognized by the recent submission to the Minister of PS of the Interim Management Framework for the Cross-Cultural Roundtable on Security which included the establishment of a more formalized process for outreach.

While, for the most part, administrative processes are in place to support the functioning of the CCRS, members noted that there is no systematic feedback to CCRS members on actions taken on, or responses to, their recommendations; although this has happened on an ad hoc basis. As well, there are limited defined performance measures in place at this time. Tracking GoC action on CCRS discussions or recommendations would provide some information that could be a useful input into performance measurement and reporting. As well, some interviewees noted a need to provide this feedback to CCRS members.  Therefore, GCS recommends that:

…the CCRS Secretariat establishes additional performance measurement indicators, including ‘actions in response to CCRS discussions or recommendations’, and begins collecting the data necessary to report on these indicators.

Impact:

The CCRS has had an impact on the Government’s understanding and sensitivity to community needs. Some concrete examples were noted of CCRS impacts on GoC policies, directives and communications materials. However, there is limited evidence that the CCRS has contributed to better community understanding of national security policies, practices and challenges. This may be due to the fact that there are no formal mechanisms in place for disseminating information to communities, and there has been limited formal CCRS member-organized outreach conducted to date. Some interviewees noted a lack of clarity as to whether or not CCRS is mandated to do outreach. Therefore, GCS recommends that:

…the CCRS Secretariat clarifies the responsibility of the CCRS to undertake outreach and dissemination of information to communities, and that
…the CCRS Secretariat establishes mechanisms for disseminating information to communities, and either directly disseminates information to communities or supports CCRS members or other stakeholders, as appropriate, in using these mechanisms.

Cost Effectiveness:

In the absence of cost information and similar comparison programs, GCS was not able to determine whether the operation of the CCRS was cost-effective. Information gathered relating to cost-effectiveness shows that stakeholders feel that not enough resources have been allocated to the program and that there are possible improvements that could be made to improve effectiveness. In particular, stabilizing the staff at the Secretariat was noted by almost half of the interviewees. As well, additional funding for outreach activities was identified as a need. Finally, clarifying the objectives of the CCRS and the roles and responsibilities of the various stakeholders were identified as priorities, as already mentioned. Therefore, GCS recommends that:

…the CCRS Secretariat explores possible ways to stabilize the CCRS Secretariat staff, and if possible to dedicates staff solely to the support of the CCRS. In addition, PS should seek additional funding for the CCRS, to allow for outreach activities.

Appendix A: CCRS Logic Model

Appendix B: Evaluation Matrix

Appendix C: List of Documents Reviewed

1. Anti-Terrorism in Europe: An Overview (August, 2005)
2. Backgrounder - PSEP and Community Outreach
3. Canadian Security Issues of Interest to the Cross-Cultural Roundtable on Security (Nov, 2005)
4. CCRS - Interactions with Communities of Interest
5. CCRS Newsletter - 2nd Edition: Newsletter Content Proposal
6. CCRS Report of the Sub-committee for the Review of Training and Recruitment
7. CCRS Report of the Sub-committee for the Review of Training and Recruitment Approaches of the RCMP, CSIS and the CBSA
8. Conflict of Interest Declaration (Proposed letter for CCRS – draft)
9. Consultations/Outreach Events Calendar (Nov. 2004 - June 2006)
10. Cross-Cultural Roundtable on Security Secretariat. Teleconference on the Review of Training and Recruitment by Agencies (Sept, 2005)
11. Cross-Cultural Roundtable on Security Selection Criteria (Qs and As) (June, 2006)
12. Cross-Cultural Roundtable on Security Subcommittees
13. Cross-Cultural Roundtable on Security Terms of Reference (March, 2005)
14. Cross-Cultural Roundtable on Security Working Meeting, Gatineau, Québec. Meeting/Event Record ([   *   ])
15. [   *   ]
16. Developing an Outreach Strategy for the Cross-Cultural Roundtable on Security (April, 2005)
17. Ethnocultural Roundtable, Saturday August 13, 2005, Burnaby, British Columbia
18. Federal Consultation with Ethno-Cultural Communities: Lessons Learned and Suggestions for Good Practice (March 2005)
19. Implementation of National Security Policy (May, 2005)
20. Interdepartmental Meeting Agendas, Participants, Meeting Records (August 30 2005, September 27 2006, September 28 2007)
21. Key Milestones for the CCRS Sub-Committee on the Review of Training and Recruitment Materials
22. Memorandum for the Minister. Appointment of Members: Cross-Cultural Roundtable on Security (May, 2006)
23. National Security: Canada's National Security Agencies
24. Outreach Activities since June 2nd 2006
25. Presentation by the Chair of the CCRS at the Second Public-Private Sector Summit on National Security (May, 2006)
26. Presentation to the Advisory Council on National Security Regarding the CCRS (Dec, 2005)
27. Principles and Procedural Rules Guiding the Dialogue of the Cross-Cultural Roundtable on Security (Roundtable) (March, 2005)
28. Proposal for Community Outreach Activity by Members of the CCRS
29. Proposed Content of the CCRS Newsletter - Issue 1
30. Public Safety Canada Media Lines (June, 2007)
31. [   *   ]
32. Recap of the Sub-Committee's Comments on Training Issues Gleaned from the Reports by the Three Agencies (Draft 2)
33. Regional (East) Meeting of the Roundtable Summary
34. Report of the Discussion Groups on Strategies to Address Community Relations in the Event of a Terrorist Act in Canada or Directed Against Canadians (Nov, 2005)
35. Request for further Information and Data Pursuant to the Teleconference of September 29, 2005
36. Roundtable News: The Cross-Cultural Roundtable on Security (Fall, 2006)
37. Scanning of U.S. Security Issues of Interest to the CCRS (Nov, 2005)
38. Scoring forms for successful and unsuccessful CCRS candidates
39. Securing an Open Society: Canada’s National Security Policy (April, 2004)
40. Security is Everyone's Concern: A Local Symposium held in Calgary on March 18, 2006, on behalf of the CCRS
41. Summary Report of the Atlantic Regional Symposium: Engaging Canadian Society in Keeping Canada Safe
42. Summary Report, Cross-Cultural Roundtable on Security, Anti-Terrorism Act Review, April 27-29, 2007
43. Summary Reports of CCRS Meetings (March 7-8 2005, May 23-25 2005, November 19-21 2005, June 3-5 2006, November 18-20 2006, April 28 2007)
44. The CCRS Montreal Regional Symposium; Oct. 29, 2006, Hotel Delta Centre-Ville, Montreal (Oct, 2006)
45. Values and Ethics Code for the Public Service, Chapter 2: Conflict of Interest Measures
46. Various correspondence regarding the selection process for the CCRS

Appendix D: List of Interviewees

CCRS Members

Interdepartmental Committee Members

Public Safety Senior Executives

Public Safety Senior Executives (Direct involvement with CCRS)

CCRS Secretariat Staff (and former staff)

Appendix E: Expenditures

1 [   *   ].

http://www.publicsafety.gc.ca/cnt/ntnl-scrt/scrng-eng.aspx at the time of drafting this report.

http://www2.parl.gc.ca/Parlinfo/Documents/ThroneSpeech/39-2-e.html, at the time of drafting this report.

4 [   *   ].

http://www.publicsafety.gc.ca/cnt/ntnl-scrt/crss-cltrl-rndtbl/trms-rfrnc-eng.aspx

6 Memorandum to the Minister: Interim Management Framework for The Cross-Cultural Roundtable on Security

7 Note: revised terms of reference were included in the Interim Management Framework for the Cross-Cultural Roundtable on Security submitted and approved by the Minister of PS in January 2008. The revised terms of reference include more detail relating to roles and responsibilities of the Chair and Vice-Chair of the CCRS, CCRS members, the Secretariat and other stakeholders.

8 Given changes to Government Committee structures, this is no longer a requirement of the CCRS Secretariat.

9 In addition, documentation provided for the evaluation references CCRS East / West meetings, however dates for these meetings were not provided.

10 The CCRS Secretariat is currently working with Citizenship and Immigration Canada to translate this booklet into other languages. It is currently available in French and English.

Date modified:

2015-12-15