In May 2012, the Canadian Career Development Foundation (CCDF) began a project, The State of Practice: Essential Skills Applications with First Nations, Inuit and Métis in Canada (FIMESA), that’s aim, is to: • Increase the understanding of Essential Skills Applications for First Nations, Inuit and Métis by developing a comprehensive inventory of current Essential Skills practices aimed at increasing employability and employment for First Nations, Inuit and Métis youth and adults living in diverse environments and; • Increase capacity in the field by developing an Essential Skills (ES) community of practice engaged in the development and assessment of the inventory, the widespread dissemination of results and promotion of ongoing innovation through the sharing of best practices in Essential Skills application and evaluation. This project is meant to solidify the field’s understanding of the state of practice with respect to ES applications tailored to First Nations, Inuit and Métis populations, help “uncover” factors which contribute to strong employability and employment outcomes and, through the establishment of an ES community of practice, identify, share and promote innovation and excellence in service delivery and evaluation. The purpose of this literature review is to describe the current level of need for Essential Skill development among First Nations, Inuit and Métis, to explore the state of practice of Essential Skills initiatives with these populations in Canada and to examine innovative practices in an effort to determine potential “markers of excellence” in ES programming.
Canadian policy decision-making has utilized case studies extensively in recent years. Johnston Research Inc. (JRI) has completed more evaluation-related case studies over the past 4 years than in the previous 15 years of our evaluation work. To understand the growing application of case studies, we interviewed clients and contacts from First Nations that had been case study sites for our government clients, to understand what aspects of case study evaluation research had helped them share their opinions and improve their programs, and what aspects had not. We then interviewed our government clients, asking how well case studies served their evaluation purposes and their programs or policy development efforts. JRI conducted and financed this study to help us improve our own approaches for conducting case studies in Aboriginal populations and to share these findings with others. This article presents our interview findings on the value of case studies for Aboriginal evaluation projects and shares some best practices for conducting case studies within, and with, First Nations. Finally, we explore the impact case studies have had on Canadian policy.
Fidelity concerns the extent to which a specific evaluation sufficiently incorporates the core characteristics of the overall approach to justify labeling that evaluation by its designated name. Fidelity has traditionally meant implementing a model in exactly the same way each time following the prescribed steps and procedures. The essential principles of developmental evaluation (DE), in contrast, provide high-inference sensitizing guidance that must be interpreted and applied contextually. In lieu of operationalizing DE fidelity criteria, I suggest addressing the degree of manifest sensitivity to essential principles. Principles as sensitizing concepts replace operational rules. This means that sensitivity to essential DE principles should be explicitly and contextually manifest in both processes and outcomes, in both design and use of findings. Eight essential principles of DE are identified and explained. Finally, 10 threats to evaluation model fidelity and/or degree of manifest sensitivity are identified with ways to mitigate those threats.
Relative to the contribution that faith-based organizations make to Canadian society, evaluations of them are rare. Thechallenge for evaluators is to develop evaluation processes that meet the scrutiny of social science yet respect the centrality of faith inherent within their interventions. The challenge is compounded when evaluating faith-based innovation. This article reviews the present status of evaluation in faith-based settings, highlighting its present limitations. It then features an innovative national faith-based evaluation framework that attempted to address these limitations. The article ends with critical reflections on the learnings of this case example in conducting evaluations of faith-based organizations and implications for other sectors.
This essay engages questions of evaluator role and indigenous peoples participation in evaluation within colonial and decolonization contexts. Specifically, I critique the Western emphasis on cultural competence and contrast the utility of ‘mainstream’ evaluation approaches alongside three indigenous inquiry models (Te Kotahitanga, Whakawhanaungatanga, and He Taniko) as utilized by/with indigenous Maori in Aotearoa/New Zealand. Using practical examples of evaluation projects conducted with and by Maori, the article highlights the very different ‘evaluation conversations’ happening amongst ‘mainstream’ practitioners—where the focus is on difference, competency, and issues of access—relative to those occurring amongst indigenous evaluators and communities—where evaluation praxis is framed within broader struggles for sovereignty and self-determination. By placing these paradigms in conversation with each other, I highlight the ways in which evaluation approaches that engage indigenous people and places are always representative of particular standpoints. This is because evaluation is unavoidably and simultaneous in dialog with the prevailing contexts of colonization and decolonization vis-a-vis the location and moment in which it occurs. The essay foregrounds the ways in which `mainstream’ evaluation’s preoccupation with issues of cultural competency fails to fully address the needs and aspirations of indigenous peoples. In contrast, the realization of Maori capacity to meet our evaluation needs as Maori, and as represented in the ongoing development and use of our own approaches and models, not only facilitates a more culturally meaningful evaluation process but also concurrently constitutes an expression of our sovereignty and agency.
Purpose: Stigma and lack of access to providers create barriers to mental health treatment for older adults living in the community. In order to address these barriers, we developed and evaluated a peer support intervention for older adults receiving Medicaid services. Design and Methods: Reclaiming Joy is a mental health intervention that pairs an older adult volunteer with a participant (older adult who receives peer support). Volunteers receive training on the strengths-based approach, mental health and aging, goal setting and attainment, community resources, and safety. Participant–volunteer pairs meet once a week for 10 weeks. Participants establish and work toward goals (e.g., better self-care, social engagement) that they feel would improve their mental health and well-being. Aging services agencies provide a part time person to manage the program, match volunteers and participants, and provide ongoing support. Outcomes evaluation for this pilot study included pre/postintervention assessments of participants. Results: Thirty-two participants completed the intervention. Pre/post assessment group means showed statistically significant improvementfor depression but not for symptoms of anxiety. Quality-of-life indicators for health and functioning also improved for participants with symptoms of both depression and anxiety. Implications: The Reclaiming Joy peer support intervention has potential for reducing depression and increasing quality of life in low-income older adults who have physical health conditions. It is feasible to administer and sustain the intervention through collaborative efforts with minimal program resources and a small amount of technical assistance.
Program evaluation can be used to assess health and education programs, health promotion programs, and social programs among others. Program evaluation can provide valuable insight into program goals, activities and target population, program strengths, areas for program improvement, and the cost-effectiveness of a program. It is an important accountability tool and can be used to justify continued funding for programs or new directions in programming. To obtain maximum benefit from program evaluations and research with Aboriginal peoples and organizations, they must be considered full partners in the project. Many Indigenous scholars have articulated respectful approaches for engaging in such evaluations. This paper will briefly review different types of program evaluation activities and discuss Indigenous approaches and ethical guidelines for engaging in a program evaluation.