Evaluation and Knowledge Impact

Approaches to evaluation that build capacity and connect people in remote and urban communities, NGOs and government, project funders and policy makers, leading to better outcomes for all involved. evallaunch

Evaluation and Knowledge Impact (EKI) comprises two program areas: EKI itself, and the Realist Research, Evaluation and Learning Initiative (RREALI).  

EKI builds the capacity of NT evaluation stakeholders in: evaluative thinking and its use in program development, design, monitoring and evaluation; commissioning evaluations; disseminating and using findings to meet accountability requirements and improve governance and service responses; and conducting policy, program and strategic evaluations. 

EKI uses similar approaches in international development, with a particular focus on our near northern neighbours in Asia. It uses multiple strategies, including undertaking evaluations; orioviding training workshops; mentoring; collaborative work with partners with a gradual handover of roles; use of innovative methods in commissioned evaluations; and facilitation of impact workshops to understand findings and develop implications for changes to policy and practice.

RREALI focuses on realist methodologies and methodological development, and uses applied research in development and testing of the methods. 

Realist methodologies are grounded in a realist philosophy of science and seek to build understanding of how and why policies and programs generate differential outcome in different contexts. This knowledge can then be used to inform decisions about, for example, the appropriate mix of programs for different contexts, how to tailor programs to different contexts, and whether – and how – to scale programs out. 

RREALI also contributes to capacity building using the same range of strategies described above.

 

The Evaluation and Knowledge Impact team

Our focus

  • Realist evaluations looking at “what works in which circumstances and for whom”
  • Project evaluations requiring analysis of numerical data (including developing surveys), or qualitative approaches (focus groups, workshops, content analysis and other creative approaches) 
  • Strategic evaluations
  • Evaluating draft policies just before they going through the legislative processes 
  • Evaluating strengths and weaknesses of proposed program approaches and identifying improvements before they are implemented
  • Evaluation training (evaluations can be contracted for programs, policies or strategies)
  • Collaborative evaluations can also be commissioned on a ‘skills transfer’ basis, where the NI evaluators work collaboratively with government and program personnel to build their internal capacity to identify, document, and address monitoring and evaluation issues.

Our Impacteval2

EKI/RREALI is a relatively new team within NI.  In its early years, it focused on engagement and capacity building strategies, and on conducting commissioned evaluations.

RREALI was established in mid-2016 and is the first realist research and evaluation hub in the southern hemisphere (there are several in England.) In the few months since then, it has won two evaluation projects and seed funding for a research program to develop Realist Economic Evaluation.

The new projects each incorporate a specific aspect of methodological development.  The innovations are being developed specifically because they hold significant potential for widespread use in other areas of policy and program planning and evaluation, and therefore have the potential for widespread policy impact.

  • Kunga Program (CAWLS, 2014-16). A developmental, realist-informed and participatory evaluation of innovative pre- and post-release interventions for violent female Indigenous offenders exiting Alice Springs prison into the community. 
  • Remote Area Rotation Initiative (RARI) Evaluation (2012-2013). A pilot initiative to attract skilled child protection practitioners to the NT to address a critical workforce shortage.  The evaluations identified the activities and outcomes of RARI, and provided recommendations for future developments.
 

Selected projects

A selection of past and current projects:

Community Services Support Program (Department for Communities and Social Inclusion SA, 2016-2018).

Evaluation of the impact of moving to a results based accountability approach for funding of six streams of community services across SA.

Indigenous Community Volunteers Monitoring, Evaluation and Learning Review, Phase 3 (ICV, 2016).

Identifying participatory, culturally informed monitoring systems suitable for remote and urban Indigenous stakeholders, as well as policy makers, including identifying metrics and building staff capacity to monitor projects in over 100 communities across Australia 

Innovative models in addressing violence against Indigenous women (ANROWS, 2015-17).

Joint project with UWA, Faculty of Law, producing a state of knowledge paper and conducting a realist informed evaluation of community responses to Indigenous domestic/family violence in the Kimberley, WA, Top End of NT, and the Cherbourg region in Queensland

Safe Streets Audit

The Safe Streets Audit is intended to be the first of several joint projects that will build the Northern Institute’s capacity to research and evaluate NT crime and safety issues. The research focused on crime trends and perceptions of public safety in the urban areas of Darwin, Katherine and Alice Springs. It involved a literature review, analysis of three years of police data from 2010 to 2012, media analysis for selected weeks during that time period, and consultations with crime reduction stakeholders in Darwin, Katherine and Alice Springs.

Targeted Family Support Scheme (TFSS) Evaluation

Targeted Family Support Services provides a range of targeted services through NGO agencies to support low risk, high need families and prevent entry or re-entry to the child protection system. It works in close partnership with Department of Community and Families (DCF) programs responsible for child protection in the Northern Territory. The evaluation worked with program teams to review progress and support in data gathering. The recommendations and approach received a very positive response.

Remote Area Rotation Initiative (RARI) Evaluation

The Remote Area Rotation Initiative (RARI) was a pilot initiative to attract skilled child protection practitioners to the Northern Territory to address a critical workforce shortage in this area. By interviewing participants and stakeholders, the evaluators identified the activities and outcomes of RARI, and provided recommendations for possible future developments.

DiversityWISE summary evaluation

This project followed an initial formative evaluation of the Darwin based enhanced employment program for refugees, set up and managed by WISE Employment. It focused on the employment context and impact of DiversityWISE. A partnership with ANU supported a cost-benefit view of the program.