evaluation

What is it? | Guides | Internet resources | Sample questions | Ethics | Clearinghouse papers | Reading evaluations | Reports | Seminars, workshops and conferences | Commercial services | Links

 

What is evaluation?

When we evaluate a domestic or family violence intervention (a policy, program, service, training, standards, etc), we are interested in seeing how useful or effective it is. Evaluation is interested in the value of something.  Evaluation uses research techniques but does not have to be complicated; even simple evaluations can provide important information.

Evaluate your programs and services to:

  • measure progress towards goals

  • refine goals and strategies

  • improve the quality of interventions

  • improve outcomes for clients, users and target groups

  • build financial and other support for a program or service

  • build the body of evidence about ‘what works” 

 

Guides to evaluation

There are many guides and articles on conducting evaluation, including online resources.  Here are just a few:

 

Evaluation resources on the internet

Action Research Resources

This site provides papers as well as an online course on action research.  

Resources for Methods in Evaluation and Social Research

Resources for program evaluation and social research methods are available on this site.  It contains information on how to conduct evaluation and evaluation methods: surveys, focus groups, sampling, interviews etc.

International Aid and Development

This site provides a forum for sharing ideas and practical examples of conducting evaluations of international development projects.

A Beginners Guide to Evaluation

This is a free resource on program evaluation. 

 

Sample evaluation questions
Most importantly,

  • did the intervention achieve its goals; e.g. did it improve the safety of victims, lower recidivism, prevent violence?

and also

  • was the problem clearly identified and did the intervention directly address the problem?
  • were the goals of the intervention valid; did users identify different goals from those of program/service operators or funders?
  • how well did the intervention reach the target group; did it capture their diversity?
  • is the program transferable to other areas or was its success dependent on local organisational relationships, key staff, social, economic, policy or legislative contexts?
  • were the strategies used based on research or evidence?
  • did they actually target the problem and how?
  • did the strategies and activities work as expected (would these changes have happened anyway?) and how well did they do this?
  • were estimations of inputs and outputs correct, were there unexpected inputs and outputs?
  • was the program cost effective?
  • were the assumptions underlying the intervention correct and were there factors not accounted for?
  • were anticipated outcomes delivered to the degree expected?
  • were they good outcomes for users?
  • were they sustainable?

 

Ethical practice in evaluation

When evaluating domestic and family violence interventions, the safety and well being of participants are critically important.  Evaluators need to keep adverse impacts of their investigations to a minimum.  Where possible, they should aim to inform, assist and empower participants, and avoid distressing or endangering them.

At a minimum, evaluators should:

  • ensure participants understand what the evaluation is about and give informed consent to participate

  • minimise the potential for harm to anyone as a result of the evaluation process

  • negotiate payment or restitution for participant's time and costs

  • ensure confidentiality and security of personal and identifying information

  • provide support and referral, if required

  • treat participants respectfully

  • share findings with participants

 

Ethical guidelines for evaluation and research are provided by a number of organisations:

 

Clearinghouse papers on evaluation

 

Reading evaluation reports

You need to have a critical eye when reading evaluation reports.  Evaluations can be limited by their scope, time, resources, access to users or staff, data available, evaluator abilities or by parameters set by funders. 

Consider:

  • did the evaluation look at the entire program or service, or only parts of it?

  • whose objectives or outcomes did it focus on, those of the users/target groups, service or program staff, or funders?

  • was the evaluation conducted during the implementation of the program or service, or following implementation, noting that memories can fade and workers can move on?

  • who was included in the sample; i.e. did it represent the diversity of the population group?

  • what questions were asked?

  • was there a control group or a comparison before and after implementation; i.e. how do you know if the intervention was responsible for the change?

  • did it measure the outcomes or only the process?

  • did the data tell you enough about the program or service or outcomes?

  • was the data validated by different sources?

  • are the conclusions of the evaluation supported by the data?

  • did the evaluation consider contextual issues that might have influenced outcomes; e.g. the  impact of individual staff members, policy, legislative, social or economic conditions?

  • did the evaluation consider if outcomes were sustainable?

 

Evaluation reports

Evaluation reports on Clearinghouse databases

Find materials on evaluation on the Clearinghouse Research and Resources database

Find materials listed on the Clearinghouse Good Practice database

 

Seminars, workshops and conferences

The Australasian Evaluation Society provides free workshops and seminars for members and non-members.  See regional branches for details

AES International Conference 2010
Workshops: 30 - 31 August 2010
Conference: 1- 3 September 2010
Wellington, New Zealand

http://www.aesconference2010.org.nz/

 

Evaluation 2009
11 to 14 November
Workshops 9 to 11 & 15 November
Orlando, Florida

  •      52 Workshops to build your knowledge and skills
  •      Over 600 Conference sessions spanning the breadth of the field
  •      In-depth exploration of Presidential Theme: Context and Evaluation
  •      Formal and informal networking opportunities 

Registration now open (early bird closes September 27) http://www.eval.org/eval2009/

 

Commercial evaluation services

The Australasian Evaluation Society provides a list of its members who provide commercial evaluation services .

 

Evaluation links

See the following organisations for more information about evaluation: