Community Food Project Evaluation handbook

The full pdf can be found here


Community Food Security Coalition
Sponsor: USDA Community Food Projects Program

Author: National Research Center, Inc.

First Edition, 2003 Second Edition, 2004 Third Edition, 2006

We welcome limited duplication of contents of the Evaluation Handbook for non-profit and educational purposes. Please credit the source in all copies, and if possible, include this page.

The Community Food Project Evaluation Handbook EaH Eritten Oy Pational QeHearc> Center; RncF
Some content Eit>in t>iH >andOooU EaH adaMted Grom MreSiouHly Eritten >andOooUH aut>ored Oy PQC includinI Outcome Handbook for City of Boulder Human Service Providers VWanuary A999X and After School Initiative Evaluation Handbook HMonHored Oy T>e Colorado TruHt V788AB7887XF


Data analysis may very well be the most intimidating part of program evaluation. A large part of data analysis is statistical in nature. Bhile many of us may have academic experience in statistical analysis, translating this into a useful analysis for direct services may be daunting. Analysis of program data, however, has become essential to program management, improvement and continued funding. In this chapter we hope to simplify the world of data analysis by describing each step in its practical relevance and application in order for programs to find ways of adapting such strategies to their setting.

It may be helpful for your program to identify the individual on staff with the most knowledge and interest in analyzing your program data. This person should be comfortable using a computer, and not be afraid of numbers. Often an administrative assistant or someone who manages the bookkeeping or finances will make a good choice (make sure they agree). Review this chapter with whoever is chosen to assist with data analysis. If your program has limited staffing resources or analytic abilities, you may consider as an alternative, seeking an outside evaluator or graduate student to help with these next steps of your evaluation.

Creating an Evaluation Notebook

Creating an evaluation notebook can be an excellent tool for tracking all of your evaluation information. Once the analysis starts, there is plenty of important information to keep track of and having it collected in one place will make it easier. The notebook should include items you may already have created, such as copies of the final evaluation tools (hard copy and electronic on disk) and data collection protocols. Mou will also want to add any items you may create as a result of going through this chapter, such as your analysis plan, codebooks, coding sheets and data printouts. Mou may also find it helpful to have pages of reflections on your evaluation process N what went well, what did not and improvements you might want to make for the next go- round. This notebook will ensure that all current information is in the same place as well as provide a quick look-up when questions arise during the next evaluation or as new staff are assigned evaluation tasks.

Developing an Analysis Plan

A simple analysis plan can be created by elaborating on your evaluation plan worksheet (see Chapter 4, Borksheet Q5). The analysis plan will list all of the specifically analysis to be performed on the evaluation data. An example of such an analysis plan is presented along with a worksheet on the following pages.

Chapter 8. Analyzing Your Data

Community Food Security Coalition! 2

The plan is quantitative in natureS that is, it focuses on counts and proportions. More complex analysis plans will need to be created for more complex evaluation designs or those using qualitative data collection methods. (For more on evaluation designs, see Chapter 5. Selecting Evaluation Strategies and Study Designs). Even with more complex analyses, however, it is important to Ustart with the end in mind,V and to let your evaluation questions guide the analysis. The worksheet provided often can be used for simple or even complex evaluations.

Chapter 8. Analyzing Your Data

Community Food Security Coalition! 3

SamMle Saluation 3lan? PeiI>Oor>ood [arden 3rodect

3roIram [oal

To increaHe IardeninI HUillH Gor MarticiMantH

To increaHe HatiHGaction amonI Yatino reHidentH Eit> Mroduce oGGered at neiI>Oor>ood GarmereH marUet

RndicatorH V4utMutH and 4utcomeHX

`ata SourceH

3erGormance Standard

`ata <nalyHiH

  • !  CountH oG SolunteerH trained
  • !  CountH oG SolunteerH >ourH EorUed
  • !  CountH oG Solunteer >ourH Oy actiSity tyMe
  • !  CountH oG yout> MarticiMatinI inIarden
  • !  CountH oG yout>>ourH
  • !  CountH andMercentaIeH oG yout> anHEerinI hHtronIly aIreei or haIreei to HtatementH?
  • !  Since R came to t>e Iarden;
    • !  Iammoreofa leader
    • !  I feel more connected tomy culture
    • !  I eat morevegetables

Increased knowledge of gardening practices

  • !  PumOer oG SolunteerH trained
  • !  Total Solunteer time
  • !  `eHcriMtion oGSolunteer actiSitieH
  • !  <dminiHtratiSe recordH
  • !  :olunteer actiSity loIH
  • !  :olunteer HurSey
  • !  A8 SolunteerH total; includinI tEo core SolunteerH
  • !  Total Solunteer time meetH need

Satisfaction with food selection

  • !  PumOer oG yout> MarticiMatinI in Iarden
  • !  PumOer oG >ourH yout> MarticiMated in Iarden
  • !  RncreaHe in yout> leaderH>iM HUillH
  • !  RncreaHe in yout> connection toculturefOacUIround
  • !  RncreaHed conHumMtionoG SeIetaOleH Oy yout>
  • !  cout> HiInBin H>eetH
  • !  SurSeyH oG yout> VMoHtBMroIramX
  • !  7DgoG MarticiMatinI yout> Eill reMort an increaHe in leaderH>iM HUillH
  • !  K8goG MarticiMatinI yout> Eill reMort an increaHe in t>eir connection to t>eir culturefOacUIroun d
  • !  =8goG MarticiMatinI yout> Eill reMort an increaHe in t>e amount oG SeIetaOleH t>ey eat

Chapter 8. Analyzing Your Data

Community Food Security Coalition! 4

3roIram [oal

To increaHe ciSic reHMonHiOility

RndicatorH V4utMutH and 4utcomeHX

`ata SourceH

3erGormance Standard

`ata <nalyHiH

  • !  CountH oG MoundH oG Good Mroduced
  • !  CountH oG MoundH oG Good Hold
  • !  CountH oG MoundH oG Good taUen >ome Gor MerHonal uHe

SamMle Saluation 3lan? PeiI>Oor>ood [arden 3rodect

Community service

  • !  `eHcriMtion oG oriIinal OarrierH to oOtaininI Good Mrior to Mrodect
  • !  `eHcriMtion oG OarrierH to oOtaininI Good aGter MarticiMation in t>e Mrodect
  • !  juantity oG Mroduce IroEn and Hold
  • !  [arden loIH oG MroduceIroEn
  • !  ^arUet loIH oGMroduce Hold
  • !  [arden loIHoG Mroduce taUen >ome Oy yout> and SolunteerH
  • !  D88 MoundH oG Mroduce Eill OeIroEn in t>eIarden in 788@
  • !  @88 MoundH oG Mroduce Eill OeHold at t>e marUetin 788@
  • !  AD8 MoundH oGMroduce Eill Oe taUen >ome Gor MerHonal uHe Oy yout> and Solunteer IroEerH in 788@

Increased collaboration

  • !  PumOer oG orIanikationHMarticiMatinI in GoodnetEorU
  • !  PumOer oG meetinIH>eld

! ^eetinI MarticiMation

tracUinI GormH

! C communityB OaHed

orIanikationH Eill attend mont>ly netEorU meetinIH

To increaHe collaOoration oG GoodB related community orIanikationH

  • !  CountH oG orIanikationH
  • !  CountH oG meetinIH
  • !  <SeraIe attendanceMer meetinI

Chapter 8. Analyzing Your Data

Community Food Security Coalition! 5

Worksheet 1: Analysis Plan

Saluation jueHtionH RndicatorH `ata SourceH 3erGormance `ata <nalyHiH RndicatorH

Chapter 8. Analyzing Your Data

Community Food Security Coalition! 6

<uantitative Data Analysis

This section describes techniques for performing quantitative data analysis, methods that produce numerical summaries of your findings. Instructions for performing simple qualitative analysis techniques begin on page 136. Those who already know how to use a statistical program such as SPSS or SAS, and are familiar with the process of creating electronic datasets from surveys or other sources may wish to skip this section. Much of this section focuses on using information from a survey. If you have quantitative data from another source, you may continue to follow most of these same steps. For example, forms used to count and classify customers at a Farmer’s Market may also be entered into an electronic dataset and analyzed in a similar fashion as that of recording food distribution.

Preparing Your Data for Analysis

Say you have just completed your first survey administration. Now you have a stack of completed surveys in front of you and you are not quite sure how to go about producing some useful information from your pile. Mou probably are considering a Uhand tallyV of the surveys at this point, but know in the back of your mind that there must be a much more efficient way of analyzing data from these surveys that may involve using your computer.

Mou are right. The question is how to go from the large stack of data to a concise computer print out. Basically, you will be taking your stack of surveys and creating a UnumericV electronic dataset that can be analyzed. Mou may be wondering what will make the dataset Unumeric.V Almost all analysis programs run more efficiently when they tally numbers rather than words. Once you have some experience with this, you will also find that you can complete the data entry much more quickly using numbers rather than letters or words. Consider the numbers just codes for the words. For example, you may use the number 1 as a code for the answer Uyes,V 2 for Ukind ofV and 3 for Unot really.V This will make more sense as we go further in this chapter.

Before creating your electronic numeric dataset, you will need to prepare the surveys for data entry using the following steps.

Coding and Identification Numbers

If your surveys were administered anonymously, asking respondents not to include their names, then each survey must be assigned an identification number before entering it electronically. This number will allow you to go back to an actual survey at any time for clarification if needed. The unique number is placed in the same spot on each survey (e.g., the upper right corner of the page)

Chapter 8. Analyzing Your Data

Community Food Security Coalition! 7

and can range from 1 to the number of surveys administered. The survey excerpt on the following page gives an example of how to ID surveys. Specifically note the area highlighted.

” #aking a )ifference. #o0ing to 2utcome-6ased 9ccountability for Comprehensi0e @er0ice Aeform. Aesource 6rief C, by Eoung, Fancy et al. Fational Center for @er0ice Gntegration, Falls Church, I9, “JJK
L 9ugmentation of 6ill Phillips in Aeport on Nessons Nearned in the Pilot Phase of the Onited Way 2utcomes Project. RSune “JJTU. Onited Way of Vreater #ilwaukee, Gnc., T.
X F. Y. Eoung, @. N. Vardner, and @. #. Coley. Vetting to 2utcomes in Gntegrated @er0ice )eli0ery #odels. Gn #aking a )ifference. #o0ing to 2utcome-6ased 9ccountability for Comprehensi0e @er0ice Aeforms. Falls Church, I9. Fational Center for @er0ice Gntegration Gnformation Clearinghouse, “JJK.
K Aeport on Nessons Nearned in the Pilot Phase of the Onited Way 2utcomes Project, Onited Way of Vreater #ilwaukee, Gnc., Sune “JJT.p Z-“X.
T 9. )onabedian. The Aole of 2utcomes in uality 9ssessment and 9ssurance. A6, Fo0ember “JJL, XTZ-XZ].

Chapter 8. Analyzing Your Data

Community Food Security Coalition! 8



Author: bryan nettles