Developmental Evaluation

• Developmental evaluation (DE) is a growing area of evaluation, first created by Michael Quinn Patton, that enables the design, monitoring and adaptation of social change initiatives in complex and uncertain environments.  
• It is best used in highly emergent, changing situations, or early-stage projects/programs where more linear forms of evaluation such as summative and/or formative are not well-suited. It is less concerned with the final result than with the process taken to reach it. 
• The purpose of DE is not to see if a program meets its initial stated objectives and outcomes, as a summative evaluation would do, but rather to track a program’s design, development and implantation implementation process in real time, in order to adapt to emergent dynamics allowing for improvements.

• Rather than using statistics, past policies, programs, and government agendas to inform policy design, DE provides on-the-ground evidence about program needs, strengths and limitations, and what works and what doesn’t. So, DE truly speaks to evidence-based policymaking which is a core requirement of program design 
• DE takes a systems dynamics/system thinking approach 
• It is a holistic approach, so the entire design is considered 
• DE is grounded in innovative thinking and innovative results because it focuses on strategic learning and the integration of the results of that learning.
• Its fluidity/flexibility allows for real-time changes in program design so that stakeholders can implement new actions/adaptations as goals emerge and evolve.
• DE supports collaborative creation as it ensures all stakeholders are included in the process.
• When there is fluidity of your contextual circumstances, the project may need to pivot at any time, and you need to have the best real-time information upon which to make decisions. 

• A certain disposition and orientation towards the unknown.
• A comfort with ambiguity while at the same time being able to sense when it’s time to push for clarity.
• An acceptance that things shift and change as well as being able to shift and change along with them.
• Transparency and openness between evaluators and organizations; ideally, the DE evaluator is embedded within the team instead of acting as an observer.
• Being honest about one’s own biases and blind spots when doing an evaluation.
• Understanding and working with truly emergent data and accepting that there may not be any final answers or recommendations at the other end.
• Being prepared to accept a program or initiative isn’t working, and that sometimes it needs to be radically overhauled, or possibly abandoned. 

• Strategic thinking: A developmental evaluator should help stakeholders develop a sense of direction by helping to develop an actionable plan in keeping with organizational principles. 
• Relationship building: A developmental evaluator needs to be able to help groups cultivate their strengths while maintaining a productive team environment. They need to be able to ask difficult questions, engage in uncomfortable conversations, and deal with tensions in the group. 
• Leadership Engagement: A developmental evaluator needs to understand how to engage organizational leadership in the DE process because evaluators are there to lead the evaluation process in collaboration with the organization and not to drive their own agendas.
• Pattern recognition: developmental evaluators need to be able to identify key patterns from an array of complex information and activities. They are analytical and are able to categorize information as well as identify emergent themes and make critical connections. 
• Tolerance for uncertainty: development tool evaluators need to be able to tolerate the kinds of ambiguity that is associated with emergent initiatives. 
• Effective listening and communication skills: developmental evaluators need to be able to connect with individuals, listen to what they have to say, as well as offer insight and input when necessary.

Note: It is essential to remember that a DE framework is not so much an evaluation in itself, but rather an organizing system for data analysis being done in an emergent / iterative fashion.  

• Start by asking a question: “What are we ultimately trying to do? 
• Once you have established the basic question, you continue to ask other questions, then analyze the answers.  
• These questions may vary depending on the context, but generally fall into the following categories: 
• What? What do we see? What does data tell us? What are the indicators of change or stability? What cues can we capture to see changing patterns as they emerge? 
• So What? So, what sense can we make of emerging data? What does it mean to us in this moment, and in the future? What effect are current changes likely to have on us, our clients, our extended network and our field of inquiry and action?
• Now What? What are our options? What are our resources? When and how can we act – individually or collectively – to optimize opportunities in this moment and the next?
• The DE never really finishes until a project finishes. However, by breaking a project into bite-size pieces, and doing DEs for each piece, you can identify decision points, tweak as you go, identify where good and bad decisions were made, and fix accordingly.

For more information on Developmental Evaluation, check out this DE backgrounder, created by Dr. Paula Romanow.