News: Data, Research & Evaluation

Digging Deeper Into Using the Impact Genome Project®

Thursday, November 21, 2019  
Posted by: Bill DeBaun, Director of Data and Evaluation
Share |

In August and September, more than 40 NCAN members participated in the Impact Survey Project, a collaboration between NCAN and the Impact Genome Project® that is made possible by the Nielsen Foundation. A recent webinar revealed the results of this effort, and a forthcoming NCAN blog post will detail those results more fully.

Developed by Mission Measurement, the Impact Genome Project® (IGP) is a publicly-funded initiative to curate the world’s evidence base, create standardized measures, and analyze benchmarks. The Genome is designed to help organizations improve the way outcomes are measured and reported in the most critical social impact areas. By creating a standard terminology, we can systematically compare programs and identify the most effective program activities (aka “genes”) for a given beneficiary and context. This data is then integrated into the Impact Genome platform which is used by social sector professionals to measure, benchmark, and forecast the impact of social programs.

Ultimately, the Impact Survey Project will help these programs (and the field) better understand their program outcomes and effective practices. It also greatly expands the data and evidence related to program-level activities and outcomes to grow the Impact Genome Project’s extensive knowledge base. Earlier this year, NCAN introduced our blog readers to the Impact Genome Project. In today’s blog post, NCAN will dive deeper into the IGP.

More Background On the IGP

The Impact Genome Project was inspired by the Music Genome Project and the Human Genome Project. The goal of the IGP is to uncover the “DNA” or components of social impact programs and understand which of these components are most effective. The “DNA” is a common language that we can use to better understand what social impact programs are trying to achieve and how they do it. What does that mean in practice? The IGP is comprised of three things: taxonomies, evidence, and benchmarks.

Interested readers can explore the Impact Genome website where they’ll find tools to benchmark their program’s impact and discover activity genes. Readers can even register through the Login button to take the full Impact Genome Survey for free. This will produce a full program scorecard with data readers can use to understand and communicate their program’s impact.

The College and Career Readiness "Genome"

Within the college and career readiness “genome,” there are four outcomes of interest. Here’s what they are and how they’re defined. (Member programs would do well to see if these definitions are in line with their own):

  • High school completion: “The attainment of a high school diploma, GED, or other high school equivalency or demonstration of on-track status in ninth-grade.”
  •  College access and readiness: “The attainment of knowledge and skills (both cognitive and non-cognitive) that prepare students to be ready for success in college (e.g. college options, college finances, college completion).”
  •  College persistence and completion: “The attainment of the academic success, financial resources, and intent necessary to complete a college degree (e.g., maintained a 2.5 GPA, acquired financing to pay for courses, etc.).”
  •  Career access and readiness: “The attainment of the skills, knowledge, and experience necessary to identify and acquire a sustainable, living wage position (e.g., marketable skills, skills to find a job, knowledge of job placement services, certifications, education, internship experience, etc.).”

Within each of these outcomes, the IGP also identifies “efficacy genes,” which are “the activities that programs utilize to produce outcomes.” These activities have been selected “based on the high frequency and high emphasis with which they appear in [the IGP] database.” The top efficacy genes for the four outcomes above are:

High school completion:

  • Provide Youth Development Interventions
  • Provide Practical Support/Orientation On Behalf Of College Attendance
  • Provide Constructive Response To Student Behavior

College access and readiness:

  • Provide Academic Opportunities On Behalf Of College Prep
  • Provide Practical Support/Orientation On Behalf Of College Attendance
  • Provide Assistance With College Selection

College persistence and completion:

  • Cultivate Positive Self-Awareness Skills
  • Cultivate Teamwork/Team-Building Skills
  • Foster Community And Cultural Engagement

Career access and readiness:

  • Provide Career Support/Advice
  • Cultivate Practical Business/Work Skills
  • Integrate Academic Work With Career Opportunities

If a lot of these look familiar, that is probably not surprising, but it is encouraging and demonstrates some alignment with IGP’s knowledge base. (Or potentially that the IGP’s knowledge base aligns with readers’ programs!)

Digging deeper into each outcome, the IGP also provides a benchmark on “Cost Per Outcome” (CPO). The CPO answers the question “what does it cost to produce a unit of social change?” This factors in the program’s self-reported effectiveness (a.k.a. the percent of beneficiaries served by the program who achieve the primary outcome), reach (the number of beneficiaries served by the program), and budget.

For example, for the college access and readiness field, the cost per outcome ranges between $1,851 and $2,590 among all nonprofit programs in the knowledge base.

Next Steps

Readers can explore the IGP evidence base, which consists of hundreds of journal articles, research reports, and other artifacts that have all been carefully coded by the IGP team. Think of it as an incredibly robust literature review that can be narrowed down and sorted.

The NCAN member programs participating in the Impact Survey Project had a wide variety of models and scopes. NCAN saw all four of these outcomes appear in the portfolio analysis that is the big output of this work. That portfolio analysis looks at which activities (“genes”) NCAN member programs are employing most often and how that compares to the IGP’s existing knowledge base. NCAN will also get a sense of what members’ estimated costs per outcome are and compare those to the IGP database.

All of this will give NCAN, members, and the field more insight into what members do and how they do it.