Sign In | Join NCAN
Data and Evaluation Toolkit

Recognizing the importance of using data to examine and improve program practice and outcomes, NCAN has produced or assembled a number of data- and evaluation-related resources for our members. Many, but not all, of these resources related to NCAN’s Common Measures, a member-developed and research-backed set of college access and success indicators. These resources are listed below with short descriptions.

Please send questions, concerns, or feedback to Bill DeBaun, director of data and evaluation, at debaunb@ncan.org.

(Photo by Allison Shelley for EDUimages)

Common Measures Quick References

Two short documents listing all of the Common Measures and the abbreviated citations to the research related to them. One covers NCAN’s "Access" and "Success" Common Measures while another covers a set of Middle School Indicators. To view the full citations for each of the Common Measures, consult the Common Measures Handbook. (Last updated February 2018).

 

Common Measures Handbook

The Common Measures Handbook is a lengthy reference that examines each of the Common Measures indicators and suggests the form in which its related data are stored, data sources, technical and tracking notes, and related research. This document will evolve as new suggestions on tracking and collection are offered by NCAN members and as new research relevant to these measures enters the field. (Last updated February 2018).

 

Driving Toward Program Improvement: Principles and Practices for Getting Started with Data

The first in a two-part series from NCAN and Exponent Partners. If you’re just getting started, dive into part one, "Driving Toward Program Improvement: Principles and Practices for Getting Started With Data". Here you’ll find best practices for building a data-driven mindset, refining your logic model, adopting data management practices, building your capacity, and more.

 

Roadmap for Tracking Your Student Results: Program Data and Systems

The second in a two-part series from NCAN and Exponent Partners. If you’re moving towards a more sophisticated approach, take a look at part two, "Roadmap for Tracking Your Student Results: Program Data & Systems." You’ll get a review of some foundational steps, frameworks for data management with a system, analysis methods, and more.

 

Data Usage and Platforms for College Access and Success: Insight from the Field (2014)

Curious about how other programs use data? Confused about which data platform might be a good fit for your organization? "Data Usage and Platforms for College Access and Success: Insight from the Field" seeks to help you answer both of those questions. Written as a product of the Common Measures Learning Community, this white paper includes data from a survey of NCAN members and profiles and testimonials of five different data platforms being used by college access and success programs nationally. Future editions of this paper will consider even more data platforms in the field.

Idea Incubator Series

In the summer and fall of 2016, NCAN conducted a series of "Idea Incubators" across the country. These gatherings of four to seven members focused on brainstorming around key questions in the college access and success field. Readers should consider these a hybrid between a white paper, a resource guide, and meeting minutes on an important topic.

NCAN Benchmarking Spotlights

The Benchmarking Project is an important collaboration between NCAN, our members, and the National Student Clearinghouse that examines college access and success programs’ collective success in helping students to enroll in and complete at postsecondary institutions. To date, the project has found evidence that students served by NCAN members enroll at rates exceeding those of students from low-income high schools and more closely resembling those from higher-income high schools. The Benchmarking Reports have also found that member-served students complete at rates exceeding their low-income peers and approaching the national average.

In addition to these important quantitative findings, the Benchmarking Report series also assists with identifying high-performing programs from which other NCAN members may be able to learn important lessons and strategies. Ann Coles, senior fellow at uAspire, conducted a series of case studies profiling some of these programs, which included extensive interviews with program staff, leadership, partners, and students. She also reviewed key program documents and processes.

 

Program Evaluation Primer

The phrase “program evaluation” is often viewed suspiciously or defensively, but the process of program evaluation can be extremely helpful for understanding what a program is successful at, where it could improve, and where its activities can and cannot be measured. This Program Evaluation Primer, written in FAQ style, is for those trying to learn more about what a program evaluation is and isn’t and whether and when a program evaluation might be right for them.

 

Student Privacy Resources

The privacy and security of student data are both critically important. The U.S. Department of Education’s Privacy Technical Assistance Center (PTAC) is a wealth of information around practices and policies that help organizations stay compliant with the law and keep student data safe. The PTAC Toolkit is organized around five important topics: security best practices, data governance, data sharing/dissemination, legal references (FERPA and cross-agency), and disclosure avoidance. Beyond the toolkit, PTAC also operates a Help Desk that programs can contact with questions or concerns on student privacy.

StriveTogether also has a very handy guide to student privacy titled "Student Data Privacy Best Practices: Five Ways Community Organizations Can Ensure Effective and Responsible Data Use." If your organization uses student data, this is an important refresher in maintaining that data's security.

 

Making Wise Decisions: A Step-by-Step Guide to Selecting the Right Data System

A valuable resource from Public Profit and B3 Consults, "Making Wise Decisions: A Step-by-Step Guide to Selecting the Right Data System" helps programs tackle one of the most frequent data questions in our field: how am I tracking and reporting data? Use the guide's self-assessment, toolkit, and templates to identify program needs and the processes and types of platforms most likely to meet those needs.

 

Other Data-Related Resources

  • Logic models are important tools that organizations can use to show how the elements of its program—its resources, activities, outcomes, etc. —come together. Creating a logic model may be a required part of program evaluation, but it’s also a great way to simply explain your program to prospective donors. This Logic Model Development Guide from the W.K. Kellogg Foundation is an excellent, detailed resource for logic model newbies and devotees alike.

  • “Data sharing agreement” is a phrase that is likely to cause a wide range of emotions in program staff; apprehension, bewilderment, and anxiousness are just some of them. The process of setting up an agreement between two parties, be they schools, districts, or non-profit entities is one that can be confusing and complicated. Fortunately, the William T. Grant Foundation’s Research-Practice Partnerships series has an entire module devoted to Developing Data Sharing Agreements, which includes guiding questions, work samples, and other related documents. The rest of the series is also worth a look, but this is the module with the most utility for NCAN members.

  • StriveTogether recently released a very handy Data Sharing Playbook that dovetails nicely with the above resource. The playbook is intended to “help community organizations effectively partner with schools on data-driven ways to improve education outcomes. This resource includes seven principles about how to begin and grow a data-driven initiative, as well as practical resources to help communities implement complex data partnerships with schools and other community partners.”

  • Organizations talk all the time about how they are data-driven, or at least want to be. But organizations do not become data-driven overnight. It’s a gradual process, but where does that process start? “Getting Started with Data-Driven Decision Making: A Workbook,” from the Nonprofit Technology Network and Idealware, asks just this question. This document is free (just put in your contact info) and well worth your time if your organization is search for square one to becoming data-driven. This guide goes step-by-step to identify questions your organization wants answered, the metrics that could answer them, who would use those metrics, how they would be collected and tracked, and more. This would make for both a great staff retreat activity and a reminder of all the steps to consider when expanding data tracking.

  • What are the specific skills to operate in the data science environment that finds itself with so much available data? The team at Leada published “The Data Analytics Handbook” to answer some of these questions. There are four volumes: data analysts and data scientists, CEOs and managers, researchers and academics, and “big data edition.” Using interviews from professionals involved in data-using top-tier organizations, the handbook attempts to get at what each of these groups should know, and ask, to get the most out of the information they have.

  • The Center for Education Policy Research at Harvard University hosts the Strategic Data Project, which pairs data analysts with school districts to perform high-impact data analyses. The SDP makes available its Toolkit for Effective Data Use. The toolkit is “a resource guide for education agency analysts who collect and analyze data on student achievement. Completing the toolkit produces a set of basic, yet essential, human capital and college-going analyses that every education agency should have as a foundation to inform strategic management and policy decisions.” Although these tools are aimed at those working in school districts, many of the skills will apply to work done in college access and success organizations. The toolkit includes dummy data sets and step-by-step directions for data cleaning using Stata software. This resource is probably not a light undertaking, but it could give a fledgling analyst some more experience with key data skills. Also of note from the SDP is the  “Strategic Use of Data Rubric,” which can be used “as a basis for gathering evidence of data use across the organization allows educational leaders to identify specific areas for improvement and highlight specific steps to move the organization toward using data more strategically.”

  • If you work for a school district or your program works closely with a school district and you wish you could get a better system of metrics established to see how students, schools, and the district overall are performing, take a look at the  College Readiness Indicator Systems (CRIS) Resource Series. This will take a little time to read through, but it’s worth it. Launched by the Bill & Melinda Gates Foundation, CRIS worked with research and school partners across the country to develop indicator systems around three dimensions of college readiness: academic preparedness, academic tenacity, and college knowledge. The resource series has six parts, but the chart in the link above describes what it included in each part and who is most likely to benefit from reading it.