Thoughts on Tracking Career Success and Labor Outcomes
Thursday, June 29, 2017
Posted by: Bill DeBaun, Director of Data and Evaluation
NCAN, through the support of Strada Education Network, is helping members understand how to incorporate career success programming into college access and success services. In a recently released white paper, NCAN shared lessons learned from this year’s career success-focused Spring Training series. That paper concentrates on three areas: securing career success data, adapting advising, and building partnerships. These areas are all important for members expanding into, or growing, their work in career success. But there is another important aspect to consider: how to track data and measure outcomes.
NCAN’s Common Measures are a set of research-backed and member-developed access and success indicators; the outcomes listed are important milestones on the way to college enrollment and completion. NCAN continues to scan the literature for research that points to career success-related milestones and outcomes and talk to NCAN members about which metrics they track. Eventually, NCAN may add career success metrics to the Common Measures. Until then, this blog post considers some metrics in this area that member programs may want to consider.
The Common Measures focus on outputs and outcomes metrics (as opposed to inputs). These terms come from logic modeling, a process that NCAN recommends all programs undertake. The distinction between these two can be hard to tease out, but generally, readers should consider outputs to be the direct result of their activities. For example, if one of the activities a program engages in is SAT preparation services for students, then some of its outputs might be the percentage of students who take the SAT and the average SAT score of students in the program. Outcomes are bigger-picture results or achievements tied to an organization’s mission. For example, a typical college access and success program’s outcomes will be the percentage of students it serves who enroll in college and then go on to complete.
Programs aiming to track their career success efforts may find tracking inputs, activities, outputs, and outcomes all to be useful. Many member programs track their inputs and program activities. They want to know, for example, how many times a student met with an advisor, how many students and parents attended financial aid nights, and how many times an advisor visited a given high school. These types of metrics can all be valuable, but inputs should not be the entirety of a program’s database because they lack the “so what?” or return on investment of time and resources that outputs and outcomes demonstrate. Programs engaged in career success work may find it helpful to track their inputs and activities in this area. For example, knowing whether a student has:
- received interview preparation or engaged in mock interviews;
- had advising around resume and cover letter writing or have submitted these for review;
- completed a career success or skill inventory; or
- been assigned a mentor or engaged in job shadowing
all are helpful building blocks to comprehend.
Inputs are also valuable because through statistical analysis programs can connect these to outputs and outcomes later on to see which had the greatest effect in achieving the intended result. Note that college access and success inputs can be combined with career success inputs in such an analysis. Programs could look at, for example, whether the receipt of SAT/ACT preparation and mock interview training contributed to students' eventual enrollment, persistence, and completion.
Now we consider some career success-related outputs and outcomes. Some outputs programs may want to consider are:
- Did the student apply to a job, internship, or fellowship?
- Does the student have professional references he or she can offer to employers?
- Did the student earn an internship or fellowship?
- Did the student enter postsecondary with a declared major? (Note that this is already in the Common Measures, although rigorous research to support this milestone’s impact has not been identified).
- Is the student on track to complete the intended major on-time?
Again, these are results stemming from the above activities. They are stepping-stones on the way to outcomes. The career success outcomes a program selects will (or should) be tied to a mission and reflect the organization’s end goal in this area. Below are some suggestions for career success outcomes that come either from NCAN’s research into this area or as suggestions pulled from NCAN members’ experiences or annual reports:
- Median earnings 10 years after first enrolling (compare to the U.S. Department of Education’s College Scorecard)
- Percentage of students earning above a high school graduate (compare to the U.S. Department of Education’s College Scorecard)
- Percentage of students with a job requiring a degree or credential within 6 months/1 year of graduating. For programs really focused on degree planning and career inventories, this outcome could easily be “percentage of students with a job requiring their degree or credential (according to the job listing) within 6 months/1 year of graduating”
- Average student starting salary (this could be difficult or sensitive to collect)
- Percentage of students working full-time in a job with health benefits
- Percentage of students obtaining professional-track jobs or graduate school admission within six months of college graduation (via NCAN member The Opportunity Network)
- Percentage of program alumni who “are in career-ladder jobs or graduate school within 12 months of college graduation” (via NCAN member Students Rising Above)
- Earnings increases over high school graduates (via NCAN member On Point for College)
- This metric is interesting from a top-level, annual-report perspective. One way to calculate it would be to take the national lifetime earnings figure from a bachelor’s- or associate-degree holder or certificate recipient and subtract the national lifetime earnings figure for a high school graduate. Then multiply the number of program alumni who earned a bachelor’s, associate, or certificate by the corresponding lifetime earnings increase. This is a rough estimate of the value of increased lifetime earnings of students served by the program.
These are just some suggestions for NCAN members looking at tracking programmatic labor outcomes. As always, tracking and analyzing data can help programs to both improve their effectiveness and scale their capacity. This is especially true when the area of work is fledgling and could use feedback to change decisions. If your program tracks these data differently or would like to contribute to this post, please email Bill DeBaun, Director of Data and Evaluation.