Grantee Plans Identify Obstacles to Changing Postsecondary Advising
Friday, February 15, 2019
Posted by: Bill DeBaun, Director of Data and Evaluation
NCAN continues to work with a number of grantees to help improve their postsecondary outcomes, and we recently gained some insight into how these grantees intend to achieve this goal. Through the To and Through Advising Challenge, NCAN is working with 20 grantees, among which are school districts, charter systems, and community-based organizations, on the development of plans that aim to change the way they do business around postsecondary advising.
The challenge has four aims for grantees:
- Incorporate fit and match into postsecondary advising for students.
- Increase access to financial aid by focusing on FAFSA completion and mitigating the effects of verification.
- Reduce summer melt by providing support to students that ensures they show up on campus in the fall.
- Accomplish each of these through a thoughtful and systemic use of data.
At the end of December, grantees delivered to NCAN and their postsecondary advising and data coaches the first draft of an implementation plan they will use next academic year to accomplish the above goals. We asked that these plans identify strategies, actions, timeframe, responsible personnel, leading and outcome indicators, and challenges/needed resources. The grantees’ coaches, NCAN Director of Technical Assistance MorraLee Keller, and I reviewed these plans and provided feedback to the grantees, and we will see at least two more iterations before the academic year concludes.
An obvious but important note: Systems change is hard. Shifting the often-calcified mindsets, beliefs, and practices of any organization is a challenge, and the grantees showed they are willing to rise to it. That said, there is value to knowing about the challenges and pitfalls others have faced in doing this work, and so below is a list of common insights or challenges across many or most of these plans about which other entities wanting to effect change in their postsecondary advising practices should be mindful.
What makes a good indicator? We asked for leading and outcome indicators in the plan drafts. I think of leading indicators as formative steppingstones along the way to a given outcome. For high school graduation, leading indicators would be credit accumulation, GPA, high levels of attendance, and low levels of disciplinary infractions.
Examples of leading indicators for applying to college are: being in a college preparatory pathway for coursework, taking the PSAT, and taking the ACT/SAT. Incidentally, NCAN’s Common Measures, a set of research-backed, member-developed access and success indicators, is very useful here.
I think of outcome indicators as the big-ticket results that we want students to be achieving. Examples include:
- % of students from a given class matriculating into a match or overmatch institution
- % of students receiving fit/match advising from a guidance counselor
- % of students completing the FAFSA
- % of students who indicated in their senior year that they would matriculate to a postsecondary institution who have a postsecondary enrollment in the following fall/year
A substantial number of the plans we received relied too heavily on vague, qualitative indicators or on indicators that would require the development and dissemination of a specific instrument to measure them. For example, “counselor awareness of XYZ skill” and “student belief in ABC” are logical constructs to want to capture. They can also be difficult to capture and require not only a survey or other kind of instrument to collect but also time to collect, clean, store, and analyze the results.
Moving forward, we will be suggesting grantees try wherever possible to make use of SMART indicators (i.e., specific, measurable, actionable, replicable, timebound).
Early awareness is key, so expand beyond grade 12. Too many grantees omitted the role of early college and financial aid awareness from their plans. The vast majority of proposed actions were in grades 11 and 12. In our comments, we sent grantees with this issue NCAN’s early awareness materials. Our belief is that increased student and family awareness about the path to college and the availability of financial aid through FAFSA can lay a foundation for success in later grades.
A specific example we saw repeatedly is the grade 12 exit survey. This is a common tool designed to capture students’ postsecondary intentions. We proposed that grantees administer this survey earlier than the spring of grade 12 because having that information at the end of grade 10 or beginning of grade 11 leaves room for interventions that can shape students’ postsecondary trajectory. Letting students know that they are on track academically to go to a postsecondary institution (or even a certain kind of postsecondary institution) could serve as a motivator. Using data from the National Student Clearinghouse StudentTracker service, a school or district could even say, “Historically, students with your academic profile have gone to a 4-year public institution 55 percent of the time, a 2-year public institution 25 percent of the time, and no postsecondary institution 10 percent of the time.”
Capacity is key. In some plans, we saw the same stakeholders repeated in the “responsibility” column. It is encouraging to see involved stakeholders here, but a handful of personnel cannot successfully shoulder the load for this work at the district level. That responsibility and buy-in must be shared and diffuse. At the same time, we have seen that there often are not enough personnel focused on postsecondary advising and preparedness. This will be an ongoing challenge for grantees to work through as they see how they can bring more hands into the work while also being mindful of these professionals’ existing responsibilities.
Process, process, process. There are a lot of new processes proposed across these drafts. That is a double-edged sword. On the one hand, it is encouraging to recognize that your current processes are insufficient and that you need new ones to accomplish your goals. On the other hand, one of the common obstacles we have seen in this work is a lack of capacity (in terms of number of personnel) to execute this work. We will be encouraging grantees to learn more about process mapping (a cousin of logic modeling) and providing training on that skill set. We hope a better understanding of process mapping will help grantees understand the specific actions they need, who will be responsible for them, etc. These maps would be ancillary and complementary to the implementation plans themselves. After all, it is one thing to say, “We will track individual FAFSA completion more closely with counselors,” but that leaves significant vagueness for implementation.
Put your data to work. In a similar vein as the above, many of the implementation plans call for a number of new data points, but the source, collection, and management of these is often vague. This is a separate issue from the vagueness of the indicators discussed above. Even where the indicators are appropriate (e.g., specific, measurable, actionable, replicable, timebound), there needs to be more clarity about the creation or maintenance of the process around them. What are you tracking? Who will track it? How will it be collected? Where will it be stored? Who will analyze it? Who will receive the analysis? These are sound questions to ask about any indicator because having these metrics exist in a vacuum will limit how they are used.
Reinventing the wheel. This was not universal, but a number of plans discuss the development of tools, templates, or even platforms around the concepts of fit and match in postsecondary advising. The admonition here is to be sure you have examined the marketplace and reached out to partners to see what they are using before rolling up your sleeves and starting from scratch. There is a wide variety of tools and templates out there for postsecondary advising and college fit and match, and these could render grantees’ creation of these tools unnecessary.
Building and maintaining healthy partnerships. We are fortunate to have a number of types of grantees in the advising challenge: school districts, charter networks, and community-based organizations. One of the challenge’s missions is to understand obstacles for changing postsecondary advising in a number of different contexts. That said, any context would be improved by strong and thoughtful stakeholder buy-in and mutual support. Too often, the first drafts of the plans we saw did not bring external partners into the work enough or there was not enough coordination between school districts and community-based organizations. This work is a big lift, and we are hopeful that future drafts will avoid planning in parallel without fully involving all of the stakeholders who could be assets for the work.
I mean for the items here to be constructive criticism, not negative nitpicking. There is a lot to learn from these grantees, their experiences, and the plans that they are developing. Given the interest in changing postsecondary advising in many communities across the country, the pitfalls described above can be instructive.
Stay tuned as we continue to share insights from the To and Through Advising Challenge.
(Photo by Alyssa Ledesma on Unsplash)
Read more about the To and Through Advising Challenge: