News: Data, Research & Evaluation

Study: Personalized, Data-driven, Text-Based Interventions Improve Student Outcomes

Monday, November 4, 2019  
Share |

By Janai Raphael, Graduate Assistance for Research and Data Analysis

Within recent years, text message interventions have emerged as popular and cost-effective tools to increase college enrollment and persistence rates by nudging students to complete key college-going milestones. These interventions often vary along several dimensions including customization levels, methods of delivery, and topics covered.

Not coincidentally, studies show these text-based interventions producing a mixed bag of results, leaving many to question their impact. However, a recent study lends evidence to positive results.

In a new study published by the American Educational Research Association, researchers launched a text-based FAFSA campaign across 66 public high schools, which together enrolled more than 17,000 senior students in eight school districts in Houston and Austin, Texas. Of the 66 high schools, researchers randomly selected 39 to be in the treatment group.

Using OneLogos Education Solutions, students received automated text messages regarding the importance of FAFSA, links to resources to help complete the FAFSA, and key state and institutional filing deadlines during the second half of their senior year. Students were allowed to respond to the text messages and received direct responses from their assigned school counselors. This campaign took a more personalized and data-driven approach by providing students individualized updates on their FAFSA submission and completion status biweekly. It further leveraged students’ existing relationships with their counselors by encouraging students to schedule on-one-one advising appointments with their counselors, instead of an external organization, for additional assistance.

Students were separated into four different groups and received different messages based on their FAFSA filing status:

  • Students who had not yet begun the FAFSA received links to helpful FAFSA resources and were encouraged to schedule an appointment for one-on-one assistance with the counselor or at a local FAFSA completion event.
  • Students who submitted but did not complete the FAFSA were congratulated for the progress, reminded of their filing status, and invited to request additional assistance.
  • Students who successfully completed their FAFSA were also congratulated and received information on next steps, such as reviewing their Student Aid Report. These students were also made aware of the possibility of being flagged for verification once their FAFSA processing was complete.
  • Students who were flagged for verification were in the fourth group, and these students received more information on the verification process and how to request additional help from their counselors or at a local FAFSA completion event.

Students’ filing statuses were updated as the district received new data every one to two weeks. This intervention reached a total of 7,500 students and cost approximately $8 per student.

At the end of the intervention in May, the researchers found that FAFSA submission and completion rates were 6 percentage points higher for students in the treatment group. By the end of the summer, the campaign’s impact dropped to a statistically insignificant 3 to 4 percentage points. These results suggest that the intervention influenced both students' overall FAFSA filing and filing timing. However, the researchers did find overall gains of 3% in seamless college enrollment, with declines in two-year and gains in four-year enrollment.

Unfortunately, results were not as positive for students who were flagged for verification. Between the control and treatment groups, seamless enrollment was 5 percentage points lower for these students overall. Verification effects were 2.2 percentage points smaller within treatment schools.

A study released in August found both a national and state-level text-based campaign had no impact on the college enrollment or financial aid receipt rates of the 800,000 students included in the study. This study used a multimodal approach (text messages, email, and postal mail) to send students one-way messages focused solely on FAFSA completion and offered one-on-one advising through an external college access program. Only 11.6% of students even responded to the messages.

That report made some hypotheses about its null finding. First, “local partners may know something important about their students and such students may react differently to messages from partners they feel are specifically invested in them or their communities.” Second, “The global scale-up in this study implied messaging content was more generic and less personalized to students than in prior interventions, perhaps resulting in lower salience for students.” That the recent AERA study made use of both counselors students had a connection to and personalized text messages may explain why it had an impact on students’ outcomes.

As more schools, districts, and college access programs continue to explore innovative ways to better serve students, this report demonstrates that personalized, data-driven, text-based interventions can be a practical, cost-effective tool to increase college access for students. There have been many studies about text message interventions in the past seven or so years. Although their findings sometimes conflict, their methods and implementation do as well. What is emerging is the importance of using text messaging to augment and expand the provision of college advising services from trusted advisers and adults rather than replacing them completely with text messaging alone.