Evaluation of Austin Community College’s Strengthening Institutions Program Grant: Annual Outcomes and Impacts Report

Evaluation of Austin Community College’s Strengthening Institutions Program Grant: Annual Outcomes and Impacts Report
Authors: Ashweeta Patnaik and Greg Cumpton
Date: August 2020
Publication Type: Report, 38pp.

Austin Community College (ACC) received a $1.7 million Strengthening Institutions Program (SIP) grant from the U.S. Department of Education (DOE) in 2015 to develop programs to help students understand smart money management and college financing.  ACC hoped to demonstrate that grant-funded activities were linked to improvements in measures of student success such as retention rates, credential attainment rates, and cohort loan default rates.  ACC partnered with the Ray Marshall Center (RMC) to perform an implementation evaluation as well as an impact evaluation of the effectiveness of SIP grant activities.

This annual report summarizes findings from the impact evaluation, focusing on ACC’s financial aid text messaging interventions for first-time in college (FTIC) credential seeking students.  The impact analyses used a quasi-experimental approach with propensity score matching.  Researchers found that texting interventions had a significant large positive impact on fall-to-fall retention – students who received the text messages were 10 percentage points more likely to return to ACC in the fall of their second year of college, compared to students who did not receive the text messaging intervention.  Researchers also found a significant positive impact on fall-to-spring retention: students who received the text messages in fall were 5 percentage points more likely to return to ACC the following spring, compared to students who did not receive the text message.  Disaggregated analyses also found small but significant positive impacts on first-semester GPA, first-year GPA, and FAFSA refiling rates for certain sub-groups.

These findings contribute to a growing body of research that suggests that low-cost, technological solutions such as text-based outreach can support students in overcoming barriers that hinder college enrollment, retention, and completion, particularly when such interventions are: personalized and tailored to students; delivered by a trusted source; and, informed by behavioral science best practices like nudging, message framing, and logistical guidance.  Such interventions may particularly benefit community college students for whom the ability to continue in college likely depends on being able to maintain financial aid.