Johnson, S. R., and F. K. Stage. 2018. “Academic Engagement and Student Success: Do High-Impact Practices Mean Higher Graduation Rates?” Journal of Higher Education 89 (5): 735–781.
SUMMARY
This study explored the association between ten high-impact practices and graduation rates among students at four-year public colleges. Specifically, the authors examined the effect of the following “high-impact” practices on graduation rates: first-year seminars, core curricula, learning communities, writing courses, collaborative assignments, undergraduate research, study abroad, service learning, internships, and senior capstone projects. These practices, often considered critical for student success, have rarely—if ever—been empirically studied using rigorous research designs that enable educators to make substantive claims about their effectiveness.
Extending this argument, the authors designed a study that merged survey data provided by academic officers at 101 institutions with those from the U.S. Department of Education’s Integrated Postsecondary Education Data System (IPEDS). Academic officers chose from four options regarding institutional availability of each of the ten practices: required for all students, required for some students, optional for students, and not offered. The dependent variables for the study included four- and six-year graduation rates reported to IPEDS for the 2013–2014 academic year.
The authors constructed models to test these relationships. They accounted for the information provided by academic officers, a composite measure of high-impact practices on any given campus (that is, does the number of practices offered at any given institution matter?), institutional selectivity, Carnegie Classification, expenditures per student, proportion of students receiving federal financial aid, proportion of students receiving federal loans, and proportion of the student body identified as White.
DISCUSSION OF THE FINDINGS
The most surprising finding was that “8 of the 10 high-impact practices had no significant relationship with either four-year or six-year graduation rates” (p. 775). Only participation in first-year seminars and internships shared significant and negative relationships with graduation rates. These findings held across institutional selectivity, type, and all other institutional conditions included in this study. To explain these “puzzling” (p. 776) and counter-intuitive trends, the authors suggest that institutions may be using most of their resources for these programs to the detriment of spending on “further engagement or guidance practices later in the student’s career” (p. 776) or that institutions have more “rigorous academic expectations, which may lead some students to delay graduation to later years” (p. 776).
In addition, the composite score of high-impact practice also failed to positively predict either four- or six-year graduation rates. This finding indicates that institutions that offer a high number of these practices are also not necessarily helping students complete their degrees in four or six years.
IMPLICATIONS FOR ACTION BY CAMPUS LEADERS
Although this study focuses on public colleges, CIC members may extrapolate many of the study’s findings to their own institutions. What constitutes a high-impact practice? How do we know when a practice is effective? For whom is it effective? Results from this study echo recent sentiments among higher education researchers that it is not the practice that matters, but the educator charged with enacting the practice that matters.
In addition, high-impact practices should be scrutinized to ensure that they effectively advance the institution’s mission and are designed and implemented by effective educators. Simply adding a practice to an institution’s menu for engaging students may be insufficient; CIC administrators should exercise caution in proceeding with any practice, even those historically positioned as high-impact.
About the Authors
Sarah Randall Johnson is associate director of institutional research at Harvard Business School.
Frances King Stage is professor of higher education, leadership, and technology at New York University.
Literature Readers May Wish to Consult
Arum, R., and J. Roksa. 2011. Academically Adrift: Limited Learning on College Campuses. Chicago: University of Chicago Press.
Kuh, G. D., and C. G. Schneider. 2008. High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter. Washington, DC: Association of American Colleges and Universities.
