Estimating College Effectiveness: The 2012 Washington Monthly College Rankings

One of my primary research interests is examining whether and how we can estimate the extent to which colleges are doing a good job educating their students. Estimating institutional effectiveness in K-12 education is difficult, even with the ready availability of standardized tests across multiple grades. But in higher education, we do not even have these imperfect measures across a wide range of institutions.

Students, their families, and policymakers have been left to grasp at straws as they try to make important financial and educational decisions. The most visible set of college performance measures is the wide array of college rankings, with the ubiquitous U.S. News and World Report college rankings being the most influential in the rankings market.  However, these rankings serve much more as a measure of a college’s resources and selectivity instead of whether students actually benefit.

I have been working with Doug Harris, who is now an associate professor of economics at Tulane University, on developing a model for estimating a college’s value-added to students. Unfortunately, due to data limitations, we can only estimate value added with respect to graduation rates—the only outcome available for nearly all colleges. To do this, we estimate a college’s predicted graduation rate given certain student and fixed institutional characteristics and then compare that to the actual graduation rate.

This measure of college value-added gives us an idea of whether a college graduates as many students as would be expected, but this does not address whether colleges are educating students in an efficient manner. To estimate a college’s cost-effectiveness for students and their families, we divide the above value-added measure by the net price of attendance (defined as the cost of attendance minus all grant and scholarship aid). Colleges can then be ranked on this cost-effectiveness measure, which yields much different results than the U.S. News rankings.

This research has attracted attention in policy circles, which has been a great experience for me. I thought that at least some people would be interested in this way of looking at college performance, but I was quite flattered to get an e-mail from Washington Monthly magazine asking if I would be their methodologist for the 2012 college rankings. I have always appreciated their rankings, as the focus is much more on estimating educational effectiveness and public service instead of the quality of the incoming student body.

The 2012 college rankings are now available online and include a somewhat simplified version of our cost-adjusted value-added measure. This measure consists of one-half of the social mobility score, which is one-third of the overall rankings (research and service are the other two components). The college guide includes an article I wrote with Rachel Fishman of the New America Foundation looking at how some colleges provide students with good “bang for the buck.”

I am excited to see some of my research put into practice, and hopefully at least some people will find it useful. I welcome any questions or comments about the rankings and methodology!

Author: Robert

I am an a professor at the University of Tennessee, Knoxville who studies higher education finance, accountability policies and practices, and student financial aid. All opinions expressed here are my own. View all posts by Robert