College Reputation Rankings Go Global

College rankings are not a phenomenon which is limited to the United States. Shanghai Jiao Tong University has ranked research universities for the past decade, and the well-known Times Higher Education rankings have been around for several years. While the Shanghai rankings tend to focus on metrics such as citations and research funding, THE has compiled a reputational ranking of universities around the world. Reputational rankings are a concern in U.S.-only rankings, but extending them to a global scale makes little sense to me.

Thomson Reuters (the group behind the THE rankings) makes a great fuss about the sound methodology of the reputational rankings, which they to their credit acknowledge is a subjective measure. They collected 16,639 responses from academics around the world, with some demographic information available here. But they fail to provide any information about the sampling frame, a devastating omission. The researchers behind the rankings do note that the initial sample was constructed to be broadly representative of global academics, but we know nothing about the response rate or whether the final sample was representative. In my mind, that omission disqualifies the rankings from further consideration. But I’ll push on and analyze the content of the reputational rankings.

The reputational rankings are a combination of separate ratings for teaching and research quality. I really don’t have serious concerns about the research component of the ranking, as the survey asks about research quality of given institutions within the academic’s discipline. Researchers who stay on top of their field should be able to reasonably identify universities with top research departments. I have much less confidence in the teaching portion of the rankings, as someone needs to observe classes in a given department to have any idea of teaching effectiveness. Yet I would be surprised if teaching and research evaluations were not strongly correlated.

The University of Wisconsin-Madison ranks 30th on the global reputation scale, which a slightly higher score for research than teaching. (And according to the map, the university has been relocated to the greater Marshfield area.) That has not stopped Kris Olds, a UW-Madison faculty member, from leveling a devastating critique of the idea of global rankings—or the UW-Madison press office from putting out a favorable release on the news.

I have mixed emotions on this particular set of rankings; the research measure is probably capturing research productivity well, but the teaching measure is likely lousy. However, without more information about the response rate to the THE survey, I cannot view these rankings as being valid.

Author: Robert

I am an assistant professor of higher education at Seton Hall University. All opinions are my own.

2 thoughts on “College Reputation Rankings Go Global”

  1. Academics who grade for a living seem to have a real sensitivity to being graded themselves. We can all complain about how the measurements were collected and even some individual rankings but they seem to yield results most find inline with their expectations. And that’s not a bad thing as the quality of any university is mostly based on some amorphous perception of the overall quality and reputation. And that gets carried into a self-reinforcing cycle so that perception=reality over time. I don[‘t see UW rushing out to hire new faculty out of Kent State or Central Florida. I’d bet most new hires come from a tight range of schools. And the higher up the college rankings you go the tighter that range becomes.
    Did UCLA suffer a major loss of faculty or did the student quality decline at all recently (actually just the opposite) so a small move is nothing to worry about. I know the UCLA med school just received a major donation that will keep it well-heeled for a generation or more. As they say-it takes decades to build a good reputation and in academe it also takes years to lose one too. .

    1. Thanks for the comment. I have no problem with the research reputation rankings (assuming the sample selection issues aren’t a concern), as they limited ratings to a researcher’s field of interest. My concern is with teaching quality, as that is difficult to rate without knowledge of the teaching process. (It’s easier to observe research than teaching from afar.)

      I agree that academics do have an extreme aversion to being graded themselves; for example, see the pushback against student evaluations. A fair amount of my work is in the area of college rankings, and I have no problem ranking colleges. But I do want the measures to be more justifiable than THE’s subjective teaching ratings.

Comments are closed.