US News liberal arts rankings vs. teaching rankings

Last week, US News released the annual rankings for liberal arts institutions. It also released a bunch of other rankings, including “Best Undergraduate Teaching.” “Wait,” you may be thinking, “How different can the best liberal arts schools and the best liberal arts schools for undergraduate teaching be?” The answer is, “More different than you would think.”

You may recall that the overall rankings for national liberal arts institutions are strongly correlated with endowments. Apparently, the things that make a school good at teaching in the eyes of US News differ from those that make a school good overall. Interestingly, the methodology for determining best teaching is similar to that for determining the best graduate programs. Namely, they ask people about their perceptions: “College presidents, provosts and admissions deans who participated in the annual U.S. News peer assessment survey were asked to nominate up to 10 schools in their Best Colleges ranking category with a strength in undergraduate teaching.”

Like the graduate school rankings, then, the undergraduate teaching rankings reflect others’ perceptions rather than a formula that schools might try to game. It turns out that, unlike the national liberal arts rankings, these perceptions are not strongly correlated with a school’s endowment (only .226 vs. .78 for the national rankings). Although there are similarities, some schools are rated much differently in the rankings for undergraduate teaching. Hendrix College has the largest difference between its overall ranking (82) and its teaching ranking (29). Other schools that are at least 40 spots higher in the teaching than overall rankings include: Beloit, Wheaton, St. Olaf, Lawrence, Berea, and Wooster.

Of the schools appearing on both lists, Bowdoin looks the worst, with its overall ranking of 4 and its teaching ranking of 29. Many high-ranking schools in the overall rankings, though, don’t appear on the list of the top 30 teaching schools at all. Eleven schools in the top 30 national rankings do not appear in the top teaching rankings, the highest-ranked of which are the US Naval Academy and Claremont McKenna, tied for 9th in the national rankings.

The takeaway from all of this seems to be that a school’s reputation for teaching is not nearly as dependent on financial wealth as its overall rankings. I think that the different methodologies for different rankings are also interesting, since graduate programs are essentially ranked by those in similar programs, who would seem to know best. Undergraduate teaching is ranked in the same way, but US News is not willing to allow these peer-nominated rankings to make up its most publicized rankings like it is for graduate programs.

Of course, both types of rankings are probably connected only tenuously to actual student experiences at various schools, but by publicizing their overall rankings, US News ensures that they will keep schools focused on the small things they can do to try to climb the rankings, while an emphasis on the perceptions of others may allow schools to shift their foci to the bigger picture, considering what is best for students instead of for US News.

—

“Like” Memoirs of a SLACer on Facebook to receive updates and links about gaming the US News system via your news feed.

Read Full Post »

Are the richest schools the “best” schools?

The other day I was having a conversation with a colleague about how much size matters to US News. That is, how closely correlated are US News rankings of national liberal arts schools with the size of their endowments? Since both the rankings and the size of endowments are available on the US News website, I decided to take a look. (Note that I don’t actually have any software for statistical analysis at home, so everything that follows will be very basic.)

The first thing I did was create a list of the top 99 schools (there were several schools at #100, so I decided to go with 99 instead of 104 or whatever it would have been) with their rankings and the size of their endowments and then order the list by endowment size and re-rank the schools in order to determine the difference between US News rank and endowment rank. On average, the US News rankings differed from the endowment rankings by just -.48, meaning that the average school is ranked about a half a position lower than its endowment would suggest.

Considering that raw endowment amounts may not be as important as endowment per student, I did the same analysis again but divided the endowment by the total number of students. The result was the same: -.48. In both cases, though, the standard deviation was large. For raw endowment, the standard deviation was 18.66, while for enrollment-adjusted endowment it was 20.99.

I also looked at the schools with the best and worst rankings relative to their endowments. The winners were the US Air Force Academy (ranked 65 spots above the location that its 90th-ranked endowment would suggest) and the US Military Academy (46 spots above its 63rd-ranked endowment). The losers were Berea College (65 spots below its 11th-ranked endowment) and Agnes Scott College (37 spots below its 52nd-ranked endowment).

Finally, I took a look at the correlation coefficients. Pearson’s r for the raw endowment was .78, while for enrollment-adjusted endowment it was .72. In both cases, there is a strong correlation between endowment and US News rankings. This isn’t particularly surprising, given that a large endowment makes a school more able to do the things that US News values while drawing the types of students that US News values. It does, however, point out that, in many cases, US News rankings don’t provide much information beyond what we could ascertain by glancing at the endowments of two schools.

—

“Like” Memoirs of a SLACer on Facebook to receive updates and links via your news feed.

Read Full Post »