Forbes has just released its annual ranking of America's Best Colleges for 2010 (online at www.forbes.com/colleges and in hard copy in the August 30, 2010 issue of Forbes now on newstands).  I admit that I got it hot off the presses and spent way too much time online today playing around with the interactive tools.  Of course, I am a college counseling professional and a bit of a data nerd.  So I love college rankings of all kinds and I feel like I can make good use of them.  But here's my warning to all you prospective applicants out there:  you've got to read the fine print and do some ranking of your own if you are going to compile a list of colleges that are right for you.

What's the fine print?  The methodology and the data sets used to compile the rankings.  All the rankings publish them, so read them and think critically about them.  You can get the description of the Forbes methodology and data online by clicking here.

What does the fine print tell me about the Forbes list of so-called "Best Colleges?"

In terms of basic methodology, I'm intrigued by and generally agree that four of the criteria they use to compile their rankings are important to consider:  student satisfaction, postgraduate success, student debt, 4 year graduation rate.  But I'm baffled by the fifth criterion they use:  what are they thinking by including national competitive awards given to students and making it almost 1/10th of the weighted score given to a college?  That just seems wacky and irrelevant to me.

In terms of data, I think Forbes falls down on the job here.  The Student Satisfaction data is particularly suspect because of the reliance on data from RateMyProfessors.com and MyPlan.com.  This data is a completely uncontrolled sample set and will be skewed by the differing participation rates at the different colleges.   For a college with a low participation rate, a few disgruntled students active on one of these sites could ruin a college's ranking.  The Postgraduate Success data is also flawed.  Perhaps not surprisingly, Forbes has considered data relevant only to one type of success — namely business/corporate success.  In the Forbes ranking, success is measured by your pay grade, whether you bothered to fill out a Who's Who listing, and if you have risen to the level of corporate officer.  Come on.  Success is much broader than that.  At a bare minimum, I think Forbes would bother to include the some data that measures whether a college has done its job of preparing you to get a job or get into graduate or professional school.  Data about the number of graduates who are employed following graduation and/or enrolled in graduate or professional degree programs is readily available and could have been incorporated.

My bottom line review of the Forbes rankings?  Pretty good methodology, pretty bad data, so not very helpful or illuminating in really revealing the "best colleges."  But if one of your primary criteria for a "best college" is the cost of education, including typical student debt, then you can get some good comparative data here.  The hard copy includes this information on the chart listing the rankings and it is available online only through the interactive tool called "Screener" found as a button about two-thirds of the way down the middle column on the landing page for the rankings article.