Are the Rankings Really So Bad?

When U.S. News & World Report released its annual rankings of America's Best Colleges on August 22, I must confess that I went straight to its website to see the 2009 list. Is Princeton still #1, or did Harvard reclaim its throne? Is Berkeley still ahead of Notre Dame by a nose? And who's the top dog of liberal arts colleges THIS year? I'm a college admission geek, and I'm instinctively, addictively, drawn to these kinds of data. But it's not just nerds like me who go there. According to an August 28 foliomag.com article by Matt Kinsman, revenue for this story alone shot up 500 percent from the magazine's usual stream, reaching seven figures for its online site. Overall ratings traffic is up 30% from last year, and according to company president Bill Holiber its print edition is enjoying its best year ever in terms of college-related advertising. Clearly, college rankings are big business.

The wave of interest in the rankings has intensified despite an outcry from some educators and college admission professionals that they are inaccurate at best, and may actually be harmful and misleading. An important counter-trend to their growing popularity is the steadily declining participation by the colleges themselves in providing their data to USN&WR. In an August 22 Inside Higher Ed article entitled "U.S. News Sees Drop in Participation," Scott Jaschik writes that college participation in the reputational survey called "peer assessment"--the single largest factor in the rankings methodology--has dropped to 46%, the fourth consecutive year it has declined. This may be the result of a concerted effort by the Education Conservancy, an anti-ratings group which has encouraged college presidents to sign a letter pledging not to fill out the survey or use the survey's results in their own outreach materials. The battle is heating up, and the stakes have never been higher.

Before becoming a professional admissions consultant, I found myself frequenting these ratings sites, hungry for information and secretly wanting to know if my knowledge (or memory) of a school's reputation corresponded with reality, with some set of objective criteria. I wasn't foolish enough to believe, for example, that 24th-ranked Colby College of Waterville, Maine is a better school than 25th-ranked Bates College of Lewiston, Maine simply because it's one notch higher on the list. But how DOES Barnard cmpare with Wellesley and Bryn Mawr, I wondered. And as a California resident with school-age children, I was curious about the UC's, how much "better" Berkeley was than UCLA, and how far down the list UC San Diego and UC Irvine actually were. So what's the harm, right?

I've come to believe there is plenty of harm, and that it's time for college admission counselors to take aim between the eyes of these rankings systems which promote themselves as sources of useful information for students and parents. "With so many students and families relying on the college rankings for their higher education research," said USN&WR Editor Brian Kelly last week on the release of its new rankings, "we're always working to find new ways to demystify the college search."

Demystify? Imagine that! Well let's start by demystifying the USN&WR rankings itself.


Flawed methodology

Deconstructing its parts:

Peer assessment (25%) - accounting for a quarter of a school's "grade," this is the single largest category. Presidents and deans are asked to evaluate peer institutions' "academic excellence." It is therefore a measure of a school's reputation only. Those on top tend to stay on top; moving up through the ranks can take years.

Retention (20%) - the measure of the percentage of freshmen who will graduate in six years, and secondarily the percentage of freshmen who will return for their sophomore year. While a high retention rate can certainly indicate strong administrative support within a campus, and possibly high student satisfaction, this metric tends to penalize those schools who have a greater number of working class students who may not be able to remain in school through graduation because of financial pressures. Those schools who tend to attract the wealthiest students will continue to score highest in this category.

Faculty resources (20%) - a measure of average class size (more points for smaller classes), faculty salaries, percentage of faculty who have the highest degree offered in their field, and percentage of full-time faculty. Important factors to be sure, but the argument against this says that it misses the less quantifiable factors of quality of instruction within the classroom and accessibility of professors outside of class.

Student selectivity (15%) - includes the SAT/ACT scores of incoming freshmen, their class rank, and the overall acceptance rate of the incoming class. In the broadest terms, selectivity may correspond with desirability. But this metric penalizes those lesser known schools, or those schools in less "desirable" locales, who may demonstrate the highest level of academic excellence but who suffer from less successful marketing and can't keep pace with the heavyweights in terms of number of applicants.

Financial resources (10%) - a measure of per-student spending in educational expenditures. No argument here. Big endowments are wonderful to have, especially if they are being re-invested in worthwhile line items.

Graduation rate performance (5%) - a calculation of whether the actual graduation rate of freshmen exceeds or falls short of the predicted graduation rate. Exceeding means the school is probably doing something right, although once again this tends to reward schools with wealthier populations.

Alumni giving rate (5%) - the number of alumni contributing to the school is considered a measure of student satisfaction. I agree, although the strength (and operating budget) of a school's alumni committee may be the determining factor here.


Can you rank excellence?

It would be whiny of me to nitpick these metrics and to pretend that they don't give us some useful information about a school. In my consulting practice, I go to these websites to mine lots of data in order to determine how an applicant compares statistically with a school's freshman profile. My biggest objection is to the ordinal rankings--the absurd notion that you can actually rank from 1 to 130 a school's level of excellence! Compounding the problem, this methodology rewards those institutions already sitting at the top because they are perceived to be more desirable. This heats up the competition for the same small handful of elites, which further rewards them in the following year's stats.

Most importantly, it focuses families' attention on the wrong things, on a grade which is largely based on subjective and self-reported data, rather than on the type of learning which takes place at the school, its particular areas of strength, the community of people it attracts and whether that will be a comfortable peer group for the applicant, and on how good a job the school does in teaching its students to think, write, communicate, solve problems, explore, discover, and prepare for their lives as adults.

In the book Cool Colleges for the Hyper-Intelligent, Self-Directed, Late Blooming, and Just Plain Different, author and consultant Donald Asher makes a great point when he says that these metrics "measure the quality of students entering the college, not the quality of students exiting the college....No national survey has taken this approach, but it would sure be interesting to see which colleges would win this ranking query." He's talking about student satisfaction surveys, about the success rate of undergrads getting into graduate schools, about how many go on to get Ph.D's, about success at job placement, about average salary offers and average lifetime income, and about alumni tithing back to the school. These are far more worthwhile measures of educational excellence than the average SAT scores of incoming freshmen!

The heart and soul of the college search process which I embrace is about matching the applicant with a community of learning in which he or she will thrive, have the greatest chance of success, and be happiest and most challenged. It is a match to be made, not a prize to be won. At the top of the USN&WR rankings are a small number of schools which are nearly impossible for any student, no matter how accomplished, to get into. But further down the lists are hundreds more which are excellent in their own ways, schools which you may never have heard of. These schools differ from each other in profound ways. Some have a required and challenging core curriculum, and some have no requirements at all. Some offer an approach of hands-on learning and career training, while others offer the classics and a more cloistered life of the mind. Some stress research opportunities and lectures by renowned experts in a variety of majors, while others excel at discussion-based learning and close contact with professors and may emphasize an inter-disciplinary approach. These are the core practices your child's college search should uncover, important truths about each school which are completely ignored by the ordinal rankings.

So go ahead, sneak a peak at the rankings. Spend the fifteen bucks for an online subscription so you can have access to all the meat and potatoes the site has to offer. But don't think for a moment that there is any worthwhile method of ranking schools according to academic excellence. The best school is the one that's best for your child, not the one with the back window decal that will impress the neighbors. Anyway, isn't it cooler to be ahead of the curve, rather than three years behind?


Jeff Levy can be reached at jeff.levy1@ca.rr.com

An Introduction to College Match

Welcome to "College Search—A Parent's Point of View," my new blog for families who are going through the process. As a parent of a college freshman and a high school sophomore, and as a practicing college admission consultant, my views are student-centric: this process will lead to success only if the starting point is the student herself. If your question is, "How can I get my daughter into one of the best schools?" you are certainly asking a question that is being asked by thousands of parents like you, but one that nonetheless misses the mark. A more productive question to ask is this: "What are the best schools for my daughter?" These questions may sound similar but in fact they are worlds apart. They represent the difference between a college search that is stressful and anxiety-producing and which possibly leads to a poor choice, and one that is exciting, enriching, full of self-discovery, and is likely to lead to a great choice.

From this tiny corner of the blogosphere in the weeks and months ahead, I hope to address some of the complex and vexing questions which are intrinsic to the process of college search, and hopefully assist families in finding their way through this thorny and uneven terrain to a vantage point from which your child's best options become clear. If your high school junior has gotten B's and C's instead of the A's he is capable of because he has spent too much time in his extracurricular pursuits, there are literally hundreds of terrific schools who would love to have someone with his specific talents as part of their community. If your senior daughter has a 3.8 GPA but bombed on her SAT's because she just doesn't test well, or was sick the day she took them, there are nearly a hundred SAT optional schools she could consider. If your highly intellectual, yet socially timid, son has blown the roof off his test scores and GPA but has none of those community service or leadership positions you know could have strengthened his application, there are many excellent schools which are looking for young people just like him, where knowledge is revered and socializing is an afterthought. The point is to go about this process with a deeper understanding of who exactly your child is, in what kind of learning environment he or she will thrive and prosper, and then to go about discovering the schools which will best meet your student's needs.

Too often parents and counselors have the upside-down notion of trying to shape a child to fit the school. In my opinion, doing that is a recipe for disaster. First, that approach diminishes and obscures the real accomplishments of your child, rather than honors them. A parent who does this is heading for trouble in that relationship which is already complicated enough. Second, it is crucial that your child's college choice be more than just a good academic match--a place where she will be challenged but still will have a great chance of success--but that it is also a good social fit where she will feel comfortable and happy in a community of peers, hopefully leading to life-long friendships and associations. And third, your child's application has a substantially better chance of acceptance at those schools which are looking for applicants with her particular attributes. A metaphor used to describe the college search process compares finding the right college to buying the right car. Yes, it's true that the applicant is a shopper and is looking for a model that best reflects her identity. But this is a dynamic process in which the commodity—in this case the institution—is also selecting the applicant. A better metaphor is that of a marriage. You are going about making a match, you learn about each other, you dance with each other, you experience each other as fully as you can, and then you take the leap of faith to ask the other for a long-term commitment. The more you know about each other, the better your chance of being truly happy throughout the years ahead.

Assisting your child with his college search requires parents to relinquish a notion they often hold dear, that hidden bias about which schools are the "best." Many of us believe either openly or secretly that the Ivies (along with Stanford and MIT) are truly the best colleges. They are the holy grail of this search, and depending upon how well your child has done in high school, and what social stratum you are from, some of the Ivies are more acceptable than others. Through the sharing of information from family to family, to the various ways we poll our friends and deduce what's valued and what's not, we design a kind of de facto hierarchy of schools, the top handful of which we hope our child applies to and gets admitted to. Without a doubt, the Ivies are terrific schools. Some have developed their reputations on the strength of their powerhouse graduate schools and research institutes, while others have developed great reputations on the strength of their undergraduate faculty. Most have huge endowments which attract the highest paid faculty, and this tends to be self-perpetuating, admittedly an advantage for those institutions.

Yet I ask parents to discard their notion of what's a great school and what isn't, and to start from scratch. Just as we need to truly understand our children's strengths and needs in order to best assist them in their choice of learning environments, we need to evaluate colleges with fresh eyes and a clear mind. What is a school's approach to learning, and is it the approach your child will most benefit from? Who will be the community of his peers, and is that a community in which he will thrive? Each school has a distinct personality and ethos, and when looking for a match for your child these are the qualities that families need to understand about a school. This doesn't mean that Harvard is not a terrific institution. It DOES mean that there are many, many more—some you may never have heard of—that could be even better destinations for your child.

One final thought. Anxiety will undermine this process and hurt your child's chance of success, so let go of it. How? By understanding that only a tiny number of schools are impossible to get into, and that 70% of all schools in the country admit at least 70% of their applicants. By understanding that college is not a prize to be won, but a match to be made. By understanding that your child's accomplishments will be honored and sought after by many, many schools. And most important, by understanding that the greater your child's ownership of the process and the ultimate choice of where to apply, the greater her chance of success at this most challenging and promising moment of her life.

Jeff Levy can be reached at jeff.levy1@ca.rr.com