The absurdity of Best/Worst lists

I’m sure I’m not the only person to believe that “Best of” lists are a marketing gimmick. The respected food or movie critic, of course, proffers a list of his or her favorite restaurants or films; that’s the stated purpose of the column or article. It’s also intuitive that an editorial staff can select the “best” stories of the year under the same clearly defined auspices of their vocation. Consumer advocates can safely maintain objectivity by publishing and adhering to the criteria used to evaluate products. Where we go off track is with the pseudoscientific crowd that purports to insert some objectivity into the subjective process of ranking the non-quantifiable.

Take, for instance, an article in today’s WSJ purporting to describe the “best and worst jobs in the U.S.” The article is based on a study that measures “five criteria inherent to every job: environment, income, employment outlook, physical demands, and stress.” The study doesn’t divulge how they’re measured, but from a quantitative standpoint, I don’t see any reasonable way to objectively measure environment, physical demands, or stress. Income (of course) and outlook are readily quantifiable, but when 60% of your criteria are subjective, how can you make any kind of objective claim?

I’m just sayin’.

The error of these types of stories is not that they exist, but that they are presented as scientific. They’ve got one guy, author of a book entitled “Jobs Rated Almanac” and some Bureau of Labor stats. From this we are led to assign some validity, as though we were perusing a peer-reviewed article in Nature. In reality we have one man’s opinion on a bunch of subjective stuff. For example, the study’s website lists 21 “stress” factors that combine to give each rated job a stress rating, but these too are subjective: “Working the public eye” may be a negative for some but a strong positive for others; ditto Competitiveness, Advocacy, Outdoor work, and many others.

Best jobs for whom? From the looks of this list, it’s someone who likes numbers (Mathematician is #1, Actuary is #2, Statistician is #3). The Left-brain set, apparently, has it made. I can think of any number of folks (myself included) who would place these types of positions near the bottom of their lists of preferred jobs. In my case, it’s not because I hate math (although there certainly was a fair amount of innumeracy in the first 16 of my 18 years of formal education). It’s rather what I discovered in graduate school about quantitative science: apply it, and things come alive. I’m sure that many of us use numbers in our work, but few of us really dig the numbers for themselves. (My dad the engineering professor is an exception; I really do think he has a tremendous admiration for numerical systems, whereas so many of us merely benefit from them.)

But it’s not just numerophobes that are left out in the cold by this study. I once had a very successful sales colleague who pulled down well over $150,000 a year. He loved his job, had great work-life balance, and is now making even more money working for one of the biggest technology companies in the country; he’s exceeded quota every quarter he’s been there and will likely continue to be successful and satisfied. Yet his job is not high on the list. My friend once confided to me that he could never go back to “sitting behind a desk”, which seems to describe the majority of the top 20 jobs listed. Granted, we can contrast my friend with a somewhat less successful counterpart in another state, who was so lost as to his role as a salesman that he literally drove around town looking for big office buildings on which to cold call before his employer finally quit giving him advances on his commissions.

The point of these examples is, of course, that different people have different criteria to match their requirements and abilities. To somehow mold the whole working populace into a single expression of priorities, capacities and desires is absurd. But it catches the eye; so long as it’s couched plausibly, the readership will ascribe some validity (“leading expert in the field…government statistics” sounds better than “some guy who wrote a book…subjective measurements”). Job satisfaction is a fairly minor issue in an economy where most of us are merely glad to have a job, but I have no doubt that this type of pseudoscience will continue to make inroads into more substantial issues, so long as there are gullible readers to be found, even among the financial geeks in the WSJ readership. Thanks, Rupert! I can already see the improvement since the Journal joined the News Corp fold.