Ratings and Reviews are Essentially Worthless

Reviews and ratings are common and can be seen everywhere. Reviews for movies, TV shows, music, video games, plays, books, cars, restaurants, you name it. However what most people don’t seem to realize is that most of the time, reviews are actually meaningless; at least free ones.

When it comes to reviewing or ratings things, there are two groups of reviewers: professionals and everybody else.

Professional reviewers get paid by magazines, websites, etc. to review things. Their reviews are generally useful because it is their job to write a review on the product/service whether they like it or not because they are being paid to do so. As such, their reviews usually do indeed reflect the quality of the thing in question. On the other hand, professional reviews are (as they have always been) susceptible to bias due to bribery.

Unfortunately, professional reviewers are few and getting rarer since thanks to the Internet, everybody and their dog can rate and review things.

There are two problems with reviews by amateurs (i.e., people who are not being paid to write it):

  • The quality of reviews by amateurs tends to be quite low. Most amateurs simply type up their general opinion instead of an objective, in-depth review (and this is assuming even a modicum of literacy on the part of the amateur which is increasingly elusive).

    Moreover, most amateurs seem not to even understand what a review is. Instead of typing up their opinion on the quality of the product/service, will simply describe it, basically just reiterating the spec-sheet/synopsis/etc. For example the vast majority of reviews on TV.com will just be an episode recap instead of describing whether the episode was good or not, likewise, on IMDB, many reviews will just describe what the movie is about. Useless—for some reason—they are giving a detailed scene-by-scene analysis, then these are completely useless and redundant.

  • Amateurs are not being paid to write reviews, so most will only bother to write a review if they either really loved it or really hated it. Most people don’t bother to write reviews for things that they liked unless they loved it so much they just have to gush about it. Likewise, people who did not like it don’t want to waste any more time on it than they already have, so they won’t bother writing a review for it unless they hated it so much that they need to vent and tell the world how horrible it was. People who thought it was so-so obviously don’t care.

    This results in a one of two kinds of reviews/ratings patterns:

    • Skewed in favor of it (you can see this in action on IMDB, TV.com, etc. where it looks like every show is great even though that is clearly not the case.

    • A polarized, bi-modal distribution where most of the curve is relatively flat with a sharp spike at the high and low ends, which may indicate the general attitude about the item, but not the actual usage. For example, a television show may have a bi-modal ratings graph, and people could on average feel that way about it, but it says nothing about how many people are actually watching, or likely to continue to watch, the show.

Professional reviews and ratings are—usually—reliable, but amateur ratings and reviews are generally completely pointless.