Amazon’s reviews may not be as reliable as you’d think.  In a study recently published in the journal Psychological Science, subjects were asked to choose between hypothetical products on Amazon’s website preferred more-reviewed items to less-reviewed ones in virtually all circumstances. The researchers also found that when it comes to Amazon’s ratings system, this bias toward popularity can actually lead people to choose what is statistically likely to be a lesser quality product.  That’s right….the more reviews don’t mean a better product.

So, if one item has an average rating of 3.4 stars and 258 reviews, or a competing item has an average rating of 3.5 stars and 14 reviews, you’d naturally assume that the product with more reviews might be better.  But you’d be wrong.

Researchers found that after analyzing 15.6 million reviews of more than 350,000 products on Amazon.com, that when a product has a ton of reviews but a mediocre rating like a three, the only thing that should be telling you as a customer is that it is definitely a mediocre product. Alternatively, the product that currently has the same rating, but a fewer number of reviews, has a better chance of actually being a better product overall since it hasn’t been reviewed by such a large group—meaning there’s a less decisive consensus that the product isn’t so great.

In fact, across 11 different conditions in which the more-reviewed product was statistically likely to be inferior, subjects opted for the morereviewed product 72.3% of the time. When it comes to consumer choices, a bias toward popularity can be self-defeating.

Why is this?  In part, it’s because retailers are trying to game the Amazon rating system.  Although many reviews on Amazon are legitimate, more and more sketchy companies are turning to compensated Amazon reviews to inflate star ratings and to drum up purchases. Businesses paid to create dummy accounts purchase products from Amazon and write four- and five-star reviews. Buying the product makes it tougher for Amazon to police the reviews, because the reviews are in fact based on verified purchases. The dummy accounts buy and review all sorts of things, and some of the more savvy pay-for-review sites even have their faux reviewers pepper in a few negative reviews of products made and sold by brands that aren’t clients to create a sense of “authenticity.” In fact, for extra cash, a company can pay one of these firms to write negative reviews of a competitor’s product.   Amazon has tried to deal with offenders and shut them down. In fact, in 2016 Amazon sued another round of companies that Amazon accused of selling fraudulent reviews. But by the time those companies were caught, their clients had already made a bunch of sales, and the fraudulent reviewers would reemerge under new names to repeat their scam.