There is a very interesting Planet Money podcast about Amazon reviewers. The podcast focusses on top reviewers. These are people who are at the top of Amazon’s own reviewer rankings. They get there by writing lots of reviews and also on the quality of their reviews measured in part by whether consumers found the reviews helpful. The podcast focusses on why they do it.
As is typical of non-paid contributors, they are motivated by lots of things. First, and initially, by a desire to communicate opinions about products. Then they notice that they are gaining recognition. This includes the helpfulness of their reviews. Sometimes this involves feedback and interaction with manufacturers. After this, the rankings become more important as this helps drive recognition. And with all that comes a move towards more professionalism as they are sent products to review, including by Amazon. The Planet Money podcast raises concerns about these latter ones but the return seems low relative to the recognition factor which should outweigh any bias. Indeed, Amazon claims their own evidence backs that up. In any case, if Amazon’s Vine reviewers (the ones who get products from Amazon) were biased, it should do Amazon no good and they would surely not be at pains to identify such reviewers to their consumers.
One interesting thing, however, is that some of the top reviewers actually go to lengths to ‘game’ the system — for instance, setting up fake accounts to vote up their own reviews and down the reviews of rivals. This is interesting because if the reviewers are motivated by recognition for doing a good job, this activity distorts that reward. After all, if you vote up your own reviews, that isn’t recognition. That said, I can imagine a situation, whereby there are some bad apples — who want to get high in the rankings to get more products to review and seemingly more influence — and some good apples doing it for pure recognition. The bad apples may pollute the behaviour of the good. This is because a good apple, if getting voted down because of bad apple behaviour may decide to compensate by engaging in that behaviour themselves. It would certainly be interesting to see if there is some ‘race to the bottom’ effect going on in these systems especially when rankings appear.