How chatbots distort customer reviews—and may confuse other consumers in the future
09/05/2024
Retailers, Reviews, and… Robots?
Irina Heimbach - September 5, 2024
A new refrigerator, the latest smartphone, or even simply just some take-out for dinner—regardless of what they’re buying, customers will likely be asked to leave a review. This nagging request, though annoying for the customer, is of great value to other potential buyers seeking to gain more insight into the quality of a product or service so they can make an informed purchase. And this feedback is helpful for sellers and service providers as well: The more reviews they receive, the higher the chances that their products will attract the attention of other buyers and increase their sales.
Today, many review requests land in customers’ inboxes as a simple email, but times are changing. More and more, companies are making use of text-based conversational agents—i.e., “chatbots”—which address customers directly and ask them for feedback. These virtual robots can, to varying extents, be given a human-esque identity to increase the level of interaction with customers. And they certainly run the gamut—from basic chat programs to fully fledged anthropomorphic chatbots that have their own names, human avatars, and unique ways of communicating. Given the quality of the program’s natural language, customers may not even be able to tell if they are chatting with man or machine. And that affects how consumer’s give their evaluations, which can have both negative and positive consequences.
Chatbots influence the quality of customer reviews
A new study from WHU – Otto Beisheim School of Management and Rotterdam School of Management at Erasmus University reveals that customer reviews assume a more optimistic bent when submitted via conversation with a chatbot. When chatbots are equipped with the mere basic humanlike features, customers experience feelings of joy when interacting with them, and their enjoyment spills over into the product reviews. Additionally, customers develop an understanding of highly anthropomorphic chatbots as very present and very social conversation partners. As we know from everyday interactions, we rarely feel comfortable giving unfavorable feedback to others face-to-face.
In both cases, the anthropomorphic nature of the chatbot biases the customer’s judgment—and the resultant reviews, which often use a star- or point-based system, end up being more positive than they would have been had the customer responded to a simple email request. This decreases the quality of the reviews, meaning they are unhelpful to other consumers making a purchasing decision. For online retailers, it’s not just about merely obtaining reviews; the methods used to obtain said reviews are equally as important.
Positive customer reviews won’t help online retailers reach their goals
For online retailers, it appears to be simple arithmetic. The more reviews they receive, and the more positive those reviews tend to be, all the better for their business… right? Early research has already shown that online retailers benefit from customer feedback, as it helps other customers and pushes them to make a purchase as well. Some companies, in particular those with a poor reputation, have even resorted to paying for fraudulent reviews to make their online shops more competitive. This new study reveals, however, that online retailers should resist the temptation of using chatbots to boost customer reviews, as doing so will only cause trouble for the business later on down the line.
Managers who fall victim to this temptation—despite knowing that chatbots positively skew customer feedback—are putting their very reputation at risk. What happens when others realize that the reviews are painting a false image? Over time, customers learn that these reviews are not to be trusted, which will influence their purchasing behavior. Additionally, reviews written and then submitted via chatbot are lower in quality and therefore less valuable than reviews submitted via conventional means. The authors of the study have concluded that online retailers are best advised, for the long term, not to use chatbots to collect customer feedback.
Policymakers should regulate the use of chatbots
How chatbots can be used to collect reviews (if at all) in the future is, according to the research team, a matter for today’s policymakers. While many countries have already enacted guidelines to ensure the veracity of online reviews, there is still a lack of regulation regarding the use of anthropomorphic chatbots. The study shows there is a risk that such reviews will cause consumers to lose trust in goods and services acquired online. If policymakers fail to act, the reviews and feedback inflated by the use of chatbots will make it increasingly difficult for consumers to distinguish reputable businesses from questionable ones.
Tips for practitioners
- As an online retailer, be aware of both the opportunities and the risks that come with using human-like chatbots. While the increased positive reviews may seem alluring, the use of chatbots will put consumer trust at risk, which can have lasting effects and damage your brand and your image. And remember that reviews obtained by chatbots are often less helpful to your other customers and may lead to poorer purchasing decisions.
- Continue to rely on more traditional methods of collecting customer feedback, such as email requests. Reviews collected in this manner are significantly more helpful to potential new customers and ensure that your company is properly represented online. Additionally, criticism can often be the catalyst for product innovation.
- As a politician, advocate the introduction of new and clear guidelines that regulate the use of chatbots when collecting customer feedback. Existing guidelines do not sufficiently consider the use of anthropomorphic chatbots, which can have negative consequences for consumers.
Literature reference and methodology
For “The robo bias in conversational reviews: How the solicitation medium anthropomorphism affects product rating valence and review helpfulness,” the research team conducted four separate empirical studies and coalesced the results. In these four studies, the team analyzed how the degree of anthropomorphism affects a medium used to collect customer feedback. They also analyzed the use of conventional methods for collecting reviews in comparison to the use of chatbots presented with varying degrees of human-like characteristics.
- Tsekouras, D./Gutt, D./Heimbach, I. (2024): The robo bias in conversational reviews: How the solicitation medium anthropomorphism affects product rating valence and review helpfulness, Journal of the Academy of Marketing Science, p. 1-22. https://doi.org/10.1007/s11747-024-01027-8
Co-author of the study
Assistant Professor Irina Heimbach
Irina Heimbach is an Assistant Professor of Digitalization at WHU – Otto Beisheim School of Management. Her research and teaching interests focus on digital services, chatbots and generative AI, algorithmic decision-making, information privacy, and the dark side of digital technology.
