New research shows that developing product search engine rankings based on best value factors — the logarithm for a hotel site search engine would be calculated not just on price but also on proximity to beach and/or proximity to shopping areas, for example — generates greater revenue than rankings based on one criteria, such as price. The research also explores a number of other issues related to product search rankings, such as how rankings impact click through rates for various classes of products or how consumers react to personalised rankings.
Researchers from New York University’s Stern School of Business and Carnegie Mellon University studied the effects of product rankings in three areas:
The research team of Anindya Ghose and Panagiotis G. Ipeirotis, both of Stern, and Beibei Li of Carnegie Mellon focused on hotel rankings, using a combination of archival data and randomized experiments. The data came from an estimated one million consumer sessions with the travel search site Travelocity. The experiments were based on a real-world hotel search application created by the researchers, which allowed them to manipulate different factors (such as the opportunity for consumers to personalize the rankings).
Beginning with the direct impact of rankings on consumer behaviour, the researchers found that ranking affect click-through rates (CTR) — the higher the ranking, the greater the number of clicks. Interestingly, because different rankings can be based on different criteria (one might use price only, while another uses proximity to downtown as well as price), the researchers could study the impact of rankings for the same hotel. The data confirmed the importance of rankings, as the same hotel went from 56 clicks to 0 clicks when different ranking criteria caused the hotel to lose 31 places.
The researchers also explored which types of rankings were most effective in getting consumers to buy. The results showed that consumer ‘utility-based rankings’ led to the highest search engine revenue. Consumer utility-based rankings are not just based on price or an unidentified algorithm, but on a combination of issues that are important to consumers, such as proximity to beach or proximity to shopping areas. In other words, the rankings are based on the ‘utility’ of the hotel that consumers felt they had gained from their stay at the hotel. The researchers found that such utility-based rankings were more profitable than rankings based on price or the algorithms of Travelocity or TripAdvisor.
A second concern of the researchers was the relationship between rankings and product ratings, specifically two types of product ratings: customer ratings and class ratings (whether the hotel was a one-star or four-star, for example). Looking at the ratings and rankings of different hotels over a long period of time, the researchers found a variety of interplay between them. For example, hotels with low user ratings benefitted more from moving up the rankings than hotels with good reputations. As for class ratings, losing five places for a budget hotel didn’t have the same impact on click-through rates than the same loss for a four-star hotel.
Finally, the researchers studied the impact of personalized rankings and whether personalization helped to make the product search rankings more profitable. With active personalized rankings, consumers can weigh the importance of a series of personal preferences, including location and service preferences. Thus, for example, a respondent might give a +1 (the highest weight) for “near the beach” and a -1 for “near public transportation”.
The results of this section of the research showed that ‘active’ personalized search engines generated less revenue than the more ‘passive’ ones, which didn’t allow users to personalize the search.
Ideas for Leaders is a free-to-access site. If you enjoy our content and find it valuable, please consider subscribing to our Developing Leaders Quarterly publication, this presents academic, business and consultant perspectives on leadership issues in a beautifully produced, small volume delivered to your desk four times a year.
For the less than the price of a coffee a week you can read over 650 summaries of research that cost universities over $1 billion to produce.
Use our Ideas to:
Speak to us on how else you can leverage this content to benefit your organization. email@example.com