How is the Overall Rating Calculated
When guests stay at an Airbnb, they can choose to leave a review and/or rate their stay across seven categories on a scale of 1 to 5 stars. After a host receives at least three reviews, the average of each individual category rating will be displayed on their listings. The overall rating is not calculated using the average of ratings across the other six categories as one might think. Rather guests give their stay an overall score in addition to rating individual categories. Given that the overall rating isn’t calculated using category ratings, this post will explore if the category ratings have an influence on the overall rating.
Introduction to the Analysis
For the analysis we’ll use data from Inside Airbnb, an activist project with the objective to provide data that quantifies the impact of short-term rentals on housing and residential communities. The full dataset contains 1.3 million listings from around the world as of April 2020.
We’ll restrict the dataset to listings with over 5 reviews. Airbnb will post aggregate scores after three guest review but doubling that threshold will provide a richer data set. Using this criterion drops the number of listings from the full data set of 1.3 million to 650k.
This analysis is going to use an approach called multiple linear regression. This technique uses several explanatory variables to predict the outcome of a response variable. In this case the explanatory variables are the individual category ratings and the response variable is the overall rating. Doing so will tell us if individual scores have an influence on the overall score.
Do Category Ratings Impact the Overall Rating
Running a multiple linear regression yielded statistically significant results with an R2 of 0.76. In other words, 76% of the overall rating is explainable by the category ratings. As mentioned at the top, this is not to say the category ratings are used to calculate the overall rating. However, the category ratings can help us explain the overall rating.
It may be a little intuitive that the category ratings strongly explain the overall rating. If a guest gives a host a high rating in all categories, then it follows they will likely give a high overall rating. What is perhaps more interesting is the flip-side of this. Category ratings fail to explain 24% of the overall rating. This says there is something intangible or at least not quantified in the available categories that goes into a guest’s decision of the overall rating.
Importance of Category Ratings to the Overall Rating
We’ve shown that the overall rating is mostly explained by the category ratings. But what’s really interesting is we can take this a step further and look at relative importance. Relative importance breaks down that 76% to how much each individual category contributed to it. This tells us which categories have the greatest influence on the overall rating. The output of that is as follows.
The way to read this is that high (or low) star ratings in the categories with the greatest importance are most likely to drive a high (or low) overall rating. We can see that accuracy, cleanliness, and value are especially important to the overall rating. Scoring high in these categories tends to lead to a high overall rating. The other side of the coin here is that categories of less importance are less likely to influence overall rating.
A host can have great check-in and communication with a guest but if they don’t deliver on the above categories, they’re unlikely to receive a high overall rating. Finally, location scores the lowest in terms of importance on the overall rating. This says that guests are factoring location in the least when deciding what overall rating to give.
Key Takeaways
- The overall score on Airbnb is not calculated using the category ratings, but can be mostly explained by them
- Cleanliness, Value and Accuracy have the greatest influence on the overall rating – scoring high (or low) is likely to affect the overall rating.
- Check-in, communication, and location have the least influence on the overall rating – scoring high (or low) on these is less likely to affect the overall rating.
