How do Airbnb guest reviews work
After a guest completes their stay, they have the option to rate their experience across six categories and one overall score. In addition to this, they can leave a written review. The review must be written within 14 days of checking out. It is only posted after the host has completed their review of the guest, or after the 14-day period has lapsed, whichever comes first.
Obviously, we can pull tons of insights from such rich text data. In this analysis, we’ll look at positive and negative written reviews across on the platform and then identify the most common reasons for bad reviews on Airbnb.
Introduction to the analysis
In natural language processing (NLP), sentiment analysis is used to score text as positive, neutral or negative. There are many applications of this approach. For example, in business it’s used to help organize and understand customer feedback.
Airbnb reviews lend themselves well to this approach. Especially since we know that they are meant to convey sentiment i.e. how the guest enjoyed their stay. By scoring the reviews, we’ll be able to see how many reviews are positive (and negative) on the platform. From there, we’ll look at the relationship between a listing’s reviews and the overall rating.
Once we’ve learned about the distribution of reviews on Airbnb, we’ll focus in on just the negative reviews. We’ll use an approach called topic modeling to uncover what guests are saying when they leave negative reviews. Then, we’ll bring in other data points to develop some deeper insights.
I try to keep the technical detail to a minimum for the purpose of this blog and focus more on the insights. However, if interested, the full code and additional detail on technical choices made can be found on the github repo for this project. Data was sourced from Inside Airbnb, an activist project with the objective to provide data that quantifies the impact of short-term rentals on housing and residential communities
Distribution of Airbnb reviews by sentiment score
Now onto the analysis! The specific approach taken to score reviews by sentiment here assigns them a score. The higher the score, the more positive the review, the lower the score, the more negative. It’s important to note that this score can be driven up (or down) simply by the length of the review; longer reviews contain more words to score.
The graph below is a density plot and shows distribution. The dotted line has been placed at 0 which, in this scoring system, is a totally neutral sentiment.
Insights from the distribution of reviews
- Airbnb reviews are overwhelmingly positive. Perhaps this is not surprising since Airbnb ratings skew so high, but the same trend appears in reviews. Only around 1% of reviews on the platform are negative
- The next thing to note is the first peak of the distribution. This low, but positive sentiment score indicates the kind of short review that guests often leave on the platform. For example: “Awesome place, great host, highly recommended”. This review, while definitively positive, is not very descriptive. We can see this type of review is quite common.
- The final takeaway is the second peak, this shows the most common type of review on the platform which is a slightly wordier but overall positive review.
Effect of reviews on the overall rating
With all reviews scored, we can average them per listing to get an average sentiment score by listing. We can then compare this average score to the overall rating for the listing. The graph below summarizes this for all listings in the data set.
From this, the conclusion may seem obvious – the higher the average sentiment score, the higher the overall rating. However, there’s a little more nuance to it than that. Above we noted that the sentiment score can be driven up by the length of the review. This tells us that higher overall ratings are correlated to longer, more descriptive positive reviews.
What negative reviews say about an Airbnb listing
We can also split the listings into two groups – those with at least one negative review, and those with no negative reviews. Below we will again use density plots to show the distribution across overall ratings of these two groups.
From this we get an interesting and telling result. The distribution of listings with no negative reviews skews massively to the right towards five stars. On the other hand, listings with at least one negative review have something resembling more of a normal distribution (with a bit of a long left tail). A normal distribution peaks in the middle and slopes to either side. In other words, for listings with one negative review, some will have a high rating, some will have a low rating, and most will have a rating somewhere in the middle.
All this to say, one bad review doesn’t necessarily mean a listing will have a bad rating, rather it’s more likely to have lower rating than a listing with no negative reviews.
Topic modelling negative reviews
So far we’ve seen that better reviews are correlated to higher overall ratings and that listings with bad reviews are more likely to have lower overall ratings. But what is the cause of these bad reviews? For that we can turn to topic modeling.
Topic modeling is an algorithmic approach that takes text as an input and identifies clusters of words that are likely related. Using this approach can be as much an art as it is a science. After pre-processing the data and making judgement calls on the most relevant output, I ended up with 17 topics.
To visualize topics, we can summarize the most frequent words that appear in them. This chart appears below. Topics were also renamed to something more descriptive. For clarity, I’ll also list the topics with the appropriate interpretation.
17 Reasons for bad reviews on Airbnb
- Broken items
- Unsafe neighborhood
- Inaccurate amenities (amenities listed were missing or not as described)
- Dirty
- Bathroom (including missing or inadequate towels and/or toilet paper)
- Uncomfortable furniture
- Loud noises (from parties or neighbors)
- Front door (including issues with key/lock or gaining access)
- Check in (including issues or miscommunications at check in)
- Host interaction (guest contacted host during stay)
- Bed (including mattress, pillows, or blanket)
- Smell (for example mold or garbage)
- Late to respond (host was late to respond to guest)
- Room temperature (too hot or too cold)
- Shower (water pressure, temperature or drain)
- Kitchen (inadequate kitchen e.g. not enough dishes, cooking essentials, etc.)
- Parking (parking is far or difficult)
Most common reasons for bad reviews on Airbnb
With these reasons for bad reviews modeled and named, we can take a few more data cuts. For consistency topics will continue to use the same colors as above.
First, we’ll look at these topics by frequency. In other words, what were the most and least common reasons for bad reviews? This chart appears below.
From this, we can see the bathroom is the most common source of complaints. Loud noises follow as the next most common complaint. This goes to show just how sensitive guests can be to this issue. Rounding out the top three is the kitchen. A previous analysis identified the kitchen as a tricky amenity to have. It is desirable and a point of differentiation from hotels but can be a source of cleanliness issues for hosts. This analysis of reviews further confirms that insight.
Reasons for angriest reviews on Airbnb
Next, we’ll bring back in the sentiment analysis from earlier. By taking the average sentiment score of each topic we are effectively looking at the severity of the complaint. In other words, the more negative the sentiment score, the angrier the guest was who wrote it! The chart below looks at severity of complaints from low to high.
Looking at this output, we can see by far the angriest complaints in reviews have to do with host interactions. When interpreting this, we must consider the bias in the data. Interactions that go well, where the guest is contacted in a timely manner and their issues are resolved, won’t show up in negative reviews. What we’re seeing here is where things have gone very wrong and led to an unpleasant interaction with a host.
Other complaints ranking as severe include smell and dirty. These complaints speak to the general cleanliness of a listing. Listings that fall below a guests expectations are likely to receive vocal and angry reviews about it.
Finally, broken items and inaccurate amenities rank high in severity. Accuracy is so important to guests that it significantly influences the overall rating. Guests will penalize hosts harshly for inaccuracies in their listing.
Negative reviews with the biggest impact of the overall rating
Since we’re analyzing reviews for listings, we can pull in the actual overall rating, take the average by topic and see which complaints are dragging down overall ratings the most. This chart appears below.
This is a bit of an unexpected result! Listings with negative reviews about the bathroom have the lowest overall rating of the topics explored here. Perhaps the state of the bathroom says a lot about the overall quality of a listing.
Less surprisingly, the complaints that rank as the most severe also tend to have the lowest rating. The opposite is true of the least severe complaints. This is called a negative correlation – as severity of the complaint rises, the overall rating falls.
In fact, we can show this relationship by plotting severity and overall rating on a scatterplot. A dotted line of best fit has been added to help show this correlation.
We can use this dotted line to draw some other interesting insights. The more distant a topic is from the line the more it breaks from the trend. For example check in appears far below the line and to the left. From this we can say complaints about the check in process won’t get the angriest response from guests, but can really drag down the overall rating. We can say the opposite of unsafe neighborhoods. While reviews about this topic will be angry, they won’t have as much of an impact on the overall rating.
How to avoid negative reviews on Airbnb
Finally, we can use everything that we’ve learned here to create some data driven tips for hosts on avoiding negative reviews on Airbnb.
- Avoid emotional confrontations with guests at all costs. These can spill over into the reviews on your listing and drag down the overall rating. Instead remain professional, keep all communications on the platform, and do your best within reason to accommodate the guest.
- Keep the list of amenities up to date and accurate. Nothing is worse for a guest than booking a listing for a specific amenity only to arrive and find it is missing or broken.
- Ensure you have a seamless check in process. Any difficulties at check in can really bring down the overall rating.
- Use an ozone machine or other method to minimize odors in your listing. Bad smells can lead to negative reviews and lower the overall rating.
- Clean listings thoroughly between guests!. Areas requiring special attention include the kitchen and bathroom.
- Ensure the thermostat works or consider giving guests access to it to minimize complaints about temperature.
- Make sure basic essentials are stocked up including toilet paper, cooking essentials, etc.
- Communicate anything the guest needs to be aware of often before their arrival. For example, if there are special instructions about parking, do everything you can to bring it to a guest’s attention. Beyond including it in the written description of the listing, consider confirming with them by message that they are aware of these issues. How frequently these issues appear in reviews will tell you how hard you’ll need to work to make guests aware of them.
