Making reviews more fair for hosts

For those who don´t follow the Air community forum, this was published last March by and Administrator

#Many of you have asked us how Airbnb can protect hosts from one-off bad reviews. When this question came up at the most recent Host Q&A, we told you we were working on ways to make the review process more fair for hosts. Specifically, we made 2 promises:

  1. We committed to launching a tool to detect outlier reviews—or one-off bad reviews. A common example is when there’s a discrepancy between the overall rating and the category ratings provided by a guest (when a guest gives a host 5 stars for cleanliness, accuracy, and the other categories but a 2-star rating overall, for instance).
  2. Based on your feedback, we also committed to exploring ways to help guests better understand that the location category rating is meant to be objective. The location category rating doesn’t impact your overall rating (or Superhost status), but we know it’s important to you, and we want to make sure the whole system is as fair as possible.

Today, we’re excited to announce two improvements to the review process that directly address these issues. Since these changes have been introduced, we’ve already noticed a tangible uptick in more accurate, fair reviews for hosts, and we hope they solve some of your pain points. Here’s what’s new:

One-off review alerts

We’ve added a step to the review process for guests when they give a host an inconsistent overall rating. For instance, the guest may have given 4-star or higher ratings for all the categories (cleanliness, accuracy, etc.), but then give an overall rating of less than 3 stars.

The new pop up screen asks guests: “Is this right?” And goes on to explain that they rated their overall stay lower than they rated it in specific categories. It gives guests an option to either change the rating or ignore the alert.

This new alert has led to higher overall review ratings for hosts. Since we launched, we’ve seen a 2.8% drop in 3-star reviews and a 3.9% drop in 2-star reviews. While these percentages may seem small, they’re driving real improvements in the accuracy of our review system, and hosts are benefitting.

Location, location, location

We’ve heard from you that the location rating can be particularly frustrating because some of you have experienced guests dinging you in this category, unexpectedly, after great stays. This category is tricky. It gives valuable information to prospective travelers, which we don’t want to lose. At the same time, we hear your concern that you’re being graded for something you can’t control: guests’ opinion of your location. This opinion is inherently subjective—one person’s “rustic rural retreat” may be another’s “too far from public transportation.” So we made it more clear in the review process that guests are rating the accuracy of your location description, rather than the location itself.

Now, when a guest goes to rate you in the location category, if they give you less than 3 stars, they see an explanation: “Was the listing’s location not described accurately?” So far, this has led to a 0.8% increase in the average rating for location.

While we were working on this, we also made similar improvements to the value category. If a guest gives you less than 3 stars there, they’ll see this message: “What would have made this listing a better value?” This has led to a 0.25% increase in the average rating for value.

These changes were designed to begin to address your concerns around unfair reviews, and to help make sure that guests understand what ratings mean. We still have a journey ahead of us to keep making the review system better, and you’ll continue to see updates from us on this throughout the year. Thank you for hosting!#

3 Likes

Interesting. Thank you for posting this.

1 Like

I read this a couple months ago and really don’t see how it will help except when a guest fat-fingers a rating on a mobile phone.

They’re doing nothing at all about retaliatory reviews by guests.

They only ask for reasons when a guest gives 3-stars and below in a category. It’s as if they’re saying to guests that 4-stars is good enough while measuring hosts by a higher standard.

Notice they don’t even talk about overall 1-star or overall 4-star reviews on the one-off alerts, which is a good indication that they don’t even bother with alerts for 4-stars and it’s done nothing to help with retaliatory 1-star reviews.

4 Likes

There’s nothing that hosts can do to ‘make reviews more fair’ especially when we are aware that there are many hosts who don’t write truthful reviews about their guests.

(‘I don’t review poor guests if they are nice people’ or ‘I don’t like to write bad reviews’ or ‘if I don’t leave a review that says it all’ or ‘other hosts will be able to read between the lines’ or the hundreds of other reasons hosts come up with to excuse their behaviour in not warning other hosts truthfully about guests.)

I have no idea if anything will ever change - until they do away with reviews altogether - so the only thing hosts can do is change the way we look at reviews. If we stop perceiving the occasional bad review as end of the world and the end of our businesses, we’ll be much better hosts and much happier people.

8 Likes

This is so true. There is one catch though: by Airbnb’s current system, even the occasional review can have a very negative impact on a host’s business if the host is just unlucky with the guests they get. I.e. the host gets a bad review very soon after listing/hosting or if the host gets multiple bad reviews a just a little too frequently.

2 Likes

My feeling (or possibly my problem?) is that I have never understood why a host should get a review that is so bad that it isn’t negated by good reviews.

I’m sure there are instances that would prove me otherwise but I just don’t see why a guest should leave a bad review if everything has gone well. I admit that there are nutcases who are simply weird but why would a host have more than one of those in a short period of time?

From Airbnb’s point of view (which is pretty difficult to imagine, I know) the reviews are the only way they can evaluate the hosts. And of course, the reviews are the only way Airbnb can evaluate the guests. Just as there are nutjob guests, I’m sure there are nutjob hosts too who write unfair and ridiculous reviews.

You, I and anyone reading this all know that one weird review means nothing in the context of other good ones. I imagine that many of us have been delighted with Amazon purchases that other people have slammed in reviews. (I certainly have). And if we can tell, I imagine guests who are reading reviews can too.

2 Likes

Thanks for posting. Note the second point says “less than 3 stars” which means 1 or 2 stars not 3 stars which I suspect are the worst most decent hosts ever get. So not much use. Notice also they aren’t dealing with outlier reviews (yet, but I suspect never since its not rocket science) where you always get 4 or 5 stars and then a 2 or 3 star review from someone who says the place was terrible. Personally I can deal with that but they should adopt TAs review display and explicitly say how many of each review type there is and let people click through. If only for the LOLz because when a hotel on TA gets 80% 4 and 5 star reviews it is fun to read the 1 and 2 star reviews from deluded guests.

I was also reading something about Yelp and their review system and they state that they are not there to police reviews or change them but that is up to the business and customer to sort out ie sue them or bribe them to take it down. Nice. It struck me that this is pretty much what AirBnB has adopted and seems intent on keeping.

1 Like

I think people just take it personally unlike businesses who expect outlier customers. Having the bad review and seeing it at the top of the review list (knowing most potential customers only read the top few, if any) makes people paranoid.

1 Like

There was also a posting about outlier retaliatory reviews being removed. I got a one star across the board from an animal in May. I am 2 years superhost. I have had at least 8 fiver s since then and I am still at 4.7.

1 Like

True - but it is a business whether we like it or not. We’re providing a service that we’re paid for, we pay taxes, we need licences and all the other things that being in business entails. Yes, of course there’s room for emotion in business because we’re not robots but we still have to be logical about it. And logic says that we’re going to get odd reviews from time to time. There’s nothing we can do about it so we have to accept it and move on.

I still don’t understand why guests should give unprovoked bad review though apart from the odd nutjob.

I should have said “corporation” rather than “business” because of course it is a business.

1 Like

All of us hope to learn something constructive from the negative review. What we are finding is that they are, outside of Airbnb’s TOS, unmoderated. Who does the TOS serve?

And what happened to removing one outlier review?

If you look at it objectively, where does anyone in the travel business with 50-500±reviews maintain a steady diet of 4.5 or higher?

Exactly. Especially when you consider the negative press received by Airbnb and the fact that so many hosts are part-timers. We all deserve a pat on the back.

1 Like

Thank you to the MOON… it’s like a miracle that finally a n y effort is made to interject rhyme or reason …it’s all we can ask… something resembling logic and coherency from guest reviews…thank you for implementing these prompts… we still have guests coming from hotels listed in air bnb with swim up bars and ‘great drink prices’… guests are clueless about apples and oranges when renting a private home… writing things like… no bartender or swim up bar on the pool… but a n y thing is better than no thing.