How Ratings are calculated

I just happened to notice that a recent guest has rated her stay a 5 in every category but the overall rating was a 4. I’ve read through some previous posts on this subject and understand a little better how it could be to reflect something other than the categories offered up. I am waiting for a response from Support to find out what numbers feed into a hosts’ overall ratings math. If it turns out it’s the 4 (which frankly, seems wrong if that’s the case), I would feel compelled to give a little education to my guests before they check out. I did find such a message on the fridge door at a place I stayed recently and felt it was useful to educate guests about the ramifications of low numbers. The only way we can learn about what we can do better is if guests tell us specifically, and I do encourage them to do so. But I’d rather they convey it in that way, than through a randomly lower overall number despite the individual ratings. @Debthecat made a good point about 5 stars being associated with a “five star hotel” which is likely contributing to the disparity. Unless someone else has the answer to my question about which numbers they use to feed into one’s overall rating, I will report back what the answer is when I get it from airbnb.

Of course, I SHOULD have done this first - I googled the question and the AI generated answer is: “An Airbnb host’s overall rating is calculated by averaging the star ratings guests provide across various categories including: cleanliness, accuracy of listing description, communication, check-in ease, location, and overall value, with each category being rated on a scale of 1 to 5 stars; the final overall rating is the average of all these individual category ratings and the guest’s overall experience rating.”
So that’s good to know (and kind of weird that Superhost Customer Service said he had to look into it to find the answer!)
So according to my math that would mean a 5 rating in all categories and a 4 in overall would mean a 4.86 being factored into the overall listing rating.

@Roocroft, don’t take the AI-generated answer as the absolute truth. Airbnb keeps the algorithm hidden because they want the ability to tweak it as needed. Moreover, the algorithm evolves over time based on their decisions, not with the goal of seeking truth, but rather to serve the needs of their business and maximize profit.

As hosts, when we evaluate a guest, the first thing we are asked to do is assign an overall star rating that reflects our experience. After that, we are additionally asked to rate specific criteria based on our satisfaction. However, these criteria are limited and don’t always fully capture the reasons behind the overall star rating we give.

From what I’ve observed on our side, a guest’s average rating is based solely on the overall star ratings we assign at the beginning of the process. The additional category ratings and detailed feedback are published but don’t seem to be factored into the guest’s overall score.

I have only used Airbnb as a guest once, a long time ago, when the review process was simpler (just star ratings). So, I can’t speak in detail about how it works from the guest’s perspective, but it’s highly likely to follow a similar pattern.

Well - this was enlightening. The ONLY rating that factors into our ratings is the “overall rating” by guests. They don’t take any of the individual categories into account. Here’s what I was sent by Superhost support:

Total number of stars divided by total number of reviews = Overall Rating

Here is the sample:

Total number of reviews: 154

(# star x number of reviews)

5 stars review x 140 = 700
4 stars review x 14 = 56
3 stars review x 0 = 0
2 stars review x 0 = 0
1 stars review x 0 = 0
0 star review x 0 = 0
total sum = 756

(total sum / total reviews = overall rating)
756 / 154 = 4.90

When asked which numbers they use he said:
The computation we provided to you is for the Overall Rating that your guest provided to you together in the review from the reservation.

Star rating from sub categories has no effect for the Overall Rating your guest provided to you.

So to me, that means it’s time for me to educate guests as to the impact of their “overall rating” number. And the suggestion from a previous string that many guests likely think of it as compared to a “5 star hotel” service, meaning the overall rating would always be lower. This is sobering! Wish I’d learned it 11 years ago!

That is completely false. The overall rating is a separate rating. As are all the other ratings. Category ratings are only averaged within that category.

2 Likes

Continuing the discussion from How Ratings are calculated:

Yup - it’s interesting that AI got it very wrong. The last word from airbnb was: " Overall rating is not dependent on other rating parameters as it’s a separate category. The star rating average is based only on what the guest selected for “overall rating”. While guest can select star ratings for each of the specific categories (ex: accuracy, cleanliness), these are used for data collection and to inform the Host on how they can improve their listing only. Even though guests may have marked five stars in all sub categories for this reservation , it seems they have selected four for overall rating."

As a side note, sometimes folks put a query into AI and get random results but do not bother to authenticate what they are posting. Please let’s not all simply google some summary from AI when a question is asked here - firsthand and accurate 1:1 discussion is why we come here.

1 Like

My query was to this forum. Posing the question to AI was after hours of back and forth trying to get a clear answer elsewhere - both here and from Superhost support. I thought it would be useful to share my research here as responses came in but I may have been wrong. I might add, I assumed kind and respectful as well as informative discussions is why we come here. I will be more careful going forward.

1 Like

You’re not very cheerful today, @Rolf. :blush: I was glad to see @Roocroft bring up this topic because I had the same issue after a year of hosting, and the CS representative I spoke with was far less helpful than the one who answered Roocroft. Moreover, even though I’m fascinated by AI, which has almost saved my life, I’ve recently noticed that some AI models can produce quite a few unexpected results. Sometimes, these results even lack basic logic (Perplexity). So, I think it’s constructive for hosts who haven’t tried AI yet to rely on their own judgment and multiple sources before making decisions. I still enjoy using Perplexity for other reasons, but I think they eventually ingested some adulterated meals…

1 Like

May I ask if someone here has designed or put together an educational note or poster that is displayed in their rental that we could see?

I would like to create such a display in our studios as well but in a funny way so people can snicker about it but actually get the point of why it is shown.

There is so much house rules stuff and fine print on the booking platform already (besides the fact that some guest don’t seem to be able to read at all!!!) that I don’t want to hang up another “annoying note” in our rentals that just bothers the guests.

@Roocroft and @Ola

The answer to this question is within the Airbnb help files, which are easily searchable. They contain the actual Airbnb policies. It’s best to check with the official information source instead of AI, a blog, a CS agent or even a forum. It’s the fastest and easiest way to get official information about Airbnb policies (and the only truly reliable way). I recommend that all hosts read through the help articles so they can understand the policies.

1 Like

Ha ha! I’m laughing because I have notes EVERYWHERE - what not to flush -by the toilet, where to put different recycling, how to open and store the sofa bed, how to work the induction stove, dishwasher and TV controls, how to treat the grand piano… anything I think will make their stay easier or keep my stuff and house safe. And frankly, almost every guest/group says how much they appreciate the notes in their review or privately. I’ve only had one guest out of thousands who said it was annoying (but so were they, so…ha ha). Though note, this is the home where I otherwise live so that adds an extra dimension to the complexity and the sense of responsibility they feel in treating it right. I also have an info binder on the kitchen island with a page of general info; “where to find things”; favourite walks; what to check when they leave etc. It’s been working for me!

2 Likes

Frankly, I still don’t feel 100% confident that I got a clear answer but I’m going with the overall rating is the only one that counts! And I think it’s true that the “5 star” association with full on luxury hotel skews the guest’s assessment as it translates to a number. I’m pondering educating my guests which is tricky to have it not come of a manipulative.

1 Like

Please scroll up to read what airbnb says - it’s the only ‘response’ that matters and is accurate.

I wonder… Does your own response above matters, @Rolf ? Indeed, the help page is accurate and the fact that until now only Overall rating is factored for average rating result is almost verified.

I appreciate seeing that and am embarrassed that I didn’t dig deeper on the site. HOWEVER, to me that still does not clearly state “how ratings are calculated” as the heading implies. It does not state that only the overall rating is considered for a host’s listing’s overall rating - just that it’s its own category. So now I know but not because it was conveyed clearly by airbnb. Moving on.

Well, a listing only has one rating, and the article says category ratings are not averaged, so it would follow that your listing’s rating wouldn’t be an average of all the category ratings.

But for sure, Airbnb’s copy writers do not seem to understand how to state things clearly and thoroughly.