Choose Your Background

  • Noah Bradley: A place to call Home

  • Noah Bradley: Their Mournful Tones

A different fundamental approach to the rating system

RRL 2.5 had an interesting alternative to the current system. A like button and a dislike button. Nothing complicated. Nothing which you could overthink. Just a simple system. If people don't like your content, then you don't have as many likes / more dislikes, if they do, then vice versa.

It would solve the problem with 4 stars being considered an insult and 2 being redundant, but it would lose a lot of data.
I'm sad to say I have to agree with a lot of the points here. My original idea does fall flat.

With that being the case, a like/dislike thing, with maybe something like a 1-5 star rating for detailed reviews, might work better.

The reason, btw, I felt like a 4-star rating was a "punishment" is that someone reviewed my story, thought it was excellent, and gave it a 4.5 star review. Despite receiving exceptionally high praise, my rating lowered.

The reason that mattered is that I'm hoping my story can eventually become reasonably found on the "Best Rated" category. High rating doesn't cut it - it needs high followers, or favorites, or total reviews, or something. I was hoping to get there.

Aiming for the top means that every single rating below 5 stars is a serious setback. If it were a ratio between "likes" and "dislikes," it'd be a different story.

But part of the problem is that, knowing the problems with the rating system, if I want to review something, I have to consider this problem. If I think a story "deserves" 4 stars, because it's good, but not amazing, what do I do? Do I give it 4 stars and contribute to its inability to rise to the top? Like "Arcane Emperor" - that story is rife with grammatical errors. If I give it a 4 star review, the rating will be harmed, even though I like the story.

It's a mess :/ I don't know what the best solution is, but the present standing just seems awful.
Writing Sorcery in Boston

- RubyCona
The ranking system doesn't actually work solely on your rating average. It ranks you up against all other fictions and how they have been scored. It also puts into account how many times has a certain review been applied to your fic, weighing frequent ratings higher than one-offs.
So even if your rating average falls if you get a 4 or a 4.5 review, your ranking will more than likely go up.
Quote:The reason, btw, I felt like a 4-star rating was a "punishment" is that someone reviewed my story, thought it was excellent, and gave it a 4.5 star review. Despite receiving exceptionally high praise, my rating lowered.

It's unfortunate, but star ratings are averaged on places like Amazon as well. For authors who are looking to eventually publish, places like RRL can help prepare them for what they will have to deal with on sites like Amazon.

That said, people who actually will publish/plan to publish are likely the minority on RRL.

Figment's reaction system or a like/dislike system would probably be better than ratings overall. A breakdown of ratings could remain on reviews, though since reviews are much less common we could see even harsher shifts in ranking placements.

The biggest problem is how to migrate to a new system.

If RRL were to nix ratings, they would have to convert them into something else or let the data slip away entirely. If it was a like/dislike system, 3 star and higher could be a like while anything under would count as a dislike (that's basically how users seem to use ratings right now anyway.) If it's a "reaction" system, then certain emoji could correlate to certain ratings.

That said, with reaction/like/dislike systems it may be prudent to have reactions available on individual chapters. Encouraging more interaction on the stories and possibly giving the site more data to pull from when it comes to ranking fictions. Or, the total reactions/likes shown on the fiction page could be the total across all chapters.

Seeing a rank drop by 130+ places over a .06 change in rating is really disheartening, even for someone like me who is preparing to publish on Amazon soon. Something a little more casual to replace ratings would probably be more encouraging to all the fresh/newbie authors on RRL.

I also question who should be allowed to review, or if reviews should be split into reader vs. literary criticism categories. Generally speaking, reviews are meant to help potential readers determine whether or not they want to give the particular book or series a chance. However, a lot of authors on RRL seem to be under the impression that reviews are meant to give feedback, tips, or criticism. Finding a way to sort and distinguish between review types may be beneficial for all involved.

I'm just spewing ideas/thoughts at this time. Think that means it's time for me to sleep. lol
A thumbs up/down system wouldn't actually change how much that .06 change would affect your rank. The underlying system would still use numbers, and basically the same numbers in a different value range at that. So it'd basically just re-map our scaling system from 1 to 10, to 0 and 1. Mid-range ratings would be completely impossible, and the maths behind the rest would be the exact same.

Our experiment for V2.5 showed that there was barely any relative movement in the final ranking after shifting to the up/down vote system compared to before (there really wasn't any fiction that shifted SIGNIFICANTLY).

The problem is not with the rating system; any sufficiently advanced rating system is only as good as the users using it. I believe our current, weighed (yes, it is) Wilson score interval based system does fit that part - you can't get into the toplist with a single 5 star rating, and it weighs both rating and popularity into your ranking.
I find the current rating/ranking system fairly robust.

I think it would be cool if Royal Road had a leveling system like Newgrounds did where reviewers could level up through leaving reviews. I've noticed that it makes people more likely to write detailed reviews, which is useful feedback as an author.

At the same time, I understand how implementing that manner of system takes a great deal of time and effort.
Now it's day and night and the irons clang and like poor galley slaves
We toil and toil, and when we die, must fill dishonored graves

Nighthunter - My Nanowrimo LitRPG
I was reading one of Luciferia's reviews, for her stories, and one of her longer reviewers said "Short and to-the-point reviews ARE FOR THE WEAK!"

I agreeish.

Longer, more in-depth reviews, should be given more weight, as those probably rate the story more accurately than those that write only to meet the word count.

Logic would justify weighing a review from a reader who has rated 15 other novels than someone who reviews for the first time. I doubt those type of reviews would sink into handing out 5s, just by their nature. The readers who take the time to review novels on regular basis probably care more to put more energy than those who don't. Of course, this isn't democratic, but though this might reek of elitism, democracy by the uninformed hurts everyone.

*Cough Cough* Trump *Cough Cough*

So in the end, instead of just reforming the entire system, improving it would be a lot better.  If the main issue of the rating system is that it's inflated by readers who give 5 stars to a good story, and still 5 stars for a great story, then the rating loses its value. All of the sudden, having a 4.5 just doesn't feel good. That's the main issue.

As an author, this current system just doesn't feel good. As a reader, I can't tell the difference between a story rated as 4 stars and a story that is rated as 4.8. Creating a system won't just serve the author and reader, but the website too. Having a reason for people to sign up (yes, I'm talking to you Guest ) and start getting engaged further in contributing to the website will benefit everyone. 

Besides, most people on this website love LitRPG elements and titles. I doubt anyone will complain about getting levels.

Giving more weight to experienced reviewers, even though Goodreads and Amazon don't do that, might add more value to the ratings. Maybe thumping up a review as useful might give more points, I don't know, but I do know we can do better. 

Just because others don't improve their systems doesn't mean we should settle for less, too.

Though personally, I think a poll, one for readers and the other for readers, and maybe reviewers, would clarify whether change is even warranted.
The problem here is that you're trying to solve the community with the system, rather than improving the community itself.
My first fiction, a story about a Demon, a Princess, and how they tried to live happily ever after: The Demon and the Princess

Looking for stat screens and apocalypse? A New Leaf explores the (mis)adventures of Treant as he fights tooth and bark for his life, with a bit of comedy in between.

Users browsing this thread: 1 Guest(s)