The Future of “Ratings”

"Ratings? Where we're going, we don't need ratings (but we'll have them anyway)."

Let’s be honest.  What’s in a rating?  What does “8″ really mean?  Is a movie rated 8 (or 80%, or 4 stars) really better than a movie rated at 6 (or 60%, or 3 stars)?  Really think about it.  What’s in a rating?

Ratings are aggregate scores that are supposed to reflect the overall sentiment toward a movie.  It’s meant to be a tool by which users can determine whether a movie is worth the time, or not.  So while an overall rating for a film is an aggregate of each rating each individual gave that movie, each individual rating is an aggregate of each individual sentiment felt or considered by the person rating the movie.  “Well, I liked the direction, but the music was terrible.  The acting was so-so and the special effects were amazing.  Plot was thin, but I really didn’t expect much anyway.  I give it a 6/10.”  All of those sentiments rolled in to one number.

The problem with ratings is the lack of context of what went into assigning the “score.”  If I’m in the mood for a great special effects movie, a number doesn’t tell me much.  Maybe if my friend who loves special effects told me a certain film was 5/5 stars, I could infer that it had great special effects.  But most of the time, we don’t even get that much.

So what’s the point?

Well, we see movies, right?  All of us.  And most of us don’t care to spend time watching movies that would be considered a waste of time.  So the point is, we deserve to have quick access to as much information about a film as possible; more than just a score.  And we’re doing what we can to make that as easy as possible.

For example:

Seeing the number of times a movie was recommended gives you an idea of how watchable it really is. Of course, there will always be ratings.

Get more in-depth reviews with just a click. The better written reviews bubble to the top with a reddit-like voting system. Top critic reviews populate the left column of each film's "Reviews" page. Blue means positive, red means negative.

Filter Chatter to the movie you want to know more about, and see what everyone is saying.

So will this all work?  Will it really replace ratings?  Probably not completely.  But our hope is that it makes more relevant information accessible in a shorter amount of time.  And it will only get better and more powerful with time.

Oh, and remember that we’re launching in a couple weeks?  Soon you’ll get to try it out yourself :D

JP

 

27 Comments on "The Future of “Ratings”"

  1. Matt says:

    Looks like you guys are really working hard on adding some great features. Can’t wait until launch :)

  2. Awesome – you keep telling me all the things I want to hear :)

  3. Alok Vaidya says:

    I’m simply impressed with the idea…. you guys are really intelligent. I could never thought of this and this never came to mind…

  4. Donna says:

    This looks really interesting, but I hope that a review’s placement isn’t primarily a factor of how many votes it gets, since that could squash insightful but unpopular opinions.

    I love the idea of showing us how often a movie is recommended!

    • John says:

      Great feedback Donna. I think we can create a better experience by allowing users to sort reviews by popularity/votes, as well as chronologically. Anything else you can think of to sort off of?

      • Matt says:

        What about if each user had a rating, and you could sort reviews by review authors with highest rating.

      • Donna says:

        A chronological sort is a good start, since the popularity sort may also favor older reviews that have been around long enough to earn votes. Maybe also a filter to show us reviews by rating? I like to see a mix of positive and negative opinions, because when reviewers who disliked a movie are specific enough about their reaction then I have a chance to decide if the aspects they hated would bother me.

        Is it safe to assume that reviews from users that we’re following will be prioritized or easy to sort out from the crowd on an individual movie’s page? That’s a big help to me over on Goodreads, at least when it comes to well-known books with a ton of reviews.

  5. Shane Helm says:

    1.21 Giga Watts!?!

    Great writeup JP! Since we’ve been working on the site for months now, these seem “present” to me. Hehe… But I guess they are new, or future, to our users.

    What you didn’t mention is that everything you’ve written about here are Phase 1 features, launching this month. We should tell people that we aren’t resting on our laurels once we launch Phase 1. We have a lot of tricks up our sleeve for future phases, and some deal with how well a movie is loved (or hated). Just thought I would interject there is more coming in the “future”…

  6. Dio says:

    Sweet! Excited to see all the tricks you guys have up your sleeves. I have been waiting for a site like this for ages. I’m curious, will the ratings system use full stars only or will members be able to give half star ratings?

  7. Yet more amazing stuff I can’t wait to try out. Remember me on that beta list :)

  8. Mo says:

    Just dying to get in and try it all out. As for ratings/reviews, perhaps something to consider, though it certainly would be work both on your end and ours (as users of the site) would be to have an option for advanced ratings or simple ratings. A simple rating is exactly what you have here. I think about it and give it x stars. An advanced rating could be giving a rating for music, acting, storyline, set/costume, special effects and that gives an overall number or rating. You could choose to give either a simple or advanced rating for each movie.

    Like I said, a lot more work, but who knows, maybe something to consider.

    • John says:

      Great idea Mo. Don’t mean to pretend we’ve thought of everything, but we definitely have thought of that. We’re just thinking through how the UI/UX would work for that without cluttering up the site, allowing users that don’t care to get into that much detail a way to avoid those details, but still making it easy to get to them. Stay tuned and thanks!

  9. movielens.org lets me export my ratings (which is also my “seen” list, obviously) as a tab separated variable file … have you guys been working on import tools so we can populate our seen list fairly quickly?

    • John says:

      No details on import tools yet Jacob, though that’s on our radar. Just been so focused on getting something up for you all to use at this point. More to come…

  10. sortile9io says:

    As for the “recommends” figure, a percentage (in relation to the total number of views) would be easier to interpret and compare to other films than the total number of recommendations. IMHO.

  11. Lisah says:

    Just had a thought…will we be able to send recommendations to our friends? You know, like send them a message ” Hey just got finished watching this movie, thought you might like it! =)”, or something like that maybe with a button that says “Recommend to a Friend”!?!

    • John says:

      Great thought, and the answer is yes. Anywhere you see a movie on the site there is the “Recommend?” action, which allows you to recommend that movie to another user. Just tag the user in the form that follows and boom, they get a notification saying, “Lisah has recommend [Title] to you.” In a coming update, we’ll even have the ability to mark that recommended movie as seen (if you’ve already SeenThat), or add it to your MustSee list (if you haven’t), directly from the notifications menu. Initially, there won’t be the ability to include custom messages, but that should also follow soon.

  12. Suka says:

    This is awesome! I’ve been looking for a site kind of like MyTVShows but for movies and I think you’re doing something way better. One (silly) thing I love from that site is that they tell you how many hours you’ve spent watching your shows. Is there gonna be something like that in SeenTh.at? And would you be able to set the day you saw the movie? And where or in what format you saw the movie? As in “Theater”, “DVD”, “Blu-Ray”, “VHS”, “TV”. Don’t mind me, I always care about the silliest things. I’m loving what I’ve seen so far, just the design itself is made of awesome. I can’t wait! (I can, it’s just a figure of speech, no pressure!) =D

    • Suka says:

      And also, how many times you’ve seen the movie? That’s all, I think. Me and my OCD are leaving now, haha.

    • John says:

      Love the feedback Suka! We’ve had a few requests for times viewed, and even total hours watched. These are things we’d love to implement down the road, but there are more basic updates that need to happen first. We’re keeping track of every idea, so keep ‘em coming!

  13. Art says:

    Excited!!!

  14. David says:

    I’m a huuuuuge fan of Goodreads, so I hope you guys have a similar ratings system. That means NO HALF STARS!

Got something to say? Go for it!