PDA

View Full Version : Reviews are awfully subjective and badly analyzed. Rating system is faulty as well.


mckracken
10-28-2003, 10:44 AM
I was browsing through some, and couldnt help but
write something, because I like the news portion of this site and
you guys seem to devote a lot of time also into the reviews.
So here are a few advices:

The rating system is non-functional because
there seems to be a lot of staff, so ones 3,5 will be another ones 4.5,
this makes comparsons betweens games futile.

With poorly analyzed, id like to take as an example the
Broken Sword review. The reviewer is so happy due to the
animated "juicy, action packed" ending, that he completely forgets to
judge it properly. (rip-off from raiders, overall shallow and more cliches than 3 B-movies combined)

Not to mention the perfect score for something thats far, far away from a flawless adventure. The only really thoughtful thing in BS, was the dialogue, which is really up there. The rest was and is highly debatable.

Plus, I HIGHLY doubt that everyone who has played their fair share of adventures (read knows what he's talking about..) would rate Broken Sword anywhere near Sam & Max or Indy 4, let alone above them.

As for another example of a 5 of 5 game, Longest Journey:
This is so loaded with hype, some might overlook the most annoying protagonist ever, the "tough teenage feminist, who takes not crap from nobody, always with a tedious witty oneliner on the short line".
I deeply hated her by the end of the game. The Story while nicely executed was done do death prior. Magic vs. Technology, with a LOT of symbolic trite.

LJ has a lot of fanboys on the net, and maybe for some this is indeed the better adventure in recent time (I found it average at best) but Broken Sword is badly misjudged.

Intrepid Homoludens
10-28-2003, 10:52 AM
;) Now aren't you being a bit subjective?

Kode
10-28-2003, 11:17 AM
;) Now aren't you being a bit subjective?
Precisely! :P

LiK
10-28-2003, 11:46 AM
of course reviews are subjective! lol, the reviewers tell u what they like and displike about the game. it depends on whether you trust their judgement or not. no review is truly objective. this is especially true for things like video games which require the gamer's tastes and their experience with past games.

every game site i know of have varying opinions on the same games. it's all because the reviewers have their own ideas if the game they're playing is good or not. this is why we should not judge one's opnion of something on one review. :)

Stinger
10-28-2003, 12:45 PM
Well, I have played my fair share of adventures, as well as your fair share and your mother's fair share also, and I'm telling you Broken Sword is one of the greatest adventures of all-time and I completely stand behind the five-star rating. The TLJ review probably could be toned down to a 4.5/5, but every publication readily admits sometimes they get carried away with original reviews of games. PC Gamer gave Alpha Centauri a 98% when it first came out. They gave Black & White and Neverwinter Nights each a 95%. Think they don't regret those scores now? Doesn't make their rating system faulty.

The VAST, and I mean VAST majority, of experienced, credible adventure game fans would sincerely disagree with your analysis of both of these adventures. So maybe you should gauge sentiment a bit before making such wild, generalizing statements. As far as the rating system being "non-functional" because different people review games...if you wouldn't mind sending me a link to a publication where one person reviews every single adventure game that comes out, I'd love it. But each reviewer understands our criteria and attempts to find a score that fits in our scale well. A 4.5 from me means the same as a 4.5 from Robert. Now, that doesn't mean Robert would have given 4.5 to Runaway as I did, but it sure doesn't make anything "non-functional" other than your ability to constructively criticize us.

I always welcome criticism. In fact, we're in very serious discussions right now on methods to improve our whole rating system. But seriously, don't come in here and say "Broken Sword has more cliches than 3 B-movies combined" and "TLJ has the most annoying protagonist ever" and accuse us of being subjective.

I'm glad you enjoy the news portion of our site, and I hope you continue reading. Thanks.

tabacco
10-28-2003, 12:47 PM
Reviews are awfully subjective and badly analyzed. Rating system is faulty as well.

Aww, that's sweet.

Jake
10-28-2003, 12:58 PM
Oops, Evan said more or less what I was in the middle of writing and editing. Lame.

Evan is right! (though perhaps a bit full of himself :shifty: )

eriq
10-28-2003, 03:35 PM
YOU'VE BEEN A BAD BAD BOY JOHAN! You didn't take out the trash! Now mommy's gingerbread is going to SMELL LIKE POO POO! BAD BAD JOHAN!

mckracken
10-28-2003, 06:55 PM
*The TLJ review probably could be toned down to a 4.5/5, but every publication readily admits sometimes they get carried away with original reviews of games.*

What is it with the perfect score of some games anyway?
Thats absurd. What if a game in the future is better than the godly Broken Sword? (low and behold...) You gonna paint-spray half a star on the 5?

*if you wouldn't mind sending me a link to a publication where one person reviews every single adventure game that comes out, I'd love it. But each reviewer understands our criteria and attempts to find a score that fits in our scale well. A 4.5 from me means the same as a 4.5 from Robert. Now, that doesn't mean Robert would have given 4.5 to Runaway as I did, but it sure doesn't make anything "non-functional" other than your ability to constructively criticize us.*

Puplications, before giving a final score, discuss in a roundtable their reviews and come to a shared rating. Thats not the best way to do it, but its better than to just go ahead and say "Robert maybe would have given it something else - but, no its not non functional"

Furthermore the most accurate way to review is by one single person who has got taste. I also run a review site (though in another field)
for four years and ive done every single one alone. With a rating system that goes from 1-10 in half increments and theorethically stops at 9.

Being overly subjective means have your view clouded from facts, somehting that reviews life from. There will and should always be a certain degreee of personal influence included, but it should be counterbalanced with the above.

tabacco
10-28-2003, 07:17 PM
So what you're saying is that no game should ever get a perfect score? In that case, why bother even rating games at all? It would be pointless to have a score you never gave out just as some kind of stupid statement.

mckracken
10-28-2003, 07:28 PM
So what you're saying is that no game should ever get a perfect score? In that case, why bother even rating games at all? It would be pointless to have a score you never gave out just as some kind of stupid statement.

Oh thats a brilliant reply. The perfect game? Made by God, published by Allah or what?

Wormsie
10-28-2003, 10:16 PM
Objectivity is nearly impossible.

As we can very well see from this thread.

DomStLeger
10-29-2003, 12:36 AM
Oh thats a brilliant reply. The perfect game? Made by God, published by Allah or what?


I think we need to be realistic McKracken, adventuregamers has a 5 star system with half stars. That allows only 11 gradings of games. You can't just arbitrarily limit it to 10 grades and say the 11th is just there for a game that may never be made. The star system doesn't mean the game is perfect but it does means it is better than another game and 5/5 makes it one of the best games available. I think part of the problem with the Ags system is not that BS got a 5/5 but that the database doesn't include some of the other games that should also have that score yet, which makes it harder for people to compare. Also if a better game does come along, we have to be objective and say lets judge these games when they were made. otherwise we'd have to constantly deflate the grades as time goes along.

And you said earlier that Broken Sword is not up to the standards of Indy 4 or Sam&Max and that most gamers would agree. Again thats highly subjective and I for one would definately not agree. For example I personally find the humour in both of your examples borders on puerile at times and are aquired tastes - equally subjective. Of course I'm sure people could say similar negative things about Broken Sword too, and so we end in a big arguement about how do you reflect everyones opinions at once. In the end Adventuregamers, Justadventure etc have to make a judgement call and say to themselves "what did I think of the game" and then "how similar is my opinion/taste/experience to the majority of other adventure gamers?" before fixing a star.

There are definately ways it could be improved. Like I said before expanding the review database with a few mini reviews perhaps, for games unlikely to make it into FlashBack friday anytime soon, could help people understand how the scores relate to their own opinions of games. A facility for mini "second opinions" to be added by other staff members whenever they like after a review, to gradually increase the range of views offered per game. Also I think a readers score system might be a good idea, to give people an idea what the general community thought. The figures from such a system could also be used to show how close an individual reviewers scores are to those of the readers, either so the readers know or just so the reviewer knows.

bpfinsa
10-29-2003, 01:20 AM
There does seem to be two solid ways people score: The "school grading" scale or a pure 0-10 scale. The school grading scale is done by virutally all the major sites. Any game lower than a 7 is poor (a failing grade in school) and shouldn't be touched. Others grade on a pure 0-10 scale where most games should either be scored a 4,5,or 6 and only a handful of titles get a 7-10. Not a lot of people use this scale, but when it is used, it riles up a lot of people on why the flavor of the month was scored a 6, so I guess that's why it isn't used too much.

--BPF

mckracken
10-29-2003, 01:23 AM
**The star system doesn't mean the game is perfect but it does means it is better than another game and 5/5 makes it one of the best games available. **

Confusing and contradicting. Youve got 5 stars, you give 5 stars. There is nothing left for improvement. You can look at that from whatever angle you want, its absurd.

**And you said earlier that Broken Sword is not up to the standards of Indy 4 or Sam&Max and that most gamers would agree. Again thats highly subjective**

It is not subjective. We know from professional reviews (print) that these two games averaged to be rated higher than Broken Sword.

**make a judgement call and say to themselves "what did I think of the game" and then "how similar is my opinion/taste/experience to the majority of other adventure gamers?" before fixing a star.**

Youre implying that a reviewer should also include the likelyness what the anonymous masses thought of game XY, ino his final rating? Disgusting.

**I think a readers score system might be a good idea, to give people an idea what the general community thought.**

The "community" is the lowest common denominator, which translates to fanboys and imbeciles. (harsh but true) Give them a voice and watch the nerds flock together in a truely gruesome fashion.

Everybody's a critic - this was never truer than it is today.

ragnar
10-29-2003, 01:51 AM
**The star system doesn't mean the game is perfect but it does means it is better than another game and 5/5 makes it one of the best games available. **

Confusing and never works out in reality. Youve got 5 stars, you give 5 stars. There is nothing left for improvement. You can look at that from whatever angle you want, its absurd.

It does work. If it would make you happier, you might think of AG:s scoring system as a 0-6 scale where no game ever gets over 5.

**And you said earlier that Broken Sword is not up to the standards of Indy 4 or Sam&Max and that most gamers would agree. Again thats highly subjective**

It is not subjective. We know from professional reviews (print) that these two games averaged to be rated higher than Broken Sword.
Are you trying to be funny? Of course that is your *subjective* view. Broken Sword is for me a clearly superior game to S&M and slightly above Indy 4. But since you think that *your* opinion is the "objective" view I don't think I can convince you that your view is highly subjective too.

**make a judgement call and say to themselves "what did I think of the game" and then "how similar is my opinion/taste/experience to the majority of other adventure gamers?" before fixing a star.**

Youre implying that a reviewer should also include the likelyness what the anonymous masses thought of game XY, ino his final rating? Disgusting.
A reviewer should include as little as possible of what anybody think including the reviewer. It should convey what type of game the reviewed game is to help the reader select what games they like. You also quite forget that the score is just a small part of a review, you should read the whole reviews also. Btw, I think the best rating system is no rating system at all since that forces the reader to actually read the review and not just complain about the rating at the end of it.

**I think a readers score system might be a good idea, to give people an idea what the general community thought.**

The "community" is the lowest common denominator, which translates to fanboys and imbeciles. (harsh but true) Give them a voice and watch the nerds flock together in a truely gruesome fashion.

Everybody's a critic - this was never truer than it is today.
Yes, but the reader score was added as a suggestion as an *addition* to the reviewers score, not as a replacement.

mckracken
10-29-2003, 02:02 AM
Broken Sword is for me a clearly superior game to S&M and slightly above Indy 4. But since you think that *your* opinion is the "objective" view I don't think I can convince you that your view is highly subjective too.

Its because youve got bad taste - it happens.
Again, looking at professional reviews, both games averaged higher than Broken Sword.

You might have also enjoyed Titanic the motion picture.
But that doesnt take away from the fact that a serious critic would decorate its toilet seat with the script.

Oh and pardon me for not taking your 0-6 imaginary friend, very serious - at all. ;)

ragnar
10-29-2003, 02:09 AM
Its because youve got bad taste - it happens.
Again, looking at professional reviews, both games averaged higher than Broken Sword.
*ROFL*. You are trying to be funny.

Everyone that doesn't agree with your taste has bad taste? Good one.
You might have also enjoyed Titanic the motion picture.
But that doesnt take away from the fact that a serious critic would decorate its toilet seat with the script.
No, Titanic isn't my favourite movie (even if it wasn't that bad). You seem to have a rather strange view of what a reviewers job is. A reviewers job is to convey what a movie/game/book/music is and try to tell if the reader will like it or not. The reviewer is there for the readers. That so called "serious" critic of yours wouldn't do his job proper if he did what you imply.

Erwin_Br
10-29-2003, 02:11 AM
I think we're missing one important point here: the purpose of the stars. I think the stars are only convenient for a quick look at the overall quality of the game.

If game A gets 5 stars and game B gets 5 stars too, then of course it's possible that game B is slightly better, which you can't see back in the rating. But who judges a game by only looking at a few stars? If game B is better than game A then you can read that back in the review.

What I'm saying is that I consider these stars as just a quick indicator, a rough pointer. If a game scored 5 stars it doesn't mean other games in the future can't be better. A better game would score 5 stars too, and looking at it's score you'll be able to tell that's a great game.

--Erwin

ragnar
10-29-2003, 02:15 AM
Oh and pardon me for not taking your 0-6 imaginary friend, very serious - at all. ;)

Just to make you observe this: It was you who invented it in the first place, you just called it 1-10, where you can't get higher than 9. I hope you don't take your own scale seriously either, then.

mckracken
10-29-2003, 02:20 AM
*ROFL*. You are trying to be funny.

Everyone that doesn't agree with your taste has bad taste? Good one.

No, Titanic isn't my favourite movie (even if it wasn't that bad). You seem to have a rather strange view of what a reviewers job is. A reviewers job is to convey what a movie/game/book/music is and try to tell if the reader will like it or not. The reviewer is there for the readers. That so called "serious" critic of yours wouldn't do his job proper if he did what you imply.

The reviewer is not, to put it bluntly, a whore for the readers.
And all I needed to know from you, was the "even if it wasnt that bad".
I think its in your interest as well, if I'm ending this conversation, ragnar.

Erwin_Br
10-29-2003, 02:30 AM
What I forgot to mention:

If you look at the definition of AG's stars you see that 5 stars means:

An instant, Hall of Fame classic. Every true adventure fan must have this game in their library. Well worth any price!

So, you're saying a game can't have 5 stars because it would leave no room for improvement.

You're wrong, because 5 stars simply means that you should have this game in your collection and that it's a classic.

So if there is a game that's better than game X which got 5 stars, it doesn't mean it has to have a higher score because the conclusion would be the same: you should have this game in your collection and it's a classic.

As I was trying to say, you have to read the review to find out the details and how it compares with other games.

--Erwin

Tom_K
10-29-2003, 02:34 AM
Puplications, before giving a final score, discuss in a roundtable their reviews and come to a shared rating.A shared rating? That makes little sense. How is it that one person can play the actual game, and allow the influence of his peers who have not played the game to effect the final rating? Why would a popular magazine like Game Informer include a "second opinion" rating if this were the case?

I think it's insulting and altogether deflates your argument[s] when you include statements like "should be done by someone with taste", what are you - the good taste ambassador?

The very nature of reviews make them subjective. I think it's ludicrous to imply otherwise. Under the confines of the reviewers experiences, each review is biased. As a reviewer, one simply tries to be as informative as possible, taking into consideration other tastes and preferences. Also, I don't know of anyone who's interested in a game that doesn't scour the net for more than a single review for comparison sake.

As to our rating system, I think Evan (Stinger) answered that question.

ragnar
10-29-2003, 02:36 AM
The reviewer is not, to put it bluntly, a whore for the readers.
And all I needed to know from you, was the "even if it wasnt that bad".
I think its in your interest as well, if I'm ending this conversation, ragnar.

So you are basically saying that just because I don't think the movie Titanic is the worst piece of shit ever produced my taste is the most horrible imaginable?

You have a strong case here. :rolleyes:

Jackal
10-29-2003, 02:44 AM
McKracken, you have a few solid points here, but you're blowing it with your Broken Sword/TLJ vs. Indy/Sam&Max tirade. Say what you will about personal preference, all of these games are quality titles with excellent production values, and AG's reviews reflect that. Your argument over the miniscule difference in scores is purely subjective, period. By all means, start another thread debating which games are better, but it's not helping your cause in this one.

Oh, and I'm sure most print reviewers would laugh themselves silly over your comment that their scoring somehow represents an "objective" standard. They are no more qualified to review games than the next schmoe; they just happen to (typically) WRITE better. "Professional" simply means "paid". Who told you it meant "superior"?

That said, your issue about the 5/5 is legitimate. I understand the spirit of the rating is to give a game a full endorsement, but I agree that a "perfect" score undermines its credibility. And that leads directly into the star system, which I also find extremely limiting, and I'm glad to hear AG is looking into ways to improve its rating system.

Some people may prefer this method, but I find it's a bit of a cop out. Each "score" actually represents a RANGE of scores. That takes some heat off reviewers, but it ends up creating too much "sameness". And many of those are for games I might be on the fence about. Sure, anything 2 or below, or 4 and above I can relate to, but take a 3 star game... Maybe I'm just a freak (quite likely), but I mathematically equate 3 out of 5 with 60%. But since technically the NEXT possible score is 3.5, or 70%, then that's a pretty darn big range this game could fall into. And if we're rounding UP, as our math teachers drill into us, then maybe a 3 is really 55-64%. Either way, still a BIG range, and where a game fits into that range can make a big difference.

Quite true what's been said about it just being a supplement to the overall review, but I'd still rather see a reviewer lay it on the line with an actual percentage.

mckracken
10-29-2003, 02:47 AM
**A shared rating? That makes little sense. How is it that one person can play the actual game, and allow the influence of his peers who have not played the game to effect the final rating? Why would a popular magazine like Game Informer include a "second opinion" rating if this were the case?**

Important games are played by more than one person, its a common procedure in game mags these days.

**The very nature of reviews make them subjective. I think it's ludicrous to imply otherwise. Under the confines of the reviewers experiences, each review is biased. As a reviewer, one simply tries to be as informative as possible, taking into consideration other tastes and preferences. Also, I don't know of anyone who's interested in a game that doesn't scour the net for more than a single review for comparison sake.**

Ok lets forget the "yada yada everything is subjective et cetera" preaching from every second person on these boards, because you guys seem to miss my point on purpose, I guess.

But why would a potential reviewer need to take into consideration other tastes and preferences when judging something? You know whats that called in print puplications? Bribe.

What makes it even more interesting (or disturbing if you want)
is that this comes from the mouth of a writer for this very site.

ragnar
10-29-2003, 02:51 AM
You quite contradict yourself you know:

Important games are played by more than one person, its a common procedure in game mags these days.

Furthermore the most accurate way to review is by one single person who has got taste. I also run a review site (though in another field)
for four years and ive done every single one alone. With a rating system that goes from 1-10 in half increments and theorethically stops at 9.

Do you want several people reviewing or one single person?

Also, where is your review site, see how "objective" your reviews are.

Kode
10-29-2003, 02:53 AM
But why would a potential reviewer need to take into consideration other tastes and preferences when judging something? You know whats that called in print puplications? Bribe.


Whether it is called that in print publications or not, at AG, I think it's called "taking into consideration other tastes and preferences." It is merely an understanding that somebody may like something that you don't.

Tom_K
10-29-2003, 03:37 AM
Important games are played by more than one person, its a common procedure in game mags these days.
Fair enough, but the primary difference is this site, the host, and reviewers are all gratis contributors. Perhaps if money were being exchanged the guidelines may, of a necessity, be a bit different. As it is, most seem to have no problem with the format.

Ok lets forget the "yada yada everything is subjective et cetera" preaching from every second person on these boards, because you guys seem to miss my point on purpose, I guess.

But why would a potential reviewer need to take into consideration other tastes and preferences when judging something? You know whats that called in print puplications? Bribe.

What makes it even more interesting (or disturbing if you want)
is that this comes from the mouth of a writer for this very site.
Yes I review games for this site, and if there's a particular problem with my personal reviews - I'm all about a critique, please let me know. As for the "argument" - I was simply stating a fact, not trying to build a case.

Why would someone consider other tastes and preferences? Well, how about a telling statement like "If you like a lot of character interaction and a linear path, then this game is for you" for starters? It's simply in an attempt to help the reader make an informed decision. What you're implying is rather extreme.

You have me interested in checking out your site. Would you care to provide a link?

DomStLeger
10-29-2003, 03:38 AM
**The star system doesn't mean the game is perfect but it does means it is better than another game and 5/5 makes it one of the best games available. **

Confusing and contradicting. Youve got 5 stars, you give 5 stars. There is nothing left for improvement. You can look at that from whatever angle you want, its absurd.

I fail to see anything absurd about it. It's absurd to suggest that 1/10th of the possible scores should never be achievable. If it were a 1-100 system then reserving 100 for a perfect game would make sense. But we're talking about a 1,2,3,4,5 system. You can't say no one can get a 5 because then you make it harder to discriminate between games. If you don't use the 5th star you have less catagories for reviewing games into, (11 catagories, 0, 0.5, 1, 1.5, 2, 2.5, 3, 3.5, 4, 4.5, 5 for hundreds of games). You have to use 5/5 to be able to discriminate between games at the top end. To reserve it for a perfect game is a nonsense on this system.

**And you said earlier that Broken Sword is not up to the standards of Indy 4 or Sam&Max and that most gamers would agree. Again thats highly subjective**

It is not subjective. We know from professional reviews (print) that these two games averaged to be rated higher than Broken Sword.

I disagree with that assessment. The reviews from "professionals" are ALL subjective, and they are definately not directly comparable as they are written by completely different people at different times. Like I said, what you think about the games is your opinion, and what I think is mine. They are our own opinions, and to try to say one is truer than the other using someone elses opinion is an utterly flawed arguement. It's like a child saying "I like the colour blue and my best friend likes the colour blue so it must be the best colour". Opinions can only be backed up with facts, not other opinions. So to say Indy4 and S&M are better than Broken Sword because X Y and Z say so it's absurd. It's also a pointless side arguement.

**make a judgement call and say to themselves "what did I think of the game" and then "how similar is my opinion/taste/experience to the majority of other adventure gamers?" before fixing a star.**

Youre implying that a reviewer should also include the likelyness what the anonymous masses thought of game XY, ino his final rating? Disgusting.

The whole job of a reviewer at AGs is to give the reader an idea as to whether a game is good or not. You have to take your readers into account if you're reviewing a game. I'm not saying a reviewer should pander to the masses. But when making their final rating, they have to standardise it - it has to fit the rating system, so you have to know what the readers expect from each rating.

**I think a readers score system might be a good idea, to give people an idea what the general community thought.**
The "community" is the lowest common denominator, which translates to fanboys and imbeciles. (harsh but true) Give them a voice and watch the nerds flock together in a truely gruesome fashion.

Everybody's a critic - this was never truer than it is today.

You're welcome to your opinions. Personally I think the community is not madeup of fanboys and imbecilies. And personally I think that was a rather obnoxious thing to say.

DomStLeger
10-29-2003, 03:45 AM
Ok lets forget the "yada yada everything is subjective et cetera" preaching from every second person on these boards, because you guys seem to miss my point on purpose, I guess.

No, more likely because you're having trouble expressing your ideas coherently. No one is deliberately missing any point on purpose.

But why would a potential reviewer need to take into consideration other tastes and preferences when judging something? You know whats that called in print puplications? Bribe.

Because the job of a reviewer is to lay out the pros and cons of the game so people know whether it is worth buying it. Anyone is free to write what they like about a game, but if you're writing a review for a site like this, it is vitally important that you are consistent with the rest of the site to some extent. This is so that people actualy know they can trust the views you express in a review as something reliable on which to base their decision to buy the game. It's as simple as that.

And I've never understood Bribe to mean what you seem to think it means.

mckracken
10-29-2003, 03:54 AM
I disagree with that assessment. The reviews from "professionals" are ALL subjective, and they are definately not directly comparable as they are written by completely different people at different times. Like I said, what you think about the games is your opinion, and what I think is mine. They are our own opinions, and to try to say one is truer than the other using someone elses opinion is an utterly flawed arguement. It's like a child saying "I like the colour blue and my best friend likes the colour blue so it must be the best colour". Opinions can only be backed up with facts, not other opinions. So to say Indy4 and S&M are better than Broken Sword because X Y and Z say so it's absurd. It's also a pointless side arguement.


I see you people here are not very familar with the term "grey".
Objectivity is something to strive for, not a mere black and white degradation.

As for the pros: Theyre paid to write reviews. They get paid for a reason: experience and journalistic talent. Of course youre entitled to tell an art critic that Michelangelo's David sucks. I mean hey its your opinion right? But please dont forget that you would also look like an idiot in the process.

mckracken
10-29-2003, 04:13 AM
Because the job of a reviewer is to lay out the pros and cons of the game so people know whether it is worth buying it. Anyone is free to write what they like about a game, but if you're writing a review for a site like this, it is vitally important that you are consistent with the rest of the site to some extent. This is so that people actualy know they can trust the views you express in a review as something reliable on which to base their decision to buy the game. It's as simple as that.


Oh excuse me, i didnt know this was Amazon.com.
Its of course not the job of a reviewer to help people decide buying a game or not. Where do you get your ideas? And what shabby standard is that?

If anything, this is a side effect, a bypass product. A review should revolve only around the actual quality and its accurate measure, and certainly not about whatsoever commerical and economical aspects.

DomStLeger
10-29-2003, 04:14 AM
I see you people here are not very familar with the term "grey".
Objectivity is something to strive for, not a mere black and white degradation.

As for the pros: Theyre paid to write reviews. They get paid for a reason: experience and journalistic talent. Of course youre entitled to tell an art critic that Michelangelo's David sucks. I mean hey its your opinion right? But please dont forget that you would also look like an idiot in the process.

So you're saying that someones reputation makes what they say somehow more worthy? I think this is where we differ; I believe everything should be judged on it's own merits. So, in your example, you would not look stupid telling an art critic Michelangelo's David sucks if you back it up with some good reasons.

But I have to be frank, I fail to see where this is going now, or the relevance of some of what you're talking about: this topic is about the subjectivity of the reviews and the faults with the reating system. I'm not quite sure where the objectivity came into it.

DomStLeger
10-29-2003, 04:21 AM
Oh excuse me, i didnt know this was Amazon.com.
Its of course not the job of a reviewer to help people decide buying a game or not. Where do you get your ideas? And what shabby standard is that?

If anything, this is a side effect, a bypass product. A review should revolve only around the actual quality and its accurate measure, and certainly not about whatsoever commerical and economical aspects.

The very job of the reviewer is to aid people in deciding whether the game is worth playing. Have you even read the ratings system on this site?
http://www.adventuregamers.com/about/editorial.php

Of course it should revolve around the actual quality of the games, but the whole point of games reviews is to help people decide whether the game is worth playing. Otherwise what on earth is the point?

mycroft
10-29-2003, 04:45 AM
there is something very wrong going on in here.

normally i would've jumped at such a heated discussion as this...but it seems that mckracken is being unreasonable...you are conveniently dodging some excellent and very valid points made against your arguments...see a couple of posts above to see what i mean .you only choose to challenge those to which you can find an answer..

dude..here is some food for thought...

there is no such thing as a perfect game.no game deserves an absolute score.if done so...then that would mean that we have reached the limits of our creative and artistic abilities.beleive me..an art critic will never say that Michelangelo's David is a 'perfect' sculpture.like objectivity...perfection is also to be strived for...but not attainable..such is the irony..

there is no such thing as a neutral review.there always is a certain bias present in a review.be it from one reviewer or a panel of reviewers.even if a review reaches the near depths of being neutral...some moron like you will take it apart for having gotten a better rating than your favourite game.if you think you have written a neutral review from the ones that you mention...please care to enlighten us ignorant fools as to the nature of this beast.post us a link so that we too may see what a neutral review looks like.in this context you cannot possibly challenge what the reviewers at AG considered a better game than another one.

the AG rating system is not flawed.its beauty has already been elaborated by erwin.see a score of 5 stars may mean an absolute score at other reviewing sites/magazines.but AG has an associated context with it.so a game receiving 5 stars is not the greatest game ever made...it is a game that is so good,it is worth having in your library.if another 'better' game has got 5 stars too..then it too must be in your library.as for which game is better if both have got a score of 5 marks..well the reviews clearly mark the pros and cons of the games don't they...see thats what i mean...here a game receiving a score of 5 stars have pros and cons.perfection is not bestowed upon it.



and as for the 'Print Publications' that you have apparently sold your soul to...what do you think of the venerable CGW when it gives a game 5 stars...there is no context in their rating system...to them 5 stars means a perfect game.absolution.no game will ever be perfect.no game ever deserves 5 stars.and as for the pros and cons in CGW...i clearly remember a reviewer giving NFS IV:High Stakes a score of 5 stars and NO cons.an absolutely perfect game in their view.while the game was good...it was far from perfection.the bugs alone are proof of that .the CGW rating system is flawed.we have been following a blind leader.


in conclusion dear mckracken...look within yourself...if you feel we are wrong..show us what right is...post a link to your reviews.

and i feel that from the begining..this thread has been in retaliation of a less than perfect score for your favourite game from AG.

Marek
10-29-2003, 04:54 AM
I'd like to add that the "pro" reviewers at large games magazines and commercial websites are just gamers like you and me. There's nothing special about them. Just gamers with opinions. They didn't study to be a 'game critic'.

As for the rest of the thread ... I don't even know what it's about anymore. McKracken, it would help a lot if you didn't take such an accusatory and -- dare I say it -- arrogant tone.

mycroft
10-29-2003, 04:57 AM
I'd like to add that the "pro" reviewers at large games magazines and commercial websites are just gamers like you and me. There's nothing special about them. Just gamers with opinions. They didn't study to be a 'game critic'.

As for the rest of the thread ... I don't even know what it's about anymore. McKracken, it would help a lot if you didn't take such an accusatory and -- dare I say it -- arrogant tone.


touche...

mckracken
10-29-2003, 05:09 AM
You dont like to read a lot, do you? Please
evaluate a bit longer before typing, otherwise I feel like im wasting my time here.

**there is no such thing as a perfect game.no game deserves an absolute score.if done so...then that would mean that we have reached the limits of our creative and artistic abilities.beleive me..an art critic will never say that Michelangelo's David is a 'perfect' sculpture.like objectivity...perfection is also to be strived for...but not attainable..such is the irony..**

RIGHT ON. Exactly what im saying since the beginning. You seem to defend my main argument. !?

**there is no such thing as a neutral review.there always is a certain bias present in a review.be it from one reviewer or a panel of reviewers.even if a review reaches the near depths of being neutral...some moron like you will take it apart for having gotten a better rating than your favourite game.if you think you have written a neutral review from the ones that you mention...please care to enlighten us ignorant fools as to the nature of this beast.post us a link so that we too may see what a neutral review looks like.in this context you cannot possibly challenge what the reviewers at AG considered a better game than another one.**

I do not review adventures. This thread was meant to wake up a few people. It is not to be taken offensive. If someone does, id consider therapy. Oh and consider this my last reply to you, as I dont like being called a moron.

**the AG rating system is not flawed.its beauty has already been elaborated by erwin.see a score of 5 stars may mean an absolute score at other reviewing sites/magazines.but AG has an associated context with it.so a game receiving 5 stars is not the greatest game ever made...it is a game that is so good,it is worth having in your library.if another 'better' game has got 5 stars too..then it too must be in your library.as for which game is better if both have got a score of 5 marks..well the reviews clearly mark the pros and cons of the games don't they...see thats what i mean...here a game receiving a score of 5 stars have pros and cons.perfection is not bestowed upon it.**

Yes it is flawed. The main guy from this site even said that they wanna improve it because its obviously not all that.

**and as for the 'Print Publications' that you have apparently sold your soul to...what do you think of the venerable CGW when it gives a game 5 stars...there is no context in their rating system...to them 5 stars means a perfect game.absolution.no game will ever be perfect.no game ever deserves 5 stars.and as for the pros and cons in CGW...i clearly remember a reviewer giving NFS IV:High Stakes a score of 5 stars and NO cons.an absolutely perfect game in their view.while the game was good...it was far from perfection.the bugs alone are proof of that .the CGW rating system is flawed.we have been following a blind leader.**

Then obviously your "venerable" CGW sucks like a hoover.

Jake
10-29-2003, 05:15 AM
**there is no such thing as a perfect game.no game deserves an absolute score.if done so...then that would mean that we have reached the limits of our creative and artistic abilities.beleive me..an art critic will never say that Michelangelo's David is a 'perfect' sculpture.like objectivity...perfection is also to be strived for...but not attainable..such is the irony..**

RIGHT ON. Exactly what im saying since the beginning. You seem to defend my main argument. !?

[....]


**the AG rating system is not flawed.its beauty has already been elaborated by erwin.see a score of 5 stars may mean an absolute score at other reviewing sites/magazines.but AG has an associated context with it.so a game receiving 5 stars is not the greatest game ever made...it is a game that is so good,it is worth having in your library.if another 'better' game has got 5 stars too..then it too must be in your library.as for which game is better if both have got a score of 5 marks..well the reviews clearly mark the pros and cons of the games don't they...see thats what i mean...here a game receiving a score of 5 stars have pros and cons.perfection is not bestowed upon it.**

Yes it is flawed. The main guy from this site even said that they wanna improve it because its obviously not all that.



It seems from these two paragraphs that you are, to a point, mad at Adventure Gamers for having a review system that leaves no room for the perfect game, a type of game you claim doesn't and will never exist. That's a little weird.

They're not out to grade the games as if they are works of art. The reviews are not reviews for reviews sake. That is a pretentious waste of time, self righteous wanking. AG's reviews are aimed at gamers interested in whether or not they should make a purchase, and whether or not game x appeals to their tastes. Adventure Gamers reviews only rate as high as "this game is amazingly good and you need to buy it!" because that's as far as the average reader/consumer is interested. Occasionally a reviewer may go above and beyond that for their own entertainment or some secret purpose of their own but in the end the main purpose of a review is to inform readers if the reviewer and the site thinks the game is good, or if it is crap.

"An art critic will never say that Michelangelo's David is a 'perfect' sculpture," but they would also never say "it's not perfect, it's 4.5/5," because that review system has nothing to do with art. The AG review system is meant for reviewing entertainment products and conveying advice to consumers (even if the review itself tells reviewers they should buy the game for its brilliant artistic acheivements - its still telling them whether or not they should buy the game). Maybe your approach to reviewing could apply to the actual artistic or technical merits of the games artwork, or its musical score, but you can't realistically think that that approach has any relevance to someone who is looking to buy the boxed game at a store? An art critic would never say David is a perfect sculpture, but they might say theres a perfect exhibition going on that is showing David (or at least "Worth paying to see more than any other exhibition in recent times, *****").

In addition to that, if the game actually is a freaking amazing new advancement in the genre that heaps one brilliant touch of innvation on after another, why the heck would anyone want to place the burden of conveying that onto a star or percent system? Read the damn review, especially if it'.s a 4.5 or 5 star review. Find out what made the thing so great, and judge for yourself if the game was simply amazing when it came out, or if its stood the test of time and is a classic.


Its because youve got bad taste - it happens.
Again, looking at professional reviews, both games averaged higher than Broken Sword.

I'd love proof of this, as you've cited it twice now as if it's fact, but haven't bothered to back it up.

I tried poking around Gamerankings.com but since all three of those games (BS1, Sam & Max, Indy: FOA) are old there aren't reliable ratings. However, they do have BS1 for the Gameboy Advance averaging 82% after averaging 39 review scores. Considering that's how professional reviewers scored the absolutely shitty bug-filled GBA version I can't imagine that the PC version scored significantly worse than the PC versions of Sam & Max and Indy: FOA (the Macintosh version of FOA has a 70% on gamerankings by the way, but that's averaging the scores of a whopping 1 review so that's hardly accurate). Please put your money where your mouth is.



Personally I think you're being a bit of a dick about all of this, mckracken.

This whole thread is incredibly dumb.

tabacco
10-29-2003, 05:43 AM
Heh... it almost sounds like you're from a developer whose game we didn't like, mckracken. Otherwise, it seems pretty odd that this thread represents every post you've ever made on this board.

Stinger
10-29-2003, 05:54 PM
This thread is just hilarious.

I think we're all being played here. I mean, there's no way anyone would actually say the things that McCrackhead is saying here and mean them.

"Its of course not the job of a reviewer to help people decide buying a game or not."

"But why would a potential reviewer need to take into consideration other tastes and preferences when judging something? You know whats that called in print puplications? Bribe."

"It is not subjective. We know from professional reviews (print) that these two games averaged to be rated higher than Broken Sword."

C'mon, you know that kinda stuff can't be said with a straight face. McCracken is just simply the most brilliant (and persistent) artist of sarcasm that we've ever seen on these forums. And for that, I take my hat off to him.

Of course, it is possible, I guess, that he's just simply too dumb for words. But somehow I doubt it.

"Its because youve got bad taste - it happens."

Comic genius.

Somebody wake me when this guy's gone.

nordic_guy
10-29-2003, 06:17 PM
I think you're missing a point here, Mckraken. Just because a game gets a five star rating, doesn't mean it's perfect.
But this is funny. Go on. :z

Erwin_Br
10-29-2003, 08:09 PM
the AG rating system is not flawed.its beauty has already been elaborated by erwin. see a score of 5 stars may mean an absolute score at other reviewing sites/magazines. but AG has an associated context with it. so a game receiving 5 stars is not the greatest game ever made...it is a game that is so good, it is worth having in your library.if another 'better' game has got 5 stars too..then it too must be in your library.

Yay! At least someone notices my (IMHO, valid) point! I was afraid I wasted my time.

--Erwin

mycroft
10-29-2003, 10:27 PM
Yay! At least someone notices my (IMHO, valid) point! I was afraid I wasted my time.

--Erwin

i'm the all-seeing eye after all... ;)

Tom_K
10-30-2003, 12:33 AM
Still waiting for the link to this famed website...

Erwin_Br
10-30-2003, 01:59 AM
Still waiting for the link to this famed website...

I wouldn't hold my breath. It's becoming clear we're dealing with an ordinary troller.

--Erwin

Curt
10-31-2003, 01:45 AM
Oh wow! I'm still trying to catch up in AG and somehow missed this beaut of a thread.

This is one of the best laughs I've had in ages. I can't even believe the likes of Dom, Ragou, Marek, Stinger, et al are even giving this McCracken fellow their time when clearly he's one of the biggest wind-up merchants in existence.

McCracken, how on earth have you managed to insult and look down upon so many people and yet still evaded the required "sod off!" response? (of course, that's only my subjective point of view, of course). What's your secret, eh? I'd say you don't have a secret - you've just been extremely lucky that you are posting on one of the most friendly and tolerant gaming sites on the net.

And to just be subjective once more for the hell of it, there is no such thing as the perfect rating system and reviews are always, yes ALWAYS, subjective.

Take off your blinkers, man!

remixor
10-31-2003, 01:52 AM
Sod off, Curt! ;)

mckracken
10-31-2003, 03:29 AM
I think we're all being played here. I mean, there's no way anyone would actually say the things that McCrackhead is saying here and mean them.

You were not played, Dick(ens)head. Yes I seem to come over sometimes a bit arrogant, Im aware of that, but I think most grownups are able to handle that. Messageboards tend to be insultive, but whats going in here, is embarrassing. (most tolerant gaming site on the net? - Sod off, Curt)

You people gotta learn to take critique, and not going spastic like complete msg board nerds, over a simple "reviews are bad on this site".

Curt
10-31-2003, 03:38 AM
You were not played, Dick(ens)head. Yes I seem to come over sometimes a bit arrogant, Im aware of that, but I think most grownups are able to handle that. Messageboards tend to be insultive, but whats going in here, is embarrassing. (most tolerant gaming site on the net? - Sod off, Curt)

You people gotta learn to take critique, and not going spastic like complete msg board nerds, over a simple "reviews are bad on this site".
I'm glad we've found your level.

Jackal
10-31-2003, 05:17 AM
Everyone here can handle arrogance. We just insist that at least a MINIMUM standard of intelligence warrants the effort. Your posts lack that. Your total inability to form a coherent argument has gone from frustrating to funny to just plain boring. Hell, I even tried to back you up on one of the few points that MIGHT have had merit, but you preferred to taunt people with no purpose whatsoever.

You're right about one thing, though. We DID overreact to a simple "reviews are bad on this site." Not because the responses were unreasonable; just because it's obviously a simpleton's opinion, and wasn't worth the time.

Marek
10-31-2003, 05:19 AM
This thread is not going anywhere so I'm just going to lock it.
PM me if for some reason you disagree with that decision.

tabacco
10-31-2003, 05:44 AM
Everyone here can handle arrogance. We just insist that at least a MINIMUM standard of intelligence warrants the effort. Your posts lack that. Your total inability to form a coherent argument has gone from frustrating to funny to just plain boring. Hell, I even tried to back you up on one of the few points that MIGHT have had merit, but you preferred to taunt people with no purpose whatsoever.

Heh... I imagined a little sign with a line on it saying "You must be more intelligent than this to post here."