As a long-standing member of AoMH, I feel it my duty to the community to speak my mind about the site, its systems and its community.
For a while, I have looked at the download section with disappointment. I feel the entire section needs a revamp. Whilst i know in terms of structure it's a lot harder to reorganise it from a moderation level, however, it's the systems such as the Best Files, Official Reviews, and general moderation which needs to be rethought.
Not to say that the moderators do a poor job, it's perfectly acceptable for them to miss poor reviews here and there as they've got plenty else on their plate. However, isn't that surely what the official review team is there for?
My comments regard mainly the campaign and single-player categories. Forbidden and Terminator Ace appear to do their specialist categories justice, however, Papaya for example, and no disrespect to him, hasn't posted a review since January 2007, and his last review of a campaign or scenario was ofEmperor by Celsus on 2nd October 2006; empireofages has given just one review this year in his category of recorded games; Alex (Pepsidude) has given just two ratings of scenarios in the last year and none since April; Khan & Steak, the official reviewers' most prolific reviewer, has given just four ratings this year and none of them are single-player scenarios or campaigns...
What is possibly a cause of the inactivity of these reviewers is that they have moved on to bigger things or have reduced their activity on the forums. But is this good enough? This sort of thing should be kept track of, and the positions reassigned to more active and effective reviewers. The inactivity of these staff members means that more and more poor reviews are getting through the fish net, even when they are reported. To use recent reviews as an example, MasterBeef'sChronicles of Kamos has received a review on 13th July 2008 which is clearly in violation of the guidelines, yet it has gone unnoticed and uncensored. Even members have been amazed by this saying:Story_Master's, Sniper1988's #1, Sniper1988's #2 and eyecandy of doom's reviews.
What is even more shocking and frustrating for the hard working designer is that thetop-rated scenario on the Download Section's Best Files database is there on a rating from an official reviewer and even that review falls short of what is acceptable as a decent review. These scenarios subsequently get hundreds of downloads whilst other scenarios are overlooked.
So what do I propose?
In essence the staff positions on the official review board need to be reassessed. There needs to be a standard that the official reviewers must adhere to in order to remain in their positions, such as quality of reviews and frequency of reviews. When non-staff members such as PurpleWorm, Baske and Banoffee appear frequently on the 'recent reviews' page having posted decent reviews it amazes me that this has gone on for so long.
What should they do?
Official reviewers should be required to review at least two files per month in their particular field (this number could vary slightly from category to category). These reviews should set a standard from which other member can follow. They should go above and beyond the guidelines. The three line rule is redundant. Just because somebody writes three lines doesn't means it's three lines of any substance, why should that review stay when a perhaps slightly more informed review that didn't post three lines in every category (although in my opinion you should post at least a decent paragraph per sub-rating) is deleted.
They should reassess the files on the Best Files page, check if the reviews still apply. Deleted, alter or re-review the files to reflect the modern perspective. Perhaps the official reviewers could dig up a monthly 'featured file' that they recommend users to download. I also believe every file in the 'Top-Rated' section should be rated official, too many times have I seen poor productions listed there.
How many should there be?
The single-player and campaign sections should have at least two official reviewers and perhaps at least one specific AOM Vanilla reviewer. Multiplayer should have at least two, recorded games at least one, mods and random map script at least one, and miscellaneous at least one. There should also be a head reviewer to 'watch the official reviewers' (as Guardian_112 once put it).
What about the Download Section itself?
The way the files are ranked is ineffective. Files which have been rated an average of 4.8 by 8 users for example should appear higher than files which have been rated 4.8 by one user. I would also say that file rated 4.8 by 8 users should appear higher than a 5 by one user.
I'm not attempting to undermine the authority here I just feel that this issue needs to be addressed and has needed to be addressed for a long time. Perhaps a Community-Staff Liaison Committee should be set up to improve site services and facilities and to improve communication between the two. I understand that this isn't a state, the internet isn't a democracy; however, what benefits us makes us happier and in turn benefits the site.
_____________________________________
ROME: 78 BC | | | | | THE LOST RUINS
Rome in your hands | Of Ancient Mesoamerica
______________________________
AOMh Member since 2004
For a while, I have looked at the download section with disappointment. I feel the entire section needs a revamp. Whilst i know in terms of structure it's a lot harder to reorganise it from a moderation level, however, it's the systems such as the Best Files, Official Reviews, and general moderation which needs to be rethought.
Not to say that the moderators do a poor job, it's perfectly acceptable for them to miss poor reviews here and there as they've got plenty else on their plate. However, isn't that surely what the official review team is there for?
My comments regard mainly the campaign and single-player categories. Forbidden and Terminator Ace appear to do their specialist categories justice, however, Papaya for example, and no disrespect to him, hasn't posted a review since January 2007, and his last review of a campaign or scenario was of
What is possibly a cause of the inactivity of these reviewers is that they have moved on to bigger things or have reduced their activity on the forums. But is this good enough? This sort of thing should be kept track of, and the positions reassigned to more active and effective reviewers. The inactivity of these staff members means that more and more poor reviews are getting through the fish net, even when they are reported. To use recent reviews as an example, MasterBeef's
Other examples of poor reviews which have slipped through the censorship committee areQuoted from CoS_Infurn4tr1x:
i'm surprised that none of the staff have made it to invade here
What is even more shocking and frustrating for the hard working designer is that the
In essence the staff positions on the official review board need to be reassessed. There needs to be a standard that the official reviewers must adhere to in order to remain in their positions, such as quality of reviews and frequency of reviews. When non-staff members such as PurpleWorm, Baske and Banoffee appear frequently on the 'recent reviews' page having posted decent reviews it amazes me that this has gone on for so long.
Official reviewers should be required to review at least two files per month in their particular field (this number could vary slightly from category to category). These reviews should set a standard from which other member can follow. They should go above and beyond the guidelines. The three line rule is redundant. Just because somebody writes three lines doesn't means it's three lines of any substance, why should that review stay when a perhaps slightly more informed review that didn't post three lines in every category (although in my opinion you should post at least a decent paragraph per sub-rating) is deleted.
They should reassess the files on the Best Files page, check if the reviews still apply. Deleted, alter or re-review the files to reflect the modern perspective. Perhaps the official reviewers could dig up a monthly 'featured file' that they recommend users to download. I also believe every file in the 'Top-Rated' section should be rated official, too many times have I seen poor productions listed there.
The single-player and campaign sections should have at least two official reviewers and perhaps at least one specific AOM Vanilla reviewer. Multiplayer should have at least two, recorded games at least one, mods and random map script at least one, and miscellaneous at least one. There should also be a head reviewer to 'watch the official reviewers' (as Guardian_112 once put it).
The way the files are ranked is ineffective. Files which have been rated an average of 4.8 by 8 users for example should appear higher than files which have been rated 4.8 by one user. I would also say that file rated 4.8 by 8 users should appear higher than a 5 by one user.
I'm not attempting to undermine the authority here I just feel that this issue needs to be addressed and has needed to be addressed for a long time. Perhaps a Community-Staff Liaison Committee should be set up to improve site services and facilities and to improve communication between the two. I understand that this isn't a state, the internet isn't a democracy; however, what benefits us makes us happier and in turn benefits the site.
_____________________________________
Quoted from AnnoDominius:
When i said the three line system is redundant i didn't mean that it should be lowered. I believe the rule should be removed as forcing people to write three lines just to keep a substandard review is an ineffective system and makes it difficult for reviews to be deleted when they aren't really up to scratch.
As PurpleWorm says, one should be aiming at writing much more than three lines per sub-category but they shouldn't be thinking all the time "I must write at least three lines of rubbish or this will get deleted", that sort of mentality doesn't help. To quote PurpleWorm again, the review isn't only about saying whether or not you liked it, it's about suggesting improvements, alternatives, and other things that could be done to make the rating higher.
I also find it frustrating when a review for example gives a review like the following:Playability: 4
Very enjoyable, its really fun to move around the map and complete the tasks. There is a great ambiance about the scenario. I really liked all the elements you included. It kept you playing right to the end
As no doubt you can see, a review with this text would have been kept as it fulfilled the 3-line rule, however, there is no substance to it. Great, so you enjoyed it... what did you enjoy, so far as the example review is concerned I can't tell if you've actually played the scenario at all. Why did you only give a 4? you've mentioned nothing negative so surely you should be giving a five here, no?
What needs to be done is to explain where the designer got each of the 5 points. You don't need to be pedantic about it (like saying you got your first point because...) but what would be a much better review would be the following:Playability: 4
Immensely enjoyable to play. There were no bugs that affected the gameplay so I have no reason to take off marks there. However, I did feel that the missions got slightly repetitive by the end of the scenario. You managed to hold my attention throughout the scenario by including many 'life' factors which made the scenario feel realistic, such as the citizens of the town going about their daily business and you got bonus points for sending them back to their houses when it got dark, a good use of the day/night cycle here. Overall, a smoothly playable map albeit repetitive. Try and vary the objective to improve.
You can see here that the reviewer is stating exactly why they have rated it a 4, detailing any bugs, general enjoyment, extra features, etc and using examples to back up these comments. If you: 1) State how much it fulfilled the category; 2) state if their were any bugs or errors; 3) detail something you really enjoyed; 4) detail something you didn't think worked; and 5) summarize by suggestive how the rating can be improved, one can easily write 5 or 6 sentences within a couple of minutes. A decent review should take roughly 10 minutes to write (2 minutes per category).
So you can see here that there really IS 15 lines of information to write, if you check out some of my own reviews you can see that at no point am i talking about something else, unless you count improvement suggestions as going off-topic. Some of my reviews have had a word count of 700+, all of relevant information.
However, i concede, I am not expecting a sudden change. I am optimistic that as one of the oldest members still around these forums my views will be taken with a degree of respect. But it is the COMMUNITY that has to band together and force a change to happen.
Rome in your hands | Of Ancient Mesoamerica
______________________________
AOMh Member since 2004
[This message has been edited by AnnoDominius (edited 08-24-2008 @ 09:46 AM).]