Archived Guidelines
These guidelines are currently outdated and continue to be retained for archive
purposes only; the latest
guidelines should be used while reviewing.
Introduction
This article will provide a description of how to write quality reviews for
Age of Kings Heaven that are scored consistently between reviewers and are
helpful to both the file’s creator as well as the potential downloader.
I (Angel SpineMan) created the reviewing system almost three years ago and
it was first used for the original Age of Empires. It has been tried and
tested and I still believe that the 5 category system of Playability, Balance,
Creativity, Map Design, and Story/Instructions, covers all the areas that a
truly great scenario should excel in. Each category is scored on a range
from 1 to 5 with 5 being the best score and 1 being the worst. The total
score is achieved by averaging each category. The system is very effective
if it is used correctly.
At the beginning, my sole responsibility at Heaven was writing reviews. As
such, the reviews were very consistent since I wrote them all. However, over
time, I took on more and more responsibilities at Heaven and I no longer had
the time to write very many reviews myself. So, we got a small team of
reviewers together and I explained the system to them in detail. That
worked well for awhile but over time, some of the original team left and
new reviewers have been constantly added to the team to try and keep up
with the immense number of scenarios and campaigns that need to be reviewed.
Unfortunately, I have not able to give each of the new reviewers as much
guidance as they needed and the reviews have started to become very
inconsistent.
This inconsistency is very troubling since scores can vary widely
depending on who is doing the review. That’s not fair to the designer or
to the downloader. I know that there will always be some variation in scores
depending on the reviewer, but lately, the amount of variation is unacceptable
and this document will detail what I expect each score to represent. As the
Angel in charge of reviewing, it’s my job to ensure that reviews remain a
valuable tool for both designers and downloaders.
General Guidelines
The first thing to always keep in mind is that reviews serve a double
purpose. They are written for both the scenario designer and for the site
visitor who is considering downloading the file. As such, reviews need to
praise the designer for things that are done well and point out areas where
the designer can improve. Reviews also need to provide enough information
about the scenario so that potential downloaders will know if the scenario
matches their interests. Obviously, don’t give away the plot or reveal secrets
that should be discovered while playing, but let the downloader know what
the scenario is about, if it’s mainly fighting, or mainly rpg, or mainly
puzzles or a mix of everything. Whatever the case, a downloader should
know what to expect from a scenario after reading a review.
Try as hard as possible to avoid vague statements in reviews. Make sure
that your review answers more questions than it raises. Don’t ever say
something like “The first part of the 2nd scenario was good” or “the part
with the wolf could be improved” without providing further explanation. Try to
always include an example from the scenario to back up any points that you
make. If you are pointing out something to the designer that you feel could
be improved, try to provide some ideas that the author could build on. Do as
much as you can to help the designer improve his work.
All scenarios have good aspects and bad aspects. Try to always say at
least one good thing about any scenario you review and never, NEVER insult
a designer. They might not be as good of a designer as you but even if you
are handing out a score of 1.0, you still should never insult the designer.
Be honest about the scenario but make every effort to encourage the designer
to do better next time.
Always spell check your reviews. I’ve seen far too many reviews that
take points off a rating because of poor spelling yet the review itself is
full of errors. Don’t embarrass yourself – spell check your work.
Lastly, the review should contain a short explanation of why you scored
each category the way you did. This does not need to be lengthy, sometimes
a sentence is enough but other times, a paragraph for each category is
needed.
Category Scoring Guidelines
Before I detail what I expect for each category, there are some general
scoring guidelines to make note of. All of the categories are subjective,
some more than others, but try to be as consistent as you can with your own
scoring. Also, take special note of a few things that should NOT affect the
score of a scenario. These things should be noted in the review, but they
should not affect the rating scores in any way.
First is the length of a scenario, or how many scenarios are included in a
campaign. There is no rule that says a scenario must last more than 15
minutes or that a campaign must include at least 3 scenarios. The scores
should only reflect how good the scenario was while it was being played. A
great 5 minutes should score much higher than a mediocre 2 hours. No
reduction in score should be made based on the length of a scenario.
Second is the number of triggers in a scenario. This is simply irrelevant
to how good or bad a scenario scores. Lots of designers like to brag about
how many triggers their scenario has but if the triggers are poorly constructed
and don’t contribute to gameplay, they might as well not have any triggers at
all. A scenario does not need to have very many triggers to be a lot of fun
to play. Scores should be based on playing the scenario, not opening the
designer and counting the triggers.
Lastly, a scenario should not be penalized for not including special extras
like music files or custom ai files. These extra items are great if used
effectively and certainly can boost a score but a scenario should not be rated
poorly just because of a lack of extras. A scenario should still be able to
achieve a score of 5.0 even without using special extra files. The AoK design
tool is so rich with extras already that a designer should not be required to
use custom files if they can achieve their design goals using what is already
built into the design tool.
Category Scoring
Playability
Playability is probably the most subjective element of the scoring. It is
simply a gauge of how much fun you had playing this particular scenario. One
thing to look out for when reviewing is to only play scenarios that use a
style you enjoy. For example, if you hate playing RPG scenarios, don’t try to
review one since you are bound to not enjoy the scenario. Try to keep within
styles that you enjoy.
There really is no specific criteria on how a score is given in Playability
but there are quite a few things that can effect playability in a negative
manner. Trigger bugs, victory condition bugs and any other
playability-destroying bugs obviously can ruin a scenario’s playability.
Lag is another playability issue that a scenario can be marked down for.
If a player is ever confused about the next goal to accomplish, that’s a
playability problem. If a player can complete an objective in a way that the
author obviously did not intend to be possible (i.e. there’s a hole in a wall
that allows the player to skip half the scenario), that’s a playability
problem. Anything that adversely affects your enjoyment of a scenario can be
deducted from the Playability score.
Balance
Balance is also somewhat subjective since each player is a different
skill level and what might be perfectly balanced for one player, might be
way too easy or way too hard for another. As a reviewer, you must take your
own skill level into account when giving a balance score. A perfectly
balanced scenario should provide a challenge for a veteran player. Most people
who are downloading scenarios from the internet have at least played through
the campaigns included with the game and have a good knowledge of the game.
Most perfectly balanced scenarios should not be able to be completed without
the player losing a few times. If a player is able to complete the entire
scenario the first time, the scenario is probably too easy. On the other
hand, a player should not need to reload 15 times to get by a certain part
of a scenario. That is frustrating and the scenario is probably way too
difficult. The ideal scenario balance happens when a player gets stuck, but
he knows that it’s possible to complete the objective if only he did something
a little differently. A player should not win by luck, the scenario should be
constructed so that a player can learn from mistakes and use his skill to
complete the objective.
One important item to note about scoring the balance category for scenarios
where no fighting takes place, such as cut-scene scenarios, some puzzle
scenarios and some rpg style scenarios, is that just because the player cannot
die in such scenarios, that doesn’t mean the scenario isn’t balanced. You also
need to take the author’s original intent into account, giving the author some
benefit of the doubt. If the author never intended the player to face a
struggle to survive, then there’s no reason to knock down the balance score
if there isn’t any fighting. So keep in mind that you do need to take the
intent and goals of the scenario into account when scoring the balance
category, especially for those scenarios where fighting is not included.
Multi-player scenarios are reviewed a bit differently in terms of balance.
Each human player should start out in an equal position with equal starting
resources and equal starting units. Obviously, the players don’t have to
match exactly, but they should be balanced. The map should also be examined
to determine if all players have access to the same amounts of on-map
resources. There are a lot of creative ways that map designers can use to
make each player different, yet still balanced. If you choose to review
multi-player scenarios, it’s your job to ensure that each starting position
is balanced with every other starting position.
Creativity
This area is probably second in subjectivity behind playability. Creativity
is found in all aspects of a scenario, from trigger tricks, to map design, to
the story, to what units a player is given, to the objectives, to sounds used,
etc… Every aspect of a scenario factors into creativity. One thing to be
careful for is not to knock points off of creativity if the designer uses a
trick you’ve seen used in another scenario. There’s nothing wrong with
using the same trick that someone else used and no reason to deduct points
because of that.
Probably the biggest creativity factors are the starting position and the
victory conditions. For example, any scenario that starts with a TC and
three villagers with a conquest victory condition is simply not very
creative. The farther a player gets from a random-style scenario, the better
the creativity score.
Map Design
Map design is one of the few categories that’s very easy to define and
give a rating to. I have pretty clear-cut rules on how map design is scored
and this is how it should work. A random map is a 3. All a designer needs
to do to score a 3 is to use a generated random map. Random maps look good,
they function well and there’s nothing wrong with using a random map in a
scenario, but it’s just average. From that basis, it’s easy to figure out
where scores of 1, 2, 4 and 5 come from.
A rating of 1 is for a pathetic map… these usually consist of large
blank areas with lots of square areas and straight lines. These maps look
completely unrealistic and are quite unattractive. A rating of 2 is
somewhere between a pathetic map and a random map.
A rating of 5 is for an outstanding map with lots of special details and
concentrated effort to make the map much better than a random map could
possibly provide. Obviously, a rating of 4 is given for maps that are
slightly better than a random.
One final note on score map design… only the portion of the map that
can be seen during play should be scored. If there are large empty areas
that a player never sees, that should not affect the map design rating.
Story/Instructions
This is another pretty clear-cut category. If there is no story or
instructions, the score is easy… it’s a 1. If there are instructions
but no story, the max score is a 3. If there is any story at all, the
rating goes up to a 4 and if the story is really good, the rating can be a
5. If the instructions are wrong, misleading or confusing, the rating goes
down. Also, keep in mind that in Age of Kings, the instructions and the
story goes far beyond the pre-scenario instruction screen. Often the story
is continued throughout the scenario by using trigger events to move the
story along. Also, since objectives can change in the middle of a scenario,
the quality of the instructions must be judged throughout the playing of the
scenario.
Some other guidelines on scoring this category: An introductory bitmap
is a nice touch and a good image can often raise the score, however, an
introductory bitmap is not required to score a 5. It certainly helps, but
it’s not an absolute requirement. Hints and History can also be judged
here… these two areas are not required, but they can also help boost a
scenario’s score. While a bitmap, hints and history are not required, it
would be difficult to give a rating of 5 if all three areas are missing.
The rating should not be effected based on whether the story is fictional
or historical. It doesn’t make a difference as long as there’s a story that
draws the player into the scenario.
The last item that factors into the rating of the story and instructions is
grammar and spelling. A designer should be diligent in this area of his
scenario since it’s very easy to copy the text into a word processor and spell
check the instructions. There’s no excuse for having spelling errors in a
scenario… it simply shows a lack of effort on the part of the designer. The
only exception I make is for designers whose primary language is not
english… I am usually quite a bit more lenient with them.
Final Thoughts
The above instructions are specifically for writing scenario and campaign
reviews. However, we are now allowing reviews of all file types available
for download so you can rate and write reviews on Mods, Random Maps, AI Files,
Utilities and Recorded Games! However, these reviews will not use a 5
category system, instead you will give a single rating to the file. When
you write your review, simply include enough information to explain why you
gave the rating that you did.
Now that you’ve made it all the way through this document, you are ready
to write reviews! If you have any further questions about what is expected
of you, please direct them to Angel SpineMan. Below, you will see the links
that you can use to submit reviews. The review form will send an email to me
so I can check your review to see if it follows the instructions given above
and provides adequate information about the scenario. If the review passes
my inspection, it will be posted on the site. Good luck and thanks for your
interest in writing reviews!