You must be logged in to post messages.
Please login or register

Modding Discussions
Moderated by Alexastor, MosheLevi, Mister SCP

Hop to:    
loginhomeregisterhelprules
Bottom
Topic Subject: TWC Tutorial: The most important Tutorial of all
posted 08-11-07 11:15 AM EDT (US)   


The most important Tutorial of all
by Young Cunhambebe, messenger of the Tupi




Cunhambebe says: Hello, friends. This tutorial will teach you the very, very basics of modding: file replacement. But STOP right now! You won't mod without these stuff:
  • Searched on Google and downloaded AoE3ED.
  • Read my List of Modding No-nos, that will tell you everything that you can't do when modding.

    Cunhambebe says: Learning how to replace files in fundamental when modding. It’s the most basic knowledge and strangely, newbies have a lot of difficulty at it. So, this tutorial will teach you how the game works, and how you’ll mimic that system to make your own job works as an original one.
    The game works with 3 basic kinds of files: Text, Image and Sound. Text is by far the most important, since that’s the file that tells the game to open the other two types. However, you won’t see anything in the game without images, and you won’t hear anything without sound. Executable icons, cursors, .dlls and other files are not important for mods – it’s hard to work on them since most are hardcoded in the .exe.

    Cunhambebe says: What’s “hardcoded”? It’s a text, image, sound or anything else that you cannot edit. It’s not viewable in any way. Those information appears in 2 situations: when it’s not that encrypted, it’s revealed by hacking tools. When it is, revealed when the Ensemble Studios decide to release the game source code – and we know this will only happen on St. Never’s day. An example of hardcoded text: the one that tells the game to open the data folder. No matter what you extract, open, edit, the game will always open that folder. We know something tells the game to do that, but we never see it. So, it’s hardcoded. When your codes don’t work (like priest conversion), that’s because the code of beast tame is “linked” to its original name. When we duplicate that ability, it doesn’t work. When we try to edit the file that links name and ability, we don’t find it anywhere. It’s hardcoded, encrypted, hidden.

    Cunhambebe says: The game reads many types of files, let me list them:

    Image:
  • .bmp – there’s a single file in the entire game with that type. It’s the splash image, the first AoE3 logo that appears when you double-click the game’s icon.
  • .ddt – encrypted form of .tga. Holds all images used in the game, textures, UI elements… Has two important properties: transparency and player color, both defined by the alpha channel, an invisible layer that is revealed by the best image editors – like Photoshop. A single alpha channel is responsible for both properties. It’s the 3D model or UI element that will tell if the marked areas will be transparent, player colored or nothing special.
  • .cur – cursors. Most cursors can be easily revealed by a hacking tool when opening AoE3, but I never tested (and needed) how to replace them.
  • .ico – icons, AoE3 and expansion pack icons are found inside the .exe, also revealed by a hacking tool.
  • .gr2 – the 3D model. Can be static models or animations. The model’s polygons are hardcoded to react in 3 different ways to the .ddt’s alpha channel: It’ll be transparent, player color or nothing special.

    Sound:
  • .wav – the most basic sound file type. Its quality is the highest possible, but files are large. All sound you hear in game is .wav.
  • .mp3 – the type used for songs. Their quality is slightly worse than .wav, but their size is way smaller. It’s possible to get a 3mb .mp3 and a 50mb .wav song!

    Text:
  • .xml – It’s a kind of code. Easy to learn, just copy-and-paste information. It’s read really fast by the. All modern games are using it. Can be easily edited by the Notepad, so don’t download any program. See an example of .xml code:
    <user>Kastor</user>
    <age>15</age>
  • .xmb – encrypted .xml. You won’t find many .xmls in the game, most are encrypted. This file type cannot be edited by anything, but you can convert it into a .xml by using AoE3ED.
  • .xs – it's completely different than a .xml. Holds random map information.

    Cunhambebe says: After a few months playing the game, you’ll notice that the British are able to train the musketeer. But how? It’s a complex group of codes, images and sounds that you’ll have to learn in order to mod your own unit. After a few tries, you’ll be able to add an unit to the game in less than 5 minutes – without a new texture, or course. Let’s see how the musketeer is constructed:

  • The game is hardcoded to load proto.xml.
  • Proto.xml tells the game about the existence of an unit called “musketeer”.
  • The musketeer is controlled by a group of tags, just like shown above. One of its tags tells the game that the unit loads a .xml that controls its animation – musketeer.xml
  • That file tells the game which model and which animations belong to the musketeer.
  • Musketeer.gr2 is loaded.
  • That file tells the game it has a texture; Musketeer.ddt is loaded.
  • Musketeer.xml tells the game that model has animations. Many of them are loaded, but they don’t have textures – they’re not models, although they are .gr2
  • The unit is fully loaded, proto.xml reading is complete.
  • The game is hardcoded to load all proto’s unit’s soundfiles. Musketeer’s one is named musketeer_snds.xml – The game is defined to automatically open any file with a name that matches ones found in proto.xml. So, “cuteymusketeer” will never open, unless you have that name in proto.xml
  • Musketeer’s sound file points to some soundsets like “BritishMusketeerSelect”
  • Soundsets.xml is loaded. BritishMusketeerSelect points to some .wav files.
  • .wav files loaded.
  • The game is hardcoded to load techtree.xml.
  • There will be a tag that points to the british.
  • There will be a tag that tells the game the british are able to train the musketeer.
  • When proto.xml was loaded, the “barracks” learned that it’s able to train an unit called “musketeer”.
  • Unit is fully loaded.

    It’s also important to know that the game will read those files only once. Doesn’t matter if you’re using control+tab to mod the game while it’s open, it’s not going to work until you restart it. There are few exceptions, the following files can be changed while the game is open, and the effects will appear as soon you click control+tab again. They are:

  • The savegame files, useful for adding experience points;
  • The homecity files;
  • And most important of all, all textures are reloaded when you return to the game! If you made any change, you can quickly test it!

    Cunhambebe says: But what the heck is file replacement? Well, this tutorial will teach you how to add anything to the game, although I’ll not teach you how you’ll make each file, that’s for other tutorials.

    I’ll use a different example this time, the turkey. Everyone with the xpack noticed that the old AoE3 peacock called “turkey” became a real turkey. But how? Did you know that you still have the old peacock? The new model and texture didn’t delete the old ones, but replaced them.

    The game cannot delete anything because most files are found inside “zipped” packs, called .bar. They work exactly like Winrar compacted files. Each AoE3 folder has its .bar file. It’s not only used to turn over 1000 files into a single one, but also to make them load faster. Bar files are protected from newbie modders, thankfully. Since you cannot edit anything that’s inside them, you have to place something outside it, and that’s what TWC does: brings a new .bar file and its turkey model is loaded instead of old AoE3’s.

    AoE3 is told to open turkey.ddt
    It’s told that file is found inside Art\units\animals\turkey\
    What will it do?



    If you look yourself, there are no folders with those names. So, the game will look inside Art’s .bar file:



    Pay attention: No matter if the game opened art2.bar or not, it’ll still look for the same file, in the same folder. The file’s address doesn’t change!
    It’s important that you notice something: game searched inside the real folder, then inside the .bar. If you ever wanted to replace a file, all you needed to do was putting a file with the correct name in the correct folder. It’ll be read before .bar, and you’ll see it in-game.
    With the arrival of TWC, how is the turkey texture loaded?



    As you see, TWC will look for that file in a folder; then, inside art4.bar. Then, inside art2.bar. It’s found first in art4, so that’s the image that’ll appear in game. I repeat: If you put a frog called “turkey.ddt” inside those folders that don’t exist (you’ll have to create), the little toad will be loaded instead of the usual texture.

    Cunhambebe says:

    Let’s talk about a more complex file, civs.xml. This will be an interesting experience. I’m considering a game with all patches installed.

    AoE3 is told to open civs.xml
    It’s told that file is found inside Data\
    What will it do?



    When you download a patch, an extra .bar file appears: datap.bar or datapx.bar. Every time you see an “x” is because that’s the expansion. As you see, the game will look for it in the folder; then, inside dataPX.bar; then, inside data2.bar (comes with xpack); then, inside dataP.bar; finally, in data.bar. The patch .bars are found outside the data folder and they contain all kinds of files. So, even sharing the “data” name, they’re different than data.bar and data2.bar. If no .xmls are found, the game will look for its encrypted form, .xmb:



    Whew, 4 files were found. What will the game do? Load the first one found. If you ever need to create your own civs.xml file, now you know that if you don’t put it in the correct folder (data) the game will directly look for it inside the .bars.
    Why is the dataPX.bar read before the other files? It must be read before the others, otherwise the game will load an old version of it, thus, no patch effect. Correct? There’s an order of .bar opening. I’ll try to list them to you, including the possible .bars that will come with the next expansion, TAD. Remember that art.bar comes from modpacks only:

    Game > data folder > dataPY.bar > data3.bar > dataPX.bar > data2.bar > dataP.bar > data.bar > art folder > art.bar > art5.bar > art4.bar > art3.bar > art2.bar > art1.bar > sound folder > soundXML3.bar > soundXML2.bar > soundXML.bar > sound3.bar > sound2.bar > sound.bar

    Although it says data files are read before the art ones, you cannot have textures inside data folder. That’s because the game is hardcoded to look for textures inside the art folder. Also, you cannot have songs anywhere but inside the sound folder.
    To finish my tutorial, how does a modded file behave?



    Extra tips:
  • .xml is read before .xmb. If you have both files in the same folder, .xmb will be ignored.
  • All textures in the game have the following dimensions: 2x x 2y. Most times it’s a 64x64 (x = y = 6). You cannot create textures with random numbers, they won’t appear.
  • Don’t forget you don’t need to restart the game to test textures!
  • You can, but you don’t need to convert .xmls into .xmbs. I think .xmb are read faster, but it takes too long for us to keep converting it from .xmb to .xml and vice-versa.

    Lord Tahattus - Register yourselves at the WotTA forum!
    WOTTA WEBSITE!
    War of the Triple Alliance Beta!
    Flags for new Civilizations, The Asian Dynasties, Knights and Barbarians and Napoleonic Era
    TWC Tutorials: Basics of Modding - The List of Modding No-nos - Adding a Nation - Music and Sounds - Initilization Failed

    WotTA back to active!

    [This message has been edited by Lord Tahattus (edited 08-25-2007 @ 01:02 PM).]

  • Replies:
    posted 08-11-07 11:32 AM EDT (US)     1 / 7  
    Very helpful, Thanks

    [This message has been edited by Gleklu (edited 08-11-2007 @ 11:32 AM).]

    posted 08-11-07 01:48 PM EDT (US)     2 / 7  
    # Text:
    # .xml – It’s a kind of code. Easy to learn, just copy-and-paste information. It’s read really fast by the. All modern games are using it. Can be easily edited by the Notepad, so don’t download any program. See an example of .xml code:
    <user>Kastor</user>
    <age>15</age>
    # .xmb – encrypted .xml. You won’t find many .xmls in the game, most are encrypted. This file type cannot be edited by anything, but you can convert it into a .xml by using AoE3ED.
    And the XS files?

    [This message has been edited by alvaro_qc (edited 08-11-2007 @ 01:49 PM).]

    posted 08-11-07 03:35 PM EDT (US)     3 / 7  
    So if you put a file in data called civs.xml and you don't convert it back to .xmb, it will be read as .xml? I didn't know that and have always been converting...


    peugeot407

    |> Robert Faber (aka peugeot407)
    |> Humble Creator of: Knights and Barbarians, Danish Civ Mod, Several Tutorials and the Tutorial Database
    |> November 2007 Modding Contest: 2nd place (me: 360 pts, Captain_Soanso: 365 pts!)
    |> January 2008 Modding Contest: first place!

    When in doubt, flat out! (Colin McRae)
    posted 08-11-07 10:39 PM EDT (US)     4 / 7  
    lol yes, all my files are in XML It's too boring re-making them XMBs
    posted 08-12-07 06:27 AM EDT (US)     5 / 7  
    I think converting to .xml.xmb has a few advantages. You can never accidently change anything in them before converting, so it serves as a sort of back-up. When you have a lot of files the file size with .xml.xmb will be considerable smaller than with .xml although this doesn't appear to be so at the start but that's because FileConverter is flawed but when zipped it does work. Not to mention that while converting the file automatically gets checked for errors.

    || argalius.elpea.net
    || Cherub at AoE3H
    || In honor of FlipBizcut
    || Mod: The Age of Crusades

    [This message has been edited by Argalius (edited 08-12-2007 @ 06:29 AM).]

    posted 08-12-07 06:42 PM EDT (US)     6 / 7  
    xmbs aren't encrypted. They are a binary representation of xml files, kind of preprocessed xmls.
    This speeds up parsing them!
    You can find more information on the format at my site.

    So converting it back to xmb is worth the time.

    I think a commandline tool to convert all xml files in a folder into xmb would be very handy here.

    [This message has been edited by Viruzz (edited 08-12-2007 @ 06:43 PM).]

    posted 08-25-07 01:04 PM EDT (US)     7 / 7  
    "encrypted" in this tutorial means you can't open it with notepad, just that. But thanks for the comment.

    Lord Tahattus - Register yourselves at the WotTA forum!
    WOTTA WEBSITE!
    War of the Triple Alliance Beta!
    Flags for new Civilizations, The Asian Dynasties, Knights and Barbarians and Napoleonic Era
    TWC Tutorials: Basics of Modding - The List of Modding No-nos - Adding a Nation - Music and Sounds - Initilization Failed

    WotTA back to active!
    Age of Empires III Heaven » Forums » Modding Discussions » TWC Tutorial: The most important Tutorial of all
    Top
    You must be logged in to post messages.
    Please login or register
    Hop to:    
    Age of Empires III Heaven | HeavenGames