You must be logged in to post messages.
Please login or register

Tech Support
Moderated by Aurilux

Hop to:    
loginhomeregisterhelprules
Bottom
Topic Subject: Game crashing after long playing time (with some debug info) - FIXED
posted 08-30-16 10:37 AM EDT (US)   
The game is crashing after a long time of playing (typically in an 8-player map with 7 toughest AIs, the game will crash after two hours if none of the AIs die early. All updates are installed manually (European version) from http://rol.heavengames.com/cgi-bin/forums/display.cgi?action=ct&f=2,920,,10.

Below is some info I found, it would be really helpful if someone good at programming can give me some help. I have been trying to come up with a fix by myself but have found it pretty hard to accomplish without help. (Update5: a fix is finally finished, go to UPDATE5 section below)

For anyone that is willing to help me, I can provide a map that will crash in one and a half hours (With some extra settings, I can provide a save data that will crash in 10 seconds).

------------------------------------------------------------
The reason of crashing:
(1) The game keeps track of a unit counter; the counter increases by one for each unit created. A grunt unit is considered as 9 units.

(2) There are many codes that uses the counter as an address reference, doing something like
mov ecx, word ptr[eax+edx*4] (edx is the unit counter)

If I translate into C code, it might be something like
Unit* unit = unit_pointer_array[count];

(3) The counter is stored as a signed short int (2-bytes), so when the count reaches 0x8000 (which is 32768), the array access would become a negative index and hence address violation which causes the game to crash

(4) Normally the maximum unit possible to exist in the game is 300 x 8 x 9 = 21600 plus some neutral units, which should not exceed 32768; but for some reason the memory for dead units are not re-used, the game creates new unis instead of using an empty-ed spot for units that are dead.

To fix the bug:
(1) Unortunately, changing the counter to be handled as an unsigned short int will not work; the crash due to negative array index can be avoided but any new units created after the count reaches 0x8000 will not appear, and those new units will overwrite memory which will cause the game to crash around two minutes later.

(2) As long as I can think of, the only possible way to fix the bug is to handle creation of new units such that memory for dead units are re-used.

(3) Fortunately, the function that creates new units actually loop through all previous units and it seems to check whether a unit is dead or not; However somehow this process doess not cause the loop to end early; It is hard to describe in words so I would show the pseudo-code in C at the beginning of unit creation:


void create_unit() {
short int new_count = 1;
for(i = 0; i < count; i ++) {
if(unit_info[i].dead == true) {
do something and call some function
if(some conditions satisfied) {
return;
}
}
new_count = new_count + 1;
}
//new_count == count + 1 at this point
count = new_count;
do actual unit creation stuff(allocate memory .etc)
}

For some reason, the for loop always executes to the end and the new count is always the previous count plus one for each new unit created. I am kind of stuck at this point what to do next and I will really appreciate if someone can give help.

------------------------------------------------
UPDATE:

I have come up with a partially working fix.

Steps:
(1) Open the legends.exe in binary/hex editor.

(2) Go to address 0x13CD25, you should see "66 83 79 7E 00 75 4C" in this order. Change the 7E to 59 and 4C to 54; it should become "66 83 79 59 00 75 54" now.

(3) Go to address 0x13CD83, you shuold see "7C B3" in this order. Change them both to 90, it should become "90 90" now.

I have confirmed the fix is working for my previously crashing save data and new units are created properly.

What the fix is doing:
The fix changes the C pseudo-code above to the following:

void create_unit() {
short int new_count = 1;
for(i = 0; i < count; i ++) {
if(unit_info[i].dead == true) {
break;
}
new_count = new_count + 1;
}
//new_count == the first dead unit index or count + 1 at this point
count = max(count, new_count);
do actual unit creation stuff(allocate memory .etc)
}

Problem:
However, this fix is not theoretically perfect, although I have not been able to test out such situation, any healing spell might not be able to revive dead squad member(s) of grunt unit.

For example, if my imperial musketeers was injured and there are only three squad members left, using the resource dominance healing spell might not be able to restore the unit to 9 squad members. The behavior of the fix in such situation is unknown.

It is going to be really hard to make the grunt reviving thing working properly with this fix. The two main issues are:

(1) I will need to increase the length of function which is practically impossible for my knowledge level, I need help from experienced hacker to tell me how to do code injection properly.

(2) I will need a way to distinguish a dead grunt member from a whole dead squad. I know how to figure out where the unit data is stored but more analysis is needed to distinguish these two types.

There is apparently another problem that non-units are sometimes used, causing an in-game error message which will terminate the game, so I will have to analyze the unit fields in depth either way... Maybe there are other objects that is created by the same function as well.

------------------------------------------------
UPDATE2:

I have fixed the non-unit error message. Do the following steps instead of the original fix:

(1) Open the legends.exe in binary/hex editor.

(2) Go to address 0x13CD25, you should see
"66 83 79 7E 00 75 4C 8B C6 83 E8 00 74 0B"
in this order. Change it to
"66 83 79 59 10 75 4C 8B C6 83 E8 00 74 4D"

(3) Go to address 0x13CD83, you shuold see "7C B3" in this order. Change them both to 90, it should become "90 90" now.

I have also confirmed that the healing will indeed not revive the dead squad members; furthermore, it will treat the next created unit as the "revived squad member" and force that unit to join the squad. I will try my best to fix this but it could be really hard.

------------------------------------------------
UPDATE3:

I managed to inject a new function which is called to determine whether break from the for loop or not inside legends.exe with relatively large maximum length restriction. Therefore the only issue left is to distinguish a dead member within a squad and a fully dead squad. This still seem to be very hard though.

------------------------------------------------
UPDATE4:

Good news.
I have found out that grunt squads are stored in linked list manner; theoretically I can traverse the list and find out whether all members are dead or not. However depending on the dying order it seems that the last dead member in the list won't have the link information so I will need to do more debugging.

------------------------------------------------
UPDATE5:

FINALLY, I have come up with a fully working fix. With this fix theoretically (since amount of testing done is perhaps not enough and I cannot guarantee unexpected bugs do not exist) one can play a game infinitely long without crashing.

Below is a list of diff, each line has the format:
00AAAAAA: XX YY
where 00AAAAAA is the address when opened in hex editor, XX is the original byte and YY is the byte after change.

[Content Removed, see UPDATE6]

What the fix is doing:
The fix changes the C pseudo-code above to the following:

void create_unit() {
short int new_count = 1;
for(i = 0; i < count; i ++) {
if(myfunction(i) == 0) {
break;
}
new_count = new_count + 1;
}
//new_count == the first dead unit index or count + 1 at this point
count = max(count, new_count);
do actual unit creation stuff(allocate memory .etc)
}

int myfunction(int i) {
//return 1 -> skip the spot
//return 0 -> use this dead unit spot for next new unit
if(unit_info[i].dead != true)
return 1;
if(unit_info[i].isGrunt != true) {
//there is a global counter, separate from the unit counter,
//if keep increasing we cannot build any more buildings after some time,
//still better than crashing but I decide to make it perfect, so I decrease it by one (which will be increased back shortly) if dead unit spot is re-used
global_counter -= 1;
return 0;
}
//grunt is stored as a linked list in alive -> dead order
//so if anyone is alive then the head (first element of the list) is alive
int i_head = unit_info[i].linked_head;
if(unit_info[i_head].dead != true)
return 1;
global_counter -= 1;
return 0;
}

UPDATE6:
The fix in UPDATE5 failed to use the dead spot of some grunt squad members under certain condition. I have fixed the issue and this is the latest fully working version. Since the fix is getting long I put the last chunk of memory as one line.

[Removed, see UPDATE7]

I have also uploaded the patching program here:
https://github.com/rc41392/rolcrashfix/blob/master/apply_patch.exe

How to use:
(1) Make sure you have made file extension visible in your Windows OS settings (i.e, such that "legends.exe" is NOT displayed as "legends" without the ".exe").

(2) Copy the "legends.exe" to a safe folder and rename it as "legendsfix4.exe.bin"

(3) Create a new file in the same folder called "fix_diff.txt"

(4) Open "fix_diff.txt" and paste the fix above (all lines with AAAA: XX YY... format) into it.

(5) Put apply_patch.exe into the same folder.

(6) Run apply_patch.exe

(7) There should be two new files named "log.txt" and "legendsfix4.exe.bin2" created.

(8) Open the "log.txt" and check there is nothing failed.

(9) Copy the "legendsfix4.exe.bin2" back to your RoL root directory and rename it back to "legends.exe". Everything done.

UPDATE7:
Apparently there are two different values to indicate a dead unit [Difference is not confirmed yet, but maybe the time after death]; the fix in UPDATE6 still crashes around 1.6 times longer (crashes at 2:30 for a map that usually crashes at 1:30)

The newest fix that checks both values is posted here. This one is confirmed to not crash in 12 hours using the same map (I think it will probably never crash now!). Use the same patching process as posted in UPDATE6.

0013CD1C: A8 E8
0013CD1D: 01 FF
0013CD1E: 75 78
0013CD1F: 58 B6
0013CD20: F6 00
0013CD21: C4 84
0013CD22: 01 C0
0013CD23: 75 90
0013CD24: 53 90
0013CD25: 66 90
0013CD26: 83 90
0013CD27: 79 90
0013CD28: 7E 90
0013CD29: 00 90
0013CD32: 0B 4D
0013CD83: 7C 90
0013CD84: B3 90
00CA4620: 00 535152568139A0970F017564A8017560F6C401755B80795910740680795990754F0FB7997602000039FB7438A1DE0F310189CA8B0C9880795910742880795990742231F60FB7997402000081FBFFFF0000741139FB741983C60183FE1074058B0C98EBE066832D2A13310101B000EB02B0015E5A595BC3

UPDATE8:
The fix in update 7 has a glitch; when a dead spot is re-used immediately after a unit dies while being attacked and the attacking animation has not finished yet, the remaining attacking ammo will go to the newly created unit across the map (sometimes you can see sentinel's ammo go across the map and hits a newly created unit nowhere near the battlefield)

To fix the glitch I have added a delay in re-using the dead unit spot; a spot is re-used only if 18 new units (or two full squads of grunts) are created after the death of the unit.

Here's the fix; use the same process as posted in UPDATE6. I will not remove the previous fix as well, since I haven't tested this for 12 hours (theoretically it should work but until it is tested I can't guarantee anything.

[removed, see update 9]

UPDATE9:
update 8 has a problem of using a conflicting address, this update changes to another address more likely to be dummy. Tested for 12 hours without crashing.

0013CD1C: A8 E8
0013CD1D: 01 FF
0013CD1E: 75 78
0013CD1F: 58 B6
0013CD20: F6 00
0013CD21: C4 84
0013CD22: 01 C0
0013CD23: 75 90
0013CD24: 53 90
0013CD25: 66 90
0013CD26: 83 90
0013CD27: 79 90
0013CD28: 7E 90
0013CD29: 00 90
0013CD32: 0B 4D
0013CD83: 7C 90
0013CD84: B3 90
00CA4620: 00 5351525689CA8139A0970F010F8591000000A8010F8589000000F6C4010F85800000008079591074068079599075740FB7997602000039FB7436A1DE0F31018B0C9880795910742880795990742231F60FB7997402000081FBFFFF0000741139FB744083C60183FE1074058B0C98EBE080BAC4000000FF7507C682C4000000008082C40000000180BAC40000002D7513C682C40000000066832D2A13310101B000EB02B0015E5A595BC3

Note:

If the administrator can grant me the permission to upload file I can upload a patch program instead of having every user to patch manually. (UPLOADED TO Github)

It would be also helpful if anyone can help test more deeply this patch; after all the amount I can test as a single person is very limited.

[This message has been edited by modder00 (edited 10-02-2016 @ 06:01 PM).]

Replies:
posted 08-30-16 06:30 PM EDT (US)     1 / 14  
I know exactly nothing about coding and modding, but seems like you did a LOT of work here. Great job
If I will replay skirmishes with ai and meet this bug I will check out your solution.
Good luck with modding and debugging!
posted 09-02-16 03:53 AM EDT (US)     2 / 14  
Cool stuff! What debugger are you using?

[This message has been edited by Ryder25 (edited 09-02-2016 @ 03:59 AM).]

posted 09-02-16 12:44 PM EDT (US)     3 / 14  
I am using visual studio 2010 for debugging; however I am not really an experienced debugger and I was myself learning how things work while debugging the crash.

[This message has been edited by modder00 (edited 09-02-2016 @ 12:44 PM).]

posted 09-02-16 05:03 PM EDT (US)     4 / 14  
Nice! I had no idea this was possible with VS, or that you can insert your own functions. Did you write your function in assembly or write it in C, and have a compiler do the work for you?

[This message has been edited by Ryder25 (edited 09-02-2016 @ 05:04 PM).]

posted 09-02-16 06:47 PM EDT (US)     5 / 14  
Visual studio was only used for debugging and gaining information. All the actual coding was done by directly modifying the legends.exe in hex editor. I wrote the function in assembly and used this site to convert it to machine code. I found an address in legends.exe which is just a bunch of "00"s and inserted my function there. Then I modified the "create_unit" function to call my function in machine code.
posted 09-02-16 07:21 PM EDT (US)     6 / 14  
Ah, gotcha. Great job! Sounds like a lot of hard work.
posted 09-02-16 08:19 PM EDT (US)     7 / 14  
It was hard. Took almost three weeks to get it done. But it was worth it; crashing after one and a half hour is really annoying for someone like me who loves playing turtling games against toughest AIs which usually takes forever.
posted 09-03-16 06:49 PM EDT (US)     8 / 14  
Wow, 3 weeks! At least now you get to enjoy the fruits of your labor. It's sad that the game was so rushed. This bug shouldn't have shipped.

I do the same kind of playing in RoN. I rarely experience a crash though. I wonder how they implemented this function there.

I recently found a software called x64dbg. Maybe try giving it a go in the future.

[This message has been edited by Ryder25 (edited 09-03-2016 @ 07:00 PM).]

posted 10-08-16 08:21 PM EDT (US)     9 / 14  
I confirm that modder00's "rolcrashfix" patch works for 12 hours with no crashes on the settings he has described.
I tested the patch and let the game run at both normal and fast speeds with minimal input every now and then and it seems it can keep going more than 12 hours.


The instructions for this crash fix patch are as follows:

Modder00's "Rolcrashfix" Patch Instructions

1 Make sure you have made file extensions visible in your Windows OS settings so "legends.exe" is NOT displayed as "legends" without the ".exe"

2 Copy the "legends.exe" to a safe folder and rename it as "legendsfix4.exe.bin"

3 Download "fix_diff.txt" here

4 Place "fix_diff.txt" in the same folder as "legendsfix4.exe.bin"

5 Download apply_patch.exe here and put it into the same folder

6 Run apply_patch.exe

7 There should be two new files named "log.txt" and "legendsfix4.exe.bin2" created

8 Open the "log.txt" and check that nothing has failed

9 Copy the "legendsfix4.exe.bin2" back to your RoL root directory and rename it back to "legends.exe"


Load the game and check these settings for the following maps:

For map maldini heights.bhs:

1 Choose a 8-player map

2 Rules: "Team Free For All", "No Death when Capital Lost"

3 Select the human player as player 3 (the green player) with the Maya civ

4 Select Toughest AI for all other players, Random nation option (Vinci, Alin, Cuotl)

For maps maldini heights 2.bhs and maldini heights 3.bhs please select Rules: "Free For All"

You can press the + and - keys on your keyboard to make the game speed faster or slower when testing this patch

[This message has been edited by alincarpetman (edited 11-23-2016 @ 07:55 PM).]

posted 03-15-18 00:02 AM EDT (US)     10 / 14  
This looks like exactly the problems i have been having with the game but after patching the game to the latest version the apply_patch.exe is noting a failure in the log file only generating a file that is 1,268kb and original is 9,906kb. Here is the error from the log file...

Write failed at address 13cd1c, original byte {81, a8} does not match.

I have tried to go in with a hex editor and try to manually find the spots to change but i am lost. Does anyone have an idea of my problem?

Thanks
posted 09-07-20 01:02 PM EDT (US)     11 / 14  
Great work Modder00! I'd possibly like to make this easy to apply by creating a user patch EXE. I'll keep you updated. My plan will be

1: Find a diff-like tool to create binary patches in a readable format (or failing that, just use a C data structure)

2: Write an EXE that opens legends.exe and patches it

It would be good to maintain this so that as new binary patches are created they can be layered on to one another and we can just have a user patch that brings an ORIGINAL / FULLY UPDATED legends.exe to a PATCHED legends.exe
posted 09-08-20 08:33 PM EDT (US)     12 / 14  
It would be good to maintain this so that as new binary patches are created they can be layered on to one another and we can just have a user patch that brings an ORIGINAL / FULLY UPDATED legends.exe to a PATCHED legends.exe
Sounds like a plan.
posted 06-27-21 09:36 AM EDT (US)     13 / 14  
Hi everyone, I did exactly as instructed in UPDATE 9, the updates were successful according to the log file, but when I rename the legendsfix4.exe.bin2 to legends.exe to legends.exe, and paste it back, the game doesn't launch at all, it says it doesn't exist on my computer. Is it really just a rename or should I mount the .bin2 file to .exe?
Thanks in advance!
posted 03-26-22 06:37 PM EDT (US)     14 / 14  
@Colonel Oneill

Try asking on the ROL Discord
Rise of Legends Heaven » Forums » Tech Support » Game crashing after long playing time (with some debug info) - FIXED
Top
You must be logged in to post messages.
Please login or register
Hop to:    
Rise of Legends Heaven | HeavenGames