next »

[grosbedo]

Hello there,

Ive recently noticed on my servers that sometimes, b3 stops to reply for a while.

Ive tracked down the bug, and it seems it just simply get stucks when my automatic logs rotation happens. B3 then simply will wait until the log reaches the same size as before, because the read() cursor wasn't updated until then.

This bug can easily be reproduced, by doing a

cat /dev/null > game_log.log

or simply by removing some of the last lines from the log while B3 is launched.

Here is an updated version of parser.py. I tried to keep the changes to a minimum.

Here is the code (you can download the full version in attachment):

Code: python

    def read(self):
        """Read date from Rcon/Console"""
        filestats = os.fstat(self.input.fileno()) # Getting the stats of the game log (we are looking for the size)
        if (self.input.tell() > filestats.st_size): # Compare the current cursor position against the current file size, if the cursor is at a number higher than the game log size, then there's a problem
            self.error('Parser: Game log is suddenly smaller than it was before (%s bytes, now %s), the log was probably either rotated or emptied. B3 will now re-adjust to the new size of the log.' % (str(self.input.tell()), str(filestats.st_size)) )
            self.input.seek(0, os.SEEK_END) # We reset the cursor position to the last line

        return self.input.readline()

[attachment deleted by maintenance]

[Courgette]

Thanks a lot. Another of your nice contributions
Keep it up !

[grosbedo]

Thanks a lot. Another of your nice contributions
Keep it up !

Thank you very much sir, that's heart warming.

But unluckily, I think my contributions rate will drop a lot now, Ive left my community. Please don't blame me if I become kind of inactive from now on.

I will still contribute if I ever stumble across a bug or a missing feature.

Thank you for your great tool guys, you've made something all servers admins and players were longing since long. And thank you Courgette for having supported me, and the OpenArena community, as fast as you did.

Goodbye.

[Courgette]

Good luck

[grosbedo]

Good luck

Thank you You too

[grosbedo]

I couldn't resist to add one last feature.

Ive edited parser.py once more, to enhance read() and run() functions, so that when they parse the log, they readlines at once instead of readline.

This permits the b3/delay option to be more useful since it will now specify the delay between each log's read, but not each log's line parsing. For the latter, Ive added a second option : b3/delay2 (but it's mainly for dev, I hardly see the purpose for standard users).

This should enhance the performances of b3 a lot, since accessing hd datas are what is the slowest and ressource consuming in the bot.

By default, delay is set to 0.05, but Ive tested with 0.5 and cannot see any difference, compared to 0.001 before. This means that the performance on this matter should be up to 500% better.

As before, Ive made as small changes as I could to not break anything, everything else should work just as before.

v1.17 attached

I suggest to add the b3/delay option in the sample b3.xml.

[attachment deleted by maintenance]

[Courgette]

Hi

Have you also fixed something regarding the timezones ? it seems so if I do a diff

[grosbedo]

Hi

Have you also fixed something regarding the timezones ? it seems so if I do a diff

Not at all, I only edited the read() and run() functions.

But I took the parser from v1.3.3 as a base, you may have a more recent version.

[Courgette]

Don't worry, I found my way through

I like your idea of reading as much lines at once from disk. Making the bot lighter on system resources is always a welcomed improvement.

About the delay, you misunderstood the point of having it here. The delay's goal is not to temporize the processing of lines but to free some CPU cycles as we are in a infinite while loop running in a thread.
So from your code, I won't keep delay2 as we do not need to have any delay between lines.

Also you removed a test

Code: python

if line:
    #blabla

which has to remain to handle cases where the game log file contains an empty (or composed of blank characters) lines.

change pushed to my github repo

[grosbedo]

About the delay, you misunderstood the point of having it here. The delay's goal is not to temporize the processing of lines but to free some CPU cycles as we are in a infinite while loop running in a thread.
So from your code, I won't keep delay2 as we do not need to have any delay between lines.

No, I understood the use, and that's why Ive added the delay2 : imagine that the b3/delay is set to a high value, like 5 seconds. Then, the next loop, the read() function can return a huge amount of lines at once if the game server is very busy with many players and events. Then, the delay2 is useful to free some CPU cycles and avoid the bot from freezing the system while processing the lines (particularly on Windows OSes).

Also you removed a test

Code: python

if line:
    #blabla

which has to remain to handle cases where the game log file contains an empty (or composed of blank characters) lines.

change pushed to my github repo

No no, I kept the check, but now since several lines are returned at once, the check is done via "if lines:" instead of "if line:". It works, I tested it carefully.

[grosbedo]

Ok, I looked at the github push, Ive seen you've added "if line:" as a supplementary test, so it's ok.

But I still maintain that the delay2 is required, or at least a time.sleep(0.001), it will avoid the hangovers with a high b3/delay.

[Courgette]

do you really think B3 would often have hundreds of lines from readlines() ?

[grosbedo]

do you really think B3 would often have hundreds of lines from readlines() ?

If you put a high delay and have a busy server (like I have), yes, definately.

Don't forget Murphy's law

[Courgette]

I'm not worried about high delay. the delay setting is not supposed to be known from end users (which is why it is excluded from the provided b3.xml) and is something only dev play with.
If you think about system resources (which was the goal of your change), having a delay between log file reads will spare some disk I/O, but even if one disk read returns thousands lines, the CPU resources required to process them does not compare to disk I/O. Also looping through those lines (even thousands) is not process which is affected by the infinite loop syndrome as we are looping through a finite number of rows.

[grosbedo]

I'm not worried about high delay. the delay setting is not supposed to be known from end users (which is why it is excluded from the provided b3.xml) and is something only dev play with.

That depends which end users you aim at : if you only aim at home users with a single server, fine. If you aim big GSPs with many servers running at the same time on the same machine, then it won't.

If you think about system resources (which was the goal of your change), having a delay between log file reads will spare some disk I/O, but even if one disk read returns thousands lines, the CPU resources required to process them does not compare to disk I/O. Also looping through those lines (even thousands) is not process which is affected by the infinite loop syndrome as we are looping through a finite number of rows.

The goal here was to avoid continuously trying to read a file. It's not only about sparing some disk I/O, but lines processing as well, since you will avoid continuously processing them, and instead process them in some bunchs with a delay between each bunchs. Meanwhile, b3 is pretty much inactive, and so it avoids sucking continuously the CPU.

Anyway, a small sleep is not only needed in infinite loops, but in finite loops as well. This permits to avoid to freeze the CPU when you process a huge amount of lines, which can happen. I can't see why you absolutely want to remove it, it doesn't affect the performances of the bot for anything, and it would avoid some annoying future bugs. It's just so simple and obvious that I can't understand why you prefer to remove a cycle sleep that would fix several future bugs.

Please stop funking into my code for no reason. I understand that you have to check on my changes, and that's fine, but you seem to remove pretty much anything you don't understand or don't see the point (even the comments).

I am an experienced coder, particularly in Python, which I develop in since more than 5 years. I am only registered since about a month (a little less in fact), and I already contributed to many plugins, made 3 on my own, respected the last standards of B3 as much as I could and helped in making the OA parser.

Please trust me more, me and my code.