next »

[WickedShell]

So today was pretty terrible and I couldn't make myself work on fixing Echelon2, nor do work I get paid for, or do work for classes (sad I know). So what did I do? I was stuck thinking how much fun it would be to get B3 working for altitude (wow the front page is terrible, ignore that one  ). If people are interested in checking it out you can play the demo for free on the site (it uses java).

Anyways, I started working on a parser for this and I believe it won't be too bad to actually see all the way through. Consistent GUID's are provided by what they call a VaporID, it's constant to the copy you purchased, which means the servers can check upon connection if a player id is banned, so persistent banning is possible. Rcon is available in a limited fashion. It can be rcon'd from in game, and there's a file command.txt that any commands appended to it are immediately run. (I foresee this being awkward for remote installs).

In terms of log parsing all log entries are provided in JSON, which makes parsing fairly trivial for once, no complicated regex's needed. Python has a json parser built in, and it serves the job just fine. Most things are already nicely broken down with type labels, consistent (per connection) player id's (similar to q3a's slot's). Which given most of the information I see exposed in the log I don't see any show stopping issues that would prevent B3 from working fine.

With regards to community reception of it; well the current tools consist of one php log parser, which does not fulfil the role that B3 does, and one community maintained VaporID tracking DB (to keep track of pro players who are alias'd). Which leaves a hole I think B3 would nicely fulfill. (As well as giving me B3 there  )

I started work on it, threw together the python code for rcon (which is only doing the local file edit version at the moment). And then I got a bit stumped. I was looking to the q3a, frostbite, and homefront engines for examples on how to setup a nice parser/what things are required. It appears to me that the q3a/frostbite/homefront abstract parsers all sorta do there own thing with only some functions common to all of them. (Indeed I didn't even see a homefront parser in the parser folder). Is there a list of functions/interfaces I need to provide for a parser, and is it worth creating a abstract parser as well as a specific one for altitude? As far as I can tell the only thing that uses this engine is altitude, and it will be the only thing to use it. I'm not opposed to splitting it into the two parts one abstract one specific, but I'm not sure where the line for that should be? Or should I just go ahead and write one parser and rcon specific for only altitude?

Or just not write it at all as B3 would not want it? (It's a pretty small game and I was looking for a project to work on when I get frustrated with the other, if you don't want it it's fine )

I can post some longer server log's if anyone has questions, or wishes to see them. Here's from start up to a couple kills on a passworded server;

Code:

{"port":-1,"time":0,"date":"2011 Apr 21 15:20:32:401 EDT","type":"sessionStart"}
{"port":27276,"time":36786,"name":"spaz's test Altitude Server","type":"serverInit","maxPlayerCount":14}
{"port":27276,"time":37039,"map":"ball_ice","type":"mapLoading"}
{"port":27276,"time":38855,"map":"ball_ice","type":"mapChange","mode":"ball"}
{"port":27276,"time":39182,"type":"serverStart"}
{"port":27276,"demo":false,"time":39217,"level":1,"player":0,"nickname":"Bot 1","aceRank":0,"vaporId":"00000000-0000-0000-0000-000000000000","type":"clientAdd","ip":"0.0.0.0:100001"}
{"port":27276,"demo":false,"time":39219,"level":1,"player":1,"nickname":"Bot 2","aceRank":0,"vaporId":"00000000-0000-0000-0000-000000000000","type":"clientAdd","ip":"0.0.0.0:100002"}
{"port":27276,"time":40285,"player":0,"team":2,"type":"teamChange"}
{"port":27276,"time":40287,"player":1,"team":2,"type":"teamChange"}
{"port":-1,"time":0,"date":"2011 Apr 21 15:22:01:422 EDT","type":"sessionStart"}
{"port":27276,"time":3600,"name":"spaz's test Altitude Server","type":"serverInit","maxPlayerCount":14}
{"port":27276,"time":3814,"map":"tbd_hills","type":"mapLoading"}
{"port":27276,"time":5365,"map":"tbd_hills","type":"mapChange","mode":"tbd"}
{"port":27276,"time":5684,"type":"serverStart"}
{"port":27276,"demo":false,"time":5708,"level":1,"player":0,"nickname":"Bot 1","aceRank":0,"vaporId":"00000000-0000-0000-0000-000000000000","type":"clientAdd","ip":"0.0.0.0:100001"}
{"port":27276,"demo":false,"time":5714,"level":1,"player":1,"nickname":"Bot 2","aceRank":0,"vaporId":"00000000-0000-0000-0000-000000000000","type":"clientAdd","ip":"0.0.0.0:100002"}
{"port":27276,"time":6166,"player":0,"team":2,"type":"teamChange"}
{"port":27276,"time":6178,"player":1,"team":2,"type":"teamChange"}
{"port":-1,"time":0,"date":"2011 Apr 21 15:23:44:968 EDT","type":"sessionStart"}
{"port":27276,"time":3531,"name":"spaz's test Altitude Server","type":"serverInit","maxPlayerCount":14}
{"port":27276,"time":3787,"map":"tdm_cave","type":"mapLoading"}
{"port":27276,"time":5495,"map":"tdm_cave","type":"mapChange","mode":"tdm"}
{"port":27276,"time":5797,"type":"serverStart"}
{"port":27276,"demo":false,"time":5846,"level":1,"player":0,"nickname":"Bot 1","aceRank":0,"vaporId":"00000000-0000-0000-0000-000000000000","type":"clientAdd","ip":"0.0.0.0:100001"}
{"port":27276,"demo":false,"time":5849,"level":1,"player":1,"nickname":"Bot 2","aceRank":0,"vaporId":"00000000-0000-0000-0000-000000000000","type":"clientAdd","ip":"0.0.0.0:100002"}
{"port":27276,"time":6447,"player":0,"team":2,"type":"teamChange"}
{"port":27276,"time":6455,"player":1,"team":2,"type":"teamChange"}
{"port":27276,"demo":false,"time":9275,"level":48,"player":2,"nickname":"WickedShell","aceRank":4,"vaporId":"6477be06-19ea-4f53-9180-01ac3a60dec9","type":"clientAdd","ip":"132.177.50.31:27272"}
{"port":27276,"time":10666,"type":"pingSummary","pingByPlayer":{"2":0}}
{"port":27276,"time":14484,"player":2,"team":2,"type":"teamChange"}
{"port":27276,"time":15665,"type":"pingSummary","pingByPlayer":{"2":5}}
{"port":27276,"time":15866,"player":1,"team":5,"type":"teamChange"}
{"port":27276,"time":15942,"plane":"Biplane","player":1,"perkRed":"Dogfighter","perkGreen":"Rubberized Hull","team":5,"type":"spawn","perkBlue":"Reverse Thrust","skin":"No Skin"}
{"port":27276,"time":16067,"player":0,"team":6,"type":"teamChange"}
{"port":27276,"time":16257,"plane":"Bomber","player":0,"perkRed":"Flak Tailgun","perkGreen":"Flexible Wings","team":6,"type":"spawn","perkBlue":"Reverse Thrust","skin":"Flame Skin"}
{"port":27276,"time":20664,"type":"pingSummary","pingByPlayer":{"2":0}}
{"port":27276,"time":21386,"player":2,"team":6,"type":"teamChange"}
{"port":27276,"time":21435,"plane":"Loopy","player":2,"perkRed":"Double Fire","perkGreen":"Repair Drone","team":6,"type":"spawn","perkBlue":"Ultracapacitor","skin":"No Skin"}
{"port":27276,"time":21679,"streak":0,"source":"plane","player":-1,"victim":0,"multi":0,"xp":10,"type":"kill"}
{"port":27276,"time":21680,"player":0,"team":5,"type":"teamChange"}
{"port":27276,"time":22737,"powerup":"Health","player":1,"positionX":1471,"positionY":690,"type":"powerupAutoUse"}
{"port":27276,"time":24809,"plane":"Explodet","player":0,"perkRed":"Director","perkGreen":"Heavy Armor","team":5,"type":"spawn","perkBlue":"Ultracapacitor","skin":"No Skin"}
{"port":27276,"time":25664,"type":"pingSummary","pingByPlayer":{"2":0}}
{"port":27276,"time":26503,"streak":1,"source":"plane","player":1,"victim":2,"multi":1,"xp":10,"type":"kill"}
{"port":27276,"time":26601,"powerup":"Health","player":1,"positionX":949,"positionY":575,"type":"powerupAutoUse"}
{"port":27276,"time":29602,"plane":"Loopy","player":2,"perkRed":"Double Fire","perkGreen":"Repair Drone","team":6,"type":"spawn","perkBlue":"Ultracapacitor","skin":"No Skin"}
{"port":27276,"time":30680,"type":"pingSummary","pingByPlayer":{"2":0}}
{"port":27276,"time":31665,"powerup":"Homing Missile","player":1,"positionX":2235,"positionY":605,"type":"powerupPickup"}
{"port":27276,"velocityX":2.37,"time":33976,"powerup":"Homing Missile","player":1,"positionX":2162.17,"velocityY":8.59,"positionY":462.89,"type":"powerupUse"}
{"port":27276,"time":34266,"streak":0,"source":"plane","player":-1,"victim":0,"multi":0,"xp":10,"type":"kill"}
{"port":27276,"time":34668,"streak":2,"source":"plane","player":1,"victim":2,"multi":1,"xp":10,"type":"kill"}

[MordyT]

I think go for it if you want to, the more games the better.

Sent from my HTC Glacier

[WickedShell]

Ugh, so I just noticed in all that great JSON, and other details it doesn't provide a cause of death, it says who did it, and who died but not the weapon used, which would sorta break xlrstats (I mean it tells you the equivelent of what "kit" is active but the actual cause of death could have been primary, secondary or a powerup, which makes XLRstats sorta hard to figure out, past a K:D...

I put in a feature request, they've been pretty good about adding "productive/community" features. Indeed the entire detailed log was a community requested feature released well after the game was.

I'll poke at it a bit cause it's interesting (and is helping explain/making me understand of the rest of the core B3 functionality/interactions) but we'll see what they say.

My bad for missing that before and not investigating that specifically before posting here.

[Courgette]

how to setup a nice parser/what things are required.

We miss a good doc here. Maybe we can take this as an opportunity to start a draft of it

Is there a list of functions/interfaces I need to provide for a parse

https://github.com/BigBrotherBot/big-brother-bot/blob/master/b3/parser.py#L1112

is it worth creating a abstract parser as well as a specific one for altitude ?

No, just make an altitude parser inheriting from b3.parser.Parser. If one day we support another game similar to altitude, then we'll assess the need for having common code.

Give me a few days, and I'll come up with a start of documentation on the wiki. That doc will become more and more complete as you ask me questions.

[WickedShell]

I noticed  Doxygen is a good start, could use some expansion of the inline code content, but the real issue is that with regards to adding something like a new parser there are only really three big parser setups that everything inherits from, and those of you who put all the excellent work into it. Which means you're very familiar with the setup, while the rest of us aren't so familair with it have a minor problem  (The other instance ones can be figured out by looking at a different instance of a specific one, similar to the way people can figure out plugins)

Okay, that's the class I was assuming to inherit from My current approach is that I just copied a Q3 parser and am editing it down to match the parser one, and appropriate helper methods. But again I'm using that as my rough guideline.

With regards to a specific altitude parser that makes sense to me, and sounds good. (Especially for my first pass without having worked on/looked at the parsers before).

No rush! Don't let me bother you to much, you have a lot on you're plate  As I said I started this on the side casue it was an itch to scratch and I'm still considering how to approach the chatlogger plugin for echelon2  (The whole config thing for multiple games is causing me headaches, as this indicates that echelon plugins should have a nice uniform way to interact with the database (allowing each plugin to maintain at least its own settings table and/or content table), but the refactoring is slow/annoying)

[Courgette]

Also good to know : b3.parser.Parser is meant to work for games that produce a game log file.

If Altitude does not produce a game log file to read from, then you have to overwritte the run() method to have it read game events in some way (Frostibe and Homefront parsers use a TCP network connection)

[WickedShell]

Altitude has a jog file, it's also incredibly easy to pull info from because python has a JSON parser built in. I can switch to using regular expressions if there's a reason to, but I'm inclined to think the JSON parser will be the fastest and most useful. Remote rcon-ing commands is going to be an issue, local is easy, but remote is an issue, and at the moment remote depends on filesystem access (It appends to the tail of a file. I need to see if I can just overwrite the file instead of having it keep trailing).

I admit I'm a bit lost at the moment. I'm staring at some of the other parsers and for the life of me I can't figure out how they all work. Within the q3a one's (which have been my rough guide, as they are log based). Specifically I picked the wop mostly. The common theme is when parseLine gets called, it tries to call a function named On<EVENT TYPE>() but what I don't get is within the client connect for example I see it append some stuff to self, but it doesn't appear to do anything. Other one's create B3 Event's but I still fail to see how this ever writes any data out to the database. Or is the only point of the parser to generate B3 Events?

I may stick in a bunch of debug's in the OA, or UrT parsers and just see what the payloads look like for the events. This is probably not a bad idea anyway so I can trace through a bit in general 

[Courgette]

ok, then assuming the game log file is composed of lines and that each line is a json object, then you do not need to overwritte the run() method.

Just overwritte the parseLine() method so that it reads json.

And the responsability of a parser is not to write to database. They are :

• read game events
• map game events to B3 events
• maintain self.game object attributes (see b3.game.Game class)
• connect players self.clients.newClient(cid, guid)
• maintain client object attributes (.team, etc. See b3.client.Client class)
• disconnect players when they leave the game (client.disconnect())
• implements parsers API (expected methods like ban(), kick(), map(), etc)

I would be you, I would start off an empty file instead of get blurred by WoP specificities

[Courgette]

for the next day you feel bored, I started up a parser for Altitude and wrote a draft of a parser tutorial.

tutorial : http://wiki.bigbrotherbot.net/customize:start#making_b3_support_a_new_game
altitude branch on my github repo : https://github.com/courgette/big-brother-bot/tree/altitude

[Freelander]

Excellent tutorial! Good job mate!

[Courgette]

Thanks, now who have Altitude ? I need friends on that game to properly test tempban, permban and a few other parser features

[Haris2201]

-sending messages to individual players

Hehe,

Smokin' Guns does not supprot this and you make a parser for it

[WickedShell]

Can't wait to feel bored again! (Gonna be at least a week and a 10 mile run and obstcale course first though...  )

With regards to people to test on I obviously have an account, it's "WickedShell". You can probably test everything with the free demo account just fine (provided the server lets demo people connect). I also happen to have an altitude server (as you might have figured from the thread in the first place)., that I can pull some logs from, give you rcon/admin rights on connect.

A bit werid but I noticed when I kicked some one earlier they got a 2 minute temp ban from the game that showed up in the bans file, which is nice to keep people from reconnecting right away.

Looks good from taking a quick glance through. The only big things I see, is that it may have an issue with some werid game modes creating some additional messages (I'll check it out), and that within sync altitude does provide a way listPlayers, which works from the client, but when I try it on the server I am discovering that that doesn't seem to be written to the log file  logPlanePostions logs by ID all postions of planes to the file correctly, but that doesn't really help.

The other one that didn't appear yet is customCommands. (These are essentially commands defined that allow players to vote for whatever, and shows up in the log, and from there it's server admin's job to do something with it).

Things I've encountered that are unfortunate;

• can not move or kick bots
• rcon only pertains to from clients, not a protocol which can be used from outside the game

Excellent start & tutorial though thanks!

[Courgette]

With regards to people to test on I obviously have an account, it's "WickedShell". You can probably test everything with the free demo account just fine (provided the server lets demo people connect). I also happen to have an altitude server (as you might have figured from the thread in the first place)., that I can pull some logs from, give you rcon/admin rights on connect.

I have an account myself and was able to run a server also.

I noticed when I kicked some one earlier they got a 2 minute temp ban from the game that showed up in the bans file

This is documented in the Altitude wiki. A kick is a 2 minutes tempban in that game.

Looks good from taking a quick glance through. The only big things I see, is that it may have an issue with some werid game modes creating some additional messages (I'll check it out)

put the B3 log level down to 8 to see methods that are to implement to act on such new game events

altitude does provide a way listPlayers, which works from the client, but when I try it on the server I am discovering that that doesn't seem to be written to the log file 

as for listBans, this is something to suggest to the Altitude dev : provide in the log file the results of these commands. Also it would be useful that the result of listPlayers include slot IDs as well as VaporIDs

logPlanePostions logs by ID all postions of planes to the file correctly, but that doesn't really help.

interesting, I did not know about this one. I have to give it a try

The other one that didn't appear yet is customCommands. (These are essentially commands defined that allow players to vote for whatever, and shows up in the log, and from there it's server admin's job to do something with it).

I thought that one was done and should raise a EVT_CLIENT_CALL_VOTE B3 event and responses to that vote should raise EVT_CLIENT_VOTE B3 events that a B3 plugin should be able to act on.

can not move or kick bots

agreed, listPlayers should also list bots

rcon only pertains to from clients, not a protocol which can be used from outside the game

there must be a way to to real rcon through the network. Maybe Altitude dev could tell us more. I noticed the Altitude server try to re-execute all commands from the command file when it restart which is a very unfortunate behavior in our case. Also, if you truncate the command file size, then the Altitude server is confused and won't run new commands put in it.

[WickedShell]

I have an account myself and was able to run a server also.

Yeah I just didn't want to make you have to buy an account/run a server all the time, if you didn't have one.

This is documented in the Altitude wiki. A kick is a 2 minutes tempban in that game.

I just pointed that out as I was suprised, but I didn't see any way that actually affected B3.

listPlayers actually shows connected bots, although a server will always have the same number of bots once it starts (they can't be removed while the game is up), and they will always fill the first x slots (where x is the number of bots). I agree listPlayers should show in the log, listBan's might be excessive. The ban list is also an XML file which isn't to bad to parse seperately. Although as far as I know, B3 doesn't ever ask the server for a list of bans does it?

I thought that one was done and should raise a EVT_CLIENT_CALL_VOTE B3 event and responses to that vote should raise EVT_CLIENT_VOTE B3 events that a B3 plugin should be able to act on.

I'm pulling from the Altitude docs relating to the release of the custom commands, but according to those the log looks like;

Code:

{"command":"startLadder","type":"customCommand","time":20428,"port":27275}

which is not handled yet in the parser. CustomCommand's are differntiated from standard commands. But I need to add some, as customCommand's can carry payload's specifying playernames/map's specifically, which may Identify differently in the JSON log entry. As I said I'll take a look at this when I get a chance.

there must be a way to to real rcon through the network. Maybe Altitude dev could tell us more. I noticed the Altitude server try to re-execute all commands from the command file when it restart which is a very unfortunate behavior in our case. Also, if you truncate the command file size, then the Altitude server is confused and won't run new commands put in it.

At least the login is done in SSL, and looking at a packet sniffer around the time I sent some RCON commands I saw no discernable plain text, and have been unable to locate any rcon protocol information on the forums, or elsewhere. I believe the truncation of the command file confusing altitude is related to the file handler that altitude maintains (that reads from the tail) not getting moved back to the begining when the file gets emptied. But it's been a while since I played with multiple things accessing the same file.