[10848] Multithreaded packet processing

[Ambal]

Ok, thank you, divisorxx I was busy so didn't had a chance to write code which will gather some statistics for us... I hope to do it today, but who knows Anyway, you should threat current patch as final version until taught otherwise

[lukaasm]

whole day testing and: no crashes, no weird problems ;] 2.5k online and lower world update diff especially on 'rush' hours ;]

[False.Genesis]

@Ambal: you may find this useful:

http://filebeam.com/e5a485ebb4bf7b9c6f7130169dd5c978

contains some network data recorded for ~2 months, with online# counts from 30 - 150 average. hope this helps

[Ambal]

@Ambal: you may find this useful:

Ohho, this is AWESOME! You've saved alot of my time, False.Genesis I'll analyze your findings and post results here as soon as I find some free time, I promise

[Toinan67]

This is really nice indeed, where did you find this False.Genesis? Did you do it yourself?

It could be a very nice core mod added to the official core (with a conf option to activate/deactivate it)

[False.Genesis]

@Toinan67: yes, this is some live data recorded, code is in the following commits (code by me):

https://github.com/fgenesis/mangos/commit/b753c8b8fb6c20acea351984ebbc53e0a08421ff

https://github.com/fgenesis/mangos/commit/c5a6be53cee4e3f12732dae12943ad3bd4729e6e

https://github.com/fgenesis/mangos/commit/86f1eb080f0d1132d3d6e9b782ba7037a9086f97

and also https://github.com/fgenesis/mangos/commit/6db47c9af2ff4998913072ab0673895720bbc8f3 for the network_usage_* table format.

dont have a clean patch and no time to work on any mangos related things, sorry

[/offtopic]

i plan to test this patch without any mtmaps stuff whenever i have the time.

and i have a question, which parts of the spell system can possibly be multithreaded (or, is it of any use to do so)? because i have observed this eating the most bandwidth, but i dont know how much CPU it uses, compared to the map/movement/visibility stuff. the thought is that *most* spells affect only 1 map .. (but not, for example, a player with a buff from another player enters another map, or a teleport/summon spell).

EDIT: i mean outgoing bandwidth here, but nevertheless i imagine the spell system could do better if response packets are built in different threads... because a LOT of them are sent.

[Ambal]

Ok, another portion of packet processing optimizations are submitted based on stats provided by False.Genesis. You can download profiling info from here http://filebeam.com/bf0b0683255ba557c88df7d366429bb7 - - rename to 'opcode_profiling.html'. If someone wishes to participate, please, install patch in the previous post made by False.Genesis and collect some useful networking data for us :rolleyes:

Opcodes CMSG_INSPECT and CMSG_MESSAGECHAT are not that easy to make thread-safe at the moment

WARNING: spell opcodes CMSG_CAST_SPELL were parallelized so pay attention on server stability and report any appearing issues!

[selector]

Perfect work. Going to test today. Wait for reports.

[darkstalker]

inspect opcodes are spammed by the GearScode addon (a very popular one nowadays)

[shauren]

inspect opcodes are spammed by the GearScode addon (a very popular one nowadays)

its CMSG_INSPECT

[lukaasm]

tests of current patch: there is no problems ;]

[Ambal]

tests of current patch: there is no problems ;]

Are you sure you are testing the latest patch from my branch? There are changes for spell packet processing which I want to be sure are really thread-safe.

[divisorxx]

I'm already testing with last changes and so far 16 hs uptime 800 users max, not crash related.

[forbiddenn]

I just tested is:

2>..\\..\\src\\game\\World.cpp(1938) : error C3861: 'RemoveQueuedPlayer' : identificateur introuvable (in english identifier not found)

Edit :

To test I just changed:

if(!pSession->Update(diff, updater)) // As interval = 0

{

RemoveQueuedPlayer (pSession);

m_sessions.erase(itr);

delete pSession;

}

By:

if(!pSession->Update(diff, updater)) // As interval = 0

{

RemoveQueuedSession (pSession);

m_sessions.erase(itr);

delete pSession;

}

Is good?

In compiling his pass and I start the server, it did not crash.

[lukaasm]

there were few problems with: "CMSG_NAME_QUERY", sometimes ppl on friends list appeard as unknown, setting it as unsafe solved it ;]

[Ambal]

Thanks to everyone for tests. Error with 'RemoveQueuedPlayer' I suppose I did when tried to merge patch with newest mangos revisions. It should definitely be 'RemoveQueuedSession'.

As of CMSG_NAME_QUERY - thanks for hint, I'll see what is causing weird behavior you've described, lukaasm.

Cheers

[Ambal]

Fixed RemoveQueuedSession() call in World::UpdateSessions() function. Also CMSG_NAME_QUERY processing set as PROCESS_THREADUNSAFE.

Lets consider this patch as final in case you won't report bugs anymore

[Ambal]

I'm going to commit this patch tomorrow in case noone will bring some bad news about bugs/crashes/glitches or performance degradation Cheers

[cyberium]

Before commit do the gain (at least for big server) is real?

Has i connot see any diff (realy little server) and in this case this patch has only risk to create any new unknow situation with some new devellopement.

Yes this partialy multithread implementation need for me at least prove enought good benefict just because it's only a partial solution.

Anyway it's still a good piece of code not any doubt about that

[faramir118]

I'm going to commit this patch tomorrow

Can't wait to see what you come up with next

[Ambal]

Applied in [10848]. Thank you for all your feedback

Lets go and fix timer issues for grid inactive objects

[cyberium]

Hi,

In an devevelopement session about Dungeon Finder system i have some question.

How to be sure i can saftly active "ThreatSafe" option under opcode that i need?

Actually i implemented 2 important thing about that but iam not sure it's enought.

EventList -> All event will be added to an list of event and processed only under "ProcessEvent" called on update.

ActionList -> All action will be added to an list of action and only processed under "ProcessAction" called olso on update.

Exemple -> player request adding to queue

- Event received and added,

- Event processed in (updateworld)

- If action decided new action is created "add this guy to list"

- Action processed just after event. And the guy is in the list.

(EventList processing can do some action too and i think about have only one list processing but it's not the purpose of this post)

So what iam not sure,

- When event is processed do new event can be added?

- Where is the case of concurent data access risk?

Thanks

[Ambal]

cyberium, if you are unsure if your packet handler is safe to run in Map::Update() - don't touch it. Also, if your packet handler is light-weight e.g. no heavy vmap/mmaps logic being called or amount of such packets is low - don't bother making it thread-safe either.

Cheers