Updating Mtmaps

[Auntie Mangos]

There you go:

mtmaps fix for win:

http://getmangos.eu/community/showpost.php?p=60214&postcount=37

[painejake]

Im currently updating the mtmaps patch as i can seem to find a repo or patch thats uptodate. I am using the patch from the TrinityCore patch queue and i am getting this error on compile:

9>c:\\Documents and Settings\\Administrator\\Desktop\\mangos\\dep\\ACE_wrappers\\ace/OS_NS_stdlib.h(210) : error C2059: syntax error : 'constant'
9>c:\\Documents and Settings\\Administrator\\Desktop\\mangos\\dep\\ACE_wrappers\\ace/OS_NS_stdlib.h(214) : error C2059: syntax error : 'constant'
9>c:\\Documents and Settings\\Administrator\\Desktop\\mangos\\dep\\ACE_wrappers\\ace/OS_NS_stdlib.inl(397) : error C2059: syntax error : 'constant'
9>c:\\Documents and Settings\\Administrator\\Desktop\\mangos\\dep\\ACE_wrappers\\ace/OS_NS_stdlib.inl(410) : error C2059: syntax error : 'constant'

If anoyone could tell me what the ": error C2059: syntax error : 'constant'" is it would be a great help.

Heres line 204 -215 of the OS_NS_stdlib.h

#if !defined (ACE_HAS_WINCE)
#  if !defined (ACE_LACKS_REALPATH)
 ACE_NAMESPACE_INLINE_FUNCTION
#  else
 extern ACE_Export
#  endif /* !ACE_LACKS_REALPATH */
 char *realpath (const char *file_name, char *resolved_name);

#  if defined (ACE_HAS_WCHAR)
 ACE_NAMESPACE_INLINE_FUNCTION
 wchar_t *realpath (const wchar_t *file_name, wchar_t *resolved_name);
#  endif /* ACE_HAS_WCHAR */

[HiTmAn]

You can use mine

http://github.com/HiTmAn/mangos.git

branch mtmaps

It works fine on linux, didn't try it on Windows

bye

HiTmAn

[painejake]

Ahh brill thanks Hitman. Good job i'm on Linux Do you have one for 0.13?

[leak]

Somewhere on forums there is a small fix for getting mtmaps to work on windows. If i'm not all wrong it is just the order of the includes being made. Well, search function...

[painejake]

I'm using linux I'm looking for a mtmaps branch (If there is one) for 3.0.9 (0.13)

[GriffonHeart]

I have same errors. How I can fix it?

[raczman]

I've written a small patch for TC that provides similiar features as mtmaps by Derex. It works by splitting workload of MapManager::Update() loop across several threads. It's been tested on TC with good results (60ms update time with 600 players at peak) and id like to share (and test ) this patch with MaNGOS community. Differences are so few that it will be easy to merge. For it to compile you need:

linux

gcc 4.X (for openmp support)

add -fomp to compile flags

link mangos-worldd with gomp library

since TC uses cmakes i cannot provide automake patch to allow it to build, but i know youll be able to do it with no problem - if you will have trouble with it though, i can do it too. Diff is at:

http://bitbucket.org/raczman/trinitycore-patches/src/tip/mtmaps2

[Wyk3d]

Why add yet another threading library when we have two already ?

Do you know what you are doing differently compared to Derex's patch that may give different results ?

Did you do an in depth search for any possible resources which might be used by two of those threads at the same time in a manner that may require locking (IIRC there were a few cases that were addressed by Derex's patch) ?

[raczman]

Why add yet another threading library when we have two already ?

Do you know what you are doing differently compared to Derex's patch that may give different results ?

Did you do an in depth search for any possible resources which might be used by two of those threads at the same time in a manner that may require locking (IIRC there were a few cases that were addressed by Derex's patch) ?

Im not adding any library, gomp is a part of gcc, and if your compiler is up-to-date everything will work correctly. If that patch needed any external library id tell so.

Also i know that i am taking completely different approach than Derex - but writing it 100% same method as him would be waste of time, dont you think?

Derex uses sophiscated system of requests to add maps to update queue, which is then executed. I on the other hand break apart for() loop into chunks that threads do parallely. Though we use different methods, outcome is more or less same: map::update() is called in own thread. I read Derex's patch many times looking for eventual pitfalls, or data access problems, and found none such thing. Or maybe i have old patch in my repo, who knows.

This started mainly as a way to disprove my friend that core with openmp forked loop for map updates will even work - i lost the bet but i do not regret it that much. Ill be as much happy to encounter and fix any bugs that my approach will bring - for me its all about learning

[leak]

As much as i like to see dev action on that part of the deal i also believe it might be on the wrong side.

For me the mtmaps patch is mainly to move the incredible ingame lag that starts at 700 users to ~1100 users, nothing more. The general lag issue is not solved by it and the systems ressources are far from being used. Vmaps are also not fully responsible from my tests, disabling them just moves the "lag border" a bit and that's all. Something still prevents mangos/trinity to eat all the ressources it can get

Some ppl claim it is all about the database but from what i've seen my 5.0.70 mysql manages about 2000 qps (delete from) without problems. I also tried the experimental high performance version of mysql (http://www.percona.com/docs/wiki/percona-xtradb:start) without getting any noticable improvements.

Atm it is just entirely beyond my understanding how someone can get lagfree past 1100 users (on core4quad 2.4ghz / 8GB for reference).

[DasBlub]

has anyone done once a performance scan which shows us, which parts of mangos are slow? i think the three bottlenecks are the maps, the vmaps and the spellsystem, because all those needs much calculations and they could run parallel. the spellsystem itself has a bad implementation (in my eyes) because everything is hardcoded

[Wowka321]

I've written a small patch for TC that provides similiar features as mtmaps by Derex. It works by splitting workload of MapManager::Update() loop across several threads. It's been tested on TC with good results (60ms update time with 600 players at peak) and id like to share (and test ) this patch with MaNGOS community. Differences are so few that it will be easy to merge. For it to compile you need:

linux

gcc 4.X (for openmp support)

add -fomp to compile flags

link mangos-worldd with gomp library

since TC uses cmakes i cannot provide automake patch to allow it to build, but i know youll be able to do it with no problem - if you will have trouble with it though, i can do it too. Diff is at:

http://bitbucket.org/raczman/trinitycore-patches/src/tip/mtmaps2

Thx u in anyway.

This patch addon for mtmaps by derex, or standalone?

Someone tested it on mangos?

[raczman]

Atm it is just entirely beyond my understanding how someone can get lagfree past 1100 users (on core4quad 2.4ghz / 8GB for reference).

So i am beyond understanding I used to hold 2k on MaNGOS it was a Q6600 and 8 gigs ram, around the time when ACE netcode was still a patch. Mainly it was database configuration issue for me. Even today i still think that DB access is the biggest bottleneck in MaNGOS/TC as of now, i remember Derex wanting to do something about it, but it seems he abandoned that idea.

Thx u in anyway.

This patch addon for mtmaps by derex, or standalone?

Someone tested it on mangos?

It's a standalone - linux only patch. I wrote it for TC1, which is a MaNGOS derivative and closely resembles MaNGOS (im a realist, and i know that 70% of TC is MaNGOS ) - if you can do automake fixes i specified you're free to test it.

[painejake]

Cheers raczman ill take a look at it update and update for automake and mangos.

In regards to the db. Have your tried PostgreSQL? As i know MySQL is supposed to be very slow however i don't know anything about PostgreSQL in means of which is faster at what if one even is faster than the other.

Also a dedicated SQL server. Im not a expert on SPARCs but surely the SPARCs arch would be faster for running a MySQL database on that any standard CPU. Unless its is the mangos side of the database connection that is the bottleneck

[raczman]

Thanks for interest painejake. I've run into several issues with my threading approach - so far im pinpointing each of them trying to prevent problems, but it's still far from being tested enough. Im delighted with it really bringing a speedup - my last project, CORBA vmap cluster failed miserably... Kind of a situation that puts your skill in question, trust me.

As for DB: i used only mySQL, which proved to have enough performance for me. If you have a spare machine in LAN with your MaNGOS server, i recommend you to move SQL there, even if the machine is slightly worse than MaNGOS one.

Also, i found out that VC90++ includes own implementation of openmp, you need to compile with /openmp switch, whatever that means, im not a windows guy myself.

[painejake]

I dont use Windows either. I will try it later on on my homeserver (if i can be bothered to set it up) or on my Mac and see how it goes. Been real busy but hoping to get round to it today

I have a few servers in my LAN but it's not too much of a isse for me however im interested in this mtmaps patch. As they say if it aint broke don't fix it

[higi]

raczman has chosen not to receive private messages or may not be allowed to receive private messages. Therefore you may not send your message to him/her.

could u eneble your PM?

and btw could u or anyone make patchfile for mangos? i woudl really like to test them , i tryied it myself, but still lots off compile errors, so if anyone skilled, it would be helpfull

[raczman]

Oi, i forgot to clean my pms. Cleaned. Also, im on rizon, quakenet and freenode IRC networks if ya need me.

[The_Game_Master]

We need mtmaps. The lag is unbelievable at around 700-1000 users.

[higi]

Oi, i forgot to clean my pms. Cleaned. Also, im on rizon, quakenet and freenode IRC networks if ya need me.

i couldnt find u on irc, could u enable your PMs? i still cant write u))

mangos forum after move to new forum, have automaticky set for old users PM disabled

[raczman]

Enabled - didnt knew pms were disabled by default after forum move.

EDIT: http://sprunge.us/YJTC diff for mangos.

[The_Game_Master]

Thanks for the patch.

I was looking for a patch of mtmaps in the archives, but couldn't find any or extract one.

I'll test your patch and give the results.

One question tough, what are

fopenmp and lgomp?

Is assume are new libs, but can you give more details about them?

[raczman]

Thanks for the patch.

I was looking for a patch of mtmaps in the archives, but couldn't find any or extract one.

I'll test your patch and give the results.

One question tough, what are

fopenmp and lgomp?

Is assume are new libs, but can you give more details about them?

Well, lests start with describing what Openmp is. Openmps is a standard that defines API for shared-memory multiprocessing. Standard is defined by OpenMP folk, and then compiler folk implement the standard. gomp is GNU openmp implementation that gcc uses, MSVC90++ has its own implementation too. -fopenmp is the magic switch that makes #pragma omp directives do its magic work.

[leak]

Just for backchecking:

This "short" patch is supposed to be a full replacement for derex mtmaps patch?