Updating Mtmaps

[raczman]

No, since both patches do things differently, its an alternative.

[reno138]

I have a branch in my repo that has the omp mtmaps patchs (from raczman).

http://github.com/reno/

[divisorxx]

I'm right now using MaNGOS but I try your patch raczman with TC2, it works... I see in other projects like infinitycore by XTZGZoReX , that they use you patch not only for maps, they used in other parts (revs 287,291,292,293) like

MapManager:: DoDelayedMovesAndRemoves

Instance saves

Battleground updates

Outdoor PvP updates

Game events

Do you think that those changes are good?, I mean should improve performance?. Thanks!

PS: I don't know if is allow to post links from other projects, so I don't do it

[XTZGZoReX]

Mind you, those parts are _completely_ untested. I don't even know if it works in practice. Raczman's patch is much more safe as-is, because map/BG/OPvP (TC only) updates are tested and confirmed working (mostly).

MapManager:: DoDelayedMovesAndRemoves

Racz does that as well (and wrote it).

Actually, rephrase: Raczman wrote BG/map/OPvP parallelization. I just added some #pragma omp task's to a few other components. So, full credits to him.

As for my changes: Instance save parallelization seems a bit obsolete to me, after going over the code recently. Instace saves are relatively fast, and not called very often. As for parallelized game events, that would definitely increase performance. When a game event activates/de-activates, a lot of things are changed in the world. And auctions, I'm fairly sure you'd get nice performance by parallelizing it, but it's far from ready for that. It needs several #pragma omp critical sections to work correctly. Parallelized auctions should especially help for those using AHBot.

[raczman]

What the hell.

Zor: I asked you on IRC to NOT include it in your copycat "project", and you said "ok". Being a relatively calm man ill just wont comment on that, but next time remember that patch author has right to say where the patch goes - and i wanted it to be mangos/tc exclusive, since i ONLY want to support those projects that are GPL compliant. And if youre gonna tell me that it's GPL - your "project" broke GPL, files from wyk3d's prepared statement patch have YOUR copyrights - wheres mangos' credit? Hell, every file is marked as if you were original author. every commit from mangos is taken 1:1 and commited without including name of original author, giving false sense of you doing all the work. Keeping a thanks file is not enough.

As for #pragma omp task - i added them in TC version for people who might happen to have omp3.0 compliant compiler, but on earlier versions they wont work as supposed - function will block waiting for the task block to finish, it doesnt matter if it is nowait claused. So basically it's useless as of now, we need to wait for omp3.0 to hit both linux and windows.

[Wowka321]

Yes. can u provide full patch whit your patch + derex. If it possible and can work together.

i tryed, but it's crash.Any of this patch working fine if standalone =/

added: ah, it's alternative...

Sorry

[The_Game_Master]

O i've tested on 700 users. Ideed low latency but more cpu.

Very good patch, but Derex is a little better.

[raczman]

O i've tested on 700 users. Ideed low latency but more cpu.

Very good patch, but Derex is a little better.

The point of it is to use more CPU Thanks for testing, but could you give me some numbers? CPU load without and with this patch - or maybe if you have it in mangos, update time diff.

[Wowka321]

The point of it is to use more CPU Thanks for testing, but could you give me some numbers? CPU load without and with this patch - or maybe if you have it in mangos, update time diff.

on 8 CPU (Xeon5410) 80% main and about 10%% other cores, 1k online

[DasBlub]

on 8 CPU (Xeon5410) 80% main and about 10%% other cores, 1k online

and what about the server uptime from the worldd? did it hold the load? (i mean memory leaks and that damn stuff )

[raczman]

on 8 CPU (Xeon5410) 80% main and about 10%% other cores, 1k online

That's bad. The load should balance evenly on all cores. How many threads did you set for mtmaps?

[Shyke]

Can we get a copy of raczman's patch working for VS2008? I get about five errors.

You'd have a better time debugging this if you tried for yourself, but if you'd like then I'll post errors.

[IAmParadox]

Can we get a copy of raczman's patch working for VS2008? I get about five errors.

You'd have a better time debugging this if you tried for yourself, but if you'd like then I'll post errors.

I have it in testing now for VS2k5 and vs2k8

Edit: It compiled without errors, always a good thing, and once I have made some final touches, it will be ready to convert to mangos. To get prepared, you can go to microsoft.com and get the Platform SDK installed. Editions of VS other than Express may come with the requirements already, but Express edition requires the Platform SDK for omp to work.

[Wowka321]

That's bad. The load should balance evenly on all cores. How many threads did you set for mtmaps?

MapUpdate.Threads = 4

compile whith -fopenmp

gcc 4.3.2

May be i do something wrong?

[Wowka321]

and what about the server uptime from the worldd? did it hold the load? (i mean memory leaks and that damn stuff )

Uptime on lasts revisions 18-36 hours. No memory leaks. (3-4 Gb used, vmaps enabe in raids, arenas) But sometime spelllags (0.1-0.5 sec) when more than 1k online =\\

[leak]

Those are the only errors i get on VC90

9>..\\..\\src\\game\\MapManager.cpp(253) : error C2057: expected constant expression
9>..\\..\\src\\game\\MapManager.cpp(253) : error C2466: cannot allocate an array of constant size 0
9>..\\..\\src\\game\\MapManager.cpp(253) : error C2133: 'update_queue' : unknown size

Troublemaker

Map* update_queue[i_maps.size()];

Since the changes in configure.ac are for linux only, is there something reqiured to make VS aware of omp?

@IAmParadox

How are the "touches" doing?

[raczman]

he pushed that patch to my patchqueue at bitbucket, with name "mtmaps3".

[pr0vieh]

is this Patch Working Correct for mangos ?

or need changes ?

[IAmParadox]

is this Patch Working Correct for mangos ?

or need changes ?

You may have to patch it manually, since the position of things can be different, but, it should work.

[leak]

Yep, it works for mangos.

11h uptime with 550 users max atm. Gonna give some more info about performance behavior later.

Little addition:

On gentoo i had to recompile gcc with +openmp USE flag and to use the -fopenmp (not -fomp, that doesn't exist - see man gcc) flag on the configure command.

[raczman]

Hmm,

+MANGOS_INCLUDES="$POSTGRE_INCLUDES $MYSQL_INCLUDES $MANGOS_INCLUDES -fopenmp"

I added -fopenmp, at least in the diff from previous page. I really dont know where did i add -fomp, but if i did - sorry - i was writing mangos diff after pulling an all-nighter - yay for no sleep.

[divisorxx]

Yep, it works for mangos.

11h uptime with 550 users max atm. Gonna give some more info about performance behavior later.

Little addition:

On gentoo i had to recompile gcc with +openmp USE flag and to use the -fopenmp (not -fomp, that doesn't exist - see man gcc) flag on the configure command.

Thanks.. i'm waiting for your results and how is compare to Derek's mtmaps. I would like to test my self... but i update to gcc 4.4.0 and seems not compile right now

[leak]

linux

gcc 4.X (for openmp support)

add -fomp to compile flags

link mangos-worldd with gomp library

I was refering to this list, but since i believe ppl who deal with this kinda stuff should be able to read man pages - no damage done

17695 mangos    20   0 3703M 3447M  9300 S  0.0 43.3  0:21.30   1  |       `- ./mangos-worldd
17710 mangos    20   0 3703M 3447M  9300 S 18.0 43.3  1h41:22   3  |           `- ./mangos-worldd <-- net?
17709 mangos    20   0 3703M 3447M  9300 S 18.0 43.3  1h42:48   1  |           `- ./mangos-worldd <-- net?
17708 mangos    20   0 3703M 3447M  9300 S  1.0 43.3 24:41.06   2  |           `- ./mangos-worldd
17707 mangos    20   0 3703M 3447M  9300 S  0.0 43.3  0:00.57   3  |           `- ./mangos-worldd
17706 mangos    20   0 3703M 3447M  9300 S 13.0 43.3  1h23:22   2  |           `- ./mangos-worldd <--map?
17705 mangos    20   0 3703M 3447M  9300 S  0.0 43.3  0:31.15   3  |           `- ./mangos-worldd
17704 mangos    20   0 3703M 3447M  9300 S 11.0 43.3  1h23:23   3  |           `- ./mangos-worldd <--map?
17703 mangos    20   0 3703M 3447M  9300 R  2.0 43.3 22:46.08   4  |           `- ./mangos-worldd
17702 mangos    20   0 3703M 3447M  9300 S 11.0 43.3  1h23:25   4  |           `- ./mangos-worldd <--map?
17701 mangos    20   0 3703M 3447M  9300 R 83.0 43.3  7h55:32   1  |           `- ./mangos-worldd <--main?
17699 mangos    20   0 3703M 3447M  9300 R  0.0 43.3  0:45.92   4  |           `- ./mangos-worldd
17698 mangos    20   0 3703M 3447M  9300 S  0.0 43.3  3:50.42   1  |           `- ./mangos-worldd
17697 mangos    20   0 3703M 3447M  9300 S  0.0 43.3  2:15.37   1  |           `- ./mangos-worldd

This htop shot was taken from 1150 users. I must say that i can't really see any different load spreading amongst the cores/threads, i also have the impression that the what i call "main" thread is going now even higher than with derex patch. Another "weird" thing that i can't find the 4th map thread i added - usually there always were 4 threads with similar load.

Also i can feel the spell lag coming back at 1100 compared to the 100% smoothness on 550 players.

(p.s. Don't get me wrong, despite the quite negative feedback of mine i'm always open for stuff like that since every other effort failed to kill the spell lag past 1100ppl has failed so far)

[raczman]

It looks like wether i made a mistake in mangos diff - or you have some problems with DB performance. But thanks for thorough testing, i appreciate that! Will look into what you posted ASAP.

[higi]

It looks like wether i made a mistake in mangos diff - or you have some problems with DB performance. But thanks for thorough testing, i appreciate that! Will look into what you posted ASAP.

tested your patch

772 root 10 -10 4560m 4.1g 9.9m S 63 25.9 12:43.79 mangosd

779 root 10 -10 4560m 4.1g 9.9m S 9 25.9 1:39.25 mangosd

11582 mysql 10 -10 4813m 2.6g 6512 S 9 16.7 81:22.00 mysqld

777 root 10 -10 4560m 4.1g 9.9m R 8 25.9 1:35.06 mangosd

778 root 10 -10 4560m 4.1g 9.9m S 8 25.9 1:37.03 mangosd

780 root 10 -10 4560m 4.1g 9.9m S 8 25.9 1:35.61 mangosd

782 root 10 -10 4560m 4.1g 9.9m S 6 25.9 0:55.51 mangosd

786 root 10 -10 4560m 4.1g 9.9m S 6 25.9 0:55.48 mangosd

784 root 10 -10 4560m 4.1g 9.9m S 5 25.9 0:56.37 mangosd

785 root 10 -10 4560m 4.1g 9.9m S 5 25.9 0:55.56 mangosd

781 root 10 -10 4560m 4.1g 9.9m S 4 25.9 0:56.32 mangosd

783 root 10 -10 4560m 4.1g 9.9m S 4 25.9 0:56.01 mangosd

787 root 10 -10 4560m 4.1g 9.9m S 3 25.9 0:55.81 mangosd

773 root 10 -10 4560m 4.1g 9.9m S 1 25.9 0:26.92 mangosd

1333testers 8 map threads, it runs almost same like derex mtmaps(derex maps are broken, some old rev was run all CPU 40,40,40,40,40,...), i could say your patch a little bit better, but it still dont take all cores anyway GJ