mtmaps (OpenMP)

[Infinity_return]

But i'm a little confuse, for use the ace patch, is necessary update the ace code?

EDIT: And is necessary compile with gcc 4.4? i compile with 4.1, and i no see any problem.

OpenMP:

In gcc 4.3.x OpenMP 2.0, gcc 4.4.x OpenMP 3.0

maybe in gcc 4.1.x, 4.2.x OpenMP 1.0 ?

i tested on gcc 4.3.3, 4.3.4, 4.4.2 works fine

ACE:

latest version(5.7.4):

http://download.dre.vanderbilt.edu/previous_versions/?C=M;O=D

i tested on ACE 5.6.9, 5.7.0, 5.7.2

but on 5.7.4 i have had problems with stability, i use 5.7.2

I recommend to use gcc 4.4.2 and ACE 5.7.2

[nanounico]

But what is the real diference, between, Ace Mtmaps, or Open Mtmaps?

I do a fast comparation, with 4 cores:

With Mtmaps, the 4 CPU Usage is 20-30%

With Ace Mtmaps, the 4 CPU Usage is 5-20%

In this momenth, i'm using Ace Mtmaps with Ace 5.7.2, but i no have much testers right now, so, with larger servers, like 2000, or more, what is the result?

EDIT: and the other important quest, is, wish give less crashes!

EDIT 2: I really want compile with gcc 4.4.2, but RHEL 5.4 dont have rpm for that :'(

[Wowka321]

I tested both. For me ACE mtmaps looking a little better. . A little better stability, performance than OpenMP.

P.S. OS -Ubuntu 8.10 amd64. GCC 4.4.2. Tested with ACE 5.6.6, now test in progress with ACE 5.7.3

[balrok]

hello - openmp crashed like hell here.. i added back the critical section at playerupdate - then it went fine again

this is what we used: http://github.com/balrok/mangos/commit/e571922be2aeb58288b7ffcf3d82f47de6aec6d6

only problem it started to lag with around 800 players (there seems to be a change between 313 and 322 which has broken openmp performance)

now i'm going to try ace - at a first glance it looked very good.. let's see how it behaves under high load

but currently received one freeze after only 15minutes with ace - i'm trying now your recommended ace-version cause we still used the systems default

[nanounico]

Balrok, finally works better the Ace MP, with 5.7.2? 5.7.3? 5.7.4? or OpenMP?

[xarly]

hello just want to note here, that we're running mangos with latest 3.1.3 commit

very stable with this patch by derex: http://pastebin.com/m7d3df483

stable means in this case no crash for 48h with 1200 user peak

actualy i had to implement two patches (but are hacks ) for some long known crashes/freeze - but don't know if they are related to the patch so i won't post them here

Hi, I'm starting with this patch, but only one question, I only need to apply http://pastebin.com/m7d3df483, nothing else?

Does it works on 64 bits? I have tried, and it compiles well, but on run says (in eventvwr):

Error al generar el contexto de activación para "c:\\mangos\\mangosd.exe". No se encontró el ensamblado dependiente Microsoft.VC90.OpenMP,processorArchitecture="amd64",publicKeyToken="1fc8b3b9a1e18e3b",type="win32",version="9.0.21022.8". Use sxstrace.exe para obtener un diagnóstico detallado.

Google translator:

Failed to generate the activation context for "c:\\mangos\\mangosd.exe". Could not find dependent assembly Microsoft.VC90.OpenMP, processorArchitecture = "amd64", publicKeyToken = "1fc8b3b9a1e18e3b", type = "win32", version = "9.0.21022.8". Use sxstrace.exe for detailed diagnosis.

Any idea?

Thank you

PD: Balrok, could you post your patches to fix crashes/freeze? Thank you

[balrok]

hello i currently run http://github.com/balrok/mangos/commits/322_server

BUT mtmaps isn't working here.. there are insanely huge (constant) lags at 900 players. (~1000ms)

and all mtmaps patches posted in this thread don't work.. they crash very often

so i took my own mtmaps-solution (almost the same as posted in this thread but i removed playerupdate-multithreading and some locks) http://github.com/balrok/mangos/commit/039cf325b03adef046ee3c0f196a96e587a130a4

but this looks quite the same as the patch i used in 313..

so in 313 everything was running smooth so i compare them:

with 313 i had this htop:

http://balrok.ath.cx/htop.png

with 322 this:

http://balrok.ath.cx/htop2.png

in 313 first thread was running 9.20, 2nd 4.11 and all other around 2..

so almost 1:2:4:4:4:4.. ratio..

with 322 first thread runs 5.19 2nd,3rd: 1.04 and all other 0.45

so around 1:5:5:6:6:6:6... ratio

also with 322 i have the problem that i can't change the number of running threads through the config anymore and there is now an additional thread compared to 313-htop

for me it looks a bit like openmp is gone crazy

if you have any suggestions you are more than welcome.. as you might have seen last commits from me in mangos were all dedicated to performance but no luck yet

[kero99]

hello i currently run http://github.com/balrok/mangos/commits/322_server

BUT mtmaps isn't working here.. there are insanely huge (constant) lags at 900 players. (~1000ms)

and all mtmaps patches posted in this thread don't work.. they crash very often

so i took my own mtmaps-solution (almost the same as posted in this thread but i removed playerupdate-multithreading and some locks) http://github.com/balrok/mangos/commit/039cf325b03adef046ee3c0f196a96e587a130a4

but this looks quite the same as the patch i used in 313..

so in 313 everything was running smooth so i compare them:

with 313 i had this htop:

http://balrok.ath.cx/htop.png

with 322 this:

http://balrok.ath.cx/htop2.png

in 313 first thread was running 9.20, 2nd 4.11 and all other around 2..

so almost 1:2:4:4:4:4.. ratio..

with 322 first thread runs 5.19 2nd,3rd: 1.04 and all other 0.45

so around 1:5:5:6:6:6:6... ratio

also with 322 i have the problem that i can't change the number of running threads through the config anymore and there is now an additional thread compared to 313-htop

for me it looks a bit like openmp is gone crazy

if you have any suggestions you are more than welcome.. as you might have seen last commits from me in mangos were all dedicated to performance but no luck yet

Yesterday we had 2700+ player online 6H uptime (yes soo bad... but good with 3.2.2a and these population) and no lag... with ACE MTMaps + gcc 4.4.2 + ACE 5.7.2, rates 60% sometimes in some core. In me experience with mangos, the secret of performance (if you have enouth CPU and RAM) are 2 important factor:

- 1) MySQL with INNODB Tunning Configuration: Without very very good config and tunnig... mangos =lag. 950+ needed

- 2) HDD IO: Without good rate IO in HDD of MySQL witn 1800+... mangos=lag

Yes.. if you have a good server and good configuration for DB. Lag is gone

Best Regards

[xarly]

hello i currently run http://github.com/balrok/mangos/commits/322_server

BUT mtmaps isn't working here.. there are insanely huge (constant) lags at 900 players. (~1000ms)

and all mtmaps patches posted in this thread don't work.. they crash very often

so i took my own mtmaps-solution (almost the same as posted in this thread but i removed playerupdate-multithreading and some locks) http://github.com/balrok/mangos/commit/039cf325b03adef046ee3c0f196a96e587a130a4

but this looks quite the same as the patch i used in 313..

so in 313 everything was running smooth so i compare them:

with 313 i had this htop:

http://balrok.ath.cx/htop.png

with 322 this:

http://balrok.ath.cx/htop2.png

in 313 first thread was running 9.20, 2nd 4.11 and all other around 2..

so almost 1:2:4:4:4:4.. ratio..

with 322 first thread runs 5.19 2nd,3rd: 1.04 and all other 0.45

so around 1:5:5:6:6:6:6... ratio

also with 322 i have the problem that i can't change the number of running threads through the config anymore and there is now an additional thread compared to 313-htop

for me it looks a bit like openmp is gone crazy

if you have any suggestions you are more than welcome.. as you might have seen last commits from me in mangos were all dedicated to performance but no luck yet

Thank you Balrok, Ii have tried to apply your patch but it look like iit's no fully compatible with rev. 8587... which was your 313 revision?

Sorry, fixed.

The problem was some libraries of VS2008 were not instaled on the server

[xarly]

i tested on ACE 5.6.9, 5.7.0, 5.7.2

but on 5.7.4 i have had problems with stability, i use 5.7.2

I recommend to use gcc 4.4.2 and ACE 5.7.2

I would like to know how use ACE MTmaps:

I have patched my rev 8850 with

mtmaps ACE (8799+):
[url]http://paste2.org/p/508050[/url]

and i have downloaded Ace 5.7.2 from"http://download.dre.vanderbilt.edu/previous_versions/?C=M;O=D", but i don't know what should i do with this, i mean, should I replace ACE_wrapper folder of Mangos or... I don't know :confused:

Could you help me?

Thank you very much

[Span1sh]

Dear, kero99 , excuse me please for my poor English, but could you post your comments for these two moments:

- 1) MySQL with INNODB Tunning Configuration: Without very very good config and tunnig... mangos =lag. 950+ needed

- 2) HDD IO: Without good rate IO in HDD of MySQL witn 1800+... mangos=lag

I'm interested in:

1) Your MySQL config.

2) What HDD are you using?

3) Any other tunning that you made in the core.

4) And if you are using MTMAPS patch with your own modifications, could you share it please or put a reference?

[nanounico]

Yes i think, could be interesting compare MySQL configuration, and personal tuning, configuration in the core.

[balrok]

hello - i found out that openmp didn't work.. it just spawns as many threads as it wants.. and disabling it helped to make the server at least playable with around 1300 users (lag ~400-600)

ace-mtmaps seems to produce some deadlocks or other errors - therefore i can't realy use it.. i will look soon if i can fix some of those.. cause the load gets realy nice divided between all cpu

and for installing ace: you either can copy the ace-version over that in mangos..

or you install ace as system library and then configure mangos, that it uses your system-ace.. i've added an ace-version output at mangos-start - so you can check if all was succesful (cause i like the system-library method quite much - since compiling ace takes hours..)

ps: i also hope to see some good mysql/mangos configs and/or a working ace-mtmaps patch

[Infinity_return]

I'm used ace mtmaps on last rev, very stable in MapUpdate.Threads = 2 uptime 8-21 h on 2800-3100 ppl online, without lags

My mysql.conf

max_allowed_packet = 128M
table_cache = 4096
join_buffer_size = 8M
sort_buffer_size = 4M
read_buffer_size = 4M
read_rnd_buffer_size = 16M
myisam_sort_buffer_size = 128M
thread_cache_size = 64
thread_stack = 256K
query_cache_size = 512M
query_cache_limit = 32M
thread_concurrency = 16
innodb_file_per_table=1
innodb_file_format=Barracuda
innodb_buffer_pool_size        = 955M
innodb_additional_mem_pool_size = 32M
innodb_flush_log_at_trx_commit = 0
innodb_flush_method            = O_DSYNC
innodb_table_locks = 0
innodb_thread_concurrency = 8

Need install Innodb plugin, +10% performance and added in my.conf

Innodb plugin for mysql 5.1.37 http://www.innodb.com/download/innodb_plugin/innodb_plugin-1.0.4-linux-x86_64-glibc23.tar.gz

ignore_builtin_innodb
plugin_load=innodb=ha_innodb.so;innodb_trx=ha_innodb.so;innodb_locks=ha_innodb.so;innodb_lock_waits=ha_innodb.so;innodb_cmp=ha_innodb.so;innodb_cmp_reset=ha_innodb.so;innodb_cmpmem=ha_innodb.so;innodb_cmpmem_reset=ha_innodb.so

Mangos compile flags:

../configure CFLAGS="-O2 -m64 -pipe -DFD_SETSIZE=4096 -march=core2 -fomit-frame-pointer -ffast-math -fno-strength-reduce -fno-strict-aliasing -frename-registers" CXXFLAGS="-O2 -m64 -pipe -DFD_SETSIZE=4096 -march=core2 -fomit-frame-pointer -ffast-math -fno-strength-reduce -fno-strict-aliasing -frename-registers" --prefix=/home/server --sysconfdir=/home/server/etc --datadir=/home/server --enable-cli --enable-ra --disable-builtin-ace

Packets and OS:

ACE 5.7.2, gcc 4.4.2, mysql 5.1.37-2, Linux Debian 5.0 64 bit

CPU Core i7 975 3.3Ghz Quad Core x2

12GB RAM DDR3

HDD 300 Gb SAS 15000rpm x2

[leak]

Few questions:

Vmap and visibilty range settings?

What exactly is wrong with the built in InnoDB plugin? (http://www.percona.com/docs/wiki/release:start I tried i high performance mod from those guys before, didn't really help with performance)

What's the "x2" on the i7 CPU? The i7 isn't a SMP CPU.

Also the innodb_buffer_pool_size looks way too smal for a 3k player char db. (Unless ofc there are only 3k players and zero unused accounts)

[Infinity_return]

Intel® Core i7 CPU 975 @ 3.33GHz Quad Core x 2, 8 cores

vmap.enableLOS = 1
vmap.enableHeight = 1
vmap.ignoreMapIds = "369, 0, 1, 571, 530"
vmap.ignoreSpellIds = "7720"
vmap.totems = 1(my patch)
vmap.pets = 1(my patch)
DetectPosCollision = 0
TargetPosRecalculateRange = 2


Visibility.GroupMode = 0
Visibility.Distance.Continents    = 66
Visibility.Distance.Instances     = 60
Visibility.Distance.BGArenas      = 66
Visibility.Distance.Object        = 60
Visibility.Distance.InFlight      = 50
Visibility.Distance.Grey.Unit   = 1
Visibility.Distance.Grey.Object = 10

#Visibility patch
Visibility.Notify.Period.OnContinents = 1100
Visibility.Notify.Period.InInstances  = 1100
Visibility.Notify.Period.InBGArenas   = 1100

innodb_buffer_pool_size normal ~900, no problems

Info for innodb plugin and mysql 5.1:

http://greenmice.info/en/node/85(russian)

[nanounico]

Using High performance Mysql (Percona Perf) 5.086, i7, gcc 4.1, ace 5.7.0, RHEL 5.4 x64, 18GB Ram, HDD 146GB 15K rpm, 2500 Without Lagg.

PD: The percona builds, at least for me, is a big problem, becouse, i need do this procedure:

1. Unistall Percona

2. Install Mysql normal

3. Compile Core

4. Unistall mysql normal

5. install Percona

Why? becouse with percona builds, or xtradb, Mangos never find the mysql libraries and that is really enoyng!

my.cnf

max_allowed_packet = 32M

table_cache = 4096

join_buffer_size = 8M

sort_buffer_size = 4M

read_buffer_size = 4M

read_rnd_buffer_size = 32M

myisam_sort_buffer_size = 128M

thread_cache_size = 64

thread_stack = 526K

query_cache_size = 356M

query_cache_limit = 64M

long-query-time = 2

innodb_file_per_table = 1

innodb_table_locks = 0

innodb_file_format = Barracuda

innodb_buffer_pool_size = 3G

innodb_additional_mem_pool_size = 32M

innodb_flush_log_at_trx_commit = 0

innodb_flush_method = O_DSYNC #or O_DIRECT

innodb_thread_concurrency = 16

innodb_checksums = 0

innodb_support_xa = 0

innodb_doublewrite = 1 # This could be improve 30% if you are using Log's correctly

innodb_adaptive_flushing = 1

innodb_max_dirty_pages_pct = 90

//Configure

 ../configure CFLAGS="-O2 -m64 -pipe -DFD_SETSIZE=4096 -march=core2 -fomit-frame-pointer -ffast-math -fno-strength-reduce -fno-strict-aliasing -frename-registers" CXXFLAGS="-O2 -m64 -pipe -DFD_SETSIZE=4096 -march=core2 -fomit-frame-pointer -ffast-math -fno-strength-reduce -fno-strict-aliasing -frename-registers" --prefix=/home/server --sysconfdir=/home/server/etc --datadir=/home/server --enable-cli --enable-ra --disable-builtin-ace

[leak]

Always post the performance related mangos settings, otherwise your info is not that useful.

Also i think we should start looking for some mysql benchmark tool and start comparing benchmarks to find out what are the most suitable settings for mangos. (Mangos itself doesn't give an objective answer on that, "lagfree" is relative)

[nanounico]

Always post the performance related mangos settings, otherwise your info is not that useful.

Also i think we should start looking for some mysql benchmark tool and start comparing benchmarks to find out what are the most suitable settings for mangos. (Mangos itself doesn't give an objective answer on that, "lagfree" is relative)

How you say the mysq configuration is not useful?? lol... try have more than 3k without tune mysql!

Off course, the next way is try use performance benchmarks, i really love Percona perf..

[Darky88]

Mangos compile flags:

../configure CFLAGS="-O2 -m64 -pipe -DFD_SETSIZE=4096 -march=core2 -fomit-frame-pointer -ffast-math -fno-strength-reduce -fno-strict-aliasing -frename-registers" CXXFLAGS="-O2 -m64 -pipe -DFD_SETSIZE=4096 -march=core2 -fomit-frame-pointer -ffast-math -fno-strength-reduce -fno-strict-aliasing -frename-registers" --prefix=/home/server --sysconfdir=/home/server/etc --datadir=/home/server --enable-cli --enable-ra --disable-builtin-ace

Can you explain the options you used in the configure and that reasons or tell me somewhere where are explaining?

Att Darky

[Infinity_return]

-DFD_SETSIZE=4096 - for login 1k+

-O2 - optimize compile flag

-pipe - optimize compile flag

-m64 - for 64bit OS

-march=core2 - optimization for CPU, if gcc 4.3+ core2 or if Xeon -march=nocona

if Core 2 Duo, Quad Core -march=prescott

-ffast-math - fast mathematical operations

-fno-strength-reduce -fno-strict-aliasing -frename-registers -fomit-frame-pointer not remember, but not over

[leak]

How you say the mysq configuration is not useful?? lol... try have more than 3k without tune mysql!

No you got me wrong. I was asking to provide the relevant mangos.conf values (like Infinity did) together with your mysql config. Stuff like vmaps and visibility range has a major impact on performance.

[Infinity_return]

If you are using mtmaps, need:

MapUpdate.Threads = 2 in mangosd.conf, 4+ very unstable

disable save,saveall commands and server save time interval 10-20 min.

And patch for database, prevent crashes on SqlDelayThread and removed ACE_Based::Thread::Sleep(10), thx Derex and Ambal.

http://paste2.org/p/528578 - updated, fixed typo in include, sorry

[Span1sh]

I'm used ace mtmaps on last rev, very stable in MapUpdate.Threads = 2

Can you give me some link to the patch that you are using please?

[Infinity_return]

Can you give me some link to the patch that you are using please?

mtmaps ACE (8799+):

http://paste2.org/p/508050