MMaps Redux

[qsa]

After Update Recast to r252. and Changes to fit new Recast API. mobs won't aggro and evade immediately.

Mmaps extracted with hi res paremeter.

Console log:

2010-11-12 12:39:51 ERROR:5712530.mmtile was built with Detour v4, expected v6

2010-11-12 12:39:52 ERROR:5712532.mmtile was built with Detour v4, expected v6

2010-11-12 12:39:52 ERROR:5712431.mmtile was built with Detour v4, expected v6

2010-11-12 12:42:31 ERROR:0005032.mmtile was built with Detour v4, expected v6

2010-11-12 12:42:32 ERROR:0004932.mmtile was built with Detour v4, expected v6

This error is there for a very good reason ...

Just take a wild guess what it means.

TIP: you have to rebuild your maps fresh.

[Konctantin]

there is a small bug, see screenshots:

[qsa]

there is a small bug, see screenshots:

What exactly is the bug?

You generated _some_ "solo mesh tiled" using recastDemo - this mesh has nothing to do with meshes generated and used in this project.

Build recastDemo from our source, then generate maps with debug output using our extractor. Then load them in debug mode.

Only this will give you idea how our actual maps looks like.

Cheers.

[KAPATEJIb]

TIP: you have to rebuild your maps fresh.

oh god... rebuilding again

and...

linux compilation

config.status: error: cannot find input file: dep/recastnavigation/Makefile.in

some places where mmaps doesn't works properly and mobs in these zones tries to jump into the air at aggro (even if you trying to attack them from land) and then evades. http://filebeam.com/9c40d57a6c794e296ce5abc3c565a155.jpg

what about this?

[qsa]

TIP: you have to rebuild your maps fresh.

oh god... rebuilding again

This is "work in process". Having to re-extract once in a while is better than having no improvements at all

PS: you better keep the old maps too, just in case you will have to revert

and...

linux compilation

config.status: error: cannot find input file: dep/recastnavigation/Makefile.in

ye, my bad, added to repository now.

Was hoping to use recast provided files, but they assume totally different things, and will require to build them first, before mangos. So, I added same files we used to have.

some places where mmaps doesn't works properly and mobs in these zones tries to jump into the air at aggro (even if you trying to attack them from land) and then evades. http://filebeam.com/9c40d57a6c794e296ce5abc3c565a155.jpg

what about this?

One thing at a time, people get used to thing getting solved

Cheers

[faramir118]

proposed changes to mmtile format:

https://gist.github.com/674959

Changes:

• writing multiple dtTiles to one file, and associate them with one mangos map tile.
  
• mmap generator version stored in file
  
• more complicated loading/unloading and tracking of loaded tiles
  

Needs some revision, and will probably benefit from a standalone class to simplify reading and book-keeping tasks.

[qsa]

proposed changes to mmtile format:

https://gist.github.com/674959

MoveMap.h is missing from that diff.

Changes looks just fine. Managing of loaded tiles in core looks just a little bit overcomplicated.

Pushing those after tinny local tests can't hurt.

Cheers.

[KAPATEJIb]

memory consuming still increasing every min for 1-2 megs with same online (i mean memory leaks still there) with https://github.com/faramir118/mangos/commit/85a8c143a74fec05023bdf38181d3d959bc13839 and cavern for that i already pointed in above posts still bugged with fresh-extracted mmaps.

Anyway it works more stable than before and total size of mmap folder is decreased (now with hiRes it takes only 1,12 gb instead of 1,29 gb as been before)

about CPU loads... with ~200 online it takes ~45-55% of my Intel Xeon (4x2.5ghz), before using mmaps server takes only ~10-15%

[Damnit]

Hello! Can someone please explain to me some things?

First.. which is the correct branch for extraction? And does mmaps require some configurations.. or I just add it and that's it?

Second.. will this interfere with vmaps.. or can they both work simultaneously?

Thanks!

[Auntie Mangos]

Hello! Can someone please explain to me some things?

It's all in the first post on page 1. Grab the mangos version from faramir118's github repository, then compile and use the new tools included. Most of the new ones depend on a working CMake installation.

$ git clone git://github.com/faramir118/mangos.git

In contrib/mmap/ you will find the extractor needed.

As usual, normal maps and vmaps still need to be extracted. Do that first, then mmap extraction.

Browse the last two or three pages of this thread for current mmaps extraction instructions.

Remember please, this is in development. You should try this only if you intend to submit feedback or code.

[KAPATEJIb]

If anyone can run under valgrind with TBB disabled! (important), and actually get leak trace, I'll be able to fix it. But prior to that, I cannot even confirm its there

seems it's the time to do it coz still getting these annoying crashes http://paste2.org/p/1089469 http://paste2.org/p/1089476

also there is new one http://paste2.org/p/1089477 (never seen this crash before). Using https://github.com/faramir118/mangos/commit/85a8c143a74fec05023bdf38181d3d959bc13839

[qsa]

If anyone can run under valgrind with TBB disabled! (important), and actually get leak trace, I'll be able to fix it. But prior to that, I cannot even confirm its there

seems it's the time to do it coz still getting these annoying crashes http://paste2.org/p/1089469 http://paste2.org/p/1089476

Good conclusion. Since without solid dumps I can't find it. So far, couple of leaks fixed, but guess, there are more.

I just wonder when they happen. Since with normal play with few users, there are none.

also there is new one http://paste2.org/p/1089477 (never seen this crash before)

This one don't look like it has anything to do with mmaps.

Thanks in advance.

Hello! Can someone please explain to me some things?

Please read the first post in this thread, it contains all the info you need. MCP's above post can also help.

[Damnit]

Thanks guys.. yeah I looked in the first thread.. but It seams I overlooked where to get the rep

[KAPATEJIb]

using valrgind now... server dies in lags it takes too much CPU time (99-100%)... is there some paramaters to increase perfomance with valgrind using?

now i will try to use this parameters

valgrind --tool=memcheck --leak-check=yes

maybe it will helps... or i will try to compile without debug info

[qsa]

using valrgind now... server dies in lags it takes too much CPU time (99-100%)... is there some paramaters to increase perfomance with valgrind using?

now i will try to use this parameters

valgrind --tool=memcheck --leak-check=yes

maybe it will helps...

Not really, you better try those things at night while relatively low online counts. Since anything over 50~ testers is unplayable.

PS: I warned you

[KAPATEJIb]

i think better to start valgrind at another (test) server where max online is 10 ppl coz i never has online below 100 players even at night

But running server with really low population wont find anything, I have one

my character table in DB has 23440 chars so these lags aren't surprising

you can run it for an hour at a time

i mean i has incredible lags... it's totally unplayable, you even can't load character list

[qsa]

i think better to start valgrind at another (test) server where max online is 10 ppl coz i never has online below 100 players even at night

well, maybe it can pull 100 on you system for a while.

But running server with really low population wont find anything, I have one

you can run it for an hour at a time, too long logs aren't best either.

[faramir118]

Can't we use a technique like the one I deleted here?

It would take some code refactoring... replacing calls to dtAlloc and dtFree, but it shouldn't impact performance like Valgrind does.

[qsa]

here is results of 2 starts with around of 5 mins of server work (with incredible lags)....

Thanks for the logs, but they do not contain memory trace.

You should with "valgrind --tool=memcheck --leak-check=yes --show-reachable=yes --num-callers=20 --track-fds=yes" flags.

Thanks.

PS: you better remove those logs...

Can't we use a technique like the one I deleted here?

It would take some code refactoring... replacing calls to dtAlloc and dtFree, but it shouldn't impact performance like Valgrind does.

This wont really help us, but I guess you meant using smart pointers.

But, well, that's exactly the thing, it wont really help us.

Just consider the case, when maps are loaded, but not unloaded. Its not really memory leak, its just bad logic. No automatic tool can handle those.

[faramir118]

As long as we have proper destructors, we can print out all remaining non-freed pointers, and where they were allocated from, right before mangos exits (during normal shutdown).

[Ambal]

As long as we have proper destructors, we can print out all remaining non-freed pointers, and where they were allocated from, right before mangos exits (during normal shutdown).

Faramir118, valgrind will do everything for you, so there is no need to write any 'proper' destructors

[qsa]

As long as we have proper destructors, we can print out all remaining non-freed pointers, and where they were allocated from, right before mangos exits (during normal shutdown).

This wont do you any good either ( in some cases ).

Since at full shutdown map destructors are called, freeing all the memory.

We need some luck here, recognizing the exact case when we allocate something and hold it while we don't need it.

I really hope its just plain old memory leak, this way with just tinny luck ( or longer time ) valgrind will catch it, giving much better result. With no additional code.

[Wojta]

KAPATEJIb: At what uptime this leaks and crashes start to appear? Because I have none of this, but maybe it is because my server just crashes too soon. Also, I am not running on newest mmap commit, but tomorrow I will do an update

[Ambal]

New map system applied in [10727]. You have to upgrade your patch to make it compatible with new Core. Personal cheers for MMaps

[qsa]

New map system applied in [10727]. You have to upgrade your patch to make it compatible with new Core. Personal cheers for MMaps

Just moving few things around, no sweat.

Cheers for pushing those changes.