MMaps Redux

[mrelfire]

My test on little server ( <10 player) give me impression that all mob have a real reaction (better than blizz?). So it's time to think about moving this to underreview...

Thank you qsa and faramir118. You are my exemple

yes it seems it is ok to go under review

[Oniryck]

I'm sorry to bring that back, but did anyone else tried to apply this great project to a mangos-one ? I'm still trying to help, but didn't find out yet why the mobs disappear when aggroed. Maybe it's the mmaps extractor that needs more modification than I already did (I just in fact changed the MAGIC ver. and deleted the "holes" part of the extraction.

If somebody more competent then me have some leads, I'll try to dig a little more. Thank to all who works and this project anyway

[qsa]

It's still relevant, but only this part is the fix for our issue:

diff --git a/src/game/Unit.cpp b/src/game/Unit.cpp
index 38b78c1..7c043b0 100644
--- a/src/game/Unit.cpp
+++ b/src/game/Unit.cpp
@@ -321,7 +321,7 @@ void Unit::Update( uint32 update_diff, uint32 p_time )
        getThreatManager().UpdateForClient(update_diff);

    // update combat timer only for players and pets
-    if (isInCombat() && (GetTypeId() == TYPEID_PLAYER || ((Creature*)this)->IsPet() || ((Creature*)this)->isCharmed()))
+    if (isInCombat() && GetCharmerOrOwnerPlayerOrPlayerItself())
    {
        // Check UNIT_STAT_MELEE_ATTACKING or UNIT_STAT_CHASE (without UNIT_STAT_FOLLOW in this case) so pets can reach far away
        // targets without stopping half way there and running off.

Why don't you push it. One bug less.

Just as question would it be possible to have NPCs using mmaps on their default movement?

It is their default movement.

On ytdb there is a certain Ironforge guard moving right through the great forge, if it was using mmaps it should not move through lava and fly across the air.

Mmaps will not work with gameobjects (forge in ironforge) - same reason why LoS don't work. We cannot implement it till LoS has it working.

About that guard flying around : I guess it uses waypoints, if they are set wrongly (he have waypoint to unreachable location), you can get some strange effects.

Other than that, these special places (like in bootybay) where movement is blocked by "strange" environment can be fix after adding the mmaps patch to the core. They are spotted by coincidence.

See post #891, those spacial places have to be fixed manually. Dock in bootybay, pillars in blade's edge arena, maybe some otehrs. In any case, there's a mechanics that allow solving those.

So it's time to think about moving this to underreview...

I think we need to list all known issues first. Then, get it tested on major sized server for some time and get proper feedback. Only then we can start talking about moving to review.

In any case, it can be nice to get some remarks about the general design from one of the devs.

I'm sorry to bring that back, but did anyone else tried to apply this great project to a mangos-one ? I'm still trying to help, but didn't find out yet why the mobs disappear when aggroed. Maybe it's the mmaps extractor that needs more modification than I already did (I just in fact changed the MAGIC ver. and deleted the "holes" part of the extraction.

Make sure the maps extracted by the ported extractor are proper. Open them in our recast demo and see if you get what you expect there.

Afterwards, you have to be sure you ported the core part properly. Keep in mind, that the current implementation of map loading/unloading system tied to terrain manager code, which in tern has to be backported first. Alternatively, you can use our old scheme, the one we used before terrain manager, but I think it is easier and safer to backport.

[cyberium]

cyberium wrote:

So it's time to think about moving this to underreview...

I think we need to list all known issues first. Then, get it tested on major sized server for some time and get proper feedback. Only then we can start talking about moving to review.

In any case, it can be nice to get some remarks about the general design from one of the devs.

Sorry i add my two cent.

Generaly not enough feedback until the code is realy released... If you want to have dev review beter chance is to place this in underreview section. Eventualy add [DEV] tag to your post

For known bug list, i think we are in acceptable area. I mean release and get last bug list directly from users.

It's not a kind of pressure

[Feanordev]

Qsa about dynamic LOS - I saw SilverIce working on it I guess, I hope he will release some under-development patch or something for it some day

[Ambal]

Yes, SilverIce already put his changes regarding dynamic LOS in his git repo - you are free to check it.

[Oniryck]

qsa : I tried to open my mmaps file into RecastDome but could manage to do it (such a noob...) I renamed them .map and .mesh in the meshes folder but the app crashes. I took the files from the mmaps folder. Could you give me a hint on how I should open the mmaps in RecastDemo ?

[qsa]

Yes, SilverIce already put his changes regarding dynamic LOS in his git repo - you are free to check it.

Qsa about dynamic LOS - I saw SilverIce working on it I guess, I hope he will release some under-development patch or something for it some day

Good to know someone working on that issue.

In general, I don't think we should have support for dynamic objects as pre-requirement to "first stage".

qsa : I tried to open my mmaps file into RecastDome but could manage to do it (such a noob...) I renamed them .map and .mesh in the meshes folder but the app crashes. I took the files from the mmaps folder. Could you give me a hint on how I should open the mmaps in RecastDemo ?

You have to compile recastDemo from our repository, then copy the binary to folder with SDL.dll, DroidSans.ttf - you can find those somewhere inside recastdemo folder.

Then, you should extract mmaps with their debug info. You have two folders now, one named "mmaps" and another "meshes", copy them into same folder as recast binary. There's no need renaming anything.

Thats about it, if the format is proper, recastDemo will have no issues opening extracted maps.

[Oniryck]

Okay, thanks qsa, I'll extract them with --debugOutput

[Oniryck]

Okay, I extracted the debug mmaps, for BWL, precisely the platform where Razorgore is standing. The strcuture looks okay but the path looks... weird. I don't know how to really interpret the results, maybe you'll have a better opinion. Here's an image of it :

http://img826.imageshack.us/i/bwlv.jpg/

[Undergarun]

I test it on a 2000+ ppl online realm and i have had some freezes:

http://pastebin.com/i8ghS0WH

http://pastebin.com/Yaj27DhT

http://pastebin.com/f8Wirm5e

http://pastebin.com/HGnG59p9

http://pastebin.com/m6xv5Hef

[faramir118]

I don't know how to really interpret the results, maybe you'll have a better opinion.

You're using the wrong tool - on the left, select Pathfind Follow or Pathfind Straight.

I test it on a 2000+ ppl online realm and i have had some freezes

Can you tell us some info about your server?

• which maps you have mmaps enabled for
  
• which compiler optimizations you applied
  
• mtmaps and how many threads
  

[Undergarun]

I test it on a 2000+ ppl online realm and i have had some freezes

Can you tell us some info about your server?

• which maps you have mmaps enabled for
  
• which compiler optimizations you applied
  
• mtmaps and how many threads
  

All mmaps enabled.

GCC => CFLAGS="-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O3 -m64 -pipe -msse4.2 -DFD_SETSIZE=4096 -fno-delete-null-pointer-checks -fno-strength-reduce -fno-strict-aliasing -frename-registers" CXXFLAGS="-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O3 -m64 -pipe -msse4.2 -DFD_SETSIZE=4096 -fno-delete-null-pointer-checks -fno-strength-reduce -fno-strict-aliasing -frename-registers"

mtmaps = 3 threads + Visibility And reloc merged.

SO: Ubuntu Server 9.10

RAM: 24 GB DDR3 ECC

[Undergarun]

Thank you QSA, works as intended! No freezes anymore for now...

[iforgotmypassword]

Anyway to reproduce it? doe's it always happen, or just sometimes?

sometimes, when you have a pet and pet follows you to the pillar, or when pet attacks target on pillar

Can you please check if you get same problem in other places in world when using off mesh connections?

I'll check it later

[Oniryck]

I don't know how to really interpret the results, maybe you'll have a better opinion.

You're using the wrong tool - on the left, select Pathfind Follow or Pathfind Straight.

Hmm *cough* thanks I was pretty sure I selected the good tools but it should have deselected it while I was searching for the "build" button ^^

Anyway, here's the result : http://img703.imageshack.us/i/bwla.jpg/

It looks ok I think, so the reason why the mob desappear should be in the server core and not caused by the mmaps generator. Probably because of the remaining differences between master and one, like ambal's modifications like you said. This is maybe still over my head but I'm still available for help I can do any good.

[Undergarun]

New freezes in another two maps:

http://pastebin.com/RtkbKT7B

http://pastebin.com/pi8Trd1Y

[qsa]

New freezes in another two maps:

http://pastebin.com/RtkbKT7B

http://pastebin.com/pi8Trd1Y

Different crash, but I guess they caused by similar issue.

Can you please try this : http://pastie.org/1568503 - thanks in advance.

PS: how often does it crash?

It looks ok I think, so the reason why the mob desappear should be in the server core and not caused by the mmaps generator. Probably because of the remaining differences between master and one, like ambal's modifications like you said. This is maybe still over my head but I'm still available for help I can do any good.

You are half way there

Anyway to reproduce it? doe's it always happen, or just sometimes?

sometimes, when you have a pet and pet follows you to the pillar, or when pet attacks target on pillar

Thanks, found a way to crash it. Now only left to find a way to solve it, or atleast understand why it does.

mehh, all those new issues coming from pretty deep in recast code

[Undergarun]

New freezes in another two maps:

http://pastebin.com/RtkbKT7B

http://pastebin.com/pi8Trd1Y

Different crash, but I guess they caused by similar issue.

Can you please try this : http://pastie.org/1568503 - thanks in advance.

PS: how often does it crash?

Just one each in 24 hours.

[Undergarun]

Hi qsa

All freezes disapear with your patches but i notice a mem-leak in somewhere

This is my 1000+ ppl realm after 6-7 hours of uptime with mmaps.

Cheers

[iforgotmypassword]

Undergarun, try valgrind

[KiriX]

Need update to latest mangos revision =(

[qsa]

Hi qsa

All freezes disapear with your patches but i notice a mem-leak in somewhere

This is my 1000+ ppl realm after 6-7 hours of uptime with mmaps.

Cheers

Unfortunately, this is not a memory leak, it is the actual memory usage under this model

You see, what used to happen, that all creatures on same map used to share dtNavMeshQuery (the engine that does all the calculation). What happens now, is that every path calculation has its own dtNavMeshQuery. Just like we used to have while back. Well, as you guessed, it is pretty expensive, memory wise.

What it made me realize, is that I was looking for the problem in wrong place.

The actual issue is thread safety. As "simple" as that.

Right now, we have single dtNavMeshQuery per map. So, what happens, is that two different instances from two different threads of that map use it simultaneously, that's why we get those results and consequentially, crashes.

Find the problem, is about 70% of the work, now its just a matter of fixing it. I'll allocate query for every instance.

Going to change our loading/unloading model a little bit, we'll have to hook somewhere in Map class

Please do correct me if I'm wrong with the logic here. From what I can tell, every instance run in thread of its own right now, not every map.

If, that is not the case, spare me some time digging into instance code.

Need update to latest mangos revision =(

I merged the latest.

Unfortunately, I do not have *nix system with mangos at the moment, so, someone else will have to make cmake files, or wait till I get to setting one up again, which wont be in any immediate future. (thanks in advance)

Cheers people.

[Schmoozerd]

shouldn't you be relatively easy (and save) when moving the movement-meshes to terrain manager?

[qsa]

shouldn't you be relatively easy (and save) when moving the movement-meshes to terrain manager?

Movement meshes are for pretty long time now loaded/unloaded from terrain manager, just like VMAPs.

They are not the issue. The issue is dtNavMeshQuery - the part that does the calculations on top of those meshes.

It is not thread safe on itself. So, we have to have one of those per instance and NOT per map.

We "cannot" have it per path calculation either, since it is pretty heavy on memory.