MMaps Redux

[qsa]

There are some problems in the extractor (ad.exe) that might be contributing to the issues qsa and I posted.

In the mmaps_rewrite version of contrib/extractor/System.cpp, the loop @ line 775 is wrong.

It iterates over quads (128x128), but acts on vertices

• sets the top-left vertex of a quad to invalid height, which affects all quads on that corner.
  Conceptually, I can see this causing both the problems qsa and I reported.
  
• ignores the right- and bottom-most edge of vertices
  This most likely isn't causing any problems, but it could in theory.
  

Offhand, I think the easiest solution is to throw this loop away, write both liquid_height and liquid_show to the .map file, and go from there. Unfortunately that is a bit of a rewrite for the core as well.

Thoughts?

That is indeed the source of this problem, causing our terrain generator to ignore polygons near bottom-right borders.

I think I can fix it without storing any extra data tho, just different logic while generating polygons. [del]I'll test it once I get back home.[/del]

EDIT: On the repository; testing needed.

@aming51: From what I remember, this issue is actually caused by GetClosePoint() in TargetedMovementGeneratorMedium returning destination point on the lower ground that the target is.

So, it is not directly related to mmaps.

Can anyone confirm this?

EDIT2: I'll confirm to myself. Same issues w/o mmaps.

It is problem in WorldObject::GetNearPoint().

Most cases should be fixed by https://github.com/faramir118/mangos/commit/e43031b928ca2036fe90215936064d9a0c57d869

Testing needed.

Since it is general mangos problem, feel free posting it in "review" section.

Cheers

[Undergarun]

I'm having crashes and crashes so i can't test it until everything is fixed ;-) I was redoing clean public repo: https://github.com/Undergarun/mangos/commits/master

[Undergarun]

Crash related to mmaps:

http://pastebin.com/vv1PsTVY

[qsa]

Crash related to mmaps:

http://pastebin.com/vv1PsTVY

I think I nkow what it is about.

Anyone know if creatures trigger grid loading? I think not.

If so, this crash happens when random movement generator makes npc to wonder to unloaded grid.

If thats the case, I'll fix it soon.

[Undergarun]

Other mmaps crash: http://pastebin.com/DmGryVfP

[qsa]

Other mmaps crash: http://pastebin.com/DmGryVfP

Can you please try :

In PathFinder.cpp : PathFinder::calculate() change "if (!m_navMesh || !m_navMeshQuery || !HaveTiles(dest) ||" to

"if (!m_navMesh || !m_navMeshQuery || !HaveTiles(dest) || !HaveTiles(start) ||"

Thanks in advance.

[Undergarun]

Can you please try :

In PathFinder.cpp : PathFinder::calculate() change "if (!m_navMesh || !m_navMeshQuery || !HaveTiles(dest) ||" to

"if (!m_navMesh || !m_navMeshQuery || !HaveTiles(dest) || !HaveTiles(start) ||"

Thanks in advance.

Works like a charm for now... ;-)

[aming51]

There are some problems in the extractor (ad.exe) that might be contributing to the issues qsa and I posted.

In the mmaps_rewrite version of contrib/extractor/System.cpp, the loop @ line 775 is wrong.

It iterates over quads (128x128), but acts on vertices

• sets the top-left vertex of a quad to invalid height, which affects all quads on that corner.
  Conceptually, I can see this causing both the problems qsa and I reported.
  
• ignores the right- and bottom-most edge of vertices
  This most likely isn't causing any problems, but it could in theory.
  

Offhand, I think the easiest solution is to throw this loop away, write both liquid_height and liquid_show to the .map file, and go from there. Unfortunately that is a bit of a rewrite for the core as well.

Thoughts?

That is indeed the source of this problem, causing our terrain generator to ignore polygons near bottom-right borders.

I think I can fix it without storing any extra data tho, just different logic while generating polygons. [del]I'll test it once I get back home.[/del]

EDIT: On the repository; testing needed.

@aming51: From what I remember, this issue is actually caused by GetClosePoint() in TargetedMovementGeneratorMedium returning destination point on the lower ground that the target is.

So, it is not directly related to mmaps.

Can anyone confirm this?

EDIT2: I'll confirm to myself. Same issues w/o mmaps.

It is problem in WorldObject::GetNearPoint().

Most cases should be fixed by https://github.com/faramir118/mangos/commit/e43031b928ca2036fe90215936064d9a0c57d869

Testing needed.

Since it is general mangos problem, feel free posting it in "review" section.

Cheers

I tested a few minutes ago, this issue( GetNearPoint bug ) is still on Undergarun's server:

http://i53.tinypic.com/vcrvkk.jpg

http://i54.tinypic.com/6p6flk.jpg

http://i53.tinypic.com/fatnwo.jpg

Best regards

[qsa]

...

I tested a few minutes ago, this issue( GetNearPoint bug ) is still on Undergarun's server:

http://i53.tinypic.com/vcrvkk.jpg

http://i54.tinypic.com/6p6flk.jpg

http://i53.tinypic.com/fatnwo.jpg

Best regards

Tested on local in exact places you showed and it worked just fine for me.

Don't have DETECT_POS_COLLISION enabled?

[aming51]

...

I tested a few minutes ago, this issue( GetNearPoint bug ) is still on Undergarun's server:

http://i53.tinypic.com/vcrvkk.jpg

http://i54.tinypic.com/6p6flk.jpg

http://i53.tinypic.com/fatnwo.jpg

Best regards

Tested on local in exact places you showed and it worked just fine for me.

Don't have DETECT_POS_COLLISION enabled?

Thanks for your reply, I am just a common player testing on Undergarun's server, maybe this bug is only on Undergarun's server. I hope Undergarun can fix this bug on his server.

Have a nice day!

[Undergarun]

Post Collision is disabled on my realms.

Single threaded tests of mmaps on low populated realm:

Note: Crash between tests so is not a just mmaps.enabled = 0 and .reload config

Tests on multithreaded high populated realm coming soon!

[Undergarun]

Something about my realms, i'm not using GridUnload, its always = 0 so don't know if mmaps can cause crashes in someway with unloaded grids, etc..

@aming51 test it now, PostCollision is enabled in all realms.

[aming51]

Something about my realms, i'm not using GridUnload, its always = 0 so don't know if mmaps can cause crashes in someway with unloaded grids, etc..

@aming51 test it now, PostCollision is enabled in all realms.

Thank you for the PostCollision enabled, now it works great!

Would you please change all realms that not only Spaish client permited to access your server but also all clients like US/EU/TW/RU ect.? Now I am using TW client, I can only test on Amberlyst realm(If choose other realms it always said: you are trying to play game in a different language server.) I can only stay at 1~60lvl places whch means I can't go to WLK places to get 80lvl for more test.

Although, I still found little bug with pathfinding: When I stand on stage and attack, of course the mob go up slope and towards me. But suddenly at about 15 yards position it turned back and moved 10 yards away from me , then towards me again. The whole action looks like unseen obstacle put on his way and he move in a circle to attack me. But the place is flat. Sometimes it moved back and back two times before access me.

1.attack:

http://i53.tinypic.com/2ah8ocy.jpg

2. first back:

http://i53.tinypic.com/1115ljk.jpg

3. move far away:

http://i52.tinypic.com/2z86ef4.jpg

4. toward me:

http://i51.tinypic.com/2v309q1.jpg

5. turn back second time:

http://i51.tinypic.com/20tgwme.jpg

I hope this test is useful.

Best regards!

[Undergarun]

Would you please change all realms that not only Spaish client permited to access your server but also all clients like US/EU/TW/RU ect.? Now I am using TW client, I can only test on Amberlyst realm(If choose other realms it always said: you are trying to play game in a different language server.) I can only stay at 1~60lvl places whch means I can't go to WLK places to get 80lvl for more test.

In order to continue with TBC and WOTLK content you should upgrade your account from Account Management. Second service on the right menu. Also my realms supports enUS enGB esES esMX deDE frFR ruRU but unfortunately not clients from asia (DBCs and installer from those clients are needed!).

Cheers.

[qsa]

Post Collision is disabled on my realms.

Single threaded tests of mmaps on low populated realm:

...

Note: Crash between tests so is not a just mmaps.enabled = 0 and .reload config

...

Tests on multithreaded high populated realm coming soon!

Looks great, Thanks you.

We can conclude that it is stable and "costs" less than 10% CPU and about 3giga memory with no unloads.

I think that its totally reasonable.

Looking forward for multithreaded results. Should scale really well.

...

Although, I still found little bug with pathfinding: When I stand on stage and attack, of course the mob go up slope and towards me. But suddenly at about 15 yards position it turned back and moved 10 yards away from me , then towards me again. The whole action looks like unseen obstacle put on his way and he move in a circle to attack me. But the place is flat. Sometimes it moved back and back two times before access me.

...

Best regards!

Thanks you for those reports.

I'll check it this weekend.

I don't sure if its case of problematic geometry or side-effect of using weighted A*.

This kind of behavior is common?

Thank you guys.

[lillecarl]

Just wanted to tell you all, im extracting all mmaps (TBC) right now, i had to manually solve a few conflicts, im not sure if i did it correct, its compiling and so but still it might not be correct, but im going to do some testing locally in a soon future

EDIT: is this regular behavior? http://filebeam.com/4ae3d0573c87add1eedbd0f68d25c1ba.jpg[/img]

EDIT2: (No polygons etc..)

- LilleCarl

[Wojta]

If it does not say it on every tile then yeah, this is correct. Some tiles are just empty, therefore theres nothing to build.

[lillecarl]

If it does not say it on every tile then yeah, this is correct. Some tiles are just empty, therefore theres nothing to build.

Thank you, i have generated all maps exept kalimdor (0) now, its on 42%, Good to know that i didnt put my laptop to 85 Celsius for like 5 hours just wasting power and collecting dust =) In a week or so i should be able to give you some reports about the mmaps for one (But they still had merging conflicts so if you (faramir) could merge it properly it would be easier for everyone to use

- LilleCarl

[lillecarl]

Sorry for doubleposting, but it feels better, sometimes when i run a stair (a rolled stair if you get what i mean) and the npc in this case is above me, he stops and thinks he is at my position, he is in 2d but not in 3d..... i will provide a movie laters (also there are some places he just airwalks etc (got it on movie aswell)

- LilleCarl

[faramir118]

Sorry for doubleposting, but it feels better, sometimes when i run a stair (a rolled stair if you get what i mean) and the npc in this case is above me, he stops and thinks he is at my position, he is in 2d but not in 3d..... i will provide a movie laters (also there are some places he just airwalks etc (got it on movie aswell)

- LilleCarl

I had forgotten about this... I have a fix that wasn't very good unless we had small tiles...

For some time now, we've had much smaller tiles as an intermediate step, so I will try this fix and see how well it works.

[lillecarl]

I didnt really understand that but ill give you video examples

[qsa]

Sorry for doubleposting, but it feels better, sometimes when i run a stair (a rolled stair if you get what i mean) and the npc in this case is above me, he stops and thinks he is at my position, he is in 2d but not in 3d..... i will provide a movie laters (also there are some places he just airwalks etc (got it on movie aswell)

- LilleCarl

Shouldn't happen unless spiral levels spaced more than 5y apart.

Unless its one of the GetNearPoint()'s features.

Please provide coordinates to reproduce.

[Undergarun]

Multithreaded MMaps Test on High populated realm:

Note: Crash between tests so is not a just mmaps.enabled = 0 and .reload config

For this kind of performance a Dedicated MySQL Server with enought memory to keep all data on RAM is needed! Also 4000 ppl online and 3 mapupdatethreads is the MaNGOS limit until everything is moved to per-map multithreading and if movement system is implemented also in multithreading way this limit can easily increased to up to 9000 - 10.000 ppl online.

@aming51 Now all my realms have latests version of mmaps and re-extracted mmaps. So please test all again

My players are reporting more falls under map that usually was. Reports about those issues coming soon but i suggest to use pathfinder on ConfusedMovementGenerator for generate random movements over valid terrain. ^^

[qsa]

...

My players are reporting more falls under map that usually was. Reports about those issues coming soon but i suggest to use pathfinder on ConfusedMovementGenerator for generate random movements over valid terrain. ^^

Thanks for the results.

Looks solid to me. No memory consumption increase (all mmaps are already loaded anyway), around ~13% CPU spread among cores. Lovely.

About falling under maps: possible when movement generator is broken before finalization. Actually, there's pretty simple solution that will give same results as old generators: using vmap heights for points + adding some error margin. Have to test it, to see if that isn't too expensive.

EDIT:Tried to use VMAP heights instead of MMAP's and it was total disaster, turns out they are even less accurate

Using ConfusedMovementGenerator instead of RandomMovementGenerator? Why? Random cannot ever be applied to players, therefore is is much cheaper. Other than that, they about the same.

EDIT: Added something to repo, should help quiet a bit.

[toyah79]

Anybody noticing creature's evading way too early? I pull some creature or mob, I go hug a wall or something or jump over some obstacle so the creature/mob cant reach me and then the mob/creature starts evading.

Now I dont know if this is intended behavior (is it?) but too me it looks like something is not right.

Dug into the pathfinding code and I found this.

In pathfinding "targetedmovementgenerator.h" there is a function called "isreachable" and it does this check.

return (i_path) ? (i_path->getPathType() & (PATHFIND_NORMAL)) : true;

shouldnt this be more elaborated with the other pathfind type like this?

return (i_path) ? (i_path->getPathType() & (PATHFIND_NORMAL|PATHFIND_INCOMPLETE)) : true;

I have tested this and the creature/mob is now following me more consistently - ie does not brake off chase so easily.

Just a thought....