qsa

Other mmaps crash: http://pastebin.com/DmGryVfP

Can you please try :

In PathFinder.cpp : PathFinder::calculate() change "if (!m_navMesh || !m_navMeshQuery || !HaveTiles(dest) ||" to

"if (!m_navMesh || !m_navMeshQuery || !HaveTiles(dest) || !HaveTiles(start) ||"

Thanks in advance.

Crash related to mmaps:

http://pastebin.com/vv1PsTVY

I think I nkow what it is about.

Anyone know if creatures trigger grid loading? I think not.

If so, this crash happens when random movement generator makes npc to wonder to unloaded grid.

If thats the case, I'll fix it soon.

There are some problems in the extractor (ad.exe) that might be contributing to the issues qsa and I posted.

In the mmaps_rewrite version of contrib/extractor/System.cpp, the loop @ line 775 is wrong.

It iterates over quads (128x128), but acts on vertices

• sets the top-left vertex of a quad to invalid height, which affects all quads on that corner.
  Conceptually, I can see this causing both the problems qsa and I reported.
  
• ignores the right- and bottom-most edge of vertices
  This most likely isn't causing any problems, but it could in theory.
  

Offhand, I think the easiest solution is to throw this loop away, write both liquid_height and liquid_show to the .map file, and go from there. Unfortunately that is a bit of a rewrite for the core as well.

Thoughts?

That is indeed the source of this problem, causing our terrain generator to ignore polygons near bottom-right borders.

I think I can fix it without storing any extra data tho, just different logic while generating polygons. [del]I'll test it once I get back home.[/del]

EDIT: On the repository; testing needed.

@aming51: From what I remember, this issue is actually caused by GetClosePoint() in TargetedMovementGeneratorMedium returning destination point on the lower ground that the target is.

So, it is not directly related to mmaps.

Can anyone confirm this?

EDIT2: I'll confirm to myself. Same issues w/o mmaps.

It is problem in WorldObject::GetNearPoint().

Most cases should be fixed by https://github.com/faramir118/mangos/commit/e43031b928ca2036fe90215936064d9a0c57d869

Testing needed.

Since it is general mangos problem, feel free posting it in "review" section.

Cheers

...

I haven't forgotten our deal, i'm just waiting the work of rsa on multithreading locking for make fully multithreaded performance tests. I still use mmaps and notice no new crashdumps related to, so consider mmaps as stable (not fully free bugs but stable at least).

Cheers.

The reason I'm so impatient is since I think its about the time this thread move to "under review" section.

Without proper test results, I cannot do it with clear conscience.

The reason I want it done, is not that I think its bug-free or perfect. The reason is to speed up development.

Proper threading model can be nice, since mmaps do scale well - but it is not "a must".

Unrelated:

There's tinny issue with terrain I noticed couple of days ago:

In some places the liquid ends just before the shore line- as seen in the picture:

http://i51.tinypic.com/dpiu4x.jpg

This causes really bad mesh generation in that area, since that shoreline is unpassable.

Was looking around both in map extractor and in our code - wasn't able to find logical error. It might be problem with the actual provided terrain. But then we'll have to complicate for it somehow.

Anyone noticed this king of issues before? faramir118?

Best regards.

Thank you. Kinda surprised nobody noticed this Oo

Most models do not have pitch/roll values at all, or they are set to low values.

PS: some testing needed.

If so, then not only in one axis - the bridge near mage tower in-game and from RecastDemo: http://valhalla-team.com/WoWScrnShot_072911_200016.jpg http://valhalla-team.com/snap7.png

Should be fixed in latest commit.

All maps must be re-extracted fresh (delete mmap dir's content at the extractor, otherwise it will just skip those).

Thanks for report. This should fix fair pile of issues.

@Undergarun: how's that report is going? We're getting old

I tried to read most of this thread and I didnt see this one noticed: http://valhalla-team.com/snap4.png http://valhalla-team.com/WoWScrnShot_072911_162911.jpg - the whole wmo is misplaced in mmap mesh, but vmaps are all right. Same thing happened for bridge on the other side of EoS.

But I have slightly modified and most likely little outdated version of mmap patch, can someone test this please?

EDIT: It seems like it is just rotated by 180 or the rotation wrong, because bridge is too low on one side and too high on the other: http://valhalla-team.com/snap5.png http://valhalla-team.com/snap6.png

EDIT2: Now when I look more closely, I think the left and right bridge is just swapped, lol Oo

EDIT3: No, they are not, more like rotated by 180 degrees.

EDIT4: By the way, the bridges are on two tiles at once: for example the one between mage tower and fel reaver ruins is on both tile 566 28 28 and 566 27 28

The mmaps generated are accurate, according to input mesh - look at it, the bridges are little bit misplaced.

There's error in our TerrainBuilder::loadVMap() logic, or maybe model data (or entire map angle)? It looks like pitch angle is off by ~3-5deg.

With low chance: http://pastebin.com/6JBf0JxK

Well, if that keeps happening, we'll have to add some asserts. Otherwise, without catching it under debugger there's no chance I can solve it.

You have no idea how to reproduce, do you? Had to ask ...

If you have more of those, please post them all, even if the trace looks the same.

On, not so related note:

Below is little modification turning our A* into weighted A*.

Two simple modification:

1) Weight up the heuristic function: in our case most of the time distance plays the role of perfect heuristics, thus expending 'only' nodes directly on the path to target - we can't go wrong here.

2) Do not return nodes from closed set back to open.

Both of the changes will do the following two things:

1) Expend less nodes. In other words, make path generator take less time and memory - should be about 30% less - no idea to now much overall it translates.

2) Provide non optimal results. While A* ensures optimal result (optimal by our metric - distance), this variant, while still ensuring completeness, will provide suboptimal solutions.

I'd love to see the performance difference under load. (thanks in advance)

diff --git a/dep/recastnavigation/Detour/Source/DetourNavMeshQuery.cpp b/dep/recastnavigation/Detour/Source/DetourNavMeshQuery.cpp
index 44aafe2..be66667 100644
--- a/dep/recastnavigation/Detour/Source/DetourNavMeshQuery.cpp
+++ b/dep/recastnavigation/Detour/Source/DetourNavMeshQuery.cpp
@@ -68,7 +68,7 @@ inline float dtQueryFilter::getCost(const float* pa, const float* pb,
}
#endif    

-static const float H_SCALE = 0.999f; // Search heuristic scale.
+static const float H_SCALE = 2.0f; // Search heuristic scale.


dtNavMeshQuery* dtAllocNavMeshQuery()
@@ -695,7 +695,7 @@ dtStatus dtNavMeshQuery::findPath(dtPolyRef startRef, dtPolyRef endRef,
            if ((neighbourNode->flags & DT_NODE_OPEN) && total >= neighbourNode->total)
                continue;
            // The node is already visited and process, and the new result is worse, skip.
-            if ((neighbourNode->flags & DT_NODE_CLOSED) && total >= neighbourNode->total)
+            if ((neighbourNode->flags & DT_NODE_CLOSED)/* && total >= neighbourNode->total*/)
                continue;

            // Add or update the node.

Best regards

Merged with master.

Added mmaps to RandomMovementGenerator.

All movement code remade fresh, pile of changes all over the place.

Keep your eyes open for new issues.

@Undergarun: Waiting for your side of the deal

I think now movement part of mmap code can be simplified alot or maybe removed at all

I didn't manage to remove mmap integration from movement generators completely, but it sure became simpler in some places, trivial in others.

I also had to make couple tinny modification to your spline code.

Feel free to comment on general design.

Cheers.

I can provide you screenshots of htop w/out mmaps but with mtmaps at 4 threads load is continuously jumping from one thread to another (CPU with 24 threads) so is quite difficult to get accurate information. But if is enought for you, deal.

The common case will do for us, thanks.

Due to recent changes (spline movement) it will take me bit longer to fulfill my side of the agreement.

All movement logic will have to be made from scratch. The good news are that the result, as I see it, will be more centralized, clear code.

@tyrael: Still no reports how mmaps work with more than 5-100 users. These tests are mandatory else you end up with something like this [1] (yeah my favorite github commit).

Regards

Skirnir

I'm using mmaps with more than 3000+ ppl online since some months ago without problems. Also i have mtmaps (4 Threads) and 48 GB DDR3 ECC with MySQL in other server.

Waiting for merge with mangos due to changes made by SilverIce.

That's pretty good news.

How about we make a deal?

I will merge with leatest mangos and all its changes, and you in return, will make a detailed report about mmap memory/cpu consumption as function of number of players. With mmaps and without.

How does that sound?

I don't know if this has been reported already, but...

If a flying monster is in the air - it evades when trying to chase some target on the ground. So there should be some check if creature can fly, has splineflag flying etc. then it shouldn't evade.

Can't confirm it using the revision in mmap master.

Tested with flying creatures with inhabit air and ground, or only one of them set.

After switching mmaps use off -> no crashes anymore. This part in the beginning of another crashdump led me here: http://pastebin.com/HjCUjEse

Maybe some kind of path building for vehicles when player is entering it? Or maybe it is not related to mmaps...

I can't see any mmap related code at any thread stacks, yet you got exactly the same vehicle calls all over.

If I were you, I'd look for too large vector allocation somewhere around Player::VehicleSpellInitialize -> WorldPacket opcode=377 -> ByteBuffer -> reserve

If you say, allocate 2giga there... with mmaps you may not have that kind of memory - fail, while you may have 2 giga free w/o.

In any case, chances this issue related to mmaps are rather slim. Besides, nothing changed in mmap code in a while now.

Cheers.

a lot of such crashes lately: http://pastebin.com/BrxJHtV8

last time merged with mmaps_rewrite on 06-27-2011

I'm using mangosR2 fork.

Nothing to do with mmaps from what I see.

std::bad_alloc -> new failed (possibly out of memory?)

Looking at the stack I'd guess something to do with vehicles (Player::VehicleSpellInitialize) ?

...

So i can test active. More stress test's not only ingame process. But isome agressive test's in very hard for pathfind locations.

...

I'm looking forward for results Thanks.

Memory/cpu diff given online count (/w /wo multithreading) can sure be valuable info.

...

3:

Need offmeshes base. Exist more places\\point's where need offmeshes. But at now we have only one.

on zero too small online to check all places for add all offmesh coordinats.

The one point provided is just syntax example

I do agree that we need some sort of database, even basic one.

Don't be shy people, post those. Don't forget to mention for which version those are generated (master, one, zero) since map vary.

small question out of the blue:

what is the performance impact vs clean mangos core? (in %, Mb/Gb) with say.. 1k players?

I'm asking the same question for quiet some time now

Take the initiative and gather this data. It will sure help the development.

bump? any news for this great feature?

bump? nay news for this great feature testing?

Sorry guys for not being too active recently, just bit busy lately.

Generally speaking, there aren't too much things that can be done without code review or bug reports, new ideas etc.

PS: I know there are some pretty esoteric problems wit some tiles on mangos zero, but I have no means debugging those at the moment.

So.. to sum it up:

1) We need some developer to review the code, general idea etc.

2) More tests and reports. Both positive and negative.

3) New idea can come in handy plus people who are willing to contribute to development process.

Without mentioned below, its just a matter of time till it dies out.

Best regards.

Recast demo??? What is this??

Look inside dep\\recastnavigation\\RecastDemo folder.

Basically, its a tool that allows you to see the actual input geometry and generated mesh.

It is extracted just fine for me (on 3.3.5). Look for file named 4293232.mmtile - if it isn't there, re-extract the map.

mmtile: http://filebeam.com/886c88901ab8b910f8983678bf84b86f

reextracted several times (with offmesh) & game serv & at my serv.

& not work inside instance.

Well, the mesh looks ok to me, beside some single poly that out of order - but that shouldn't affect much.

But I cannot match it to the geometry of old maps. That's since 335s 32,32 tile on that map is totally different. Furthermore, I cannot test it ingame.

You can always try and extract with debug flag and upload geometry too. Or better, see for yourself using the recast demo.

Sometimes the path generated by confused movement generator is very large, for example in warsong gluch, you use fear in the base and the system sometimes generates a path to the roof. You are going to walk until you reach the destination point, ignoring if the enemy fear stills on you. Also I think that if you use trinket when feared (or confused etc), you don't stop unitl you reach the next destination point.

.

Should be solved in latest (mmaps_rewrite branch).

KuraiNeko:

I don't think Intimidating Shout has anything to do with my changes. See spell 5246, its first triggered effect is stun, which is used for the glyph. It is what prevents you from running You can clearly see it by using ".aura 5246" on self.

missing tile's

Map: 429 (Dire Maul)

Zone: 2557 (Dire Maul)

Area: 2557 (Dire Maul)

mmap tileloc: 4293232.mmtile

gridloc [32,32]

Calc [02,01]

Dt [??,??] (invalid poly, probably no tile loaded)

It is extracted just fine for me (on 3.3.5). Look for file named 4293232.mmtile - if it isn't there, re-extract the map.

About those issues with fear and confusion : well, obviously they need some more work

I'll push few more changes that should solve atleast some of the problematic cases soon enough.

Thanks for the testing and input.

bugs:

Player get incorrect z coord.

x\\y - correct but fale on z coord. at result some times player get under textures or in-to wmo.

Tested in zul'gurub at ladder on enter & at searing gore by spells 29848 & 21060.

In ZG sometimes sheep gone down with ignore of LOS. In to wmo.

feelings movement generator.

Very bugged.

1 - cast self 12096

After fade fear player run as in fear. before player not command some wasd positions. If player not controlled - gamer afk than player run forewer

2 - duel. 2 warriors

Used warriors fear. Feared players stand at positions & not run anywhere. Only stand untill fear fade.

This sure sounds like the behavior of fear on maps/tiles with no mmaps loaded. Make sure those do work there.

Added little safety check for z value, just in case - but it really shouldn't change too much for proper mesh data.

I'll check ZG tomorrow, but on 335 client, don't have earlier versions. As one of the above posts said : you can easily see the mesh generated.

PS: please include exact coordinates next time ... searing gore isn't small place.

Outside karazhan, (arround X: -11155.110352, Y: 1978.645874, Z: 54.798313) the system has some troubles, the NPCs can't find a proper path. Is all over the area, not only at that coords. But it seems related with bad vmaps data.

Those coordinates on map 0 sent me to Westfall , I guess there's minus missing at Y coord.

Anyhow, there's indeed strange issue there.

For some strange reason, when you stand there and at couple more locations, the closest polygon selected in the one from the basement just beneath and not the ground.

No idea why it happens only there. This will need a closer look little bit later.

I can only report for mangos-zero which I mostly use mmaps on, and yet I only discovered a few spots where things go wrong, but these are also failing without mmaps, and caused by vmap data errors, e.g. in Deadmines before the Ogre boss, there is a spot not recognized as part of any valid map, but this happens without mmaps, too.

Haven't had the time to look at that location (nor I can on zero client only 3.3x), but, generally speaking ...

You (as general "you", not particular) can debug alot of the problems all by yourself. For instance, you can generate mmaps for that map using the --outputDebug flag, then open it in recast viewer and the the actual mesh generated.

As for mangos one/zero - it can be nice if someone maintained those branches, someone who actually use those ... wink, wink ...

About pathfinding, does exist a way to detect gameobject size and go around them

It's just a simple question

Short, simple answer would be "yes".

I mean, sure, way(s) always do exist.

A little bit longer answer:

Does current code support it - no.

Why? Since none implemented it.

Why? Since VMAPs do not support those and we rely on them to generate our input meshes.

Why don't "we" implement it? go ahead, I think we already had someone working on those.

As for mmaps, I think we already spoke about ways to make it happen on mmaps without modifying meshes on runtime.

Had some time on my hands yesterday, so I finally decided to try and add pathinding to ConfusedMovementGenerator and to FleeingMovementGenerator.

Pushed the changes into git yesterday.

FleeingMovementGenerator had to be totally remade, including all its logic. The good, is that it is much simpler now. The bad, is that it will work really poorly on maps with no MMAP enabled (fearing off cliffs, etc ..)

Changes to ConfusedMovementGenerator were less extensive, the logic left the same, just the paths generated by pathfinding system and some minor logic changes, like not generating all points in advance, etc...

As always, lots of testing is needed.

On the subject : unfortunately, I see no correlation between the number of people using our little system, and the number of reports/feedback we are getting. Thats a shame. There are at least 6 (I know of) hugely popular sub/projects/branches using mmaps, yet we have even less feedback from the times we started.

I know I sound like a broken record, same "bla, bla bla", but it would be shame, if this initiative will die out after all work invested in it.

If you find a bug, please make sure it is not a known issue (stuff like dynamic GOs, etc), then report it here. Include exact coordinates and way to reproduce it + system info.

Thanks in advance.

...

This change will not affect at Charge because charge uses PointMovementGenerator instead of Targeted that is changed by this commit.

...

Where is this coming from?

Charge do not use point movement generator, or any movement generator at all. Charge calls MonsterMoveByPath() directly, which in tern forces the destination.

Given examples (movies), show only one thing : retail ensure that if succeeded casting charge, it will get you to the destination. I can grantee, this is not done by having 2nd set of maps, but by appending destination to any generated path.

I did read your reply, unfortunately, this was not mutual.

Just to rephrase it in simpler manner : charging walls has nothing to do with walkable angles.