MMaps Redux

[qsa]

Noticed couple problems with VMAPs which affect mmaps.

Nothing new, they just became visible due to mmap related movement.

Case 1:

http://i55.tinypic.com/29atcus.jpg

Just check the diff between the actual ground level and level claimed by VMAP.

Case 2:

".go 3624.958 2034.717 75.228 571" and soot the bear next to you. Now observe the Z coordinate GetClosePoint() gives you as your target position. Needless to say it is far from actual ground level -> no path can be generated.

While not directly related to mmap code, we may want to do something about it. Or at least properly test and fill bug report Those things sure cause some nasty problems to mmap generated movement.

Another issue is the following: http://i53.tinypic.com/1087f43.jpg

Only partial water data and partial terrain is there, something to do with how we generate the terrain.

Not very problematic in most cases, but, those things below water line shouldn't be there.

We really need to think how underwater terrain can be handled.

forbiddenn : you have problem with merging. It compiles and links just fine on vs9. PS: some actual info can be nice. But hey! I must be a prophet, guessing those.

scargo: That 2nd report directly related to first case. Thanks.

PS: pushed some things that may need some real testing.

[iforgotmypassword]

navigation in Booty Bay:

map 000, navmesh 0005931

Issues

#1:

http://filebeam.com/d3ec2671304861b66280badcd3349e10.jpg

X: -14428.005859 Y: 450.399872 Z: 15.412142 Map: 0

#2:

http://filebeam.com/a00738e6471c73736d37d813b6cb72b6.jpg

http://filebeam.com/6ca72feba71acf17a6287b4781089fe2.jpg

X: -14438.644531 Y: 453.203949 Z: 3.743556 Map: 0

Badlands, strange corner

http://filebeam.com/c324dcf793f8816ec1e835235d39fb42.jpg

X: -6566.770508 Y: -3291.101074 Z: 242.517136 Map: 0

strange curves in path

http://filebeam.com/9e2036876a9e01618ae86a697f4f21b2.jpg - straight line

http://filebeam.com/8454939e69fd9df38f70c80b6a1c8005.jpg - curve

[faramir118]

navigation in Booty Bay:

Looks like there are consequences to such small voxel sizes... small, insignificant gaps become problems.

I'm really surprised to see this...

Badlands, strange corner

http://filebeam.com/c324dcf793f8816ec1e … 39fb42.jpg

X: -6566.770508 Y: -3291.101074 Z: 242.517136 Map: 0

strange curves in path

http://filebeam.com/9e2036876a9e01618ae … 4f21b2.jpg - straight line

http://filebeam.com/8454939e69fd9df38f7 … 1c8005.jpg - curve

These are due to A* heuristic, and the fix is rather CPU expensive.

Due to player range limits, it's hard to make a bad detour like this in-game.

[Neo2003]

Hello all,

Here a small multi-threaded python program that will spawn as many MoveMapGen processes as the number of cores you have and will maintain this load until all is extracted.

Untested on Linux, it probably requires some parameters to subprocess.call in order to open an xterm per MoveMap Generator or all outputs are mixed.

The list of maps I hardcoded is probably not complete. Be sure to keep 0,1,530,571 in the beginning in order to max load your PC. The collection is unstacked by the program so from left to right.

#!/usr/bin/python

import os, sys, threading, time, subprocess
from multiprocessing import cpu_count
from collections import deque

mapList = deque([0,1,530,571,13,25,30,33,34,35,36,37,42,43,44,47,48,70,90,109,129,169,189,209,229,230,249,269,289,309,329,349,369,
   389,409,429,449,450,451,469,489,509,529,531,532,533,534,540,542,543,544,545,546,547,548,550,552,553,554,555,556,557,558,559,
   560,562,564,565,566,568,572,573,574,575,576,578,580,582,584,585,586,587,588,589,590,591,592,593,594,595,596,597,598,599,600,
   601,602,603,604,605,606,607,608,609,610,612,613,614,615,616,617,618,619,620,621,622,623,624,628,631,632,641,642,647,649,650,
   658,668,672,673,712,713,718,723,724])

class workerThread(threading.Thread):
   def __init__(self, mapID):
       threading.Thread.__init__(self)
       self.mapID = mapID

   def run(self):
       name = "Worker for map %u" % (self.mapID)
       print "++ %s" % (name)
       if sys.platform == 'win32':
           stInfo = subprocess.STARTUPINFO()
           stInfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
           stInfo.wShowWindow = 7
           cFlags = subprocess.CREATE_NEW_CONSOLE
       else:
           stInfo = None
           cFlags = 0
       retcode = subprocess.call(["MoveMapGen", "%u" % (self.mapID),"--silent"], startupinfo=stInfo, creationflags=cFlags)
       print "-- %s" % (name)

if __name__ == "__main__":
   cpu = cpu_count() - 0 # You can reduce the load by putting 1 instead of 0 if you need to free 1 core/cpu
   if cpu < 1:
       cpu = 1
   print "I will always maintain %u MoveMapGen tasks running in //\\n" % (cpu)
   while (len(mapList) > 0):
       if (threading.active_count() <= cpu):
           workerThread(mapList.popleft()).start()
       time.sleep(0.1)

Have fun.

PS: on Windows, properly setup your compilation flags and you get a -33% time to extract compared to what is on GIT currently (btw this is true for core too, release mode can have 20% speed increase by putting other compilation flags, but that's another story).

[forbiddenn]

Re, I updated a core 10835 and I the following error

12>Édition des liens en cours...
12>Detour.lib(DetourAlloc.obj) : .netmodule ou module MSIL compilé avec /GL trouvé ; redémarrage de l'édition de liens avec /LTCG ; ajoutez /LTCG à la ligne de commande de l'édition de liens pour améliorer les performances de l'Éditeur de liens
12>   Création de la bibliothèque .\\mangosd__Win32_Release\\mangosd.lib et de l'objet .\\mangosd__Win32_Release\\mangosd.exp
12>game.lib(Chat.obj) : error LNK2001: symbole externe non résolu "protected: bool __thiscall ChatHandler::HandleUnFreezeCommand(char *)" (?HandleUnFreezeCommand@ChatHandler@@IAE_NPAD@Z)
12>game.lib(SpellEffects.obj) : error LNK2001: symbole externe non résolu "public: void __thiscall Unit::MonsterJump(float,float,float,float,unsigned int,unsigned int)" (?MonsterJump@Unit@@QAEXMMMMII@Z)
12>..\\..\\bin\\Win32_Release\\mangosd.exe : fatal error LNK1120: 2 externes non résolus
12>Le journal de génération a été enregistré à l'emplacement "file://c:\\Users\\Nemesis\\Desktop\\Emulateur Mangos Serveur en Développement\\Emulateur V 0.0.3\\Master\\win\\VC90\\mangosd__Win32_Release\\BuildLog.htm"
12>mangosd - 3 erreur(s), 0 avertissement(s)

I do not know how to correct ...

[faramir118]

12>game.lib(Chat.obj) : error LNK2001: symbole externe non résolu "protected: bool __thiscall ChatHandler::HandleUnFreezeCommand(char *)" (?HandleUnFreezeCommand@ChatHandler@@IAE_NPAD@Z)

12>game.lib(SpellEffects.obj) : error LNK2001: symbole externe non résolu "public: void __thiscall Unit::MonsterJump(float,float,float,float,unsigned int,unsigned int)" (?MonsterJump@Unit@@QAEXMMMMII@Z)

Again, not a problem with mmaps - you probably have merge errors.

[iforgotmypassword]

Server uptime: 7 Hour(s) 11 Minute(s) 57 Second(s).

Server uptime: 9 Hour(s) 54 Minute(s) 59 Second(s).

Online: 200

GridUnload: off

and crazy lags after .mmap stats command

mmaps directory size is 1.4 GB

[qsa]

navigation in Booty Bay:

Looks like there are consequences to such small voxel sizes... small, insignificant gaps become problems.

I'm really surprised to see this...

I think we can allow some particular problematic tiles to use different world metrics.

I'll push it, if I'll have luck making them coexist properly, link together.

...

Have fun.

PS: on Windows, properly setup your compilation flags and you get a -33% time to extract compared to what is on GIT currently (btw this is true for core too, release mode can have 20% speed increase by putting other compilation flags, but that's another story).

Thanks, I'll push it once I have time to test it on *nix machine, or maybe someone can do it.

This sure can be useful to some people.

About compiler flags, you had anything particular in mind? Since all I can think about are cpu dependent.

...

Online: 200

GridUnload: off

and crazy lags after .mmap stats command

mmaps directory size is 1.4 GB

9 hours ain't too bad with mmaps and mtmaps (which you obviously use). Cores aren't too busy either.

About memory usage : since you don't unload grids, it actually may be expected sizes. Don't really know, have nothing to compare it with. Its not like we have formula for "proper" memory usage for mangos. online_count*X*grid_loaded*Y*...

Still, 7g +9g virtual looks just too much.

Again, if you imply memory leaks, please do provide us a trace. You can find example few pages back in this topic.

"crazy lag" is nothing unexpected from "mmap stats" command, the more grids you have loaded, the slower it goes. keep in mind, that it is debug command, thus should not be used on anything but test machine.

Thanks for testing and feedback. The better quality of reports we get, the faster problems will be solved.

Cheers people

[iforgotmypassword]

qsa:

before commit 4c154a764cba03560d0d176528d4ea3c06f6328a memory usage was 6.5Gb after 2d uptime.

But,

maybe this crazy memory usage is my bad

seems like I forgot to backport some code

I will test it today.

And thank you

[qsa]

qsa:

before commit 4c154a764cba03560d0d176528d4ea3c06f6328a memory usage was 6.5Gb after 2d uptime.

That commit is from August 23, 2010, since then we had like... I don't want to count, how many changes. You better update.

[iforgotmypassword]

currently I'm using commit 09f8acfebf39944f4da25f3a292b9fb6488cecd7

[qsa]

navigation in Booty Bay:

Looks like there are consequences to such small voxel sizes... small, insignificant gaps become problems.

I'm really surprised to see this...

I think we can allow some particular problematic tiles to use different world metrics.

I'll push it, if I'll have luck making them coexist properly, link together.

Pushed something to git.

It should "solve" this little issue in bootybay and similar.

The good news, is that you can extract only affected tile using bigger metrics, and due to chosen sizes, we maintain same ratio, thus neighboring tiles ( which use normal metric) will link with this one.

The bad news are, it is not really generic solution. Having some tiles extracted one way, others another.

Guess we'll have to make list of those or something and script to extract them afterwards.

[Neo2003]

About compiler flags, you had anything particular in mind? Since all I can think about are cpu dependent.

Yes, of course

Activate fast-math on g3dlite, Detour, Recast and MoveMapGen (FloatingPointModel="2" in vcproj files or in project properties C/C++ | Code Generation | Floating Point Model).

As far as I remember, this option was first put to strict mode as default in VS2005. It has a big performance impact and code size impact due to x2 more instruction generated when put to strict.

On my test PC MoveMapGen.exe (Win32/release) is by default 636Kb in size and takes 328 seconds to extract map 33. After it's 253Kb in size and process the same map 33 in 243 seconds.

Note: I will soon patch all mangos project files so that release libraries and binaries are at least compiled with /O2 (ex: g3dlite is /O0 currently on VC80 and VC90, i386 and x64).

Neo2003

[iforgotmypassword]

how to detect memory leaks under *nix\\win64?

my developer machine has 5GB of memory and win7 64, and centos 5.5 on virtual machine

[Ambal]

Activate fast-math on g3dlite, Detour, Recast and MoveMapGen (FloatingPointModel="2" in vcproj files or in project properties C/C++ | Code Generation | Floating Point Model).

Be very careful with such floating-point optimizations to not end up with something like in http://stackoverflow.com/questions/3042925/strange-results-with-fpfast. All FP comparisons might be affected by this change, so it is better to put a patch on forum so ppl can test it before pushing into repo.

Cheers.

[qsa]

how to detect memory leaks under *nix\\win64?

my developer machine has 5GB of memory and win7 64, and centos 5.5 on virtual machine

Post #592 in this topic can give you basic idea.

Activate fast-math on g3dlite, Detour, Recast and MoveMapGen (FloatingPointModel="2" in vcproj files or in project properties C/C++ | Code Generation | Floating Point Model).

Ambal beat me to it.

It may not be very safe. Some proper testing needed. Simply since, only god knows what assumptions Recast/g3dlite code takes.

Making 0.0f == 0.0f error, is well too easy to make. Even tho, none in right mind will write this code regardless of optimizations used.

If anyone can properly test it for mmaps and report, we'll be grateful.

[shauren]

dont use fast math on vmap code, BIH doesnt go well with it

[Neo2003]

dont use fast math on vmap code, BIH doesnt go well with it

BIH is in vmap code so in game project, not in g3dlite project.

If you download G3D 8.00 sources and open the VC project files included, you will see that G3D library is set to compile with the following flags in release mode by default with VS compilers:

Optimization: Full (/Ox)

InlineFunctionExpansion: Any suitable (/Ob2)

EnableIntrinsicFunctions: Yes (/Oi)

FavorSizeOrSpeed: Favor Speed (/Ot)

WholeProgramOptimization: No

EnableFunctionLevelLinking: Yes (/Gy)

EnableEnhancedInstructionSet: SSE2 (/arch:SSE2)

FloatingPointModel: Fast (/fp:fast)

But you are right, I did not dig Detour & Recast behavior and which flags the author uses. So it's unsafe without big testings on those ones.

[iforgotmypassword]

a new issue with navmesh in booty bay (almost same place)

X: -14435.139648 Y: 420.237518 Z: 8.844869 Map: 0

--bigBaseUnit true

http://filebeam.com/e2cb821599cfe493dd34cd67abb881de.jpg

--bigBaseUnit false

http://filebeam.com/89c92f181cf826d54df8816887b3caee.jpg

[iforgotmypassword]

1545 MB memory used on just-started server with new mmaps

after loading map Deadmines 1770 MB memory used

628 MB memory used on just-started server without new mmaps

after loading map Deadmines 661 MB memory used

deadmines with new mmaps = 225 MBytes

deadmines w\\o mmaps = 33 MBytes

sorry for my english, I'm almost sleeping

[scargo]

1545 MB memory used on just-started server with new mmaps

after loading map Deadmines 1770 MB memory used

628 MB memory used on just-started server without new mmaps

after loading map Deadmines 661 MB memory used

deadmines with new mmaps = 225 MBytes

deadmines w\\o mmaps = 33 MBytes

sorry for my english, I'm almost sleeping

Did you mean, ram+swap ?

For me less than 80 megs for 1 player (committed on swap 1,5 gigs)

On instance (AQ20) 12 player 650 megs of ram

Rel 10878 sd2 and and some other patches win7 x64

Cheers

[iforgotmypassword]

I told about first map loading, it is not related to players count.

Map initializing takes ~230-250 MBytes WITH mmaps and ~30-50 MBytes WITHOUT

Just start the server without any players, remember memory usage, and enter the world

[scargo]

Understood, mine, on players number, was only a puntualizzation (useless)

But i need another explanation, for 'maps inizializzation memory usage' you mean the difference between initial ammount of used ram, after server loadsup and the ammount after someone enter the worlds ?

Thanks for your time

Cheers

[faramir118]

Some memory is pre-allocated when the navmesh is loaded (before any tiles are loaded). I didn't think it would be that high though, I will look into it.

edit:

Probably because we hard-coded 2^21 maxTiles

[qsa]

a new issue with navmesh in booty bay (almost same place)

X: -14435.139648 Y: 420.237518 Z: 8.844869 Map: 0

--bigBaseUnit true

http://filebeam.com/e2cb821599cfe493dd34cd67abb881de.jpg

--bigBaseUnit false

http://filebeam.com/89c92f181cf826d54df8816887b3caee.jpg

We are trying to apply discrete logic and methods to continuous world. Therefore, corner cases will always be possible.

The best we can hope for, is to minimize their occurrence. There's no "silver bullet" for this problem.

I don't think we can dynamically solve this particular issue.

I'd love to see someone proving me wrong.

Alternatively, you can test our parameters to find the best values which will give you best results for that particular tile.

1545 MB memory used on just-started server with new mmaps

after loading map Deadmines 1770 MB memory used

628 MB memory used on just-started server without new mmaps

after loading map Deadmines 661 MB memory used

deadmines with new mmaps = 225 MBytes

deadmines w\\o mmaps = 33 MBytes

sorry for my english, I'm almost sleeping

You see, I'll say it again ( just like I told you in PM) : memory log, or it didn't happen.

This report provides us with no valid information.

I can tell you as a fact, that on my test system, with one player loading 5~ mmap grids take at _very most_ 20mega.

Pretty easy to test. Start server, log into character on map with no mmap files, teleport to map with mmap - see the change.

I bet someone can provide 2nd opinion on this one.

Besides, knowing this report is for mmap port to Trinity, doen't make it too reliable, sorry.

As I said, report problems in code in our repository. We cannot maintain changes/bugs made by others, sorry.

By no means, I'm not saying there are no memory issues in mmap code, its just people cannot show there are.

Either show me valgrind log, OR test I mentioned on CLEAN mangos+empty database.

Best regards