MMaps Redux

[KiriX]

Crash again =(

Registers:
EAX:0B4580F8
EBX:47AE90C0
ECX:00000000
EDX:00000002
ESI:0B458180
EDI:47AE90DC
CS:EIP:B45001B:7C80BEF7
SS:ESP:60023:0B4580F4  EBP:0B458148
DS:E070023  ES:0023  FS:003B  GS:1700000
Flags:00000206

Call stack:
Address   Frame     Function      SourceFile
7C80BEF7  00000000  RaiseException+3C
7857DF56  00000000  _CxxThrowException+48
004753F0  00000000  operator new[]+40
0069364D  00000000  dtCustomAlloc+D
00401965  00000000  dtNavMesh::init+95
0096F998  00000000  MMAP::MMapManager::loadMapData+278

Call stack:
Address   Frame     Function      SourceFile
7C93860C  00000000  KiFastSystemCallRet+0
7C821C8D  00000000  WaitForSingleObject+12

Call stack:
Address   Frame     Function      SourceFile
7C93860C  00000000  KiFastSystemCallRet+0
7C8024FD  00000000  Sleep+F
00A80460  00000000  ACE_Based::Thread::Sleep+30
00A86810  00000000  SqlDelayThread::run+50
00A80249  00000000  ACE_Based::Thread::ThreadTask+19
00292E34  00000000  ?invoke@ACE_OS_Thread_Adapter@@UAEKXZ+74
78543433  00000000  _endthreadex+44
785434C7  00000000  _endthreadex+D8
7C82482F  00000000  GetModuleHandleA+DF

mmaps tune off =(

Latest revision of mmaps...

I can't say anything elso...

[faramir118]

Seems like we need to do some memory profiling.

If there are memory management issues, it would be nice to know before we send it to under review.

And even if there aren't leaks, it seems like we should know the system requirements for running this code.

[qsa]

Crash again =(

Registers:
EAX:0B4580F8
EBX:47AE90C0
ECX:00000000
EDX:00000002
ESI:0B458180
EDI:47AE90DC
CS:EIP:B45001B:7C80BEF7
SS:ESP:60023:0B4580F4  EBP:0B458148
DS:E070023  ES:0023  FS:003B  GS:1700000
Flags:00000206

Call stack:
Address   Frame     Function      SourceFile
7C80BEF7  00000000  RaiseException+3C
7857DF56  00000000  _CxxThrowException+48
004753F0  00000000  operator new[]+40
0069364D  00000000  dtCustomAlloc+D
00401965  00000000  dtNavMesh::init+95
0096F998  00000000  MMAP::MMapManager::loadMapData+278

Call stack:
Address   Frame     Function      SourceFile
7C93860C  00000000  KiFastSystemCallRet+0
7C821C8D  00000000  WaitForSingleObject+12

Call stack:
Address   Frame     Function      SourceFile
7C93860C  00000000  KiFastSystemCallRet+0
7C8024FD  00000000  Sleep+F
00A80460  00000000  ACE_Based::Thread::Sleep+30
00A86810  00000000  SqlDelayThread::run+50
00A80249  00000000  ACE_Based::Thread::ThreadTask+19
00292E34  00000000  ?invoke@ACE_OS_Thread_Adapter@@UAEKXZ+74
78543433  00000000  _endthreadex+44
785434C7  00000000  _endthreadex+D8
7C82482F  00000000  GetModuleHandleA+DF

mmaps tune off =(

Latest revision of mmaps...

I can't say anything elso...

You are running out of memory again.

If you turn off the mmaps from configuration file, mmaps are still loading, just not used

If you want to prevent them from loading, just rename mmap folder to something else.

Seems like we need to do some memory profiling.

If there are memory management issues, it would be nice to know before we send it to under review.

And even if there aren't leaks, it seems like we should know the system requirements for running this code.

I think its something like ~4mega per tile and some pre-allocation per map.

Someone real testing can be nice from someone who's OS shows the actual memory usage

[ankso]

I have mmaps since some weeks, no significant changes on resources usage.

I use cloud server with one core @2GHz and 3Gb RAM, 50% memory usage and 35% CPU with 80-70 testers. Also I have a cata test realm on same machine, and other tasks runing, so memory is always completely used (10-20Mb free). On Ubuntu 8.04 LTS x64. No crashes related with memory usage, I have 12-24 hours uptime with a lot of custom modifications. May be with the crashfixes commited today to master branch I increase more my uptime.

I could do more test if you want, but I don't know how to test detailed resources usage on linux . Greetings.

[KiriX]

qsa, thx. I think what if I turn off mmaps in config - they will be not use and can't crash server...

I can rename mmaps folder but it is not resolve problem in fact =)

I afraid that I can't correctly understand this:

You are running out of memory again.

Not enough memory?

But I've already got crash from memory full error - and it was another crash-log...

I can build debug and give you debug crash-dump...

P.S: Always: MMAP:: MMapManager:: loadMapData +278

[Lynx3d]

Well operator new[] followed by an exception means most likely out of memory

Out of available contiguous addresses, more precisely, which on 32bit windows (without tweaks) can happen when the process uses about 1.5GB RAM already...

[qsa]

Well operator new[] followed by an exception means most likely out of memory

Out of available contiguous addresses, more precisely, which on 32bit windows (without tweaks) can happen when the process uses about 1.5GB RAM already...

This is pretty much nonsense on any modern OS (since VAXes) thanks to virtual memory. God knows what really goes on in windows universe

I never saw allocator throwing anything on normal usage. But, then again, I never run anything that takes over 1Giga memory on windows either.

qsa, thx. I think what if I turn off mmaps in config - they will be not use and can't crash server...

I can rename mmaps folder but it is not resolve problem in fact =)

I afraid that I can't correctly understand this:

You are running out of memory again.

Not enough memory?

But I've already got crash from memory full error - and it was another crash-log...

I can build debug and give you debug crash-dump...

P.S: Always: MMAP:: MMapManager:: loadMapData +278

We need some info about your system and how much it tried to allocate.

I'm pretty sure installing *nix will magically solve this problem. But, I feel even windows server edition will.

If anyone have similar problems, please do report. Otherwise I will consider it as "user's system issue"

I have mmaps since some weeks, no significant changes on resources usage.

I use cloud server with one core @2GHz and 3Gb RAM, 50% memory usage and 35% CPU with 80-70 testers. Also I have a cata test realm on same machine, and other tasks runing, so memory is always completely used (10-20Mb free). On Ubuntu 8.04 LTS x64. No crashes related with memory usage, I have 12-24 hours uptime with a lot of custom modifications. May be with the crashfixes commited today to master branch I increase more my uptime.

I could do more test if you want, but I don't know how to test detailed resources usage on linux . Greetings.

Well, thanks for report - even positive reports are good indicators. If you notice issues, do report.

hm... that gives me an idea. Now that we have all mmap loading handled from singelton, we can keep some statistics there. Things like number of maps loaded, overall number of tiles, etc. + some commands to show this data. Can be nice to do some theoretical calculations Since without those, knowing memory usage don't tell us much, well unless something really really wrong, but that's different story.

[Lynx3d]

Well operator new[] followed by an exception means most likely out of memory

Out of available contiguous addresses, more precisely, which on 32bit windows (without tweaks) can happen when the process uses about 1.5GB RAM already...

This is pretty much nonsense on any modern OS (since VAXes) thanks to virtual memory. God knows what really goes on in windows universe

I never saw allocator throwing anything on normal usage. But, then again, I never run anything that takes over 1Giga memory on windows either.

Well sorry exchange "RAM" by "Memory", but it's not nonsense. Virtual memory doesn't help with the fact that you eventually run out of contiguous 32bit addresses to allocate a memory chunk, virtual or not.

[iforgotmypassword]

mmaps needs ~2500 MB of memory for all continents, battlegrounds and instances on x64 architecture because one map use 20-30 MB

[qsa]

Well operator new[] followed by an exception means most likely out of memory

Out of available contiguous addresses, more precisely, which on 32bit windows (without tweaks) can happen when the process uses about 1.5GB RAM already...

This is pretty much nonsense on any modern OS (since VAXes) thanks to virtual memory. God knows what really goes on in windows universe

I never saw allocator throwing anything on normal usage. But, then again, I never run anything that takes over 1Giga memory on windows either.

Well sorry exchange "RAM" by "Memory", but it's not nonsense. Virtual memory doesn't help with the fact that you eventually run out of contiguous 32bit addresses to allocate a memory chunk, virtual or not.

Your post made me think about it once again. Thanks.

OS takes the upper quoter of our address space, which leaves us with around 3giga.

Lets assume all mmaps are loaded and never unloading, so we take another 2.5giga. Add all other things that have to be loaded, maps, vmaps, add some fragmentation and we can get pretty grim picture. We can have problem with continues address space.

I can't say we can solve it right now from our side. Upgrade to x64 architecture going to handle it tho.

Don't know if we can significantly lower memory usage right now. Using bigger world metrics will do the trick, but it will lower the quality of our meshes.

But thinking about it once more, if you do unload unused grids there's no freaking way you can fill the entire 2.5g.

[Lynx3d]

Windows actually still divides virtual address space 2GB/2GB by default from what i can find out, and changing it to 3/1 requires tweaking of Boot.ini (XP) or with BCDEdit (Vista/7), and the binary you run needs the "large address aware" bit set (special linker option) so the process will really get 3GB useable address space...hence, somewhere between 1.5 and 2GB used memory most applications will crap out without tweaking.

You can find many discussions on the web of people having trouble with large data sets in Photoshop, CAD programs etc. since many software providers were slow on migrating to 64bit.

[KiriX]

I have 1,5Gb RAM, but my online is not over 2-3 people =) Windows.

If I turn on mmaps I get crash at memory full error, but crash-log at that will be another...

I understand that 1,5Gb RAM is not enough for mmaps, but aslo I think that it is a error if I have crash with turn off mmaps in config...

I hope, you understand me =)

[faramir118]

The mmap.enabled setting just controls whether or not pathfinding is performed. mmap data is always loaded

We did it this way because if the setting is changed from off to on (with .reload conf), we have no good way to load the mmap data on every grid

This should probably be explained in the config file.

[Ambal]

We did it this way because if the setting is changed from off to on (with .reload conf), we have no good way to load the mmap data on every grid

If you are trying to make mmaps support config reload - this is, IMO, bad idea since such functionality should be initialized only on server startup.

Upd: if you like you can support config reload only while patch is under development. When it will be ready to be included in Core, please, remove this logic.

Best regards.

[faramir118]

such functionality should be initialized only on server startup.

The config file offers a persistent setting, so you can work around known issues - maps with a crappy mesh, or where unimplemented features like spawned GOs cause problems.

We need some runtime config option to handle other cases, like player abuse.

Admittedly, I was lazy and used the mangos.conf + .reload conf to satisfy both needs

How about this instead:

• add chat commands to disable pathfinding
  
• change the config settings so that they can't be reloaded
  
• change the config settings so that they actually prevent mmap data from loading
  

[Ambal]

faramir118 - for development purposes you can preload mmaps data into memory on server startup and process config reloads on runtime. But when you'll finish - it is better to stick with the way vmaps and regular geodata are loaded by the server - on-the-fly.This will save alot of memory for those who wishes to use mmaps but also wants to save some RAM.

And don't forget that 99% of server setups are "set it and forget it" .

Cheers.

[ankso]

I have a couple of doubts about warrior charges on retail, what happens if the enemy is unrecheable? and if the path to the enemy is very large? (For example, you are in your base in WG, and you charge to an enemy that is in the roof.) And finally, the stun is applied just after use the hability or when you reach the target?

[Feanordev]

Path distance doesn't matter for charges.

Look on youtube for some charge-fun videos

for example in HFC:Ramparts

you can charge thru whole instance from the beggining to end

Stun is added when you reach target.

[ankso]

Stun is added when you reach target.

Humm... When using mmaps, stun the target just after use the spell could be a problem. But this is related with MaNGOS, isn't it?

Anyway, on retail, imagine that you charge and you must follow a large path. I supose that, if the target moves when you are charging, the position of the target is updated and you will finish the charge at target's real position, not at the target's initial position, and apply the stun in that moment. Or I'm wrong?

[michalpolko]

I supose that, if the target moves when you are charging, the position of the target is updated and you will finish the charge at target's real position, not at the target's initial position, and apply the stun in that moment.

I was also thinking about this, seems to be important thing, maybe somebody knows something from playing on retail experience?

[faramir118]

For charge effects, it looks like proper implementation is a long way off. Thinking aloud:

• Mangos currently uses Relocate, so server-side travel is instant. Judging from videos, charge is not instant - in fact, speed appears to be constant and independent of distance. A special charge movement generator would be more appropriate.
  
• The stun spell is triggered - if travel is not instant, the stun would happen too early. So the trigger spell effect should not be processed immediately, instead being handed off to the charge movement generator. When the destination is reached, the spell should be triggered.
  
• If the charger gets stunned or something, the charge movement generator should be interrupted.
  
• Also judging from videos, it appears that the destination is never updated during travel. We can do better though, just depends on how blizzlike people want it
  

[faramir118]

Created ChargeMovementGenerator patch for make and VS2010.

Handled everything except the delayed spell trigger - I'm not too sure how to approach that.

[Feanordev]

Stun is added when you reach target.

Humm... When using mmaps, stun the target just after use the spell could be a problem. But this is related with MaNGOS, isn't it?

Anyway, on retail, imagine that you charge and you must follow a large path. I supose that, if the target moves when you are charging, the position of the target is updated and you will finish the charge at target's real position, not at the target's initial position, and apply the stun in that moment. Or I'm wrong?

Not sure if it happens in all cases but if warrior doesnt reach target on charge he will NOT stun target (target blinking directly after warrior start charging will make him 'avoid' stun)

and yes instant charge stun is normal bug becouse in DBC its trigger-effect therfore is always casted on charge cast

[Oniryck]

I know I'm bringing the subject back from the abyss, but do you think this amazing project will be ported to mangos-one one day ? We'd really need something like that too And congratulation for the fabulous work, in any case!

[faramir118]

Created ChargeMovementGenerator patch for make and VS2010.

Handled everything except the delayed spell trigger - I'm not too sure how to approach that.

Just noticed my diff didn't have the new movement generator.

Updated with all of the code.

I know I'm bringing the subject back from the abyss, but do you think this amazing project will be ported to mangos-one one day ? We'd really need something like that too And congratulation for the fabulous work, in any case!

vmaps v3 need to be ported first. Finding accurate documentation of old file and struct layouts is a challenge.

If it would get us more testers, I guess I can start poking around in old MPQs for mangos zero/one.