[10167] vmap rewrite

[caeruleaus]

Yeah i had seen the accidental use of the delete also.

To get it to those lines (yes I'm sorry it's actually two lines =P lol) I put a lot of printf statements to figure out where the program hung up. It calls so many different writeToFiles though that I didn't trace all the way through lol.

I could be wrong (which is quite possible) but that's just where it seemed to be happening.

[faramir118]

ceruleaus was very close!

The crash is at WorldModel.cpp:129, GroupModel::writeToFile()

if (result && fwrite(&vertices[0], sizeof(Vector3), count, wf) != count) result = false;

error is vector subscript out of range

Will have to find out which is the problem:

1. * Vertex info not getting written to disk? (in extractor)
   * Vertex info not getting read? (in assembler convert)
   * Some models don't have vertex info?
   

[Lynx3d]

Ah that was the "somewhere between to printf" method of finding the code line ^^

Inside writeToFile() then...makes a lot more sense to me.

So...empty group model somewhere...probably WMO because i just had added an empty-check to calculateTransformedBound() *scratches head*

I can also understand why it doesn't crash on linux then, because if 'vertices.size()' is zero, the invalid pointer '&vertices[0]' is probably never dereferenced by fwrite, since i tell it to write zero elements...interesting that MSVC is able to catch that actually...

I think that was very helpful, thanks!

[faramir118]

Yup, 40manarmygeneral.wmo actually reads one groupmodel, then crashes on the second because it has 0 vertices.

PS - Thanks for that tip about push_front earlier, fixed my problem. I need to keep track of what I change better

[Lynx3d]

I hope i fixed that one, and it finally has no reason to crash anymore.

I really wasn't aware that WMO group files without collision geometry get extracted too, the intersection code may have caused problems with that too, so i properly check for that now.

Maybe they are of no use, but i'm not sure yet.

[Stab]

I hope i fixed that one, and it finally has no reason to crash anymore.

I really wasn't aware that WMO group files without collision geometry get extracted too, the intersection code may have caused problems with that too, so i properly check for that now.

Maybe they are of no use, but i'm not sure yet.

No longer crashes but now it's only converting map 0 WMO files. You then get completed and then closes when you press any key to continue. Result is a less then 100Mb vmaps folder 000_27_29.vmtile - 001_25_39.vmtile and all the vmo files are from map 0 like stormwind.vmo, undercity.vmo, ironforge.vmo but no darnasus.vmo.. no orgramar.vmo none for any instance, northlend or outlands either.

[caeruleaus]

worked for me i think. I ended up with 1.30 gb and 8532 files

[faramir118]

I'm up and running with 3.3.3 maps.

edit: 8783 files (5980 .vmo, 2701 .vmtile, 102 .vmtree), 541 MB

This extractor+assembler seems much faster than the previous one!

Can pull from my branch to get it running on windows.

Build Instructions (make sure you change the build config to Release)

* compile libmpq (located in contrib/libmpq/win)

* compile extractor (located in contrib/vmap_extractor_v2/win)

* compile assembler (located in contrib/vmap_assembler)

* compile mangos

Run Instructions:

* copy contents of contrib/vmap_extract_assembler_bin to WoW directory

* run makevmaps_SIMPLE.bat

* copy vmaps directory from WoW directory to mangos directory

I think we're ready to merge mine and yours, Lynx. There are only a few conflicts, and mine has a few additions.

edit2: These two diffs are just code changes. The full diff also includes source for bzip2, zlib, and VS Solution/Project files

libmpq: http://paste2.org/p/796276

extractor: http://paste2.org/p/796277

[faramir118]

I was having a problem with 30 errors like the following:

warning: file World\\Maps\\Kalimdor\\Kalimdor_45_12.adt has size 0; cannot read.

I pushed a fix to my repo, and I get more vmap tiles files than before. I'm not sure if it's proper though.

[TOM_RUS]

I was having a problem with 30 errors like the following:

warning: file World\\Maps\\Kalimdor\\Kalimdor_45_12.adt has size 0; cannot read.

I pushed a fix to my repo, and I get more vmap tiles files than before. I'm not sure if it's proper though.

May be that file isn't used anymore? They do such things for unused files... Does it 0 size in patch*.mpq only or in all other mpq's as well?

[KAPATEJIb]

EDIT: okay, i find where the problem. This not related to your patch :rolleyes:

will test your patch at windows server soon

[Lynx3d]

faramir118: That looks promising

Is that file count above already with the fix for 0-size ADT files? Because i have a couple files less...hm i even have one map less...wtf...these MPQs are picky...

Maybe the loading order is still not correct (stormlib code did everything in reverse order to map extractor code...), but i guess your fix for it can't break anything.

Will look at your patches, first impression is that it looks quite sane.

And yes, the extractor had a horrible code section that unnecessarily slowed down extraction a lot.

And the assembler is faster for the same reason that vmaps are smaller, WMOs are not copied/transformed each spawn and hence don't need a new BSP tree each time, but the time difference here is not that dramatic for me.

Oh btw, forgot to mention: please always make sure the destination dir is EMPTY before running vmap_assembler, it only appends to the tile files.

[Stab]

* compile libmpq (located in contrib/libmpq/win)

* copy contents of contrib/vmap_extract_assembler_bin to WoW directory

There is no /win folder in libmpq

using the .exe's in vmap_extract_assembler_bin result to the current .vmap MaNGOS uses. You have to pull the .exe files from each projects /bin folder as they save there not in /vmap_extract_assembler_bin.

/sigh still trying.. but I'm still only getting Map 0 stuff. After each try I have deleted my buildings and vmaps folders.

Here is what I have done.. maybe i'm missing something..

First in VC90 I compiled /contrib vmap_extractor_v2 in release and took /contrib vmap_extractor_v2/bin/bin/Win32/ReleaseAS/vmap_extract_v2.exe and put it in my WoW client folder.

Next in VC90 I compiled /contrib/vmap_assembler in release and took /contrib/vmap_assembler/Release/vmap_assenbler.exe and put in my WoW folder.

Lastly I took the makevmaps_SIMPLE and the splitConfig.txt and put them both in WoW and ran the SIMPLE bat file.

Now if I run the bat like that, after it list all the mats loaded It just sits at a propt after the new Ruby map. never quits nor starts assemble like it should.

If I redo the bat file leaving out vmapextract so it starts at mdvmaps then it only makes the vmtile and vmo files of map 0 into vmaps and then propts completed.

[KAPATEJIb]

i was compiled all in VS 2008 as Faramir118 said in his guide and have Vmap folder with 8885 files for 548 MB (575*348*528 bytes). It is ok? Going to test it in game ^^

for hint: your compiled extractor in VS2010 (that placed in folder "vmap_extract_assembler_bin") will not start on most systems because required some libs from compilator, i was already tried to find them in the internet, but that what i find is not working

anyway i was compiled my own in VS2008 and it works.

[Lynx3d]

Stab: You are obviously looking at the wrong repo...

-edit-

KAPATEJIb: I haven't tested faramir's fix for the Kalimndor tiles yet, but seems reasonable...

[Stab]

Stab: You are obviously looking at the wrong repo...

I'm doing git clone http://github.com/Lynx3d/mangos.git;cd mangos;git pull http://github.com/Lynx3d/mangos.git vmap_rewrite

Look for yourself.. there is no /win in /contrib/libmpq/ - http://github.com/Lynx3d/mangos/tree/vmap_rewrite/contrib/libmpq/

[KAPATEJIb]

I'm doing git clone http://github.com/Lynx3d/mangos.git;cd mangos;git pull http://github.com/Lynx3d/mangos.git vmap_rewrite

Look for yourself.. there is no /win in /contrib/libmpq/ - http://github.com/Lynx3d/mangos/tree/vmap_rewrite/contrib/libmpq/

he means you are doing git clone from wrong repo, just replace Lynx3d to faramir118 in your link

here my compiled extractor (in VS2008) for those who can't compile by self or just lazy http://filekeeper.org/download/insider/vmaps_rewrite/333_vmaps_rewrite_extractor.7z

extracted vmaps http://filekeeper.org/download/insider/vmaps_rewrite/333_vmaps_rewrite.zip

[TOM_RUS]

i was compiled all in VS 2008 as Faramir118 said in his guide and have Vmap folder with 8885 files for 548 MB (575*348*528 bytes). It is ok? Going to test it in game ^^

for hint: your compiled extractor in VS2010 (that placed in folder "vmap_extract_assembler_bin") will not start on most systems because required some libs from compilator, i was already tried to find them in the internet, but that what i find is not working

anyway i was compiled my own in VS2008 and it works.

Learn to use google.

x86

http://www.microsoft.com/downloads/details.aspx?FamilyID=a7b7a05e-6de6-4d3a-a423-37bf0912db84&displaylang=en

x64

http://www.microsoft.com/downloads/details.aspx?familyid=BD512D9E-43C8-4655-81BF-9350143D5867&displaylang=en

[KAPATEJIb]

also better to get rid of things like that when loglevel=0

      2623     2A41 -233.319,-117.438,-28.319 | -23.643,117.981, 43.542 | z=-7.0
03755, p_z=-7.003662
VMapManager::getAreaInfo(): x:-8836.264648 y:639.547913 z:94.953362 => x:25902.9
31641 y:16427.119141 z:94.953362
trying to intersect 'Stormwind.wmo'
     2623     2A41 -233.319,-117.438,-28.319 | -23.643,117.981, 43.542 | z=-7.0
03755, p_z=-7.003662
VMapManager::getAreaInfo(): x:-8836.264648 y:639.547913 z:94.953362 => x:25902.9
31641 y:16427.119141 z:94.953362
trying to intersect 'Stormwind.wmo'
     2623     2A41 -233.319,-117.438,-28.319 | -23.643,117.981, 43.542 | z=-7.0
03755, p_z=-7.003662
VMapManager::getAreaInfo(): x:-8836.264648 y:639.547913 z:94.953362 => x:25902.9
31641 y:16427.119141 z:94.953362
trying to intersect 'Stormwind.wmo'
     2623     2A41 -233.319,-117.438,-28.319 | -23.643,117.981, 43.542 | z=-7.0
03755, p_z=-7.003662
VMapManager::getAreaInfo(): x:-8836.264648 y:639.547913 z:94.953362 => x:25902.9
31641 y:16427.119141 z:94.953362
trying to intersect 'Stormwind.wmo'
     2623     2A41 -233.319,-117.438,-28.319 | -23.643,117.981, 43.542 | z=-7.0
03755, p_z=-7.003662

and this flood doesn't stop even if i stay at one place.

is there a way to fix Dalaran and Orgrimmar fishing? As i remember it not works in clean mangos because need to read additional VMO data, but now we has this data...

* indoor/outdoor check which allows automatic unmounting of players

this not works for me, i still can enter in the houses of stormwind but doesn't get dismount

[faramir118]

May be that file isn't used anymore? They do such things for unused files... Does it 0 size in patch*.mpq only or in all other mpq's as well?

That file is over 600KB, and it is in patch.mpq and common-2.mpq. Problem was that there was a 0-size file in another mpq.

I don't know which mpq is proper to pick data from, so do we just get one that has usable data? That's what my fix does.

i was compiled all in VS 2008 as Faramir118 said in his guide and have Vmap folder with 8885 files for 548 MB (575*348*528 bytes). It is ok? Going to test it in game ^^

Here is my count, using my repo's code, starting with empty Buildings and vmaps directory.

Buildings/
   4531 m2        6840kB
   1985 wmo     252926kB
      1 dir_bin  41916kB
   6517         301682kB

vmaps/
   6080 vmo     497744kB
   2700 vmtile   53055kB
    103 vmtree   10895kB
   8883         561696kB

[faramir118]

here my compiled extractor (in VS2008) for those who can't compile by self or just lazy http://filekeeper.org/download/insider/vmaps_rewrite/333_vmaps_rewrite_extractor.7z

extracted vmaps http://filekeeper.org/download/insider/vmaps_rewrite/333_vmaps_rewrite.zip

Also available in contrib/vmap_extract_assebler_bin/ from http://github.com/faramir118/mangos/tree/vmap_rewrite

[fabian]

also better to get rid of things like that when loglevel=0

      2623     2A41 -233.319,-117.438,-28.319 | -23.643,117.981, 43.542 | z=-7.0
03755, p_z=-7.003662
VMapManager::getAreaInfo(): x:-8836.264648 y:639.547913 z:94.953362 => x:25902.9
31641 y:16427.119141 z:94.953362
trying to intersect 'Stormwind.wmo'
     2623     2A41 -233.319,-117.438,-28.319 | -23.643,117.981, 43.542 | z=-7.0
03755, p_z=-7.003662
VMapManager::getAreaInfo(): x:-8836.264648 y:639.547913 z:94.953362 => x:25902.9
31641 y:16427.119141 z:94.953362
trying to intersect 'Stormwind.wmo'
     2623     2A41 -233.319,-117.438,-28.319 | -23.643,117.981, 43.542 | z=-7.0
03755, p_z=-7.003662
VMapManager::getAreaInfo(): x:-8836.264648 y:639.547913 z:94.953362 => x:25902.9
31641 y:16427.119141 z:94.953362
trying to intersect 'Stormwind.wmo'
     2623     2A41 -233.319,-117.438,-28.319 | -23.643,117.981, 43.542 | z=-7.0
03755, p_z=-7.003662
VMapManager::getAreaInfo(): x:-8836.264648 y:639.547913 z:94.953362 => x:25902.9
31641 y:16427.119141 z:94.953362
trying to intersect 'Stormwind.wmo'
     2623     2A41 -233.319,-117.438,-28.319 | -23.643,117.981, 43.542 | z=-7.0
03755, p_z=-7.003662

and this flood doesn't stop even if i stay at one place.

is there a way to fix Dalaran and Orgrimmar fishing? As i remember it not works in clean mangos because need to read additional VMO data, but now we has this data...

this not works for me, i still can enter in the houses of stormwind but doesn't get dismount

I have no error and unmount in houses works for me

[KAPATEJIb]

I have no error and unmount in houses works for me

this is strange... i was tried in Orgrimmar too, have same flood

897461 y:21489.873047 z:16.994158
trying to intersect 'Ogrimmar.wmo'
     1949     2841 -236.212,-115.504,-32.576 | -73.973,186.099,117.556 | z=-13.
666945, p_z=-13.667599
VMapManager::getAreaInfo(): x:1662.769043 y:-4423.206055 z:16.994158 => x:15403.
897461 y:21489.873047 z:16.994158
trying to intersect 'Ogrimmar.wmo'
     1949     2841 -236.212,-115.504,-32.576 | -73.973,186.099,117.556 | z=-13.
666945, p_z=-13.667599
VMapManager::getAreaInfo(): x:1662.769043 y:-4423.206055 z:16.994158 => x:15403.
897461 y:21489.873047 z:16.994158
trying to intersect 'Ogrimmar.wmo'
     1949     2841 -236.212,-115.504,-32.576 | -73.973,186.099,117.556 | z=-13.
666945, p_z=-13.667599
VMapManager::getAreaInfo(): x:1662.769043 y:-4423.206055 z:16.994158 => x:15403.
897461 y:21489.873047 z:16.994158
trying to intersect 'Ogrimmar.wmo'
     1949     2841 -236.212,-115.504,-32.576 | -73.973,186.099,117.556 | z=-13.
666945, p_z=-13.667599
VMapManager::getAreaInfo(): x:1662.769043 y:-4423.206055 z:16.994158 => x:15403.
897461 y:21489.873047 z:16.994158
trying to intersect 'Ogrimmar.wmo'
     1949     2841 -236.212,-115.504,-32.576 | -73.973,186.099,117.556 | z=-13.
666945, p_z=-13.667599

and unmounting not work

are you trying patch at windows?

This i get at server shutdown

VMapManager2: unloading file 'Generallantern02.m2'.
VMapManager2: unloading file 'Cavekoboldbucket.m2'.
VMapManager2: unloading file 'Innbarrel.m2'.
VMapManager2: unloading file 'Rowboat01.m2'.
VMapManager2: unloading file 'Elwynntreecanopy01.m2'.
VMapManager2: unloading file 'Redridgefallentree02.m2'.
VMapManager2: unloading file 'Elwynnwoodpost01.m2'.
VMapManager2: unloading file 'Elwynnpine01.m2'.
VMapManager2: unloading file 'Elwynntreestump01.m2'.
VMapManager2: unloading file 'Elwynnwoodfence01.m2'.
VMapManager2: unloading file 'Abbeygate01.wmo'.
VMapManager2: unloading file 'Farm.wmo'.
VMapManager2: unloading file 'Wallpiece01.wmo'.
VMapManager2: unloading file 'Wallpost01.wmo'.
Error reading ModelSpawn!
VMapManager2: unloading file 'Elwynntreecanopy02.m2'.
VMapManager2: unloading file 'Elwynntreecanopy04.m2'.
VMapManager2: unloading file 'Redridgefallentree03.m2'.
VMapManager2: unloading file 'Redridgefallentree01.m2'.
VMapManager2: unloading file 'Canopylesstree01.m2'.
VMapManager2: unloading file 'Elwynntreemid01.m2'.
VMapManager2: unloading file 'Deadminecargoboxes.m2'.
VMapManager2: unloading file 'Ballista.m2'.
VMapManager2: unloading file 'Elwynntreecanopy03.m2'.
VMapManager2: unloading file 'Stormwind.wmo'.
Error reading ModelSpawn!
VMapManager2: unloading file 'Jarorc06.m2'.
VMapManager2: unloading file 'Bucket.m2'.
VMapManager2: unloading file 'Orcsword03.m2'.
VMapManager2: unloading file 'Orccrate10.m2'.
VMapManager2: unloading file 'Jarorc01.m2'.
VMapManager2: unloading file 'Orccrate06.m2'.
VMapManager2: unloading file 'Orccrate07.m2'.
VMapManager2: unloading file 'Orcshield01.m2'.
VMapManager2: unloading file 'Westfallwagon01.m2'.
VMapManager2: unloading file 'Firewoodpile03.m2'.
VMapManager2: unloading file 'Animalcage01.m2'.
VMapManager2: unloading file 'Orcbrazier_Lightpostbarrens.m
VMapManager2: unloading file 'Blastedlandsrock01.m2'.
VMapManager2: unloading file 'Cauldronempty.m2'.
VMapManager2: unloading file 'Goblintent06.m2'.
VMapManager2: unloading file 'Badlandspalmtree01.m2'.
VMapManager2: unloading file 'Durotarcliffrock07.m2'.
VMapManager2: unloading file 'Wetlandswagon03.m2'.
VMapManager2: unloading file 'Wetlandswagon01.m2'.
VMapManager2: unloading file 'Durotarcliffrock08.m2'.
VMapManager2: unloading file 'Wetlandswagon02.m2'.
VMapManager2: unloading file 'Md_Warmcave_Medium.wmo'.
VMapManager2: unloading file 'Md_Warmcavetunnel01.wmo'.
Error reading ModelSpawn!
VMapManager2: unloading file 'Durotarcliffrock06.m2'.
VMapManager2: unloading file 'Durotarcliffrock01.m2'.
VMapManager2: unloading file 'Durotarcliffrock05.m2'.
VMapManager2: unloading file 'Durotarcliffrock04.m2'.
VMapManager2: unloading file 'Durotarrock05.m2'.
VMapManager2: unloading file 'Durotarrock06.m2'.
VMapManager2: unloading file 'Stonetalonharpysnest02.m2'.
VMapManager2: unloading file 'Stonetalonharpysnest01.m2'.
VMapManager2: unloading file 'Kalidarnest01.m2'.
VMapManager2: unloading file 'Kodoegg01.m2'.
VMapManager2: unloading file 'Dragonegg01.m2'.
VMapManager2: unloading file 'Kodoegg02.m2'.
VMapManager2: unloading file 'Badlandscactus_2.m2'.
VMapManager2: unloading file 'Badlandscactus_3.m2'.
VMapManager2: unloading file 'Badlandscactus_8.m2'.
VMapManager2: unloading file 'Badlandscactus_4.m2'.
VMapManager2: unloading file 'Badlandscactus_9.m2'.
VMapManager2: unloading file 'Badlandscactus_5.m2'.
VMapManager2: unloading file 'Badlandscactus_10.m2'.
VMapManager2: unloading file 'Feedtroph01.m2'.
VMapManager2: unloading file 'Orccrate01.m2'.
VMapManager2: unloading file 'Jarorc04.m2'.
VMapManager2: unloading file 'Orctable01.m2'.
VMapManager2: unloading file 'Orctoolrackburningsteppes.m2'
VMapManager2: unloading file 'Jarorc03.m2'.
VMapManager2: unloading file 'Jarorc05.m2'.
VMapManager2: unloading file 'Orcbarrel01.m2'.
VMapManager2: unloading file 'Taurensack03.m2'.
VMapManager2: unloading file 'Orccrate02.m2'.
VMapManager2: unloading file 'Durotarpalm02.m2'.
VMapManager2: unloading file 'Badlandspalmtree02.m2'.
VMapManager2: unloading file 'Durotartree04.m2'.
VMapManager2: unloading file 'Durotarpalm01.m2'.
VMapManager2: unloading file 'Durotarpalm03.m2'.
VMapManager2: unloading file 'Orccrate03.m2'.
VMapManager2: unloading file 'Goblintent07.m2'.
VMapManager2: unloading file 'Smallbasket01.m2'.
VMapManager2: unloading file 'Flatbasket01.m2'.
VMapManager2: unloading file 'Orccrate08.m2'.
VMapManager2: unloading file 'Orccrate09.m2'.
VMapManager2: unloading file 'Durotarrock04.m2'.
VMapManager2: unloading file 'Durotarrock03.m2'.
VMapManager2: unloading file 'Orcspear03.m2'.
VMapManager2: unloading file 'Trolldungeonpottery03.m2'.
VMapManager2: unloading file 'Taurensack01.m2'.
VMapManager2: unloading file 'Orcspear01.m2'.
VMapManager2: unloading file 'Trolldungeonpottery01.m2'.
VMapManager2: unloading file 'Orcsword02.m2'.
VMapManager2: unloading file 'Mummytroll01.m2'.
VMapManager2: unloading file 'Taurenbowl09.m2'.
VMapManager2: unloading file 'Taurenbowl03.m2'.
VMapManager2: unloading file 'Trollskullpile.m2'.
VMapManager2: unloading file 'Orcshield02.m2'.
VMapManager2: unloading file 'Orcshield03.m2'.
VMapManager2: unloading file 'Durotarbush04.m2'.
VMapManager2: unloading file 'Orcspear02.m2'.
VMapManager2: unloading file 'Redridgerock03.m2'.
VMapManager2: unloading file 'Durotarbush03.m2'.
VMapManager2: unloading file 'Taurensack02.m2'.
VMapManager2: unloading file 'Redridgerock01.m2'.
VMapManager2: unloading file 'Redridgerock02.m2'.
VMapManager2: unloading file 'Jarorc02.m2'.
VMapManager2: unloading file 'Badlandscactus_1.m2'.
VMapManager2: unloading file 'Generaltorch01.m2'.
VMapManager2: unloading file 'Orcbrazierstreetlamp.m2'.
VMapManager2: unloading file 'Orcsignpost03.m2'.
VMapManager2: unloading file 'Orcfence.m2'.
VMapManager2: unloading file 'Orcfencepost.m2'.
VMapManager2: unloading file 'Durotarrock07.m2'.
VMapManager2: unloading file 'Durotarrock08.m2'.
VMapManager2: unloading file 'Hordebanner02.m2'.
VMapManager2: unloading file 'Barrenstree04.m2'.
VMapManager2: unloading file 'Mediumbrazier01.m2'.
VMapManager2: unloading file 'Durotartree05.m2'.
VMapManager2: unloading file 'Md_Warmcave.wmo'.
VMapManager2: unloading file 'Orchut.wmo'.
VMapManager2: unloading file 'Orczeppelinhouse_Durotar01.wm
Error reading ModelSpawn!
VMapManager2: unloading file 'Durotartree03.m2'.
VMapManager2: unloading file 'Ogrimmar.wmo'.
Error reading ModelSpawn!
Halting process...

[Thyros]

use you this patch with your insider42 repo?

[KAPATEJIb]

use you this patch with your insider42 repo?

yes, i am using patch with my repo. I will try clean now, but i think my core mods NOT related to that problem. This can happen because of Ru-ru locale?