[10167] vmap rewrite

Thyros · April 22, 2010

and what can i do, to fix this problem?

Lynx3d · April 22, 2010

Eh..."ChamberofAspectRed" is the last map currently in game, so of course it stops there, it's done ^^

-edit-

Wait, are you sure you actually let it finish? After it read the map files it starts converting them without further output (except some warnings about empty ADT files and a final success/error message), takes a few minutes.

Stab · April 22, 2010

Lynx3d I sent you a PM regarding my compile issues on CentOS 5

faramir118 · April 22, 2010

I added debug configs to my branch.

Has anybody run into trouble on Windows?

Woweur · April 22, 2010

I will try your's wmap rewrite soon with ~2.6K ppl, and i repport here performance and bug issue.

Thanks you for the work !

przemratajczak · April 22, 2010

vmapExtractor_VC90.sln (from Lynx3d branch)

Windows XP SP2 Visual c++ 2008 with SDK6.1

Client build 11723 enUS

1>------ Build started: Project: vmapExtractor, Configuration: Release Ansi Static Win32 ------

1>Compiling...

1>wmo.cpp

1>..\\extractor\\mpq_libmpq.h(93) : warning C4100: 'f' : unreferenced formal parameter

1>..\\extractor\\mpq_libmpq.h(94) : warning C4100: 'f' : unreferenced formal parameter

1>.\\vmapextract\\wmo.cpp(358) : warning C4018: '<' : signed/unsigned mismatch

1>.\\vmapextract\\wmo.cpp(371) : warning C4244: '=' : conversion from 'int' to 'uint16', possible loss of data

1>.\\vmapextract\\wmo.cpp(383) : warning C4018: '<' : signed/unsigned mismatch

1>wdtfile.cpp

1>..\\extractor\\mpq_libmpq.h(93) : warning C4100: 'f' : unreferenced formal parameter

1>..\\extractor\\mpq_libmpq.h(94) : warning C4100: 'f' : unreferenced formal parameter

1>vmapexport.cpp

1>..\\extractor\\mpq_libmpq.h(93) : warning C4100: 'f' : unreferenced formal parameter

1>..\\extractor\\mpq_libmpq.h(94) : warning C4100: 'f' : unreferenced formal parameter

1>.\\vmapextract\\vmapexport.cpp(95) : warning C4244: '=' : conversion from 'int' to 'char', possible loss of data

1>.\\vmapextract\\vmapexport.cpp(184) : warning C4018: '<' : signed/unsigned mismatch

1>.\\vmapextract\\vmapexport.cpp(234) : warning C4101: 'id_filename' : unreferenced local variable

1>.\\vmapextract\\vmapexport.cpp(491) : warning C4800: 'int' : forcing value to bool 'true' or 'false' (performance warning)

1>.\\vmapextract\\vmapexport.cpp(109) : warning C4505: 'ShowProcessedFile' : unreferenced local function has been removed

1>mpq_libmpq.cpp

1>e:\\work\\contrib\\extractor\\mpq_libmpq.h(93) : warning C4100: 'f' : unreferenced formal parameter

1>e:\\work\\contrib\\extractor\\mpq_libmpq.h(94) : warning C4100: 'f' : unreferenced formal parameter

1>model.cpp

1>..\\extractor\\mpq_libmpq.h(93) : warning C4100: 'f' : unreferenced formal parameter

1>..\\extractor\\mpq_libmpq.h(94) : warning C4100: 'f' : unreferenced formal parameter

1>dbcfile.cpp

1>..\\extractor\\mpq_libmpq.h(93) : warning C4100: 'f' : unreferenced formal parameter

1>..\\extractor\\mpq_libmpq.h(94) : warning C4100: 'f' : unreferenced formal parameter

1>adtfile.cpp

1>..\\extractor\\mpq_libmpq.h(93) : warning C4100: 'f' : unreferenced formal parameter

1>..\\extractor\\mpq_libmpq.h(94) : warning C4100: 'f' : unreferenced formal parameter

1>wave.cpp

1>parser.cpp

1>..\\extractor\\libmpq\\parser.cpp(58) : warning C4100: 'search_value' : unreferenced formal parameter

1>..\\extractor\\libmpq\\parser.cpp(205) : warning C4189: 'i' : local variable is initialized but not referenced

1>mpq.cpp

1>..\\extractor\\libmpq\\mpq.cpp(202) : warning C4189: 'i' : local variable is initialized but not referenced

1>..\\extractor\\libmpq\\mpq.cpp(204) : warning C4189: 'mpq_h' : local variable is initialized but not referenced

1>..\\extractor\\libmpq\\mpq.cpp(358) : warning C4018: '>' : signed/unsigned mismatch

1>..\\extractor\\libmpq\\mpq.cpp(328) : warning C4189: 'i' : local variable is initialized but not referenced

1>..\\extractor\\libmpq\\mpq.cpp(557) : warning C4018: '>' : signed/unsigned mismatch

1>..\\extractor\\libmpq\\mpq.cpp(599) : warning C4389: '==' : signed/unsigned mismatch

1>..\\extractor\\libmpq\\mpq.cpp(538) : warning C4189: 'i' : local variable is initialized but not referenced

1>huffman.cpp

1>extract.cpp

1>..\\extractor\\libmpq\\extract.cpp(159) : warning C4100: 'in_length' : unreferenced formal parameter

1>explode.cpp

1>common.cpp

1>..\\extractor\\libmpq\\common.cpp(198) : warning C4389: '==' : signed/unsigned mismatch

1>..\\extractor\\libmpq\\common.cpp(198) : warning C4389: '==' : signed/unsigned mismatch

1>..\\extractor\\libmpq\\common.cpp(198) : warning C4389: '==' : signed/unsigned mismatch

1>..\\extractor\\libmpq\\common.cpp(417) : warning C4389: '!=' : signed/unsigned mismatch

1>..\\extractor\\libmpq\\common.cpp(472) : warning C4389: '!=' : signed/unsigned mismatch

1>Generating Code...

1>e:\\work\\contrib\\extractor\\libmpq\\huffman.cpp(739) : warning C4701: potentially uninitialized local variable 'p_item1' used

1>e:\\work\\contrib\\extractor\\libmpq\\parser.cpp(43) : warning C4706: assignment within conditional expression

1>Linking...

1>Embedding manifest...

1>Build log was saved at "file://e:\\work\\contrib\\vmap_extractor_v2\\bin\\Win32\\ReleaseAS\\BuildLog.htm"

1>vmapExtractor - 0 error(s), 39 warning(s)

========== Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========

and crashed at attempt to execute

Extract V2.4 2007_07_12. Beginning work ....

Game path: D:\\Client 3.3.3a\\Data\\

Opening data files from locale directories.

Loading patch levels from data directory.

Loading patch levels from locale directories.

Locale: enGB

Locale: enUS

Locale: deDE

Locale: esES

Locale: frFR

Locale: koKR

Locale: ruRU

Opening D:\\Client 3.3.3a\\Data\\enGB/locale-enGB.MPQ

Error opening archive 'D:\\Client 3.3.3a\\Data\\enGB/locale-enGB.MPQ': File operati

on Error

przemratajczak · April 22, 2010

the same with faramir's. am i doing something wrong that app recognize my client as enGB? :<

caeruleaus · April 22, 2010

i got it to extract fine. But when i run the assembler i get an error that says "Error: file name too long!" and then on the next line something like

min guid: 5579 max guid:5497953

Then it starts to assemble the vmaps but crashes on stormwind.wmo

faramir118 · April 22, 2010

Hrm, I hadn't tried to compile my changes into the extractor. I'll work on it and push the changes to my repo.

I fixed an overflow bug in libmpq that was making it error on files toward the end of large mpqs.

Very weird issue...

typedef long      off_t;
typedef __int64   libmpq__off_t;
typedef __int64   longlong;

off_t         = uint32 + longlong + uint32  // overflow, ends up being negative
libmpq__off_t = uint32 + longlong + uint32  // fixes error

Stab · April 23, 2010

vmapExtractor_VC90.sln (from Lynx3d branch)
Windows XP SP2 Visual c++ 2008 with SDK6.1

Client build 11723 enUS

and crashed at attempt to execute

I got this too, in the extract project search for enGB... change the enGB to enUS and change enUS on the next line to enGB save and recompile.

This is what I did and extract is currently running now instead of crashing on that error

Stab · April 23, 2010

I'm stuck in assemble I get to spawning map 249 and then min Guid: 5154 max 5497977 and the it just hangs.

Lynx3d · April 23, 2010

Just jound a bug in extractor that caused a crash when enGB (first search locale) was not present.

Also improved scan code to not print irritating errors for not installed locales.

But i have no clue why assembler fails, need more detailed information.

How many files did extractor create? How big is the dir_bin file created? Maybe a checksum (MD5 or SHA1) would also help, but i'm not sure it's really locale independent...

What exactly means "hang"? 100% CPU usage? Memory usage?

It should definitely go til "Spawning Map 724" before starting printing that Guid ranges, and after that printing which model file it converts...

Stab · April 23, 2010

Just jound a bug in extractor that caused a crash when enGB (first search locale) was not present.
Also improved scan code to not print irritating errors for not installed locales.

But i have no clue why assembler fails, need more detailed information.

How many files did extractor create? How big is the dir_bin file created? Maybe a checksum (MD5 or SHA1) would also help, but i'm not sure it's really locale independent...

What exactly means "hang"? 100% CPU usage? Memory usage?

It should definitely go til "Spawning Map 724" before starting printing that Guid ranges, and after that printing which model file it converts...

It runs a bunch of guid's then..

Converting Model Files

Converting 1000Needlesbridge.wmo

Converting40Manarmygeneral.wmo

Then you get a pop-up window that the application quit and cmd.exe then says pause hit any key to continue and closes when you do.

It always happens at that same point too trying both your and Faramar's assembler. You have to relaunch assembler after extracting too as it lists the full list of maps inclding the new Ruby 724 but then just hangs.

As for the enGB issue, I posted a soltion that worked for me. I changed enGB to enUS and enUS to enGB and ran visual studio again and that got extactor to work.

caeruleaus · April 23, 2010

For me it created 6517 files in the buildings directory.

When i run the assembler it says the min guid max guid thing right off the bat and then crashes once it gets to "Converting stormwind.wmo"

faramir118 · April 24, 2010

I made some changes so that the extractor would run correctly on windows. Problems I'm having:

* some maps aren't getting loaded
I noticed that Map.dbc is stored in multiple mpq archives. Is Map.dbc split across multiple mpqs, or something? (sorry, I'm clueless about this sort of thing)
* Having an error with WMO group ND_Forsaken_Apothecary.wmo
* assembler crashes while converting wmo files

I uploaded the output from extractor + assembler: http://paste2.org/p/790489

Look at lines 29, 34, 200, 211, and the last few lines.

And of course, the map count is only 118

KAPATEJIb · April 24, 2010

It runs a bunch of guid's then..
Converting Model Files

Converting 1000Needlesbridge.wmo

Converting40Manarmygeneral.wmo

Then you get a pop-up window that the application quit and cmd.exe then says pause hit any key to continue and closes when you do.

It always happens at that same point too trying both your and Faramar's assembler. You have to relaunch assembler after extracting too as it lists the full list of maps inclding the new Ruby 724 but then just hangs.

I have same problem at Windows 7.

As I understand I can extract vmaps in Linux and use these files for windows server with this patch, it will work?

Lynx3d · April 24, 2010

I made some changes so that the extractor would run correctly on windows. Problems I'm having:

* some maps aren't getting loaded
I noticed that Map.dbc is stored in multiple mpq archives. Is Map.dbc split across multiple mpqs, or something? (sorry, I'm clueless about this sort of thing)
* Having an error with WMO group ND_Forsaken_Apothecary.wmo

You know what, this problem sounds very familiar...

You even noticed "(...) that Map.dbc is stored in multiple mpq archives."

Now you have to give me a very good reason why you did this:

    }
-    gOpenArchives.push_front(this);
+    gOpenArchives.push_back(this);
}

In other words, you deliberately broke it

Some things indeed have a reason that they are as they are...

About the crash of assembler, please give me at least a hint about the type of crash...is it a segmentation fault, stack overflow, floating point exception etc.

And maybe someone even knows how to use a debugger to find the causing code line...

@KAPATEJib: Yes Linux assembled vmaps should work on windows too unless i overlooked something gross...just make sure the CPU has the same endianness.

KAPATEJIb · April 24, 2010

About the crash of assembler, please give me at least a hint about the type of crash...is it a segmentation fault, stack overflow, floating point exception etc.

it (Win7) doesn't give so much info as on Windows XP

Problem signature:
Problem Event Name: BEX

Application name: vmap_assembler.exe

Application version: 0.0.0.0

Timestamp application: 4bd2ab64

Name of the module with error: MSVCR90.dll

Version of the module with error: 9.0.30729.4926

Timestamp module with an error: 4a1743c1

Exception offset: 0006c955

Exception code: c0000417

Exception data: 00000000

OS version: 6.1.7600.2.0.0.256.48

Language code: 1049

Additional info 1: 730f

Additional info 2: 730ff60f5430b8d7e4ce30de7787852e

Additional info 3: 6761

Additional info 4: 6761ef3421c6da3cfbf1d982367e36d3

Maybe somewhere translation typo's because i just translated from Russian to English

caeruleaus · April 24, 2010

it's crashing because of a a vector check actually. I'll go run it again and debug it for you.

 #endif /* _HAS_ITERATOR_DEBUGGING */
       _SCL_SECURE_VALIDATE_RANGE(_Pos < size()); <---------- Debugger says it crashes there

       return (*(_Myfirst + _Pos));
       }

EDIT: I compiled the assembler in debug and then ran it. It's throwing an assert that the vector subscript is out of range (which makes sense since that what it's checking here.)

Thyros · April 24, 2010

I've Update the bat files to the vmap extractor, can someone test this?

DOWNLOAD NOW!

Patch file:

commit 8a38aee5c46fc96b9e8d0c4fcb0ce0d60077d503
Author: unknown <Zynum@.(none)>
Date:   Sat Apr 24 13:03:28 2010 +0200

   update the bat files.

diff --git a/contrib/vmap_extract_assembler_bin/make vmaps.bat b/contrib/vmap_extract_assembler_bin/make vmaps.bat
index cb6714f..ec5a905 100644
--- a/contrib/vmap_extract_assembler_bin/make vmaps.bat    
+++ b/contrib/vmap_extract_assembler_bin/make vmaps.bat    
@@ -1,46 +1,46 @@
-@echo off
-cls
-echo.
-echo                    Welcome to the vmaps extractor and assembler
-echo.
-echo You need 2GB of free space in disk, CTRL+C to stop process
-echo Hit Enter to start . . .
-pause>nul
-cls
-echo.
-echo.
-echo.
-IF EXIST buildings\\dir (ECHO The buildings folder already exist do you want to delete it?
-echo If YES hit Enter to continue if no CLOSE the program now! . . .
-pause>nul
-DEL /S /Q buildings)
-vmapextract_v2.exe
-cls
-echo.
-echo.
-echo.
-IF NOT %ERRORLEVEL% LEQ 1 (echo The vmap extract tool finalized with errors.
-echo Hit Enter to continue . . .
-pause>nul)
-cls
-echo.
-echo.
-echo.
-echo Vmaps extracted check log.txt for errors, now it's time to assemble the vmaps press any key to continue . . .
-pause>nul
-md vmaps
-vmap_assembler.exe buildings vmaps splitConfig.txt
-cls
-echo.
-echo.
-echo.
-IF NOT %ERRORLEVEL% LEQ 1 (echo The vmap assembler tool finalized with errors.
-echo Hit Enter to continue . . .
-pause>nul)
-cls
-echo.
-echo.
-echo.
-echo Process done! copy vmaps folder to the MaNGOS main directory
-echo Press any key to exit . . .
+@echo off
+cls
+echo.
+echo                    Welcome to the vmaps extractor and assembler
+echo.
+echo You need 2GB of free space in disk, CTRL+C to stop process
+echo Hit Enter to start . . .
+pause>nul
+cls
+echo.
+echo.
+echo.
+IF EXIST buildings\\dir (ECHO The buildings folder already exist do you want to delete it?
+echo If YES hit Enter to continue if no CLOSE the program now! . . .
+pause>nul
+DEL /S /Q buildings)
+vmapExtractor3.exe
+cls
+echo.
+echo.
+echo.
+IF NOT %ERRORLEVEL% LEQ 1 (echo The vmap extract tool finalized with errors.
+echo Hit Enter to continue . . .
+pause>nul)
+cls
+echo.
+echo.
+echo.
+echo Vmaps extracted check log.txt for errors, now it's time to assemble the vmaps press any key to continue . . .
+pause>nul
+md vmaps
+vmap_assembler.exe buildings vmaps splitConfig.txt
+cls
+echo.
+echo.
+echo.
+IF NOT %ERRORLEVEL% LEQ 1 (echo The vmap assembler tool finalized with errors.
+echo Hit Enter to continue . . .
+pause>nul)
+cls
+echo.
+echo.
+echo.
+echo Process done! copy vmaps folder to the MaNGOS main directory
+echo Press any key to exit . . .
pause>nul
\\ No newline at end of file
diff --git a/contrib/vmap_extract_assembler_bin/makevmaps_SIMPLE.bat b/contrib/vmap_extract_assembler_bin/makevmaps_SIMPLE.bat
index bf01c24..e5d115d 100644
--- a/contrib/vmap_extract_assembler_bin/makevmaps_SIMPLE.bat
+++ b/contrib/vmap_extract_assembler_bin/makevmaps_SIMPLE.bat
@@ -1,4 +1,4 @@
-vmapextract_v2.exe
+vmapExtractor3.exe
md vmaps
vmap_assembler.exe buildings vmaps splitConfig.txt

diff --git a/contrib/vmap_extract_assembler_bin/vmap_assembler.exe b/contrib/vmap_extract_assembler_bin/vmap_assembler.exe
index 9c8b5f2..b0fe3dc 100644
Binary files a/contrib/vmap_extract_assembler_bin/vmap_assembler.exe and b/contrib/vmap_extract_assembler_bin/vmap_assembler.exe differ
diff --git a/contrib/vmap_extract_assembler_bin/vmapextract_v2.exe b/contrib/vmap_extract_assembler_bin/vmapextract_v2.exe
deleted file mode 100644
index 213705f..0000000
Binary files a/contrib/vmap_extract_assembler_bin/vmapextract_v2.exe and /dev/null differ

faramir118 · April 25, 2010

Now you have to give me a very good reason why you did this:
    }
-    gOpenArchives.push_front(this);
+    gOpenArchives.push_back(this);
}
In other words, you deliberately broke it

Ah damn, my bad.

The reason why I did that is because at one point I had changed gOpenArchives to a vector instead of a deque - I was trying to fix the crash at

if(gOpenArchives.front() != archive)

but eventually switched to checking the error code. Nothing else seemed to work for me.

Anyway, I'll change it and test again.

And I'll try to debug the assembler in VS to see what the deal is.

My dev machine went down yesterday though, and I'm just now getting the OS running. It will be a little bit before I can get back into the swing of things.

caeruleaus · April 26, 2010

it's crashing at this line in WorldModel::writeFile()

            for(uint32 i=0; i<groupModels.size() && result; ++i)
               result = groupModels[i].writeToFile(wf);

Lynx3d · April 26, 2010

Hm that's two lines in my world

Anyway, i don't see how anything can go wrong in those two lines, the cause must be somewhere else.

Only thing i could find is an accidetly use of "delete" instead of "delete[]".

I ran it through valgrind and now it has no complaints at all, also can't find any other place where i might write out of bounds and are only lucky that std::vector grew more in capacity than current size would require...

If it still doesn't work, then i don't know...

fabian · April 26, 2010

any news about the vmap_assembler.exe crash?

or can anyone upload the maps from linux extractor

Stab · April 26, 2010

Still getting crash in assebler.exe at Converting Pvp_Alterac_Ent01.wmo using VS90 Express

Could someone maybe post a Linux guide too? I'm basically lost in Linux systems beyond ./config;make;make install; =)

Sign In

[10167] vmap rewrite

Recommended Posts

Link to comment

Share on other sites

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Popular Days

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Recently Browsing 0 members

Contact Us

Repositories

Important Information