Jump to content

[10167] vmap rewrite


Auntie Mangos

Recommended Posts

  • Replies 298
  • Created
  • Last Reply

Top Posters In This Topic

I compiled and applied this mod, as mentioned in first post. went alright. extracted maps, dbc, and vmaps.

Now when I am ingame, everything looks fine. But when I enter a cave / building / anything with a roof the NPCs / Mobs fly in the air.

I'm not sure if it is a problem with the vmaps, or with a patch I applied. I encountered the "flying attack" bug and changed the code as mentioned in another thread (Thread | Post). Well, could be both. any help would be appreciated.

Also I am trying to get mmaps running. I know, not the right thread, but anyone got the generator working with Linux?

Link to comment
Share on other sites

  • 40 years later...

Disclaimer:

Current status of this project is "working, but needs more testing". Use at own risk, although feedback so far has been mostly positive.

You probably wonder why one would rewrite the vmap system.

I'll try to keep it brief, feel free to ask for more detail...

It basically started with arrai starting to implement indoor/outdoor information for vmaps, and me trying to figure out how area information stored in WMO works.

While reding the vmap code I came to the conclusion that there's enough rough edges to try a different approach.

Basic things i did not like:

  • * all objects are saved as a transformed copy, which makes Vmaps very big
    * no proper mesh format, each triangle defines new corner points, makes Vmaps even bigger
    * to counter this, coordinate precision was severely reduced (16bit, fixed range) which made deeprun tram impossible even

Also several things were done quite confusingly or inefficiently, making the decision to rewrite even easier.

Since the models are stored with transformed coordinates, it was more difficult to check the WMO bounds that delimit the information we were trying to query.

This lead me to this approach:

  • * keep models stored in original coordinates, transform rays at intersection instead
    * to be able to intersect the world, assembler creates a static tree for the entire map
    * instead of the AABSPTree i chose a Bounding Interval Hierarchy (BIH) which basically is much better at "cutting off" empty space without having to reference models in multiple tree nodes
    * Since models are re-used all over the game world, model allocations are handled globally

All in all, this reduced Vmap size to less than one third, and should improve precision. However, there's much more files now, each model, each tile ("grid") and each map needs one file.

I can't really say anything about performance, but i would be very surprised if it's slower than the old code. If anyone can setup some benchmark, i'd love to see some numbers...

New features thanks to arrai (and indirectly tom_rus):

  • * indoor/outdoor check which allows automatic unmounting of players
    * additional area information from WMOAreaTable.dbc, removed existing "hacks"
    * WMO Liquid information for swimming and fishing correctly in buildings/cities/caves/instances

Now that you know what this is about, you may get the code from:

http://github.com/Lynx3d/mangos/tree/vmap_rewrite

How to use (linux)

First, you need to compile libmpq in dep/libmpq the "traditional" way:

First './autogen.sh' then './configure' followed by 'make'.

Then you can compile vmap_extractor_v2 and vmap_assembler with Cmake like you do with extractor (see contrib/extractor/README.linux if you never did that).

The rest should work like it did before, i.e. run vmapextractor in WoW base folder, create <destination dir> and run 'vmap_assembler Buildings <destination dir>'. Should take ~5-10min each.

Finally, copy/move the <destination dir> to your data dir and name it 'vmaps' (if not already named like that anyway). 'Buildings' can be deleted afterwards.

Update:

How to use (windows)

Before extracting and assembling, you need to build the new vmap_extractor and vmap_assembler binaries, the precompiled ones are not yet updated. But usage should still be the same...I think.

First, you need to build the solution in dep/libmpq/win in the same configuration you want to build the extractor (release or debug)

Then, you can build the solutions for vmap_extractor_v2 and vmap_assembler.

Windows build setup thanks to faramir118, you may still check his repository for new improvements to build/usage (use at own risk...well...not that mine would give you any guarantees):

http://github.com/faramir118/mangos/tree/vmap_rewrite

General Info:

To enable indoor check for automatic unmounting, set "vmap.enableIndoorCheck" in mangosd.conf to 1.

The .gps command also prints a few additional infos to check if area detection works as intended.

Link to comment
Share on other sites

libmpq is not compatible with windows!

Great to see you actually read his post.

I switched out StormLib for libmpq (the former just didn't want to work on 64bit linux...) in the belief it is cross platform, but now i noticed there's no build setup for windows available.

Anyway, amazing job Lynx.

Link to comment
Share on other sites

It looks to me like the code in contrib/extractor/libmpq is already cross-platform. Why not use that?

Here's an alternative:

Built using Eclipse CDT, MinGW 5.1.6, and MSYS 1.0.10 for Win32:

http://rapidshare.com/files/377488543/libmpq-win32.zip.html

The whole eclipse project is in the zip, along with a Win32-Release binary.

I had to make similar changes as the cross-platform version in contrib/extractor/libmpq, because I wasn't able to get config.h.

//#include "config.h"

//const char *libmpq__version(void) {
//
//	/* return version information. */
//	return VERSION;
//}

Link to comment
Share on other sites

Thanks for your effort faramir118.

I should mention that i did not touch the project files for vmap_extractor/assembler yet, maybe they even have been/can be created with cmake anyway, i don't know.

It looks to me like the code in contrib/extractor/libmpq is already cross-platform. Why not use that?

Because no one can tell me how old it is, where it even comes from (.cpp files??) or if its even save to use on 64bit systems, so i have to hope it doesn't wreak havoc each time i run it ;)

Also, current libmpq just makes code nicer because it encapsulates internal stuff better, so i'd really like to upgrade libmpq for both tools.

I had to make similar changes as the cross-platform version in contrib/extractor/libmpq, because I wasn't able to get config.h.

Hm isn't it possible to run the configure script in MSYS?

I would prefer if people not wildly hack inside library code this time...

There's not really anything in my config.h that looks like it would be impossible to make a "generic" one that works for MSVC or MinGW:

http://paste2.org/p/781286

Link to comment
Share on other sites

I started with Cygwin.

./autogen.sh went smoothly

./configure throws error autom4te: invalid parameter '--trace=', along with a bunch of suggestions/warnings.

I got stuck there, not experienced enough with *nix build tools.

I didn't bother trying with MSYS after that, just figured it was hopeless. I'll try it tomorrow and let you know how it goes.

Link to comment
Share on other sites

Hm i just installed MinGW and MSYS on my notebook (only computer with windows here...) and after tediously downloading and unpacking all the components (luckily there's ready zlib and bz2 for mingw), autogen.sh, configure and make worked and produced a libmpq.a (~480kb).

Guess it's statically linked to zlib+bz2, so would that one file be enough to compile the extractor with MSVC?

Link to comment
Share on other sites

Guess it's statically linked to zlib+bz2, so would that one file be enough to compile the extractor with MSVC?

I got the lib compiled with MSYS. It does appear as if it contains the bz2 and zlib objects (open libmpq.a with a text editor and search for function names from bz2 or zlib). However, my libmpq.a is only 35.1kB

I hacked around a while and finally got Visual Studio to attempt linking libmpq into vmap_extractor_v2, but I get:

libmpq.lib(mpq.o) : error LNK2019: unresolved external symbol _fseeko referenced in function _libmpq__archive_open

I went back and looked in MinGW's stdio.h, and all it has is fseeko64

With all of these problems, something in libmpq is going to have to be changed at some point.

Link to comment
Share on other sites

Quote from the libmpq.org site regarding compiling libmpq 0.4.2:

You need zlib >= 1.1.3 for compiling libmpq.

You need bzip2 >= 1.0.4 for compiling libmpq.

Also, the wowmodelviewer uses libmpq. There's a vcproj file listed in their source directory for libmpq at '../trunk/src/libmpq'. Check it out at Google Code.

Maybe that can shed some light on working with VC++ and libmpq?

There is a Windows native MPQ wrapper for Stormlib, written in C++ and .NET 3.5, named MpqLib.

Binary and sources are available, under the GPL, at the MagosX forums. (That's not a misspelled. lol)

Link to comment
Share on other sites

Also, the wowmodelviewer uses libmpq. There's a vcproj file listed in their source directory for libmpq at '../trunk/src/libmpq'. Check it out at Google Code.

Which seems to pretty much the same god-knows-why-C++ derivate from who knows of unknown age that we are using too ^^

If you're fine with that, i'll just make a compile option to use that or official libmpq...

faramir could've told me the headers of libmpq are not MSVC compatible in the first place, so it's not only the build setup.

Though i do wonder why there options in config.h to specify if stdint.h etc. exist, when they still get unconditionally included :P

Anyway, can someone confirm that at least core and vmap_assembler now build with VS 9?

And come on, is there not a single linux user interested in this?

Link to comment
Share on other sites

Sorry, I guess I could have told you earlier that VS90 didn't like libmpq includes.

There are three options:

* use contrib/extractor/libmpq

This library is ported to C++

This library seems to be a slightly modified version of rev5 of libmpq

* use wowmodelviewer/src/libmpq

This library is ported to C++

This library is heavily modified

* attempt to make libmpq compatible with Windows on our own

I'll try number 3, but don't count on me. If I do make progress, I promise it won't be super hacky!

I'll probably also make some diffs to see what the differences between the three are.

Link to comment
Share on other sites

Here's another example of a libmpq vcproj file for VC++ 9.0, posted on Pastebin very recently.... http://pastebin.com/sFs7u05n

Were you aware someone else has also tried rewriting the map extractor, ad.exe, with an attempt to update libmpq for MaNGOS and Trinity? Plans for rewriting vmap extractor are forthcoming.

The project is called madex. You can find it here... http://bitbucket.org/imbecile/madex/src

Quote from the project's readme:

This is a rewrite of the old mangos map_extractor to use the latest LibMPQ available from libmpq.org.

Hope all this info helps you guys. I admire the brilliance and craftsmanship of all you coders here. Wish I knew how to do this, but that's for another day.

Link to comment
Share on other sites

UnkleNuke, i really don't think it is possible to compile libmpq with MSVC without some code changes...at least not with ver. 8 and 9, because they don't have all C99 headers required.

For now i went with suggestion 1) of faramir. If your computer didn't explode in the past when extracting maps/dbc, it should work. But i can't test, there's really no way to make room for WoW on my notebook.

I don't think it's a good idea using some untested "quick shot" patch for libmpq just to get current version working, but if someone wants to try a serious porting attempt, please also suggest at libmpq.org.

Link to comment
Share on other sites

I have not yet tried madex, Thyros, but I kept it bookmarked for future reference. Oddly enough, it turned out to be a useful thing, my being a code packrat (which is made more odd by the fact that I have yet to open up my editor/compiler and really learn some of this gobbledygook.).

I also found a reference in some Starcraft 2 forum. There was a guy who claimed he got libmpq to compile with VS 2008, after making heavy modifications, but he has not posted any examples or code for his private project.

The other data I provided is just for examples to help give you ideas for possible solutions to this thorny problem.

I use Windows as my server platform and VC++ 9.0 Express SP1 for compiling, so I'd be happy to test anything you have to offer with my MaNGOS core. Please bear in mind I am not a developer, so it would be ideal if you can provide some simple instruction for compiling and testing

Link to comment
Share on other sites

Ok, try this: http://github.com/faramir118/mangos/tree/vmap_rewrite

Amazingly, I didn't have to change a single line of code to get it to work. All I did was work with the includes.

Changes:

  • * Added bzip2 src to contrib/libmpq/dep/bzip2
    * Added zlib src to contrib/libmpq/dep/zlib
    * Added test.cpp to contrib/libmpq/libmpq
    Tests mpq read and extraction.
    There's no error checking, it just assumes that you give it a valid filename (and optionally a file index)
    * Added VS90 solution to contrib/libmpq/win

Windows binaries (libmpq.lib, test.exe) goes to contrib/libmpq/bin

Happy testing!

edit: forgot to mention that I didn't edit the debug configurations for VS, so they won't compile. Make sure you're compiling for Release.

edit: fix path typos

Link to comment
Share on other sites

faramir118, i just saw, that you've done a git-pull from mangos/master to merge the newest updates. this causes, that not all your commits are on top now. may i suggest you to use this command instead?

git pull --rebase upstream master

(replace "upstream master" by your name for the mangos repo). the "--rebase" will call "git rebase" instead of "git merge" and thus rebase all your own commits to the HEAD of the repo.

this will also make it easier for you to keep track of your commits :)

Link to comment
Share on other sites

Ok, try this: http://github.com/faramir118/mangos/tree/vmap_rewrite

Amazingly, I didn't have to change a single line of code to get it to work. All I did was work with the includes.

Changes:

  • * Added bzip2 src to contrib/libmpq/dep/zlib
    * Added zlib src to contrib/libmpq/dep/bzip2
    * Added test.cpp to contrib/libmpq/libmpq
    Tests mpq read and extraction.
    There's no error checking, it just assumes that you give it a valid filename (and optionally a file index)
    * Added VS90 solution to contrib/libmpq/win

Windows binaries (libmpq.lib, test.exe) goes to contrib/libmpq/bin

Happy testing!

edit: forgot to mention that I didn't edit the debug configurations for VS, so they won't compile. Make sure you're compiling for Release.

The compilem works fine now, now when the extractor startet then comes an error that he can not open the enGB file.

Okay i have edited so i delete the different language and now stops the extract by Map - ChamberofAspectRed.

Link to comment
Share on other sites

The compilem works fine now, now when the extractor startet then comes an error that he can not open the enGB file.

Okay i have edited so i delete the different language and now stops the extract by Map - ChamberofAspectRed.

That file can be bugged or have different version, because it's not finished zone for patch 3.3.5.

Link to comment
Share on other sites

Guest
This topic is now closed to further replies.
×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue. Privacy Policy Terms of Use