[Mangos Zero] Stuck Installing Release 21 - Asset Extraction

[publiccert]

Hi all,

First time poster. I'm working on setting up a local install on Ubuntu 14.04. I cloned the repo and all that good stuff. I got to step #5. The instructions seem to trail off around here; cd ~/server/contrib/extractor

I can't find that directory and am a little lost at this point. Thanks!

[publiccert]

Thanks for your help! I'll give this a shot. I'm planning on running the server on Ubuntu but doing most of my work on a Mac. I believe I have access to Windows so I'll see if that makes a difference.

Edit: Will the extraction process work with a Mac version of the game client? I could get another version but thought I'd ask since its what I have available. Thanks!

[Olion]

Edit: Will the extraction process work with a Mac version of the game client? I could get another version but thought I'd ask since its what I have available. Thanks!

In theory, they will. The only thing I'm unsure of, is the extractor dependence on the wow.exe binary. It is artificial and, as far as I understand, was introduced to track actual client version (build). There was a later change of extractor regarding MAC client, so it is at least worth to be tested.

[publiccert]

Olion, thanks for your help. I started setting up Windows and VS but realized you had suggested a simpler method. I acquired a Windows version of the game client, went into (being verbose in case someone else has this problem) ~/zero/bin/tools and copied everything to /path/to/wow as you suggested. The extraction is now running.

Is the extraction tool intentionally limited to 4 cores? I'm running this on a 40 core server with 512Gb RAM and would like to take advantage of that.

[Olion]

Is the extraction tool intentionally limited to 4 cores? I'm running this on a 40 core server with 512Gb RAM and would like to take advantage of that.

All extractors (AFAIK) are single-threaded. The extremely time-consuming operation is mmap extraction. It uses 4 cores because the script starts 4 extractor copies, each with appropriate list of maps. Anyone can reformat that list for different number of the extractor copies running in parallel.

P.S. However, the most time-consuming Kalimdor and Eastern Kingdom maps will limit the extraction step still, so you can barely get any advantage. You'll get it later while running the server; the data extraction step must be redone very rarely.

[H0zen]

The movemap-generator has an option called --threads. To take advantage of your multicore, use ./movemap-generator --threads 40 For this, don't use the supplied script. Just use the movemap-generator alone

[Olion]

The movemap-generator has an option called --threads.

Great, was not aware of. But the gain will not be huge still because of the mentioned single thread per map scheme. So the whole process will take time of the longest map extraction, for map IDs 0 and 1.

[publiccert]

Thanks for the responses everyone! Ran the process with the 4 cores. Took about 20 minutes. I gather from other research that I won't have to do this again so its not too awful. Thanks again for all your help.

[H0zen]

[MENTION=5581]Olion[/MENTION] - Indeed, a map per thread is not an optimal solution. That's the reason I tried a tile per thread one. Merge pull request #9 from Hzen/master * mangos/Extractor_projects@6df8d7a * GitHub

[Olion]

That's the reason I tried a tile per thread one.

Thanks, I have totally missed that. I suppose being not alone with it.. It is a sufficient reason for changing the mmap extraction script to use this functionality.

[antz]

There is a valid reason why the mmap extractor was limited to 4 cores....

As it stands, where processing under 4 cores, the split is this:

cpu 1: Eastern Kingdoms

cpu 2: Kalimdor

cpu 3: A mix of large, medium and small maps

cpu 4: A mix of large, medium and small maps

Kalimdor is the single biggest map, followed by Eastern Kingdoms.

All the other maps put together are smaller than the Eastern Kingdoms.

Even if you run it across 8 cores, it would be no faster than the time taken on cpu 2

[Olion]

At this step, you must have the compiled binaries in your install ($CMAKE_INSTALL_PREFIX) dir. Under that directory, find subdirectory tools/.

To get further, you need the game client files accessible by your system. (Only MPQ and wow.exe files are needed, providing the client directory structure kept.) The simplest way is to copy all files from tools/ to the client root folder. Then run

sh ExtractResources.sh

and follow on-screen instructions. For a start, you would like to postpone mmap extraction (providing pathfinding scheme) that usually takes several hours. If so, answer No to the very first question and disable mmap extraction (dbc and maps are absolutely needed, vmaps are optional but extracted rapidly), then do not forget to disable pathfinding in the mangosd.conf:

mmap.enabled                      = 0

The generated data (dbc/, maps/, vmaps/, optionally mmaps/) may be placed anywhere providing DataDir mangosd config parameter pointing to there. The default DataDir setting "." means you have these directories in the server bin/ dir, along with mangosd and other stuff.

Another option is repeating the whole build process at a Windows system where you have the client and do the extraction there. Then the files need to be transferred to your linux server, meaning up to 900MB for dbc+maps+vmaps and up to 800MB for mmaps.

Having the data extracted and placed properly, go to the 6th step of the Install guide.