[7428] New .map file format and some fixes

[Auntie Mangos]

Is it possible to do this with vmaps too ?

this patch produces smaller vmaps (even if it's not the main goal).

I still use it with a 2.4.3 branch.

[Auntie Mangos]

Good job DiSlord, you earned some extra respect points from all of us.

[Auntie Mangos]

Guys i have a problem when trying to compile the extractor_v2 on linux, this is what i got:

[ 7%] Building CXX object loadlib/CMakeFiles/loadlib.dir/loadlib.cpp.o

/root/mangos-old/extractor_v2/loadlib/loadlib.cpp:8:27: error: ..\\mpq_libmpq.h: No such file or directory

/root/mangos-old/extractor_v2/loadlib/loadlib.cpp:54:25: warning: multi-character character constant

In file included from /root/mangos-old/extractor_v2/loadlib/loadlib.cpp:7:

/root/mangos-old/extractor_v2/loadlib/loadlib.h:4: error: â__int64 does not name a type

/root/mangos-old/extractor_v2/loadlib/loadlib.h:8: error: â__int64 does not name a type

/root/mangos-old/extractor_v2/loadlib/loadlib.cpp: In member function âbool FileLoader::loadFile(char*)â:

/root/mangos-old/extractor_v2/loadlib/loadlib.cpp:27: error: variable âMPQFile mf has initializer but incomplete type

/root/mangos-old/extractor_v2/loadlib/loadlib.cpp:30: error: âprintf was not declared in this scope

/root/mangos-old/extractor_v2/loadlib/loadlib.cpp:44: error: âprintf was not declared in this scope

make[2]: *** [loadlib/CMakeFiles/loadlib.dir/loadlib.cpp.o] Error 1

make[1]: *** [loadlib/CMakeFiles/loadlib.dir/all] Error 2

make: *** [all] Error 2

i corrected that line by switching the "\\" for "/", but after that i got this:

[ 7%] Building CXX object loadlib/CMakeFiles/loadlib.dir/loadlib.cpp.o

/root/mangos-old/extractor_v2/loadlib/loadlib.cpp:54:25: warning: multi-character character constant

In file included from /root/mangos-old/extractor_v2/loadlib/loadlib.cpp:7:

/root/mangos-old/extractor_v2/loadlib/loadlib.h:4: error: â__int64 does not name a type

/root/mangos-old/extractor_v2/loadlib/loadlib.h:8: error: â__int64 does not name a type

In file included from /root/mangos-old/extractor_v2/loadlib/loadlib.cpp:8:

/root/mangos-old/extractor_v2/loadlib/../mpq_libmpq.h:16: error: â__int64 does not name a type

/root/mangos-old/extractor_v2/loadlib/../mpq_libmpq.h:20: error: â__int64 does not name a type

make[2]: *** [loadlib/CMakeFiles/loadlib.dir/loadlib.cpp.o] Error 1

make[1]: *** [loadlib/CMakeFiles/loadlib.dir/all] Error 2

make: *** [all] Error 2

my Machine does not have 64bit, it is a regular pc. I'm trying to compile the ad.exe because even after a fresh installation of the mangos server i'm having the error:

Map file '/opt/mangos/Data/maps/0004331.map' is non-compatible version (outdated?). Please, create new using ad.exe program.

Correct *.map files not found in path '/opt/mangos/Data/maps' or *.vmap/*vmdir files in '/opt/mangos/Data/vmaps'. Please place *.map/*.vmap/*.vmdir files in appropriate directories or correct the DataDir value in the mangosd.conf file.

In which i read i need to run a new ad.exe, but even after downloading an extractor_v2 file which contains a procompiled version of it my server wont start up because of that outdated thing.

And also tried applying the maps.patch as i read on post #67 of this threat.

Ohh i forgot to mention that i extracted the maps from my client version 3.0.9 (9551) (Release), Feb 3 2009

and i use the ad.exe which extract the maps and size of them ~220MB.

So, can you guys help me on this matter.

[Auntie Mangos]

do you have a link for the latest resource then, cause to be honest i have been reading and reading and reading forums looking for a solution on my Map file '/opt/mangos/Data/maps/0004331.map' is non-compatible version (outdated?). Please, create new using ad.exe program.

Correct *.map files not found in path '/opt/mangos/Data/maps' or *.vmap/*vmdir files in '/opt/mangos/Data/vmaps'. Please place *.map/*.vmap/*.vmdir files in appropriate directories or correct the DataDir value in the mangosd.conf file. error thing

Dont know what am i doing wrong.

For example, this is what i did:

i did the installation guide from http://getmangos.eu/community/viewtopic.php?id=4011.

i downloaded the DB files from http://unifieddb.svn.sourceforge.net/svnroot/unifieddb/trunk/Full_DB/ using first the Clean install, and the running the upgrade to core 7069

i update DB's using the https://unifieddb.svn.sourceforge.net/svnroot/unifieddb/trunk/Updates/0.11.0_additions/

i ran all those, then i used the ad.exe from the files above to extract the maps from my client 3.0.9.9551 copied the maps and dbc files into my /opt/Data directory

and still have the same error.

[Auntie Mangos]

Well Vladimir, i found that ad.exe on the post #67 of this threat.

Also i modified the code trying to make it work to get a newest version compiled for me. That doesn't mean that the ad.exe got a modified code on it. It came on the file i downloaded and also that files contains the source code which i tried to compile.

Vladi, can you send me your 0004331.map file to see if yours will allow my server to run.

[Auntie Mangos]

Well Vladimir, i found that ad.exe on the post #67 of this threat.

Also i modified the code trying to make it work to get a newest version compiled for me. That doesn't mean that the ad.exe got a modified code on it. It came on the file i downloaded and also that files contains the source code which i tried to compile.

Vladi, can you send me your 0004331.map file to see if yours will allow my server to run.

Then you're using the wrong Ad.exe, the one at post 67 was for testing, its not valid anymore, the correct one is in your source when you clone from git. contrib/extractor i believe.

He cannot send you maps/vmaps, that is ILLEGAL

[DiSlord]

Continue work under .map files:

How possible limit .map size? Liquids level stored as 128x128 float array in every griid, but in most cases liquids have constant level and need just know it, or if no liquids in greed - not need store it (its also for area data and height data). I also add some improvements in .map file format.

Now .map size 619 Mb (1 Gb before) (on load in memory maps also need only 619 Mb). And code should be faster as before.

I also rewrite .adt extractor, and don`t have any error on extract maps (also i hope fixed water damage as lava).

Download http://depositfiles.com/files/3ebnha1gf

[added]

Next try pack also height data (lost some accuracy but get better file size)

now all maps req 290 Mb ram.

Now height stored as uint8/uint16/float - depend from max grid heigh diff (in 40% use uint8 ).

+ less maps size

- lost some accuracy in some places (but not more than 0.01)

http://depositfiles.com/ru/files/8gxr54z34

[added]

Not store height data in deep ochean (dark water), limit it to -230. Its allow save some memory for maps.

Also use better float->int conversion and get better accuracy (max mistake = 0.002 by my tests)

Now all maps size = 217 Mb

DOWNLOAD

[The_Game_Master]

OMG! I bow to you! Very very impressive!

Can you please explain more when you'll have the time.

Couln't the same optimizations be used for vmaps?

[DiSlord]

Example in greed water:

~ water

# earth

1 )

~~~~~~~~

~~~~~~~~

~~~~~~~~

~~~~~~~~

~~~~~~~~

~~~~~~~~

~~~~~~~~

~~~~~~~~

Add have constant level 0.0f - example ochean.

Store only uint8 liquidType, and float liquidLevel = 0.0f - only 5 bytes (except 64 kb before).

2)

~~~#####

~~~~####

~~~~~###

~~~~~~##

~~~~~~~~

~~~~~~~~

~~~~~~~~

~~~~~~~~

And constant level - store only matrix uint8 16x16 liquid type and float level - 260 bytes

3)

#####~~#

####~~##

###~~###

#~~~~~~#

#~~~~~~#

########

########

And not constant level (waterfall for example)

Select only water area (xoffs, y offs, width, height)

####~~

###~~#

##~~##

~~~~~~

~~~~~~

Store uint8 16x16 liquid type and width*height liquid height data < 64 kb

4)

########

########

########

########

########

########

########

########

No liquids - nothing store - 0 bytes

[karol]

this sound like revolution such as Vmaps!!

great

[crashuncle]

This makes sense, I think it will also improve the performance not only save space and theoretically it also need less RAM?

[henhouse]

The recent one fixed Shadowstep, now maybe this should fix all the others?

[Ebrithil]

Beautiful work DiSlord!!

Keep rockin'

[The_Game_Master]

Great example! Thank you. I think now it's posibble to implement exhaustion. If the sea have any special extra flags from the rest of the water.

[Trainzfreak]

nice work, i have added this to my core

this works not under windows 7

[Sephiroth1983]

nice work, i have added this to my core

this works not under windows 7

You are talking about an operating system which currently is still in beta.

[crackm]

Good work indeed but terrain is sometimes unpredictable since you do not see creatures some times.

[DiSlord]

Update patch

Fixed:

- Memory leak

- Not closed map file in some cases

- Speedup (now as fast as possible) get liquid data

- Extract map by use WDT files (contain grid lists)

- Store dark water data from MH2O to .map file

- Fixed wrong asrea store in some cases

http://depositfiles.com/files/3ebnha1gf

PS need reextract maps

[Darkpeninsula]

ty DiSlord.

[higi]

excelent, gonna test)

[amrok]

graveyards arent working since i've added this maps to my core....

[DiSlord]

graveyards arent working since i've added this maps to my core....

Its not this patch problem i think (test it on clean mangos and get some results)

Need fill game_graveyard_zone table

[xsopher]

I am using the new map format everything works great thanks Dislord.

[amrok]

Its not this patch problem i think (test it on clean mangos and get some results)

Need fill game_graveyard_zone table

but my game_graveyard_zone table is full ive never got problems with it... im using newest release of ytdb

[xsopher]

are you using vmaps with maps or just maps?

dunno why it would make a diff just wondering i run vmaps also on a udb db