mangos to load scripting library as a module/plugin

[trapni]

Hey guys,

what i was always wondering about, is:

mangos comes with a default (pretty no-op) script library.

this library seems never to be used and just serves as an example for this interested in runnign mangos.

This actually leads to to case where though nearly everybody needs to "patch" the mangos to replace the statically linked script libary with their own, or the one they want, however.

Why is this done the ugly way?

What are the cons against loading this library dynamically and letting the configuration decide what plugin to load (and from where).

This way, it would come in handy for more ppl, not just sysads

I'd try to write the little chunks to load this piece dynamically, but I fear that my calls would just work on UNIX systems as i've no clue of win32 implementations (though, could provide empty #if's)

[Wyk3d]

Have you looked at the code ? The script library is loaded dynamically and can even be reloaded at runtime.

[trapni]

Have you looked at the code ? The script library is loaded dynamically and can even be reloaded at runtime.

ok, i found it, but still, you've to patch mangos build tree, and give it a fixed name instead of a configurable one.

[DasBlub]

why you need to patch mangos build tree?

have you ever heard from scriptdev2 ?

[trapni]

why you need to patch mangos build tree?

have you ever heard from scriptdev2 ?

i feel a little misunderstood, yes, i heard of sd2, and that's why i even thought about suggesting this - to make mangos a little more configurable.

sd2 is currently compiled as part of mangos, thus, the Makefiles have to be patched, using "git-am src/bindings/ScriptDev2/patchfile".

this is sooo hardcoded and ugly, and that's why I wanted to propose to provide some kind of configuration line, e.g.

LoadScript = /usr/libexec/mangos/ScriptDev2.so

which shall then be part of mangosd.conf.

Then SD2 shall be compilable from the outside world, and not from within mangos itself. This at least would make packaging mangos and sd2 as a pluggable part much more easier.

[DasBlub]

the patchfile doesn't change that.

you even don't need it on windows. you compile sd2 and the final library gets placed into the bin folder from mangos.

but yes, a dynamic path to the script library may really be nice... but i'm unsure if it is needed, it would only be a nice 'gadget'

[spermik]

We discussed it long long time ago, the scripting library is HARCODED and HARDLINKED (dynamically but is) to the core. The problem is that mangos does not have any plugin interface, so all "plugins" (like various chatlogs, bots, scripting libraries, etc,...) are hadroced to the core by patching the sources. So if you call UnloadScriptingModule() nothing will happen (im 100% sure on unix systems) because mangos-worldd is linked to that during linking process (g++ -l...). Try to delere that lib and ld will not allow mangos to start. Btw even if you will remove that link and open that library runtime you will have problems with all AIs spawned in game, those objects are allocated in the library and they will be freed during unloading, so all scripted creatures will have i_AI pointer to hell But everything is possible to rewrite

Best regards

Tabak

[jolan]

mangos does not need patching to use a scripting lib. the scripting lib is loaded at runtime using a fixed name in lib search patch (wich include mangos install dir on linux and windows) . SD2's patch is a build patch, to allow building mangos and sd2 in one step. it does not modify mangos itself. if SD2 were using "universal" dir instead of "ScriptDev2" i think it would not need patching to achieve one step building.

SD2 shall be compilable from the outside world

theoretically a lib could be built w/o mangos, even if this would be difficult, as it would have to includes lots of mangos headers. providing a mangos scriptlib dev package with required headers would ease that. but I don't think it worth the pain.

[Wyk3d]

So if you call UnloadScriptingModule() nothing will happen (im 100% sure on unix systems) because mangos-worldd is linked to that during linking process (g++ -l...). Try to delere that lib and ld will not allow mangos to start.

It's the other way around .. the scripting library is linked against the core (though i can only vouch for this on Windows). You cannot say that you are patching the sources when changing scripts because doing so does not actually change the mangosd executable (even if you rebuild with the common build system) so they are not part of its source. Technicly the library is not needed to run the application although there is a hardcoded check at startup and it is considered a fatal error (so the server exits) if it is not found. The library however is not required to actually contain any actual scripts and the default library included in the source tree does not. After you have built the core, you can rebuild just the scripts any number of times without the rest and it does work (or rather you build everything but if you only changed the scripts then mangosd did not change so only the scripts get built). You can also switch between multiple scripting libraries without having to rebuild mangosd, as long as they were all built against the same core. It is only as minor incovenience to have to rename them.

Btw even if you will remove that link and open that library runtime you will have problems with all AIs spawned in game, those objects are allocated in the library and they will be freed during unloading, so all scripted creatures will have i_AI pointer to hell But everything is possible to rewrite

Yes that would prevent you from reloading it at runtime however it could be fixed by looping all creatures and removing their AIs and adding them back after the library is loaded again, doesn't need any rewriting. So the implementation may have some bugs and missing features which vary in degree with the platform used but overall the library is not hardlinked or hardwired or whatever.

[VladimirMangos]

Have you looked at the code ? The script library is loaded dynamically and can even be reloaded at runtime.

More correctly: can even be relaoded in past. After adding AI classes this motly impossible without crash.

AI classes allocated at script DLL side and can't be correctly recreated at scriptlib reloading, and more: it still pointed in Crearture data.

[added]Ah, already remarked.

Yes that would prevent you from reloading it at runtime however it could be fixed by looping all creatures and removing their AIs and adding them back after the library is loaded again, doesn't need any rewriting. So the implementation may have some bugs and missing features which vary in degree with the platform used but overall the library is not hardlinked or hardwired or whatever.

At reloading we also lost AI state and this in some cases important like scripted instances...

[spermik]

I have already investigated scripts reloading in the past, so despawning all scripted objects in game is a solution for safe library unload.

You are right that the SD2 patch does not modify the mangos sources, but it is linked to mangos binary by ld, so without modifying this build process the library will not be freed and reloaded - dlopen and dlclose will not work as it should (im still talning about *nix system, dunno how it works under win).

So if you wanna reloadable scripts, you will have to :

1) change the linking so mangos will not be linked to any scripting lib

2) make the load and unload safe

3) and also excluding that lib from mangos build process whould be great

Btw creating some mangos module/plugin interface whould be great to, because many ppl develop and distribute their modules as source patches, and thats not the right way.

Tabak

[VladimirMangos]

o without modifying this build process the library will not be freed and reloaded - dlopen and dlclose will not work as it should (im still talning about *nix system, dunno how it works under win).

This is _not_true_, i personally reload scripting lib in past and more recently and if AI objects not created this is not problems. script lib reloaded and replaced.

Exist some specific for Windows/Unix. In windows case you need reload with new name script lib used that allowed by command. Ofc, maybe some additiona code required for current used numbering for scripts name schame...

[spermik]

Vladimir, you obviously dont belive me. So lets make a little proof. (Im still talking about mangos under linux, I dont understand win stuff). So lets open some SD2 cpp file (for example ScriptMgr.cpp) and add this:

class Test
{
public:
 Test()
 {
   printf("TEST CREATED\\n");
 }

 ~Test()
 {
   printf("TEST DESTROYED\\n");
 }
};

Test t;

So when out library is initialized the Test t shoud be created and message should appead in stdout. So lets recompile and run mangos.

1st strange thing: the message "TEST CREATED" appear in output immidiatly after starting the binary and NOT when the dlopen is called later in boot process (thats right because it is HARD linked to mangos and that dlopen practicaly does nothing).

2nd stragne thing: use loadscripts to reload that library. Both messages should appear (DESTROYED and CREATED) but neither does. (again the same reason, the library is NOT freed and loaded again, dlclose practicaly does nothing in this case).

3rd strange thing: the DESTROYED message is printed when the server is shutting down when all linked libraries are freed.(the same reason)

So if you will use loadscripts to load another library the first will remain loaded, because it was linked to the core just all other libraries (ZThread, ACE). And that is what Im trying to explain here

Best regards

Tabak

[VladimirMangos]

Ok.Maybe i wrong or something chnage from times when i use reload scripts. Any way currently impossinle this do in result existance AI objects allocated in script DLL

[trapni]

Ok.Maybe i wrong or something chnage from times when i use reload scripts. Any way currently impossinle this do in result existance AI objects allocated in script DLL

it has never been the way you described it, unless someone can proof it.

I would wonder if windows would do it differently.

Although, I'd highly appreciate a _dynamic_ load/unload (ignoring the reload with state-keeping for now).

This shall be possible by designing _clean_ load() and unload() hooks the dynamically script library is hooked into (and implements properly himself).

The unload shall get some special care, because any reference from withing a library to be unloaded may result into funny results after having unloaded the libarary (on access) - this shall also apply for windows, I assume.

( But again, valgrind can help there

From a UNIX library's API point-of-view, a library is completely open, unless compiled with -fvisibility=hidden which would require to declare anything in the public API to be __attribute__((visible)) - think of a nice macro here, that makes it fine for windows (declspec) and GCC/UNIX.

Regards.

[Wyk3d]

I would wonder if windows would do it differently.

It does it as I said above with the library being properly loaded dynamically, only when LoadScriptingModule is called. You also have the possibility of building SD2 from its own separate solution on windows.

[Patman128]

This is one instance where windows has unix beat. On windows, the scripting library is compiled separately and is swappable with any other compatible library after build. On unix, it is linked at compile time and you are stuck with whatever library you built mangos with. Maybe the project should work towards having a universally-dynamic linked library for scripts.

EDIT: Tested what you wrote, and on windows it doesn't display TEST CREATED until the part where it loads MangoScript.dll.

[Wyk3d]

This is one instance where windows has unix beat.

It is probably just a minor error made by those who worked on the build system .

[Patman128]

It is probably just a minor error made by those who worked on the build system .

Any fixes in the near future?