AI specifying, loading and packaging/storing

[hoijui]

follow up to a discussion that started here:
viewtopic.php?f=15&t=15019&start=100

should we package AIs in archives, like the mods?
pro:
* configuration specification files can be supplied
* additional libraries
(* lower file size)
and more
contra:
* lots of work

alternatives?

[Agon]

Additional you can add for pro: Lobbys can easier recognize if an AI is a native, managed or a jar file/dll. And can choose the right interface.
Also with a archives we could use the "Archivemover" to move an AI archive to its right place like with maps and mods.

It is true that this requires some work but the futures we would gain are not bad for AI developers and end user.

[AF]

The problem here is that configuration files are usually written to by the AIs anyway so putting them in the archive wouldn't make much sense.

However if archive mover could see to it that the necessary config files are copied into the real file system that would be fixed.

I also think if your serious about this we should go for archive names instead of AI names in the script file the lobby writes.

[AF]

hmm no, because then we'd have no way of debugging AIs using script files without forcing the compiler to auto package the AIs which doesnt sound very nice. Changing the code at runtime in visual studio would also be problematic among other issues too.

OK Im getting confused, is this for storing AIs or for installing AIs?

[hoijui]

@AF
aehm...
as i understood it, the end user will not have DLLs, but archive files in Bot-libs (that was the idea). not that the AIs are shipped in archives which the user extracts to his Bot-libs folder.

i think we have 2 different configurations here. one is static, the otherone dynamic (somehow). in the static one, the AI exports information to the engine or the lobby, which can not be changed. eg, the interface DLL and version required. then there are dynamic properties, like AI difficulty or if the AI should cheat. static ones would fit well into a config file inside an AI archive, the dynamic ones definatly not.
i still think it would be much easier to have a folder with all DLLs that can be used directly, eg Interface-libs (which contains Javainterface.dll, CSharpInterface.dll, and possibly AAI.dll, NTai.dll, ...). and have the other, the real AI implementations in Bot-libs eg.
then the engine or unitsync would call the GetPossibleNames() method on all DLLs in Interface-libs, and in script.txt we would have a tag for the interfaceDLL and one for the bot-lib. it is flexible, simple, needs little change.

dynamic configuration can still be added later. eg setup a standard like AAI.dll has the config file specification in AI/Global/AAI/configSpec.txt and the ocnfig file in AI/Global/AAI/config.txt with a standard like for mod configurations. this makes it possible for lobbies to supply a GUI for AI configs, and also makes it easy to config them by hand, editing the config files.

[AF]

NTai has learning files it updates, these are text files on the hard drive, AAI has build tables, these are all now stuck inside zip or 7zip files. Do you all not see the problem here? AIs can't write to their config files anymore.

I also object to having to read in and parse text files with the lobby. Too complicated.

[AF]

May I point out that lobby interfaces and AI interface do nto use the same paradigm.

AI interface redesign the goal is flexibility and future proofing.

Lobby developers tend to be rare, overworked, and with very long to do lists. Make it simple and extremely easy to implement in as small a time as possible.

As a lobby developer I would rather completely drop AI support for 4-5 weeks than have to deal with calling dlls or parsing text files and other headaches. These sound more like hacks and kludges to me.

Why don't you have ntai.dll and ntai.lua and make the engine search look inside the lua file and look at that for the details rather than asking an external project to remind it what its supposed to be doing.

[hoijui]

NTai has learning files it updates, these are text files on the hard drive, AAI has build tables, these are all now stuck inside zip or 7zip files. Do you all not see the problem here? AIs can't write to their config files anymore.

even if we use archives for the AIs, to store everything static in there, like a DLL or jar, and possibly additional libraries, and possibly a static configuration or configuration speciification, all dynamic stuff, like the configuration or the cache files would have to be outside the archive, in the normal file system. i never though on saving that in the archive, not to say that i think we really need archives

I also object to having to read in and parse text files with the lobby. Too complicated.

for dynamic configurations, the system wcould be the smae like for the mod configurations. though as said already, that can be done at a later state, if there is a real need for configurable AIs (difficulty, cheating, ...).

Lobby developers tend to be rare, overworked, and with very long to do lists. Make it simple and extremely easy to implement in as small a time as possible.

My solution is the closest one we have to that, so far.

The main change would be, that the lobby does not scan a dir for DLLs, but ask the engine for available AIs, which shifts future changes to the engine (a singe place, not multiple places (the lobbies) currently). Then the engine would have to deal with finding out which interface DLL to use if any, and which directories to scan for which files and so on.

it is not possible that the lobbies will end up with no change at all, when we want to support more languages. calling a function on unitsync instead of scanning a directory and possibly write a tag with an other name (eg ainame instead of aidll)... thats a 5 minutes change, which frees you of future changes for AIs.

Why don't you have ntai.dll and ntai.lua and make the engine search look inside the lua file and look at that for the details rather than asking an external project to remind it what its supposed to be doing.

could be used with my adapted idea, but it could also be done without ntai.lua.

[Kloot]

Static config-spec files can be supplied now too, the only advantage
of bundling them with the AI would be that their naming conventions
could be standardized.

Additional libraries can be statically linked, which would be necessary
for archived AI's anyway (you can't link against an archive member at
runtime). At the very least Spring would have to unpack the AI library
to load it, so the whole point would be lost.

The filesize argument also hardly holds, AI libs are typically miniscule
compared to maps, mods, etc even when not fully stripped of debug
symbols.

The other approach for non-C++ AI's sounds much more realistic.

[AF]

Archives are a good idea as a means of installing AIs, but not for storing AIs and loading from them. Archivemover should extract the archive into the spring folder rather than move/copy it.

Also any references to lobby interface and tags should be accompanied by examples. The examples being given by people are too ambiguous and could mean various different things, and many interpretations because the language being used is not clear enough.

[hoijui]

The other approach for non-C++ AI's sounds much more realistic.

Which approech are ou reffering to here?

Static config-spec files can be supplied now too, the only advantage
of bundling them with the AI would be that their naming conventions
could be standardized.

Additional libraries can be statically linked, which would be necessary
for archived AI's anyway (you can't link against an archive member at
runtime). At the very least Spring would have to unpack the AI library
to load it, so the whole point would be lost.

The filesize argument also hardly holds, AI libs are typically miniscule
compared to maps, mods, etc even when not fully stripped of debug
symbols.

i agree

Archives are a good idea as a means of installing AIs, but not for storing AIs and loading from them. Archivemover should extract the archive into the spring folder rather than move/copy it.

+1

Also any references to lobby interface and tags should be accompanied by examples. The examples being given by people are too ambiguous and could mean various different things, and many interpretations because the language being used is not clear enough.

is this your way to say:
"explain your idea better, hoijui."
?
i would try to do so then

[AF]

I know sometimes even when I say these things it doesn't come out quite right and can be misinterpreted, so a quick example would be best.

I also like the idea of changing from aidll to ainame, I get it now that aidll isn't quite applicable when the AI has a .jar extension.

I think that AI options as such should all be strings key value pairs, considering what tobi said somewhere about floating point values. I also think we could put AI options inside the mod/map options in script.txt so that we don't need to make lobby protocol modifications to support them, taking the form <ainame>_<keyname>=<value>;

[Tobi]

AIs are unsynced though, so just like player colors it doesn't matter that much. (The options would just be ignored on other clients anyway because the AI isn't even instantiated there.)

[AF]

Yes but say we're in a multiplayer game and your not the host, and theres an option on the AI that makes it do things that would affect the game quite a lot you'll want to know what options are enabled, such as maphacks etc, or lets say the AI has a rather effective comnapping routine, or the predictive dgunning krogothe demonstrated a year or so ago.

That and the host might be an autohost.

[hoijui]

Code: Select all

// interfaces
spring/AI/Global/Interface-libs/OldCppInterface.dll
spring/AI/Global/Interface-libs/NewCppInterface.dll
spring/AI/Global/Interface-libs/JavaInterface.dll
spring/AI/Global/Interface-libs/CSharpInterface.dll
// Old C++ AIs
spring/AI/Global/Bot-libs/AAI.dll
spring/AI/Global/Bot-libs/KAI.dll
spring/AI/Global/Bot-libs/KAIK.dll
spring/AI/Global/Bot-libs/NTAI.dll
spring/AI/Global/Bot-libs/RAI.dll
// Java AIs
spring/AI/Global/Bot-libs/RAI.jar
spring/AI/Global/Bot-libs/Hoijui.jar
// C# AIs
spring/AI/Global/Bot-libs/CSAI.dll

All the DLLs in Interface-libs use the AI C interface. All the AI implementations are in Bot-libs, and one of the interfaces provided by the DLLs in Interface-libs. A Lobby calls a function on unitsync, that looks like this:

Code: Select all

int getAvailableAINames(char** availableAINames);

unitsync redirects this call to the engine. There, the implementations somehow like this:

Code: Select all

int getAvailableAINames(char** availableAINames) {
	
	for-each (interface-DLL in "spring/AI/Global/Interface-libs/") {
		availableAINames.append(interface-DLL.getAvailableAINames());
	}

	return availableAINames;
}

When the engine calls getAvailableAINames() on JavaInterface.dll for example, it simply returns a list of the jars in Bot-libs: "RAI", "Hoijui".
If we have two equal names, we resolve the conflict by prefixing the interface name to the AI.
In the end, the lobby will receive a list of strings, that looks like this, for our example:

Code: Select all

AAI
KAI
KAIK
NTAI
OldCppInterface_RAI
JavaInterface_RAI
Hoijui
CSAI

The lobby can now present this list to the user, when he wants to add a bot. When generating script.txt, it writes one of these names to the ainame tag:

Code: Select all

ainame=OldCppInterface_RAI

[Kloot]

unitsync redirects this call to the engine

There is no link between unitsync and Spring, it can't "redirect"
because the engine process doesn't exist yet at that point. So
unitsync itself would need to query the AI's (not impossible, but
more work and more dangerous because every AI then has the
chance to corrupt unitsync/lobby memory).

[hoijui]

hmm...
we could have a listAIs.dll, which is used by the engine and by unitsync, and contains basically this:

Code: Select all

DLL_EXPORT int getAvailableAINames(char** availableAINames) {
   
   for-each (interface-DLL in "spring/AI/Global/Interface-libs/") {
      availableAINames.append(interface-DLL.getAvailableAINames());
   }

   return availableAINames;
}

Or export this function from spring.exe

Or have a command line parameter for spring.exe, eg:

Code: Select all

spring.exe --list-available-ais

which prints the list of names to stdout or writes them into a file. unitsync would then call spring.exe to get the list of names this way.
is that a a good idea?
(at least no corruption of lobby memory is possible)

[Pxtl]

To me, a more important feature would be lobby support for limiting which AIs are applicable to which mods, and more information for the user about the AI when listing it in the lobby. Right now, the AI list is a guessing-game for users.

And yeah, I'd like AIs packaged in SD7s/zips - developers can use SDDs while working on them. But I imagine it wouldn't work out anyways given how many scratch-files AIs need.

[AF]

I still maintain that sd7 and sdz archives should only be used for download and installation. Archive mover should extract their contents into the spring directory. AIs should not be read from the archive by the engine, and we would not have an AI folder full of archives either.

I also think AIs should expose a name and a description. Because my lobby uses html and spring lobby uses richtext, we should use bb tags for formatting and let the lobby replace them accordingly.

Also AI support for mods is problematic in that an AI might flag a mod as unsupported, and a new version of the mod comes out which fixes support. Or a new game comes out which doesn't support the AI but the AI was released before and doesn't know of its existence.

Then there's the quasi questionable situations where an AI is considered supported by some people, and unsupported by others.

Instead I think we should have a list of AIs that definately work, and a list of AIs that definately don't. But these should include AI names and AI version numbers, which means the lobby needs the AIs name, a description, and a version number. A human readable name would be better too I'd rather see Java RAI than javainterface_rai

[hoijui]

good point Pxtl!

i often play LAN games of spring with mates, against AIs. We use a lot of different mods, and it is VERY hard to select the AIs. sure, for the mods we play often, and that change seldomly, we already know it, but it is still a nightmare, and for newcommers, it is a pain in the ass.
the most simple way to fix it, is maybe to include the information in the description, simply as text, like this:
KAIK
This AI is good for TA based mods (they usualy have a 2 letter abreviation ending in A, like BA, CA, ...). It attacks pretty early in the game. In addition to TA mods, it also supports mod X and Y quite well, and is works with mod H and K, but is considerably less challenging there. Beta versions of mod L are supported too, but still unstable; please report errors in the forum.

its not the cleanest way, but well... i can't imagine that AI devs want to always test thier AIs agianst all mods in different versions, to include the info. in practice, a very general information, like in the text above, would already help select the AI a lot and it is litte work for the AI dev.
if the text sais that mod H is supported, but a new version of H gets released which renders the AI less challenging or even unusable.. well... after one game, the players will know it, and it is still much more usefull then no information at all.

a lot to the