Hard Light Productions Forums
Modding, Mission Design, and Coding => FS2 Open Tools => Topic started by: ngld on March 29, 2015, 05:36:06 pm
-
This specification attempts to record all metadata that's needed to install, launch and update a mod. It's more or less a successor to the FSO Installer's text files (http://www.hard-light.net/wiki/index.php/FSO_Installer_Text_Files).
The actual spec is available on GitHub (https://github.com/ngld/fsmod-json-spec/blob/master/repo.js).
The install process for a mod works like this:
- The installer has a list of packages it wants to install.
- It then recursively resolves the package's dependencies.
- For every package it needs to check which files are already present. If a file is missing or doesn't match the provided checksum, it will be added to the list of missing files.
- The installer then downloads the necessary archives, verifies their integrity by generating and comparing their checksum and then extracts them. Using the file list from the mod metadata, it moves the extracted files to the mod's folder.
- Now the mod folder should match the file list contained in the mod metadata.
- The installer should save a mod.json file in the mod folder which contains a list of installed packages for this mod.
- Important: You shouldn't parse the "actions" field. It's only used by the server to generate the "filelist" property.
- Note: There are two special mod IDs which can be used as dependencies: retail and self. The "self" dependency can be ignored during installation. It's only relevant if you need to generate a -mod flag (which is documented below). The "retail" dependency expresses that the mod depends on the retail VPs AKA it's not a TC.
A few notes on launching a mod:
- The launcher reads the mod.json contained in the mod folder. It then resolve the mod's dependencies.
- It should use the cmdline property from the metadata and any flags the user specified.
- To determine the "-mod" flag, it has to flatten the dependency tree. The following rules apply:
- If one of the dependencies has its own dependencies, they are added behind the dependency.
- If a mod appears several times in the list, only the last appearance is kept.
- The special mod ID "self" refers to the mod which listed it as a dependency.
- The special mod ID "retail" refers to the directory containing the retail VPs. If a mod doesn't list retail as a dependency, it's a TC and you should launch FSO without the retail VPs.
- The generated "-mod" flag has to be deterministic and should match the output of other launchers implementing this specification.
Here's an example:
A depends on Z, self and B.
B depends on C.
C depends on nothing.
Z depends on C.
If you wanted to run mod A, you'd get this: -mod Z,A,B,C
If you want to run mod B, you'd have to use -mod B,C
Other notes:
- The "environment" property lists special hardware/OS dependencies. If those dependencies aren't met, the package can't be installed and shouldn't be displayed to the user.
- The "executables" property contains all provided executables. This is used by the FSO and FSO Nightlies package. Mods should only use this if they need to provide a custom build.
- "files" contains all downloadable files. You should use "filelist" to determine where the files are saved.
The "official" server-side implementation is Nebula (https://github.com/Hellzed/hlp-nebula). A prototype is running on fsnebula.org (https://fsnebula.org).
Mod metadata is available at https://fsnebula.org/repo/test.json. Here's the metadata for FSO 3.7.2 RC5 (also included in the previous link): https://fsnebula.org/repo/fso_3.7.2_RC5.json
Known client-side implementations are m!m's Launcher (http://www.hard-light.net/forums/index.php?topic=88119.0) and Knossos (https://dev.tproxy.de/knossos/).
Thanks go to m!m and Hellzed who have helped me write this spec.
Feel free to post questions, constructive criticism and other comments. If you want to talk about Nebula or Knossos, please post here (http://www.hard-light.net/forums/index.php?topic=86364.new).
-
A few notes on launching a mod:
- To determine the "-mod" flag, it has to flatten the dependency tree. The following rules apply:
- If a mod appears several times in the list, only the last appearance is kept.
Wouldn't this mean that a less important dependency overrides a more important dependency?
In your example Z and B depend on C but only the dependency of B is kept although Z is "more important" (as is appears earlier in the dependency list).
I would propose to change the "ordering" of the mods to be specified by topological sorting the dependency graph (AFAIK that or something similar is what most package managers use). That would make sure that dependencies are resolved in the right order without allowing for much interpretation by the implementation.
A few notes on launching a mod:
- To determine the "-mod" flag, it has to flatten the dependency tree. The following rules apply:
- The special mod ID "self" refers to the mod which listed it as a dependency.
I know that the "self" dependency is designed to allow ST:R to specify a "primarymod" dependency but do we really have to make it this complicated? How do you implement the dependency resolution when a mod has a "self" dependency?
I use a graph to represent dependencies but when a mod has a "self" dependency I would need to restructure the graph and find a new root node. What do you think about simply providing a "primary dependencies" functionality that prepends the dependency to the -mod line? Most mods will never need to use it and it would greatly simplify the dependency resolution implementation.
-
Wouldn't this mean that a less important dependency overrides a more important dependency?
In your example Z and B depend on C but only the dependency of B is kept although Z is "more important" (as is appears earlier in the dependency list).
I would propose to change the "ordering" of the mods to be specified by topological sorting the dependency graph (AFAIK that or something similar is what most package managers use). That would make sure that dependencies are resolved in the right order without allowing for much interpretation by the implementation.
You're right about that but what happens if B overrides files contained in C? If C appears anywhere in the mod list before B, those files won't be overriden. My approach ensures that all overrides are applied.
I know that the "self" dependency is designed to allow ST:R to specify a "primarymod" dependency but do we really have to make it this complicated? How do you implement the dependency resolution when a mod has a "self" dependency?
I use a graph to represent dependencies but when a mod has a "self" dependency I would need to restructure the graph and find a new root node. What do you think about simply providing a "primary dependencies" functionality that prepends the dependency to the -mod line? Most mods will never need to use it and it would greatly simplify the dependency resolution implementation.
I wanted to avoid adding new fields. Couldn't you ignore the "self" dependency when you build your graph? If it's easier for you, I can add a new field instead, however that means that I'll also need to add that field to the Nebula page. :blah:
-
Alright, here's a new proposal.
Instead of adding special mod IDs to the dependencies, we can add the following new fields:
"primarylist": ["<mod id>", "<mod id>", ...],
"type": "<mod|tc|engine|addon>",
"primarylist" works pretty much the same way it does now: The mods contained in this list will appear before the mod in the "-mod" list. Those mods still have to be listed as dependencies.
"type":
- The value "mod" should be obvious.
- "tc" stands for Total Conversion. This tells the launcher that this mod doesn't depend on retail data.
- "engine" should be used for FSO builds. This allows a launcher to easily generate a list of all FSO builds. (I'm not sure if this is actually useful.)
- "addon" is a mod which is supposed to be added on to other mods (i.e. the Cockpit Mod (http://www.hard-light.net/forums/index.php?topic=84459.0)). If the player wants to start an addon, they have to select another mod as the base. The -mod list is then generated by putting the addon's folder in front of the mod's "-mod" list.
If the addon has dependencies of its own, those will have to be resolved, too... This will probably work like the "primarylist" mechanism.
I hope that made sense... It's 1am here so I might've missed something.
-
So excited to see you back on this, you have no idea. FotG was just discussing how we'd like to package and distribute our first closed beta, whether to just do manual downloads, SVN access, or perhaps do a limited release over one of the distribution systems like this.
-
Glad you're excited about this. I haven't been able to spend much time on this lately, hence the long silence.
I hope we can resolve this soon because this is the only thing left to resolve before I can finish TC support for the Knossos launcher.
@chief: Do you have a rough estimate as to when you're going to release FotG?
-
No, as I said, we were just discussing preliminary packaging details yesterday. If we had this at our disposal to test with though, we might be able to get something releasable very soon.