[Vidmaster]

Yes, hundrets and hundrets of megabytes  , that is why I was asking. I got nine versions of bpc-audio1.vp and so on...

[Mito [PL]]

Generally, you can delete all but the most recent versions of all mods except for MediaVPs and probably FSPort/FSPort-MediaVPs. These are the "base" mods that are used as a requirement by the majority of existing mods, and many of these mods can rely on older versions of these.

I personally force whatever mods I play to use the latest and greatest MediaVPs by making a brute force modification in their mod.json file - replacing MediaVPs dependency version with 4.3.3 (at the moment). This is NOT supported, and quite likely to cause issues in case of more advanced mods, so your mileage may vary wildly. Although I'd personally say that most retail-like mods (Homesick, Sync, Uncharted Territory...) are pretty safe to have this done to them, lately I also forced The Aftermath: Reboot in the same way and found no real issues across the bunch of missions I tried.

Also, Blue Planet *does* have some mods that use it as a dependency. I recall that Mantle and Friends and Foes relied on some quite old version of BP, and there's Blue Planet: Battle Captains and possibly VOTC that might do so also. I think the rest of BP-reliant mods (Battle of Neptune, Warmachine, incoming Mote in the Sunbeam) should be happy with the most recent version.

[Vidmaster]

Yes, all valid points and I know how to do all this stuff too. After all, I have been part of this community for almost half of my life (is  or  more appropriate?).
The thing is just that I wanted to switch to Knossos due to the ease of updating but I am running into problems now that make me question if a manually managed install might still be the better option.

Is the Knossos source-code open and available? Depending on the underlying tech, I might even consider attempting to do a Pull/Merge Request to fix the issues / missing functions I was mentioning...

For those who have just tuned in, we were talking about

Is there a "clean-up" function anywhere, computing a dependency graph for all the currently installed games/mods and then removing everything not referenced by that graph anymore?
On that note, a verify all that verifies all the installed mods/games in order would be very nice too  .

[chief1983]

It is available on github, under ngld's repositories.  On mobile at the moment so forgive me for not having a link handy

[Vidmaster]

Found it. Gosh, it is python-based and some web-stuff on-top (or is Node-JS used for Python today as well?) Anyway, not my cup of tea.
Which does not mean that the community, including myself, does not appreciate your work, ngld .

Anyway, back to topic:

Generally, you can delete all but the most recent versions of all mods (...)

I manually cleaned up my installation now, which included updating and verifying. I got back over 65GB of storage from the tons of unneeded data. I also found several GBs inside its temp-folder.
Dare I say it, as cool as Knossos is, it seems to focus too much on the fresh install and not on maintaining one.

[Mito [PL]]

Well, given that this is an early beta that has a bunch of feature requests piled up but its main coder is curently unavailable... Dare I say, praise what you have at the moment

Also, I am absolutely sure that incremental updates (that is, downloading only files/archives that were modified between mod updates) and cleanup of leftovers are definitely in the plans if ngld gets his free time back.

[darckense]

Hi, I'm coming back to Free Space after several years. I would liket to play some of the new campaigns. 

I see that there is now a nice mod manager, Knossos.

I'm on a Debian linux system, and I tried hard, but I cannot manage to make it run on my computer.

Does anyone here know how to run it on Debian ?

I have tried to install the ubuntu package from the ppa repository, but it crashed.

Any advices ?

Many thanks,

--
Darckense

[praseodym]

You may want to check this one:

https://wiki.ubuntuusers.de/Howto/Knossos/

18.04 packages dont work with newer systems. Some dependencies may be missing?! You may want to compile it for yourself. Ubuntu 18.04 and 20.04 manual is there

[darckense]

This is for Ubuntu. (and in German...  )
I tried the Ubuntu package, unsucessfully. I guess that I have no choice but try to compile Knossos from source.
I will give it a try.

Thanks.

[AdmiralRalwood]

Also, I am absolutely sure that incremental updates (that is, downloading only files/archives that were modified between mod updates) and cleanup of leftovers are definitely in the plans if ngld gets his free time back.

As I understand it, they're supposed to already be in, but there might be some bugs there. As an example, I just updated BP from 1.1.1 to 1.1.2. The first thing the update process did was go over every installed file looking for ones that haven't changed. Once that was done, all the unchanged packages were removed from the "to be downloaded" list; instead of reinstalling the whole thing, I only had to download the updated "visuals" packages (1, 3, 4, and 5). Then, the unchanged old files were copied over. And then, since I had no mods depending on 1.1.1, the 1.1.1 folder was emptied out; the old (empty) folder unfortunately sticks around, but at least my harddrive doesn't have 7+ gigs of duplicate data.

[ngld]

Hi, it's... been a while 

I'm sorry for not being around as much as I probably should've been. Either way, I'm finally (slowly) working on Knossos again. However, it's probably going to take quite some time before I can release anything substantial.
Let's recap and take a look at the current issues with Knossos:

• The way Knossos manages mod metadata (both within Knossos itself as well as downloading from Nebula) is being pushed way past the limits it was designed for. .json files aren't databases.
• Basing most of Knossos' UI around a Qt5-wrapped Chromium was probably a mistake (especially since most of the non-UI code is written in Python and has to go through the PyQt5 bindings as well).
• There are a bunch of subtle bugs which I never figured out: Issues with the joystick setup preventing FSO from launching, Knossos hanging/crashing on launch
• The transfer of data from Python -> PyQt5 -> Qt5 -> Chromium -> JavaScript worked better than it had any right to but only at first. With the current amount of mods and releases, bugs start to appear which seem to mostly result in things not showing up in the UI although they really should.
• The current dependency resolution is far too messy and mostly the result of chasing an ideal that probably isn't realistic/practical for mods.
• Building new Knossos releases is a pain since it involves bundling Qt5/PyQt5 with PyInstaller which can fail in fascinating ways.

And that's just what I can recall right now. I'm pretty sure there are a few more bugs around uploading mods and metadata processing which I don't recall right now.

First of all, I want to make a few changes to how mods work (this will make mods more reliable and simpler in the long run):

• Dependencies always point to a fixed version of each dependency. Users will always get exactly the release specified by the uploader. I've tried to avoid this because it means that someone has to update all mods depending on MVPs after each MVPs release. However, this will prevent mods from breaking after a dependency updates.
• To balance this change, users will get a screen where they can easily edit (add/remove/change) dependencies. I'll put a disclaimer on that page that those changes aren't supported and might cause breakage. Hopefully, that's enough to prevent people from messing up their mods.
• FSO builds are linked directly to each release instead of (indirectly) through mod packages. Uploaders can select a specific or minimum FSO version. This should fix the various issues where Knossos couldn't find an executable to run.
• Transitive dependencies aren't allowed. Whenever you add a dependency, all required mods for that dependencies are added as dependencies to your mod as well. This makes dependency resolution much simpler and gives you complete control over the mod versions Knossos users.

I think those are the most important changes. The next section will talk about more technical aspects and the different implementations of the current Knossos / Nebula implementation and what I'm planning instead. If you're not interested in that, skip to the end.


With that said, my goal has been to rewrite Knossos (and Nebula) from scratch. My current priorities are to make a) maintanance easier (to make sure Knossos stays stable even if I can't actively work on it) b) make bundling / compiling easier and c) to make working on the code fun again. All of these bugs are very frustrating to deal with especially since most of them don't have a clear or straightforward solution.

So here's the current plan and what I've done so far to get there:
I'm rewriting Nebula first. This time it's a modern web application (SPA). The benefit here is that I'm more comfortable with that since web dev is more or less my day job. It also ensures that you can always manage any mods you have uploaded to Nebula regardless of whether you can currently use Knossos or install the mods. The downside is that managing mods requires an active internet connection. Having a proper web application also simplifies account management (registration, login, password, reset).

The current Nebula implementation is a bare-bones Flask application implemented in Python which likes to waste memory for seemingly no reason (three processes are currently sitting at 1.3 GiB each).
The new implementation is done in Go. There are many trade-offs involved but for me the benefits are: Vastly simpler to deploy (a single static binary vs installing Python, dependencies and a WSGI server) and easier background tasks (goroutines vs threads). Aside from that, the type checking is nice and the implementation will probably be faster but that doesn't really matter for Nebula since the current Python implementation usually processes requests in less time than a TCP handshake takes (0-3ms for DB-only stuff and 30-500ms for upload processing; mod releases take way longer but that's mostly because Nebula has to export a 150MiB .json file).

The current database is MongoDB but I'll replace it with PostgreSQL. This means I'll have to conform to a proper schema but that's not a bad thing. The main reason for this move is that Nebula is the only stuff I'm running that uses MongoDB while I have several applications that need PostgreSQL. Having less databases to maintain simplifies things. By now I also have far more experience troubleshooting and optimizing PostgreSQL.

On the client side, we currently have a Python application that uses PyQt5 to render the UI and Qt5's Chromium implementation (QtWebEngine) to render most of the UI. As I pointed out before, this lead to a bunch of issues. What makes it worse is that Knossos kind of grew into this and was never designed around the problems this kind of UI brings with it. Here are a few screenshots from ancient Knossos versions:



You can see how more and more of the UI was replaced with the Chromium widget. That's not necessarily a bad thing. However, the current Python code is still designed around the assumption that it has direct access to the UI widgets and that it has to filter mods, etc. For example, performing the mod search and filtering in JavaScript would avoid quite a few problems Knossos currently has to deal with.

With that said, my goal for the new Knossos implementation is to embrace the web UI completely by replacing Qt5 with CEF (Chromium Embedded Framework). I'll implement most of the desktop-specific code (archive extraction, downloads, etc.) in Go (since I'm more comfortable with it than C++) and then write a fairly simple launcher in C++ which initializes CEF, opens the main window and loads the UI. Additionally, it will give JavaScript access to the Go-code (CEF allows us to define additional browser APIs. Through this we can call C++ functions from JavaScript and C++ functions can call Go functions through cgo). I know this sounds complicated (and probably error-prone) but it's far simpler than the current Python -> PyQt5 -> Qt5 -> Chromium -> JavaScript interface. To avoid type conversions (which caused most of the current issues with the Python -> JS interface), I'll pass raw bytes encoded in Protocol Buffers, Flat Buffers or a similar protocol. The goal is to pass statically typed struct-like objects between both languages with little overhead.
In case anyone's wondering why I'm not using Electron: In my opinion it's overly complicated and uses too much memory. Some of that is caused by Chromium itself (I suppose we all know how memory hungry Google Chrome is) but Electron seems to make it worse. I'm hoping this implementation is goind to be more efficient. I've also used CEF several times before but haven't used Electron for anything.

For file uploads, I can use TUS to replace the current resumable uploads. TUS should be more reliable since a) I've had pretty good results with it in a different project b) it's a mature and active project.
To extract archives, I'll use libarchive. This means Knossos won't have to move files around after extraction, instead it'll create the files at the final location.

I'm not entirely sure which format I'll use for metadata sync between Knossos and Nebula. Currently, Nebula exports the list of all public mods with all releases as a single .json file which Knossos downloads and parses. To put it mildly, this is a bad idea and leads to several issues (including running out of memory).
My currently favored solutions are either splitting the metadata into several files (probably one per mod) and encode them with Protocol Buffers since it's fairly compact and easier to parse than JSON. An alternative would be to use an embedded database like SQLite, Bolt, ... to store the metadata and have Knossos download and load that database.
The benefits of the latter approach are that we won't have to worry about limits since these databases can handle several GiBs worth of data and we'll probably never reach that with metadata alone. However, it'd require Knossos to always download the full DB dump which seems like a waste of bandwidth.
Having one Protocol Buffer file per mod would allow us to only download metadata for updated mods (a list of mods and last change date / revision would be stored in an index to make that possible). The problem with Protocol Buffers is that you always have to load the whole file in memory before you deserialize it (you can't stream it). This essentially puts an upper limit on the file size. I'm not sure if a single mod can ever reach a problematic size but I suppose it's better to assume it might happen and think of a way to split the files further.


With that huge info dump out of the way, I can finally talk about how far I am. I've already started the Nebula rewrite a while ago and user registration, login and password reset are already functional (including the relevant mails). I'm currently working on importing the old Nebula database into the new DB schema. Once that's done, I'll have to finish the mod UI (view, edit, download for people without Knossos). After that, I'll try to get the desktop client into a state where it can download, install and launch mods.
We'll see how long this takes. I'm hoping that once I reach that milestone, the new client will be more reliable and will allow people to spend more time actually playing mods and less time troubleshooting Knossos.

As I understand it, they're supposed to already be in, but there might be some bugs there. As an example, I just updated BP from 1.1.1 to 1.1.2. The first thing the update process did was go over every installed file looking for ones that haven't changed.

This is correct. Knossos compares the checksums to find identical files, however this doesn't always work. Sometimes the reason is that while the VP contents haven't changed, the VPs were repacked which caused the file checksums to change as well.
The empty folders (and sometimes unused mods) are left behind because Knossos generally errs on the side of caution and only deletes stuff if it's sure you want to remove it. Specifically, uninstalling mods or updating one installed version to the next. If you have multiple versions of the same mod installed, the older versions are never deleted.
Empty folders are sometimes left behind because Knossos usually only looks at individual packages and never at a whole mod. Once the last package is removed, the mod isn't installed anymore as far as Knossos is concerned. Knossos doesn't touch anything not part of a package because everything else is was probably created by the user.

This is for Ubuntu. (and in German...  )
I tried the Ubuntu package, unsucessfully. I guess that I have no choice but try to compile Knossos from source.
I will give it a try.

Thanks.

I've made the last release available to the new distros as well. However, I can't rebuild the package right now so I have no idea if it even works. It's not even the latest version (the PPA still has 0.13.3 instead of 0.14.3) because I never got around to updating the Ubuntu build scripts. You'll probably be better of building from source.

Found it. Gosh, it is python-based and some web-stuff on-top (or is Node-JS used for Python today as well?) Anyway, not my cup of tea.
Which does not mean that the community, including myself, does not appreciate your work, ngld .

It's even worse: Python, PyQt5, Qt5, Chromium (QtWebEngine) and Vue (UI framework in JavaScript). The Node.JS stuff is only used to compile the web assets.

Dare I say it, as cool as Knossos is, it seems to focus too much on the fresh install and not on maintaining one.

I think a better way to put it would be that Knossos is paranoid about accidentally deleting something you might want to keep. Having to manually delete old mod versions seemed the lesser evil at the time.

It should be possible to compute a dependency graph for all the currently installed games/mods and to remove everything not referenced by that graph anymore. Thoughts?

Knossos doesn't track whether mods were installed by the user or to satisfy a dependency. That would have to be added first but aside from that it sounds like a good idea.

It would be wonderful if someone could get Knossos installable again by simply getting it from a PPA, but I gather that is quite a bit of work and requires a decent amount of maintenance to keep it functioning.  Praseodym and Anagram have both provided instructions for successfully building Knossos 0.14.3.

Here's the build script for Ubuntu. Essentially, you have to create an archive with Knossos' sources (see the tar -czf line in the linked script), create a new empty folder, place that archive and the releng/ubuntu/debian folder inside, update the changelog and run "dpkg-buildpackage -us -uc". If anyone can make that work (and the built .deb file works), send me that folder (with the .orig.tar.gz and debian folder) and I'll publish it on the PPA.


If an SCP dev wants access to the automated error reports, I can give them an account. I can also publicly post the most common errors if anyone's interested in fixing them.


TL;DR: I'm still alive, a new Knossos version might be coming... maybe even before the end of the year but who knows.

[Asteroth]

• Dependencies always point to a fixed version of each dependency. Users will always get exactly the release specified by the uploader. I've tried to avoid this because it means that someone has to update all mods depending on MVPs after each MVPs release. However, this will prevent mods from breaking after a dependency updates.

While this would be more stable, I worry about users that have many different space-hogging copies of MVP versions or the like only differing in tiny compatability changes because various mods point at slightly different versions. This is already an issue to some extent.

[ngld]

To balance this change, users will get a screen where they can easily edit (add/remove/change) dependencies. I'll put a disclaimer on that page that those changes aren't supported and might cause breakage. Hopefully, that's enough to prevent people from messing up their mods.

I'll probably have to do this on the mod installation screen as well. One of the issues with the current dependency resolution is that Knossos always prefers installed versions over available versions which means that as an uploader or mod developer, it's hard to tell which combination of mod versions a user will play with. Pinning dependencies makes the result reproducable and most of all consistent.
If we can find some way to guarantee that a mod will work with all possible dependency versions, I'd support it but I don't see that as realistic (not unless we somehow manage to automate testing and build a mod CI).

I worry about users that have many different space-hogging copies of MVP versions or the like only differing in tiny compatability changes

EDIT: I roughly remember m!m implementing some kind of VFS in FSO which could be used to avoid duplicate files for cases like this. Not sure, I'll have to look it up again.

[Mongoose]

This is some unexpected but incredibly welcome news. Great to see you back!

[EatThePath]

I will hopefully add more over the weekend but I want to get this out before I forget it.

To balance this change, users will get a screen where they can easily edit (add/remove/change) dependencies. I'll put a disclaimer on that page that those changes aren't supported and might cause breakage. Hopefully, that's enough to prevent people from messing up their mods.

I'll probably have to do this on the mod installation screen as well. One of the issues with the current dependency resolution is that Knossos always prefers installed versions over available versions which means that as an uploader or mod developer, it's hard to tell which combination of mod versions a user will play with. Pinning dependencies makes the result reproducable and most of all consistent.
If we can find some way to guarantee that a mod will work with all possible dependency versions, I'd support it but I don't see that as realistic (not unless we somehow manage to automate testing and build a mod CI).

I can totally see your perspective here, but speaking as someone that has been on the receiving end of the problems you wish to avoid, I don't really lik this solution. The versioning as is when used properly is exceedingly useful, I believe FSU has stuck to it pretty well with good results. For myself, Warmachine depends on BP and the MVPs both. If the MVPs do a bugfix right now and increment the third version, I have to do nothing. BP didn't follow the scheme for a long time and that caused hell for those of downstream of it, but now that they've switched over it's been pretty okay. Yeah if the upstreams screw up the versioning, it's a nightmare, but so is the maintence prospect of always hard pinned versioning.

I think part of the problem with the current versioning is that it's not been clearly or loudly communicated, either in the UI or in any prominent place in the community as far as I've seen, how it works and is supposed to be used. Maybe there's a post in this thread, I've never sifted through all 1k posts. The UI just has a TODO in place of any guidance currently. Better communication there, as well in the develop tab sections where you actually set the dependencies, would help. Also helpful I think would be very prominent display of the versions of each mod being loaded somewhere on the launching interface, right along with the easy overrides you're talking about.

If nothing else, I beg something like a 'trust upstream version numbering' opt-in buried somewhere in the develop tab, covered in warnings and explanation. "Newest" as a version targeting should almost certainly die in a fire, for sure, and being able to hard target specific versions with no wiggle room should be an easily accessed option, but being able to automatically load bugfix updates is IMO vital to the sanity of anyone maintaining mods that depend on anything.  Please don't make it so that BP has to update every time FSU fixes a bug to get the fix, and then I have to update after they do.

[ngld]

Thanks for the input, those are some very good points. The versioning scheme was explained in the mod creation guide. I can put the same explanation next to the version field. IIRC the issue with that was that the UI didn't have enough space but I can fix that during the rewrite.

What do you think about this solution? The UI will by default use a "simple" mode where you only select the major version for each dependency which then translates into >=M.0.0,<(M + 1).0.0 where M is the major version. Or would >=M.N.0,<M.(N+1).0 be a better default?
I'll also add an advanced option which accepts any valid semver version range (~1.0.0, ^4.0.0, >=5.9.8-32fds, ...) which you can use to specify whatever version range you want / need.

Also helpful I think would be very prominent display of the versions of each mod being loaded somewhere on the launching interface, right along with the easy overrides you're talking about.

That's a good idea. I'll also record the specific mod versions an uploader used to test the mod and display those next to the version you're currently using. That might help with troubleshooting.

The only issue with this is that I'm forced to deal with transitive dependencies again unless I force uploaders to classify any release that adds or removes dependencies as breaking. Any ideas on how to handle this?

This is some unexpected but incredibly welcome news. Great to see you back!

I'm glad to be back and it's nice to see quite a few familiar faces again.

[jadeddragoon]

So lets get the ball rolling on requests for the re-write...

One of the things I've run into is the differences in the way knossos handles "dev-mode" mods and the final user-mode mod and how that affects the load order for modular tables. Let me see if i can explain the issue succinctly.

First we need to understand how modular tables are resolved. There are two overlapping methods:

• Reverse Alphabetical by Filename - the appropriate `.tbl` file is loaded... then `.tbm` files in the same folder/vps are loaded in reverse alphabetical order. Each entry in the `.tbm` files can override matching entries in the the previously loaded `.tbm`s and the matching `.tbl` as well. In this way, `ships.tbl` is loaded first then `z-shp.tbm` is next and finally `a-shp.tbm` last. As a result if any entries are shared between the files the ones in `a-shp.tbm` will override the ones in `z-shp.tbm` and `ships.tbl` and the ones in `z-shp.tbm` will override the ones in `ships.tbl`. (still with me?)
• Reverse `-mod` Order - when specifying mods to load via the `-mod` launch flag, the `.tbm` files of the last folder path passed to `-mod` are all loaded first (applying the Reverse Alphabetical by Filename to any `.tbm`s within that folder). In this way, `-mod fs2\foo,fs2\bar` would load all modular tables in the `fs2\bar\tables\` folder then all modular tables in the `fs1\foo\tables\` folder. This means that entries in modular tables in the "foo" mod would take priority over matching entries in the modular tables in the "bar" mod. When separate `-mod`s have modular tables within them which must also be resolved the Reverse Alphabetical order is used within each `-mod` and those results are resolved based on Reverse `-mod` Order.

And then there are `.vp` files which... I am unsure about. I believe FSO loads `.vp` files in reverse alphabetical order as well... treating them like separate `-mod` entries as it goes. I also assume Loose files take precedence over the modular tables in `.vp` files. But I have no data to back this up and there doesn't appear to be any documentation on the subject. This is now confirmed.

Knossos allows mod developers to specify separate packages within their mod. For the MediaVPs, for example, we have packages like  `MV_Root` and `MV_Advanced1`. This allows for better organization of the mod's contents as well as giving the option for users to skip downloading and installing components they don't want. When a user downloads a mod via Knossos it places all the files for each selected package into a single folder combining the packages together. Then Knossos can load the entire mod with a single path entry in the `-mod` launch flag when starting FSO or Fred. The only way I am aware of to keep the contents of packages separate is to have each package as a `.vp` file... which Knossos helpfully provides a setting for. However, if a package is set as required for the mod it appears Knossos ignores the instruction to bundle the package as a `.vp` file.

On the developer side things are a bit different. Packages are not combined but instead kept in separate folders. This is great for organization. Absolutely a good thing. However, it means that loading the mod into FSO for testing requires Knossos to pass each package as a separate path to `-mod`. See the problem? Because packages are combined on the user side the "Reverse Alphabetical" method of resolving modular tables is king... but on the developer side the "Reverse `-mod` Order" method takes precedence which can introduce a discrepancy between how the modular tables are resolved for the developer and how they are resolved for the user. That's a problem. That's a big problem.

Knossos currently automatically orders package paths in alphabetical order (by the folder name, not the package name) when passing them to `-mod`... which means that, because FSO loads those paths in the reverse order from how they were specified, packages are loaded in reverse alphabetical order. Since Knossos uses the package folder name to determine the name of `.vp` files it creates from them simply bundling each package as a `.vp` seems like a solution to restore consistency. That way, hypothetically, packages get loaded in reverse alphabetical order on the dev side and `.vp`s (maybe) get loaded in reverse alphabetical order on the user side. However, this doesn't quite work because "required" packages are not bundled as `.vp` files even if we tell Knossos we want those specific packages bundled.

Thus the solution i've been using is two-fold. First, I name package folders such that they will be loaded on the developer side in the order I want them to be:



Then I also name the `.tbm`s such that they will be loaded on the user side in the order I want them to be:



Needless to say this is an annoyance and quite prone to error. A better way to ensure load order consistency for modular tables would be much appreciated.

[ngld]

However, this doesn't quite work because "required" packages are not bundled as `.vp` files even if we tell Knossos we want those specific packages bundled.

That's a bug. The required/recommended/optional status doesn't influence whether VP packing works (see for example the MediaVPs they have several required packages that contains VPs built by Knossos). Most likely you were affected by a bug that causes Knossos to not reupload a package if the only change is whether Knossos is packing the VP or not. Once you modify a file and upload again, the VP packing should work. That said, this shouldn't be an issue in the rewritten version.

When a user downloads a mod via Knossos it places all the files for each selected package into a single folder combining the packages together.

I originally implemented it that way to be compatible with existing FSO launchers. I don't think this matters as much now and reducing the differences between mods in dev and user mode is more important (mostly due to the issues you brought up). The new implementation will create one folder for each package, most likely even if they're packed as VPs since it simplifies the logic involved. As a side effect, VP load order in FSO no longer matters since it'll receive one folder in the -mod list for each package. That should guarantee that the load order is consistent. It creates a few folders that would be unnecessary with the current system but I think it's worth the benefits.

[jr2]

• To balance this change, users will get a screen where they can easily edit (add/remove/change) dependencies. I'll put a disclaimer on that page that those changes aren't supported and might cause breakage. Hopefully, that's enough to prevent people from messing up their mods.

It might prevent some future troubleshooting headaches if, wherever there are settings to customize or change things like this, there is also a "reset to default" option (but you might have already set it up so that this capability is either inherent or included).

[jadeddragoon]

However, this doesn't quite work because "required" packages are not bundled as `.vp` files even if we tell Knossos we want those specific packages bundled.

That's a bug. The required/recommended/optional status doesn't influence whether VP packing works (see for example the MediaVPs they have several required packages that contains VPs built by Knossos). Most likely you were affected by a bug that causes Knossos to not reupload a package if the only change is whether Knossos is packing the VP or not. Once you modify a file and upload again, the VP packing should work. That said, this shouldn't be an issue in the rewritten version.

When a user downloads a mod via Knossos it places all the files for each selected package into a single folder combining the packages together.

I originally implemented it that way to be compatible with existing FSO launchers. I don't think this matters as much now and reducing the differences between mods in dev and user mode is more important (mostly due to the issues you brought up). The new implementation will create one folder for each package, most likely even if they're packed as VPs since it simplifies the logic involved. As a side effect, VP load order in FSO no longer matters since it'll receive one folder in the -mod list for each package. That should guarantee that the load order is consistent. It creates a few folders that would be unnecessary with the current system but I think it's worth the benefits.

Sweet, thanks.

Some other thoughts...

What about being able to set a mod as mod type "Modder's Resource" to indicate the mod isn't intended to be launched directly but instead be used only as a dependency of other mods. This way such mods don't need the full set of dependencies applied as that would be handled by the mod they are a dependency of... which in turn reduces the risk of dependency conflicts and other dependency related issues likely to crop up over time for mods that hardly ever change.

And what about being able to set a mod's mod type to "Mod Pack" to indicate that the mod adds nothing in and of itself but instead is simply combining other mods via dependencies. For example, I've implemented Lafiel's hud icons script as a mod in the developer tab. I've also implemented the freecam script as another. Lastly I have my own HD Hud for BluePlanet Complete (and associated HD Radar Icons). I created a mod in the develop tab that combines these for use when I want to play blueplanet complete:



Like so. `Jaded's Blueplanet` is, in this case, an empty package... but could be used to resolve conflicts between "depended" mods if there were any. Currently I haven't published any of these mod packs to knossos as I don't want to gunk up the mod list with these. If i can mark them as mod packs then maybe they can be filtered out by default or shown in a separate tab. Just an idea.

Also, since your talking about having packages handled as separate `-mod`s anyway... what about package-level dependencies? That way even if i have every FSU MediaVPs package installed when I play a mod that doesn't require the Animglows package the animglows package isn't added to the list of `-mod`s (unless I, as a user, have manually specified otherwise).

Sorry in advance for shotgunning you with ideas. lol :-P