Hard Light Productions Forums
Modding, Mission Design, and Coding => FS2 Open Coding - The Source Code Project (SCP) => Topic started by: m!m on August 06, 2014, 05:46:48 am
-
Hello everyone,
Here is a small project of mine which I started after giving up on fixing wxLauncher. I know what you are thinking "What? Another launcher? But we already have one!".
Before I discuss why I think this step is needed here are the important things:
Download (http://vpgui.fsmods.net/launcher/setup.exe) (Updates will be installed automagically)
This application is only compatible with Windows Vista and later, see below for an explanation why.
Screenshots: Imgur Album (http://imgur.com/a/7Lqm9)
Source code: https://github.com/asarium/FSOLauncher
Changelog:
Version 0.1.0.2:
Version 0.1.0.1:
- Selecting the audio device added
- Fixed executable selection grouping
Version 0.1:
Now that the important stuff is complete I will try to explain what the main goals of the project are and why I decided that this step was necessary.
Main goals of this project:
- Easy to use one-stop solution to installing, playing and installing/updating mods for FSO
- Make modding easier (this is a secondary goal but it is still important)
Known issues:
- Will not run on non-Windows platforms
- There are still quite a lot of things that are not implemented
- Non-Antipodes builds use a different Joystick than the selected one
- Adding profiles is not implemented
Planned features:
- Install and update mods
- Install FreeSpace from the GOG package
Why only Windows?
Windows has the biggest user base and provides the best libraries to create a nice UI for the launcher. I decided to use C# as the language this launcher is written in as it is a very good language and while there is an implementation for non-Windows platforms the UI framework I use is only available on Windows.
I am aware of this limitation which is why I separated the code that actually handles the management of FSO into a separate assembly that should be usable on other platforms so a launcher based on that code for other platforms is still possible.
Support for previous FSO versions
This launcher will not support any version before the current official release. That also means that the launcher will not be able to launch Retail FS2 but who does that anyway :P
If you have questions or feature requests please post them here. If you want to help developing this then you can take a look at the source code.
Thanks for reading!
-
Hello everyone,
Here is a small project of mine which I started after giving up on fixing wxLauncher. I know what you are thinking "What? Another launcher? But we already have one!".
Actually we already have 4 that I can think of, 2 are in active use.
I am also curious as to what the insurmountable issue you were having with wxLauncher that you would rather start from scratch than build on something that does work.
-
I saw multiple issues:
- The code that created the UI was also doing other stuff like querying the joysticks which made it nearly impossible to understand what was going on. I tried improving that by porting wxLauncher to wxWidgets to version 3 (which worked surprisingly well) and redesigning the UI in a designer but that didn't help much.
- Blocking operations were executed on the UI thread which freezes the UI until those were done.
- It is written in C++ and that was the biggest issue, that language is simply not good for programming an UI.
I spend hours for implementing something that should be trivial and then I recognized that a new launcher could do a better job when I rewrite the launcher and have more possibilities in the future.
-
Will Goober's own Installer code be of any help, or are you to go through the same motions and troubles he did for the past years?
e: I must say I applaud the effort and all, but the aspect of it being windows-only tells me this is like a dead end from the get go. People with other systems will either be left out or the software for them will have to be updated anyway, etc. My tenth of two cents what do I know! Good luck!
-
The code of the installer can be of help to get things to work (like extracting the installer) but I can't use the code directly as it is written in Java.
I though about the Windows-only problem a bit before starting the project and I agree that doing it this way is not perfect but given the trouble and feature limitations I would need to handle I came to the conclusion that it was simply not worth it. The code that actually handles most of the FSO management will be cross-platform compatible so it can also be used on other platforms if someone wants to write a frontend (which should be easy to do).
-
Was wxWidgets not an option for the UI framework for some reason?
-
wxWidgets is a C++ framework and I wanted to avoid that language.
-
Hmm. This (http://wxnet.sourceforge.net/) exists, but it's been inactive for 3 years. Whether that means it's abandoned or just complete, I don't know.
-
Windows has the biggest user base and provides the best libraries to create a nice UI for the launcher.
I don't understand the effort. FSO's stated objective IS for a multi-platform engine; it therefor needs a multi-platform launcher. And speak for yourself. I happen to be an OSX user, and I'm sure there are plenty of linux users here. You say this step is "needed" but, to me, this would be a step backward. We are in the process of getting rid of a windows only launcher being the main one. Now you want to go back?!? The current windows only one still works, and it was already decided to be a dead end. Frankly, people then didn't see the benefit in maintaining 32 different launchers, dividing the user experience, and in general, being a support nightmare.
-
I saw multiple issues:
The code that created the UI was also doing other stuff like querying the joysticks which made it nearly impossible to understand what was going on. I tried improving that by porting wxLauncher to wxWidgets to version 3 (which worked surprisingly well) and redesigning the UI in a designer but that didn't help much.
I have made some progress on addressing this in my work on wxL, but it remains an issue, although I hope to continue working on addressing it.
Blocking operations were executed on the UI thread which freezes the UI until those were done.
I was working on a patch a while back to retrieve the news/highlights in a separate thread. Need to finish that. Probably wouldn't be too much extra work.
It is written in C++ and that was the biggest issue, that language is simply not good for programming an UI.
Can't do anything about this one, although consider that Qt is in C++ and is widely used (http://en.wikipedia.org/wiki/Qt_%28software%29#Uses), although some may use alternate language bindings. We use Qt extensively at work, where I work on the RIFT MMORPG.
EDIT: Typo correction
-
I don't understand the effort. FSO's stated objective IS for a multi-platform engine; it therefor needs a multi-platform launcher. And speak for yourself. I happen to be an OSX user, and I'm sure there are plenty of linux users here. You say this step is "needed" but, to me, this would be a step backward. We are in the process of getting rid of a windows only launcher being the main one. Now you want to go back?!? The current windows only one still works, and it was already decided to be a dead end. Frankly, people then didn't see the benefit in maintaining 32 different launchers, dividing the user experience, and in general, being a support nightmare.
As I have said before, only the UI is Windows specific. The main part of the application can also be used on Linux and Mac OSX but you will have to write an UI for that.
Can't do anything about this one, although consider that Qt is in C++ and is widely used (http://en.wikipedia.org/wiki/Qt_%28software%29#Uses), although some may use alternate language bindings. We use Qt extensively at work, where I work on the RIFT MMORPG.
Just because something is widely used doesn't make it good :P
There is also the problem with the inactivity of the wxLauncher project, I opened a ticket with a patch three months ago which still hasn't seen any activity. That may only be a temporary phase but from my perspective the project looks like a dead end.
-
C# does have cross-platform UI options (http://stackoverflow.com/questions/779283/is-there-a-cross-platform-gui-framework-for-c-net), most notably a compatibility layer over WinForms (http://mono-project.com/WinForms). I seriously advise that you take a look at them, in this day and age it's not acceptable to be writing Windows-only tools.
-
I am aware of the possibilities for cross-platform UIs in C# but
- I am not familiar with WinForms
- Most of these solutions will need special handling for the different platforms anyway or the user experience will suffer
-
As far as 2 is concerned it's far better than relying on someone else coming along and writing an entire frontend from scratch.
-
I agree that it would be better than nothing but I have never worked with WinForms before and I have no intentions of learning how to use it as it is clearly inferior to WPF.
EDIT: WinForms is also somewhat abandoned by both Microsoft and Mono.
-
Well then all I can do is reiterate what I said earlier: FSO is a cross-platform engine, and cross-platform player-end tools are now very much standard, and have been for years. Your launcher will not be an acceptable alternative for wxL until you can provide that, no matter how much better you think it is.
-
I agree that it would be better than nothing but I have never worked with WinForms before and I have no intentions of learning how to use it as it is clearly inferior to WPF.
EDIT: WinForms is also somewhat abandoned by both Microsoft and Mono.
Luckily (Depending on your perspective), I am spending my work days deep in the guts of a WinForms app.
-
Well then all I can do is reiterate what I said earlier: FSO is a cross-platform engine, and cross-platform player-end tools are now very much standard, and have been for years. Your launcher will not be an acceptable alternative for wxL until you can provide that, no matter how much better you think it is.
I know that a launcher needs to cross-platform compatible and when the management library is feature complete I will try to implement an UI using either GTK# (http://www.mono-project.com/GtkSharp) or XWT (https://github.com/mono/xwt).
Luckily (Depending on your perspective), I am spending my work days deep in the guts of a WinForms app.
That is most likely not a pleasant occupation. If your scheduling allows that I would appreciate any help with this project :P
-
There is quite a large movement to use web technologies in desktop apps. However, they are all so young that we can only guess which implementations eventually evolve into something mature and commercially viable. The movement is understandable because today web browsers are possibly the closest thing we have for uniform cross-platform graphics rendering engine.
https://github.com/atom/atom-shell
https://github.com/adobe/brackets-shell
https://github.com/rogerwang/node-webkit
But this is mostly irrelevant post as far as this launcher is concerned.
-
i think i would have used lua + iup. you can quickly slap together a multi-platform gui with just a few lines of lua script. its probibly not viable for this purpose because i haven't been able to get sdl bindings to work in a lua app without building a custom interpreter, so i couldn't retrieve joystick indices that would work in the game, and other low level system calls needed to configure itself properly.
lua also has other good points though, the launcher could pass lua variables to a mod. so you could run a pre-launch configuration script to let users tweak mod options.
-
Main goals of this project:
- Easy to use one-stop solution to installing, playing and installing/updating mods for FSO
- Make modding easier (this is a secondary goal but it is still important)
A few months ago, ngld and I worked on a somewhat similar project. The app itself worked pretty well for an alpha software (it had some bugs of course, but no big show stopper), but we ran into some big issues :
- Mods have currently no standard files structure. The only workaround is to handle many different types of archives, and use "descriptor" files to rebuild files structure from scratch, for each mod. It adds a maintenance burden on whoever runs the mode repository.
- fs2_open itself and VP files have no clue about versioning. Versions had to be handled by the "descriptor" files too. When you have mods from different sources (the main repository, and some updated or unsupported mod repos), you get conflicts. Solving these conflicts is a PITA.
- Some fs2_open command line flags are undocumented (especially anything related to sound). You can try to use OpenAL standards, and fs2_open behaves in a completely differend, or completely weird way. Development process becomes a game of trial and error, much more than it should.
- Same applies for saved settings : different defaults apply to Windows/*nix platforms. I believe it's fixed in trunk, with Windows builds switching to .cfg files, instead of using the registry.
You can try to fix these issues by yourself, which is ok, but it will quickly become a hassle.
It would be better to fix the root issue : fs2_open needs a mod package standard that every launcher and mod manager out there could implement.
ngld and I both prioritized RL since march, so our project is dead at the moment, but if you want to take a look, the dev logs are still on the forum, and the git repository is on github (everything is written in Python, with a Qt UI).
The end goal was to integrate a new version of the mod database and the launcher/installer like Steam does.
-
- Mods have currently no standard files structure. The only workaround is to handle many different types of archives, and use "descriptor" files to rebuild files structure from scratch, for each mod. It adds a maintenance burden on whoever runs the mode repository.
- fs2_open itself and VP files have no clue about versioning. Versions had to be handled by the "descriptor" files too. When you have mods from different sources (the main repository, and some updated or unsupported mod repos), you get conflicts. Solving these conflicts is a PITA.
My idea was to have multiple "repositories" of JSON files that contain information about what versions a mod has and where they are available, those would be what you call the descriptor files.
I don't understand what you mean by by "rebuild files structure from scratch". Mods have a very well defined file structure and any released mod will most be distributed as VP or later 7zip/Zip so the installer would only have to unpack the files somewhere and set the correct -mod paths. Could you explain what you mean by that?
Versioning is a problem but for our limited application we probably only need to handle "new version available? Download the whole thing!" although being able to have incremental patches would be a nice thing. Now that I think of, it shouldn't be that hard to implement...
- Some fs2_open command line flags are undocumented (especially anything related to sound). You can try to use OpenAL standards, and fs2_open behaves in a completely differend, or completely weird way. Development process becomes a game of trial and error, much more than it should.
- Same applies for saved settings : different defaults apply to Windows/*nix platforms. I believe it's fixed in trunk, with Windows builds switching to .cfg files, instead of using the registry.
Yeah, most of the launcher stuff is undocumented but I tend to just take a look at the FSO source code anyway so that is not a problem for me.
The configuration is more or less easy to handle, you just need to implement the different backends (registry on Windows and .ini files on other platforms). Currently every branch still uses the old way of saving/loading stuff from the config but I have a patch that moves everything to using .ini files and once that hits trunk I will gladly delete the registry stuff from this launcher :P
-
FYI as far as OSX is concerned, GTK# can make use of the quartz backend and does not need to use X11 (which is good as x11 doesn't even come with OSX anymore) However, I can tell you from experience that GTK# is a #$%@ to get bundled in an OSX app. It is possible, but it took me many months to wrangle xboard (http://www.open-aurec.com/wbforum/viewtopic.php?f=19&t=52964) into one.
-
@m!m: I know mods have a well defined file structure once installed, but many mods are broken into different archives for easier downloading, and these archives follow no rule regarding their root folder (some have all files directly put in the root of the archive, others use a folder named like the mod as their root). It can even change from one archive to the other for a single version of a mod. It gets even worse for mods not using VPs.
That's not a big issue, but you have to keep that in mind when you build your JSON descriptor files. The only reliable information you can feed your mod installer with will come from these files, because you can't trust what's in the archives.
How do you plan to handle conflicts ?
Let's say A mod needs an old version of fsport, and B mod needs a newer version of fsport. Both versions of fsport share the same name, and are supposed to install in the same folder.
ngld and I thought about mod.ini rewriting and alias creation but it would have been messy (imagine fsport_33 and fsport_34 aliases, and mod depending on these having their "primary" mod list rewritten to fsport_xx on install)
To me, the only "clean" way to solve this is to build a better mod selection logic inside fs2_open.
Each mod should know it dependencies (min and max version of said dependencies too), and pass them to fs2_open. fs2_open would then search the root folder for the right mod version (whatever the mod folder name is), and output a clear error in some cases (dependency not found, dependency too recent or too old). Each mod should come with a "version file" containing a version string, so different versions of a same mod could be installed in parallel.
-
@m!m: I know mods have a well defined file structure once installed, but many mods are broken into different archives for easier downloading, and these archives follow no rule regarding their root folder (some have all files directly put in the root of the archive, others use a folder named like the mod as their root). It can even change from one archive to the other for a single version of a mod. It gets even worse for mods not using VPs.
That's not a big issue, but you have to keep that in mind when you build your JSON descriptor files. The only reliable information you can feed your mod installer with will come from these files, because you can't trust what's in the archives.
Now I understand, yes that is an issue but that can be handled by the JSON files.
How do you plan to handle conflicts ?
Let's say A mod needs an old version of fsport, and B mod needs a newer version of fsport. Both versions of fsport share the same name, and are supposed to install in the same folder.
ngld and I thought about mod.ini rewriting and alias creation but it would have been messy (imagine fsport_33 and fsport_34 aliases, and mod depending on these having their "primary" mod list rewritten to fsport_xx on install)
To me, the only "clean" way to solve this is to build a better mod selection logic inside fs2_open.
Each mod should know it dependencies (min and max version of said dependencies too), and pass them to fs2_open. fs2_open would then search the root folder for the right mod version (whatever the mod folder name is), and output a clear error in some cases (dependency not found, dependency too recent or too old). Each mod should come with a "version file" containing a version string, so different versions of a same mod could be installed in parallel.
I was thinking of completely deprecating the usage of mod.ini files. The JSON mod descriptors have all the information needed to get a commandline to work. To handle conflicts I would install the mods into a folder structure like {modname}/{version}. The installer then could use that folder structure to construct the -mod commandline argument. I already build the mod handling with something like that in mind.
Integrating the mod selection logic inside FSO doesn't sound like a good idea as that is something the launcher should handle as implementing that logic inside FSO would take a lot of work and the launcher has all the information anyway.
-
How do you want to write / generate those JSON files and what are they going to contain?
I wrote a python script which takes Goober's installer text files, parses them, downloads the mod archives and generates JSON files which contain all mod dependencies, file checksums and so on.
If you want to make the launcher multi-platform you'll have to look out for path case issues: Most mod developers use Windows which is case-insensitive. If you install the MediaVPs, you'll end up with MediaVPs_2014 and mediavps_2014. The mod installer has to choose one of those folders and move all files inside that one. Goober's installer doesn't handle this correctly.
One problem left in fs2mod-py is dependency tracking. The mod.ini files list mod folders like "blueplanet" but you never know exactly which version is required and which FSO version is needed to run the mod.
-
Ideally the new format would be created based on a discussion in the community.
What I am currently thinking of is at least:
- The user facing name of the mod
- A description
- An unique ID that can be used for installation folders
- The current version in the Major.Minor.Release.Patch format to handle versioning consistently
- A list of mod IDs that this mod depends on along with a version specifier, either minimum and maximum or some special format (I like the format Composer (https://getcomposer.org/doc/01-basic-usage.md#package-versions) uses but that may be a bit too complicated to implement)
- A list of URLs where the files of the mod can be downloaded along with mirrors and checksums
- I'm not really sure about this but maybe some sort of optional script to do special things on installation
While I don't think that case issues is something the launcher/installer should handle as that is a problem of the mod author I think extracting everything to lower case file names should not be hard to do.
Dependency tracking would be included in the JSON file along with information about what version of the dependency is required.
-
We need a common way to support for optional files in the mod (compatibility packages, hi res stuff, cutscenes...).
EDIT : we also have to add fs2_open compatibility information, with "branches" (standards fso builds, blue planet builds, diaspora builds...) and min/max versions for each branch. Installers should check if a compatible build is installed on mod install, and if it's not, download the latest compatible build and create launcher settings for the mod, with the right build.
At this point, it becomes too complicated for good old .fs2mod files, I want to drop them in favour of the new common JSON files.
EDIT 1.5 : some optional files will need the ability to override fs2_open build compatibility. Let's say a modder adds a new shader pack which only works with newer builds, this feature would be needed.
EDIT 2 : If we still allow mods to be packaged in weird ways (all kinds of archive formats, all kinds of file structure with no clear relation to fs2_open mod folders standards), the new JSON should store the final path for unpacked files.
Writing a php toolbox to manage these repositories shouldn't be a problem. With Symfony framework, I would only need the regex to make sure every bit of information we store is ok. Then generating JSON from a template would be easy.
-
with "branches" (standards fso builds, blue planet builds, diaspora builds...)
Except both Blue Planet and Diaspora run on standard builds... :nervous:
-
with "branches" (standards fso builds, blue planet builds, diaspora builds...)
Except both Blue Planet and Diaspora run on standard builds... :nervous:
They do now but they used to have custom builds because SCP couldn't keep up with official stable releases. It is not safe to assume that mods/tc's use official builds every time.
-
Ok, new try : Wing Commander Saga ?
Last march, before I took a break from working on fs2mod-py, BP was still using custom builds. Since then, I updated BP, but as I run on svn trunk builds I didn't check standard builds.
I don't have anything to add to what Fury said. Mods using different branches happened before, it may happen again. Better be on the safe side.
-
WCS doesn't count. They forked the code-base. For all real-world purposes, they are no longer using SCP at all.
-
And there's also an alpha release of a port of WCS to FSO: http://www.hard-light.net/forums/index.php?topic=85375.0
-
And I'm working improving said release so that someday one will be able to run WCS on regular FSO builds without a hitch.
-
To be honest, I'd rather see us get together and actually plan what we're going to do about a launcher properly before people run off and start coding them. Cause this sort of nonsense is exactly what happened last time.
-
I have been updating the launcher over the past few months and now I think I have implemented most features required for a standard launcher.
There is currently a discussion (http://www.hard-light.net/forums/index.php?topic=86364.new#new) about what format to use for mod package management which I want to support when there is an agreement on what is needed.
To be honest, I'd rather see us get together and actually plan what we're going to do about a launcher properly before people run off and start coding them. Cause this sort of nonsense is exactly what happened last time.
It's fairly obvious what a simple launcher needs to do but I see your point. I would like to see a discussion about what the launcher should be able to do and how it should do so. If anyone has suggestions I would be glad to hear them and implement any changes that can improve the launcher.