Hard Light Productions Forums
Modding, Mission Design, and Coding => FS2 Open Coding - The Source Code Project (SCP) => Topic started by: General Battuta on December 24, 2013, 11:43:04 pm
-
I've said this before but I think the forum metrics from the past year only make it more urgent: we must have an end-user-friendly install solution built to professional standards if we are going to maintain or grow the FreeSpace community.
It must be self-contained. It must not require reference to any outside website or list of instructions. It must be legal. We need to do everything we can to stress-test it and work out any kinks. We need an installer that is useful not only to power users but that works for the evangelizer, the newcomer, the skeptic.
Goob's been working on a solution to the problem, but for whatever reason (and it may be a very good reason), that effort is not complete. So please, let me renew the plea: what can we do to achieve this goal?
I think this is the single most important thing we can do for the future of HLP and FreeSpace. If we cannot achieve it, I think the only alternative is for individual mods to start shipping preconfigured standalones.
-
I'm kind of wondering if Nexus mod manager can provide a suitable solution for this. I know it doesn't fit all your criteria, but it has built up a massive user base despite that and it might be able to deal with all the necessary modifications to retail FS2 with minimal oversight. I meant to look into this further but got sidetracked by Christmas.
-
I'd like to take a look at integrating or cloning innoextract with Python. If that can be done, it'd be fairly straightforward to create standalone installers that work on Windows, Mac, and Linux. (Why Python? No reason, other than that's what I'm most familiar with. It's pretty easy to make standalone executables that don't need Python already installed.)
I think the biggest aid would be a more centralized repository of mods, SCP builds, tools, and standalone TCs with a small index file the installer could download when run. I'd be willing to host that myself, since I have a server I'm otherwise not doing much with.
-
I still think this functionality should be built into the launcher. That was the plan from day one after all and I believe it's still on the roadmap.
-
We should also agree on a file format to manage the mod versions so we could host some sort of mod repository with informations like mod version and install commands in one or more file.
Doing this with a widely supported format like JSON or XML will make is easy to integrate it into the launcher. That way we could also add dependency tracking to each mod to make sure everything is installed for the mod to work correctly. Hosting the actual files could be done somewhere else, we would just include the URL of the download so hosting the mod files on a separate server would not be a problem.
-
I think there's value in having a separate Freespace2/retail installer for OSX/Linux (i.e. to mirror the retail installer for Windows), no matter what form the other installer takes (Goober's/wxLauncher/new(?!?). Hellzed has made some good progress (http://www.hard-light.net/forums/index.php?topic=86364.0) already on this, with the script written in bash it should probably run on OSX as well as Linux. This could even be run by the other installer on Linux/OSX.
Of course, that is a secondary concern, the primary concern is mod downloading/dependency resolution/installation/verification. If wxLauncher was intended to do this then perhaps we could get a "report" from the wxL devs on the current state of wxL mod handling, what the current plans are, and info on how non wxL devs can contribute?
edit: clarify / refine the post
-
Battuta is right, this is the most pressing thing at this point
90% of the people will not install something if it requires them to read more than 2 lines
-
installing fso takes a lot more than '2 lines' and frequently results in users getting bombarded with useless errors we have to decipher for them
-
How is it supposed to be stand alone and legal? At least if it is going to result in a functional game at the end of the process.
-
i don't think somebody is talking about standalone
rather a case of reducing a noobs usual way of installing stuff from:
- install fs2
- read through confusing instructions, get an up to date launcher and copy all the files by hand
- read through confusing instructions, get an up to date version of fso and copy all the files by hand
- read through even more confusing instructions, get the up to date mediavps (several seperate downloads, a noob will have no idea what he needs)
- select the exe and mod you want to use (read: mediavps) in the launcher
- have crashes because you installed freespace into a directory with a too long file path
to:
- install fs2
- install everything fso from one .exe, and be ready to go for playing with the latest stable fso and the latest mediavps
(the installer should come with a BIG warning that freespace can crash from too long file paths)
-
I agree with General Battuta (and many things said in later posts).
@niffiwan : i really hope my current work *will* be superseded by a graphical tool included in a launcher... I'm not a real programmer, I have close to zero experience, so I'm a bit worried about the quality of what I'm doing. And I need comments, advice, help for testing... Even comments and advice from people who do not use Linux.
I think we really should go for something a bit different from a mod installer linked to a repository : people should have the choice to add mods from other sources and still manage these from the mod installer. I'm not sure about my current implementation, but it could be a starting point for a discussion about new ways to do this.
-
Turey's old installer is was brought me to (and kept me in) the FSO community. It installed everything I needed, and if I recall correctly, I could even choose which mods to install and/or update. At the time I first used that, I would have been overly-intimidated to try to follow the current manual procedures, but it was easy enough to use that I got hooked and stuck around. Yet, as fightermedic noted, I still get confused sometimes about which files I need to download or I am uncertain whether I am using the latest version of something.
-
I'm an FSO power user, and I find the current install process an active turn-off to getting my friends involved...never mind them getting themselves involved. We need something at least as easy as Turey's.
Let's assume that Goob's installer is a long-term solution that will not be finished in the next couple years (sorry Goob, take it as a testament to your monumental ambition). Does anyone want to put forward a proposal as to what we can do? Could a single huge archive make an effective solution to drop onto an existing FS2 install? It'd be impossible to maintain.
-
i don't think somebody is talking about standalone
rather a case of reducing a noobs usual way of installing stuff from:
- install fs2
- read through confusing instructions, get an up to date launcher and copy all the files by hand
- read through confusing instructions, get an up to date version of fso and copy all the files by hand
- read through even more confusing instructions, get the up to date mediavps (several seperate downloads, a noob will have no idea what he needs)
- select the exe and mod you want to use (read: mediavps) in the launcher
- have crashes because you installed freespace into a directory with a too long file path
to:
- install fs2
- install everything fso from one .exe, and be ready to go for playing with the latest stable fso and the latest mediavps
(the installer should come with a BIG warning that freespace can crash from too long file paths)
(1) was **EXACTLY** my experience with my current, 2 month old install.
And I had installed FSO previously! A couple of years ago, but still - I already knew what the MediaVPs were.
To me, this issue takes precedence over nice graphics, scripting, AI, upgraded models, etc.
If we cant get new people playing fairly easily and quickly, then whats the point?
BH
-
What about focussing on first things first?
ie,
1) FS/FSO installer, with appropriate stable MVPs.
Get a stable, up-to-date installation of FSO running, FIRST.
Never mind mod repositories, just get the damn game and its upgrade installed, FIRST, ASAP.
Hell, a simple batch script, or even NSIS to just GET THE GAME & ITS STABLE UPGRADE ONTO THE COMPUTER AS EASILY & QUICKLY AS POSSIBLE.
THEN move on to forever discuss/sporadically design/roughly build a mod friendly/universal installer.
BH
-
Geez guys, at least wait until after the holiday break before posting this. :p
-
I will take that statement with guarded optimism. :yes:
-
Cross platform modding tools and the WxFred port are more important IMHO. Installer support is nice, but once WxFRED and all modding tools are ported to Linux/OSX you should submit a post to slashdot and other news websites. It'll help promote/grow the community. You'll also be the first 3D space game in Linux history to have a working level editor. Submit screenshots/videos of the engine upgrades along with the news of the editor port in the links you give to phoronix.com and slashdot to get better coverage. Phoronix.com might make a negative comment about the engine's tech level, so if you can, upgrade it so it supports some features from OpenGL 4, even if no one really uses them. That will make the game get more favorable reviews, they will compare the game from 2002 as a game that can go toe to toe with the latest company tech from 2014 as most linux games coming out right now barely support OpenGL3 and there's almost no OpenGL 4 titles.
-
Even as a content creator with a big stake in improved, cross-platform versions of the tools I use, I don't think content creation tools are nearly as important as content consumption tools.
In your specific scenario, we can get all the positive press we want, but when it turns out that the game's an obnoxious bear to install, that press will curdle.
Additionally, getting the installer working is a lot easier than WXFred.
-
As a developer, I would also disagree. wxFRED is an important part of our toolset, yes, and a priority for development. But making it easy for people to actually consume the content we're producing is far more important.
-
Indeed. One of the reasons the installer is taking so long is the amount of bulletproofing that needs to be done so that it's suitable for use by the general public.
-
hope it all happens soon so i can play the new Jad. but on a unrelated but still related note...last time i used tureys i was able to get 3.6.14 and i think the RC for 3.7 so maybe it is being updated?
-
You can install JAD without using any kind of fancy installer.
-
i don't think somebody is talking about standalone
rather a case of reducing a noobs usual way of installing stuff from:
- install fs2
- read through confusing instructions, get an up to date launcher and copy all the files by hand
- read through confusing instructions, get an up to date version of fso and copy all the files by hand
- read through even more confusing instructions, get the up to date mediavps (several seperate downloads, a noob will have no idea what he needs)
- select the exe and mod you want to use (read: mediavps) in the launcher
- have crashes because you installed freespace into a directory with a too long file path
to:
- install fs2
- install everything fso from one .exe, and be ready to go for playing with the latest stable fso and the latest mediavps
(the installer should come with a BIG warning that freespace can crash from too long file paths)
(1) was **EXACTLY** my experience with my current, 2 month old install.
And I had installed FSO previously! A couple of years ago, but still - I already knew what the MediaVPs were.
To me, this issue takes precedence over nice graphics, scripting, AI, upgraded models, etc.
If we cant get new people playing fairly easily and quickly, then whats the point?
BH
I agree!!!!!
If growing the community is a goal, the installation process is the most important step. Think of the foreign people with lame English literacy. I myself - as an Austrian speaking a quite good English - had a hard time with the manual install, since there is no automation available at the time. I'm not speaking of translation. I think the most common FS player (from whereever) is able to understand the game and to handle a step-by-step windows-installer in English.
Happy new year
tom
-
Bah I've regarded this issue as a big ****ing mess for years now. My strategy to get friends playing this is a lot more illegal and all the **** you can throw at me (and I won't correct you on the morality of it), but at least it functions. I just make a copy of my FS2 folder (though a lot simpler and leaner w/out the rubbish I have) and give them a pen with it all. I've tried the "correct" way with them, but the shenanigans are so large and so paramount that even with the astonishing level of voluntarism and generosity of all the forum's people with errors and bad installations they just quit the next day. As far as they can tell, it's just not worthy that much trouble, and they go back to play GTA or other ****. And I can't blame them. Who can?
-
I can.
-
This is how I see things :
https://www.dropbox.com/s/801qnuxhhdrsnz2/hlpcat.zip
(unpack the HTML page and its data directory somewhere, and open it in any modern browser)
(imagine this page with a HLP header and a menu bar to select a mod type : "Great War/FS2/Post Capella/TCs...")
Next generation FreeSpace mod database should enable a "click & play" feature. Browse mods as a slide show, click "play !", and FreeSpace 2 will launch (using the latest options profile used with selected mod). If the mod is not installed, an installer launches, downloads the mod files, and then we resume FS2 launch process.
What would need to be done :
- a HTML+CSS template to present mods.
- a simple mod database referencing a nice HD screenshot, mod title, author, mod description, size...
- a PHP script to generate the final pages.
- a tool to launch when a fs2:// protocol link is clicked, launch the game if the mod is installed, launch the installer if it is not installed.
No need for a unified installer/launcher. Just a modern browser would do the trick.
The drawback is that users would have to be online to launch FreeSpace with this method.
The workaround is a standalone webkit/Qt window and a locally cached list of mods.
-
I can.
For what do you blame them, exactly? For wanting to play a game that they remember from ages ago, or that they read about, with minimal issues? For being interested in the stuff we've produced, but not in learning how to tame a game and its various technical issues?
-
For not be willing to take 5min to follow simple instructions.
-
It's not 5 minutes, and it's not that simple. The number of people willing to invest the mental work necessary to read, understand and follow the guides is not that large, and even a good portion of that set will resent having to do it just to have a bit of fun with a game.
Games, IMHO, should not be made for power users. Games should be easy to access, easy to get going, and easy to patch. Right now, FSO is only partially there. Demanding that your audience engage in what is for all intents and purposes the digital equivalent of housekeeping is not going to make said audience want to play the game.
I don't know about you, but I want the maximum number of people to be able to enjoy this game and all the stuff we've produced over the years. I want the install process to be as easy as "Step 1: Run GOG installer, Step 2: Download this executable and let it patch the game up for maximum awesome".
Personally, I am frequently astonished at the amount of problems some people have to get this thing going. But then I remember that most of them probably do not have to deal with Computers everyday, or even care about learning how to operate them properly. And that's totally fine! Just as knowing how to write a novel should not be a prerequisite for enjoying a book, knowing how to mod FS2 should not be a prerequisite for enjoying the game.
-
I can.
Blame them away, see if they care one yota. These are smart people, and they are very fast at calculating the sort of work it's ahead of them. First they will have to understand the kind of problem they are facing, then they will have to understand if the solutions that are sort of given in a fragmented, scattershot way have any basis in current reality, if they still work or not, what webpage giving instructions is reliable or not, and what kinds of mistakes they are allowed to do or not.
What is a "launcher" (remember we are not in the nineties anymore), now you telling me I shouldn't download 3.7.0, the stable latest build if I want to have the latest mediavps? Openal? wtf? What are these words you are talking about?
I had enough of this, let me play this other game that seems a lot more 2014, fun and easy to install. See? I just need to click "Install" and press "accept". That was easy. Now bang bang here I go! And stop whining how fun that 90s game is. It's from the bloody 90s, can't be that good anyway. It's just too much work, and that kind of **** I do already too much of it during workhours. I want to have some fun, not bloody scream at the screen.
-
I don't think there can be any reasonable doubt that the FSO install process should be:
1. Press 'Download FreeSpace Open'
2. Press 'Install FreeSpace Open'
3. Click 'FreeSpace Open Launcher'
-
Well as a retired IT professional that recently decided to get back into space flight sims I can tell you the install process for FS2 Open is anything but clear and straight forward. There are so many old threads in this forum and changes over the years as to what to do and how to do it you can spend weeks searching and reading. I have tried to get some friends (some of which are also IT) to try this out and when they see the mess they pass.
I have also recently gotten back into flight sims. I purchased IL-2 and was invited to join a squad. Well they have the right idea. Someone wrote an app that automatically updates and downloads and installs all patches and high resolutions mods. No messing around. And since everyone's final install is the same (same mods) playing together online becomes much easier.
I totally agree with those that feel this is priority #1. I applaud all those that have worked on this project and its campaigns (quite amazing really) but unless you come up with a better approach to installing and upgrading you will never see the full extent of users who would get to use all your hard work.
There seems to be a growing interest in space sims once again. Some of these projects are years away. Others really do pale compared to what you guys have done. Others will have a bad business model (like free to play but it will cost you hundreds in the end). Freespace 2 Open is there now.
Here are some of the things I feel need to be address to make Freespace 2 Open attractive to the general sim player.
1) An automatic installer and updater (this is the 21st century not the 90's)
2) Build in cockpits
3) Fix the joystick limitations so it can handle more advanced setups without the need for PPJoy
4) Something like Hyperlobby to support online play
Thanks again for all the hard work. It would be great to see even more people enjoying it.
-
I will admit, I haven't installed FS2 on my new computer, whilst I'm sure I could figure out the install process, as I've done it enough times before, it's just the thought of taking on the whole thing again that puts me off.
I think the problem may be that the launcher was developed alongside the code and media updates, so it changed to adapt to their needs rather than initially having been written to support them, and that resulted in a method that feels 'clunky', and I can understand how that might put potential new players off.
-
So, any criticism about some web based installation/modding/launcher experience ?
[attachment deleted by an evil time traveler]
-
If you post a candidate I'll happily give my thoughts.
-
Only if it works on more than just Linux.
-
Only if it works on more than just Linux.
If I understand it correctly it is just a web page so it will work anywhere where you can have a web browser but we will still need some sort of installer tool that can handle the special protocol.
-
That's my issue. The only languages I know are Bash and php. I know some HTML and CSS. But I can't say I'm really good at coding in general.
@m!m: exactly !
Hopefully, the utility script called after clicking the fs2:// protocol links would not be a full installer/launcher. It could hook within wxlauncher for launching the game, and a real installer for installing the game files and the mods. The only real task of this piece of code would be to detect if a mod is installed or not and pass the right arguments to the right software.
wxlauncher would need to have a hook allowing to bypass most of its GUI and skip directly to its fs2_open arguments flags parsing before launching the game.
Of course, it does not do anything to solve the initial install issue...
-
I always wanted the launcher to do all this. Having only one entry point to the game simplifies things a lot.
Basically have the launcher work as the online installer for mods. Make it capable of accepting 7z files too and we solve the problem of people who don't want an online installer.
-
Technically we wouldn't need much, some sort of mod repository in an easily machine-readable format (json (probably the best if we want a web interface), xml, ...) and an additional wxLauncher tab. On installation wxLauncher could also register itself as the handler for our fs2:// protocol and handle mod installation from the web that way.
-
I proposed the idea years ago but everyone ran off and did things their way. Which got us into the mess we're in now. If we're going to do something, I strongly suggest we get our **** together with a plan we all agree on first and only then start coding.
-
At the risk of sounding like a broken record, I'll say again that the Installer really is almost ready.
I know people will respond that it's been years without anything released yet, and they'd be correct, but I'll remind them that people said the same thing about ST:R. :)
-
On a scale of urgency I would rate the release of a proper launcher slightly above that of ST:R :p
-
At the risk of sounding like a broken record, I'll say again that the Installer really is almost ready.
I know people will respond that it's been years without anything released yet, and they'd be correct, but I'll remind them that people said the same thing about ST:R. :)
GOOBER ROCKS!!
oh wait that's the other thread where I'm supposed to use caplock.
-
On a scale of urgency I would rate the release of a proper launcher slightly above that of ST:R :p
Very slightly. If you consider an increment of a quintilliontude to be marginal. ;)
-
Well there have to come a point where you need to draw a line in the sand and say the development and changes are frozen then work on upgrading the better campaigns to use those changes. This falls into the reason why people shy away from all this. This campaign only works with this Build and or this MediaVPs. Its great to see all these addons but there needs to be some order to all of it. It might not be the first priority but it is directly related to standardizing and polishing the experience for new comers.
-
I don't think there can be any reasonable doubt that the FSO install process should be:
1. Press 'Download FreeSpace Open'
2. Press 'Install FreeSpace Open'
3. Click 'FreeSpace Open Launcher'
unfortunately it cannot be unless you are assuming a preexisting FS2 install.
-
There is no possible way to avoid assuming a pre-existing FS2 install.
-
Yes there is.
Like I said years ago the installer should be able to send people to GOG or point them in the direction of the standalone games. Expecting new users to know they need to pay to download the game from GOG by themselves, THEN come to HLP is a rather unnecessary hurdle.
-
Of course, but you still can't avoid assuming a pre-existing install before installing the MVPs and MVPs-based mods.
-
Any FreeSpace 2 installer should either :
- ask where GOG version is installed before upgrading the install, and point to GOG to buy the game first if it isn't installed anywhere
- set up a base install with fs2_open and launcher builds, and if its missing retail data, point to GOG.
The second option is much better for Linux and OS X users since they can't install a native version from GOG (and the FS2 installer has to use innoextract to get retail data).
(I have attached a screenshot of how I handled this issue in the bash installer I made)
[attachment deleted by an evil time traveler]
-
I don't think so. I think the best approach is to offer to install any TC available, and offer the option of upgrading an existing FS2 install (and if no FS2 install can be found, exit), not create a situation where the user may end up confused over whether or not everything's fine. I can just see the support posts that would come from that: "Hey, I used this installer thing, and now I get an error that says 'Web Cursor bitmap not found, this may be due to you not having FS2 installed', what's up with that?"
-
One solution: Bundle innoextract with the installer. Ask the user to select their GOG installer, with a short explanation about how FS2_Open works on top of FS2 rather than instead of and instructions and a link to GOG's website if they haven't downloaded it yet. Would also have the option to select their FS2 install directory if they already have it installed.
-
That also works, I suppose. The only thing I really want to avoid is a situation where the installer downloads FS2 mods without having verified that FS2 is actually installed.
-
@Kopachris :
I was thinking about that. Currently, the script uses the distro's own packaging system to install innoextract if it is not installed yet (It works on Ubuntu but I'm missing info for other distros). I'll change that to download a distro agnostic version of innoextract and store it with the config files of the script.
Maybe a fully unified installer would even use innoextract on windows.
Currently, to avoid redundancy with Goober's installer, I'm looking into splitting the script into small functions, port the whole thing to python to prepare windows compatibility, and dropping the "pseudo GUI" thing.
My ultimate goal would be to write a cross-platform command line utility for managing FS2 installs with a syntax similar to apt-get. I don't know if anyone would be interested, but sounds fun to me.
$ fs2tool --install-from-gog <your GOG installer>
$ fs2tool --install-from-cd <your drive with retail CD>
$ fs2tool --add-mod <mod descriptor or mod descriptor url>
$ fs2tool --update-mod <installed mod name>
$ fs2tool --uninstall-mod <installed mod name>
$ fs2tool --remove-mod <installed mod name>
$ fs2tool --get-binary [stable|nightly]
-
Hellzed, I think we have similar goals, although mine are a bit more extensive. If you already know Python or can learn it fairly easily, I have some ideas of my own I've been thinking about, using PyInstaller to package installer, mod manager, and innoextract into a single binary. Care to collab?
-
Uhhhh
guys
While I applaud your enthusiasm, you are missing a few important points here:
1. We do not want multiple ways of achieving the same goal. We especially do not want multiple infrastructures that we have to maintain.
2. The end goal is to integrate wxLauncher and the installer, so that the interface to getting FS mods installed is the same across all platforms
-
@Kopachris :
At the moment, I would prefer to wait before starting anything.
There are 3 big unanswered questions :
- How well do Goober's installer and wxlauncher integrate (if they integrate at all, apart from sharing the same FS2 directory) ?
- How does Goober's installer manage downloads ? It will be a repository, but that's the only thing I know.
- How does it handle GOG install ?
@The E:
1) I do not host whole mods, only mod descriptors, files that take 5 minutes to create and 2 minutes to update, from mod release posts on the forum. Basically, it takes more from a manual install automated with a bash script than a "real" full blown installer as what's Goober is doing. Maintenance is only on my side, and once the script is stable, it will be a piece of cake to manage these small files.
Anyway, I hear what you mean, that's still a bit redundant.
2) That will be awesome when it's done. I have no intention to code something that does exactly the same thing. I wish I could hook into these pieces of software to come up with a nice web interface for managing FreeSpace using these tools. A bit in the way Steam does things : they both have a website and a client, and clicking "Play" in both interfaces launches the game in the same way.
-
1) I do not host whole mods, only mod descriptors, files that take 5 minutes to create and 2 minutes to update, from mod release posts on the forum. Basically, it takes more from a manual install automated with a bash script than a "real" full blown installer as what's Goober is doing. Maintenance is only on my side, and once the script is stable, it will be a piece of cake to manage these small files.
Anyway, I hear what you mean, that's still a bit redundant.
Yeah, I understand that, but you do realize that you're setting yourself up to be the single point of failure for this whole venture, right? Way I would do it (note that I do not know how Goober's planning to solve this) would be to use the HLP wiki as the source for the mod metadata; a place I can be reasonably sure will exist for the foreseeable future.
2) That will be awesome when it's done. I have no intention to code something that does exactly the same thing. I wish I could hook into these pieces of software to come up with a nice web interface for managing FreeSpace using these tools. A bit in the way Steam does things : they both have a website and a client, and clicking "Play" in both interfaces launches the game in the same way.
Which gets me to the whole "install a web handler thingie". For several reasons, I really dislike this. Not the idea in theory, the idea is great, but the actual realities of it all, with all the security checks that have to be done to ensure that noone does bad things with it. I do realize that, in the grand gaming ecosystem of the internet, we're a very small community and as such less likely to be targeted than Steam for example, but I still dislike the idea of opening ourselves up to attack like that.
(This is probably paranoia on my end)
-
My version of the installer does check to see whether FS2 is installed in the destination directory, and provides a friendly notification message if not. It also checks to see if there are any miscellaneous VPs in the folder that don't belong. It checks to see whether downloaded files can be validated against an MD5 or SHA hash. And several other validation steps.
Anyone can check the current status by downloading the source from SVN here:
svn://svn.icculus.org/fs2open/trunk/fsoinstaller
For mod metadata, it uses a very similar text format to what Turey used. It can be easily modified and maintained by mod teams and I even plan to make a GUI install builder to generate them after I finish the installer itself.
Right now, everything is hooked up and working except for the actual queueing of downloads. It should only take a weekend of work (or less) to finish. This weekend is unavoidably blocked off, but I fully expect to have an end-to-end working installer released by January 19th.
-
@The E:
That's why my Dropbox url is not hardcoded in the script. Each mod descriptor can have its independent update source url. I also wrote a tutorial about how to make these mod descriptors. Even if my dropbox goes down now, anybody can replace me.
I didn't feel like posting all this stuff on the wiki, because it was an untested one man project (at the moment only 3 people have successfully tried it). It would have felt like "squatting".
I checked the security of this method (custom protocol) :
Steam had security issues because they put undocumented broken functions in their released client code, for example allowing it to silently download remote files (terrible idea indeed).
The whole point of using a protocol is that it doesn't download anything : it is limited to the features we put in it. The safe way to do it is what they did with the APT links in Ubuntu. When you click an APT link, you don't download the DEB package directly from the website where you found the link, but you actually start a download from your *distribution's official repository*, thanks to a small helper script.
The way to safely do that with FS2 is to make links like fs2://mediavps. The installer would launch and only get the "mediavps" string. Here we perform the usual checks in order to avoid XSS-like issues (escaping strings...). If the end result string corresponds to something we have in the repo, we install. Else, we display an error.
Sounds safe enough to me. Modern browsers have fixed custom protocols safety issues some time ago (mostly dealing with protocol handler re-association, silent downloads and other ugly stuff... But here, we are not directly concerned since its a browser/user side issue).
-
@Kopachris :
At the moment, I would prefer to wait before starting anything.
There are 3 big unanswered questions :
- How well do Goober's installer and wxlauncher integrate (if they integrate at all, apart from sharing the same FS2 directory) ?
- How does Goober's installer manage downloads ? It will be a repository, but that's the only thing I know.
- How does it handle GOG install ?
Fair 'nuf.
-
For mod metadata, it uses a very similar text format to what Turey used. It can be easily modified and maintained by mod teams and I even plan to make a GUI install builder to generate them after I finish the installer itself.
Can we please change the format to something that can be easier parsed by existing tools after you finished your initial version?
-
@Goober :
Thanks for the answer !
I'm not sure I understand this right, do you mean the "repository" only hosts metadata (with "links" pointing to the real place where mods archives are hosted), or it hosts both metadata and the full mod archives ?
Is the available mod list limited to a single official list, or is it possible to add other update sources ?
-
For mod metadata, it uses a very similar text format to what Turey used. It can be easily modified and maintained by mod teams and I even plan to make a GUI install builder to generate them after I finish the installer itself.
Can we please change the format to something that can be easier parsed by existing tools after you finished your initial version?
Turey's format is nonstandard, but it's perfectly parseable. And I've already written a bunch of parsing utilities that are in the fsoinstaller project and working. (In fact, those were the first classes I created.) They're in GPL, so you can use them in another project if you need to.
@Goober :
Thanks for the answer !
I'm not sure I understand this right, do you mean the "repository" only hosts metadata (with "links" pointing to the real place where mods archives are hosted), or it hosts both metadata and the full mod archives ?
Is the available mod list limited to a single official list, or is it possible to add other update sources ?
There is a repository on fsoinstaller.com that provides a list of URLs. Each URL points to a text file that describes a mod, its description, and its required files.
When the new installer is ready, it will add the capability of repository mirrors on other places such as scp.indiegames.us.
-
The problem is that those utilities are written in Java. If we want to integrate this repository into a web service or another external application we would have to write the whole parser again in Javascript, Python or something else.
IMHO changing this format at some point after the initial release of the installer is crucial if we want to pursue this multi-application mod installer solution any further.
-
The problem is that those utilities are written in Java. If we want to integrate this repository into a web service or another external application we would have to write the whole parser again in Javascript, Python or something else.
No we wouldn't. First of all, web services can be written in Java. Second of all, there are transcoders that could re-emit the code in JavaScript, with no rewriting necessary.
IMHO changing this format at some point after the initial release of the installer is crucial if we want to pursue this multi-application mod installer soltuion any further.
Why?
-
No we wouldn't. First of all, web services can be written in Java. Second of all, there are transcoders that could re-emit the code in JavaScript, with no rewriting necessary.
That's all possible but this will again limit our possibilities, we will require a web server that has a Java environment which is not as common as a standard PHP server and having to use code transformers to get something as easy as a parser for our format is inefficient if we have the possibility to use an existing data format with existing parser solutions.
Why?
Mostly to be able to keep the web services/applications we will have to create maintainable. Keeping the old format just because we have a working solution is perfectly fine for one application but as soon as we have more of them we will have to maintain a parser for each used language and the SCP is already lacking coders.
-
I am not sure I understand that objection.
To my mind, it doesn't matter what the format is for this metadata, so long as it's a consistently defined format. If someone wishes to implement the installer in a different language (and really, I can see just one occasion where that is necessary, namely when integrating the Launcher and the Installer into one package), he or she will have to implement the parser anyway, regardless of how that metadata is stored. Whether you're parsing a text file, or loading JSON, or loading XML, doesn't make that much of a difference.
Secondly, what's all this talk about web services etc? I am not sure I understand how we suddenly need a web service to handle anything aside from simple file hosting.
-
You are right the format of the metadata doesn't matter but it matters if you have to write a parser for the file format from scratch or if you can use an existing library which will parse it for you, for example for C/C++ we don't have a parser for the current metadata format, if we were to use JSON we could just use jansson which is well tested.
The web service is just a secondary idea which would allow us to have some sort of online mod gallery with an underlying database which would allow us to dynamically generate the required metadata automatically while allowing mod creators to publish their mods/missions without the need for someone else to edit the metadata by hand. Please note that this is an idea I had and nothing that should be set in stone but it will make updating or releasing a mod much easier.
-
My two cents:
JSON is one of the best data interchange formats.
If the updater needs any server side scripts, the architecture is wrong.
-
JSON is one of the best data interchange formats.
"One of the most popular" != "one of the best", although it is certainly one of the easiest to parse.
-
If you compare the spec to e.g. XML you will see the beauty of simplicity:
http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf (http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf)
http://www.w3.org/TR/2004/REC-xml11-20040204/ (http://www.w3.org/TR/2004/REC-xml11-20040204/)
Also it solves the old problem of how to represent numbers very nicely.
-
If you compare the spec to e.g. XML you will see the beauty of simplicity:
That's true of comparing anything to XML. :P
I really dislike XML.
-
The question is what else do you want to compare it to?
INI, YAML
The beauty of JSON is that it has all features you need for most use cases.
With INI you often end up with custom parsers for values.
YAML is as bloated as XML.
-
/*quietly cheers*/ JSON! JSON!
The way I see it there are two basic use cases, a user gets the launcher/installer and A) has some form of FS installed already, or B) does not. for B there is also a sub use case of B2) user gets some form of FS. A is trivial and the expected use case, but B is likely to happen often, there are enough standalone TCs out there that it makes since that the original FS should not be a requirement to run FSO, it should be well within reason to setup the directory structure and install a few stand alone mods. The problem is B2, do you have the user install over any existing fs2 folder? have GOG/whatever FS2 install somewhere else and have the launcher import one into the other?
also, asking the user where something is installed should be the last option, it should be fairly easy to find any existing FSO folders automatically, they tend to be in only a handful of locations.
-
As good as all these ideas are, can we please have the new installer out first before we try to make it better (for dubious values of better)?
-
Well the new installer is pretty much up to Goober. So where is the need to wait?
-
@Bobboau :
In case B), we also have to think about Linux and OS X users : they cannot use GOG installer out of the box, wine (windows compatibility layer, used to perform a full windows FreeSpace 2 install) is overkill, and innoextract is perfect but has no GUI.
A complete FS2 installer should come bundled with innoextract, and provide a GUI for using it.
-
YAML is as bloated as XML.
And has some very ambiguous definitions of how it handles certain types of values, too. About the only thing YAML has going for it is that it's a lot easier for a human to read/edit than just about anything else, but that's it.
...So yeah, if you're comparing to XML, YAML, and INI files, JSON all the way.
-
I created a Java library which migrates the current installer data format to a proposed JSON format: https://github.com/asarium/fsoi2json (https://github.com/asarium/fsoi2json)
Here is the output when I run it with the current installer data: http://pastebin.com/7i71r8Kg (http://pastebin.com/7i71r8Kg)
It already includes preliminary dependency tracking but we will have to improve that with proper version detection.
-
:pimp:
I might be able to help you guys with this. I package applications as my regular job and work almost exclusively in AdminStudio/Installshield. I can build complex MSI packages to order and would be willing to make a nice professional install sequence for you if MSI format works for your needs?
Let me know/
-
I think the issue is that it needs to be an online installer, because so many of the individual components are updated so constantly.
-
I'm just going to close this, because there's no use having half a dozen people all duplicating each others' efforts. Especially not when, as I said, the FSO Installer is nearly ready to go.
My intended date for release is January 19th and I'm going to stick to that. We can also revisit Hellzed's and m!m's ideas for an online portal at that time.
And if it turns out I can't make that deadline for whatever reason, I'll reopen this thread.