[Goober5000]

My version of the installer does check to see whether FS2 is installed in the destination directory, and provides a friendly notification message if not.  It also checks to see if there are any miscellaneous VPs in the folder that don't belong.  It checks to see whether downloaded files can be validated against an MD5 or SHA hash.  And several other validation steps.

Anyone can check the current status by downloading the source from SVN here:
svn://svn.icculus.org/fs2open/trunk/fsoinstaller

For mod metadata, it uses a very similar text format to what Turey used.  It can be easily modified and maintained by mod teams and I even plan to make a GUI install builder to generate them after I finish the installer itself.

Right now, everything is hooked up and working except for the actual queueing of downloads.  It should only take a weekend of work (or less) to finish.  This weekend is unavoidably blocked off, but I fully expect to have an end-to-end working installer released by January 19th.

[Hellzed]

@The E:
That's why my Dropbox url is not hardcoded in the script. Each mod descriptor can have its independent update source url. I also wrote a tutorial about how to make these mod descriptors. Even if my dropbox goes down now, anybody can replace me.
I didn't feel like posting all this stuff on the wiki, because it was an untested one man project (at the moment only 3 people have successfully tried it). It would have felt like "squatting".

I checked the security of this method (custom protocol) :
Steam had security issues because they put undocumented broken functions in their released client code, for example allowing it to silently download remote files (terrible idea indeed).
The whole point of using a protocol is that it doesn't download anything : it is limited to the features we put in it. The safe way to do it is what they did with the APT links in Ubuntu. When you click an APT link, you don't download the DEB package directly from the website where you found the link, but you actually start a download from your *distribution's official repository*, thanks to a small helper script.
The way to safely do that with FS2 is to make links like fs2://mediavps. The installer would launch and only get the "mediavps" string. Here we perform the usual checks in order to avoid XSS-like issues (escaping strings...). If the end result string corresponds to something we have in the repo, we install. Else, we display an error.
Sounds safe enough to me. Modern browsers have fixed custom protocols safety issues some time ago (mostly dealing with protocol handler re-association, silent downloads and other ugly stuff... But here, we are not directly concerned since its a browser/user side issue).

[Kopachris]

@Kopachris :
At the moment, I would prefer to wait before starting anything.
There are 3 big unanswered questions :
- How well do Goober's installer and wxlauncher integrate (if they integrate at all, apart from sharing the same FS2 directory) ?
- How does Goober's installer manage downloads ? It will be a repository, but that's the only thing I know.
- How does it handle GOG install ?

Fair 'nuf.

[m!m]

For mod metadata, it uses a very similar text format to what Turey used.  It can be easily modified and maintained by mod teams and I even plan to make a GUI install builder to generate them after I finish the installer itself.

Can we please change the format to something that can be easier parsed by existing tools after you finished your initial version?

[Hellzed]

@Goober :
Thanks for the answer !
I'm not sure I understand this right, do you mean the "repository" only hosts metadata (with "links" pointing to the real place where mods archives are hosted), or it hosts both metadata and the full mod archives ?
Is the available mod list limited to a single official list, or is it possible to add other update sources ?

[Goober5000]

For mod metadata, it uses a very similar text format to what Turey used.  It can be easily modified and maintained by mod teams and I even plan to make a GUI install builder to generate them after I finish the installer itself.

Can we please change the format to something that can be easier parsed by existing tools after you finished your initial version?

Turey's format is nonstandard, but it's perfectly parseable.  And I've already written a bunch of parsing utilities that are in the fsoinstaller project and working.  (In fact, those were the first classes I created.)  They're in GPL, so you can use them in another project if you need to.

@Goober :
Thanks for the answer !
I'm not sure I understand this right, do you mean the "repository" only hosts metadata (with "links" pointing to the real place where mods archives are hosted), or it hosts both metadata and the full mod archives ?
Is the available mod list limited to a single official list, or is it possible to add other update sources ?

There is a repository on fsoinstaller.com that provides a list of URLs.  Each URL points to a text file that describes a mod, its description, and its required files.

When the new installer is ready, it will add the capability of repository mirrors on other places such as scp.indiegames.us.

[m!m]

The problem is that those utilities are written in Java. If we want to integrate this repository into a web service or another external application we would have to write the whole parser again in Javascript, Python or something else.
IMHO changing this format at some point after the initial release of the installer is crucial if we want to pursue this multi-application mod installer solution any further.

[Goober5000]

The problem is that those utilities are written in Java. If we want to integrate this repository into a web service or another external application we would have to write the whole parser again in Javascript, Python or something else.

No we wouldn't.  First of all, web services can be written in Java.  Second of all, there are transcoders that could re-emit the code in JavaScript, with no rewriting necessary.

IMHO changing this format at some point after the initial release of the installer is crucial if we want to pursue this multi-application mod installer soltuion any further.

Why?

[m!m]

No we wouldn't.  First of all, web services can be written in Java.  Second of all, there are transcoders that could re-emit the code in JavaScript, with no rewriting necessary.

That's all possible but this will again limit our possibilities, we will require a web server that has a Java environment which is not as common as a standard PHP server and having to use code transformers to get something as easy as a parser for our format is inefficient if we have the possibility to use an existing data format with existing parser solutions.

Why?

Mostly to be able to keep the web services/applications we will have to create maintainable. Keeping the old format just because we have a working solution is perfectly fine for one application but as soon as we have more of them we will have to maintain a parser for each used language and the SCP is already lacking coders.

[The E]

I am not sure I understand that objection.

To my mind, it doesn't matter what the format is for this metadata, so long as it's a consistently defined format. If someone wishes to implement the installer in a different language (and really, I can see just one occasion where that is necessary, namely when integrating the Launcher and the Installer into one package), he or she will have to implement the parser anyway, regardless of how that metadata is stored. Whether you're parsing a text file, or loading JSON, or loading XML, doesn't make that much of a difference.

Secondly, what's all this talk about web services etc? I am not sure I understand how we suddenly need a web service to handle anything aside from simple file hosting.

[m!m]

You are right the format of the metadata doesn't matter but it matters if you have to write a parser for the file format from scratch or if you can use an existing library which will parse it for you, for example for C/C++ we don't have a parser for the current metadata format, if we were to use JSON we could just use jansson which is well tested.
The web service is just a secondary idea which would allow us to have some sort of online mod gallery with an underlying database which would allow us to dynamically generate the required metadata automatically while allowing mod creators to publish their mods/missions without the need for someone else to edit the metadata by hand. Please note that this is an idea I had and nothing that should be set in stone but it will make updating or releasing a mod much easier.

[Eli2]

My two cents:
JSON is one of the best data interchange formats.
If the updater needs any server side scripts, the architecture is wrong.

[AdmiralRalwood]

JSON is one of the best data interchange formats.

"One of the most popular" != "one of the best", although it is certainly one of the easiest to parse.

[Eli2]

If you compare the spec to e.g. XML you will see the beauty of simplicity:
http://www.ecma-international.org/publications/files/ECMA-ST/ECMA-404.pdf
http://www.w3.org/TR/2004/REC-xml11-20040204/

Also it solves the old problem of how to represent numbers very nicely.

[AdmiralRalwood]

If you compare the spec to e.g. XML you will see the beauty of simplicity:

That's true of comparing anything to XML.

I really dislike XML.

[Eli2]

The question is what else do you want to compare it to?
INI, YAML

The beauty of JSON is that it has all features you need for most use cases.
With INI you often end up with custom parsers for values.
YAML is as bloated as XML.

[Bobboau]

/*quietly cheers*/ JSON! JSON!

The way I see it there are two basic use cases, a user gets the launcher/installer and A) has some form of FS installed already, or B) does not. for B there is also a sub use case of B2) user gets some form of FS. A is trivial and the expected use case, but B is likely to happen often, there are enough standalone TCs out there that it makes since that the original FS should not be a requirement to run FSO, it should be well within reason to setup the directory structure and install a few stand alone mods. The problem is B2, do you have the user install over any existing fs2 folder? have GOG/whatever FS2 install somewhere else and have the launcher import one into the other?
also, asking the user where something is installed should be the last option, it should be fairly easy to find any existing FSO folders automatically, they tend to be in only a handful of locations.

[The E]

As good as all these ideas are, can we please have the new installer out first before we try to make it better (for dubious values of better)?

[karajorma]

Well the new installer is pretty much up to Goober. So where is the need to wait?

[Hellzed]

@Bobboau :
In case B), we also have to think about Linux and OS X users : they cannot use GOG installer out of the box, wine (windows compatibility layer, used to perform a full windows FreeSpace 2 install) is overkill, and innoextract is perfect but has no GUI.
A complete FS2 installer should come bundled with innoextract, and provide a GUI for using it.