[mjn.mixael]

Yeah, the times it stalled on setting up.. seemed to indicate it never got to the timeout.. but I lost the log.. (I was testing too much at once, silly me). If it happens again, I'll post the log.

I'll definitely do a writeup/wiki article. Though, the VERSION does seem to be shown to the user under More Info (if available).

[Axem]

Can the installer only add files, or can it remove them too?

[mjn.mixael]

It looks like it can remove them as well if you add the correct commands to the mod.txt file. I'm going to be testing all of that as I do the writeup on the txt files.

[niffiwan]

Some more feedback; I tried this on my work PC (Centos 6) and again, this version of OpenJDK has the problem with the window being too tall.  jre1.6.0_25 worked as expected.

The proxy seemed to work as expected (using our non-authenticated proxy)

I tried a complete install to an empty dir but lots of stuff failed.  In addition, it said that "The Basic Installation configuration could not be validated against the current list of mods. This option will not be available".   Maybe that's because this file can't be downloaded through our proxy because that particular port is not allowed: http://173.8.193.66:7701/svn/MediaVPs/mv.txt

I then tried it a 2nd time (complete download to the same dir I used before).  Some more mods worked, but others still failed.  Since the log seems to append to itself, here's the one log from attempts so far: http://pastebin.com/Cy73eG3Q

I think there's a bug in the destination pathnames here - on Linux you need to use '/' as the dir separator not '\'.

Code: [Select]

INFO: Downloading and extracting from http://www.freespacemods.net/e107_files/downloads/fs2_ogg.zip to local directory /home/me/fsot/data\movies

Re: valid URLs, I used wget to test all the attempted downloads recorded in the log.

These fsoinstaller files failed:
(NOTE: works fine without proxy shenanigans)

Code: [Select]

http://173.8.193.66:7701/svn/MediaVPs/mv.txt

These fsoinstaller files worked:

Code: [Select]

http://blueplanet.fsmods.net/blueplanet2.txt
http://blueplanet.fsmods.net/blueplanet.txt
http://fsport.hard-light.net/installer/fsport.txt
http://scp.indiegames.us/fsoinstaller/basic_config.txt
http://scp.indiegames.us/fsoinstaller/filenames.txt
http://scp.indiegames.us/fsoinstaller/version.txt
http://scp.indiegames.us/scp_files.txt
http://swc.fs2downloads.com/builds/LINUX/fs2_open_3.7.0.tar.bz2
http://www.fsoinstaller.com/files/installer/java/basic_config.txt
http://www.fsoinstaller.com/files/installer/java/cutscene.txt
http://www.fsoinstaller.com/files/installer/java/dmtools.txt
http://www.fsoinstaller.com/files/installer/java/filenames.txt
http://www.fsoinstaller.com/files/installer/java/fsmods.net.txt
http://www.fsoinstaller.com/files/installer/java/homesick.txt
http://www.fsoinstaller.com/files/installer/java/multi.txt
http://www.fsoinstaller.com/files/installer/java/ransomarceihn.txt
http://www.fsoinstaller.com/files/installer/java/version.txt
http://www.lunardigitalproductions.com/bta/downloads/demo/installer/mefistofele.txt
http://www.sectorgame.com/goober/dem.txt

For data files; these ones failed:

Code: [Select]

http://blueplanet.fsmods.net/bp-audio1.zip
http://blueplanet.fsmods.net/bp-audio2.zip
http://blueplanet.fsmods.net/bp-core.zip
http://blueplanet.fsmods.net/bp-visuals1.zip
http://blueplanet.fsmods.net/bp-visuals2.zip
http://www.rateofinjury.com/campaigns/installer/Sync.zip
http://www.rateofinjury.com/campaigns/installer/Transcend.zip

And these ones worked:

Code: [Select]

http://blueplanet.fsmods.net/bp2-audio1.zip
http://blueplanet.fsmods.net/bp2-core.zip
http://blueplanet.fsmods.net/bp2-visuals1.zip
http://blueplanet.fsmods.net/bp2-visuals2.zip
http://freespacemods.net/files/installer_files/campaigns/brshivans.zip
http://freespacemods.net/files/installer_files/campaigns/bwo_demo.zip
http://freespacemods.net/files/installer_files/campaigns/derelict_core.zip
http://freespacemods.net/files/installer_files/campaigns/inf1.zip
http://freespacemods.net/files/installer_files/campaigns/pi103.zip
http://freespacemods.net/files/installer_files/campaigns/stl.zip
http://freespacemods.net/files/installer_files/campaigns/twilight.zip
http://freespacemods.net/files/installer_files/campaigns/warzone.zip
http://fsport.freespacemods.net/releases/fsport3_4.zip
http://fsport.freespacemods.net/releases/fsport-missions.zip
http://www.freespacemods.net/e107_files/downloads/fs2_ogg.zip
http://www.freespacemods.net/e107_files/downloads/homesick.zip
http://www.freespacemods.net/files/installer_files/multi/multi-mission-pack.zip
http://www.fsoinstaller.com/files/downloads/dmtools.zip
http://www.lunardigitalproductions.com/bta/downloads/demo/BTA_DEMO_Assets1.7z
http://www.lunardigitalproductions.com/bta/downloads/demo/BTA_DEMO_Assets2.7z
http://www.lunardigitalproductions.com/bta/downloads/demo/BTA_DEMO_Root.7z
http://www.lunardigitalproductions.com/bta/downloads/demo/installer/BTA_DEMO_Assets1.zip
http://www.lunardigitalproductions.com/bta/downloads/demo/installer/BTA_DEMO_Assets2.zip
http://www.lunardigitalproductions.com/bta/downloads/demo/installer/BTA_DEMO_Root.zip
http://www.sectorgame.com/goober/DeusExMachinaInterlude.zip
http://www.sectorgame.com/goober/DeusExMachina.zip

Or in other words, if you're testing, don't bother with BP, Sync or Transcend until the fsoinstaller files for them are updated.

I'll keep doing tests and let you know the results.

[mjn.mixael]

DELETE commands don't seem to be working as well. Here's a log. FSPort asks it to delete fsport3_3.vp and I added a command to the BtA demo as well to test how the installer works with file flow... but since the command didn't work for FSPort either...

What I mean is that the BtA Demo installs to 'games\freespace2' because it unzips the folder 'BtA Demo' instead of individual VPs.. so I was guess I had to do something like this...

Code: [Select]

FOLDER
BtA Demo
DELETE
BTA_OLD.vp
FOLDER
\
URL
http://www.lunardigitalproductions.com/bta/downloads/demo/installer/
BTA_DEMO_Root.zip
BTA_DEMO_Assets1.zip
BTA_DEMO_Assets2.zip

Anyway, here's the log. It says some stuff about deleting, but it doesn't seem to actually do the job, which seems odd.

[attachment deleted by an evil time traveler]

[MatthTheGeek]

And Goober, I was going to throw up a news post about this, but would you prefer to hold off on that for now, since this is still something of a WIP?

Thing is, until downloading become reliable (which it's definitely not right now) we'll just end up in the exact same situation as we were with Turey's installer: lots of broken installs everywhere for support to have to fix.

Noone wants that - this is literally what the new installer was meant to fix.

[Goober5000]

Some more feedback; I tried this on my work PC (Centos 6) and again, this version of OpenJDK has the problem with the window being too tall.  jre1.6.0_25 worked as expected.

This is actually a bug in OpenJDK:
https://bugs.launchpad.net/ubuntu/+source/openjdk-6/+bug/742830

It's listed as "confirmed", with a few example screenshots, but not fixed, assigned, or updated in almost three years.

I tried a complete install to an empty dir but lots of stuff failed.  In addition, it said that "The Basic Installation configuration could not be validated against the current list of mods. This option will not be available".   Maybe that's because this file can't be downloaded through our proxy because that particular port is not allowed: http://173.8.193.66:7701/svn/MediaVPs/mv.txt

Yes, that's likely to be the reason.  I think we should ask the MediaVP guys to host their config file somewhere else.

I think there's a bug in the destination pathnames here - on Linux you need to use '/' as the dir separator not '\'.

Code: [Select]

INFO: Downloading and extracting from http://www.freespacemods.net/e107_files/downloads/fs2_ogg.zip to local directory /home/me/fsot/data\movies


That's actually a problem in the config file itself; it specifies data\movies as the location.  I'll have to add a processor that replaces slashes with system-appropriate symbols.

Or in other words, if you're testing, don't bother with BP, Sync or Transcend until the fsoinstaller files for them are updated.

I've updated the file for Sync and Transcend.  Ransom's new upload location now uses .rar files, so this will be a nice way to test that the .7z engine can handle .rar.

I'll keep doing tests and let you know the results.

Great.

Though you might want to hold off until I change the logging mechanism.  I want to set things up so that each mod has its own log file.  This will avoid overlapping log statements when downloads are done in parallel.

And Goober, I was going to throw up a news post about this, but would you prefer to hold off on that for now, since this is still something of a WIP?

Thing is, until downloading become reliable (which it's definitely not right now) we'll just end up in the exact same situation as we were with Turey's installer: lots of broken installs everywhere for support to have to fix.

That was already agreed upon, so I'm not sure why you're bringing it up again.

Noone wants that - this is literally what the new installer was meant to fix.

The differences are twofold.  One, Turey's installer was coded with everything smashed together: GUI code mixed with configuration code mixed with download code, which made it very hard to understand, let alone debug.  The new installer was designed to separate out functionality so that individual features could be added, debugged, and changed in isolation.  (This has paid off already; instead of the whole thing failing, most of it currently works.)

Two, Turey wasn't around to update the installer or fix bugs after it was released.  That's not the case with me.  And even if I vanished tomorrow, the code is on the SCP SVN and the mod files are on indiegames.

[niffiwan]

This is actually a bug in OpenJDK:
https://bugs.launchpad.net/ubuntu/+source/openjdk-6/+bug/742830

It's listed as "confirmed", with a few example screenshots, but not fixed, assigned, or updated in almost three years.

Ah.  Thanks for finding that. Since ubuntu only packages OpenJDK and doesn't maintain it, I had a look at the upstream project http://openjdk.java.net/ and their bugtracker.  I didn't find any trace of the bug there, so I installed OpenJDK 7 and success - it works as expected now   Maybe this could be noted as a "known issue" in the OP? Here's a link with instructions for installing OpenJDK 7 on Linux.

Code: [Select]

$ java -version
java version "1.7.0_51"
OpenJDK Runtime Environment (rhel-2.4.4.1.el6_5-x86_64 u51-b02)
OpenJDK 64-Bit Server VM (build 24.45-b08, mixed mode)


Though you might want to hold off until I change the logging mechanism.  I want to set things up so that each mod has its own log file.  This will avoid overlapping log statements when downloads are done in parallel.

No worries, just let me know when the next version is ready.

[Mongoose]

And Goober, I was going to throw up a news post about this, but would you prefer to hold off on that for now, since this is still something of a WIP?

Thing is, until downloading become reliable (which it's definitely not right now) we'll just end up in the exact same situation as we were with Turey's installer: lots of broken installs everywhere for support to have to fix.

Noone wants that - this is literally what the new installer was meant to fix.

Yeah, I was going to put some big bold letters stating THIS IS VERY MUCH A BETA RELEASE AND YOU SHOULD NOT BE USING IT AT ALL UNLESS YOU'RE AN FSO EXPERT, but given how that never seems to stop anyone, your point is well-taken.

[Goober5000]

Okay, new version has been uploaded, with quite a few fixes...

• The installer will now navigate through the mod tree and install child mods in addition to the parent mods
• Logging is now broken down into multiple files, with mod-specific information in its own file
• Logging format is improved, fitting onto one line and containing more information than previously, while looking cleaner
• Certain errors will now be properly logged, while previously they were eaten without a trace
• Application properties are now loaded correctly (this doesn't matter unless you're releasing a custom-branded installer for a standalone mod)
• If mod developers provide a destination folder with slashes, these will be corrected to the system-specific path separators
• The automatic updater mechanism has been fixed (it will now correctly prompt you to download a new version if the current version is out of date)

There are still a bunch of errors relating to mod downloading and dependency resolution, but these changes should make it a lot easier to track them down.

[MatthTheGeek]

1) Could we have a version number appearing somewhere? When you re-DL it it's not obvious whether you do have the latest version or not.

2)

3) As a future possible improvement, would it be possible for the installer to run hardware/OS detection via maybe cpu-z or something (that thing's open source, right) and select automatically the correct FSO version (SSE-2 / no SSE / linux / whatever).

Why should that be a future improvement?  The installer already does this.   (You need to select the Basic configuration to see it.)

It would probably be more user-friendly if the other exes didn't appear at all when you select Basic. The average user doesn't need to worry about it, it will only confuse him and everything is available to download manually on the forums anyway.

3) The issue where all the MV_assets, etc progress bars didn't appear at all is fixed. However everything fails if the parent install fail, which is probably expected behaviour. Still get 25% DL failures for DLs that were attempted at all.

4) When the MVP parent install fails, everything is stuck (like I said, probably expected behaviour) but as a result the "Next >" button is never made available. "Cancel" shows the "cancelling now will stop all current DLs, are you sure" popup, and clicking yes does nothing, still on the DL window. You have to press the cross on top of window to actually quit.

logs here

[Fury]

Does the new installer know how to update itself should there be newer version available? If not, that should be addressed.

[Goober5000]

1) Could we have a version number appearing somewhere? When you re-DL it it's not obvious whether you do have the latest version or not.

That's reasonable.  I might put it in the title bar.

It would probably be more user-friendly if the other exes didn't appear at all when you select Basic. The average user doesn't need to worry about it, it will only confuse him and everything is available to download manually on the forums anyway.

I'm inclined against that.  The whole reason for the mod selection screen appearing after Basic/Complete/Custom is so that the user can modify the selection if desired.

3) ... Still get 25% DL failures for DLs that were attempted at all.

4) ... "Cancel" shows the "cancelling now will stop all current DLs, are you sure" popup, and clicking yes does nothing, still on the DL window. You have to press the cross on top of window to actually quit.

Yes, I noticed these myself.  Will investigate them soon.

Does the new installer know how to update itself should there be newer version available? If not, that should be addressed.

That's what the last bullet point in my previous post was all about.  So yes.

[mjn.mixael]

Stalled out here, couldn't hit next and couldn't cancel.


Deleting files still isn't working properly. Logs attached.

EDIT: And each time I try a fresh install of mods, the first files fail, causing everything else to fail. Oddly, though the 'failed' files are still downloaded and put in the right folders. Second set of logs attached.

And the java process is still not killing itself properly.

[attachment deleted by an evil time traveler]

[MatthTheGeek]

It would probably be more user-friendly if the other exes didn't appear at all when you select Basic. The average user doesn't need to worry about it, it will only confuse him and everything is available to download manually on the forums anyway.

I'm inclined against that.  The whole reason for the mod selection screen appearing after Basic/Complete/Custom is so that the user can modify the selection if desired.

Except the user has no idea what those choices mean (how many times did we get "what's SSE?" on the forums), so he'll either just click them all and then panic when it's time to select an exe in the launcher, or wholeheartedly give up at that point cause "even the installer was too complicated for me".

We're talking about users for which following manual install instructions is already a stopper. We should aim to make things as clear and ambiguity-free as possible for them. "Advanced" users can use the custom install if they want to have a finer control toward what exactly to DL.

[AdmiralRalwood]

If you don't want to hide them if the user selects "Basic", how about greying them out with a note that "The installer has automatically selected the correct build for your system" or something?

[niffiwan]

The check-for-the-most-recent-version seems to be broken. 

1) downloaded the .jar from the OP& ran it (i.e. http://scp.indiegames.us/fsoinstaller/FreeSpaceOpenInstaller.jar)
2) it says that the installer is out of date
3) Click yes, go to the website and it downloads a 32kb .jar file (i.e. http://www.fsoinstaller.com/files/installer/java/FreeSpaceOpenInstaller.jar)
4) new .jar does not run

So maybe it's just grabbing the wrong file as the most-recent version?

[Goober5000]

It shouldn't...  I tested the updating mechanism before I uploaded it.

I'll have another version ready this evening.  This one should fix the child mod dependencies, and it will also show the version string in the title bar.

[Goober5000]

Okay, new version is up again, with the following changes:

• Dependencies between parent and child mods should now be fixed!  The GUI should no longer get stuck if a child mod fails to install.  (Though the download bar still bounces back and forth... I'll still need to fix that.)
• Renaming and deleting should now be fixed
• Not just mod folders, but mod files will now have their slashes standardized to the system-specific path separators
• The version number will be shown in the title bar (version as of this post is 2.0.2)
• Mods will be logged in the order they are shown in the list (minor, but useful for context when you have a package simply called Voice Files)
• The scroll bar will scroll in increments of mods, not pixels (so on windows, one tick of the scroll wheel will scroll past three mods)
• Various logging improvements

In the process of making these fixes, I discovered a problem in certain mod config text files.  All references to files on the local file system are taken relative to the download folder.  Most mods, such as FSPort, download to their actual mod folder; but some, such as BTA, download to the root folder and then create the folder in the process of extracting the files.  For the second category of mods, you'll need to adjust the file references.

That probably isn't very clear, so let me illustrate it with a fixed version of BTA's config file.  (Incidentally, MjnMixael, you can only have one FOLDER, so I had to juggle things slightly.)

Code: [Select]

NAME
Between the Ashes: Mefistofele
DESC
The day is August 3, 2335. Thirteen months have passed since the Lucifer was destroyed, mysteriously locking the Sol system away. It's barely been two months since the Hades superdestroyer was destroyed in orbit of Deneb, and the rogue GTI reduced to silence. There are many wounds to lick for the GTA, as the weakened fleet struggles to maintain order in its systems. Although the cease-fire treaty with their new Vasudan allies is still holding, the PVE is still busy dealing with the terrorist HoL faction.

Now is our chance to get past them. We no longer need to so carefully hide. Now we can make our deal with the devil.
ENDDESC
FOLDER
\
DELETE
BtA Demo\BTA_OLD.vp
URL
http://www.lunardigitalproductions.com/bta/downloads/demo/installer/
BTA_DEMO_Root.zip
BTA_DEMO_Assets1.zip
BTA_DEMO_Assets2.zip
HASH
MD5
BtA Demo\BTA_DEMO_Root.vp
53bc5ece74ce324ba893d3065e9fea44
HASH
MD5
BtA Demo\BTA_DEMO_Assets1.vp
5bf52c3633b1afbb08e5259130b159ae
HASH
MD5
BtA Demo\BTA_DEMO_Assets2.vp
908266bf60d357e768b3a45a8b10365f
NOTE
This requires MediaVPs 3.6.12!
ENDNOTE

NAME
Low-End Compatibility
URL
http://www.lunardigitalproductions.com/bta/downloads/demo/installer/
BTA_DEMO_00LowEnd.zip
HASH
MD5
BtA Demo\BTA_DEMO_00LowEnd.vp
e500623ed6ee6e4b6f5d943bf4023e18
VERSION
Version 1.1
END

VERSION
Version 1.1
END

[mjn.mixael]

Aha, that makes a lot of sense. I'll make sure to highlight that in the tutorial/wiki article.

EDIT: The 'NOTE' section doesn't seem to be shown to the user anywhere.. is that just a legacy 'table flag', bug, or future feature?