[Goober5000]

All right!  Many thanks to m!m for spotting the cause of all the failed downloads.  It was just an accidental mistake, but it caused all the downloads to share the same buffer instead of having their own separate buffers.  This was the equivalent of tossing all downloads into the same blender and hitting "frappé".  No wonder they all got corrupted.

New version is up again, with the following changes:

• Fixed parallel downloads being corrupted (thanks again to m!m)
• Fixed cancellation mechanism; you should now be able to smoothly cancel mods and downloads
• Made 7zip initialization thread-safe (fixes "SevenZipJBinding wasn't initialized successfully last time")
• Fix bug where deep directory paths weren't created
• Compute all hashes, even if some of them turn out to be bad
• Allow window to be resized
• Various thread-safety fixes
• Various GUI and logging fixes and tweaks

I've also fixed the DMTools mod config file.  Unfortunately, SectorGame seems to be down, so some of the mod selections will be unavailable.  (The installer is already able to filter them out.)

[zookeeper]

One remaining minor issue is the lack of clear feedback when downloads are complete. All bars seem to be done, but the text above will still say "Installing..." while only one of the bars says "Done!" and unless you collapse the bars, it's not obvious at all that there's anything special about the topmost bar. Also, considering how massively big some downloads can be, it wouldn't hurt to have some unannoying notification pop up (flash the taskbar item, for example) when it's done.

[MatthTheGeek]

Going much better now. Still DLing basic, but at this point I had already failed half the things on the older versions so that's a good sign.

As a future improvement, a DL rate indicator in kB/s would be nice to have somewhere.


EDIT: I tried to stress-test it by going custom and toggling ALL THE THINGS, it got stuck for 20min after DLing the exes, had to cancel it.

logs here

[niffiwan]

I've also run a stress test downloading everything available.  Runtime was just under 40 mins.  There were a few failures but I believe they can all be attributed to issues in the config files.

Code: [Select]

The following errors were encountered:

The hash value for 'BtA Demo/BTA_DEMO_Assets2.vp' did not agree with the expected value.  This could indicate a corrupted download.  The file has been deleted.
Low-End Compatibility: Skipped because parent mod was not installed.
Blue Planet: The file 'bp-core.zip' could not be downloaded.
Blue Planet: The file 'bp-audio1.zip' could not be downloaded.
Blue Planet: The file 'bp-audio2.zip' could not be downloaded.
Blue Planet: The file 'bp-visuals1.zip' could not be downloaded.
Blue Planet: The file 'bp-visuals2.zip' could not be downloaded.
Enhanced Graphics: Skipped because parent mod was not installed.
The hash value for 'fsport-str.vp' did not agree with the expected value.  This could indicate a corrupted download.  The file has been deleted.
Silent Threat: Reborn Voice Pack: Skipped because parent mod was not installed.


One remaining minor issue is the lack of clear feedback when downloads are complete. All bars seem to be done, but the text above will still say "Installing..." while only one of the bars says "Done!" and unless you collapse the bars, it's not obvious at all that there's anything special about the topmost bar. Also, considering how massively big some downloads can be, it wouldn't hurt to have some unannoying notification pop up (flash the taskbar item, for example) when it's done.

I also felt the feedback on completion status could have been clearer. Would collapsing each mods bars when it's completed make it easier to see the progress? Or maybe remove each mod/item from the list entirely when they're complete?

And one more suggestion, I really like the mouse-wheel-scrolls-one-mod-per-click in the mod selection window.  Could you also add that to the download status window?

[mjn.mixael]

It's still not killing itself properly. (Win7 Pro)

[AdmiralRalwood]

And one more suggestion, I really like the mouse-wheel-scrolls-one-mod-per-click in the mod selection window.  Could you also add that to the download status window?

Could you do the same scrolling fix to the other sections of the installer (installing... and install review)?

Uh... I could, but that's more tricky, because the Installing section has panels of variable height.  (Especially if you expand/collapse a few of them.)  It's possible, but it would take some custom implementation.

At the very least, though, the default scroll distance needs to be increased; trying to move down the list with the scroll wheel takes forever at the moment.

[Goober5000]

As a future improvement, a DL rate indicator in kB/s would be nice to have somewhere.

At the very least, though, the default scroll distance needs to be increased; trying to move down the list with the scroll wheel takes forever at the moment.

I'm going to take it as a positive sign that people are now complaining about silly things rather than major things.

It's still not killing itself properly. (Win7 Pro)

wat

I've made several changes in the code that should have fixed this -- not to mention I've never seen it at all on my end.  Could you post all your logs?

Actually, a clarification would be helpful.  Try starting it via command-line: go to the directory and do "java -jar FreeSpaceOpenInstaller.jar".  Then run it as usual.  Now, after the installer is complete, a) does the window close?  b) does the program return control to the command prompt (i.e. can you type more commands)?  c) is javaw.exe still running in the background?

EDIT: Also, d) could you run "java -version" and tell me what you get?  And e) if javaw.exe persists, then does it still persist if you exit the program on the first screen?  f) on the screen just before installation?  g) if you hit the close button rather than "Finish" on the last screen?

[MatthTheGeek]

c) is javaw.exe still running in the background?

Yup here.

[Goober5000]

Please answer each of the questions; I need all the answers together to get the whole picture.

[AdmiralRalwood]

I'm going to take it as a positive sign that people are now complaining about silly things rather than major things.

You should! Now that downloads actually work, it looks like the biggest thing that needs fixing is the data (mod config files and the downloads they link to) rather than the program.

[Luis Dias]

Yes, the development of this thread is highly optimistic in tone and I love it! Keep it up .

[MatthTheGeek]

Now that downloads actually work

Someone hasn't read my previous post.

[mjn.mixael]

Well, the first thing I ran into is that the Java installation didn't set the environment variables... is that something that's going to be required for proper .jar termination?

[AdmiralRalwood]

Now that downloads actually work

Someone hasn't read my previous post.

c) is javaw.exe still running in the background?

Yup here.

I did, actually!

However, assuming you meant your post-before-last, I also read that one, and it doesn't match my experience or niffiwan's; I was referring more to the fact that all downloads no longer share a buffer, so it is actually possible for them to succeed now. Still, maybe Goober will spot something useful in your logs; I couldn't see anything obviously wrong.

Speaking of config file weirdness, Windmills is installing to /Windmills/Windmills/ and Transcend is installing to /Transcend/Transcend/ (no other nested folders in any of the other mods, although obviously BtA:M, BP:AoA, and ST:R don't succeed so can't guarantee their install paths).

[Goober5000]

Now that downloads actually work

Someone hasn't read my previous post.

I did notice your post and I'll take a look at your logs later.  For extra information, could you tell me exactly which mods were complete and which mods seemed to have hung?  Or try running it again, with a new set of logs and a screenshot.

Well, the first thing I ran into is that the Java installation didn't set the environment variables... is that something that's going to be required for proper .jar termination?

That shouldn't matter.

[Goober5000]

EDIT: I tried to stress-test it by going custom and toggling ALL THE THINGS, it got stuck for 20min after DLing the exes, had to cancel it.

logs here

I figured out what happened here.  The Installer used a thread pool that was limited to 20 tasks at any one time.  Some of these tasks will spawn additional tasks to handle parallel downloads, but if you run ALL THE THINGS at the same time, thread scheduling may cause a situation where certain tasks are waiting for downloads that can never complete because they are waiting in the queue.

I've changed this so that the installer can now run unlimited tasks in parallel.  My original reason for limiting the pool was to limit the number of downloads occurring in parallel, but I can do this using another method (semaphores, to be specific).  But I'm wondering whether I actually need to do that or not.

The change in thread pool setting might fix the latent javaw.exe, but then again, it might not.  If it recurs, please post answers to the questions a-g in this post.

I've uploaded a new version of the Installer that restricts parallel downloads to 4*.  I've also updated fsoinstaller.com, fixed the Transcend and Windmills install paths, and fixed the hashes for ST:R.


*You can change this at the command line with the option -DmaxParallelDownloads=X, where X is any number >= 1.  Example: java -DmaxParallelDownloads=2 -jar FreeSpaceOpenInstaller.jar.  Make sure you spell it right!

[niffiwan]

OK - I did another complete install from scratch and only BP:AoA and BTA:M failed this time

There's one more issue I noticed; Linux has a case sensitive filesystem which caused issues with the mediavps_3612 install.  All the patch files went into one dir while the rest of the files went into another.

Code: [Select]

$ ls  ~/fsot/[mM]*
~/fsot/mediavps_3612:
FSU-MVP.bmp  MV_Advanced.vp   MV_Assets.vp   MV_Music.vp       MV_Root.vp
mod.ini      MV_AnimGlows.vp  MV_Effects.vp  MV_RadarIcons.vp  readme.txt

~/fsot/MediaVPS_3612:
MV_Assets.3612.vp  MV_Effects.3612.vp  MV_Root.3612.vp

This seems like a tricky issue to fix as the difference in directory capitalisation is in the zip files themselves, not the config file.  Maybe this just needs to be a note/warning to mod creators to ensure they have uniform capitalisation in all their archives?


And now a few questions/feature requests

When creating a new configfile it may be useful to be able to pass the configfile directly to the installer so that the creator can write/test/fix the file rapidly, without needing to upload the file anywhere and without letting "the world" see and accidentally use a potentially buggy file. Is there a -Dxxx cmdline option to allow this, and if it doesn't exist could it be created?

I really like having control of the number of simultaneous downloads.  I set mine to 100 and it halved my download time for the complete package from 40 mins to 20 mins. Would you consider adding an option to control this to the interface?

And one more suggestion, I really like the mouse-wheel-scrolls-one-mod-per-click in the mod selection window.  Could you also add that to the download status window?

Could you do the same scrolling fix to the other sections of the installer (installing... and install review)?

Uh... I could, but that's more tricky, because the Installing section has panels of variable height.  (Especially if you expand/collapse a few of them.)  It's possible, but it would take some custom implementation.

At the very least, though, the default scroll distance needs to be increased; trying to move down the list with the scroll wheel takes forever at the moment.

Oops...    and I should add that it seems to take ages to scroll using the scroll arrows as well, clicking within the scrollbar is fine though.

[AdmiralRalwood]

Performed a Complete install with the new version:

Code: [Select]

FreeSpace Open 3.7.0 Windows 9X: The file 'mod.ini' could not be downloaded.
FreeSpace Open 3.7.0 Windows SSE2: The file 'mod.ini' could not be downloaded.
FreeSpace Open 3.7.0 Windows SSE: The file 'mod.ini' could not be downloaded.
FreeSpace Open 3.7.0 Windows NO-SSE: The file 'mod.ini' could not be downloaded.
The hash value for 'BtA Demo\BTA_DEMO_Assets2.vp' did not agree with the expected value.  This could indicate a corrupted download.  The file has been deleted.
Low-End Compatibility: Skipped because parent mod was not installed.
Blue Planet: The file 'bp-core.zip' could not be downloaded.
Blue Planet: The file 'bp-audio2.zip' could not be downloaded.
Blue Planet: The file 'bp-audio1.zip' could not be downloaded.
Blue Planet: The file 'bp-visuals1.zip' could not be downloaded.
Blue Planet: The file 'bp-visuals2.zip' could not be downloaded.
Enhanced Graphics: Skipped because parent mod was not installed.

Also, Deus Ex Machina: Interlude has nested folders.

[jr2]

Nice work, Goober! 

[Goober5000]

Thanks!  Now update your install guide.

EDIT: I've uploaded a new version, with the following changes:

• Added a new "validate" mode.  Running java -jar FreeSpaceOpenInstaller.jar validate will not start up the wizard; instead it will prompt for a text file on the local file system.  It will then try to parse this text file the same as if it had been accessed from a remote site through the install screen.
• Post-install notes will now appear on the post-install screen.

I've also fixed the mod.ini error in the SCP installation, and removed the Blue Planet mod txt files until they have the opportunity to build new ones.  The nested DEM: Interlude fix will need to wait until I can get in touch with Hunter at SectorGame.