[ngld]

Thanks, Hellzed! I'll add support for the version files.
You're right about the update process. Right now it simply compares the filelist in the fs2mod file or from Goober's files with the files the user has and replaces any files with wrong checksums and adds missing files. I'll add detection for outdated and user-added files.

@niffiwan: Thanks for testing! I'm using the latest version of six which is 1.5.2. If you have pip installed you can run "pip install --upgrade six" to update it. If you don't want to update the package system-wide (or don't have pip) you can simply download the latest version and put the six.py file in the same directory as manager.py. That should fix the issue on Linux Mint.
"Select installed FS2 (Open)" should open a dialog which asks you to select a directory. What desktop environment are you using? Did the script show something on the console?
Just to check: If you click on "Install FS2 with the GOG installer", a window should open asking you for the path to the installer and a destination. If you click on the three dots next to "GOG installer", does anything happen?

[niffiwan]

Thanks, I installed six 1.5.2 on Mint (home) PC and it's working there as well now. With python2 at least, python3 complains that it can't find 7z, but it's installed! And I get the same issue with the "Select installed FS2 (Open)" bit

I'm using the MATE desktop.

There was nothing output to the console when clicking on the "Select installed FS2 (Open)" button (although clicking on other buttons output "FIXME: handle dialog start.")

When selecting "Install FS2 with the GOG installer" and either source or destination "..." button I get the normal directory browser that you'd expect.

[ngld]

I'm not sure what the exact problem is but I've added a workaround to the develop branch which avoids the native dialog.
Can you check if this solves the problem?

[Hellzed]

I was wondering : do you think it would be convenient to register fs2mod-py as a protocol handler for "fs2://" links ?
Imagine you click on "fs2://run/Derelict". Instead of showing the main window, it updates the list of available mods from the current sources, then checks if an item named "Derelict" is in the list. If it is not installed, we start installing Derelict.

I'm still trying to familiarise with python and github, so I forked the project and patched it with a few lines to read the version string from fs2mod files. I'm also looking into libunity API to check if we can do something for a better Unity/Ubuntu integration (using hard light icon, and maybe adding a progress bar on the icon when something is installing), the problem being that it's distro specific and I have no idea how not to break everything else trying to implement this.

[ngld]

Did you take a look at the documentation in the Ubuntu Wiki?

To avoid breaking other platforms you can use a try ... except block like this:

Code: [Select]

try:
  from gi.repository import Unity
except ImportError:
  # Can't find Unity.
  Unity = None

If you want do anything Unity-specify in the code just wrap it in an if:

Code: [Select]

if Unity is not None:
  # Update the icon....

If the user has Unity, the import succeeds and you can use the module in your code. If the user doesn't have it, the import fails and Unity is set to None. Any Unity-specific part would be skipped in that case.

[niffiwan]

I'm not sure what the exact problem is but I've added a workaround to the develop branch which avoids the native dialog.
Can you check if this solves the problem?

I tried commit 0b6c47fd5ad0b42f23c9623b1b350a5a487c580b on the develop branch and I get the same result.

I should also clarify; I get the output to the console from the working-as-expected "Install FS2 with the GOG installer" directory picker. 
I don't get any console output from clicking on the "Select installed FS2 (Open)" directory picker.

i.e. here's the output I get from running manager.py, clicking on "Select..." multiple times, then clicking on "Install..." once only, then pressing cancel.

Code: [Select]

$ python manager.py
Got bus address:  "unix:abstract=/tmp/dbus-NgcGCjGTPB,guid=b3a6fba5e5ce8542b3576e6e0000076b"
Connected to accessibility bus at:  "unix:abstract=/tmp/dbus-NgcGCjGTPB,guid=b3a6fba5e5ce8542b3576e6e0000076b"
Registered DEC:  true
Registered event listener change listener:  true
FIXME: handle dialog start.
FIXME: handle dialog end.


[ngld]

Do you have the qt-at-spi package installed? That might cause this problem.
It seems that the package can cause crashes when a qt application opens the directory picker but I'm not sure since the program doesn't crash.
If this doesn't help, try this build(33MB!) it should use my version of Qt.

EDIT: @Hellzed: Your idea with fs2:// links sounds nice. I've tried to implement it in the current dev version. It works on Windows and in Firefox on Linux. If you click on this link Firefox should ask you how it should open it. If you select the manager.py file (and that file is executable) and press OK it should start fs2mod-py.
On Windows you can simply start the script then go to "Settings" and click on "Install handler for fs2:// links". Windows will ask you if you want to modify the registry. If you accept all fs2:// links on windows should be opened with fs2mod-py.
I'm not shure about other browsers on Linux... Chromium uses xdg-open which in turn always complains "xdg-open: file 'fs2://run/Derelict' does not exist".
Well, at least it works in some cases.

EDIT2: HLP doesn't allow non-http links... Fixed it using redirection.

[AdmiralRalwood]

Did you take a look at the documentation in the Ubuntu Wiki?

To avoid breaking other platforms you can use a try ... except block like this:

Code: [Select]

try:
  from gi.repository import Unity
except ImportError:
  # Can't find Unity.
  Unity = None

If you want do anything Unity-specify in the code just wrap it in an if:

Code: [Select]

if Unity is not None:
  # Update the icon....

If the user has Unity, the import succeeds and you can use the module in your code. If the user doesn't have it, the import fails and Unity is set to None. Any Unity-specific part would be skipped in that case.

Python programmer here, just chipping in that the latter check can be written more concisely thusly:

Code: [Select]

if Unity:
    # do stuff

(It also avoids silly bugs, like if you accidentally typed "Unity = False", it won't say, "Hey, 'False' isn't 'None'! Let's use it!")

[Hellzed]

@ngld : I just tried to update both Blue Planet mods. It says it's only missing bp-compat.vp and bp2-compat.vp, but it started to download all archives for both mods when I confirmed. Is it a bug, or a check not implemented yet ?

From command line (python3 manager.py fs2://run/Derelict), directly installing Derelict worked, but I still cannot register the fs2:// protocol in firefox.

[AdmiralRalwood]

@ngld : I just tried to update both Blue Planet mods. It says it's only missing bp-compat.vp and bp2-compat.vp, but it started to download all archives for both mods when I confirmed. Is it a bug, or a check not implemented yet ?

From command line (python3 manager.py fs2://run/Derelict), directly installing Derelict worked, but I still cannot register the fs2:// protocol in firefox.

Does this help?

[Hellzed]

I tried this first, didn't help. Now the protocol is registered but Firefox complains about no app being able to open it.

[ngld]

Thanks, Hellzed, that was actually a bug. The script generated wrong paths while checking during installation.

Does ./manager.py work? Maybe you need to change the first line of manager.py to "#!/usr/bin/python3" to make it work.
Firefox will complain about having no app for that protocol if it can't run the program it knows. Did you try entering the link or did you use my link?

EDIT: Try to put the following in ~/.local/share/applications/fs2mod-py.desktop.

Code: [Select]

[Desktop Entry]
Name=fs2mod-py
Exec=/path/to/manager.py %U
Type=Application
Terminal=false
MimeType=x-scheme-handler/fso;

I had to change the protocol to fso:// because xdg-open doesn't allow numbers in protocol names.

[Hellzed]

You may also check the repository/fs2mod file adding logic : I can add empty stuff in there, and that leaves blank spaces in the sources list.

A lot of these bugs look awfully similar to the bugs I encountered with the bash version (wrong path forcing full re-download, adding empty stuff to the cache...)

Also, we're lucky that Dropbox keeps copies of old revisions, because (Ireally don't know how) Derelict.fs2mod was replaced by an empty text file using the same name, 2 times. Any idea what could be causing that ? (maybe it had something to do with adding and deleting empty repositories and it's fixed already ?)
(I'm not sure why, but when a fs2mod file is added, it's also always rewritten. Looks unsafe to me)

Each time I switch between launching the installer with python2 and python3, it loses all its cache and settings, and ends up doing all checksums again on next launch. Is it possible to avoid the issue ?
"#!/usr/bin/python3" may be necessary in Ubuntu, since it defaults to python2 if we directly launch the script.

The desktop entry alone doesn't work, it also needs to be referenced in "~/.local/share/applications/mimeapps.list"

Code: [Select]

[Desktop Entry]
Name=fs2mod-py
Exec=python3 /home/loris/fs2mod-py-develop/manager.py %U
Icon=/home/loris/fs2mod-py-develop/hlp.png
Type=Application
Terminal=false
MimeType=x-scheme-handler/fso;
Append this to "~/.local/share/applications/mimeapps.list"

Code: [Select]

x-scheme-handler/fso=fs2mod-py.desktop;
Something else : secondary progress bars appear in python3, not in python2. The main progress bar is ok in both.

I am also experiencing priority issues between the HLP repository and fs2mod files. Not sure which priority is intended behaviour.
(how complicated would it be to allow reordering of sources in the settings tab ?)

If I try to launch a mod from manager.py, fso currently crashes. I heard parsing flags and tabbed settings for fs2_open is not something easy. You could be interested in wxLauncher source code. I'm not sure what is your intention about launching mods directly from the mod installer (something like a "light" launcher ?)

Currently, wxLauncher is "profile centric" : it can store different settings profiles, with basic options, advanced options, and mod settings, for each profile.
A one-click mod install/launch process would need a "mod centric" launcher : base settings (screen resolution, AA, AF, controller, sound device) should be shared between all mods, while advanced settings (especially lighting, hud features...) should be set separately for each mod. When you chose to launch a mod, right settings are selected. wxLauncher has no way to that currently, and implementing it would end up with a bloated code base.

Something like fso://settings (for default profile, the one which is copied for default settings when we start a new mod for the first time) and fso://settings/Derelict (for Derelict profile) link to bring up a really simple settings screen would be great, with only the base settings, and a button to access a text zone to enter advanced flags (a bit like what steam does).

[ngld]

Well, that's the update mechanism... Right now I'm fetching the file from the given URL in root/update whenever the user clicks on "Update List". If this fails, it leaves an empty file.
I guess I should save the downloaded file somewhere in /tmp and only replace the actual file if the download succeeds.
BTW Dropbox is kinda annoying when you want to download the fs2mod files. Right now I'm sending a "curl/7.x.x" (x are actual numbers in the code) User-Agent to convince Dropbox to give me the file and not a download page. The program also has to follow a redirect on every fetch.

When the fs2mod file is replaced with an empty file, did the script display some kind of error or exception?

EDIT: The settings are saved with pickle. Python 3 supports the new protocol version 3 while python 2 doesn't (and won't) support it. I'll set it to 2 for now, that should prevent the settings loss.
I'm not sure about the progress bars... might be a PySide bug.

[Hellzed]

I didn't save the console logs, sorry.
My bash script made a copy of fs2mod files inside its config directory, before running the update on the copy. The original, wherever it is on the user's disk then becomes useless.

I guess we'll have to move fs2mod files somewhere on HLP. Since they are small files, they can be hosted as attachments on the wiki. I'll have to ask about it.

I think fso:// links should use fs2mod files names instead of the actual mod title, since space and special characters in urls are generally a bad idea (I just experienced some issues with "Media VPs 3612"...)

[ngld]

@niffiwan: I've finally found and fixed the bug you found. PyQt4 behaved a little bit different than PySide and thus skipped my directory picker. The develop branch should work now.

The python 2/3 settings issue is partly resolved. It works but all images will be lost when switching from 2 to 3. The manager should now ignore all mod entries which have no name.
I hope that I've fixed the fs2mod file issue now. The issue with the secondary progress bars is also resolved now the script had a rounding issue.

Actually allowing the user to sort the mod sources isn't too hard. Applying that order in tasks.py (in FetchTask.finish) is more work because I'll have to reorder the results there but it shouldn't be too hard. I'll see how long it takes.
FSO crashes on your installation? I'd like to see the error message...
The logic to run fs2 is actually pretty simple (run_mod() and run_fs2() in manager.py): It reads ~/.fs2_open/data/cmdline_fso.cfg on Linux and <FS2>/data/cmdline_fso.cfg on all other systems. It then tries to find the -mod flag and replaces whatever value follows that flag with the selected mod. Once that's done it simply runs the given FS2 binary. Could you post your cmdline_fso.cfg, please?

Right now it works for me but I'll take a look at the wxLauncher. Thanks for the pointer!
I could host the fs2mod files, if you want. I'm hosting the JSON files already anyway or you could use GitHub Pages to host them.

EDIT: What happend to the thread title?

[Hellzed]

Some connection issues with the forum, the page reinitialised while I was typing a quick reply.

I'm not sure about the current syntax for fso:// links : which string should I use now ?

This is my cmdline_fso.cfg :

Code: [Select]

-mod mediavps_3612 -missile_lighting -3dshockwave -post_process -fxaa -dualscanlines -orbradar -rearm_timer -ship_choice_3d -weapon_choice_3d -3dwarp -warp_flash -snd_preload -fb_explosions -keyboard_layout "azerty" -ambient_factor 75 -spec_exp 11 -spec_point 0.6 -spec_static 0.8 -spec_tube 0.4 -ogl_spec 60
I just tried to install mediavps_3612.fs2mod and some (if not all) files were actually downloaded 2 times. They are downloaded a first time, then their size drops to zero, and they are downloaded again. Any clue ?

[niffiwan]

Do you have the qt-at-spi package installed? That might cause this problem.
It seems that the package can cause crashes when a qt application opens the directory picker but I'm not sure since the program doesn't crash.

@niffiwan: I've finally found and fixed the bug you found. PyQt4 behaved a little bit different than PySide and thus skipped my directory picker. The develop branch should work now.

Hurrah! It works now on both my PCs, Mint 13 (which I needed to remove qt-at-spi from) & Centos 6.  And it looks really good    Installing mods works very well, as does using innoextract to pull the FS2 data out of the GOG installer.  I like the download progress bars and how they remove themselves from the window when they finish.

My only comment is something that I think I previously mentioned to Hellzed (); I believe that the FS2 VPs from the GOG installer should be extracted into the root directory just like the retail installer did, and not into the data dir.  It's also what all our manual install guides recommend, and I believe it's what all the people on HLP who do support expect.

Another comment is that selecting a nested option doesn't automatically select the parent.  i.e. if you select "Derelict Voice Pack", "Derelict" itself is not selected or installed. It'd be nice to have the parent automatically selected when the child is selected.  It would also be good to apply this to dependencies.

Aside from that, this is a great effort 

[Hellzed]

Found a bug, latest develop branch :

Code: [Select]

INFO:Thread-9:util.download: Downloading "https://www.dropbox.com/s/os2ihzqx8b1c4o9/mediavps_3612.fs2mod?dl=1"...
ERROR:Thread-9:fs2mod.update_info: Failed to update Media VPs 3612!
Traceback (most recent call last):
  File "/home/loris/Bureau/fs2mod-py-develop/fs2mod.py", line 286, in update_info
    shutil.move(path, self.update[2])
NameError: global name 'shutil' is not defined
WARNING:Thread-9:progress.add_task: Added an empty task of type "FetchTask". Ignoring it!

Maybe the "Working" window should be a bit more robust : currently if you close the window while something is downloading, you can restart the same download at the same time. I think it should send a warning if you try to close it, then kill all current tasks so we are sure we don't restart the resume the same tasks next time it's open.

By the way, my personal branch now has full Unity launcher integration (it has a quicklist to access tabs from right click, not really interesting except for right click -> Add Source which is a direct access to the "Add mod" window. And the icon in the Unity launcher shows a copy of the global progress bar.). I'll push it to my repository once it's rebased.

[ngld]

Thanks, niffiwan. The develop version automatically selects all dependencies of a selected mod (including its parent) but the default mod source (http://dev.tproxy.de/fs2/all.json) is missing some dependencies. If you change it to http://dev.tproxy.de/fs2/hlp.json it should work. The newer repo isn't compatible with the released version (master branch) so I changed the link.

@Hellzed: Thanks, fixed again... I should actually test the code before I commit.
Your cmdline looks alright so that shouldn't cause the crash... hm.
I've been able to reproduce the double download bug but I'm still trying to find out why it happens.

You don't need to change your fso:// links, I've just added support for links like "fso://run/FreeSpace Port". Special characters and whitespace are automatically encoded by the browser which results in "fso://run/FreeSpace%20Port". Fs2mod-py can now decode that and will look for "FreeSpace Port" and not "FreeSpace%20Port".

I'll prevent the user from closing the "Working..." window and instead add a "Cancel" button but I'm not sure how to implement it. After all, I can't simply abort a task but I'll have to revert all changes made by it.
Thanks for your work on the Unity integration!