Author Topic: [Preview] Knossos 0.5.3  (Read 23084 times)

0 Members and 1 Guest are viewing this topic.

Offline ngld

  • Administrator
  • 29
  • Knossos dev
Re: [Preview] Knossos 0.5.3
Thanks!

Unfortunately, my hard drive died a few days ago. Nothing was lost because I had everything backed up but restoring everything took a while. I haven't made much progress since my last post, however I finally finished two concepts:
I think I'm going to rename packages because the name seems to confuse people. I just don't know what to name them instead (packs?). Anyway, Knossos will create a folder for each package you create through the dev tab. When you start a mod from there you will be able to choose which packages to enable. You can also tell Knossos to put everything in the package's folder into a VP file when you upload your mod.

As a reminder if you (the reader) don't know what packages are:
A mod has a version, title, description, etc. A mod is what shows up in lists.
Each mod has at least one package but can have as many as a modder wants. A package always belongs to exactly one mod (and version). A package has a title, included files and dependencies.
A package serves to seperate mod files into groups.


I'm also done with the concept for BitTorrent support. The communication with the tracker will go through HTTPS and communication between clients will be SSL encrypted. Due to this it should be impossible for an ISP to tell that users are using BitTorrent. They will be able to detect that P2P connections are used but that shouldn't be too surprising nowadays (see Windows updates and various game launchers).
Knossos will allow the user to completely turn BitTorrent support off in which case it falls back to normal HTTP(S) downloads. It will also allow people to limit the download and upload bandwidth. I might also add an option to stop seeding once you've uploaded as much as you downloaded.
If BitTorrent downloads are enabled, the HTTP(S) mirror will still be used so even if noone else is seeding, it should be at least as fast as normal downloads.

 

Offline mjn.mixael

  • Cutscene Master
  • 212
  • Chopped liver
    • Steam
    • Twitter
Re: [Preview] Knossos 0.5.3
Randomly, I just noticed tonight that the Install % does not update unless you switch tabs now.

I created a new folder for Knossos to install FS2 and mods to. Told it to download and install BtA. Had to keep switching to Explore, then back to Home to see the update bar progress and to see the percentage number increase. Pretty sure this worked before. Then after the install should have completed, the mods did not show in Home. Odd. Tomorrow I'm gonna poke around with the dev tab UI design anyway.
Cutscene Upgrade Project - Mainhall Remakes - Between the Ashes
Youtube Channel - P3D Model Box
Between the Ashes is looking for committed testers, PM me for details.
Freespace Upgrade Project See what's happening.

 

Offline m!m

  • 211
Re: [Preview] Knossos 0.5.3
Knossos will allow the user to completely turn BitTorrent support off in which case it falls back to normal HTTP(S) downloads. It will also allow people to limit the download and upload bandwidth. I might also add an option to stop seeding once you've uploaded as much as you downloaded.
If BitTorrent downloads are enabled, the HTTP(S) mirror will still be used so even if noone else is seeding, it should be at least as fast as normal downloads.
Does that mean that BitTorrent is enabled by default? I already voiced my concerns about enabling BitTorrent by default so how about showing a message at the first startup which explains that Knossos can use BitTorrent and what advantages that would provide? The user could then choose to enable or disable it but they will at least be notified that by enabling it they might use more of their data volume than they expected.

Also, I just tried to run Knossos from the development branch and now it doesn't start anymore. I'll try to bisect to find the commit that caused this.

EDIT: This may be a system error on my side. The last message that is shown by "ninja run" is this:
Code: [Select]
Check failed: NamespaceUtils::DenySetgroups()

It seems to be related to the chromium sandbox and I found a few references to this message in the chromium bug tracker. I also see the following message in "dmesg" every time I try to run Knossos:

Code: [Select]
[ 2245.963242] traps: python3.5[11392] general protection ip:7f7dc35ef602 sp:7ffe045cf960 error:0 in libc-2.24.so[7f7dc35b8000+1bc000]
« Last Edit: July 16, 2017, 03:40:44 am by m!m »

 

Offline ngld

  • Administrator
  • 29
  • Knossos dev
Re: [Preview] Knossos 0.5.3
@mjn: Weird. I've only noticed that bug on Ubuntu before where it was a bug in QtWebKit which didn't properly foward the events. I can't test it right now because the Explore tab is empty. I'll look into it later.

Does that mean that BitTorrent is enabled by default? I already voiced my concerns about enabling BitTorrent by default so how about showing a message at the first startup which explains that Knossos can use BitTorrent and what advantages that would provide? [...]
We already came to that conclusion in the last thread.


EDIT: This may be a system error on my side. The last message that is shown by "ninja run" is this:
Code: [Select]
Check failed: NamespaceUtils::DenySetgroups()
Are you trying to run Knossos under a Ubuntu guest account? There Chrome/Chromium didn't work there until they added an AppArmor exception for Chrome. Since that exception is path specific I'm going to guess that they didn't add Qt's QtWebEngineProcess to that exception which in turn would cause the same bug for Knossos.

It seems to be related to the chromium sandbox [...]
You can try to disable the sandbox by running it like this:
Code: [Select]
QTWEBENGINE_DISABLE_SANDBOX=1 ninja run

 

Offline m!m

  • 211
Re: [Preview] Knossos 0.5.3
We already came to that conclusion in the last thread.
Ok, I wasn't sure what exactly you meant in your post.

Are you trying to run Knossos under a Ubuntu guest account? There Chrome/Chromium didn't work there until they added an AppArmor exception for Chrome. Since that exception is path specific I'm going to guess that they didn't add Qt's QtWebEngineProcess to that exception which in turn would cause the same bug for Knossos.
No, I'm on Fedora and I can't recall any significant changes to my system since I last ran Knossos. It could have been broken in a recent kernel update, I also found a few kernel commit references in the chromium bug report.

You can try to disable the sandbox by running it like this:
Code: [Select]
QTWEBENGINE_DISABLE_SANDBOX=1 ninja run
Yes, that makes it work. Although now most of the mods don't show up (I'm not sure if they are supposed to show up, maybe you are doing some refactoring and it doesn't work at the moment). Here is the stacktrace of one parse failure:

Code: [Select]
ERROR:Thread-1:tasks.work1: Failed to parse "/fso/blueplanet/mod.ini"!
Traceback (most recent call last):
  File "knossos/tasks.py", line 146, in work1
    mod.set_base(base_mod)
AttributeError: 'IniMod' object has no attribute 'set_base'

 

Offline ngld

  • Administrator
  • 29
  • Knossos dev
Re: [Preview] Knossos 0.5.3
I had to move fsnebula.org to a new server (the old server is outdated and is being replaced) and I didn't want to transfer the old stuff hosted there since I'll have to replace it anyway. That's why Knossos currently can't download a mod list and thus can't show any mod on the "Explore" tab.

The set_base error will be fixed with the next commit. It might cause some mods you have locally installed to not show up but shouldn't affect anything else.

Regarding the sandbox issue: The issue here is that the Chromium sandbox isn't initialized properly because it failed to create the namespace. This most of the time is a permission problem. Fedora uses SELinux which probably denied access to this API. The cause is probably a recent QtWebEngine or policy update. For reference here's Fedora's bug tracker. I've seen similar issues there but none seem to be the root cause for this problem.
« Last Edit: July 16, 2017, 11:20:13 am by ngld »

 

Offline m!m

  • 211
Re: [Preview] Knossos 0.5.3
Regarding the sandbox issue: The issue here is that the Chromium sandbox isn't initialized properly because it failed to create the namespace. This most of the time is a permission problem. Fedora uses SELinux which probably denied access to this API. The cause is probably a recent QtWebEngine or policy update. For reference here's Fedora's bug tracker. I've seen similar issues there but none seem to be the root cause for this problem.
SELinux is disabled at the moment so it's can't be that. Just to be sure, I reenabled it.

 
Re: [Preview] Knossos 0.5.3
Is this going to be the replacement for FSO installer for mod installation/management?

 

Offline ngld

  • Administrator
  • 29
  • Knossos dev
Re: [Preview] Knossos 0.5.3
Probably, at least if enough people here like it.
My intention was to combine installer and launcher (i.e. FSO Installer and wxLauncher) into one program AND make it easier to use. Whether that works out or not is something we will see once this is finally finished. It will take a while but hopefully I'll be able to release 1.0.0 this year.

I'm finally done with the dev tab UI, now I still have to write the saving and upload code. Afterwards I'll have to test and fix the code which launches mods. I'll probably have to fix the install/update/uninstall parts, too. I guess I'll release a new beta at that point and work on UI polishing, bug fixing and eventually torrent support. Once that beta is polished and stable enough, I'll be able to release 1.0.0 at which point (I hope) more and more people will use it.

@m!m: I've built a Fedora VM and wasn't able to reproduce your issue on it. I cloned the repo, installed the neccessary packages with dnf and finally ran Knossos both with "ninja debug" and "ninja run". It worked both times without issue. Any idea what I did differently?

 
Re: [Preview] Knossos 0.5.3
Probably, at least if enough people here like it.
My intention was to combine installer and launcher (i.e. FSO Installer and wxLauncher) into one program AND make it easier to use. Whether that works out or not is something we will see once this is finally finished. It will take a while but hopefully I'll be able to release 1.0.0 this year.

I'm finally done with the dev tab UI, now I still have to write the saving and upload code. Afterwards I'll have to test and fix the code which launches mods. I'll probably have to fix the install/update/uninstall parts, too. I guess I'll release a new beta at that point and work on UI polishing, bug fixing and eventually torrent support. Once that beta is polished and stable enough, I'll be able to release 1.0.0 at which point (I hope) more and more people will use it.

@m!m: I've built a Fedora VM and wasn't able to reproduce your issue on it. I cloned the repo, installed the neccessary packages with dnf and finally ran Knossos both with "ninja debug" and "ninja run". It worked both times without issue. Any idea what I did differently?

I like it so far. But when I tried to use the explore search bar it didn't work for me. Is that going to get fixed?

 

Offline ngld

  • Administrator
  • 29
  • Knossos dev
Re: [Preview] Knossos 0.5.3
Are you talking about the fact that no mods are showing up? That's going to change once I'm done with the uploading code which will allow me to add mods again.

 

Offline mjn.mixael

  • Cutscene Master
  • 212
  • Chopped liver
    • Steam
    • Twitter
Re: [Preview] Knossos 0.5.3
I haven't poked around with this in a little while.. has the mod.json been updated to include to additional mod images/descriptions/etc yet?
Cutscene Upgrade Project - Mainhall Remakes - Between the Ashes
Youtube Channel - P3D Model Box
Between the Ashes is looking for committed testers, PM me for details.
Freespace Upgrade Project See what's happening.

 

Offline ngld

  • Administrator
  • 29
  • Knossos dev
Re: [Preview] Knossos 0.5.3
Kind of... You can open the dev tab, go to "Mod details" and edit everything there (including the new fields), however you can't save that, yet.
« Last Edit: July 25, 2017, 04:30:53 pm by ngld »

 
Re: [Preview] Knossos 0.5.3
Are you talking about the fact that no mods are showing up? That's going to change once I'm done with the uploading code which will allow me to add mods again.

No mods showing up is precisely what I meant

  

Offline m!m

  • 211
Re: [Preview] Knossos 0.5.3
@m!m: I've built a Fedora VM and wasn't able to reproduce your issue on it. I cloned the repo, installed the neccessary packages with dnf and finally ran Knossos both with "ninja debug" and "ninja run". It worked both times without issue. Any idea what I did differently?
My guess is that the chromium version Qt is using is outdated and can't handle my system setup for some reason. Google Chrome is working fine so it will probably be fixed by a Qt update in the future. In the meantime I'm just going to run Knossos with QTWEBENGINE_DISABLE_SANDBOX=1.

EDIT: I'm also getting an uncaught error on startup:
Code: [Select]
Traceback (most recent call last):
  File "knossos/launcher.py", line 364, in main
    run_knossos()
  File "knossos/launcher.py", line 184, in run_knossos
    center.mods.load_json(mod_db)
  File "knossos/repo.py", line 81, in load_json
    self.set(json.load(stream))
  File "knossos/repo.py", line 92, in set
    self.add_mod(Mod(mod, self))
  File "knossos/repo.py", line 392, in __init__
    self.set(values)
  File "knossos/repo.py", line 417, in set
    if pkg.check_env():
  File "knossos/repo.py", line 630, in check_env
    logging.warn('Invalid value for environment check in mod %s (%s)!' % (self.mid, self.version))
AttributeError: 'Package' object has no attribute 'mid'
I "fixed" that by simply removing that line but you may want to take a closer look at this.

 

Offline ngld

  • Administrator
  • 29
  • Knossos dev
Re: [Preview] Knossos 0.5.3
@m!m: Thanks, should be fixed now.

I've finally implemented saving and the FSO settings section. You can now install retail files and create/edit/delete mods.
Launching should work now, however, I'm still working on the -mod flag. This means that you can only launch retail and TCs right now.

Knossos now allows multiple versions of a mod to be installed (since it was easier to implement than I thought). After the -mod flag is done, I'll make sure that the UI works with multiple versions of the same mod and then I'll look for problems with launching FSO and fix those.
I'm not yet sure if I'll release a new beta at that point or if I should wait until uploading is implemented. That will take a while because I'll also have to write the server part for that. Either way, I'll release a new beta after that and I think that only polishing and bug fixing will be necessary after that point to reach 1.0.0.

 

Offline mjn.mixael

  • Cutscene Master
  • 212
  • Chopped liver
    • Steam
    • Twitter
Re: [Preview] Knossos 0.5.3
Updated and upon launching got..

Code: [Select]
ninja: error: 'html/templates/flag-editor.html', needed by 'knossos/data/resoures.rrc', missing and no known rule to make it
Cutscene Upgrade Project - Mainhall Remakes - Between the Ashes
Youtube Channel - P3D Model Box
Between the Ashes is looking for committed testers, PM me for details.
Freespace Upgrade Project See what's happening.

 

Offline ngld

  • Administrator
  • 29
  • Knossos dev
Re: [Preview] Knossos 0.5.3
Try removing the build.ninja file and then running windows_run.bat again. I renamed a file and build.ninja still references the old location. I haven't found a better way to deal with this, yet.

 

Offline mjn.mixael

  • Cutscene Master
  • 212
  • Chopped liver
    • Steam
    • Twitter
Re: [Preview] Knossos 0.5.3
That works
Cutscene Upgrade Project - Mainhall Remakes - Between the Ashes
Youtube Channel - P3D Model Box
Between the Ashes is looking for committed testers, PM me for details.
Freespace Upgrade Project See what's happening.

 

Offline mjn.mixael

  • Cutscene Master
  • 212
  • Chopped liver
    • Steam
    • Twitter
Re: [Preview] Knossos 0.5.3
Working on some of the cosmetics. Here's a few thoughts.

Can this handle Steam install?
The "Install Retail" button is persistent even after I installed retail. It's that correct behavior? I would have thought that button would disappear when retail is installed? Then include a "reset/reinstall retail" button in the settings somewhere. But not a big deal. If it's permanent, then I might redesign the button a bit.

Saving mod details gives
Code: [Select]
A critical error occurred! Knossos might not work correctly until you restart it. The error has been reported and will hopefully be fixed soon. Seems like Video links saving has an error based on the CMD output window.

When we're in the Settings page, the Gear should be green and the tab links (Home/Explore/Dev) should be gray. I could not find the tab-bar elements in the html files anymore.

Still would like to see the Uninstall button moved to the dropdown for each mod, but that will require javascript to get working. I can move the button semi easily, but I could never get it to be functional from the dropdown.
Cutscene Upgrade Project - Mainhall Remakes - Between the Ashes
Youtube Channel - P3D Model Box
Between the Ashes is looking for committed testers, PM me for details.
Freespace Upgrade Project See what's happening.