Author Topic: [Preview] Knossos 0.5.3  (Read 23070 times)

0 Members and 1 Guest are viewing this topic.

Offline mjn.mixael

  • Cutscene Master
  • 212
  • Chopped liver
    • Steam
    • Twitter
Re: [Preview] Knossos 0.5.3
No, it needs some significant work on the backend in relation to handling additional mod data and how it works with the online portion.
Cutscene Upgrade Project - Mainhall Remakes - Between the Ashes
Youtube Channel - P3D Model Box
Between the Ashes is looking for committed testers, PM me for details.
Freespace Upgrade Project See what's happening.

 

Offline ngld

  • Administrator
  • 29
  • Knossos dev
Re: [Preview] Knossos 0.5.3
Sorry for the late reply. I've been busy studying for exams which is why I haven't worked much on Knossos recently. The exams are this week so I should be able to continue my work soon.

The current state is: The whole uploading part (interface & logic) is missing, parts of the settings screen (mostly joystick & network options) are missing, a few parts of the dev tab (the team screen and (I think) the screen for the -mod flag) as well as a bunch of details (menu items to launch FRED, filtering of TCs, sorting of mods, etc.) are also missing.
The uploading code is the biggest piece left. To start on that, however, I need to finish the server. I've already started working on that and will upload the code tomorrow but it will take a while to finish.
Once the server and uploading code work, we should have the basic functionality in place and could start the next beta phase.

 

Offline ngld

  • Administrator
  • 29
  • Knossos dev
Re: [Preview] Knossos 0.5.3
As promised, I've uploaded the server code. It's still in a very early stage but it's there if someone wants to look at it.

Something I haven't been able to decide on is whether to upload files in one piece or split them into several blocks. The latter approach is useful when uploads fail. If I upload each block in a single request, I only have to retry the block that failed. If I upload files without splitting them, it would be easier (faster) to implement but the whole file would have to be retried which can be very annoying if you're working with huge files.

 

Offline chief1983

  • Still lacks a custom title
  • Moderator
  • 212
  • ⬇️⬆️⬅️⬅️🅰➡️⬇️
    • Minecraft
    • Skype
    • Steam
    • Twitter
    • Fate of the Galaxy
Re: [Preview] Knossos 0.5.3
I do agree that anything that could be done to make uploading saner for mod developers, it would be helpful.  But, it can probably be iterated in later when we have something that works at all, if upgrading the upload method later would be not much harder than doing it that way first.  Seems odd though, that breaking up uploads is the only way to do what sounds like a type of resume-able upload?  Is it not possible to just resume uploads from arbitrary byte indices?
Fate of the Galaxy - Now Hiring!  Apply within | Diaspora | SCP Home | Collada Importer for PCS2
Karajorma's 'How to report bugs' | Mantis
#freespace | #scp-swc | #diaspora | #SCP | #hard-light on EsperNet

"You may not sell or otherwise commercially exploit the source or things you created based on the source." -- Excerpt from FSO license, for reference

Nuclear1:  Jesus Christ zack you're a little too hamyurger for HLP right now...
iamzack:  i dont have hamynerge i just want ptatoc hips D:
redsniper:  Platonic hips?!
iamzack:  lays

 

Offline ngld

  • Administrator
  • 29
  • Knossos dev
Re: [Preview] Knossos 0.5.3
Changing the upload method later on isn't very hard.
Resuming an interrupted upload without splitting the upload is hard because normally the web application code is only run once the request is complete. An incomplete request like an interrupted upload is discarded. I could work around that but I'd rather avoid the additional complexity.
I've looked at how tus.io and Amazon's S3 implement resumable uploads and they both split files into multiple parts which are then uploaded individually.

I'll implement simple file uploading for now and later replace it with tus.io.

 

Offline ngld

  • Administrator
  • 29
  • Knossos dev
Re: [Preview] Knossos 0.5.3
I'm finally done with exams and started working on this. I've finished the basic functionality on the server (register, login, password reset, mod creation/updating, file uploads). The UI work on Knossos is mostly done (only the register, login and password reset buttons are missing). This only leaves the main part: The code which controls the actual upload. I should be able to finish an alpha build either this weekend or next week. It probably will be buggy and it will have a bunch of issues.

While I work on the uploading code I wanted to discuss a few issues Knossos currently has:
  • If you have multiple versions of a mod installed, Knossos will only show the latest version.
    The code is able to handle multiple versions but I haven't figured out a way to display them in the UI. Should I display each version as a seperate mod (tile)? Should I put the mod version in the title? Or should I add a version select to the detail page and the dev tab?
  • The dev tab shows all installed mods.
    I think it makes sense to limit this to mods which you are actually allowed to edit & upload.
  • Knossos currently supports two different folder structures for mods and the dev tab isn't aware of it.
    Knossos has always used a folder per mod which contained the various files (and folders) belonging to the mod. This is what most FSO users currently have.
    As a result of feedback on this thread and my work on the dev tab, I've added a "dev mode". In this mode each package is a sub folder which is necessary to separate the files during uploading and this will allow modders to enable/disable parts of the mod each time they launch FSO (I use the -mod flag for this, each active package is listed as a mod folder).

    These two modes aren't clearly separated in the code or the UI. I will probably add a new field to the mod.json file to indicate the current mode and maybe display it in the UI. As a general rule of thumb: Mods on the dev tab should be in dev mode and all others shouldn't be.
  • There is no easy way to create a new version.
    The dev tab should have a simple button which allows a modder to create a new version. This would allow them to play the released version for bugfixing/testing and work on the new version at the same time (both versions would obviously be in different folders).
  • The only place to change the commandline flags is on the dev tab.
    Players should be able to change these on the mod detail screen. However, I'm not sure where to save them and what to do with these when a mod is updated. Should I overwrite the user's flags with the new mod flags or should I keep the user's flags?
  • There is no way to edit or delete uploaded mods.
    I'm not sure how to handle this case. It consists of two problems: a) the UI b) If a mod is deleted, the ID can't be re-used because it would cause issues for users which still have the old mod installed.
    If I allow mod deletion (and I probably should), I'll keep a list of deleted mod IDs on the server which should solve b). The same problem more or less applies to mod versions.
  • Knossos doesn't have a proper documentation.
    I think the dev tab has become complex enough to warrant an explanation, tutorial or at least a bit more help text. For example, dependencies and packages are never explained.

I think most of these issues are UI related which might be because I don't know much about UI/UX design.
I want feedback on these issues. I want to create some mockups before I start changing the UI code but before I work on those, I want to hear your thoughts. (@mjn feel free to create/share your own, I think they'll be better than mine).

  

Offline mjn.mixael

  • Cutscene Master
  • 212
  • Chopped liver
    • Steam
    • Twitter
Re: [Preview] Knossos 0.5.3
I think a version select dropdown is best. We don't want to the tile page to be filled up with multiple mod versions. It just ends up looking and feeling cluttered. Especially since the use-case for running older mod versions is way more niche.

Agree on dev tab after some thought.

Agree on folder structure points.

Agree on adding a version button to the dev tab. The dev tab needs some UI work in general that I'd like to get done before a release. Been waiting on feature completion.

Do people change their flags on a per-mod basis often? I would be happy with an option in settings for the user to have a set of custom flags. Then they can specify "prefer player flags" or "prefer mod flags". Knossos would override any same mod flags with player flags if the preference is selected. Maybe. I dunno, just a thought.

Keep a list of deleted mod IDs and the metadata. We don't need to do anything on Knossos for UI on this if a deleted mod is installed on a player's machine. Just warn them before they delete it. "This mod has been removed from Nebula, if you continue to delete it, you won't be able to redownload it."

Documentation is good.

I've also been waiting on feature completion of other mod details (banner image, I'd love for it to pull a list of mod staff if possible and list that on the details page, I think there were other things, but it's been a while since I worked on the UI.)
Cutscene Upgrade Project - Mainhall Remakes - Between the Ashes
Youtube Channel - P3D Model Box
Between the Ashes is looking for committed testers, PM me for details.
Freespace Upgrade Project See what's happening.

 

Offline ngld

  • Administrator
  • 29
  • Knossos dev
Re: [Preview] Knossos 0.5.3
My main problem with flags is that they are specific to each FSO build. If I allow a user to globally set flags, I need to get the flag list from somewhere (maybe allow the user to select a FSO build? That might be confusing though...) and I risk passing a new flag to an older build which will then complain about the unknown flag.

Regarding mod deletion: We have to put a delete button somewhere because the Nebula doesn't have a UI which leaves Knossos as the only way to add/edit/delete mods.

 
Re: [Preview] Knossos 0.5.3
Do people change their flags on a per-mod basis often?
Speaking for myself, I know I've done so on several times through profiles in wxLauncher, for instance I had base settings for standard FSO, profiles with darker or brighter lighting depending on a mod's mood, a profile with shadows disabled for resource intensive mods, etc.
I'm not sure that's a common use case though, and if mods can supply recommended settings, I'd probably use that 8 times out of 10.

 

Offline chief1983

  • Still lacks a custom title
  • Moderator
  • 212
  • ⬇️⬆️⬅️⬅️🅰➡️⬇️
    • Minecraft
    • Skype
    • Steam
    • Twitter
    • Fate of the Galaxy
Re: [Preview] Knossos 0.5.3
I'm not saying I agree one way or the other with how we allow flags to be set, but just from a technical standpoint, I don't see why allowing global flag preferences shouldn't be able to work.

The global settings (all known flags to exist across any builds) could be saved somewhere.  When configuring for a specific build, any removed flags would be hidden but still saved.  When running a build with global flags saved that don't exist yet or anymore in that build's flags, couldn't we just silently not pass them in?  You'd have to compare the global flags to the available flags for that build but that should allow to strip out any flags that aren't supported before actually passing them to the build I would think.  Then, instead of the harsh unsuppported flags popup, the launcher could also determine if or how to notify the user that some chosen global flags are unavailable with the current mod/build configuration.
Fate of the Galaxy - Now Hiring!  Apply within | Diaspora | SCP Home | Collada Importer for PCS2
Karajorma's 'How to report bugs' | Mantis
#freespace | #scp-swc | #diaspora | #SCP | #hard-light on EsperNet

"You may not sell or otherwise commercially exploit the source or things you created based on the source." -- Excerpt from FSO license, for reference

Nuclear1:  Jesus Christ zack you're a little too hamyurger for HLP right now...
iamzack:  i dont have hamynerge i just want ptatoc hips D:
redsniper:  Platonic hips?!
iamzack:  lays

 

Offline ngld

  • Administrator
  • 29
  • Knossos dev
Re: [Preview] Knossos 0.5.3


So... I finished the upload code and it works! The fsnebula.org server is running again, registration, login, upload & download work. If you build the latest git version, you should be able to test it. Go to Settings > Nebula settings, enter values for all three fields and click "Register". You should now receive an email with a confirmation link. Click it, switch back to Knossos and then click Login. Once that worked, go to your dev tab, select a mod and click "Upload". That's it, your mod is now available to everyone using Knossos.

I'll make an alpha build sometime this week so more people can test this but I want to fix a few issues first:
  • The server doesn't prevent the upload of older or duplicate versions. I'll probably need to add more sanity checks.
  • The "Reset Password" button is missing
  • The directory structure issue I already mentioned
  • The staff list mjn mentioned which I'd also use to edit permissions (it would allow multiple people to edit the same mod)
  • Mod deletion

On another note, I'm confident that private mods (i.e. for beta testing) will be part of the stable release.

@chief: I like your concept. Any idea where I could get a list of flags for all existing builds? On second thought... Only 3.8.0 is currently supported, that makes this easy. I'll just have to edit the list whenever the SCP removes a flag but that doesn't happen too often from what I can tell.

 

Offline mjn.mixael

  • Cutscene Master
  • 212
  • Chopped liver
    • Steam
    • Twitter
Re: [Preview] Knossos 0.5.3
Excellent. I'll have to poke around to make sure the UI is solid.

Are the additional mod fields supported yet? (details banner, etc.)

Can't the launchers ping builds for available flags? Could that be used silently when selecting mods?
Cutscene Upgrade Project - Mainhall Remakes - Between the Ashes
Youtube Channel - P3D Model Box
Between the Ashes is looking for committed testers, PM me for details.
Freespace Upgrade Project See what's happening.

 

Offline Cyborg17

  • 29
  • Life? Don't talk to me about life....
Re: [Preview] Knossos 0.5.3
Just a note, since it's so easy to upload a mod, you may want to have a report mod button, so that we can report anyone who uploads stuff that isn't a mod.

 

Offline ngld

  • Administrator
  • 29
  • Knossos dev
Re: [Preview] Knossos 0.5.3
@mjn: The additional fields haven't been added, yet, I want to focus on fixing the bugs in the upload and install code, first.
Knossos can ask the builds for flags but the global flags aren't associated with any build.

@Cyborg17: That's a great idea. I need to add moderation functions while I add that (I think moderators should be able to hide or delete mods. I'm not yet sure how to handle this for private mods).

During testing I also realized that I need to add a delete function soon because I accidentally uploaded broken metadata.

 

Offline ngld

  • Administrator
  • 29
  • Knossos dev
Re: [Preview] Knossos 0.5.3
Since I still haven't finished the alpha build, I wanted to update you on the progress so far.

I've added the upload checks (prevents upload of the same or an older version), added a report button and a delete button. The version selector and a button to create new versions have also been finished.
You can now also reset your password. Now I'm only missing a way for users to change their passwords. The appropriate parts have already been implemented on the server and in Knossos but I'm not sure how to put it in the UI so I've left it out for now (the Nebula section of the settings screen feels already pretty crowded).

I still have to deal with the directory structure issue and the staff list. I might delay the staff list a bit and release the alpha after I solve the first issue.

@mjn: Well... the UI's a bit of a mess right now. I think we can convert some of the buttons on the sidebar into tabs which might make their purpose clearer.


EDIT:
The code should now be aware of the new directory structure and refuses to edit mods which aren't in "dev mode". I've also finished the -mod editing tab and fixed the launching logic. So far everything seems to be working fine. I can create, edit, upload, install and launch mods.
There are only a few issues remaining (apart from the unfinished staff page):
  • The FSO build I uploaded requires retail files although it obviously shouldn't.
  • If you deleted your local dev files for a mod, you can't delete it anymore since it doesn't show up in your dev tab.
  • Ubuntu seems to have some weird UI glitches since Knossos still uses WebKit there instead of Chromium. I'm tempted to change that since Knossos uses Chromium on every other supported platform but that means that we can only support Ubuntu >= 17.04. The previous OS releases don't ship the necessary packages.
  • The UI needs a lot of polish and a few things like sorting, filtering, the dropdown menu in the mod list and the concept for tools (i.e. FRED2, qtFRED, ...) hasn't been implemented.
« Last Edit: September 22, 2017, 05:35:41 pm by ngld »

 

Offline mjn.mixael

  • Cutscene Master
  • 212
  • Chopped liver
    • Steam
    • Twitter
Re: [Preview] Knossos 0.5.3
What sidebar area were you thinking for tabs? Your post wasn't clear.

I'll spend some time this afternoon working on the UI in places.
Cutscene Upgrade Project - Mainhall Remakes - Between the Ashes
Youtube Channel - P3D Model Box
Between the Ashes is looking for committed testers, PM me for details.
Freespace Upgrade Project See what's happening.

 

Offline ngld

  • Administrator
  • 29
  • Knossos dev
Re: [Preview] Knossos 0.5.3
The area which contains the "Edit Mod Details", "Upload" buttons, mod path and the list of packages.

The code's now in a state where I could release an alpha build. Creating mods, uploading, installing & launching should all work. Now I'm wondering, should I release the current build or should I wait until I've put some more polish in (i.e. finish the dropdown menus on the "Home" tab, the sorting menu (the one on the upper right) and the staff list)?

 

Offline mjn.mixael

  • Cutscene Master
  • 212
  • Chopped liver
    • Steam
    • Twitter
Re: [Preview] Knossos 0.5.3
I think we can give the UI a bit of polish. It won't take a whole lot of work comparatively.
Cutscene Upgrade Project - Mainhall Remakes - Between the Ashes
Youtube Channel - P3D Model Box
Between the Ashes is looking for committed testers, PM me for details.
Freespace Upgrade Project See what's happening.

 

Offline ngld

  • Administrator
  • 29
  • Knossos dev
Re: [Preview] Knossos 0.5.3
Here's an example of what I meant regarding the button / tab placement:


I've also changed a few other things: I think the current CREATE button in the mod list doesn't fit there so I replaced it with the one you see. I've also removed the "INSTALL RETAIL" button because the Knossos styled buttons have a static width and the text wouldn't fit with that. Not sure what to do about that.
The middle column has been cleaned up: All buttons use the proper style and a few of them have been converted to tabs on the right side. I just realized that I forgot the "Add Package" button...  :banghead: Not sure where I can put that. The tab bar listing all packages seems to be the most logical place. Maybe I can just add a tab labeled "+"?

This is obviously just a sketch but I want to finish the new UI concept before I start implementing it.

@mjn: Can you give me a list of missing mod fields and how they should look? I forgot...


EDIT: Here's the staff page:

The access field allows you modify who can edit/upload/delete a mod and who can install it (for private mods). The checkbox at the top allows the user to switch the mod between public and private mode.
« Last Edit: October 01, 2017, 06:40:11 am by ngld »

 

Offline chief1983

  • Still lacks a custom title
  • Moderator
  • 212
  • ⬇️⬆️⬅️⬅️🅰➡️⬇️
    • Minecraft
    • Skype
    • Steam
    • Twitter
    • Fate of the Galaxy
Re: [Preview] Knossos 0.5.3
Private mods should probably not be seen by those without access either.  Although there may be mods that want themselves to be visible but not installable, like unlockable content maybe?  I dunno if there's a good case for that or not.
Fate of the Galaxy - Now Hiring!  Apply within | Diaspora | SCP Home | Collada Importer for PCS2
Karajorma's 'How to report bugs' | Mantis
#freespace | #scp-swc | #diaspora | #SCP | #hard-light on EsperNet

"You may not sell or otherwise commercially exploit the source or things you created based on the source." -- Excerpt from FSO license, for reference

Nuclear1:  Jesus Christ zack you're a little too hamyurger for HLP right now...
iamzack:  i dont have hamynerge i just want ptatoc hips D:
redsniper:  Platonic hips?!
iamzack:  lays