Author Topic: Possible new UI systems for FSO  (Read 14864 times)

0 Members and 1 Guest are viewing this topic.

Offline m!m

  • 211
Possible new UI systems for FSO
The current systems for presenting a user interface in FSO are severely lacking. The first system is to one used to render the main UI (e.g. options menu, briefing screen, ...). It is limited to the two main resolutions and is unable to scale correctly to the screen size which means that this system is not usable for adding additional user interfaces and upgrading it would be a monumental task. The second system ist the WMCGUI which is used in the lab but that is a pain to work with. There is a third option which involves rendering the UI with Lua (and that has been done already by some recent mods thanks to the system developed by Axem) but I will take a look at that option later.

My primary goal would be to build a new system that could be used and extended by mods to provide custom UI screens. I have looked at a few third- and first-party solutions and I will list the advantages and disadvantages I see if we would use these systems. Here is my list (in no particular order):
  • CEF/Chromium Embedded Framework This is basically the chromium web-rendering engine as an embeddable component. This allows to use HTML/CSS and JavaScript to create user interfaces. I already integrated it into FSO some time ago so most of the work is already done.
    Pros:
    • Extremely flexible. Since it uses modern web technologies the UI can be pretty much everything.
    • Overwhelming amount of resources available online since web technologies are pretty much everywhere today
    Cons:
    • No direct scripting support. All dynamic actions have to be coded in JavaScript and Lua can't influence the UI directly. If a custom UI would be implemented using this it would need to be developed in Lua and JavaScript.
    • Uses a lot of resources. The chromium DLLs and resources are very big and would need to be distributed with FSO which increases the download size
    • Uncertainty about what is contained in the code. There were a few instances in the past where Google added something to Chromium which compromised the users privacy.

  • MyGUI: A relatively simple UI system which also has a WYSIWYG editor.
    Pros:
    • Self-contained solution with pretty small footprint
    • Has a renderer interface so we could integrate it into our own rendering engine
    Cons:
    • Not widely used so it may have bugs or incompatibilities
    • Limited amount of existing widgets
    • No built-in Lua support so we would need to build that ourself

  • CEGUI: A very mature GUI system that is used by a lot of other projects. Has a WYSIWYG editor.
    Pros:
    • Modular design which allows to use our own implementations for the renderer and resource loader
    • Mature project. A lot of resources available online
    • Seems to have support for Lua scripting but I am not entirely sure about that
    Cons:
    • Development slowed done somewhat. Not necessarily bad since we might not need any new features
    • Bigger footprint which may make is harder to integrate into FSO

  • libRocket: A UI system which uses an HTML and CSS dialect to declare how the UI should look like.
    Pros:
    • Has a decent amount of documentation since it's also in use in other projects
    • Designed for extensibility. Rendering backend can be exchanged.
    • Designed to be used in game engines
    • Has built-in Lua support
    Cons:
    • Development slowed down a bit in the last few years and the main website has not been updated in a while
    • Uses something that looks like HTML but is not entirely like it which may cause confusions

  • Use an HTML and CSS renderer (e.g. litehtml) and build our own system around it
    Pros:
    • Has the advantage of existing resources without the overhead of an entire web browser in FSO
    Cons:
    • Requires a lot of implementation work
    • Lua interface would need to be written by us
    • May be missing UI elements

  • Implement our own solution using Lua: Axem already demonstrated that the existing Lua interface can be used to create great UIs so we could extend that and integrate it into the engine.
    Pros:
    • No external dependencies
    • Can be expanded by simply editing the scripts that draw the UI
    Cons:
    • Requires some additional work to make the library universally usable
    • The entire UI would need to be written in Lua
    • Reinvents parts of the wheel


I have no strong preference for any of the solutions but IMO libRocket and CEGUI are probably the best solutions but I would like to hear alternative thoughts about that.

Those are all the solutions that I can think of at the moment. If you have an alternate solution then please add it below and I'll update this list. I would like to hear some thoughts from both coders and modders so we can find the best solution for everyone involved.

 

Offline m!m

  • 211
Re: Possible new UI systems for FSO
I have done some research and it appears that neither CEGUI nor libRocket support animated images directly. I found some resources which describe how such a feature could be added but it is not supported natively (the custom HTML engine obviously also doesn't support it).

 

Offline niffiwan

  • 211
  • Eluder Class
Re: Possible new UI systems for FSO
Well that's a pain, like you I'd considered those two to be the best to integrate. Do all the other options support animated images?
Creating a fs2_open.log | Red Alert Bug = Hex Edit | MediaVPs 2014: Bigger HUD gauges | 32bit libs for 64bit Ubuntu
----
Debian Packages (testing/unstable): Freespace2 | wxLauncher
----
m|m: I think I'm suffering from Stockholm syndrome. Bmpman is starting to make sense and it's actually written reasonably well...

 

Offline m!m

  • 211
Re: Possible new UI systems for FSO
APNG support can be added to CEF but AFAIK it's not supported natively. There is a Chrome extension for APNG display so it is definitely possible with some JavaScript. EFFs will not work since they are a custom FSO format.

I haven't looked at MyGUI in detail but it's pretty simplistic so I wouldn't expect it to have support for that.

The Lua solution obviously supports it since that uses our rendering code.

 

Offline Dragon

  • Citation needed
  • 212
  • The sky is the limit.
Re: Possible new UI systems for FSO
TBH, I don't think that something as bloated as Chromium is a good idea for FSO. I don't know what exactly do you mean by "UI", but if it messes with either the HUD or the pause dialog, it needs to run in-mission and not degrade performance. And even if it doesn't, I certainly wouldn't want loading times to go through the roof just because of UI (models and textures make it long enough). One thing I noticed about Chromium-based browsers is that they tend to gobble memory (Opera 12 FTW).

 

Offline m!m

  • 211
Re: Possible new UI systems for FSO
Performance wise, Chromium isn't too bad since the rendering doesn't actually happen in the FSO process but you are right that it is quite bloated which is why I currently prefer libRocket or CEGUI.

 

Offline m!m

  • 211
Re: Possible new UI systems for FSO
Since there haven't been any responses which favor a particular solution, I'll go ahead and try to integrate libRocket into FSO (I still have other stuff to do so it may take some time). I'm confident that I can integrate the existing lua bindings into FSO so modders will have full support for how the user interface behaves and looks. There are quite a few existing samples for libRocket so feel free to take a look at them. Here is a sample from the libRocket repository: https://github.com/libRocket/libRocket/blob/master/Samples/luainvaders/data/game.rml

 

Offline chief1983

  • Still lacks a custom title
  • Moderator
  • 212
  • ⬇️⬆️⬅️⬅️🅰➡️⬇️
    • Minecraft
    • Skype
    • Steam
    • Twitter
    • Fate of the Galaxy
Re: Possible new UI systems for FSO
Hopefully though, we create native menus for native engine features, so modders would only need to be adding menus for things the mod itself has added, probably via scripting.  Would feel weird to have to write Lua menus to access things in the engine itself, right?
Fate of the Galaxy - Now Hiring!  Apply within | Diaspora | SCP Home | Collada Importer for PCS2
Karajorma's 'How to report bugs' | Mantis
#freespace | #scp-swc | #diaspora | #SCP | #hard-light on EsperNet

"You may not sell or otherwise commercially exploit the source or things you created based on the source." -- Excerpt from FSO license, for reference

Nuclear1:  Jesus Christ zack you're a little too hamyurger for HLP right now...
iamzack:  i dont have hamynerge i just want ptatoc hips D:
redsniper:  Platonic hips?!
iamzack:  lays

 

Offline m!m

  • 211
Re: Possible new UI systems for FSO
What would be the advantage of locking down "native" menus this way? If possible, those screen could also be replaced with RML/Lua so that it can be made as flexible as possible.

 

Offline The E

  • He's Ebeneezer Goode
  • Moderator
  • 213
  • Nothing personal, just tech support.
    • Steam
    • Twitter
Re: Possible new UI systems for FSO
Yeah, at this point, I am convinced that the best way to do this is by eating our own dog food. If the end goal is to create a system that can be easily used from within lua, it makes little sense to then hardcode a bunch of stuff when we can just as well use embedded scripts that can be overridden by modders.
If I'm just aching this can't go on
I came from chasing dreams to feel alone
There must be changes, miss to feel strong
I really need lifе to touch me
--Evergrey, Where August Mourns

 

Offline m!m

  • 211
Re: Possible new UI systems for FSO
I want to add that an initial implementation would only allow modders to use whatever system we use to define additional screens. Porting the old UIs to the new system will likely require some additional lua API functions so it doesn't make sense to include that in the initial version.

 

Offline chief1983

  • Still lacks a custom title
  • Moderator
  • 212
  • ⬇️⬆️⬅️⬅️🅰➡️⬇️
    • Minecraft
    • Skype
    • Steam
    • Twitter
    • Fate of the Galaxy
Re: Possible new UI systems for FSO
Not every FSO dev also knows Lua, but it would be nice for them to be able to help expand and maintain the menus for primary engine features that have been added.  But embedding some Lua that does that works too I suppose.
Fate of the Galaxy - Now Hiring!  Apply within | Diaspora | SCP Home | Collada Importer for PCS2
Karajorma's 'How to report bugs' | Mantis
#freespace | #scp-swc | #diaspora | #SCP | #hard-light on EsperNet

"You may not sell or otherwise commercially exploit the source or things you created based on the source." -- Excerpt from FSO license, for reference

Nuclear1:  Jesus Christ zack you're a little too hamyurger for HLP right now...
iamzack:  i dont have hamynerge i just want ptatoc hips D:
redsniper:  Platonic hips?!
iamzack:  lays

 

Offline m!m

  • 211
Re: Possible new UI systems for FSO
Lua isn't that hard to learn but I understand your point. I would like to create a system where a mod could say "Hey, I want the briefing text to flicker randomly because the ship is damaged" or something like that. A native C++ solution would require code changes that would need to be maintained by the devs for something that might only be used once. If we have a Lua driven UI then that feature can be implemented in a custom mod and the SCP never has to worry about that. I would like to keep the UI easy to develop for SCP devs as well but I don't see a way of keeping the customizability of a fully scripted UI while still using C++.

 

Offline chief1983

  • Still lacks a custom title
  • Moderator
  • 212
  • ⬇️⬆️⬅️⬅️🅰➡️⬇️
    • Minecraft
    • Skype
    • Steam
    • Twitter
    • Fate of the Galaxy
Re: Possible new UI systems for FSO
I'm mostly just talking about any out of game menu, to control the actual things that the engine supports, that are currently controlled by command line options, etc.  People have clamored for an expanded options menu to adjust those kinds of things for years.  Especially all the graphical options, at some point enhanced input device control, etc.  I don't really see the need for Lua extensibility there as much as for them to be maintainable by the widest number of developers.  Adding Lua hooks to allow menus like that to be expanded even farther would be great, just not sure I want a situation where only devs that know Lua can maintain menus that were added to control those kinds of features.  We're not suggesting we actually replace existing menus with this kind of system though are we?  Although I guess something would have to be done make any new menus actually accessible.
Fate of the Galaxy - Now Hiring!  Apply within | Diaspora | SCP Home | Collada Importer for PCS2
Karajorma's 'How to report bugs' | Mantis
#freespace | #scp-swc | #diaspora | #SCP | #hard-light on EsperNet

"You may not sell or otherwise commercially exploit the source or things you created based on the source." -- Excerpt from FSO license, for reference

Nuclear1:  Jesus Christ zack you're a little too hamyurger for HLP right now...
iamzack:  i dont have hamynerge i just want ptatoc hips D:
redsniper:  Platonic hips?!
iamzack:  lays

 

Offline mjn.mixael

  • Cutscene Master
  • 212
  • Chopped liver
    • Steam
    • Twitter
Re: Possible new UI systems for FSO
As much as you might say LUA is easy to learn, that's relative to other code languages. You may find it simple, but I've tried many times to learn it and just can't get a grasp on it.

I'm not saying don't use it for this at all. Just keep in mind that not everyone speaks computer as well as you SCP guys.

EDIT: I think what I am saying is that with continuing to add more LUA, more coders might find themselves faced with more script/LUA requests and to be OK with that. :)
« Last Edit: February 17, 2017, 01:46:57 pm by mjn.mixael »
Cutscene Upgrade Project - Mainhall Remakes - Between the Ashes
Youtube Channel - P3D Model Box
Between the Ashes is looking for committed testers, PM me for details.
Freespace Upgrade Project See what's happening.

 

Offline m!m

  • 211
Re: Possible new UI systems for FSO
I'm mostly just talking about any out of game menu, to control the actual things that the engine supports, that are currently controlled by command line options, etc.  People have clamored for an expanded options menu to adjust those kinds of things for years.  Especially all the graphical options, at some point enhanced input device control, etc.  I don't really see the need for Lua extensibility there as much as for them to be maintainable by the widest number of developers.  Adding Lua hooks to allow menus like that to be expanded even farther would be great, just not sure I want a situation where only devs that know Lua can maintain menus that were added to control those kinds of features.  We're not suggesting we actually replace existing menus with this kind of system though are we?  Although I guess something would have to be done make any new menus actually accessible.
Replacing existing menus with the new system is definitely something that I would like to do at some point once we have a functioning UI system in place. The current UI is old and suffers from a lot of issues such as being unable to scale correctly to widescreen resolutions which could be fixed with the new UI system. Running the UI from within the engine would require duplicating a lot of systems such as event handling that would be handled pretty much automatically in Lua. I'm not a huge fan of introducing another language but at least for an options menu we could make the lua API function in such a way that it would be easy to add new options without having to edit the Lua code.

As much as you might say LUA is easy to learn, that's relative to other code languages. You may find it simple, but I've tried many times to learn it and just can't get a grasp on it.

I'm not saying don't use it for this at all. Just keep in mind that not everyone speaks computer as well as you SCP guys.

EDIT: I think what I am saying is that with continuing to add more LUA, more coders might find themselves faced with more script/LUA requests and to be OK with that. :)
That's not really news since every UI change requires a code change at the moment but with a new UI system it would actually be feasible to change the code :p

The huge advantage of a markup language based system is that a modder can change how the UI looks without having to worry about how it behaves and, if required, the behaviors can be changed without having to dive into engine development which can get a lot more complicated than editing a few lines in a more or less simple Lua script.

 

Offline mjn.mixael

  • Cutscene Master
  • 212
  • Chopped liver
    • Steam
    • Twitter
Re: Possible new UI systems for FSO
:yes:
Cutscene Upgrade Project - Mainhall Remakes - Between the Ashes
Youtube Channel - P3D Model Box
Between the Ashes is looking for committed testers, PM me for details.
Freespace Upgrade Project See what's happening.

 

Offline AdmiralRalwood

  • 211
  • The Cthulhu programmer himself!
    • Skype
    • Steam
    • Twitter
Re: Possible new UI systems for FSO
I'm all for replacing the existing UI entirely; after going all over the UI code as part of making the FS2 demo data not crash, I discovered that our UI code is even more of a complete mess than I only vaguely thought it was (as part of the PR that fixed demo compatibility, I fixed at least one instance of a bitmap that never unloaded until FSO closed... not by design, but because it was unlocked/released in the wrong order).
Ph'nglui mglw'nafh Codethulhu GitHub wgah'nagl fhtagn.

schrödinbug (noun) - a bug that manifests itself in running software after a programmer notices that the code should never have worked in the first place.

When you gaze long into BMPMAN, BMPMAN also gazes into you.

"I am one of the best FREDders on Earth" -General Battuta

<Aesaar> literary criticism is vladimir putin

<MageKing17> "There's probably a reason the code is the way it is" is a very dangerous line of thought. :P
<MageKing17> Because the "reason" often turns out to be "nobody noticed it was wrong".
(the very next day)
<MageKing17> this ****ing code did it to me again
<MageKing17> "That doesn't really make sense to me, but I'll assume it was being done for a reason."
<MageKing17> **** ME
<MageKing17> THE REASON IS PEOPLE ARE STUPID
<MageKing17> ESPECIALLY ME

<MageKing17> God damn, I do not understand how this is breaking.
<MageKing17> Everything points to "this should work fine", and yet it's clearly not working.
<MjnMixael> 2 hours later... "God damn, how did this ever work at all?!"
(...)
<MageKing17> so
<MageKing17> more than two hours
<MageKing17> but once again we have reached the inevitable conclusion
<MageKing17> How did this code ever work in the first place!?

<@The_E> Welcome to OpenGL, where standards compliance is optional, and error reporting inconsistent

<MageKing17> It was all working perfectly until I actually tried it on an actual mission.

<IronWorks> I am useful for FSO stuff again. This is a red-letter day!
* z64555 erases "Thursday" and rewrites it in red ink

<MageKing17> TIL the entire homing code is held up by shoestrings and duct tape, basically.

 

Offline Axem

  • 211
Re: Possible new UI systems for FSO
So how does one make a UI with librocket, exactly? You say its something that looks like html (and boy does it) but can we use like html... things to preview UI screens?

 

Offline m!m

  • 211
Re: Possible new UI systems for FSO
No, unfortunately libRocket uses its own dialect of HTML so you won't be able to preview it in a browser. It has a visual debugger tool which allows you to inspect the properties of individual elements and I'm sure that I could add "Reload" option so you can edit the markup without having to restart the game.