Author Topic: wxLauncher 2.0 Request for Comments  (Read 25964 times)

0 Members and 1 Guest are viewing this topic.

Offline Iss Mneur

  • 210
  • TODO:
wxLauncher 2.0 Request for Comments
As you may already know kkmic and I, along with input from (alphabetical) chief1983, FUBAR, Fury, Karajorma, Portej05, The_E, Zacam, and others (sorry, I don't have records of all the names) have been developing a new launcher for FreeSpace and it's Total Conversions. It is now time for the wxLauncher team to start planning version 2.0, whose purpose is to complete the package and add the installer to wxLauncher. For those that remember the initial proposal that kkmic and I presented, we asked for suggestions on what features the community would like for this component of the launcher. I don't know about kkmic but I have received a great deal of feedback about what features are in demand.

This document is not about the how, but the what. That is, this discussion is to be about the short and long term features of wxLauncher 2.0, not specifics of how we implement (ie. algorithms and designs, not code libraries and languages) these features.  The discussion of the specific how will start at a later date.

Table of Contents
The following essay has the following major topic categories:
  • Terms
  • Overview of wxLauncher 2.0
  • Looking for input
  • Scripting
  • Names

Terms
I know most of you will likely know the definitions of these terms, but I am defining them here so that we are clear.
  • MOD - Content released for the FS2Open Engine that changes what the TC contains.  For the purposes of wxL, anything released for the FS2Open Engine that is not a TC is considered a MOD. This this includes campaigns, texture or ship packs, etc.
  • modder - The person or team that produces a TC or MOD.
  • TC - A Total Conversion. If a FS2Open binary was dropped into a TC folder it would run without error.  For the purposes of wxL, Freespace 2 itself is a TC.
  • wxL - wxLauncher, see the wxLauncher Public Alpha release thread.
  • wxL2 - wxLauncher 2.0, see the Overview of wxLauncher 2.0 section.

Overview of wxLauncher 2.0
The general idea is to make wxLauncher into a platform like Steam or Impulse and allow for the complete management of MODs and TCs.

The specific ideas that we are pulling from Steam or Impulse are: a central location for all FSO engine content (TCs and MODs) to be catalogued, make installing the catalogued MODs and TCs easy for all users, and allow modders (TCs in particular) to offer off-line installers.

Contrary to some opinions, wxL2 would not invalidate the existing installs, except for non-mod.ini compliant installs.  Any MOD that uses the mod.ini will be a dropin for the wxLauncher. To be clear, in the eyes of the wxLauncher team, having wxL2 break existing installs and/or invalidate current install instructions is not negotiable.  The result of this policy is that even if wxLauncher was used to install everything, there is nothing stopping anyone from manually installing a MOD.  That being said, for installations that are not mod.ini compliant (user error (or age, but this will be harder because of need for meta-data)), wxLauncher will be capable of correcting a damaged and/or out-dated TC folder. That is, if the user has put VPs in the wrong spot or has truncated files wxLauncher will be able to detect and fix these issues on a per file basis.

The wxL2 platform's ability to integrate with off-line installers would also allow projects like the Bablyon Project who like to offer DVD iso installs of the content to be able to to still do this but use wxL2 as their actual installer and gain the benefits that wxL provides.  The wxL2 platform would also allow the modders to just provide an installer that is basically just a downloader.

The platform would also allow for a modder to provide a "net" installer that only downloads what is needed.  For example, Blue Planet, who are going to release Blue Planet 2 as a MOD that depends on having Blue Planet 1 installed (and updated).  wxL2 could for the release of BP2 could use wxL2 as a net installer that will check and update in needed the BP1 install then download the BP2 install.  As well, BP does require the MediaVPs so wxL2 could also check and update, if needed, the MediaVPs.

If this is starting to sound like a package manager, you are correct, package management is one of a few things that Steam is good at. In keeping with how wxLauncher is currently setup, anything that requires internet access will be optional and wxLauncher will prompt on first run about these types of things.

The general design of the central location to catalogue and distribute the MODs and TCs will likely be an integration of both the local wxLauncher front-end (install tab of the current wxLauncher builds) and a website/server to provide the back-end support required for the versioning that the installer will require. Potentially we would also allow for installing previous versions of MODs and TCs.

At this point we are not looking at supporting any FSO build older that 3.6.9, thought this could be bumped to 3.6.10 depending on the time lag, difficulty of implementing the support for .9 and the demand for .9 support. This also implys that retail executable support is outside of the scope of this project, unless of course there is sufficient demand. Though we do plan on implementing the ability for wxLauncher to read the retail installer files (gog.com and OEM/RTM disks) on all platforms.  This is so that we can expedite and ease the installation of FreeSpace on all platforms.  Yes, this means that wxLauncher is not going to be as simple to install/upgrade as the current (win32) launcher, but as the intention of this project is to make FSO easy(-ier) for new people, like the people that WCS and TBP already attract and the people that Diaspora and Fate of the Galaxy could attract to the community.

As for the installer capability, we are looking at being able to have wxLauncher able to edit .vp files on the spot and to have on-the-fly compression of the .vp files so that we can lower bandwidth requirements for updates. The purpose of this capability is to allow the wxL2 to apply diffs of the contents of .vps.

Looking for input
The following are items that we are looking for more information on.
  • Should wxLauncher be able to build FSO from source on Linux at least, maybe windows and osx?
  • Because we are keeping the old files so that we can provide specific diffs, do we allow the user to request older versions of resources? How long do we keep them the old files?
  • Who/Where are the mod files going to be stored?  In particular wxL2 would require MOD/TC data hosting with resume support that allows hot linking.

Scripting
What language or syntax is meant by "scripting" for wxL2 is still up in the air.  At this point it is between Python, Lua, or a custom "script" based on .tbl.

Names
We are currently looking for a catchy name for wxL2 that is not as generic.  To be clear I am not opposed to it continuing to be called "wxLauncher 2.0".  This name will likely also be used in the domain for the wxL2 central catalogue website.  Current suggestions are (in no particular order):
  • wxL or wxL2 - Truncated wxLauncher 2.0
  • NILS - New Installer and Launcher System)
  • F-Soil - FreeSpace Open Installer and Launcher
  • Launcher - Simplictiy at its finest
  • FOUL - FreeSpace Open Updater & Launcher
  • FROLIC  - Freespace Open Launcher & Installer Combo
  • FSOELIP - Feespace Open Engine Launcher & Installer Platform
  • FSELIP - Feespace Engine Launcher & Installer Platform


Change log:
  • 2010-04-09 - 22-49: Added FROLIC.
  • 2010-04-10 - 17-33: Added Chief to contributors list
  • 2010-06-09 - 16-06: Added FSOELIP and FSELIP; Removed TL;DR from top
« Last Edit: June 12, 2010, 02:24:41 pm by Iss Mneur »
"I love deadlines. I like the whooshing sound they make as they fly by." -Douglas Adams
wxLauncher 0.9.4 public beta (now with no config file editing for FRED) | wxLauncher 2.0 Request for Comments

 

Offline chief1983

  • Still lacks a custom title
  • Moderator
  • 212
  • ⬇️⬆️⬅️⬅️🅰➡️⬇️
    • Minecraft
    • Skype
    • Steam
    • Twitter
    • Fate of the Galaxy
Re: wxLauncher 2.0 Request for Comments
I know you can't remember everyone, but me?  Even though we talk about this thing like every other day?  Is this punishment for not testing OS X soon enough?  :P
Fate of the Galaxy - Now Hiring!  Apply within | Diaspora | SCP Home | Collada Importer for PCS2
Karajorma's 'How to report bugs' | Mantis
#freespace | #scp-swc | #diaspora | #SCP | #hard-light on EsperNet

"You may not sell or otherwise commercially exploit the source or things you created based on the source." -- Excerpt from FSO license, for reference

Nuclear1:  Jesus Christ zack you're a little too hamyurger for HLP right now...
iamzack:  i dont have hamynerge i just want ptatoc hips D:
redsniper:  Platonic hips?!
iamzack:  lays

 

Offline Tomo

  • 28
Re: wxLauncher 2.0 Request for Comments
Not sure if you're already doing this, but there should be buttons for FS2-Open and FRED, and the option to run the Debug and Release of either, all using the same MOD.

The current launcher needs quite a bit of background knowledge and doesn't do much handholding, which isn't great for new people.

On rebuilding from source:
  • On Linux, this feature is required due to the differences in APIs between distributions - a lot of apps don't work correctly (or at all) if not built from source.
  • Under Windows, this feature is not a good idea:
    • No build environments are included by default, so one would have to be provided. We can't provide MSVC which is the build environment currently used for Windows development as that's owned by MS.
    • It would take a very long time compared to what Windows users are used to. My PC runs FS2-Open pretty well, but takes around half an hour to do a full build of FS2-Open 3.6.12 in MSVC 2008. Windows users don't expect to do this.

However, there does need to be a choice of Win32 Standard/SSE/SSE2 and Win64 builds under Windows - selection should be automatic based on the detected OS and CPU capabilities, but an 'advanced' user could override it (in case they want to try something or the detection is incorrect)

I don't use Macs so can't comment on that platform.

On old versions of MODs
Is there any particular reason for not leveraging an existing proper version-control system that already handles the full revision history?
- Each individual MOD folder would be its own repository and thus have a contiguous revision history.
(Eg Updates to BP2 do not affect BP1 revision numbers)

Each 'commit' to these version control repositories would only ever be a proper release (never beta), so all commits should be good. (At least as good as shrinkwrap software anyway)
Each MOD lists the revision numbers of all the dependencies that it has been tested with.
If the local version is older than the oldest 'tested' revision it warns and suggests updating to the newest or newest tested one.

Example:
  • MVP Revision 20 is released and I download it.
  • BP1 Revision 5 was tested with MVP Revisions 12 thru 15.
  • System defaults to MVP Rev 20, offering to switch to MVPs Rev 12 thru 15 as alternatives on user selection. Other MVP revisions would not be offered.

It would then be up to a given modder to test with a sensible range of versions of the items it depends on. We don't have multiple inheritance so this isn't an onerous task - eg BP2 depends on BP1 and MediaVPs, but not anything that BP1 says it depends on.

If a MOD is old and therefore completely untested, then the latest version of all dependencies is assumed, but the end user can roll any dependency back to any older version.

When a modder wishes to test their MOD with other revisions, they activate the 'Expert mode' to get the full, dated list of dependency revisions to consider which to test with.
- Administrators of the new system can also add tags to any MOD to indicate recommended versions, to handle orphaned and untested works (eg People can post here saying that MOD x doesn't work with MVP release y but is fine with z)

New MODs are self-evidently tested by the modder with the versions of the dependencies that they had at the time, even if they didn't want to bother testing with other versions.
I expect that only very large MODs would be tested with more than one version of dependencies at release.

As the new system offers automatic mod.ini creation anyway, this same automatic system can be used by the modder to generate the 'release' mod.ini (while in Expert Mode)

This gives the information required to determine when to archive/discard very old versions of things:
  • There will be the meta-data available saying which revisions are no longer required by anything in the system.
  • It will be possible to tell if a very old revision of something is only used by one or two MODs, and then user-testing can be done to determine if these can use the very newest version without problems.
- 'Untested' MODs are a problem, but they won't stay untested for long if anyone is playing them.

Otherwise we are likely to discard things that someone is relying on.

Selection of version control engine is left open. No holy wars please!
However, it should be something where you can switch between revisions without redownloading them, as otherwise testing across multiple revisions becomes a pain in the proverbial.

Scripting
Lua is the only choice.

It's the one used in FS2-Open, so many people already know it from both ends.
There is no point in asking modders to learn a new language, or coders to learn new ways of handling hooks.

Further notes
The entire system is going to need fairly powerful Administrators.
This is going to be quite a lot of work.

Turey's Installer fell over for a while because nobody was doing the important work needed to keep that up to date.

So whatever system is built, there will need to be a very simple and easy to use multiplatform tool to handle the updates that can be taught to your average modder in a few minutes over IRC or similar.

Otherwise, sooner or later the new system will lose a key person (even if only temporarily) and fall apart.

 
Re: wxLauncher 2.0 Request for Comments
One word: Versioning.
STRONGTEA. Why can't the x86 be sane?

 

Offline Mongoose

  • Rikki-Tikki-Tavi
  • Global Moderator
  • 212
  • This brain for rent.
    • Minecraft
    • Steam
    • Something
Re: wxLauncher 2.0 Request for Comments
I feel like this came up already in one of the other discussions on the subject, but the ability to create "profiles" of specific launcher flag combinations for particular mods (beyond the -mod flag itself) would be very useful.  It doesn't come up all that often, but there are certain mods that require specific launcher flags to be turned on or off; beyond the obvious TBP and WCS flags, I know Windmills needed certain lighting settings to achieve its effect.  I can't think of a specific application, but one could even go beyond just flags to general settings as a whole, like changing the video resolution.  I'm not sure if this would be best achieved by integration with the existing mod.ini system, or some external means, but I'll leave that to those who are actually technically-oriented.

 

Offline The E

  • He's Ebeneezer Goode
  • Moderator
  • 213
  • Nothing personal, just tech support.
    • Steam
    • Twitter
Re: wxLauncher 2.0 Request for Comments
I think I mentioned this on IRC before, but I'll mention it again here.

IMHO, the best model for version management would be something akin to Debians' apt. The launcher would record metadata about the mod (comprised of something like version + md5 checksum of any given asset), and use that to determine whether or not an update is needed.
If I'm just aching this can't go on
I came from chasing dreams to feel alone
There must be changes, miss to feel strong
I really need lifе to touch me
--Evergrey, Where August Mourns

 

Offline Iss Mneur

  • 210
  • TODO:
Re: wxLauncher 2.0 Request for Comments
Is this punishment for not testing OS X soon enough?  :P
Yes :P (I have edited the first post to include you)

Not sure if you're already doing this, but there should be buttons for FS2-Open and FRED, and the option to run the Debug and Release of either, all using the same MOD.
Yes we are, something related to this is going to be in wxLauncher by 1.0.

The current launcher needs quite a bit of background knowledge and doesn't do much handholding, which isn't great for new people.
This is more related to wxLauncher 1.0, but yes, this is one of those "benefits that wxL provides".

On rebuilding from source:
  • On Linux, this feature is required due to the differences in APIs between distributions - a lot of apps don't work correctly (or at all) if not built from source.
  • Under Windows, this feature is not a good idea:
    • No build environments are included by default, so one would have to be provided. We can't provide MSVC which is the build environment currently used for Windows development as that's owned by MS.
    • It would take a very long time compared to what Windows users are used to. My PC runs FS2-Open pretty well, but takes around half an hour to do a full build of FS2-Open 3.6.12 in MSVC 2008. Windows users don't expect to do this.
Thank you, this is why it is on the list of questions, we are looking for more input on this item, namely on how integrated we want it to be with wxL.  That is, do we just provide some pre-built binaries for the major linux distros (.debs, .rpms, etc) and coerce the platform's package manger into doing our bidding; or do we make the linux users compile, then copy the binaries into the TC folder as we do now; or do we have wxL interface with the platform's build system to build the binaries for the user?

Also, VC++ Express editions are freely available to anyone that wants to download it. The VC++ Express edition allows the compiling of FSO on windows, even though FRED is currently not available (but FRED not being build-able is a problem on Linux as well).

However, there does need to be a choice of Win32 Standard/SSE/SSE2 and Win64 builds under Windows - selection should be automatic based on the detected OS and CPU capabilities, but an 'advanced' user could override it (in case they want to try something or the detection is incorrect)
While standard builds are supposed to vanish, it is an interesting thought.

On old versions of MODs
Is there any particular reason for not leveraging an existing proper version-control system that already handles the full revision history?
[snip]
This is essentially what I had in mind for the installer. Though it will likely be more ackin to a package manager than a version control system. Also, I am not convinced that we will be able to drop in an existing version control system for two reasons they don't handle large groups of binary files that well, and they are hard to tailor to the specifics of what we will need to do.  That being said, as I noted in the OP, I do not want to discuss the specifics of the implementation wxL2 at this point. Also see my response to The_E at the bottom of this post.

Scripting
Lua is the only choice.

It's the one used in FS2-Open, so many people already know it from both ends.
There is no point in asking modders to learn a new language, or coders to learn new ways of handling hooks.
Noted. Still looking for more input from others, of course.

Further notes
The entire system is going to need fairly powerful Administrators.
This is going to be quite a lot of work.

Turey's Installer fell over for a while because nobody was doing the important work needed to keep that up to date.

So whatever system is built, there will need to be a very simple and easy to use multiplatform tool to handle the updates that can be taught to your average modder in a few minutes over IRC or similar.

Otherwise, sooner or later the new system will lose a key person (even if only temporarily) and fall apart.
Agreed. The administrative interface of the wxL2 platform is something that is going to be a large part of this project.  Of course we will try and exploituse the input of the end users to help with the administration. 

Yes, this will be a lot of work, and to be clear this is not even in the prototyping phase. That is, actual technical work will not start on this until wxLauncher 1.0 is released.

I feel like this came up already in one of the other discussions on the subject, but the ability to create "profiles" of specific launcher flag combinations for particular mods (beyond the -mod flag itself) would be very useful.  It doesn't come up all that often, but there are certain mods that require specific launcher flags to be turned on or off; beyond the obvious TBP and WCS flags, I know Windmills needed certain lighting settings to achieve its effect.  I can't think of a specific application, but one could even go beyond just flags to general settings as a whole, like changing the video resolution.  I'm not sure if this would be best achieved by integration with the existing mod.ini system, or some external means, but I'll leave that to those who are actually technically-oriented.
Is this any different than the current planned implementation of profiles wxL? (See the Overview of wxLauncher linked in the first paragraph of the first post).

I think I mentioned this on IRC before, but I'll mention it again here.

IMHO, the best model for version management would be something akin to Debians' apt. The launcher would record metadata about the mod (comprised of something like version + md5 checksum of any given asset), and use that to determine whether or not an update is needed.

Yes, is why I mentioned package managers in the 6th paragraph of the "Overview of wxLauncher 2.0".  And a package manger model is the most likely solution, but Tomo's response is the reason that we have posted this Request for Comments to start with.
"I love deadlines. I like the whooshing sound they make as they fly by." -Douglas Adams
wxLauncher 0.9.4 public beta (now with no config file editing for FRED) | wxLauncher 2.0 Request for Comments

 

Offline Mongoose

  • Rikki-Tikki-Tavi
  • Global Moderator
  • 212
  • This brain for rent.
    • Minecraft
    • Steam
    • Something
Re: wxLauncher 2.0 Request for Comments
I feel like this came up already in one of the other discussions on the subject, but the ability to create "profiles" of specific launcher flag combinations for particular mods (beyond the -mod flag itself) would be very useful.  It doesn't come up all that often, but there are certain mods that require specific launcher flags to be turned on or off; beyond the obvious TBP and WCS flags, I know Windmills needed certain lighting settings to achieve its effect.  I can't think of a specific application, but one could even go beyond just flags to general settings as a whole, like changing the video resolution.  I'm not sure if this would be best achieved by integration with the existing mod.ini system, or some external means, but I'll leave that to those who are actually technically-oriented.
Is this any different than the current planned implementation of profiles wxL? (See the Overview of wxLauncher linked in the first paragraph of the first post).
No, that's pretty much exactly what I was hoping for. :) I thought I'd remembered it being discussed earlier, but I didn't happen to check the link.

 

Offline Tomo

  • 28
Re: wxLauncher 2.0 Request for Comments
While standard builds are supposed to vanish, it is an interesting thought.
Oops - I didn't mean Standard/Inferno, rather 'no SSE', SSE and SSE2. (Although I suppose that eventually there won't be any machines that don't have SSE)

Thank you, this is why it is on the list of questions, we are looking for more input on this item, namely on how integrated we want it to be with wxL.  That is, do we just provide some pre-built binaries for the major linux distros (.debs, .rpms, etc) and coerce the platform's package manger into doing our bidding; or do we make the linux users compile, then copy the binaries into the TC folder as we do now; or do we have wxL interface with the platform's build system to build the binaries for the user?

Getting the core executables into the 'common' platform package managers would be a very good thing, but that would fall outside the scope of this project - if it's going into the genuine package manager then it needs to be managed by the distribution, not us.
I'm unsure of exactly how to ask Debian, Ubunbtu, Red Hat etc to add a given set of binaries to their package managers, and they may not anyway as the required game data isn't free, and can't be included in any free package.

So, getting wxL to interface with the platform's build system would be perfect, and a very good measure regardless of whether Debian et al add FS2-Open to their package managers.
The 'official' packages will always be a bit behind, and now that Linux is starting to go mainstream there will be an increasing number of users who aren't sure how to rebuild and copy themselves.

Also, VC++ Express editions are freely available to anyone that wants to download it. The VC++ Express edition allows the compiling of FSO on windows, even though FRED is currently not available (but FRED not being build-able is a problem on Linux as well).
While VC++ Express editions are available for zero money, it's not 'freely available'.
A Windows user has to download it from Microsoft (at a location that moves each year) and install then activate it providing quite a lot of info to Microsoft, and agreeing to a rather large licence agreement.

We can't ask Windows end users to do this simply to play the game - it will scare them!
- Anyone interested in the code for the game engine can be pointed to this forum, where the latest available of VC++ Express, suitable svn tools etc are easy to find.

 

Offline The E

  • He's Ebeneezer Goode
  • Moderator
  • 213
  • Nothing personal, just tech support.
    • Steam
    • Twitter
Re: wxLauncher 2.0 Request for Comments
Well, what you can do is offer to build from source if VS Express or the full VS is installed.
If I'm just aching this can't go on
I came from chasing dreams to feel alone
There must be changes, miss to feel strong
I really need lifе to touch me
--Evergrey, Where August Mourns

 

Offline chief1983

  • Still lacks a custom title
  • Moderator
  • 212
  • ⬇️⬆️⬅️⬅️🅰➡️⬇️
    • Minecraft
    • Skype
    • Steam
    • Twitter
    • Fate of the Galaxy
Re: wxLauncher 2.0 Request for Comments
Tomo, you can probably fire up the command line compiler after install without ever needing to activate the project.  But there are non-MSVC compilers for Windows too, it's just that we don't use them currently for anything else in the SCP area.  But source releases for Windows end users are probably just a bad idea in general.
Fate of the Galaxy - Now Hiring!  Apply within | Diaspora | SCP Home | Collada Importer for PCS2
Karajorma's 'How to report bugs' | Mantis
#freespace | #scp-swc | #diaspora | #SCP | #hard-light on EsperNet

"You may not sell or otherwise commercially exploit the source or things you created based on the source." -- Excerpt from FSO license, for reference

Nuclear1:  Jesus Christ zack you're a little too hamyurger for HLP right now...
iamzack:  i dont have hamynerge i just want ptatoc hips D:
redsniper:  Platonic hips?!
iamzack:  lays

 

Offline The E

  • He's Ebeneezer Goode
  • Moderator
  • 213
  • Nothing personal, just tech support.
    • Steam
    • Twitter
Re: wxLauncher 2.0 Request for Comments
Yep. Using the precompiled exes is the best way to go on Windows.
If I'm just aching this can't go on
I came from chasing dreams to feel alone
There must be changes, miss to feel strong
I really need lifе to touch me
--Evergrey, Where August Mourns

 

Offline Iss Mneur

  • 210
  • TODO:
Re: wxLauncher 2.0 Request for Comments
Tomo, you can probably fire up the command line compiler after install without ever needing to activate the project.  But there are non-MSVC compilers for Windows too, it's just that we don't use them currently for anything else in the SCP area.  But source releases for Windows end users are probably just a bad idea in general.

Yes you can, the activation is only for visual studio itself and not build tools.

To be clear, I was not planning on having wxLauncher build the binaries on Linux, OSX or especially Windows unless actually needed.  As the newbie linux users are likely to be on the major popular distros anyway, building on linux will not be needed in all cases. But as Tomo noted previously, the linux installs will likely have to build the releases anyway because we are not going to be able to maintain binary packages for all distros.  But because of this potential need to build the binaries anyway, there is no reason that we cannot expand this ability to all platforms.
"I love deadlines. I like the whooshing sound they make as they fly by." -Douglas Adams
wxLauncher 0.9.4 public beta (now with no config file editing for FRED) | wxLauncher 2.0 Request for Comments

 

Offline Spicious

  • Master Chief John-158
  • 210
Re: wxLauncher 2.0 Request for Comments
Which APIs aren't compatible across distros?

What is scripting meant to achieve in this context?

 

Offline Iss Mneur

  • 210
  • TODO:
Re: wxLauncher 2.0 Request for Comments
Which APIs aren't compatible across distros?
Its not that the APIs necessarily change but the location of the .so (dynamic link library) is not consistent.  Some platforms have support for the LSB to help minimize this, but I don't know if the APIs that we require are actually specified in the LSB.

Also some of the APIs that we link to are only source compatible based on the version that is actually installed. Lua and the SDL would be good examples of APIs that we use that we are source compatible with, that is, we can use a range of versions of the APIs but only when compiled against those APIs.

What is scripting meant to achieve in this context?
Scripting mostly for configuring wxLauncher so that it is able to configure FSO.  That is, automatically create profiles for the user to choose.

Scripting may also be needed to deal with some of the non-standard installers that some mod teams use, though this may or may not be a problem depending on the specifics of how wxL2 is implemented and how it handles the MOD data.  That is, does wxL2 force all data to be distributed with the wxL2 platform as zipped .vps or do we allow the mod teams to provide an self extracting installer. If we do force .vp only systems we are also potentially going to need someway for the TCs (in particular) to create shortcuts to starting the TC.

I realize that everything that I describe above about scripting could be accomplished by a .tbl or .ini-like configuration file or even in the MOD/TC's meta-data that wxL2 has on file.  On the other hand allowing a MOD or TC to have some code executed on its behalf to implement features that wxLauncher will not implement could be handy (like extracting unsupported file archives).

The potential FSO binary building system could also exploit a proper scripting language to allow for different scripts to execute on different platforms so that we can move this data out of the wxL binary so that it can be updated without forcing an update to wxL itself to support a new platform.

I also realize that the above ideas about scripting could be needlessly complex, but that is why we are discussing this here, to help determine these things.
"I love deadlines. I like the whooshing sound they make as they fly by." -Douglas Adams
wxLauncher 0.9.4 public beta (now with no config file editing for FRED) | wxLauncher 2.0 Request for Comments

 

Offline Iss Mneur

  • 210
  • TODO:
Re: wxLauncher 2.0 Request for Comments
What is scripting meant to achieve in this context?
Scripting mostly for configuring wxLauncher so that it is able to configure FSO.  That is, automatically create profiles for the user to choose.

Scripting may also be needed to deal with some of the non-standard installers that some mod teams use, though this may or may not be a problem depending on the specifics of how wxL2 is implemented and how it handles the MOD data.  That is, does wxL2 force all data to be distributed with the wxL2 platform as zipped .vps or do we allow the mod teams to provide an self extracting installer. If we do force .vp only systems we are also potentially going to need someway for the TCs (in particular) to create shortcuts to starting the TC.

I realize that everything that I describe above about scripting could be accomplished by a .tbl or .ini-like configuration file or even in the MOD/TC's meta-data that wxL2 has on file.  On the other hand allowing a MOD or TC to have some code executed on its behalf to implement features that wxLauncher will not implement could be handy (like extracting unsupported file archives).

The potential FSO binary building system could also exploit a proper scripting language to allow for different scripts to execute on different platforms so that we can move this data out of the wxL binary so that it can be updated without forcing an update to wxL itself to support a new platform.

I also realize that the above ideas about scripting could be needlessly complex, but that is why we are discussing this here, to help determine these things.

I remembered the other reason that I wanted scripting. I promised The_E (and the rest of the support ninjas) to have wxL help in diagnosing the users problems.  That is, have wxL run tests and diagnostics to help the users before the post even makes it to the forum.  Since starting wxL I have read the support forums very regularly, and it seems a large number of the issues can be dealt with by very simple automation.  Of course some of the automation can be done in C++, but other parts can and should be more easily changeable without requiring that everyone download a point release of wxL, when only a couple of lines change.
"I love deadlines. I like the whooshing sound they make as they fly by." -Douglas Adams
wxLauncher 0.9.4 public beta (now with no config file editing for FRED) | wxLauncher 2.0 Request for Comments

 

Offline chief1983

  • Still lacks a custom title
  • Moderator
  • 212
  • ⬇️⬆️⬅️⬅️🅰➡️⬇️
    • Minecraft
    • Skype
    • Steam
    • Twitter
    • Fate of the Galaxy
Re: wxLauncher 2.0 Request for Comments
That sounds like it could open up a bunch of opportunities, I like it.
Fate of the Galaxy - Now Hiring!  Apply within | Diaspora | SCP Home | Collada Importer for PCS2
Karajorma's 'How to report bugs' | Mantis
#freespace | #scp-swc | #diaspora | #SCP | #hard-light on EsperNet

"You may not sell or otherwise commercially exploit the source or things you created based on the source." -- Excerpt from FSO license, for reference

Nuclear1:  Jesus Christ zack you're a little too hamyurger for HLP right now...
iamzack:  i dont have hamynerge i just want ptatoc hips D:
redsniper:  Platonic hips?!
iamzack:  lays

 
Re: wxLauncher 2.0 Request for Comments
Thank you, this is why it is on the list of questions, we are looking for more input on this item, namely on how integrated we want it to be with wxL.  That is, do we just provide some pre-built binaries for the major linux distros (.debs, .rpms, etc) and coerce the platform's package manger into doing our bidding; or do we make the linux users compile, then copy the binaries into the TC folder as we do now; or do we have wxL interface with the platform's build system to build the binaries for the user?

Getting the core executables into the 'common' platform package managers would be a very good thing, but that would fall outside the scope of this project - if it's going into the genuine package manager then it needs to be managed by the distribution, not us.
I'm unsure of exactly how to ask Debian, Ubunbtu, Red Hat etc to add a given set of binaries to their package managers, and they may not anyway as the required game data isn't free, and can't be included in any free package.

So, getting wxL to interface with the platform's build system would be perfect, and a very good measure regardless of whether Debian et al add FS2-Open to their package managers.
The 'official' packages will always be a bit behind, and now that Linux is starting to go mainstream there will be an increasing number of users who aren't sure how to rebuild and copy themselves.

I have a package for openSUSE... but users need to build it. That's becasue the non-commercial part of the license. Such a thing is against OSI's Open Source definition and FSF's Free Software definition, so most distros will not package it. And it could be also a problem for openSUSE because of Novell (and RedHat/Fedora? Others?). Things would be easier without that part of the license, but I suppose it would be difficult to change.

Which APIs aren't compatible across distros?
Its not that the APIs necessarily change but the location of the .so (dynamic link library) is not consistent.  Some platforms have support for the LSB to help minimize this, but I don't know if the APIs that we require are actually specified in the LSB.

Also some of the APIs that we link to are only source compatible based on the version that is actually installed. Lua and the SDL would be good examples of APIs that we use that we are source compatible with, that is, we can use a range of versions of the APIs but only when compiled against those APIs.

The dynamic libraries can be anywhere specified at /etc/ld.so.conf (and you can add more paths with a file in /etc/ld.so.conf.d/ with a sane /etc/ld.so.conf). But in every normal distro they will be at /usr/lib or /usr/lib64. What's the standard place in Windows? If you prefer you can use rpath and put the libraries in the same dir than the binary or any other path (absolute or relative, with $ORIGIN). Or naturally static link them. But I don't know why this was raised, the binary has no the library paths hardcoded and you don't need to provide any library, only a binary.

About the libraries that the binary uses... If they break source or binary compatibility in Linux they will also break it in Windows, and:
- libSDL: stable if just because it is rarely updated. I don't understand what you said about it, all the 1.2.x series are binary compatible.
- libvorbis: stable
- libvorbisfile: stable
- libogg: stable
- libtheora: stable
- libpng: ok, it recently broke backward compatibility. But has been stable for a long time, and anyway distros will provide both 1.2 and 1.4 versions for a time.
- libGL: stable
- libGLU: stable
- libopenal: stable if just because it is rarely updated
- libjpeg: similar to png case
- liblua: no idea. It seems little stable, good candidate to static link.
- libstdc++: stable, broke for last time with... gcc 3.4? That was 2004, and because of an exceptional event. It even uses symbol versioning to make things easier.
- libc/m/pthread: rock stable. It even uses symbol versioning to make things easier.
« Last Edit: July 08, 2010, 11:49:53 am by RedDwarf »

 

Offline chief1983

  • Still lacks a custom title
  • Moderator
  • 212
  • ⬇️⬆️⬅️⬅️🅰➡️⬇️
    • Minecraft
    • Skype
    • Steam
    • Twitter
    • Fate of the Galaxy
Re: wxLauncher 2.0 Request for Comments
libopenal is actually one of the worst recent offenders.  Some distros have switched to seemingly only offering openal-soft, which uses a differently names .so and does not provide a symlink even though the new one is backwards compatible.  Thus, binaries built on Debian don't seem to work on Ubuntu and vice versa.

Fate of the Galaxy - Now Hiring!  Apply within | Diaspora | SCP Home | Collada Importer for PCS2
Karajorma's 'How to report bugs' | Mantis
#freespace | #scp-swc | #diaspora | #SCP | #hard-light on EsperNet

"You may not sell or otherwise commercially exploit the source or things you created based on the source." -- Excerpt from FSO license, for reference

Nuclear1:  Jesus Christ zack you're a little too hamyurger for HLP right now...
iamzack:  i dont have hamynerge i just want ptatoc hips D:
redsniper:  Platonic hips?!
iamzack:  lays

 
Re: wxLauncher 2.0 Request for Comments
libopenal is actually one of the worst recent offenders.  Some distros have switched to seemingly only offering openal-soft, which uses a differently names .so and does not provide a symlink even though the new one is backwards compatible.  Thus, binaries built on Debian don't seem to work on Ubuntu and vice versa.
Uhm, openSUSE provided the symliink, though everybody did.
But this is another very exceptional case. When an specific library breaks binary compatibility just static link (or rpath) it for some time*. Isn't that the same that you did in Windows?

It seems to me easier than try to automatically compile it (ask the native package manager for -devel packages with a different name in every distro).


* Or, in this specific case, dlopen it. If .so.0 fails try .so.1.
« Last Edit: July 08, 2010, 12:04:17 pm by RedDwarf »