Hard Light Productions Forums
Modding, Mission Design, and Coding => FS2 Open Coding - The Source Code Project (SCP) => Topic started by: chief1983 on April 16, 2014, 02:47:44 pm
-
Currently there's still quite a bit of variation around the build naming of the various projects, between the different platforms. I'm currently working on adding release build support based on my nightly build code, and this is proving to be a bit of a headache to work with. Here's the current situation:
--Current--
Windows (VS 2008):
Nightly: fs2_open_3_7_1[_SSE/_NO-SSE][-DEBUG]-20140414_r10560.exe
Archive: fso-WIN-Standard-20140414_r10560.7z
Release: fs2_open_3_7_2[_RC1][_SSE/NO-SSE][-DEBUG].exe
Archive: fs2_open_3_7_2[_RC1][_SSE/NO-SSE].zip
OSX:
Nightly: FS2_Open[ (debug)]-20140414_r10560.app
Archive: fso-OSX-Standard-20140414_r10560.tgz
Release: FS2_Open 3.7.2[ RC1][ (debug)].app
Archive: FS2_Open-3.7.2[_RC1].dmg
Linux/BSD:
Nightly: fs2_open_3.7.1[_DEBUG]-20140414_r10560
Archive: fso-LINUX-Standard-20140414_r10560.tgz
Release: fs2_open_3.7.2[_RC1][_DEBUG]
Archive: fs2_open_3.7.2[_RC1].tar.bz2
Now, I'd like to propose something a little different. I can move the injected build dates and revisions on the nightly builds closer to the other version info, and the debug info to the end where it should be, or, as many have requested, I can not inject revision info at all into the nightly filenames, allowing nightly executables to just overwrite previous nightlies. Also changing some hyphens to underscores, for a bit more consistency, and fso to fs2 in the nightly archive names. Also adds the version to the OS X project, which I think is easy enough to do without actually hardcoding it in this case even. All this will really help eliminate a lot of complexity in the build automation code.
--Proposed--
Windows (VS 2008):
Nightly: fs2_open_3_7_1_20140414_r10560[_SSE/_NO-SSE][-DEBUG].exe
Archive: fs2_Standard_20140414_r10560.7z
Release: fs2_open_3_7_2[_RC1][-DEBUG].exe
Archive: fs2_open_3_7_2[_RC1][_SSE/NO-SSE].zip
OSX:
Nightly: FS2_Open 3.7.1_20140414_r10560[ (debug)].app
Archive: fs2_Standard_20140414_r10560.dmg
Release: FS2_Open 3.7.2[ RC1][ (debug)].app
Archive: FS2_Open-3.7.2[_RC1].dmg
Linux/BSD:
Nightly: fs2_open_3.7.1_20140414_r10560[_DEBUG]
Archive: fs2_Standard_20140414_r10560.tar.bz2
Release: fs2_open_3.7.2[_RC1][_DEBUG]
Archive: fs2_open_3.7.2[_RC1].tar.bz2
So, what do people think about getting rid of the build revision from the nightly build filenames? Or anything else here in general? Ultimately whatever I go with will probably be whatever makes my automation code the simplest, but I wanted to get opinions on anything I should try to do or not do aside from that.
-
So, what do people think about getting rid of the build revision from the nightly build filenames?
plzno
I've got enough builds named just fs2_open_3_7_1_SSE2.exe laying around...
-
Really need to have revision numbers in build names.
-
I guess I could have clarified some of the arguments against it. For one, FRED seems to associate its memory of last mission directory, etc from the build filename. Every new build name, you have to re-locate your mission folder. Also, if you set up per-game presets in your video card driver, for instance, those are usually associated with an executable name. And as long as you keep the archives around, you can always just extract the one you know you wanted to be using at the time. I may even be able to get the binaries built with this system to include the build revision internally, with some of this work I did on regex replacement.
-
I've certainly been annoyed at stuff depending on the executable name, but even so I've found it to be the lesser evil when working with nightlies.
However, with Git, the hashes won't really tell you much and sorting of the builds by filename needs to be determined by the date instead.
-
The date will, as with SVN, still be in the name before the short Git revision.
-
IIRC the standard advice on our support boards is to use nightlies, in which case build numbers are useful for debugging purposes.
-
And I can probably inject that into version.h now so it can be in the debug log, which should hopefully cover that. Unless "the debug log isn't being written" :p
-
I'd suggest getting the wxLauncher guys opinion on this as well since they do some parsing of the filename, therefore any changes made here may require some changes in wxLauncher.
And I support having the date + short git rev in the filename, plus the short git rev in the debug log ;)
-
I've certainly been annoyed at stuff depending on the executable name, but even so I've found it to be the lesser evil when working with nightlies.
This is my opinion too. Yeah its a pain to have Windows go "hey, open the firewall for this new build?" but when I'm dealing with a lot of nightlies to find a bug, having the builds with their numbers in the filename saves me from insanity.
-
Honestly, it doesn't really matter what you do to the filenames it will probably break our parsing in some way, it is just that brittle. Having said that, what Chief proposed doesn't look like it will break anything, but it hard to say without testing it out.
Obviously, I have put a lot of thought into this over the years. I know I have discussed the parsing of filenames in wxLauncher with Chief several times in the past as well and that it is a behaviour that should be depreciated.
At this point I think my favourite idea is for the binary to have a parseable representation of all of the metadata that we want to record about the binary embedded in the binary it self. This representation should probably be a simple key value pair delimited by sentinels. This would allow the data collection system will work no matter what which OS the launcher is running on, this avoids the issues of not being able to id a Linux binary on Windows, for example. More importantly though it allows for far more detail than what could ever be embedded into the file name. This block of data could be outputted at the start of every log (like how the wxLauncher log starts to id which build it comes from (both for the support ninjas and so that wxLauncher or any other tool can match up the logs with the build that produced it).
14059045326:INFO :wxLauncher Version 0.9.4
14059045326:INFO :Build "67777df3c3e6 default tip" committed on (Tue Jun 04 22:03:25 2013 -0700)
14059045326:INFO :02/27/14 21:53:26
I was thinking something like this.
==FreeSpace 2 Open==
Version: 3.7.0.9755
Revision: 12345
Build date: 2014-04-14T11:25:25 EDT
Class: RC1
Debug: Yes
SSE: Yes
====
==FreeSpace 2 Open==
Version: 3.7.1.10560
Revision: 10560
Build date: 2014-04-14T11:25:25 EDT
Class: Official Nightly
Debug: Yes
SSE: Yes
====
Passed cmdline options:
-missile_lighting
-3dshockwave
-3dwarp
-warp_flash
-snd_preload
-window
The hope would be there is enough information that a launcher could figure out the correct FRED, and the correct debug build (for b oth) to go along with the release fs2_open that the user is running.
-
Embedding all the metadata into the exec sounds quite interesting, but how does it work? :) I realise that a literal char array is stored in part of the executable, and that the tools "strings" and "grep" allow you to retrieve this info from the exec, but I suspect that you have something more sophisticated in mind.
(should I just have a look at the wxlauncher source code? :D)
-
I don't think the launcher will look at the executable data but instead FSO could be invoked with a special commandline option that would print the information to stdout which the launcher could read and process afterwards.
-
The exe already does something like this. The launcher uses a special commandline flag (whose name escapes me at the moment), which causes the exe to emit a binary file containing all the commandline flags supported by the executable.
-
I know (AFAIK the name is -get_flags) but that file does not contain any version information but that could be added very easily. The main problem with the file is that you can't start all FSO executables you have at the same time as they will overwrite the flag file another executable has generated which would not be a problem if the information is printed to stdout.
Also the binary file format is pretty limited, it would be nice if FSO could print the information in either a simple text format or just use JSON which can be parsed very easily.
-
No, this was going to be just reading the executable file. Remember, a binary is still just a file.
I see the implementation as being nothing more clever than the C equivalent of strings and grep. We might be able to optimize it later but the first implemention is would be a brute force linear search just link strings.
I had considered -get_flags to be separate from this proposal, but now that it has been mentioned, the scope is actually pretty similar. Get the launcher the information about the features that the binary supports. I think the primary problem with this would be how to represent it in the binary so that it could be read out, though I suppose the flag file is separate from the parsing of the flags already.
There are other problems with -get_flags beyond that it is always written to the same file. wxLauncher works around the same file issue on some platforms already. Also my previous posts on the topic have some solutions to that problem as well. The primary problem with -get_flags is that when it doesn't work, it fails in very hard to troubleshoot ways, such as pop ups, error dialogs, etc.
Also noticed that I forgot the most important part of the metadata in my previous post. The bitness and platform.
-
Why not just use stdout for returning the data ?
-
For my original metadata suggestion, that requires the binary actually execute on the platform wxLauncher is running on but would otherwise work.