Hard Light Productions Forums

Modding, Mission Design, and Coding => FS2 Open Coding - The Source Code Project (SCP) => Cross-Platform Development => Topic started by: subliminal on November 01, 2014, 08:27:08 am

Title: OpenBSD port
Post by: subliminal on November 01, 2014, 08:27:08 am
I managed to compile fs2_open on OpenBSD 5.5 AMD64. It works nicely.
Only two problems were encountered while compiling and those are:
1) Missing OpenBSD entry in the configure script (it was treated as an unknown OS).
2) Missing lua.pc (in OpenBSD it's named "luaversionnumber.pc", "lua51.pc" in this case).

Step by step tutorial:
Install all the necessary dependencies (all of which are available in the package repository).
Download the source code tarball (e. g. 3.7.2 RC4).
Replace AUTOMAKE_VERSION, AUTOCONF_VERSION and lua**.pc with the ones you have installed on your system.
Code: [Select]
cd /home/user/Downloads
tar xzvf fs2_open_3_7_2_RC4_src.tgz
cd fs2_open_3_7_2_RC4
export AUTOMAKE_VERSION=1.9  AUTOCONF_VERSION=2.69
su
cp /usr/local/lib/pkgconfig/lua51.pc /usr/local/lib/pkgconfig/lua.pc
exit
vi configure.ac

Find the line saying:
Code: [Select]
case "$target" in and add this after the FreeBSD entry:

Code: [Select]
*openbsd*)
# openbsd
echo "OPENBSD $host_cpu-$host_os"
fs2_os_bsd="yes"
if test "$host_cpu" = "amd64"
then
D_CFLAGS="$D_CFLAGS -m64 -march=athlon64 -ansi"
fi
D_CFLAGS="$D_CFLAGS -DLUA_USE_LINUX"
LDFLAGS="$LDFLAGS -L/usr/local/lib"
CPPFLAGS="$CPPFLAGS -I/usr/local/include"
;;

Then just compile as usual:
Code: [Select]
./autogen.sh
su
make install
exit

Play the game!
Code: [Select]
cp /usr/local/bin/fs2_open_3.7.2_RC4 /home/user/freespace2
cd /home/user/freespace2
su
chmod 0666 /dev/drm*
exit
chmod +x fs2_open_3.7.2_RC4
./fs2_open_3.7.2_RC4
Add command line options to your liking:
http://www.hard-light.net/wiki/index.php/Command-Line_Reference

The chmod 0666 part is essential if you don't want your game started in software mode (it's slow!).
You can download the port from here:
https://github.com/subliminalshadow/openbsd/tree/master/ports/games
Title: Re: OpenBSD port
Post by: chief1983 on November 01, 2014, 09:10:54 am
Interesting, I was contemplating adding another nightly build arch to experiment with Vagrant.  Maybe I'll make it an OpenBSD one.
Title: Re: OpenBSD port
Post by: subliminal on November 01, 2014, 04:02:27 pm
wxLauncher compiles as well. Just follow the readme, be sure you have all the dependencies installed and add:
Code: [Select]
-DPYTHON_EXECUTABLE=/usr/local/bin/python2.7 (else it will complain of missing markdown).

Like this:
Code: [Select]
cmake -DPYTHON_EXECUTABLE=/usr/local/bin/python2.7 -DUSE_OPENAL=1 -DCMAKE_INSTALL_PREFIX=/usr/local ../Copy the resulting binary to your freespace2 directory and have fun.
Title: Re: OpenBSD port
Post by: subliminal on November 06, 2014, 12:08:47 pm
I made the ports for fs2_open and wxlauncher. You just need to download the archives and untar into /usr/ports/games. Then, cd to each directory and run
Code: [Select]
make install Finally, you can run both the fs2_open binary and wxlauncher from your freespace2 data directory.

https://github.com/subliminalshadow/openbsd/tree/master/ports/games

NOTE: These are the first OpenBSD ports I made, so, feel free to report any bugs and mistakes considering the Makefiles.
UPDATE: 3.7.2 RC4 suffers from segmentation fault when beams are fired
UPDATE2: -no_3d_sound fixes the crash for 3.7.2 RC4
UPDATE3: recent revisions work perfectly
Title: Re: OpenBSD port
Post by: chief1983 on November 06, 2014, 02:34:51 pm
I still need to fire up an OpenBSD VM to try this out for myself.
Title: Re: OpenBSD port
Post by: subliminal on January 03, 2015, 04:23:32 pm
Get the newest builds from:
https://github.com/subliminalshadow/openbsd/tree/master/ports/games
Title: Re: OpenBSD port
Post by: chief1983 on January 04, 2015, 11:35:25 am
I missed this update about beam crashes, I wonder if that's related to mantis 3130.  Would be crazy to find out it does hit other systems besides windows.  Did you test on nvidia by any chance?
Title: Re: OpenBSD port
Post by: subliminal on January 04, 2015, 02:10:14 pm
OpenBSD doesn't support nouveau (open source nvidia driver) yet. I tested 3.7.2 RC4 on Intel GMA X3100.
Title: Re: OpenBSD port
Post by: chief1983 on January 04, 2015, 02:24:29 pm
Interesting, might get a message later about that.  In the meantime, you might want to watch/comment on #3130 (http://scp.indiegames.us/mantis/view.php?id=3130) and help us identify if it's the same crash or something different.
Title: Re: OpenBSD port
Post by: AdmiralRalwood on January 04, 2015, 03:19:32 pm
I missed this update about beam crashes, I wonder if that's related to mantis 3130.  Would be crazy to find out it does hit other systems besides windows.  Did you test on nvidia by any chance?
It can't be the same problem as 3130; not because it's not crashing in the Windows nVidia drivers, but because 3130 doesn't go away if one goes back to 3.7.0. It could be a similar problem related to whatever is triggering 3130 in the first place, though.
Title: Re: OpenBSD port
Post by: chief1983 on January 04, 2015, 03:34:31 pm
True.  Still, since this bug at least seems revision dependent, doing a divide-and-conquer throughout different revisions to identify the change that is causing the OpenBSD crash would be very helpful.  It could shed light on other issues with the beam firing code to know what's causing this one, but that's likely a long shot.
Title: Re: OpenBSD port
Post by: subliminal on January 05, 2015, 12:54:06 pm
The last fully working revision on OpenBSD is r10797.
Title: Re: OpenBSD port
Post by: AdmiralRalwood on January 05, 2015, 01:12:14 pm
Almost certainly not the same problem, then; r10798 is sound-related (and my patch :().
Title: Re: OpenBSD port
Post by: chief1983 on January 05, 2015, 01:20:39 pm
Just out of curiosity, what version of OpenAL( Soft) is on your system, and can you see if -no_3d_sound prevents the crash with newer revisions?
Title: Re: OpenBSD port
Post by: AdmiralRalwood on January 05, 2015, 01:23:56 pm
Additionally, can you compile a debug build (of r10802 or later) and see if you trip any assertions when a beam fires?
Title: Re: OpenBSD port
Post by: subliminal on January 05, 2015, 04:04:07 pm
Just out of curiosity, what version of OpenAL( Soft) is on your system, and can you see if -no_3d_sound prevents the crash with newer revisions?
It indeed does prevent the crash. Now all revisions work! I have OpenAL Soft version 1.15.1 installed.
The original crash happened during Psamtik beam fire in Surrender, Belisarius! mission, so that's why I thought it was beam-related. It also happened in custom missions during beam fire, but now it all works.
On a side note, I'm also able to max it out with 2014 MediaVPs and get mostly lag-free gameplay on Intel GMA X3100 using 1280x800 as my resolution.
Title: Re: OpenBSD port
Post by: chief1983 on January 05, 2015, 04:30:15 pm
Just checking, because the commit that broke for you had most of the changes only affect when 3d sound is enabled.  We can probably figure out a fix for it so you don't have to force that off though.
Title: Re: OpenBSD port
Post by: AdmiralRalwood on January 05, 2015, 05:33:27 pm
You might also want to try a newer version of OpenAL Soft (pretty sure 1.16 is still the latest); I still want to know if a debug build triggers an Assertion with 3D sound on.
Title: Re: OpenBSD port
Post by: subliminal on January 06, 2015, 04:00:33 am
Building a debug build results in this:
Code: [Select]
windows_stub/stubs.cpp:47:23: error: malloc_np.h: No such file or directory
windows_stub/stubs.cpp: In function 'void* _vm_malloc(int, char*, int, int)':
windows_stub/stubs.cpp:586: error: 'malloc_usable_size' was not declared in this scope
windows_stub/stubs.cpp: In function 'void* _vm_realloc(void*, int, char*, int, int)':
windows_stub/stubs.cpp:608: error: 'malloc_usable_size' was not declared in this scope
windows_stub/stubs.cpp: In function 'void _vm_free(void*, char*, int)':
windows_stub/stubs.cpp:690: error: 'malloc_usable_size' was not declared in this scope
*** Error 1 in code (Makefile:5907 'stubs.o')
Title: Re: OpenBSD port
Post by: oldlaptop on January 07, 2015, 06:44:42 am
I was able to reproduce this issue on one of my systems; with retail data, it segfaults in Surrender, Belisarius! as soon as Psamtik fires a beam (after the charge effect is all done, but before the beam itself renders). Unfortunately, I was also able to reproduce the build failure with debug builds.  I did, however, build a release build after making this change to configure.ac (forcing release builds to be built with debug compiler flags):

Code: [Select]
diff --git a/configure.ac b/configure.ac
index 85de273..4a0f911 100644
--- a/configure.ac
+++ b/configure.ac
@@ -230,8 +230,8 @@ if test "$fs2_debug" = "yes" ; then
        fi
 else
        AC_DEFINE([NDEBUG])
-       D_CFLAGS="$D_CFLAGS -O2 -Wall -Wno-write-strings -funroll-loops"
-       D_LDFLAGS="$D_LDFLAGS "
+       D_CFLAGS="$D_CFLAGS -O0 -g -Wall -Wextra -Wno-unused-parameter -Wno-write-strings -Wshadow -funroll-loops"
+       D_LDFLAGS="$D_LDFLAGS -g"
 
        if test "$fs2_fred" = "yes" ; then
                AC_SUBST(FS2_BINARY, ["wxFRED2_${PACKAGE_VERSION}"])

and was able to get a good backtrace:
Code: [Select]

Program received signal SIGSEGV, Segmentation fault.
0x000015bd120ae201 in ds3d_update_buffer (channel=773, min=1500, max=3000, pos=0x7f7ffffe6dd4, vel=0x0) at sound/ds3d.cpp:46
46              ALuint source_id = Channels[channel].source_id;
(gdb) bt
#0  0x000015bd120ae201 in ds3d_update_buffer (channel=773, min=1500, max=3000, pos=0x7f7ffffe6dd4, vel=0x0) at sound/ds3d.cpp:46
#1  0x000015bd120b41aa in snd_update_3d_pos (soundnum=773, gs=0x15bfe7f15208, new_pos=0x7f7ffffe6dd4, radius=0, range_factor=1)
    at sound/sound.cpp:723
#2  0x000015bd120de8dc in beam_recalc_sounds (b=0x15bd13727140) at weapon/beam.cpp:1917
#3  0x000015bd120e1745 in beam_move_all_post () at weapon/beam.cpp:1099
#4  0x000015bd11f9245c in obj_move_all (frametime=0.0309906006) at object/object.cpp:1515
#5  0x000015bd11c5174d in game_simulation_frame () at freespace2/freespace.cpp:4094
#6  0x000015bd11c56349 in game_frame (paused=false) at freespace2/freespace.cpp:4487
#7  0x000015bd11c56882 in game_do_frame () at freespace2/freespace.cpp:4903
#8  0x000015bd11c56b69 in game_do_state (state=2) at freespace2/freespace.cpp:6586
#9  0x000015bd11d0d0ea in gameseq_process_events () at gamesequence/gamesequence.cpp:409
#10 0x000015bd11c5332b in game_main (cmdline=0x15bf1d3a20d0 "") at freespace2/freespace.cpp:7153
#11 0x000015bd11c53505 in main (argc=1, argv=0x7f7ffffe71c8) at freespace2/freespace.cpp:7288
(gdb)

This is with r11204 (https://github.com/scp-fs2open/fs2open.github.com/commit/7250cd13a63a22857af7a45bfb8c00038e68b3c2) on OpenBSD/amd64 5.6 (OpenAL Soft version 1.15.1).
Title: Re: OpenBSD port
Post by: m!m on January 07, 2015, 07:15:26 am
snd_update_3d_pos uses the sound number as the channel ID. I'm not very familiar with the sound code but other function suggest that ds_get_channel should be used to get the actual channel id that can be passed to ds3d_update_buffer.
Title: Re: OpenBSD port
Post by: AdmiralRalwood on January 07, 2015, 01:34:43 pm
snd_update_3d_pos uses the sound number as the channel ID. I'm not very familiar with the sound code but other function suggest that ds_get_channel should be used to get the actual channel id that can be passed to ds3d_update_buffer.
I'll try to take a look at this soon, but any idea why this is only a problem on OpenBSD?
Title: Re: OpenBSD port
Post by: chief1983 on January 07, 2015, 01:40:37 pm
I'm still not entirely convinced it is :P  it could just be that the sound numbers happen to coordinate to channel ids for many windows users, even if it's not the right one, and occasionally it could still be wrong, leading to a crash.  But I'm probably just way too optimistic this actually fixes several reports that we've tied to #3130.
Title: Re: OpenBSD port
Post by: AdmiralRalwood on January 07, 2015, 02:49:32 pm
I'm still not entirely convinced it is :P  it could just be that the sound numbers happen to coordinate to channel ids for many windows users, even if it's not the right one, and occasionally it could still be wrong, leading to a crash.  But I'm probably just way too optimistic this actually fixes several reports that we've tied to #3130.
That's the thing, though. If it's not the right channel, it shouldn't have sounded like the beam was panning at all, and instead some unrelated sound would have been panning. I'm just completely baffled that the code worked as it was at all. I just compiled, tested, and committed a fix to make ituse ds_get_channel() as m!m suggested. Could somebody try compiling latest SVN on OpenBSD and make sure 3D sound works with beams?
Title: Re: OpenBSD port
Post by: chief1983 on January 07, 2015, 03:15:52 pm
Hopefully by tomorrow so that if it does for sure fix OpenBSD, I can run a full suite of nightlies and see if it helps with anyone's beam crashes at all.  I really am setting myself up for disappointment on this though, but either way I'll be running RC5 shortly after.
Title: Re: OpenBSD port
Post by: AdmiralRalwood on January 07, 2015, 03:23:28 pm
Hopefully by tomorrow so that if it does for sure fix OpenBSD, I can run a full suite of nightlies and see if it helps with anyone's beam crashes at all.  I really am setting myself up for disappointment on this though, but either way I'll be running RC5 shortly after.
I think we can conclusively say that this has nothing to do with 3130 since 3130 happens with code that predates that revision (and even happens with sound turned off).
Title: Re: OpenBSD port
Post by: subliminal on January 07, 2015, 04:30:56 pm
I confirm it works. It would be nice if you would patch configure.ac, so it compiles out of the box on OpenBSD without applying this patch manually:
Code: [Select]
--- configure.ac.orig Fri Aug 15 23:22:48 2014
+++ configure.ac Thu Nov  6 17:59:28 2014
@@ -169,6 +169,18 @@ case "$target" in
  LDFLAGS="$LDFLAGS -L/usr/local/lib"
  CPPFLAGS="$CPPFLAGS -I/usr/local/include"
  ;;
+ *openbsd*)
+ # openbsd
+ echo "OPENBSD $host_cpu-$host_os"
+ fs2_os_bsd="yes"
+ if test "$host_cpu" = "amd64"
+ then
+ D_CFLAGS="$D_CFLAGS -m64 -march=athlon64 -ansi"
+ fi
+ D_CFLAGS="$D_CFLAGS -DLUA_USE_LINUX"
+ LDFLAGS="$LDFLAGS -L/usr/local/lib"
+ CPPFLAGS="$CPPFLAGS -I/usr/local/include"
+ ;;
  *)
  # Unknown
  echo "fs2_open does not support $host_cpu-$host_os"
Title: Re: OpenBSD port
Post by: chief1983 on January 11, 2015, 04:38:30 pm
Added OpenBSD to the FreeBSD line to keep it simple.  It could turn out those presets even work for NetBSD, so I'm kind of curious to try it now.  But a fresh svn checkout should work without changes now on OpenBSD, from 11214+.
Title: Re: OpenBSD port
Post by: subliminal on January 12, 2015, 03:33:52 pm
There's also DragonFlyBSD, and then you've covered all the BSDs.
Title: Re: OpenBSD port
Post by: chief1983 on January 13, 2015, 10:02:45 am
Apparently Net uses /usr/pkg by default, due to pkgsrc as its default manager.  I don't know about DragonFly yet but could look into it too :P

Have you tested the latest SVN yet?
Title: Re: OpenBSD port
Post by: subliminal on January 13, 2015, 12:29:43 pm
Latest SVN works great.

There's one more thing that can be fixed:
Running ./configure wants lua.pc, so this has to be done manually:
Code: [Select]
ln -s /usr/local/lib/pkgconfig/lua51.pc /usr/local/lib/pkgconfig/lua.pc
for fs2_open to compile. Could you make it find and use /usr/local/lib/pkgconfig/lua51.pc instead? (as that's the default on OpenBSD)
Code: [Select]
## Checking for lua.pc and if that fails lua5.1.pc ****and if that fails lua51.pc
**** [PKG_CHECK_MODULES([LUA], [lua51 >= 5.1.3 lua51 < 5.2], [])
Title: Re: OpenBSD port
Post by: chief1983 on January 13, 2015, 01:01:03 pm
If I'm right, you can just keep adding fallbacks to the check modules as a 4th argument, so I added another level and cleaned it up a bit so it's still somewhat legible.  Mind trying this out and see if it finds lua without any changes for you?

Code: [Select]
Index: configure.ac
===================================================================
--- configure.ac (revision 11214)
+++ configure.ac (working copy)
@@ -425,9 +425,22 @@
  [LUA],
  [lua >= 5.1.3 lua < 5.2],
  [], ## do nothing special
- [PKG_CHECK_MODULES([LUA], [lua5.1 >= 5.1.3 lua5.1 < 5.2], [], [
-        PKG_CHECK_MODULES([LUA], [lua-5.1 >= 5.1.3 lua-5.1 < 5.2], [])
-    ])])
+ [PKG_CHECK_MODULES(
+ [LUA],
+ [lua5.1 >= 5.1.3 lua5.1 < 5.2],
+ [], ## do nothing special
+ [PKG_CHECK_MODULES(
+ [LUA],
+ [lua-5.1 >= 5.1.3 lua-5.1 < 5.2],
+ [], ## do nothing special
+ [PKG_CHECK_MODULES(
+ [LUA],
+ [lua51 >= 5.1.3 lua51 < 5.2],
+ [], ## do nothing special
+ )]
+ )]
+ )]
+)
 D_CFLAGS=$D_CFLAGS" $LUA_CFLAGS"
 FS2_LIBS=$FS2_LIBS" $LUA_LIBS"
 
Title: Re: OpenBSD port
Post by: subliminal on January 13, 2015, 01:13:47 pm
That works.
There's also this one, when building a debug build:
Code: [Select]
windows_stub/stubs.cpp:47:23: error: malloc_np.h: No such file or directory
windows_stub/stubs.cpp: In function 'void* _vm_malloc(int, char*, int, int)':
windows_stub/stubs.cpp:586: error: 'malloc_usable_size' was not declared in this scope
windows_stub/stubs.cpp: In function 'void* _vm_realloc(void*, int, char*, int, int)':
windows_stub/stubs.cpp:608: error: 'malloc_usable_size' was not declared in this scope
windows_stub/stubs.cpp: In function 'void _vm_free(void*, char*, int)':
windows_stub/stubs.cpp:690: error: 'malloc_usable_size' was not declared in this scope
*** Error 1 in code (Makefile:3340 'windows_stub/stubs.o')

malloc_np.h and jemalloc are FreeBSD features.
Title: Re: OpenBSD port
Post by: chief1983 on January 14, 2015, 10:36:56 am
I wonder our Solaris' #define MALLOC_USABLE(pointer) (*((size_t*)(pointer)-1)) would work fine on all the BSDs then (Net doesn't have that function either) instead of #define MALLOC_USABLE(pointer) malloc_usable_size(pointer).  Would actually clean up that section of code a bit instead of making it more complicated with more OS-specific checks.  Or, I could try to use HAVE_MALLOC_NP_H somewhere instead.  I have some ideas, I'll experiment and hopefully come up with something even cleaner than what we have now.  Might need help testing a patch or two.  Or I'll try to find a Vagrant VM for OpenBSD I can fire up quickly.
Title: Re: OpenBSD port
Post by: niffiwan on January 14, 2015, 06:07:18 pm
Using the block one before the pointer returned by malloc to get the size of the allocated memory is implementation specific, it may not work at all especially given that OpenBSD seems to take a very secure approach to malloc and if I'm reading the net correctly one that's not really based on the other implementations. You could just define it as something that returns zero the entire time, I think the only issue will be that some memory usage tracking will be effectively disabled in debug (but do we really need that anyway?).
Title: Re: OpenBSD port
Post by: chief1983 on January 14, 2015, 06:16:12 pm
Yeah I was wondering if we might have to resort to disabling that behavior on all but FreeBSD for now.
Title: Re: OpenBSD port
Post by: subliminal on January 15, 2015, 04:10:18 pm
It would be nice if you could implement the option to choose a custom executable path in wxLauncher. Lets say the port installs fs2_open as /usr/local/bin/fs2_open_3.7.1. This would eliminate the need to copy the executable to local data directory (e. g. $HOME/freespace2), as it's possible to run the executable (from /usr/local/bin) while residing in local data directory and it does use data files correctly.
Title: Re: OpenBSD port
Post by: chief1983 on January 15, 2015, 04:17:17 pm
It's not the launcher's limitation, it's the engine's.  It doesn't support having the data live anywhere else other than ./ currently, but it is something we'd like to fix in the future.  It would also allow the launchers and mods more flexibility, such as requiring specific versions and having multiple versions of the binary available, and the launcher picking the best one, etc, without having to worry about manually selecting them.  That's just a dream at the moment though.
Title: Re: OpenBSD port
Post by: niffiwan on January 16, 2015, 03:09:12 am
I think it is a launcher limitation  :nervous:

As long as you launch FSO with your current working directory being the game data directory the binary can be anywhere. IIRC the old YAL allowed you to do that but wxlauncher doesn't yet.  Here's (https://github.com/wxLauncher/wxlauncher/issues/92) the "issue" for this enhancement.
Title: Re: OpenBSD port
Post by: m!m on January 16, 2015, 03:38:54 am
I am not sure about Linux but at least on Windows the engine will always change the current directory to the directory of the executable.
Title: Re: OpenBSD port
Post by: niffiwan on January 16, 2015, 05:25:09 am
I am not sure about Linux but at least on Windows the engine will always change the current directory to the directory of the executable.

Ah right, yes, I was thinking of the *nix OS's only...
Title: Re: OpenBSD port
Post by: chief1983 on January 16, 2015, 07:31:20 am
And that's the first I've heard about other OS handling that differently than Windows.  Interesting.