Hard Light Productions Forums
Modding, Mission Design, and Coding => FS2 Open Coding - The Source Code Project (SCP) => Test Builds => Topic started by: m!m on March 26, 2015, 07:58:16 am
-
Mjn.Mixael: H264 for cutscenes, music seek-to-time SEXPs, and model translation animation support
Hmm, H264 support, that doesn't sound like it's too difficult to implement.
After a few weeks of looking through documentation and cursing those who wrote it I arrived at an acceptable solution: http://www.mediafire.com/download/2dovsb3t83w5d2d/FSO-FFmpeg.7z
Source code: https://github.com/asarium/fs2open.github.com/tree/feature/ffmpeg
What does this do?
The changes in those builds add support for decoding a wide range of video and audio formats, notably H.264. It also improves the performance of cutscene playback by using a separate thread for video and audio decoding.
It uses the ffmpeg (https://www.ffmpeg.org/) libraries which do all the hard work. The currently version is capable of smoothly playing a 1080p Theora (Ogg) or H.264 encoded cutscene (I tried playing the amazing end cutscene of ST:R with RC5, it was horrible :shaking:).
Known issues
The build is based on the Antipodes branch which is a bit out of date so you should only use these build with retail data or mods that support older versions of FSO. The easiest way to test these build is to put the movie you want to see in data/movies and renaming it intro.ogg or intro.mp4 (depending on what format you are using).
-
Well well.... I know what I'm doing later.
-
We have to jump through quite a few hoops to legally use FFmpeg (https://www.ffmpeg.org/legal.html). For this example, was #1 followed?
FWIW, OpenH264 (http://www.openh264.org/) is BSD-licensed, hasn't been out too long so it might have not been around the last time people were thinking about video codecs.
-
Well, looks like I have to compile ffmpeg myself (seriously, **** GPL. Who thought such a piece of crap would be a good software license? :hopping:).
I am currently using precompiled binaries that were compiled with --enable-gpl so I have to compile ffmpeg myself without that flag so I can link those libraries dynamically to FSO (I hope it will still have the necessary features...).
-
We have to jump through quite a few hoops to legally use FFmpeg (https://www.ffmpeg.org/legal.html). For this example, was #1 followed?
FWIW, OpenH264 (http://www.openh264.org/) is BSD-licensed, hasn't been out too long so it might have not been around the last time people were thinking about video codecs.
I'm wary of trying/using yet another generally untested video thing...
-
There may be other options, it's just the first one I found that's not LGPL/GPL. Not saying we can't do LGPL, but in the FFmpeg case we definitely have to disable the GPL and non-free parts, and we also have to meet all the other LGPL license requirements to the best of our ability.
-
I compiled my own FFmpeg binaries without "--enable-gpl" and it looks like all the needed features are still there. Link in the first post has been updated.
For anyone curious, here are the resulting binaries: http://www.mediafire.com/download/irvgy2ry51r8z3l/FFmpeg-lgpl-shared.7z
-
It also improves the performance of cutscene playback by using a separate thread for video and audio decoding.
Multi-threading in mai freespace???
-
Multi-threading in mai freespace???
Don't worry, it's just one additional thread :P
-
ngld kindly checked and fixed the code for linux so this should also compile and run on linux.
-
First thing first: AWESOME job, this feature should make adding cutscenes to games/mods a lot easier! (Since devs can use more or less "standard" tools for creating video files instead having to deal with the vagaries of Theora).
IIRC, FFMpeg supports a bunch of codecs (https://www.ffmpeg.org/ffmpeg-codecs.html), are all these features available for developers or should they only stick with h264? If features are available, what codecs (at what specs, e.g. hi10 vs 8-bit encoding) are recommended for use?
-
The list of supported formats depends on how the libraries were compiled. I needed to disable a few features because we can't use the GPL version but here is a list of what the current libraries support:
- Formats (http://pastebin.com/YXKCiub6)
- Codecs (http://pastebin.com/5ZHkbdM3)
Once FFmpeg is merged into trunk/master we probably will create some kind of list of formats that must be supported by "official" builds.
The current build still checks all available extensions but I have already changed that to only check ogg, mp4 and webm. Those formats will always be supported.
-
As I said on IRC. I approve of this build.
It plays 1080p cutscenes (while scaling them for my current monitor) like they were nothing. I suspect I could stick to releasing just 1080p cutscenes and not multiple resolutions when this gets into the master branch.
-
I uploaded new builds that should be more in sync with the current master branch.
-
I'm looking forward to this. I remember that when I tried to play the 1080 version of the FS1 endgame cuscene when it first came out, I could only play it well in a dedicated media player. In game it froze and stuttered. This is gonna be awesome.
-
Just out of interest, has anyone tried using this code to see what performance is like when the movie is in a VP file? Cause originally Diaspora VP'd all our movies but eventually put them loose in the data/movies folder due to performance issues.
-
Hmm. I thought .vp was uncompressed? How would that cause a difference (besides for a split second to open the .vp)? :confused:
-
There is a bit of overhead from reading in a VP file as the read pointer has to be moved to the right position in the file. For optimal performance cutscenes should not be stored in a VP but I can't confirm or deny that VPs cause a performance hit (it would need to be very serious hit as the decoder thread is idle for quite a long time even when reading a 1080p video).
-
I'll test it out this evening
-
Playing from a VP seems just as fast. I tested a 1080p .ogg video file. (STR's Endgame)
-
Cool, thanks a lot for that mjn.
-
You bet. Now let's get this merged! :)
-
Not just yet, I found a few memory leaks :nervous:
-
I think I fixed most of the remaining issues.
I also satisfied a few points of the "License Compliance Checklist" (now there is a notice about FFmpeg usage in the credits and a text file describes how to compile the libraries). Please be aware that the checklist is designed for proprietary applications and not for open-source projects that can't use the GPL version so some of the points on the checklist are not relevant for us.
-
To the FOSS community I doubt they would consider us legally "open source" due to the volition license so i'd be careful with that.
-
What should we be careful about? We may not be a legally open-source project but the source is still freely available and we satisfy as many points of the checklist as possible.
-
Because of the copyleft (http://en.wikipedia.org/wiki/Copyleft) nature of the license to software we are considering using.
We need to demonstrate clearly that the combined work is not a derivative of FFmpeg, because the FS2Open project is not in a position to change the license on the Freeespace 2 code we use (only the rights holder can do that). And there are components to the existing Freespace 2 license which are incompatible with a free (as in libre) open source license.
-
As an alternative, you all could use VP8 (https://en.wikipedia.org/wiki/VP8) instead of H.264. The former has the benefit of being freer on a patent basis than H.264 with roughly the same (or slightly larger) file size for the same bitrate. In addition, it has its own self-contained decoder library in libvpx (https://github.com/webmproject/libvpx/), which is BSD licensed. Also, as it uses Vorbis as the audio stream, it wouldn't mean adding a new audio library. This also opens the door to use VP9 with the same library, which may offer even smaller file sizes still (though I wouldn't recommend it right now; it's still experimental).
-
Ugh, vp8 is far inferior to h264 in many ways. vp8 is basically the same quality as theora, so we won't really gain any quality.. just another version of the same quality. The goal here is to get something better.
-
The current version already checks the .webm extension which is VP8.
-
vp8 is basically the same quality as theora, so we won't really gain any quality.. just another version of the same quality. The goal here is to get something better.
Not true. Theora was based on VP3, which was 5 iterations before VP8. And it's improved a lot since then. In your remastered cutscenes, Theora has blocks all over the place. VP8 (which you can see from YouTube, or I can post the file someplace) doesn't. And it's an eighth the size, even with more bandwidth allocated to the audio track. So even if it wasn't better than Theora -- which it is -- it is definitely smaller. When I looked at the H.264 YouTube rips, they were twice the size of the VP8 ones for no discernable difference in quality.
I admit, I don't whether you uploaded the original source files for processing or uploaded the Theora videos. So it may not have been an objective test. But if the idea is to look for something better, it behooves the project to look at all of the options (including even H.265 and VP9). Regardless, I will agree that Theora sucks and the sooner we retire it, the better, and it seems like the library is a step in that direction. So yay for that.
-
I realize this is a moot point since m!m's builds can apparently do both. But I have worked in the video production industry for about 7-8 years. Trust me when I tell you that h.264 is objectively better than VP8 in all ways except file size. Especially for our purposes. VP8 is great for web video, streaming things, and stuff you want to get the data of fast at the cost of some (minor) quality. (See: YouTube) However, we don't care about streaming... and the size of the cutscenes is not going to make or break the download size of mods big enough to have cutscenes. For sheer video quality, h.264 is the way to go.
-
Just to keep everyone updated, I fixed a few issues of the cutscene player on linux and also added a way to use prebuilt LGPL binaries instead of relying on the system libraries which are GPL most of the times.
As far as I know GPL allows to use (and even modify) the libraries if they are not distributed so building a binary for private use may actually not be against the rules of the GPL. That would mean that only the release and nightly builds would need to include the LGPL libraries which would simplify development a lot. If someone knows more about this I would like to know if what I wrote is actually right as I don't know much about all that licencing stuff.
-
As far as I know GPL allows to use (and even modify) the libraries if they are not distributed so building a binary for private use may actually not be against the rules of the GPL.
Yep, section 2 of the GPL states this, that you can use and modify, and allow others to modify, the work under you sole discretion, provided that none of the involved party intends to distribute the "modified work" without appropriate licensing under the GPL. If you have the GPL'd library on your machine already, or otherwise can get it separately, there's nothing legal stating that you can't link the GPL'd library to the program.
The SCP can't distribute such versions of FSO, since that's expressively forbidden by the GPL, but conveying a version of FSO that's linked to an unmodified LGPL'd library is permitted, and of course the source code of FSO is freely available so folks savvy with build systems can use whatever GPL flavor they desire--so long as they don't distribute them.
The safe bet here for us would be to stick with LGPL libraries and leave the custom GPL linking to users that wish to do so, and probably warn them somewhere in bold friendly letters that they cannot distribute/convey copies of their GPL-linked version.
-
So... I'm gonna poke this again. I don't understand all the license stuff, but I'm still fed up with Theora. :)
-
As far as I can tell this project is ready to be merged but it still needs antipodes and CMake so it might take a while.
Mac support is also an issue because my CMake configuration currently doesn't work there and I don't have a mac so I can test it :sigh:
-
Someone needs to hackintosh (http://www.tonymacx86.com/). Maybe your system isn't compatible, but maybe it is! Then you could possibly test these things (actually, can't you already boot up mac in VirtualBox (http://lifehacker.com/5583650/run-mac-os-x-in-virtualbox-on-windows)?
Although all these things take Time (https://www.youtube.com/watch?v=_P8eFRCfno8). :warp:
-
There's enough reasonable devs with Macs these days, that we can probably muddle our way through the necessary CMake changes.
-
chief1983 and I had some success on his machine but without direct access to a Mac fixing bugs is very difficult and time consuming.
-
What about subtitles much like players? What about the .srt file?
-
No plans for them at this time.
-
These changes have been merged into the master branch and are available in the nightlies starting from this build (http://www.hard-light.net/forums/index.php?topic=92614.0).
-
YeeaAAAHH
-
Are there any supported formats it can play? Why can't be in the Wiki?
-
The list of supported formats depends on how the libraries were compiled. I needed to disable a few features because we can't use the GPL version but here is a list of what the current libraries support:
- Formats (http://pastebin.com/YXKCiub6)
- Codecs (http://pastebin.com/5ZHkbdM3)
Once FFmpeg is merged into trunk/master we probably will create some kind of list of formats that must be supported by "official" builds.
The current build still checks all available extensions but I have already changed that to only check ogg, mp4 and webm. Those formats will always be supported.