[Nightmare]

The first time an animation plays, there will always be a little big of lag, but after it loops through it tends to get better.

Does the game not pre-load files referenced by scripting files? While experimenting with some custom death flash I've noticed that there's some lag too. Or does that effect only APNG/PNG files, and is there a way to prevent that?

[Axem]

Welllll...

It's complicated. I'll try and run down the basics of how this all works.

When a VN Scene is loaded, the script does go through the whole file and looks for any actor animations that could be called. It will load them, then unload them right after so we're sort of caching the file. Sort of. A cold file load on an APNG for JAD is pretty noticable, it would cause a very noticeable spike of lag. Maybe 0.25 - 0.5 s? I never timed it, but its pretty lame. But by having the files loaded and unloaded, enough stuff is cached that the file can load again almost right away. So this is always done at VN start so you think its just part of the script loading.

Now, we've got the file loaded, but that doesn't necessarily mean you're out of the woods. All the APNGs are streamed in. This is so I don't have to load... well there's like 2GB of just APNGs in JAD. Obviously not everything would be used in one scene, but I think to draw it on the screen it needs to be decompressed to essentially a raw bitmap (think .bmp plus alpha!) and all of this needs to get thrown from your hard disk and into RAM. So it's going to steal all that RAM and take forever. This is all on top of already loaded assets inside FreeSpace as well!

Anyway, back to streaming. The APNGs are streamed in (eternal thanks to niffiwan for that), and how that works is each frame gets loaded as needed, then thrown away once I need another animation. So the first run through takes a little bit of time to actually get loaded up. So if an APNG has 90 frames, we're basically dividing up the load into 90 smaller bits in semi-real time, which is more seamless than loading all 90 together while waiting for a full second or something. And then the next time it loops, the animation is already in memory and is painlessly simple to play over again. I did my best to try and stagger animations so that initial load would just be done on one animation at a time. But sometimes I can't because the scene needs 4 people to be there at the start...

Things could be improved if the dark art of multi-threading was attainable, or perhaps some sort of movie codec with a reliable transparency feature. But I think for a silly mod like JAD, the system is as good as it needs to be. Maybe I can revisit when I go commercial.

Oh and by the way, if you are wondering "gee, if it was like that before, how did you improve performance?" Wings of Dawn is using the VN script too, and Spoon wanted a way to have characters have a frame around them, so I added that feature to the VN script. Problem is, if there is no base, the script still tries to load one. And if it doesn't see one, it will keep trying to find and load it every. single. frame. That's a bit of a slowdown. So added some instructions to tell the script "don't bother looking if there is no base".

So I made things worse by accident, and I'm hoping they are better now.

[Volta]

The VN scenes do work much smoother as long i don't rush trough the dialog and the interrogation works now again too, thanks!

But the interrogation scene doesn't cache anything for some reason 
Do the APNG script cache stuff into the OpenGL Nvidia/Amd GPU cache folder? i wonder if my GPU cache is corrupt but i did delete the folder so there souldn't be a problem (it regenerates on it's own if you play a OpenGL based game)

Is it possible to just use a Normal PNG file without the animations? Or is it also possible do use a lower resolution APNG version of the mod? 

[m!m]

When a VN Scene is loaded, the script does go through the whole file and looks for any actor animations that could be called. It will load them, then unload them right after so we're sort of caching the file. Sort of. A cold file load on an APNG for JAD is pretty noticable, it would cause a very noticeable spike of lag. Maybe 0.25 - 0.5 s? I never timed it, but its pretty lame. But by having the files loaded and unloaded, enough stuff is cached that the file can load again almost right away. So this is always done at VN start so you think its just part of the script loading.

As far as I can tell from looking at the source, unloading a texture will remove all related data from the bmpman system of FSO so the next time it still needs to load the complete file. However, your operating system caches data which was loaded from your disk in RAM so the next time the file is opened it doesn't need to be read from the disk which is the probably cause for the improved performance you are seeing.
I would like to add support for movie files with an alpha channel but so far I could not find a way to support that with FFmpeg

[Axem]

Yeah, it's not a real cache. That's why I said it sort of caches it, its probably the OS that still has the file in recent memory. Or something. I'm not sure how it works, but it do.

The interrogation scene is probably slowest because there's always 4 characters active, even if 2 are visible. Maybe its too much for your amount of memory or something? The best I can suggest right now is to grin and bear it. That sequence is probably the worst out of the campaign.

I would like to add support for movie files with an alpha channel but so far I could not find a way to support that with FFmpeg

Yeah... In my research about that, I saw claims that VP8 could do that, but I could never get it to work. There was even a proof of concept file they had and it worked, but I could never make my own working files.

[Nightmare]

Wow, I didn't realize it was that difficult. The flash I played around with takes about a second or so to load, but the VN parts of JAD never gave me any problems.

[niffiwan]

Someday I really should get back to this...

Code: [Select]

$ git diff feature/multithread_apng_processing^ feature/multithread_apng_processing --stat
 code/bmpman/bmpman.cpp     |   2 +-
 code/pngutils/pngutils.cpp | 162 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------------------------------------------
 code/pngutils/pngutils.h   |  14 +++++++++++++-
 3 files changed, 128 insertions(+), 50 deletions(-)

(IIRC it works, just with bugs. Like crashes if you interrupt the apng loading  )

[m!m]

It looks like FFmpeg already has support for APNG so we don't really need new code for getting multi-threaded decoding since everything is already in place. Since APNG decodes to RGBA instead of YUV a few changes to the movie renderer are needed but nothing extensive. Maybe this would also be the right time to add scripting support for playing movie files since I wanted to do something like that for a long time already.

[Nightmare]

So the reason for that mini-lag is that PNG/APNG loading procedure and not because files only referred to in scripting files (like VN anime) aren't preloaded at the mission start?

[Axem]

I split the thread since this isn't really JAD troubleshooting anymore...

Anyway, that's basically the case. I try to pay that lag cost up front, but I guess it doesn't always work as well as it could.

[m!m]

Sorry for derailing the thread...

I'll work a bit on the FFmpeg APNG support and get that working a bit more efficiently. Maybe the multi-threading of the movie player will help APNG performance a bit if used properly.

[Axem]

I wish all thread derails could be the cause for some performance improvements.

[m!m]

I finished adding the scripting API for movie files (https://github.com/scp-fs2open/fs2open.github.com/pull/1794) and it looks very promising. This also supports displaying APNG files with full transparency support so it's basically a drop-in solution for the streaming animation API. However, since the movie player code preloads a few frames and also uses a separate thread it uses more resources than the streaming animation code so it might not be ideal in all circumstances. Also, it doesn't support seeking in the movie file yet so looping animations are not supported currently.

I used this script for testing the playback so you can use that to see how displaying a cutscene is done from Lua. player:isPlaybackReady() is basically the part where the background thread is involved since that checks if enough frames have been preloaded to start playback.

Code: [Select]


#Conditional Hooks

$Application: FS2_Open
$State: GS_STATE_MAIN_MENU
$On State Start:
[
player = gr.openMovie("test")
playbackActive = true
]

$State: GS_STATE_MAIN_MENU
$On Frame:
[
if playbackActive and player:isPlaybackReady() then
if lastPlayerTime == nil then
lastPlayerTime = time.getCurrentTime()
end
local currentTime = time.getCurrentTime()

playbackActive = player:update(currentTime - lastPlayerTime)

player:drawMovie(100, 100)

lastPlayerTime = currentTime
end
]

$State: GS_STATE_MAIN_MENU
$On State End:
[
player:stop()
player = nil
]

#End