Author Topic: Let's talk about bmpman  (Read 14016 times)

0 Members and 1 Guest are viewing this topic.

Offline niffiwan

  • 211
  • Eluder Class
Re: Let's talk about bmpman
So, after talking with Axem on IRC today it seems that a *single* JAD mission is exceeding the current BMPMAN limits.  He's got 2800+ slots filled with talking head frames (:nervous:), so being more aggressive with unloading bitmaps isn't going to help. I'd also wager that the increased slot fragmentation of unloading/loading moderate-to-lengthy animations means that across multiple missions the BMPMAN limit is going to be hit well below the current 4750 slots.

Anyway, so maybe the focus should be on implementing texture/animation atlasing (which I 1st heard of here in this thread, from Swifty!) independently of messing with BMPMAN? i.e. as I understand it so far, we could have a single mega-texture stored in a single BMPMAN slot rather than animation frames strewn across heaps of slots.

(Am I just being Captain Obvious here?)

(also, I thought I should have a look at this when I realised that apng support isn't going to fix anyone's BMPMAN slot issues, at least, not if I follow the current animations framework within FSO)
Creating a fs2_open.log | Red Alert Bug = Hex Edit | MediaVPs 2014: Bigger HUD gauges | 32bit libs for 64bit Ubuntu
----
Debian Packages (testing/unstable): Freespace2 | wxLauncher
----
m|m: I think I'm suffering from Stockholm syndrome. Bmpman is starting to make sense and it's actually written reasonably well...

 

Offline m!m

  • 211
Re: Let's talk about bmpman
We would still need to keep track of where a single frame is in the atlas which would consume a bmpman slot so this probably won't fix our problems...

 

Offline z64555

  • 210
  • Self-proclaimed controls expert
    • Minecraft
    • Steam
Re: Let's talk about bmpman
Nah, I'm thinking we should update how animations are stored in RAM, instead needing a slot for every frame in the animation, we'd have a single slot for the start of the animation and then reference the other frames through that one.
Secure the Source, Contain the Code, Protect the Project
chief1983

------------
funtapaz: Hunchon University biologists prove mankind is evolving to new, higher form of life, known as Homopithecus Juche.
z64555: s/J/Do
BotenAlfred: <funtapaz> Hunchon University biologists prove mankind is evolving to new, higher form of life, known as Homopithecus Douche.

 

Offline m!m

  • 211
Re: Let's talk about bmpman
It could help but it's not a permanent solution.
Also, maybe I'm missing something but shouldn't animations never be stored in RAM but only in the video RAM? I don't see much point in keeping them in the normal RAM where we can't really do much with them.

 

Offline niffiwan

  • 211
  • Eluder Class
Re: Let's talk about bmpman
Wouldn't you want them in RAM to avoid reading from disk when you need them? I also figure that loading them only into video memory for the duration of a mission is likely to exhaust it (presuming that model textures will tend to be much larger than most animations?)

Also, why would we use bmpman to store which location the frame is within an atlas? Couldn't that info be stored elsewhere? i.e. generic anim's already store the current frame in the generic_anim struct.
Creating a fs2_open.log | Red Alert Bug = Hex Edit | MediaVPs 2014: Bigger HUD gauges | 32bit libs for 64bit Ubuntu
----
Debian Packages (testing/unstable): Freespace2 | wxLauncher
----
m|m: I think I'm suffering from Stockholm syndrome. Bmpman is starting to make sense and it's actually written reasonably well...

 

Offline AdmiralRalwood

  • 211
  • The Cthulhu programmer himself!
    • Skype
    • Steam
    • Twitter
Re: Let's talk about bmpman
The only problem is that if you change animations to not consume multiple BMPMAN slots, you break every script that uses animation frames. On the other hand, the benefits would probably outweigh the cost there.
Ph'nglui mglw'nafh Codethulhu GitHub wgah'nagl fhtagn.

schrödinbug (noun) - a bug that manifests itself in running software after a programmer notices that the code should never have worked in the first place.

When you gaze long into BMPMAN, BMPMAN also gazes into you.

"I am one of the best FREDders on Earth" -General Battuta

<Aesaar> literary criticism is vladimir putin

<MageKing17> "There's probably a reason the code is the way it is" is a very dangerous line of thought. :P
<MageKing17> Because the "reason" often turns out to be "nobody noticed it was wrong".
(the very next day)
<MageKing17> this ****ing code did it to me again
<MageKing17> "That doesn't really make sense to me, but I'll assume it was being done for a reason."
<MageKing17> **** ME
<MageKing17> THE REASON IS PEOPLE ARE STUPID
<MageKing17> ESPECIALLY ME

<MageKing17> God damn, I do not understand how this is breaking.
<MageKing17> Everything points to "this should work fine", and yet it's clearly not working.
<MjnMixael> 2 hours later... "God damn, how did this ever work at all?!"
(...)
<MageKing17> so
<MageKing17> more than two hours
<MageKing17> but once again we have reached the inevitable conclusion
<MageKing17> How did this code ever work in the first place!?

<@The_E> Welcome to OpenGL, where standards compliance is optional, and error reporting inconsistent

<MageKing17> It was all working perfectly until I actually tried it on an actual mission.

<IronWorks> I am useful for FSO stuff again. This is a red-letter day!
* z64555 erases "Thursday" and rewrites it in red ink

<MageKing17> TIL the entire homing code is held up by shoestrings and duct tape, basically.

 

Offline mjn.mixael

  • Cutscene Master
  • 212
  • Chopped liver
    • Steam
    • Twitter
Re: Let's talk about bmpman
Are there any major scripts that would be affected? I'm wondering about the explosion effects in the MediaVPs, for example. As long as there is an alternate solution for those scripts and we have an idea of which ones to fix up... we could deal with the fallout pretty quickly.
Cutscene Upgrade Project - Mainhall Remakes - Between the Ashes
Youtube Channel - P3D Model Box
Between the Ashes is looking for committed testers, PM me for details.
Freespace Upgrade Project See what's happening.

 

Offline Axem

  • 211
Re: Let's talk about bmpman
I think the explosion effects in that script are just particles, so I don't think they would be affected (not sure, I'd have to look at the script when I get home).

But with regards to animations and scripts, is that really a problem? The smart way to do that right now is with gr.loadTexture(), with an argument to load as an animation and then you draw the frames and change the frame index every frame. The script never actually directly touch the raw files, even with effs (and as such, the script writer needs to make sure he/she unloads the texture!). Or would it break even then?

Unless you mean scripts that intentionally use a single frame of an animation like an image from a larger eff? Or I might be misunderstanding something...

 

Offline AdmiralRalwood

  • 211
  • The Cthulhu programmer himself!
    • Skype
    • Steam
    • Twitter
Re: Let's talk about bmpman
But with regards to animations and scripts, is that really a problem? The smart way to do that right now is with gr.loadTexture(), with an argument to load as an animation and then you draw the frames and change the frame index every frame. The script never actually directly touch the raw files, even with effs (and as such, the script writer needs to make sure he/she unloads the texture!). Or would it break even then?

Unless you mean scripts that intentionally use a single frame of an animation like an image from a larger eff? Or I might be misunderstanding something...
I... hmm. For some reason I thought scripts were using BMPMAN slots directly, but a closer look at the only example I had on hand shows it using the animation returned by loadTexture(), so I guess they'd actually be fine.
Ph'nglui mglw'nafh Codethulhu GitHub wgah'nagl fhtagn.

schrödinbug (noun) - a bug that manifests itself in running software after a programmer notices that the code should never have worked in the first place.

When you gaze long into BMPMAN, BMPMAN also gazes into you.

"I am one of the best FREDders on Earth" -General Battuta

<Aesaar> literary criticism is vladimir putin

<MageKing17> "There's probably a reason the code is the way it is" is a very dangerous line of thought. :P
<MageKing17> Because the "reason" often turns out to be "nobody noticed it was wrong".
(the very next day)
<MageKing17> this ****ing code did it to me again
<MageKing17> "That doesn't really make sense to me, but I'll assume it was being done for a reason."
<MageKing17> **** ME
<MageKing17> THE REASON IS PEOPLE ARE STUPID
<MageKing17> ESPECIALLY ME

<MageKing17> God damn, I do not understand how this is breaking.
<MageKing17> Everything points to "this should work fine", and yet it's clearly not working.
<MjnMixael> 2 hours later... "God damn, how did this ever work at all?!"
(...)
<MageKing17> so
<MageKing17> more than two hours
<MageKing17> but once again we have reached the inevitable conclusion
<MageKing17> How did this code ever work in the first place!?

<@The_E> Welcome to OpenGL, where standards compliance is optional, and error reporting inconsistent

<MageKing17> It was all working perfectly until I actually tried it on an actual mission.

<IronWorks> I am useful for FSO stuff again. This is a red-letter day!
* z64555 erases "Thursday" and rewrites it in red ink

<MageKing17> TIL the entire homing code is held up by shoestrings and duct tape, basically.

 

Offline niffiwan

  • 211
  • Eluder Class
Re: Let's talk about bmpman
After more investigation it seems that talking head EFFs only use two BMPMAN slots each, at least when they're being rendered. One slot is the 1st frame, one is the current frame. On the other hand, it looks like FSO is reserving enough BMPMAN slots for every frame; whether this is actually a problem or not I'm not 100% sure, it needs some more investigation.

As part of this investigation I've found a way to get a list of all BMPMAN slots used per mission; just add this to your debug_filter.cfg and it's dumped to the log just before the mission starts.

Code: [Select]
+BMP DEBUG

(and you probably want the usual suspects as well)
Code: [Select]
+General
+Warning
+Error
+BMP DEBUG

Exactly how much of this output is "lying" is yet to be seen, "best case" maybe all the EFFs (or at least the talking heads) only need two frames... wouldn't that be an easy win! :)
Creating a fs2_open.log | Red Alert Bug = Hex Edit | MediaVPs 2014: Bigger HUD gauges | 32bit libs for 64bit Ubuntu
----
Debian Packages (testing/unstable): Freespace2 | wxLauncher
----
m|m: I think I'm suffering from Stockholm syndrome. Bmpman is starting to make sense and it's actually written reasonably well...

 

Offline Axem

  • 211
Re: Let's talk about bmpman
Bmpman. ¯\_(ツ)_/¯

Am I rite coders?

(Seriously though, if the best case is true, I will be very very happy)

 

Offline Spoon

  • 212
  • ヾ(´︶`♡)ノ
Re: Let's talk about bmpman
Bmpman is so weak, it needs to lift all these bitmaps! GET FIT.

JAD and WoD can't function as long as you remain a scrawny little excuse for a line of code! GO TO THE GYM, BMPMAN.
Urutorahappī!!

[02:42] <@Axem> spoon somethings wrong
[02:42] <@Axem> critically wrong
[02:42] <@Axem> im happy with these missions now
[02:44] <@Axem> well
[02:44] <@Axem> with 2 of them

 

Offline niffiwan

  • 211
  • Eluder Class
Re: Let's talk about bmpman
I think I've got headanis (only) using only 3 bmpman slots each; not quite finished but close.  Here's my TODO list:

Code: [Select]
[X] only look for 3 frames to hold eff-anim instead of all frames (NEEDS TESTING)
[ ] reset 3rd frame back to _0001 after finishing playing anim (so we don't keeping using up slots)
[ ] update unload/release functions to handle the 3-frame anis
[x] see if we can release headani's between missions (don't just unload them) (looks like they are already released)
[x] correctly deal with EFFs that don't have all the frames AND are streaming - cap frames to "reset" to 3 (NEEDS TESTING)
[ ] deal with someone trying to use an EFF for streaming AND non-streaming (not sure where to store the "type" data just yet...)
[ ] Test Axem's headani script
[ ] Remove all temp debug code

Note that I'm pretty sure I can't help the effects animations (at least not easily), in the new deferred rendered these are handled differently to headanis so the same easy win doesn't apply to them. Atlasing could help, but that's a far bigger & more complicated change.
Creating a fs2_open.log | Red Alert Bug = Hex Edit | MediaVPs 2014: Bigger HUD gauges | 32bit libs for 64bit Ubuntu
----
Debian Packages (testing/unstable): Freespace2 | wxLauncher
----
m|m: I think I'm suffering from Stockholm syndrome. Bmpman is starting to make sense and it's actually written reasonably well...

 

Offline Spoon

  • 212
  • ヾ(´︶`♡)ノ
Re: Let's talk about bmpman
You are a star, niffi.

Be sure to let me know if you need anything tested, you know where to find me!
Urutorahappī!!

[02:42] <@Axem> spoon somethings wrong
[02:42] <@Axem> critically wrong
[02:42] <@Axem> im happy with these missions now
[02:44] <@Axem> well
[02:44] <@Axem> with 2 of them

 

Offline m!m

  • 211
Re: Let's talk about bmpman
I've been thinking about a complete rewrite of bmpman in order to fix these issues once and for all (until other issues arise :nervous:) but before doing any serious work it would be best to have some sort of specification what bmpman should be able to do. Here is what I have come up with so far, feedback and additions are obviously welcome:

  • Load a bitmap by file name and return an integer handle that represents it, for animations handle+i refers to the i-th frame of the animation
  • On mission load the different subsystems specify which bitmaps should be loaded for this mission. Actual loading is done after subsystems are done loading, could be multi-threaded
  • There should be an "unlimited" amount of possible bitmap slots but the image data may not actually be loaded yet
  • It should have a "lock" function to load the bitmap data into system memory
  • The data format of the image in memory should be explicit (e.g. RGBA 8-bit for each channel) as opposed to the current bits-per-pixel system which is not very helpful in most cases
  • No support for palettes, just use the decompressed pixel data
  • Support for compressed textures if the GPU supports that format (e.g S3TC, ASTC)
  • Allow using texture atlassing in the future (needs changes in the rendering engine)
« Last Edit: September 14, 2015, 05:05:44 am by m!m »

 

Offline Echelon9

  • 210
Re: Let's talk about bmpman
  • Support for S3TC

Can we also add support for ASTC (see also here for technical comparison)? This is Adaptive Scalable Texture Compression and is a lossy, block-based algorithm originally designed by ARM and importantly is royalty-free.

Native hardware support is starting to appear on modern graphics chipsets, e.g. Intel's Skylake.

 

Offline niffiwan

  • 211
  • Eluder Class
Re: Let's talk about bmpman
For animations I really think we should use atlasing; WoD has hit issues from all the effects anims chewing up the available slots. One altas per animation could be the simplest way to do this; but it would be better to have multiple anims per atlas, e.g. the approx 6x used in the mediavps_2014 explosions/flashy-death scripts could be atlased as one image. Naturally this'll require a bunch of changes outside BMPMAN in order to track & process the uv coords (I think that's the correct term) for each frame.

Or have the option to load animations in two modes, either "fat n+i" mode, or streaming mode with a single slot used and the textures rebound for each frame through the one slot.

edit: ACK - of course limits only matter if we hit perf issues from having too many slots in-use at once and that'll depend on the underlying data structures (see Chandler Carruth on high-perf C++ data structures). On rereading your list I realise that the number of slots used may not matter. Still - for faster GPU performance I think atlasing is the answer, not more/unlimited slots (but it may be able to be separated from this proposed change).
« Last Edit: September 14, 2015, 12:55:19 am by niffiwan »
Creating a fs2_open.log | Red Alert Bug = Hex Edit | MediaVPs 2014: Bigger HUD gauges | 32bit libs for 64bit Ubuntu
----
Debian Packages (testing/unstable): Freespace2 | wxLauncher
----
m|m: I think I'm suffering from Stockholm syndrome. Bmpman is starting to make sense and it's actually written reasonably well...

 

Offline Fury

  • The Curmudgeon
  • 213
Re: Let's talk about bmpman
While ASTC has been an official extension to OpenGL since 2012, it never became part of the core spec afaik. ASTC became part of Direct3D spec only in 11.3 and 12. No word on Vulkan regarding ASTC, but it is safe to assume ASTC is in the spec. This pretty much means that only GPUs that officially supports full spec of D3D 11.3 and 12 or Vulkan are guaranteed to have hardware support for ASTC textures.

Intel Skylake supports D3D 12, so obviously it would also support ASTC. Neither NVIDIA or AMD has yet to release next-gen GPU's that have full D3D 12 support, partial support does not guarantee ASTC support and there simply is no reliable information on the subject.

 

Offline m!m

  • 211
Re: Let's talk about bmpman
Can we also add support for ASTC (see also here for technical comparison)? This is Adaptive Scalable Texture Compression and is a lossy, block-based algorithm originally designed by ARM and importantly is royalty-free.
ASTC is still not widely supported by the hardware so most times we would need to decompress the texture in software and then send the decompressed data to the GPU. Bmpman can obviously be designed in such a way that different compression methods can be added later and I intend to do just that.

Re Atlasing: Would the modder have to create the texutre atlas manually and then just tell FSO the layout or should FSO generate the atlas by itself? OpenGL (and probably Vulkan) has support for "Array Textures" which are similar to texture atlases but apparently avoid some of the problems. If we use array textures then FSO needs to be able to separately access the data of the individual frames and that would mean that the modder keeps the data in separate images and FSO only combines them via the OpenGL/Vulkan functions.

I don't think that texture arrays should be done as part of the bmpman rewrite because that's something the OpenGL backend has to handle, not bmpman. I will still try to make sure bmpman can provide all the necessary information for building the texture atlas.

I updated the list with to reflect the recent comments.

 

Offline Spoon

  • 212
  • ヾ(´︶`♡)ノ
Re: Let's talk about bmpman
Re Atlasing: Would the modder have to create the texutre atlas manually and then just tell FSO the layout or should FSO generate the atlas by itself?
Considering I don't even have the vaguest of clues what that even means, and considering the sheer list of stuff we already have to deal with during modding, my initial response to this is "Dear FRED, not more **** to deal with, please let FSO do everything for us!"
Urutorahappī!!

[02:42] <@Axem> spoon somethings wrong
[02:42] <@Axem> critically wrong
[02:42] <@Axem> im happy with these missions now
[02:44] <@Axem> well
[02:44] <@Axem> with 2 of them