A thought then (blind as I've not looked at the code):
Open the EFF file, process and close.
Spawn a thread that does the following:
- Create a number of file handles to open the data, either the number of frames or a sensible maximum number (eg 10). (This does not limit the number of frames)
- Open these files
- Start to load the frames into memory.
- As soon as the first frame is in memory, close its file and reassign the file handle to the next unassigned frame (eg 11), start to load that.
- If the last frame already has a handle, delete the completed handles as they're not needed anymore.
- The thread dies once all frames are loaded.
The loading thread has to inform the caller when it's sufficiently far ahead for it to start playing based on framerate and speed of loading each frame.
Once the loader gets sufficiently far ahead you can start playing the effect.
For effects that are fully loaded into memory, obviously that doesn't matter.
- It might be faster to have the loader thread only access one file at a time, depends on the setup time for file handles.