Hard Light Productions Forums
General FreeSpace => FreeSpace & FreeSpace Open Support => Topic started by: Bryan See on March 29, 2017, 12:56:09 pm
-
Over a long term, mission loading poses same problems since Retail.
One example is, while it's loading, Alt-Tabbing it could cause FS2_Open to close unexpectedly in some circumstances (happened in Windows 10). Also, the application may hang itself before it happens.
And, after a long while, it fails to load as the game hangs for some reason.
-
The "some reason" is because of FSO ignoring system messages during the loading process, which was fixed in nightly builds months ago.
-
I know it was fixed in nightly builds months ago, but it hangs for some reason. I know it has something to do with code streamlining and optimization in order to speed up mission performance without the need for a mission designer to sacrifice some mission size (e.g. ships, events, etc.)
-
If you already know how this bug happens, feel free to explain it.
Or, better yet, prove that you actually are a coder and submit a fix.
-
OK fine. The bug may lie in the use of malloc, and other areas. Replacing malloc with calloc will solve the problem, but implementing streaming will also solve the problem. Streaming in-game is something that FSO hasn't tried.
-
OK fine. The bug may lie in the use of malloc, and other areas. Replacing malloc with calloc will solve the problem
Please explain this. Be sure to point out what the difference between calloc and malloc are, and why they are relevant to this issue.
, but implementing streaming will also solve the problem. Streaming in-game is something that FSO hasn't tried.
Please explain why a streaming load system would be a good idea for FSO.
-
I propose a roadmap for implementing streamloading of missions with support also for parallel retail data, but this should be done using a design pattern that is compatible with 2D allocation techniques and socket-based buffering. Do you have further ideas on this, Bryan?
-
Yes. I do have. Given the streamloading of bitmaps, effects, models, shaders, maps, sounds are a must, it could involve resources being loaded in an instant rather than in a single operation between missions. With the system in place, loading times were reduced to as little as a few seconds.
But I'm afraid that implementing such a system on the potentially infinite variations of PC hardware setups in use poses serious performance problems.
Textures are loaded to memory but only moved to the system's video card when needed and audio files are loaded with an unusual "soundcache" system: only the first 0.125 seconds of each file are pre-cached, and the clip is used to cover the asynchronous buffering of the full sound file in the background when it is first requested.
Both systems keep data in the heap until there is no more room and older resources are flushed out, and when either is held up or otherwise slowed down the engine will either freeze or go into a temporary loop until the data arrives. 'Stuttering', or 'hitching' as it is sometimes known, is the result of these pauses.
While stutter can be caused by poor system performance, it has also been noted on hardware setups that should be more than powerful enough to cope with the data rate, and despite many theories, the precise cause remains unknown to the public even over two years since the engine's debut. Most solutions that have been found involve bypassing the caching system, as it cannot be directly disabled, or system-specific optimizations (e.g. driver updates).
So, I've got a fork of FS2 SCP and Zookeeper and me will be announcing streamloading of missions very soon.
-
Be sure to point out what the difference between calloc and malloc are
From man malloc:
Name
malloc, free, calloc, realloc - allocate and free dynamic memory
Synopsis
#include <stdlib.h>
void *malloc(size_t size);
void free(void *ptr);
void *calloc(size_t nmemb, size_t size);
void *realloc(void *ptr, size_t size);
Description
The malloc() function allocates size bytes and returns a pointer to the allocated memory. The memory is not initialized. If size is 0, then malloc() returns either NULL, or a unique pointer value that can later be successfully passed to free().
[...]
The calloc() function allocates memory for an array of nmemb elements of size bytes each and returns a pointer to the allocated memory. The memory is set to zero. If nmemb or size is 0, then calloc() returns either NULL, or a unique pointer value that can later be successfully passed to free().
In other words, both functions allocate a block of memory, but calloc erases whatever was in that memory before (by setting all those bytes to 0), while malloc leaves it in. (The arguments are also passed in a slightly different way, but don't worry about that.) The "c" in calloc stands for "clear".
I'm not familiar with the code in question, so I couldn't say why this would be relevant. Allocating space for a null-terminated array or an array of pointers is slightly safer with calloc, I suppose?
-
Yes. I do have. Given the streamloading of bitmaps, effects, models, shaders, maps, sounds are a must, it could involve resources being loaded in an instant rather than in a single operation between missions. With the system in place, loading times were reduced to as little as a few seconds.
Careful, your cluelessness is showing again. With this statement, you have shown conclusively that you have no idea what streaming systems are, how they work, and why they aren't a good fit for FSO's mission architecture.
Do you get the point yet, Bryan?
But I'm afraid that implementing such a system on the potentially infinite variations of PC hardware setups in use poses serious performance problems.
That's not the reason why we aren't doing this. Which you would know, if you had a single ****ing clue of what you are talking about.
Textures are loaded to memory but only moved to the system's video card when needed and audio files are loaded with an unusual "soundcache" system: only the first 0.125 seconds of each file are pre-cached, and the clip is used to cover the asynchronous buffering of the full sound file in the background when it is first requested. Both systems keep data in the heap until there is no more room and older resources are flushed out, and when either is held up or otherwise slowed down the engine will either freeze or go into a temporary loop until the data arrives. 'Stuttering', or 'hitching' as it is sometimes known, is the result of these pauses.
While stutter can be caused by poor system performance, it has also been noted on hardware setups that should be more than powerful enough to cope with the data rate, and despite many theories, the precise cause remains unknown to the public even over two years since the engine's debut. Most solutions that have been found involve bypassing the caching system, as it cannot be directly disabled, or system-specific optimizations (e.g. driver updates).
Okay, this whole paragraph didn't seem like something you'd write (it's far too coherent, and the bit about the sound engine bears no relation to what FSO is doing), so I checked where you copied it from. Based on the results of that search (an article on common issues in Valve's Source engine (http://half-life.wikia.com/wiki/Source#Common_issues)), I have to ask: Are you aware that FSO isn't Source? That it is in no way related to it, shares no code and very few concepts with it, and is (to put it succinctly) a completely different engine altogether?
So, I've got a fork of FS2 SCP and Zookeeper and me will be announcing streamloading of missions very soon.
Did you tell zookeeper about this too?
I'm not familiar with the code in question, so I couldn't say why this would be relevant. Allocating space for a null-terminated array or an array of pointers is slightly safer with calloc, I suppose?
It is utterly irrelevant for FSO. In most cases where we allocate memory, we want to fill it with reasonable data; zeroed out memory isn't particularly useful.
-
Textures are loaded to memory but only moved to the system's video card when needed and audio files are loaded with an unusual "soundcache" system: only the first 0.125 seconds of each file are pre-cached, and the clip is used to cover the asynchronous buffering of the full sound file in the background when it is first requested. Both systems keep data in the heap until there is no more room and older resources are flushed out, and when either is held up or otherwise slowed down the engine will either freeze or go into a temporary loop until the data arrives. 'Stuttering', or 'hitching' as it is sometimes known, is the result of these pauses.
While stutter can be caused by poor system performance, it has also been noted on hardware setups that should be more than powerful enough to cope with the data rate, and despite many theories, the precise cause remains unknown to the public even over two years since the engine's debut. Most solutions that have been found involve bypassing the caching system, as it cannot be directly disabled, or system-specific optimizations (e.g. driver updates).
Okay, this whole paragraph didn't seem like something you'd write (it's far too coherent), so I checked where you copied it from. Based on the results of that search (an article on common issues in Valve's Source engine (http://half-life.wikia.com/wiki/Source#Common_issues)), I have to ask: Are you aware that FSO isn't Source? That it is in no way related to it, shares no code and very few concepts with it, and is (to put it succinctly) a completely different engine altogether?
What about Image-Based Rendering, a technique in which 2D elements are manipulated to appear in a 3D world, pushed by the visionary Gabe Newell?
So, I've got a fork of FS2 SCP and Zookeeper and me will be announcing streamloading of missions very soon.
Did you tell zookeeper about this too?
I am also planning to implement an event-based asynchronous file loading architecture and possibly adding support for zero-dimensional point data in a model format for allowing dynamic mesh operations.
-
calloc is potentially slower than malloc if you don't need to zero out the memory; it's certainly not faster.
-
I am also planning to implement an event-based asynchronous file loading architecture and possibly adding support for zero-dimensional point data in a model format for allowing dynamic mesh operations.
GEOMOD WHEN?
-
I am still half-convinced that Bryan is some sort of Turing test gone rogue.
-
calloc is potentially slower than malloc if you don't need to zero out the memory; it's certainly not faster.
That too, but I'm more at a loss for how it's supposed to help FSO not miss Windows system events :/
-
calloc is potentially slower than malloc if you don't need to zero out the memory; it's certainly not faster.
That too, but I'm more at a loss for how it's supposed to help FSO not miss Windows system events :/
Still waiting for that explanation from Bryan. He is, after all, the visionary coder here.
-
you see windows alerts you to system events by writing junk data into your allocated pages; but if you use malloc they're already full of junk and you can't detect it! if fso switches to calloc it will be easier and more reliable to detect system junk