Author Topic: Mission loading  (Read 1370 times)

0 Members and 1 Guest are viewing this topic.

Offline Bryan See

  • Has anyone really been far as decided to use even go want to do look more like?
  • 210
  • Trying to redeem.
    • Skype
    • Steam
    • Twitter
Over a long term, mission loading poses same problems since Retail.

One example is, while it's loading, Alt-Tabbing it could cause FS2_Open to close unexpectedly in some circumstances (happened in Windows 10). Also, the application may hang itself before it happens.

And, after a long while, it fails to load as the game hangs for some reason.
Bryan See - My FreeSpace Wiki User Page (Talk, Contributions)

Campaigns:
FreeSpace: Reunited - Shattered Stars

Ships:
GTS Hygeia, GTT Argo, SC Raguel

 

Offline AdmiralRalwood

  • 211
  • The Cthulhu programmer himself!
    • Skype
    • Steam
    • Twitter
The "some reason" is because of FSO ignoring system messages during the loading process, which was fixed in nightly builds months ago.
Ph'nglui mglw'nafh Codethulhu GitHub wgah'nagl fhtagn.

schrödinbug (noun) - a bug that manifests itself in running software after a programmer notices that the code should never have worked in the first place.

When you gaze long into BMPMAN, BMPMAN also gazes into you.

"I am one of the best FREDders on Earth" -General Battuta

<Aesaar> literary criticism is vladimir putin

<MageKing17> "There's probably a reason the code is the way it is" is a very dangerous line of thought. :P
<MageKing17> Because the "reason" often turns out to be "nobody noticed it was wrong".
(the very next day)
<MageKing17> this ****ing code did it to me again
<MageKing17> "That doesn't really make sense to me, but I'll assume it was being done for a reason."
<MageKing17> **** ME
<MageKing17> THE REASON IS PEOPLE ARE STUPID
<MageKing17> ESPECIALLY ME

<MageKing17> God damn, I do not understand how this is breaking.
<MageKing17> Everything points to "this should work fine", and yet it's clearly not working.
<MjnMixael> 2 hours later... "God damn, how did this ever work at all?!"
(...)
<MageKing17> so
<MageKing17> more than two hours
<MageKing17> but once again we have reached the inevitable conclusion
<MageKing17> How did this code ever work in the first place!?

<@The_E> Welcome to OpenGL, where standards compliance is optional, and error reporting inconsistent

<MageKing17> It was all working perfectly until I actually tried it on an actual mission.

<IronWorks> I am useful for FSO stuff again. This is a red-letter day!
* z64555 erases "Thursday" and rewrites it in red ink

<MageKing17> TIL the entire homing code is held up by shoestrings and duct tape, basically.

 

Offline Bryan See

  • Has anyone really been far as decided to use even go want to do look more like?
  • 210
  • Trying to redeem.
    • Skype
    • Steam
    • Twitter
I know it was fixed in nightly builds months ago, but it hangs for some reason. I know it has something to do with code streamlining and optimization in order to speed up mission performance without the need for a mission designer to sacrifice some mission size (e.g. ships, events, etc.)
Bryan See - My FreeSpace Wiki User Page (Talk, Contributions)

Campaigns:
FreeSpace: Reunited - Shattered Stars

Ships:
GTS Hygeia, GTT Argo, SC Raguel

 

Offline The E

  • He's Ebeneezer Goode
  • Global Moderator
  • 213
  • Nothing personal, just tech support.
    • Steam
    • Twitter
If you already know how this bug happens, feel free to explain it.

Or, better yet, prove that you actually are a coder and submit a fix.
Let there be light
Let there be moon
Let there be stars and let there be you
Let there be monsters and let there be pain
Let us begin to feel again
--Devin Townsend, Genesis

 

Offline Bryan See

  • Has anyone really been far as decided to use even go want to do look more like?
  • 210
  • Trying to redeem.
    • Skype
    • Steam
    • Twitter
OK fine. The bug may lie in the use of malloc, and other areas. Replacing malloc with calloc will solve the problem, but implementing streaming will also solve the problem. Streaming in-game is something that FSO hasn't tried.
Bryan See - My FreeSpace Wiki User Page (Talk, Contributions)

Campaigns:
FreeSpace: Reunited - Shattered Stars

Ships:
GTS Hygeia, GTT Argo, SC Raguel

 

Offline The E

  • He's Ebeneezer Goode
  • Global Moderator
  • 213
  • Nothing personal, just tech support.
    • Steam
    • Twitter
OK fine. The bug may lie in the use of malloc, and other areas. Replacing malloc with calloc will solve the problem

Please explain this. Be sure to point out what the difference between calloc and malloc are, and why they are relevant to this issue.

Quote
, but implementing streaming will also solve the problem. Streaming in-game is something that FSO hasn't tried.

Please explain why a streaming load system would be a good idea for FSO.
Let there be light
Let there be moon
Let there be stars and let there be you
Let there be monsters and let there be pain
Let us begin to feel again
--Devin Townsend, Genesis

 

Offline zookeeper

  • *knock knock* Who's there? Poe. Poe who?
  • 210
I propose a roadmap for implementing streamloading of missions with support also for parallel retail data, but this should be done using a design pattern that is compatible with 2D allocation techniques and socket-based buffering. Do you have further ideas on this, Bryan?

 

Offline Bryan See

  • Has anyone really been far as decided to use even go want to do look more like?
  • 210
  • Trying to redeem.
    • Skype
    • Steam
    • Twitter
Yes. I do have. Given the streamloading of bitmaps, effects, models, shaders, maps, sounds are a must, it could involve resources being loaded in an instant rather than in a single operation between missions. With the system in place, loading times were reduced to as little as a few seconds.

But I'm afraid that implementing such a system on the potentially infinite variations of PC hardware setups in use poses serious performance problems.

Textures are loaded to memory but only moved to the system's video card when needed and audio files are loaded with an unusual "soundcache" system: only the first 0.125 seconds of each file are pre-cached, and the clip is used to cover the asynchronous buffering of the full sound file in the background when it is first requested.

Both systems keep data in the heap until there is no more room and older resources are flushed out, and when either is held up or otherwise slowed down the engine will either freeze or go into a temporary loop until the data arrives. 'Stuttering', or 'hitching' as it is sometimes known, is the result of these pauses.

While stutter can be caused by poor system performance, it has also been noted on hardware setups that should be more than powerful enough to cope with the data rate, and despite many theories, the precise cause remains unknown to the public even over two years since the engine's debut. Most solutions that have been found involve bypassing the caching system, as it cannot be directly disabled, or system-specific optimizations (e.g. driver updates).

So, I've got a fork of FS2 SCP and Zookeeper and me will be announcing streamloading of missions very soon.
Bryan See - My FreeSpace Wiki User Page (Talk, Contributions)

Campaigns:
FreeSpace: Reunited - Shattered Stars

Ships:
GTS Hygeia, GTT Argo, SC Raguel

 
Be sure to point out what the difference between calloc and malloc are

From man malloc:

Quote
Name

malloc, free, calloc, realloc - allocate and free dynamic memory

Synopsis

#include <stdlib.h>

void *malloc(size_t size);
void free(void *ptr);
void *calloc(size_t nmemb, size_t size);
void *realloc(void *ptr, size_t size);

Description

The malloc() function allocates size bytes and returns a pointer to the allocated memory. The memory is not initialized. If size is 0, then malloc() returns either NULL, or a unique pointer value that can later be successfully passed to free().

[...]

The calloc() function allocates memory for an array of nmemb elements of size bytes each and returns a pointer to the allocated memory. The memory is set to zero. If nmemb or size is 0, then calloc() returns either NULL, or a unique pointer value that can later be successfully passed to free().

In other words, both functions allocate a block of memory, but calloc erases whatever was in that memory before (by setting all those bytes to 0), while malloc leaves it in. (The arguments are also passed in a slightly different way, but don't worry about that.) The "c" in calloc stands for "clear".

I'm not familiar with the code in question, so I couldn't say why this would be relevant. Allocating space for a null-terminated array or an array of pointers is slightly safer with calloc, I suppose?

 

Offline The E

  • He's Ebeneezer Goode
  • Global Moderator
  • 213
  • Nothing personal, just tech support.
    • Steam
    • Twitter
Yes. I do have. Given the streamloading of bitmaps, effects, models, shaders, maps, sounds are a must, it could involve resources being loaded in an instant rather than in a single operation between missions. With the system in place, loading times were reduced to as little as a few seconds.

Careful, your cluelessness is showing again. With this statement, you have shown conclusively that you have no idea what streaming systems are, how they work, and why they aren't a good fit for FSO's mission architecture.

Do you get the point yet, Bryan?

Quote
But I'm afraid that implementing such a system on the potentially infinite variations of PC hardware setups in use poses serious performance problems.

That's not the reason why we aren't doing this. Which you would know, if you had a single ****ing clue of what you are talking about.

Quote
Textures are loaded to memory but only moved to the system's video card when needed and audio files are loaded with an unusual "soundcache" system: only the first 0.125 seconds of each file are pre-cached, and the clip is used to cover the asynchronous buffering of the full sound file in the background when it is first requested. Both systems keep data in the heap until there is no more room and older resources are flushed out, and when either is held up or otherwise slowed down the engine will either freeze or go into a temporary loop until the data arrives. 'Stuttering', or 'hitching' as it is sometimes known, is the result of these pauses.

While stutter can be caused by poor system performance, it has also been noted on hardware setups that should be more than powerful enough to cope with the data rate, and despite many theories, the precise cause remains unknown to the public even over two years since the engine's debut. Most solutions that have been found involve bypassing the caching system, as it cannot be directly disabled, or system-specific optimizations (e.g. driver updates).

Okay, this whole paragraph didn't seem like something you'd write (it's far too coherent, and the bit about the sound engine bears no relation to what FSO is doing), so I checked where you copied it from. Based on the results of that search (an article on common issues in Valve's Source engine), I have to ask: Are you aware that FSO isn't Source? That it is in no way related to it, shares no code and very few concepts with it, and is (to put it succinctly) a completely different engine altogether?

Quote
So, I've got a fork of FS2 SCP and Zookeeper and me will be announcing streamloading of missions very soon.

Did you tell zookeeper about this too?

I'm not familiar with the code in question, so I couldn't say why this would be relevant. Allocating space for a null-terminated array or an array of pointers is slightly safer with calloc, I suppose?

It is utterly irrelevant for FSO. In most cases where we allocate memory, we want to fill it with reasonable data; zeroed out memory isn't particularly useful.
« Last Edit: April 04, 2017, 01:41:26 pm by The E »
Let there be light
Let there be moon
Let there be stars and let there be you
Let there be monsters and let there be pain
Let us begin to feel again
--Devin Townsend, Genesis

 

Offline zookeeper

  • *knock knock* Who's there? Poe. Poe who?
  • 210
Quote
Textures are loaded to memory but only moved to the system's video card when needed and audio files are loaded with an unusual "soundcache" system: only the first 0.125 seconds of each file are pre-cached, and the clip is used to cover the asynchronous buffering of the full sound file in the background when it is first requested. Both systems keep data in the heap until there is no more room and older resources are flushed out, and when either is held up or otherwise slowed down the engine will either freeze or go into a temporary loop until the data arrives. 'Stuttering', or 'hitching' as it is sometimes known, is the result of these pauses.

While stutter can be caused by poor system performance, it has also been noted on hardware setups that should be more than powerful enough to cope with the data rate, and despite many theories, the precise cause remains unknown to the public even over two years since the engine's debut. Most solutions that have been found involve bypassing the caching system, as it cannot be directly disabled, or system-specific optimizations (e.g. driver updates).

Okay, this whole paragraph didn't seem like something you'd write (it's far too coherent), so I checked where you copied it from. Based on the results of that search (an article on common issues in Valve's Source engine), I have to ask: Are you aware that FSO isn't Source? That it is in no way related to it, shares no code and very few concepts with it, and is (to put it succinctly) a completely different engine altogether?

What about Image-Based Rendering, a technique in which 2D elements are manipulated to appear in a 3D world, pushed by the visionary Gabe Newell?


Quote
So, I've got a fork of FS2 SCP and Zookeeper and me will be announcing streamloading of missions very soon.

Did you tell zookeeper about this too?

I am also planning to implement an event-based asynchronous file loading architecture and possibly adding support for zero-dimensional point data in a model format for allowing dynamic mesh operations.

 
calloc is potentially slower than malloc if you don't need to zero out the memory; it's certainly not faster.
The good Christian should beware of mathematicians, and all those who make empty prophecies. The danger already exists that the mathematicians have made a covenant with the devil to darken the spirit and to confine man in the bonds of Hell.

 

Offline qazwsx

  • POST DRUNK GET TITLE
  • 29
    • Minecraft
I am also planning to implement an event-based asynchronous file loading architecture and possibly adding support for zero-dimensional point data in a model format for allowing dynamic mesh operations.

GEOMOD WHEN?
<Achillion> I mean, it's not like he's shoving the brain-goo in a usb slot and praying to kurzweil to bring the singularity

<dsockwell> idk about you guys but the reason i follow God's law is so I can get my rocks off in the afterlife

 

Offline Mongoose

  • Rikki-Tikki-Tavi
  • Global Moderator
  • 212
  • This brain for rent.
    • Minecraft
    • Steam
    • Something
I am still half-convinced that Bryan is some sort of Turing test gone rogue.

 
calloc is potentially slower than malloc if you don't need to zero out the memory; it's certainly not faster.

That too, but I'm more at a loss for how it's supposed to help FSO not miss Windows system events :/

 

Offline The E

  • He's Ebeneezer Goode
  • Global Moderator
  • 213
  • Nothing personal, just tech support.
    • Steam
    • Twitter
calloc is potentially slower than malloc if you don't need to zero out the memory; it's certainly not faster.

That too, but I'm more at a loss for how it's supposed to help FSO not miss Windows system events :/

Still waiting for that explanation from Bryan. He is, after all, the visionary coder here.
Let there be light
Let there be moon
Let there be stars and let there be you
Let there be monsters and let there be pain
Let us begin to feel again
--Devin Townsend, Genesis

 
you see windows alerts you to system events by writing junk data into your allocated pages; but if you use malloc they're already full of junk and you can't detect it! if fso switches to calloc it will be easier and more reliable to detect system junk
The good Christian should beware of mathematicians, and all those who make empty prophecies. The danger already exists that the mathematicians have made a covenant with the devil to darken the spirit and to confine man in the bonds of Hell.