Hard Light Productions Forums

Modding, Mission Design, and Coding => FS2 Open Coding - The Source Code Project (SCP) => Topic started by: Lightspeed on June 06, 2004, 12:09:05 pm

Title: There is something wrong.... very wrong.
Post by: Lightspeed on June 06, 2004, 12:09:05 pm
Err...

okay

let's sum it up:

Bobs Builds

I tried playing the last days and had certain... err.. problems

1) The game seems to hang from time to time as if it was loading stuff - but stuff that should technically have been loaded before. Example: I watch some cruisers. Framerate is at 40, I start turning away from them and the game shudders and locks up for nearly a second, dropping down to 2.1 FPS, then speeds up and is at 80 FPS. If I turn back it does the same thing again.

2) When textures come into the screen they sometimes are only white rectangles. (this also applies to beamglows and trails appearing). The white rectangles vanish after about 1/10 second and show the normal texture. This applies to background images, explosions, beams, and sometimes laser shots.

3) The framerate drops to horrendous values in some situations (which it shouldn) - to about 4 FPS to be precise. It stutters and locks around and jumps wildly between 4.1, 6.8, 12.7, etc.

4) Any time an ANI (shockwave, explosion, etc) gets played it locks up for a bit and then stutters back to normal speed. I have exlcuded any of the new effects as I ran it with retail data and had the same thing happening.

---

Now, I tested around and tried various things. I dont know if its code based or something wrong with my system, but SOMETHING seems to be killing off my performance (wayyy more than it should). It seems to happening with any of Bob's latest builds. As I tried and removed stuff back to literally retail level, and still didnt get any noticeable speedup, I reverted to the 3.6 exe. Now, there everything seems to be in fine order. My framerates are (literally) TEN times larger, and none of the above is happening.

This is why I came to the conclusion, that anything with the new features has to be messed up. Either decals, or env mapping. As i was not using ANY decals when I tested, it seems to have something to do with env mapping. It also isn't the alpha-env mapping, as I've disabled it to test - same thing. (see screenies below).

This here, is a comparison of the 3.6 and fs2_open_Bobboau_7-5-04 exes. As you can see the situation is exactly the same (same mission, same everything), but the framerates are different by a multiplier of 10.

3.6 RELEASE:
(http://www.penguinbomb.com/lightspeed/SCPTest/HELL.jpg)

NEW BUILD:
(http://www.penguinbomb.com/lightspeed/SCPTest/HELL02.jpg)

As you can see i'm rather depressed, and feel this shouldnt be happening. If you have any ideas what this could possibly be, please help me. It might be a problem of the code, might also be a problem of my data (I've checked about everything though and that didnt help either). All I know is, somethings wrong. No way that the new features would cause a THAT severe slowdown, right?

And no, my system is not specifically low spec, as I've got a 2.4 GHz machine with 512 MB RAM, and a Radeon 9800.
Title: There is something wrong.... very wrong.
Post by: Turambar on June 06, 2004, 12:26:12 pm
I have a similar machine with a somewhat less L337 gfx card and i have similar issues.  WTF Bob?!?
Title: There is something wrong.... very wrong.
Post by: phreak on June 06, 2004, 12:30:19 pm
try playing my new one
Title: There is something wrong.... very wrong.
Post by: mrduckman on June 06, 2004, 12:36:04 pm
I had the same thing. But I've checked only on the first mission. When I was pointing to the Psamtik the FPS dropped dramatically to 5.
Title: There is something wrong.... very wrong.
Post by: Bobboau on June 06, 2004, 01:16:43 pm
did this start with environment mapping and never got better?
I'm thinking it might be decals as there is some particularly hairy code in there.
Title: There is something wrong.... very wrong.
Post by: RandomTiger on June 06, 2004, 02:43:53 pm
Lightspeed, please run your tests again with this flag on:

-d3d_bad_tsys
Title: Re: There is something wrong.... very wrong.
Post by: Lightspeed on June 06, 2004, 03:53:59 pm
:eek2: :eek2: WHAT the heck does that flag do - its a difference like night & day?

Here's the results:

Bobs Builds - WITH FLAG

1) The game seems to hang from time to time as if it was loading stuff - but stuff that should technically have been loaded before. Example: I watch some cruisers. Framerate is at 40, I start turning away from them and the game shudders and locks up for nearly a second, dropping down to 2.1 FPS, then speeds up and is at 80 FPS. If I turn back it does the same thing again. STILL HAPPENING - The drops are less intense though

2) When textures come into the screen they sometimes are only white rectangles. (this also applies to beamglows and trails appearing). The white rectangles vanish after about 1/10 second and show the normal texture. This applies to background images, explosions, beams, and sometimes laser shots. STILL HAPPENING

3) The framerate drops to horrendous values in some situations (which it shouldn) - to about 4 FPS to be precise. It stutters and locks around and jumps wildly between 4.1, 6.8, 12.7, etc. RESOLVED - The framerate is still worse enough, but never falls to such low values anymore.

4) Any time an ANI (shockwave, explosion, etc) gets played it locks up for a bit and then stutters back to normal speed. I have exlcuded any of the new effects as I ran it with retail data and had the same thing happening. RESOLVED - This seems to be gone

---

3.6 RELEASE:
(http://www.penguinbomb.com/lightspeed/SCPTest/HELL.jpg)

NEW BUILD with flag:

Varies between 27-55 FPS. All-in-all it feels slower, but I think that's mainly due to the hanging problem (see 1)

-----

Its still bad enough, but A LOT better than it was before.
Title: There is something wrong.... very wrong.
Post by: RandomTiger on June 06, 2004, 04:09:39 pm
That flag changes between the two texture systems in D3D.
Interesting stuff thanks.
Title: There is something wrong.... very wrong.
Post by: Lightspeed on June 06, 2004, 04:26:55 pm
Well, the 'bad' one is about five times better as it seems.
Title: There is something wrong.... very wrong.
Post by: RandomTiger on June 06, 2004, 04:35:56 pm
I've updated my sig with a new exe that has a fix attempt.
Please (at least at first) run it *without* -d3d_bad_tsys.
And tell me how it does.
Title: There is something wrong.... very wrong.
Post by: Bobboau on June 06, 2004, 09:59:56 pm
you know the white square thing might be relaited to environment mapping, I have it set a texture directly with D3D_setTexture(), or it might be related to the state block code, but I find that less likely.
Title: There is something wrong.... very wrong.
Post by: JarC on June 07, 2004, 01:46:39 am
Ah this is good news to wake up with....;) not this 'bug' I mean, but the fact that it is a bug afterall, I already had this attirbuted to my low-end machine...glad to see others have it too...
Title: There is something wrong.... very wrong.
Post by: Lightspeed on June 07, 2004, 12:16:44 pm
Quote
Originally posted by RandomTiger
I've updated my sig with a new exe that has a fix attempt.
Please (at least at first) run it *without* -d3d_bad_tsys.
And tell me how it does.


I cant. It doesnt seem to recognize any of bob's command lines.
Title: There is something wrong.... very wrong.
Post by: RandomTiger on June 07, 2004, 12:44:21 pm
That build is a plain 3.6 release with some bug fixes.
I would be great of you could give it a test though.
Title: There is something wrong.... very wrong.
Post by: Lightspeed on June 07, 2004, 12:57:36 pm
That wouldn't have any comparable results as the system drain is completely different. I could test it but that wouldn't show if at all/ how good/ the situation has been fixed.

I did some more testing with the 7-5 build and -d3d_bad_tsys turned on. Well, point three...

"The framerate drops to horrendous values in some situations (which it shouldn) - to about 4 FPS to be precise. It stutters and locks around and jumps wildly between 4.1, 6.8, 12.7, etc. RESOLVED - The framerate is still worse enough, but never falls to such low values anymore."

has now officially proven wrong. It DOES drop to such low values - but stays there only for seconds, and stutters around till speeding up again. It happens almost never though, whereas without the flag -d3d_bad_tsys it happened every time.

Well, there's definately some rather nasty things going on somewhere and those will have to be eliminated.
Title: There is something wrong.... very wrong.
Post by: RandomTiger on June 07, 2004, 03:00:26 pm
7-5 build? I take it thats Bob's build?

If you can help me test for bugs on that build then I can get them fixed up. Then Bob can update and it will be easier for him to fix his.
Title: There is something wrong.... very wrong.
Post by: Lightspeed on June 07, 2004, 03:22:03 pm
Okay - i'll see what I can do.

In the meantime. Here's a somewhat unusual test on Bob's 7-5 build. I recorded the FPS to a logfile every second, and put it into some 1337 excel tables.

y-scale = FPS, x-scale = seconds (the shortest hangers didnt even get recorded)

Red diagram = no flag
Yellow diagram = -d3d_bad_tsys

Click (http://www.penguinbomb.com/lightspeed/SCPTest/FPS.jpg)

The two curves are not synched (they're just in the same diagram so you can compare overall shapes), but I think you can see the problem.

I've also noticed, that I can magically slow down my game to the 4 FPS crawl. All I have to do is play without the flag and switch to external camera mode (, on the keypad) - my FPS instantly drop to 4 and stay there, no matter if/what there's (anything) in the background.
Title: There is something wrong.... very wrong.
Post by: Lightspeed on June 07, 2004, 04:57:46 pm
More testing with bobs builds!

The white blobs are almost definately due to texture loading. There seem to be some bad flaws in the way beams and suns are stored. Whenever a 1024² sun enters my screen (a lot of other action going on) it starts as the white rectangle and slows down the FPS to about 5 - then the sun image appears and the FPS go back up to where they were before. If I turn away from the sun it stutters again (as it leaves the screen). As soon as it re-enters the screen it stutters again. The same seems to happen for some effect graphics and ANIs.

However, it does NOT happen with 2048² planets, or 1024² background images. Haven't seen it on any ship textures yet, either.

So, it seems to load and unload textures wildly, which it shouldnt.
Title: There is something wrong.... very wrong.
Post by: Bobboau on June 07, 2004, 11:06:34 pm
arn't some of the alpha blended textures loaded diferently that the normal ones?

or you know what, tell me if this happens in any build before 6-3-04
Title: There is something wrong.... very wrong.
Post by: Bobboau on June 10, 2004, 12:32:57 am
try this build for framerate problems
http://freespace.volitionwatch.com/blackwater/fs2_open_Bobboau_6-10-04.zip
Title: There is something wrong.... very wrong.
Post by: Lightspeed on June 10, 2004, 09:31:55 am
Okay, off to testing I am. :)
Title: There is something wrong.... very wrong.
Post by: Lightspeed on June 10, 2004, 11:51:45 am
More Information

Okay, I did some more more testing today.

1. The 6-10 build

overall framerates a bit faster than on previous builds, but the problems remain (though the white square thingy seems to be pretty much fixed). The difference bewteen normal and d3dbadtsys got smaller. Now both ways lock up if I use the external camera to watch multiple ships (see below). The average FPS with the normal rendering is 0,2 FPS better than with -d3d_bad_tsys, but has a lot more hanging and stuttering.

Also, 'new' in this build, is the bug where any glows (thruster glows) suddenly get incredibly huge and go about 500% of their normal size, and then swap back immediately. Missiles tend to get rather messy too.

Another thing I noticed, is that without "d3d_badtsys" there seem to be glitches with background musics being played (namely: The game chose a wrong briefing music). With d3d_bad_tsys the right music was chosen and played.

2. The problems

I ran through all this with -stats and -timerbar to analyse where the problem seems to be coming from. I discovered some really interesting things.

Problem1: External camera
---
If I switch to external camera with a bit of fighting behind me, the framerate will instantly drop to x.x values and stay there. Sometimes the textures of ships (mostly the player ship) will whiten out.

TIMERBAR says: 10 complete red lines, 5 complete blue lines, 12 dark red lines, 13 black lines + some tiny snippets (green, pink, etc)

STATS: What came to my attention almost instantly is that the free texture memory (normally really small values such as 3 or 7) was relatively large while it was hanging, namely at 88 MB. (from 128 MB memory on my vid card, mind you). This value kept fluttering around (hard to see at 4 FPS) around this value.

Problem2: Hanging during gameplay
---
If textures come into my screen, the game will sometimes hang for a short time, and then go on normally. Note that this wont happen only once, so the same texture can make me hang twice. This also happens if ships explode, or just randomly while i'm turning around.

TIMERBAR says:
a) Lots of dark red lines.
b) Lots of black lines
c) Some blue lines

A and B can happen seperately (i.e. only dark red, or only black). Some stutters also seem to have a lot of bright red lines in it.

STATS: While I kept watching the stats I noticed that everytime the game slows down to one of the hangs there was a lot of texture memory freed. The Texture memory free went from 5 MB to around 30 MB in most cases. (the other memory consumption was not changed in any way). After the hanger, it was back at 5 MB. I *think* the hanging is produced by the fact that the game reloads some textures unneccessarily - An object comes into your FOV, the game trashes the textures it has ALREADY loaded for it, and then reloads them (leading to the hanging). Whatever happens, vid card memory is freed and filled immediately afterwards every time the hangings occur.

Problem 3: Explosion hanger
---
Even with retail explosions the game seems to slow down to 5 FPS sometimes (some other times it doesnt affect my FPS *at all*.). This seems to occur randomly when something explodes.

TIMERBAR says: 57+ complete black lines, 3 complete blue lines, 1 red line. Sometimes the red lines flicker up and grow to about 20 red lines - this doesn't happen every time it hangs on an explosion.

STATS: Nothing unusual, everything stays about the same as far as I could see.

Side note: For some reason, FS2 SCP seems to be consuming 1400+ MB of memory. This is a bit.... much, isn't it?
Title: There is something wrong.... very wrong.
Post by: Lightspeed on June 10, 2004, 08:34:27 pm
bump :(
Title: There is something wrong.... very wrong.
Post by: Gregster2k on June 10, 2004, 10:21:50 pm
Well, I THOUGHT something was wrong here when I was using Bob's build with my combi-Omniscaper super mega giga Star Trek ship pack combo i whipped together. I can't even face the Galaxy Class once without the FPS dropping to zilch. In fact, "Sector 001" is now unplayable (the one with teh super high polygon Earth model).

Oh yeah, and to make matters worse, Bob's build here, it kinda, well, CANT DISPLAY ANY DDS TEXTURES (dxt flag or not)...a problem that has been around ever since Bob's first tinkering build I ever tested, and that was a long time ago (prolly the very first environment mapping build, i gather.).

3.6 vanilla build displays ships with DDS textures; Bob's borks it every time without fail, displaying nothing where the damn model should be when it cant find the DDS textures that are RIGHT THERE.
Title: There is something wrong.... very wrong.
Post by: Bobboau on June 10, 2004, 11:15:26 pm
you know I think I'm going to manualy add features into the current CVS tree some time this weekend, I'm hardly seeing any of these issues.
Title: There is something wrong.... very wrong.
Post by: Lightspeed on June 11, 2004, 12:50:24 pm
Some of the problems happen with 3.6 too, but are a lot less intense there.

This is NOT specifically limited to Bob's builds.
Title: There is something wrong.... very wrong.
Post by: Gregster2k on June 11, 2004, 02:34:52 pm
I didn't see the DDS bug until Bob's first environment mapping build. Although it could have been before them, I'm speaking only from my own experience. i never said that I'm sure that Bob was responsible, but i suspect it. im open to suggestions from other people who have similar problems, of course.

and i too experience white boxes, uniquely on the latest builds only, seemingly every modified build, 3.5 or 6, around the time of and following the Bobbeau Environment Mapping 2 build has been somewhat unstable for me...mainly problems with INTERFACE GRAPHICS (like the mainhall and the actual Quit Game? screen!) would be disappearing, causing crashes...

...At the present time I am using JPG's and DDS's to make up for the fact that even a Radeon 9800 Pro 128MB will not put up with super high res barely-compressed TGA's in large quantities. I really hope someday someone takes the time to DDSize all the textures in FS2. Everyone who retextures anything these days puts in TGA textures at the best size vs. performance possible, but without accounting for the possibility of other custom textures adding up. All these texture remakes are costing us all big time. In a nutshell, IMHO, TGA textures absolutely suck.

 DDS is superior performance wise and unless one is trying to destroy every last drop of their graphics card's memory (especially if they are using Anisotropic Filtering and/or Antialiasing which effectively double or triple video memory requirements), TGA's are not a good idea.

Personally...I'd recommend PNG. It's not as compressed as JPG, and a lot better in quality than JPG, but its not nearly as bloated as TGA. And, the difference between a PNG and a TGA isn't really all that noticeable. Did I mention PNG's also support alpha channels JUST LIKE TGA?
Title: There is something wrong.... very wrong.
Post by: CP5670 on June 11, 2004, 03:48:48 pm
I fully agree, the tgas have almost no compression and make for unnecessarily large filesizes. I think 32-bit pcxs would be ideal, as they have some of the best lossless compression I have seen. dds has good compression but can mess up the quality on certain kinds of images.

I have not seen this white square thing at all with the original 3.6 but haven't tried Bob's decal builds (computer was completely messed up with heat problems for the last few days and I only just got it fixed). I will run one of those a bit later and see what happens.
Title: Hey Gregster2K
Post by: Star Dragon on June 11, 2004, 03:58:10 pm
I got a big problem. I've been trying to play any mod with 3.6 , even the 3.6c  to free up resources, but to no avial nomatter which mod I try (inferno, Fstrek, RT, B5) it always says MOd name can't contain spaces!!! Is there a way I can TRICK the exe into thinking it ISN'T a mod like renaming it one of the core VP files? That way it runs it regardless?

   Does anyone know WHY this inability of the launcher to allow a mod to be chosen occurs? I like the SCP build for the Original Campaign but I want to at least enjoy Omni's ships!
Title: There is something wrong.... very wrong.
Post by: Gregster2k on June 11, 2004, 04:09:40 pm
I have no idea. Check your directory names, then try possibly reducing the number of flags.

ALSO, make sure u are using launcher version 5, not 4. v5 is designed for 3.6.
Title: There is something wrong.... very wrong.
Post by: Lightspeed on June 11, 2004, 07:40:44 pm
You all realize that once loaded into your texture memory ALL file formats have exactly the same file size, right?

The only format that doesnt (and only on modern video cards) is DDS. And DDS is not precisely usable for anything with colour gradients.

But please, don't off topic this thread as these problems really ought to be looked in to and eventually be sorted out.
Title: There is something wrong.... very wrong.
Post by: Bobboau on June 11, 2004, 09:13:07 pm
some people arn't as sensitive to the gradient issue as you are, I hardly see any diference, and the performence diference can be dramatic
Title: There is something wrong.... very wrong.
Post by: CP5670 on June 11, 2004, 09:45:20 pm
I just tried some missions on the 6/10 Bob build and the framerates were always good and consistent. The external view worked fine and there were no white squares. I actually used to get the ani slowdown in pre-3.6 builds, but it stopped occurring with 3.6 and does not happen in anything more recent. I did notice the erratic thrusterglow sizes though, which was especially apparent on missiles. The glowpoints were shown too big in the tech room view, which might be related to that. It also doesn't seem to remember the last used pilot, which is a minor issue but can get annoying

This is all without the d3d_bad_tsys thing.

Quote
You all realize that once loaded into your texture memory ALL file formats have exactly the same file size, right?


I know, I am talking about the space they take up on the hard drive and possibly in downloads. It doesn't make any sense to have the vp files so large when simply changing the tga's to 32-bit pcx's cuts the vp sizes to around half without losing any image quality.
Title: There is something wrong.... very wrong.
Post by: Lightspeed on June 14, 2004, 06:02:28 pm
and you wouldnt have any textures in game, as 32-bit pcx are not supported. Yay!
Title: There is something wrong.... very wrong.
Post by: WMCoolmon on June 14, 2004, 07:06:48 pm
Do you want to force people to download large files or something? :wtf:
Title: There is something wrong.... very wrong.
Post by: CP5670 on June 14, 2004, 09:38:25 pm
Quote
and you wouldnt have any textures in game, as 32-bit pcx are not supported. Yay!


yeah...which is why I was suggesting that support be added at some point. :p I mean, the size difference is so dramatic that there really isn't any point in using the uncompressed tga format over something like 32-bit pcx.

It seems that my computer messed up again and is not going to be operational for a while (at least one of the memory dimms has died or is about to and I need to find a replacement); I was going to also test that version with the d3d_bad_tsys thing but I guess that has to wait. Maybe I could try the game on this laptop, although I bet it would look like crap with that color reduction issue I'm having.

By the way, was this place down again last night or was it just me?
Title: There is something wrong.... very wrong.
Post by: Bobboau on June 14, 2004, 10:46:12 pm
it's been down for a few days
Title: There is something wrong.... very wrong.
Post by: Lightspeed on June 15, 2004, 05:47:17 am
uhm the problems not the textures, you can use maximum quality JPGES for them at a rather good quality (i.e. no visible detail loss). Thats what I did with my shinemap zip.

The problem is the Shinemaps. Those HAVE to be TGA, as TGA is the ONLY FORMAT that allows an alpha channel.

Except of course, DDS. But as I have already mentioned, DDS is not a valid choice (for me).