[Aardwolf]

I'm making a first-person shooter where you fight giant space bugs!



But it's too slow



The problem is primarily the AI and the physics. I did a simple experiment, recording the framerates with and without each of these two features.

frames per second		AI off		AI on
Physics off		~30		~15
Physics on		12-14		4-7

I've actually made some changes that make the AI faster since I recorded those data, but you get the gist.




Of course, this only tells me that the physics is taking a lot of my time; it doesn't tell me that I could do it any better than Bullet could, or that I'm even using Bullet as efficiently as I could be. Still, there are a few issues I have with Bullet...

Ragdolls. It's not reflected in the above statistics because I refrained from killing any enemies for that experiment, but just a handful of ragdolls can cause a considerable performance hit. I wish there were a way I could tell Bullet "all of these rigid bodies are parts of a whole, and that whole will never be bigger than some size; thus, you should only check for collisions between the parts of two separate ragdolls if they are near one another" ... But afaik there is no easy way to do it (there are ways, but they aren't easy). Unfortunately, all of my past attempts at implementing ragdoll physics have been miserable failures 

Terrain. I don't know if Bullet is putting the triangles of the terrain mesh into some sort of spatial partitioning tree, but it would be profoundly stupid of it not to do so. I recently wrote a system for destructible terrain, and I want to be able to integrate that into this game engine, but the polygon counts this thing produces are much higher than what I have currently. However, the triangle data is already conveniently partitioned into cubic chunks. It would make sense to me to keep track of which physics objects (or game entities) are in which chunks, and consult that to figure out which objects need collision detection... maybe Bullet has something I can use for that, but making it match up with my destructible terrain system may prove difficult.




So yeah, suggestions?

[The E]

Okay, stupid question first, have you run a profiler on your program to see where the hotspots in the AI and physics code are?

[Aardwolf]

Yes, I have run a profiler (AMD CodeAnalyst)... however, I am thinking about switching to another one, because some of the info I'm getting from this one is bull****.

But anyway, here's the top entries (sorted by "total")...

Name		Self	Children	Total
btGjkPairDetector::getClosestPointsNonVirtual		37	2062	2099
btMinkowskiPenetrationDepthSolver::getPenetrationDirections		48	1861	1909
btDiscreteDynamicsWorld::debugDrawConstraint		3	1688	1691
btCompoundCollisionAlgorithm::CreateFunc::CreateCollisionAlgorithm		3	1000	1003

I'm not sure how I managed to not notice it previously, but that third one is quite peculiar, as I never used the debug drawer while I was recording this stuff.

I'm going to double-check that the version of Bullet I'm static-linking with is not a debug build, since lower on the list (10th highest as far as "total") is _wassert, which shouldn't be getting called in a release build 

Then again, maybe that is related to the bull**** I mentioned I'm getting from AMD CodeAnalyst... for example, it says that CibraryEngine::SkinnedCharacter::UpdatePoses (one of my functions) is being called by __GLeeLink_GL_VERSION_1_2, and that it's calling __GLeeLink_GL_VERSION_1_3... which is complete and utter bull****. The function in question has nothing to do with OpenGL. Similar bull**** exists elsewhere. 

[The E]

Which version of Visual Studio are you using (if any)? If yes, have you tried the VS profiler?

Another question would be if you are building with all the optimizations switched on, especially the SSE2 options.
Also, which OpenGL version are you developing against? Are you using all the nice little go-faster switches, like Vertex Buffer Objects, non-immediate mode rendering and all that stuff?

[Aardwolf]

It turns out it was a debug version of Bullet. I tried again with a release build and it's much faster.

I am using MSVC++ 2010 Express; no I have not tried the VS profiler.

I have not tried SSE2.

OpenGL version... it's a hodge-podge  I know I ought to pick one and stop using the deprecated features, but I haven't done it. Yes, I'm using VBOs for most things.

[Aardwolf]

Now for another issue with my game engine! It sometimes likes to crash just after loading completes (or just before the game starts, I'm not entirely sure which).

I have occasionally managed to catch this when running a debugger, and the crash is in lua5.1.dll ... however, the call stack is rubbish; the only stack frames it contains are from lua5.1.dll! I don't know what to make of this.

I have a hunch, that maybe it has something to do with my loading screen being multi-threaded?

[BritishShivans]

Those "bugs" look like Shivans. Are you sure they're not baby Sathanases in disguise? 

[The E]

Hang on. A multi-threaded loading screen? Somehow, that strikes me as a somewhat bad idea, given that most loading involves HDD access, and given that that is usually among the slowest things possible, and not really suitable for multithreading...

[Nuke]

i bet hes just running his gui in a thread and loading data in another.

[Aardwolf]

i bet hes just running his gui in a thread and loading data in another.

Precisely! It lets me have a cursor which moves around (kinda pointless, but it makes it obvious the game hasn't frozen), and it lets me press Esc to abort mid-load (although it still has to finish whatever operation it was on).

I had to jump through some hoops to make it not fubar the OpenGL stuff... so it still pauses at the very end when it's time to call the OpenGL stuff. But it's better than nothing.

[The E]

Ah, Ok. That does sound more sensible indeed.

[Aardwolf]

Bleh. Inconsistent crashes are annoying, because you can't tell if they're fixed. And this one is particularly annoying, because the few times I've managed to catch it when I'm running a debug build, the only thing on the call stack has been lua dll stuff

I don't know what to do about this!

[Nuke]

this is where id load my code with printf()s to see exactly where its happening at.

[Aardwolf]

I suppose I'll have to.

Still, it will be a bit iffy, because there are two threads. And for all I know it could be in a destructor or something (which would make it hard to debug that way).



Edit: Much to my amazement, after some tweakage of my project settings, this error is now giving me a more meaningful call stack!

Code: [Select]

lua5.1.dll!00101212()
[Frames below may be incorrect and/or missing, no symbols loaded for lua5.1.dll]
lua5.1.dll!00113aab()
lua5.1.dll!00114fd1()
lua5.1.dll!001058f2()
lua5.1.dll!0011192d()
msvcr80.dll!63874d83()
Test.exe!_nh_malloc_dbg_impl(unsigned int nSize, int nhFlag, int nBlockUse, const char * szFileName, int nLine, int * errno_tmp)  Line 239 + 0x19 bytes C++
Test.exe!_nh_malloc_dbg(unsigned int nSize, int nhFlag, int nBlockUse, const char * szFileName, int nLine)  Line 302 + 0x1d bytes C++
Test.exe!malloc(unsigned int nSize)  Line 56 + 0x15 bytes C++
Test.exe!operator new(unsigned int size)  Line 59 + 0x9 bytes C++
Test.exe!std::basic_string<char,std::char_traits<char>,std::allocator<char> >::_Copy(unsigned int _Newsize, unsigned int _Oldlen)  Line 1932 + 0x15 bytes C++
00fc63e8()
Test.exe!boost::`anonymous namespace'::create_current_thread_tls_key()  + 0xb bytes C++
Test.exe!boost::call_once<void (*)(void)>()  + 0xe9 bytes C++
Test.exe!boost::detail::thread_data<Test::TestGame::Loader>::run()  Line 62 C++
Test.exe!boost::`anonymous namespace'::thread_start_function()  + 0x63 bytes C++
Test.exe!_callthreadstartex()  Line 314 + 0xf bytes C
Test.exe!_threadstartex(void * ptd)  Line 297 C
kernel32.dll!763e3677()
ntdll.dll!770b9f42()
ntdll.dll!770b9f15()

Why is malloc calling lua? And what's that msvcr80.dll doing in there? 

[Aardwolf]

Gameplay video if anyone is interested:

http://youtu.be/847Vlwl2eBc

[BritishShivans]

I'm surprised you didn't see my post asking if the enemies were baby Sathanases in disguise. Too busy?

[Aardwolf]

Eh... I saw it, but it seemed too silly to be worth replying to  But since I'm less frustrated with my game not working now...

Shivans only have 5 legs, and they're in a weirder arrangement...

At one point I made an alien that looked more a Sathanas x Zergling (or according to some people, Sathanas x Bunny), but these bugs just look like crabs 

[Aardwolf]

I'm still occasionally running into this crash... can anyone help?

Code: [Select]

lua5.1.dll!00251212()
[Frames below may be incorrect and/or missing, no symbols loaded for lua5.1.dll]
lua5.1.dll!00263aab()
lua5.1.dll!00264fd1()
lua5.1.dll!002558f2()
lua5.1.dll!0026192d()
msvcr80.dll!74cb4d83()
Test.exe!_threadstartex(void * ptd)  Line 292 + 0x5 bytes
kernel32.dll!773b339a()
ntdll.dll!77e59ef2()
ntdll.dll!77e59ec5()
The one stack frame that it has source for is the _threadstartex function in threadex.c

I'm still not sure why it's doing anything with msvcr80.dll, as I've tried to get rid of that dependency in as many places as I could think of...

Edit: A little more info, here's MSVC++ 2010's debugger's output. It's not the output that goes with the above stack trace, this is from a successful play of the game.

Code: [Select]

'Test.exe': Loaded 'C:\Programming\C++\MSVCCibraryEngine\Release\Test.exe', Symbols loaded.
'Test.exe': Loaded 'C:\Windows\SysWOW64\ntdll.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\kernel32.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\KernelBase.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Program Files\Alwil Software\Avast5\snxhk.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\opengl32.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\msvcrt.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\advapi32.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\sechost.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\rpcrt4.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\sspicli.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\cryptbase.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\gdi32.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\user32.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\lpk.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\usp10.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\glu32.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\ddraw.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\dciman32.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\setupapi.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\cfgmgr32.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\oleaut32.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\ole32.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\devobj.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\dwmapi.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Program Files (x86)\Lua\5.1\lua51.dll', Binary was not built with debug information.
'Test.exe': Loaded 'C:\Program Files (x86)\Lua\5.1\lua5.1.dll', Binary was not built with debug information.
'Test.exe': Loaded 'C:\Windows\winsxs\x86_microsoft.vc80.crt_1fc8b3b9a1e18e3b_8.0.50727.6195_none_d09154e044272b9a\msvcr80.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\ws2_32.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\nsi.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\OpenAL32.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\winmm.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\imm32.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\msctf.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\atioglxx.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\version.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\atiadlxy.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\wintrust.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\crypt32.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\msasn1.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\MMDevAPI.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\propsys.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\wdmaud.drv', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\ksuser.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\avrt.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\shlwapi.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\AudioSes.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\msacm32.drv', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\msacm32.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\midimap.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\wrap_oal.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\dsound.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\powrprof.dll', Cannot find or open the PDB file
'Test.exe': Loaded 'C:\Windows\SysWOW64\clbcatq.dll', Cannot find or open the PDB file
The thread 'Win32 Thread' (0x11e4) has exited with code 0 (0x0).
The thread 'Win32 Thread' (0xfb0) has exited with code 0 (0x0).
The thread 'Win32 Thread' (0x768) has exited with code 0 (0x0).
The thread 'Win32 Thread' (0x10dc) has exited with code 0 (0x0).
The thread 'Win32 Thread' (0x12c8) has exited with code 0 (0x0).
The thread 'Win32 Thread' (0xc78) has exited with code 0 (0x0).
'Test.exe': Unloaded 'C:\Windows\SysWOW64\wrap_oal.dll'
'Test.exe': Loaded 'C:\Windows\SysWOW64\wrap_oal.dll', Cannot find or open the PDB file
The thread 'Win32 Thread' (0x12c0) has exited with code 0 (0x0).
WARNING: 0:19: implicit cast from int to float
WARNING: 0:20: implicit cast from int to float
WARNING: 0:21: implicit cast from int to float
WARNING: 0:19: implicit cast from int to float
WARNING: 0:20: implicit cast from int to float
WARNING: 0:21: implicit cast from int to float
WARNING: built-in varying gl_TexCoord [3] defined in Vertex shader and not used in Fragment shader
WARNING: built-in varying gl_TexCoord [4] defined in Vertex shader and not used in Fragment shader
WARNING: 0:19: implicit cast from int to float
WARNING: 0:20: implicit cast from int to float
WARNING: 0:21: implicit cast from int to float
WARNING: built-in varying gl_TexCoord [3] defined in Vertex shader and not used in Fragment shader
WARNING: built-in varying gl_TexCoord [4] defined in Vertex shader and not used in Fragment shader
WARNING: built-in varying gl_TexCoord [0] defined in Vertex shader and not used in Fragment shader
WARNING: built-in varying gl_TexCoord [1] defined in Vertex shader and not used in Fragment shader
WARNING: built-in varying gl_TexCoord [2] defined in Vertex shader and not used in Fragment shader
WARNING: built-in varying gl_TexCoord [3] defined in Vertex shader and not used in Fragment shader
WARNING: built-in varying gl_TexCoord [4] defined in Vertex shader and not used in Fragment shader
WARNING: built-in varying gl_TexCoord [0] defined in Vertex shader and not used in Fragment shader
WARNING: built-in varying gl_TexCoord [1] defined in Vertex shader and not used in Fragment shader
WARNING: built-in varying gl_TexCoord [2] defined in Vertex shader and not used in Fragment shader
WARNING: built-in varying gl_TexCoord [3] defined in Vertex shader and not used in Fragment shader
WARNING: built-in varying gl_TexCoord [4] defined in Vertex shader and not used in Fragment shader
The thread 'Win32 Thread' (0xbdc) has exited with code 0 (0x0).
The thread 'Win32 Thread' (0x1028) has exited with code 0 (0x0).
The thread 'Win32 Thread' (0x1308) has exited with code 0 (0x0).
The thread 'Win32 Thread' (0x1068) has exited with code 0 (0x0).
The thread 'Win32 Thread' (0xfe4) has exited with code 0 (0x0).
The thread 'Win32 Thread' (0x10c0) has exited with code 0 (0x0).
The thread 'Win32 Thread' (0x1344) has exited with code 0 (0x0).
The thread 'Win32 Thread' (0x13e0) has exited with code 0 (0x0).
The thread 'Win32 Thread' (0xca8) has exited with code 0 (0x0).
The thread 'Win32 Thread' (0x1020) has exited with code 0 (0x0).
The program '[5108] Test.exe: Native' has exited with code 0 (0x0).
Ignore the OpenGL warnings, and don't worry too much about C:\Program Files\Alwil Software\Avast5\snxhk.dll - that's just Avast antivirus.

One of the things that I'm concerned about is C:\Windows\winsxs\x86_microsoft.vc80.crt_1fc8b3b9a1e18e3b_8.0.50727.6195_none_d09154e044272b9a\msvcr80.dll ... it's right after the lua DLLs (maybe they are what's loading it?), and the directory it's in has got me seriously -ing.

[The E]

Welcome to WinSxS, MS' attempt to alleviate DLL hell. If you do not know what it is, let me explain. Assume you have a program that is compiled against a specific version of a system library, in this case, msvcr80.dll. The problem is that MS will update these libraries periodically. If your program depended on a given behaviour in that version of msvcr80.dll that was around when it was compiled, and that behaviour was changed in the new version, then obviously the behaviour of your program will be affected. To avoid this, the MSVC compiler embeds a manifest in your exe, telling the DLL loading routines which version of the dll to load. In order to keep multiple versions of the same dll around, all of these varying versions are stored in the WinSxS structure (SxS standing for Side-by-Side).

So yeah. Nothing to worry about, just MS protecting you from stupid errors.

As for the crash, well, unless you can find a lua dll that has debug info, there's really nothing you can do. AMong other things, this is why FSO's Windows builds have the lua library statically linked into the exe.

[Aardwolf]

Y'know, I bet if I had googled "WinSxS" I could have found that out. But instead I googled the name of the DLL file, and just found a bunch of error reports for WoW.

I've had the error occur another time today, and this time the call stack is slightly different!

Code: [Select]

lua5.1.dll!00101212()
[Frames below may be incorrect and/or missing, no symbols loaded for lua5.1.dll]
lua5.1.dll!00113aab()
lua5.1.dll!00114fd1()
lua5.1.dll!001058f2()
lua5.1.dll!0011192d()
Test.exe!_read_nolock(int fh, void * inputbuf, unsigned int cnt)  Line 230 + 0x15 bytes
msvcr80.dll!65424d83()
Test.exe!_threadstartex(void * ptd)  Line 292 + 0x5 bytes
kernel32.dll!773b339a()
ntdll.dll!77e59ef2()
ntdll.dll!77e59ec5()
_read_nolock is in read.c

Something to do with reading a file, and multithreading. Bleh. 




Edit: Changed something. I won't see if it's fixed, but I'll see if it isn't. 

I think the problem was that I was using bools to communicate between threads whether the load had finished, and I should have known better. So now I made those bools private and I'm using getters and setters which do mutex locks.



Edit II: Wow, I should've at least tested it once before I said "maybe fixed". With the mutex locks it now crashes consistently.