Author Topic: Multi-threading  (Read 2023 times)

0 Members and 1 Guest are viewing this topic.

Offline Macfie

  • 210
  • If somebody made a campaign I've probably got it
Are there any plans to incorporate multi-threading into the game engine?  I'm finding that the capacity of the processor is limiting my ability to run the SCP with all the bells and whistles.
Normal people believe that if it isn't broke, don't fix it. Engineers believe that if it isn't broke, it doesn't have enough features yet.
The difference between Mechanical Engineers and Civil Engineers is:
Mechanical Engineers build weapons.  Civil Engineers build targets
An optimist sees the glass half full; the pessimist sees it half empty. An engineer sees that the glass is twice as big as it needs to be.

 

Online Spoon

  • 212
  • ♪ ♬ ヾ(´︶`♡)ノ ♬ 淫画
Are there any plans to incorporate multi-threading into the game engine?  I'm finding that the capacity of the processor is limiting my ability to run the SCP with all the bells and whistles.
To repeat what I've heard endlessly about this subject:
Not gonna happen, pretty much requires a rewrite of the whole engine
Urutorahappī!!

[02:42] <@Axem> spoon somethings wrong
[02:42] <@Axem> critically wrong
[02:42] <@Axem> im happy with these missions now
[02:44] <@Axem> well
[02:44] <@Axem> with 2 of them

 

Offline Kolgena

  • 211
2.4GHz on a C2D from my experience is more than enough to run this game maxed out. I am by far GPU limited at 1680x945 with a ATI 3650m HD. It's not like the game requires THAT much CPU horsepower.

 

Offline Macfie

  • 210
  • If somebody made a campaign I've probably got it
I'm running about 2.34 Ghz and hit the peak output periodically while playing on 1440 x 900.  I really haven't had a problem with my GPU
Normal people believe that if it isn't broke, don't fix it. Engineers believe that if it isn't broke, it doesn't have enough features yet.
The difference between Mechanical Engineers and Civil Engineers is:
Mechanical Engineers build weapons.  Civil Engineers build targets
An optimist sees the glass half full; the pessimist sees it half empty. An engineer sees that the glass is twice as big as it needs to be.

 

Offline Kolgena

  • 211
Hmm. You probably play much more strenuous mods on a real GPU. War in Heaven R1 is always GPU throttled for me.

 

Offline Valathil

  • ...And I would have had a custom title if it wasn't for you meddling kids!
  • 29
  • Custom Title? Wizards need no Custom Title!
Not only would it require a rewrite of the whole engine. It would also require to profile this ***** so perfectly accurate to find things that would we be able to run in parallel that it would be better to just rewrite the whole engine with multithreading in mind. So yeah you can either rewrite the whole engine or think about it REALLY hard and then rewrite the whole engine.
┏┓╋┏┓╋╋╋╋╋╋╋╋╋┏┓
┃┃╋┃┃╋╋╋╋╋╋╋╋╋┃┃
┃┃┏┫┃┏┳━━┓┏━━┓┃┗━┳━━┳━━┳━━┓
┃┃┣┫┗┛┫┃━┫┃┏┓┃┃┏┓┃┏┓┃━━┫━━┫
┃┗┫┃┏┓┫┃━┫┃┏┓┃┃┗┛┃┗┛┣━━┣━━┃
┗━┻┻┛┗┻━━┛┗┛┗┛┗━━┻━━┻━━┻━━┛

 

Offline Kolgena

  • 211
Well... seeing as the core of the engine is already hopeless spaghetti code, why not?




Jk, don't kill me.

 

Offline Valathil

  • ...And I would have had a custom title if it wasn't for you meddling kids!
  • 29
  • Custom Title? Wizards need no Custom Title!
You start ok? i look over the code once your finished
« Last Edit: July 09, 2011, 06:51:35 pm by Valathil »
┏┓╋┏┓╋╋╋╋╋╋╋╋╋┏┓
┃┃╋┃┃╋╋╋╋╋╋╋╋╋┃┃
┃┃┏┫┃┏┳━━┓┏━━┓┃┗━┳━━┳━━┳━━┓
┃┃┣┫┗┛┫┃━┫┃┏┓┃┃┏┓┃┏┓┃━━┫━━┫
┃┗┫┃┏┓┫┃━┫┃┏┓┃┃┗┛┃┗┛┣━━┣━━┃
┗━┻┻┛┗┻━━┛┗┛┗┛┗━━┻━━┻━━┻━━┛

 

Offline Zacam

  • Magnificent Bastard
  • Administrator
  • 211
  • I go Sledge-O-Matic on Spammers
    • Minecraft
    • Steam
    • Twitter
    • ModDB Feature

Well, on windows you enable processor affinity to run on more than 1 core (which is what FSO does by default)

However, mileage may vary and it's not a supported operation if you choose to under take it.

As already mentioned, we would need to have a lot more HLOMs (High-Lover Order Managers) in place that are thread safe and thread aware to co-ordinate before we could implement multi-threading. Which means re-writing a lot of stuff into being thread safe first so that it can even be managed in the first place.

That being said, there are still a few places where the engine can gain some enhancement without resorting to a multi-threading re-write project. While there may come a day when that becomes the final obstacle, we'll hopefully have the code base in a condition where the transition will take less effort than it would right now. Assuming of course that it ever becomes a constraining limitation on the performance of the engine, which at this point is still some what doubtful as there many places that still need improvement. (Collision detection and handling, Asteroid field management, just to name the biggest two).

There is also a significant question as to HOW we could approach going about adding multi-threading. For example, we can't really leverage OpenMP (which is related to Multi-Processing, not necessarily Multi-Threading but could still be something to cite as being able to give gains to the FSO Engine) because that excludes anybody that uses MSVC Express edition and that doesn't exactly help us any. Just as an example of being cautious about how to go about doing things.
« Last Edit: July 10, 2011, 12:19:50 am by Zacam »
Report MediaVP issues, now on the MediaVP Mantis! Read all about it Here!
Talk with the FSU on #SCP-FSU Talk with the SCP on #SCP
"If you can keep a level head in all this confusion, you just don't understand the situation"

¤[D+¬>

[08/01 16:53:11] <sigtau> EveningTea: I have decided that I am a 32-bit registerkin.  Pronouns are eax, ebx, ecx, edx.
[08/01 16:53:31] <EveningTea> dhauidahh
[08/01 16:53:32] <EveningTea> sak
[08/01 16:53:40] * EveningTea froths at the mouth
[08/01 16:53:40] <sigtau> i broke him, boys

 

Offline chief1983

  • Still lacks a custom title
  • Moderator
  • 212
  • ⬇️⬆️⬅️⬅️À➡️⬇️
    • Minecraft
    • Skype
    • Steam
    • Twitter
    • Fate of the Galaxy
Not to mention, there are still (supposedly) areas of the code that could be optimized to keep single-core performance sufficient for years.  Profiling needs to be done to identify those though.
Fate of the Galaxy - Now Hiring!  Apply within | Diaspora | SCP Home | Collada Importer for PCS2
Karajorma's 'How to report bugs' | Mantis
#freespace | #scp-swc | #diaspora | #SCP | #hard-light on EsperNet

"You may not sell or otherwise commercially exploit the source or things you created based on the source." -- Excerpt from FSO license, for reference

Nuclear1:  Jesus Christ zack you're a little too hamyurger for HLP right now...
iamzack:  i dont have hamynerge i just want ptatoc hips D:
redsniper:  Platonic hips?!
iamzack:  lays

 

Offline Fury

  • The Curmudgeon
  • 213
I apologize for resurrecting this old topic, but I wanted to have these old posts as they are still valid and my post is directly related.

As has been mentioned before, implementing multi-threading would require rewrite of the engine. But I just have to ask, were OpenMP and Pthreads ever considered as solutions to the problem?
http://en.wikipedia.org/wiki/OpenMP
http://en.wikipedia.org/wiki/POSIX_Threads
Both supports linux, OS X and Windows platforms and so does Visual Studio, GCC and likely all other relevant compilers.

Now, the question is whether either of those actually helps avoiding rewrite of the entire engine?

Chances are that I'm being stupid and those two were looked at and dismissed. But still, posting this anyway.

 

Offline The E

  • He's Ebeneezer Goode
  • Global Moderator
  • 213
  • Nothing personal, just tech support.
    • Skype
    • Steam
    • Twitter
They are an option.

But, and I am SURE we've mentioned this before, figuring out which multithreading framework to use (if any) takes a back seat to figuring out how to use multithreading in the first place, how to make it work for us. That's the big holdup.
**** every cause that ends in murder and children crying. ― Iain Banks
Join the fun at the HLP IRC channel. Get the latest spam and gossip as long as it's fresh!

 

Offline Tomo

  • 28
The framework doesn't matter, there's plenty of good ones.

The first thing is the simple question: Why?

Which tasks really can be done in parallel?
- Many things are reliant on the answer from other functions, and so 'multi-threading' them would just mean one core waiting on the other and might even make it slower overall.

Of those, which of those are big and slow enough for it to be worth bothering with?

Actually the most intensive ones already are done in parallel because the GPU is doing them.

To take a literal wild guess, quite likely the only two things that is likely to be worth even trying to spin off into another thread (or pile of) are the load/cache of data (so other things can continue while doing so), and calculating each frame's collisions.

Getting either of those wrong would be pretty nasty to the game, with hard-to-reproduce issues.
« Last Edit: August 06, 2013, 03:10:14 pm by Tomo »

 

Offline The E

  • He's Ebeneezer Goode
  • Global Moderator
  • 213
  • Nothing personal, just tech support.
    • Skype
    • Steam
    • Twitter
Loading level data cannot be multithreaded (As you can't multithread disc access without sacrificing performance through overhead).

Collision detection (Well, physics as a whole, really) can be multithreaded, and would be the area we're looking at. We're already kinda-sorta offloading our audio processing into a separate thread as well through OpenAL. Our game logic (AI/Sexps/Scripting) isn't heavy enough to warrant going into multithread territory.
**** every cause that ends in murder and children crying. ― Iain Banks
Join the fun at the HLP IRC channel. Get the latest spam and gossip as long as it's fresh!

 

Offline Tomo

  • 28
I didn't mean multi-threading the actual disk access. That generally doesn't help.

I meant spinning off disk access into its own thread, allowing the rest of the application to continue doing other stuff (where possible) instead of sitting there waiting for the disk to come back.

I recently did that to a large data-processing application (it processes a stack of XML files that are approx. 200MB when zipped) and found that it scrubbed about 30sec off a 1 min 45 sec loading time.
Most of the saving came from ensuring the loading processing didn't stall while waiting for the disk, as whenever "need file Y" occurs, processing earlier file X could continue while Y was being pulled off the disk.

However, it's a relatively small gain (an earlier optimisation knocked 45 minutes off!) and in the case of FSO, would only have any effect in cases where other processing can continue before the file's loaded - I suspect that's pretty much only cache generation.

 

Offline karajorma

  • King Louie - Jungle VIP
  • Administrator
  • 214
    • Karajorma's Freespace FAQ
Well loading times could use some improvement, especially on debug. I actually had to stop coding for multiplayer on my old PC cause the time it took to actually load the mission was so long the other PC would time out.
Karajorma's Freespace FAQ. It's almost like asking me yourself.

[ Diaspora ] - [ Seeds Of Rebellion ] - [ Mind Games ]

 

Offline The E

  • He's Ebeneezer Goode
  • Global Moderator
  • 213
  • Nothing personal, just tech support.
    • Skype
    • Steam
    • Twitter
Then that's an area that we need to profile. I have no clue where the slowdown would be in that area (aside from stuff we cannot control, like disc response times and data rates etc).

At any rate, that would also be an area where major restructuring would be unavoidable.
**** every cause that ends in murder and children crying. ― Iain Banks
Join the fun at the HLP IRC channel. Get the latest spam and gossip as long as it's fresh!

  

Offline karajorma

  • King Louie - Jungle VIP
  • Administrator
  • 214
    • Karajorma's Freespace FAQ
Bizarrely it's in the generation of icons. For some reason it was taking upwards of a minute in Diaspora builds I was using. Admittedly I had a very slow PC but still.....
Karajorma's Freespace FAQ. It's almost like asking me yourself.

[ Diaspora ] - [ Seeds Of Rebellion ] - [ Mind Games ]