Hard Light Productions Forums
Modding, Mission Design, and Coding => FS2 Open Coding - The Source Code Project (SCP) => Topic started by: jg18 on July 08, 2014, 01:38:43 am
-
Hey guys,
I'm back and continuing my work for the SCP. Personal details are in WHIYL (http://www.hard-light.net/forums/index.php?topic=53623.msg1755743#msg1755743) if you're interested.
I'm now looking into the sound code for possible improvements, and I'd like to get the community's take on
(1) bugs/problems with the current sound code
(2) features that the sound code should support but currently doesn't. Please justify all features you mention, meaning please don't just say "It would be cool if X were possible" without explaining why.
Thanks!
-
I think it'd be really cool to allow the number of concurrent channels to be set by the user (based on the assumption that autodetecting this seems hard (http://scp.indiegames.us/mantis/view.php?id=2680)). This would allow those with the support for it to have really rich & dense soundscapes. For instance, with taylors old code I set the max OpenALSoft channels to be 256, and it sounded glorious :)
On a more mundane note, I've noticed with newer versions of OpenALSoft, this warning is being spewed to STDOUT.
AL lib: alcOpenDevice: Option 'format' is deprecated, please use 'channels' and 'sample-type'
This would be good to fix prior to the format option disappearing completely (although it may require some version specific hackery in order to work with both OpenAL and OpenALSoft).
Lastly, it'd be really good to know exactly why OpenAL & OpenALSoft work for some people and not others. i.e. it's hard to recommend just one of them for everyone, which leads to (maybe) version specific hackery as mentioned above.
-
There is a way to get the number of supported mono sources (http://stackoverflow.com/questions/2871905/openal-determine-maximum-sources) which is better than nothing.
I tested it with OpenALSoft and it reported 255, maybe that could be used for determining the maximum number of sources.
On a more mundane note, I've noticed with newer versions of OpenALSoft, this warning is being spewed to STDOUT.
AL lib: alcOpenDevice: Option 'format' is deprecated, please use 'channels' and 'sample-type'
This would be good to fix prior to the format option disappearing completely (although it may require some version specific hackery in order to work with both OpenAL and OpenALSoft).
From looking at the OpenALSoft source code this seems to be a configuration file issue and not an issue caused by FSO.
-
There is a way to get the number of supported mono sources (http://stackoverflow.com/questions/2871905/openal-determine-maximum-sources) which is better than nothing.
I tested it with OpenALSoft and it reported 255, maybe that could be used for determining the maximum number of sources.
That looks decent, as you probably saw from the mantis ticket the idea of "create them until it fails" doesn't seem to work at all.
On a more mundane note, I've noticed with newer versions of OpenALSoft, this warning is being spewed to STDOUT.
AL lib: alcOpenDevice: Option 'format' is deprecated, please use 'channels' and 'sample-type'
This would be good to fix prior to the format option disappearing completely (although it may require some version specific hackery in order to work with both OpenAL and OpenALSoft).
From looking at the OpenALSoft source code this seems to be a configuration file issue and not an issue caused by FSO.
:nervous: Thanks, and removing "format" from ~/.alsoftrc has indeed has fixed the "issue".
-
Something that might be worth looking into is the horrible soundclipping that occures when multiple of the same sounds play at the same time. For example, listen to the unholy noise your speakers will produce when you're right next to 6 fighters warping in at the same time.
I'm aware this is mostly caused by hardware limitations, onboard motherboard soundcards etc. But this was something that didn't happen as badly with the old soundcode, I think mostly because a lot of concurrent sounds got pruned back in retail.
(unless I somehow completely missed this being fixed in the mean time, in that case never mind me)
-
Is there a way to use a chorus effect type thing instead of just overlapping sounds?
e.g., instead of just playing 6 of the same sound, play one sound with a chorus effect set for 6?
No I don't know if that's possible, but it seems to make sense to me.
-
There's something I'd like to see for fast-firing weapons. It'd be nice if it was possible to define a sound that would loop when a weapon is being fired, and be created once, instead of for all individual guns (maybe with number of guns defining the volume).
The reason is that when multiple fast guns are firing in unison, it doesn't sound good, and can sometimes "saturate" the sound system completely. That way, just a single sound would be played per ship, despite there being 10-50 shots per second. That could also make the gun sound faster than it actually is, which could be useful, too.
-
How about smooth transition between sound channels? Prime example of this would be when you have a beam weapon firing past you to your left, its sound is coming from left channel. When you begin turning past the beam, it suddenly switches to right channel. There is no transition, its either one or the other and that can be jarring. Alternative to a beam weapon would be jump-in or jump-out sound, or basically anything that keeps looping for prolonged time.
-
How about smooth transition between sound channels? Prime example of this would be when you have a beam weapon firing past you to your left, its sound is coming from left channel. When you begin turning past the beam, it suddenly switches to right channel. There is no transition, its either one or the other and that can be jarring. Alternative to a beam weapon would be jump-in or jump-out sound, or basically anything that keeps looping for prolonged time.
I think that was a bug and has been fixed in trunk.
-
How about smooth transition between sound channels? Prime example of this would be when you have a beam weapon firing past you to your left, its sound is coming from left channel. When you begin turning past the beam, it suddenly switches to right channel. There is no transition, its either one or the other and that can be jarring.
That bug has been fixed (http://scp.indiegames.us/mantis/view.php?id=2725) since r10798.
Alternative to a beam weapon would be jump-in or jump-out sound, or basically anything that keeps looping for prolonged time.
I'm not aware of any similar problem with warp sounds...
-
Oh well, haven't been playing FSO at all past year. Good to see stuff gets fixed. :p
-
My biggest issue with the sound code right now is that audio playback is weirdly tied to framerate somehow. What I mean is that low FPS causes sound stuttering. I've only ever noticed this in FSO. Any other games I play on PC don't have audio issues when my framerate decreases or becomes unstable.
-
Really? I've seen something like that happen in multiple games, though usually when my framerate hits dire levels, like 5 or less.
-
I can't remember seeing FPS that low on other games. Either way, should FSO really be doing it when the rate drops from 60 to 40?
-
That certainly is oddball, but I don't believe I've seen it occur, e.g. BP Artemis station goes from 60-20 fps for me, and the sound is smooth throughout.
Or on other words, what's your sound setup/environment? (I've got Linux, SB Audigy, OpenALSoft 1.14, Headphones)
-
Win 7, SoundBlaster Recon3D, OpenALSoft
In fact, I went and bought a dedicated card to get off my mobo onboard card specifically to try and remedy the FSO sound issues. It wasn't until I searched high and low and learned about the OpenALSoft config file that I was able to mitigate it a little.
-
Thanks, guys! This also helps give me the extra motivation needed to work on this.
Between wrapping my head around the current sound code and getting up to speed on OpenAL and game audio, all on top of RL issues, there likely won't be anything ready to test for months.
FWIW here's my development setup:
- Onboard audio (Realtek ALC898 on Gigabyte GA-Z77X-UD5H)
- Discrete audio (Sound Blaster Z)
- Headphones (JVC HA-RX 900)
niffiwan/m!m: just changing MAX_CHANNELS won't do much with the current sound code because of the engine's per-sound concurrency limits (see Iss Mneur's comments in Mantis 2266 about his fix for it). Maybe changing those limits would help (see my posts in the internal).
Also, I wish I knew why taylor decided that 48 sources should be the minimum (why MAX_CHANNELS + 16?), setting aside the problem that his test doesn't work.
Spoon: I believe this should be fixed.
jr2: I'm not sure what you mean by chorus effect; could you elaborate?
Dragon: Interesting, thanks for mentioning it.
Mjn: Hmm, once I'm up to speed, I'll have to see if I can repro that.
Also, I've heard from Battuta that explosions are loud and clippy, although apparently he's out for a month, so I won't be getting his opinion anytime soon. :sigh:
-
A chorus effect basically gives a sound multiple 'voices', as if they were a singing chorus.
I did a quick search using the term
Chorus effect
And this seems to explain it:
http://testtone.com/fundamentals/what-chorus-effect
What is a Chorus Effect?
A chorus (or ensemble) is a modulation effect used to create a richer, thicker sound and add subtle movement. The effect roughly simulates the slight variations in pitch and timing that occur when multiple performers sing or play the same part.
Chorus is a common guitar pedal effect that gives a clean electric guitar a "dreamy" quality. It's also widely used on acoustic guitar, electric piano, and clavinet. On strings and synth pads, chorus creates a richer, more complex sound. Stereo chorus effects also can be used to widen a stereo image.
How does it work?
The chorus effect is based upon a short delay. Incoming audio is split and run through the delay, then mixed with the original audio and sent to the effect's output. With short delay times (20 to 50ms), the delayed audio blends with the original audio instead of creating a distinct echo.
To add movement, chorus effects slowly modulate the delay time with a low-frequency oscillator (LFO). As the LFO cycles the delay time up and down, the delayed audio shifts up and down in pitch by a little bit.
Principal controls
Apple's Chorus for GarageBand
Apple's Chorus for GarageBand
GarageBand's plugin is a typical
single-voice stereo chorus. Its controls
adjust the speed and depth of the LFO,
and the wet/dry mix is fixed at 50%. Left
and right stereo channels are modulated
180 degrees out of phase to create a
wide stereo image.
Most chorus effects include knobs to adjust the LFO speed (a.k.a. rate or period) and depth (a.k.a. amplitude or intensity). LFO speeds are usually in the range of natural human vibrato (up to about 10 Hertz).
Some plugins include control of the wet/dry mix. At 100% wet, the pitch modulation of a chorus effect sounds like vibrato. To blend the modulated audio with the original, the wet/dry mix is often set at 50% of each.
Common variations
Mono, stereo, and surround chorus
A mono chorus operates on a mono input, or sums its inputs to mono before running them through a single modulated delay. The output of the effect may be stereo. Guitar pedals are usually mono effects generating stereo outputs.
A stereo chorus applies the effect to left and right stereo inputs, or to a duplicated mono input, using two independent delays. The two delays may share the same LFO, but the delays are often modulated 180 degrees out of phase. This makes the left channel cycle up while the right channel cycles down, and vice versa, creating a richer chorus and a wider stereo image.
A surround chorus applies the effect to each surround channel using independent delays. Like a stereo chorus, the surround chorus may use the same LFO for all of the delays, but modulate the delays out of phase to create a wider surround image.
Single and multi-voice chorus
Apple's Ensemble for Logic
Apple's Ensemble for Logic
Logic's plugin is a multi-voice stereo and
surround chorus that uses a single delay
modulated by three summed LFOs. LFO
phase offsets for the stereo or surround
outputs may be adjusted to narrow or
widen the stereo/surround image.
A single voice chorus uses a single delay that creates a single modulated duplicate of the incoming audio. Basic chorus effects and inexpensive guitar pedals are often single-voice.
A multiple voice chorus uses multiple modulated delays to create a richer sound with more movement. Some chorus effects use the same LFO to modulate all of the delays in sync, but at different points in the LFO's cycle. Other effects use multiple LFOs to modulate the delays independently. The latter creates a richer, less obviously cyclical effect, but with added complexity in the user interface and a higher CPU use.
Complex LFOs
A single LFO chorus can create a noticeable up-and-down pitch wobble. To make the wobble less obvious, a plugin may use multiple LFOs summed together. When each LFO has a different speed and depth, the resulting modulation waveform is complex and the wobble less obvious.
Uncommon variations
Reverb chorus
Audio Damage's Vapor
Audio Damage's Vapor
Audio Damage's plugin uses a diffuse
reverb algorithm instead of a delay. Plugin
controls adjust the LFO speed and depth,
the delay time, the wet/dry mix, and the
amount of reverb diffusion. A low-cut
filter removes low frequencies that can
muddy the diffusion.
Some chorus effects replace the delay with a diffuse reverb tail algorithm. Instead of a single echo to double the sound, reverb creates a diffuse scattering of echos that are pitch shifted up and down randomly.
Delay time and feedback
Some multi-effect plugins include controls to adjust the delay time and delay feedback. Neither of these are strictly necessary for chorusing, which always uses a short delay time and little or no feedback. The additional knobs allow the plugin to perform a wider range of effects. For instance, a short delay time and high feedback creates flanging. A deep LFO and no delay creates vibrato.
Related effects
Unison mode on a synthesizer creates a bigger sound by playing multiple slightly detuned notes each time a keyboard key is pressed. The effect can be similar to chorus.
Flanging is the same as chorus except that it uses a shorter delay time and delay feedback. This creates a distinctive "zipper" noise as the effect's LFO modulates the delay time back and forth.
Phasing is similar to chorus except that it uses multiple all-pass filters instead of a delay. When the filtered audio is mixed with the original audio, it creates comb filtering effects. When modulated, phasing creates a "swirling" feel to the sound.
Further reading
Chorus effect (http://en.wikipedia.org/wiki/Chorus_effect) at Wikipedia. A brief explanation and links.
Chorus effect (http://en.wikiaudio.org/Chorus_effect) at WikiAudio. A brief explanation.
Mixing Audio (http://www.mixingaudio.com/) by Roey Izhaki. An excellent book on audio technology, including chorus effects.
Now, the question is, is that something that would work for multiple simultaneous sound events instead of what we get now.
-
I might be wrong of course, but I don't see how chorus would allow you to decrease the number of sounds. If several sounds start playing at the same time, then sure, it could work, but not if they start at different times, which would practically always be the case.
Effects like that would be useful in other ways, though, if their parameters could be adjusted in real-time by some in-game conditions. For example, the sound emitted by a beam could get gradually chorused if you get very close to it, or the sound of your engines could be pitch-shifted based on how much thrust you're using.
-
niffiwan/m!m: just changing MAX_CHANNELS won't do much with the current sound code because of the engine's per-sound concurrency limits (see Iss Mneur's comments in Mantis 2266 about his fix for it). Maybe changing those limits would help (see my posts in the internal).
Indeed. Something like dynamically adjusting the current per-sound concurrency limits based on the total number of sounds available would be very cool, at the same time avoiding clipping issue that Spoon mentioned.
-
Something like dynamically adjusting the current per-sound concurrency limits based on the total number of sounds available would be very cool, at the same time avoiding clipping issue that Spoon mentioned.
Ah, instead of hardcoded limits. I wonder if that's the kind of thing Jason Gregory covers in the new edition of his game engine programming book (http://www.amazon.com/Game-Engine-Architecture-Second-Edition/dp/1466560010/). Now I really want to see what he says. Even if a complete version of his description is not feasible for integrating into FSO, just borrowing a few ideas might greatly improve how FSO games sound.
Ergh, it was supposed to release on July 18, but now it's not until August 4? What gives?! :banghead:
- New chapter on audio technology covering the fundamentals of the physics, mathematics, and technology that go into creating an AAA game audio engine
EDIT: Jason was one of the lead programmers on The Last Of Us, so you can trust that he knows what he's talking about. He gave a great talk at this year's GDC on the games' character dialogue system.
-
can I throw doppler effects into this mix?
-
Many video players have clipping detection that drops/normalizes the volume of all audio temporarily to prevent distortion. That might be more useful than chorus effect to reduce the ugliness of 15 fighters warping in or lots of things exploding at once.
Does anyone know how SC2 handles sound code? You can set 128 audio channels and have a giant ball of infantry shooting at once without clipping issues.
-
can I throw doppler effects into this mix?
Also: Sound morphing based on distance. HL2 has an interesting system where it uses stereo soundfiles, where hte left channel is the "close" sound and the right channel is the "far" sound, and then mixes them as a mono source based on distance. Alternatively, simply allow different sound files that work similar to model lods.
Sure, there is no sound in space, but I think it still would be a great boon to the immersion. Also, for mods that actually take place in athmospheric conditions (sound delay based on distance too, since even the most realistic games often forget about this - although I recognize that it may be *bloody hard*).
-
Thanks, guys! Keep it coming.
Bobboau: Not sure how Doppler was used in retail, but I'm not sure where it would be logical to use it. Flyby sounds? Weapons fire (projectiles moving past you)?
Kolgena: No idea, sorry.
-Joshua-: Can you give an example or two of where distancebased sound morphing is used in HL2?
As for modifying the speed of sound and such, I wonder if this is where the EFX/EAX 2.0 sound environment presets that taylor added in the 3.6.12 sound code could be of use. My vague impression is that they currently don't work, but I really don't know. I'm not sure how much documentation there is on them, either.
More generally, It'd be good to look more into what OpenAL's EFX can do for us. I'll have to add that to the TODO list.
I would think the general rule for sound modifiers or any changes to the existing sound code would be
(1) does it sound good/better than before? and ideally also
(2) does it provide additional useful feedback to the player?
I suppose that immersion falls under both questions to some extent.
Also, Mjn, about sound stuttering when the frame rate drops, does that apply to all sounds? As in sound effects, voiceover, music? Can you check if it applies to sounds/music that are played via SEXP, e.g. play-sound-from-file?
-
Doppler should be used for everything, but maybe have a weight added to the formula to allow for scaling the effect. For standard FreeSpace flight speeds this wouldn't be much of a deal. However, for the faster mods such as DE and FringeSpace the effect would become unbearably high or low pitched quickly. (The weight would be a property of each sound byte).
Normalization means take the loudest sound that's currently playing, make that 100% volume, and then scale all other sounds down relative to it. It's more or less simple multiplacation and division.
Speed of sound is not difficult to simulate, but it may conflict with other effects, which may be why its not used as widespread as it should be.
-
When taylor first added the new sound code, there was doppler shifting applied to everything, and man was it distracting and annoying. It might have been ok if it was only weapon flyby effects, but the doppler shifting due to things like ships warping in just made everything sound weird. If doppler shifting does go back in, I'd really have to say just make it a sound entry option for mods that want it.
-
Yeah TBH I'm pretty skeptical that Doppler will sound good or provide useful information to the player (my two criteria from before). except for maybe if just applied to flyby/weapons fire (even beams?)
FWIW I think the speed of sound is a variable you can control through the OpenAL API, but I'm not sure if it's intended to be adjusted on the fly.
The attenuation model, which IIRC controls how sounds fade over distance (sorry it's late and I'm too tired to fact check right now), might be something worth making moddable if there's enough interest. I forget which model FSO uses, but I do remember seeing it in the code somewhere.
EDIT: Clarification.
-
can I throw doppler effects into this mix?
Also: Sound morphing based on distance. HL2 has an interesting system where it uses stereo soundfiles, where hte left channel is the "close" sound and the right channel is the "far" sound, and then mixes them as a mono source based on distance. Alternatively, simply allow different sound files that work similar to model lods.
Sure, there is no sound in space, but I think it still would be a great boon to the immersion. Also, for mods that actually take place in athmospheric conditions (sound delay based on distance too, since even the most realistic games often forget about this - although I recognize that it may be *bloody hard*).
I think the main problem with this is that this requires modified content which would need to be produced. For now, I think it's better to make existing content sound as good as possible.
-
I want Doppler on everything, sounds like a launcher flag. it also sounds like the support code is in place.
-
-Joshua-: Can you give an example or two of where distancebased sound morphing is used in HL2?
Gunfire. If you play some of the more open missions you can really tell if those SMGs are being fired far away or rather close to you. It's esp. prevalent on Minerva (the mod) and the sand buggy mission (Highway 17?)
I personally think it would work really well on explosions in FS2.
I think the main problem with this is that this requires modified content which would need to be produced. For now, I think it's better to make existing content sound as good as possible.
True that, but one can dream ;) (I'd actually be interested in producing that content, but I know enough about the nature of developing stuff in ones freetime to withhold myself from claiming full commitment :P).
-
If Doppler sounds bad, maybe make the effect 25% of "realistic", just so it's there, and neat, but not distracting?
-
-Joshua-: Can you give an example or two of where distancebased sound morphing is used in HL2?
Gunfire. If you play some of the more open missions you can really tell if those SMGs are being fired far away or rather close to you. It's esp. prevalent on Minerva (the mod) and the sand buggy mission (Highway 17?)
I personally think it would work really well on explosions in FS2.
I think the main problem with this is that this requires modified content which would need to be produced. For now, I think it's better to make existing content sound as good as possible.
True that, but one can dream ;) (I'd actually be interested in producing that content, but I know enough about the nature of developing stuff in ones freetime to withhold myself from claiming full commitment :P).
I think this is already done in trunk, or in one of m|m's github branches. The trick would be to combine a sound effect and adjust the ranges and attenuation schemes.
-
can I throw doppler effects into this mix?
Also: Sound morphing based on distance. HL2 has an interesting system where it uses stereo soundfiles, where hte left channel is the "close" sound and the right channel is the "far" sound, and then mixes them as a mono source based on distance. Alternatively, simply allow different sound files that work similar to model lods.
Sure, there is no sound in space, but I think it still would be a great boon to the immersion. Also, for mods that actually take place in athmospheric conditions (sound delay based on distance too, since even the most realistic games often forget about this - although I recognize that it may be *bloody hard*).
I think the main problem with this is that this requires modified content which would need to be produced. For now, I think it's better to make existing content sound as good as possible.
You could apply an EQ rolloff and reverb (of the early reflections variety) to sound effects depending on distance from the player to get results without needing new files.
That would be simulating the attenuation of different frequencies of sound by the atmosphere and the sound bouncing off other objects in the environment (neither of which is present in a typical FS mission, but rule of cool). No guarantees it would sound any good.
-
How's the current support for 5.1 surround sound? I haven't played any trunk builds for months, but about half a year ago 5.1 was pretty garbage, if it was even working. Is it also viable to make vertical separation good as well??
-
OpenAL Soft (http://kcat.strangesoft.net/openal.html) can mix sound for 5.1 output so you could try installing that and see if that works better for you.
-
From time to time, i have this issue where one or 2 in-mission voices (people over comms) get played VERY quietly. Like 10% of what they should. This can happen to any voice, but is quite rare and almost never struck the same line twice. I think it may be present when there is lots happening sound-wise, big battles with beams and lots of explosions, but also happened in nebula once, where for entire mission i heard none of the voices. The thunder-strikes were LOUD tho. Restart the mission and everything works allright.
Same behavior on both Realtek HD Audio (built into mainboard) and USB Soundcard that comes with Creative Fatality Headset.
Win7 x64, 3.7.2 RC3 and this has been going on for a long time. I think i first noticed it after updating FSO halfway between mediavps 3.6.10 and 3.6.12.
-
A I've always liked the idea of the possibility to route the messages voices of the pilots to a secondary audio card.
That would give us the possibility to send these messages to a headset and use the primary card for music and sound effects even with 5.1 system.
That would be great for cockpit projects and nice inmersion of the game. ;)
I don't know if it's too difficult or impossible, but i wanted to share the idea.
Thanks.