Author Topic: A possible way to improve campaign voiceovers  (Read 623 times)

0 Members and 1 Guest are viewing this topic.

Offline Zarax

  • 210
A possible way to improve campaign voiceovers
Someone is using deep learning to do voiceovers of text to speech including emotion starting from a determined source: https://15.ai/
Copyright issues aside (not a small thing, I know) this could potentially be used to reproduce our favourite FS2 characters with customized text or mod campaigns voiceovers without any voice actors.

The site owner is available to add content sets as long as they are cleaned with corresponding text (for which FS2 content is perfectly fitting), do you guys think it could be useful for FSO?
The Best is Yet to Come

 

Offline Goober5000

  • HLP Loremaster
  • 214
    • Goober5000 Productions
Re: A possible way to improve campaign voiceovers
It's a great site, but not really the sort of thing that would be useful for FSO.  Generating these voices takes massive amounts of computing power and can't be done in real-time.  That site would be more suitable for modders who generate voice files ahead of time and include them in their mod release.

 

Offline General Battuta

  • Poe's Law In Action
  • 214
  • i wonder when my postcount will exceed my iq
Re: A possible way to improve campaign voiceovers
Someone is using deep learning to do voiceovers of text to speech including emotion starting from a determined source: https://15.ai/
Copyright issues aside (not a small thing, I know) this could potentially be used to reproduce our favourite FS2 characters with customized text or mod campaigns voiceovers without any voice actors.

The site owner is available to add content sets as long as they are cleaned with corresponding text (for which FS2 content is perfectly fitting), do you guys think it could be useful for FSO?

If this works as advertised it would be a slam dunk great thing for modders. Darius has already used a lot of prerecorded TTS in Solaris, to quite good effect.

 
Re: A possible way to improve campaign voiceovers
That site works more or less as advertised. The real trick will be getting some voices submitted that aren't MLP or MLP adjacent.
"if disco is dead then I am the laser lich"
"...[Warmachine] keeps changing as fast as EatThePath can force the engine to do his dark bidding..."

 

Offline Zarax

  • 210
Re: A possible way to improve campaign voiceovers
It's a great site, but not really the sort of thing that would be useful for FSO.  Generating these voices takes massive amounts of computing power and can't be done in real-time.  That site would be more suitable for modders who generate voice files ahead of time and include them in their mod release.

The training part is the computationally expensive one but the author said that he's open to add new voice sets if the proper material is given (and it's notoriously hard to find quality datasets) so FS2 clips could be used as training sets and with that you can get in return new machine-generated audio with little effort (as a pre-trained model).

As for feasibility I wasn't considering training in-engine as that would be way too intensive but instead using a pre-trained model.
With a pre-trained model one could either generate audio files in batches from submitted text (similar to the way the site works now, only generating files as output) or, with an higher effort, the pre-trained model could in theory be integrated in FRED for example and generate the audio clips on mission save.
The nice thing about neural networks of all sorts is that while training is crazy expensive from a computational side, running the model is relatively trivial and is routinely done by low-power devices like smartphones.

That said, this is merely a speculative thought on how AI content generation could be useful to stretch source material, not a feature request of course so I will leave any value consideration to you guys.
The Best is Yet to Come

 

Offline Galemp

  • Actual father of Samus
  • 212
  • Current projects: GVCa Tanen, Vasudan Imperium
    • Steam
    • User page on the FreeSpace Wiki
Re: A possible way to improve campaign voiceovers
I propose we move this topic to FS Modding, since it's more feasible as a resource for mission designers to voice their work and package the output files when distributing, but not as an implementation directly into FSO.
"Anyone can do any amount of work, provided it isn't the work he's supposed to be doing at that moment." -- Robert Benchley

Members I've personally met: RedStreblo, Goober5000, Sandwich, Splinter, Su-tehp, Hippo, CP5670, Terran Emperor, Karajorma, Dekker, McCall, Admiral Wolf, mxlm, RedSniper, Stealth, Black Wolf...

 

Offline Colonol Dekker

  • HLP is my mistress
  • 213
  • Aken Tigh Dekker
    • My old squad sub-domain
Re: A possible way to improve campaign voiceovers
As the guys on discord heard, spongebob saying "Dive Dive Dive! HIT YOUR BURNERS PILOT!" is unsettling...

 

Offline Goober5000

  • HLP Loremaster
  • 214
    • Goober5000 Productions
Re: A possible way to improve campaign voiceovers
I propose we move this topic to FS Modding, since it's more feasible as a resource for mission designers to voice their work and package the output files when distributing, but not as an implementation directly into FSO.

Done.