Author Topic: Possibility for AI upscaled FMVs of original FS1 and FS2?  (Read 7184 times)

0 Members and 1 Guest are viewing this topic.

Offline sfried

  • 27
Possibility for AI upscaled FMVs of original FS1 and FS2?
While most mods I've seen outright replace the cinematic FMVs with new renders, I was hoping someone in the community would try to use AI upscaling, namely Topaz Video Enhance AI, to try and upscale/"restore" the original FMVs that were included in Freespace and Freespace 2.

I feel this option deserves a chance considering how it worked well for many other such projects, such as the Wing Commander 3 and 4 games, and most recently Independence War 2. The results are quite astounding and I much prefer having the choice to play the Freespace games "faithfully" with the Volition-made FMVs as opposed to just choosing fan recreations of the same cutscenes.

 

Offline DefCynodont119

  • 210
  • Ascended GTSC-Faustus Artist
    • Steam
Re: Possibility for AI upscaled FMVs of original FS1 and FS2?
I don't think that will work very well here:


The issue with this is that the FMVs use outdated models for ships, these models are little better then boxes, and no matter how many post effects are applied to the video that still won't change the fact that the ships and models are very low ploy.


I don't want to discourage you or anyone else from trying, but I don't think prettying up the videos in-post will make them look any better, the issue with the original cutscenes was mostly the low poly-count of renders themselves, not the video resolution.
My gift from Freespace to Cities Skylines:  http://steamcommunity.com/sharedfiles/filedetails/?id=639891299

 

Offline Novachen

  • 29
  • The one and only capella supernova
    • Twitter
Re: Possibility for AI upscaled FMVs of original FS1 and FS2?
Well... i think the (Ancient) Monologue cutscenes could be upscaled as they does not use any ship models (with wrong weapon points etc.  :lol:) ... but well... would they actually look very different to the Remakes?

I think especially these ones are very faithfully already.
Female FreeSpace 2 pilot since 1999.
Former Global moderator in the German FreeSpace Galaxy Forum.
Developer of NTP - A Multi-Language Translation Library Interface, which allows to play FreeSpace in YOUR Language.

Is one of my releases broken or not working? Please send a PM here, on Discord at @novachen or on Twitter @NovachenFS2, a public tweet or write a reply in my own release threads here on HLP, because these are the only threads i am still participating in.

 
Re: Possibility for AI upscaled FMVs of original FS1 and FS2?
I feel this option deserves a chance considering how it worked well for many other such projects, such as the Wing Commander 3 and 4 games, and most recently Independence War 2. The results are quite astounding and I much prefer having the choice to play the Freespace games "faithfully" with the Volition-made FMVs as opposed to just choosing fan recreations of the same cutscenes.

While I wouldn't deny the technical possibility per se, there are simply hurdles in upscaling from ~360p to 1080p that you simply cannot scale without rebuilding/rerendering everything from scratch.

But even in case WC and IW mods did exactly that smh, it'd be helpful if you'd provide YT links instead of just mentioning them.

 

Offline sfried

  • 27
Re: Possibility for AI upscaled FMVs of original FS1 and FS2?
While I wouldn't deny the technical possibility per se, there are simply hurdles in upscaling from ~360p to 1080p that you simply cannot scale without rebuilding/rerendering everything from scratch.
I disagree. AI upscale has improved a lot since then, especially with tools such as Topaz Labs Video Enhance AI, which I've seen multiple modders already toy around with the restoring and upscaling FMVs from old games, even far more impressive than what Ptelograph did with C&Cs Remastered FMVs.
But even in case WC and IW mods did exactly that smh, it'd be helpful if you'd provide YT links instead of just mentioning them.

With regards to Wing Commander (III and IV), you can find the AI upscaled work done on the FMVs on Owen Davies' youtube page. But here's an example of the recent improvements they've done since their "HD Video Pack 5.0" for Wing Commander IV.

(The usage of interpollation to increase the framerate from 30fps to 60fps seems a bit jarring IMO, as it gives the game a rather soap opera look.)

Here's an example given to Wing Commander III, which had no DVD version:

As for Independence War 2, the video raplacement files can be downloaded here. The only comparison example I could give is the one posted on the i-war2.com website:



Edit: I found the youtube examples the author posted! Mind you this was before they fine-tuned the AI model settings to give better results in the finished product:

I don't think that will work very well here:


The issue with this is that the FMVs use outdated models for ships, these models are little better then boxes, and no matter how many post effects are applied to the video that still won't change the fact that the ships and models are very low ploy.


I don't want to discourage you or anyone else from trying, but I don't think prettying up the videos in-post will make them look any better, the issue with the original cutscenes was mostly the low poly-count of renders themselves, not the video resolution.
I disagree. I feel that preserving [V]'s original efforts with the FMV would be very helpful in keeping with the intent they envisioned for the games. And no offence, but the newer cinematics, for me, will always just be fan recreations. Like Black Mesa, they can be spectacular fan recreations that can go above and beyond what the source material has done, but nothing should ever replace the originals outright. They are alternatives, but it's best to experience both, IMO.

AI upscale allows use to experience the original cinematics with much better quality while preserving the oldness and limitations presented by that era which contribute to the overall feel of the base game for modern hardware. It's the best of both worlds with regards to preservation.
« Last Edit: March 27, 2022, 03:35:45 am by sfried »

 
Re: Possibility for AI upscaled FMVs of original FS1 and FS2?
Well that looks much better than I expected. Anyway, the issue with the lowpoly ships remains, so you'd have highres pics of lowpoly ships, though you're welcome to experiment with it if you want.

 

Offline sfried

  • 27
Re: Possibility for AI upscaled FMVs of original FS1 and FS2?
I don't mind the lowpoly ships. The same goes for the lowpoly aesthetic of the humans in the games. They're definitely products of its time and yet there's a certain charm to seeing them at a higher resolution.

 
Re: Possibility for AI upscaled FMVs of original FS1 and FS2?
Well that's up to you ofc. If you want to experiment with it you might be able to get a free trial version of that software, I don't think anybody else has the tool in question here + it's rather unlikely somebody else does it if you dont step ahead

 

Offline mjn.mixael

  • Cutscene Master
  • 212
  • Chopped liver
    • Steam
    • Twitter
Re: Possibility for AI upscaled FMVs of original FS1 and FS2?
I have used AI upscaling on other projects. I can tell you it is not as easy as loading in the original FMV and hitting the go button. It takes significant work to get good results. Poorly done upscales result in a very strange wobbly/painterly effect as the computer tries to fill in detail that doesn't exist in the original. Often you have to go scene by scene or shot by shot, tweaking the settings as you go and then stitching it back together later. It's a lot of work to do right.

If you'd like to try it, then go for it. It's not a project I'm personally interested in.
Cutscene Upgrade Project - Mainhall Remakes - Between the Ashes
Youtube Channel - P3D Model Box
Between the Ashes is looking for committed testers, PM me for details.
Freespace Upgrade Project See what's happening.

 

Offline sfried

  • 27
Re: Possibility for AI upscaled FMVs of original FS1 and FS2?
I have used AI upscaling on other projects. I can tell you it is not as easy as loading in the original FMV and hitting the go button. It takes significant work to get good results. Poorly done upscales result in a very strange wobbly/painterly effect as the computer tries to fill in detail that doesn't exist in the original. Often you have to go scene by scene or shot by shot, tweaking the settings as you go and then stitching it back together later. It's a lot of work to do right.
Oh I definitely understand that it isn't going to be easy. Most people who work using AI upscale remastering also use other video edition tools and also tinker around with video encoder/decoder formats to "extract the best quality"...which does make me want to ask, what encoder/format is the .mve/.mv2 file that Freespace and its sequel use for cinematics?

 

Offline sfried

  • 27
Re: Possibility for AI upscaled FMVs of original FS1 and FS2?
Now that I have my hands on a GPU with Tensor cores and some funds to get Topaz Video Enhance AI, I intend to go forward with this project.

How do I extract/repackage the .mve/.mv2 video files from both Freespace games?

 

Offline Antares

  • 28
  • Author of The Shivan Manifesto
Re: Possibility for AI upscaled FMVs of original FS1 and FS2?
Now that I have my hands on a GPU with Tensor cores and some funds to get Topaz Video Enhance AI, I intend to go forward with this project.

How do I extract/repackage the .mve/.mv2 video files from both Freespace games?

You might be better off just grabbing them as OGGs and working from there.

FS1 cutscenes

FS2 cutscenes

EDIT: If you prefer, they're also available in AVI format.

FS1 AVIs

FS2 AVIs
« Last Edit: November 17, 2022, 05:49:20 pm by Antares »
We have returned to continue our purification of this galaxy. It is again your turn to be crushed beneath the great force that is the Antaran army. All your petty squabbling with the other beings in this galaxy is meaningless. The Antaran fleet will destroy you all, one by one. You may not surrender. You cannot win. Your only option is death.

 

Offline Novachen

  • 29
  • The one and only capella supernova
    • Twitter
Re: Possibility for AI upscaled FMVs of original FS1 and FS2?
For the best quality, you are also able to convert the original MVE files into AVI by yourself with MVE2AVI.

Not sure if the downloadable OGGs and AVIs are maybe way too compressed.
Female FreeSpace 2 pilot since 1999.
Former Global moderator in the German FreeSpace Galaxy Forum.
Developer of NTP - A Multi-Language Translation Library Interface, which allows to play FreeSpace in YOUR Language.

Is one of my releases broken or not working? Please send a PM here, on Discord at @novachen or on Twitter @NovachenFS2, a public tweet or write a reply in my own release threads here on HLP, because these are the only threads i am still participating in.

  

Offline sfried

  • 27
Re: Possibility for AI upscaled FMVs of original FS1 and FS2?
For the best quality, you are also able to convert the original MVE files into AVI by yourself with MVE2AVI.

Not sure if the downloadable OGGs and AVIs are maybe way too compressed.
I know this is coming late, but thanks! I have a lot of free time on my hands now, so hope to make something out of it.

Edit: Just tried it but I got a "DirectDraw Init FAILED" error

Edit 2: I managed to overcome this by downloading DDrawCompat, throwing the .dll into the MVE2AVI folder, and the extraction worked flawlessly! Now to throw the resulting AVI to Topav Video AI...
« Last Edit: March 07, 2024, 01:18:45 am by sfried »