Hard Light Productions Forums

Modding, Mission Design, and Coding => FS2 Open Coding - The Source Code Project (SCP) => Topic started by: Gregster2k on August 03, 2008, 09:25:14 am

Title: ZIP format support
Post by: Gregster2k on August 03, 2008, 09:25:14 am
I was poking around at my Media VPs for a while the other day, and then it occurred to me that we could just use (in lieu of VP's) a more common storage format like Uncompressed ZIP. For example, Quake 3 uses files (pk3's) which essentially are ZIPs with different file extensions.

Food for thought.
Title: Re: ZIP format support
Post by: Stormkeeper on August 03, 2008, 11:09:28 am
The FS code is setup to read from VPs, so if you want it to read from Zip, you'll need to modify the code. And iirc, VPs and zip files are quite different.

Besides, why would you want to do that? Its basically the same thing, except for when reading from a zip, the program will have to uncompress/recompress upon opening/shutting down.
Title: Re: ZIP format support
Post by: Jeff Vader on August 03, 2008, 11:25:29 am
And, I currently believe that if we have vps instead of zips, new people are less likely to break their FreeSpace.
Title: Re: ZIP format support
Post by: Stormkeeper on August 03, 2008, 11:27:28 am
True that. Can you imagine dl-ing a C&C3 mod in a zip file, and accidentally leaving it in your FS directory? FS tries to read it an ka blamms itself.
Title: Re: ZIP format support
Post by: Jeff Vader on August 03, 2008, 11:30:49 am
Well, that too. But I was thinking of something like "Oh! These are zips. I know that file format!", followed by the opening of the zips, some tinkering, failure and a new thread at HLP saying "freespace doesnt start up you most make an update".
Title: Re: ZIP format support
Post by: Snail on August 03, 2008, 11:44:51 am
Well, that too. But I was thinking of something like "Oh! These are zips. I know that file format!", followed by the opening of the zips, some tinkering, failure and a new thread at HLP saying "freespace doesnt start up you most make an update".
Well, that's bound to happen with INFASA since we're not using .VPs.
Title: Re: ZIP format support
Post by: karajorma on August 03, 2008, 01:09:09 pm
Your idiocy doesn't have to be repeated by others. :p
Title: Re: ZIP format support
Post by: Mr_Maniac on August 03, 2008, 04:00:07 pm
Well...
I don't know if a new file-format would be needed...
But I think it would be good to support compression for the VP-Files and - as far as I know - that is somewhere on Taylors To-do-list (lesser Priority).
Title: Re: ZIP format support
Post by: Jeff Vader on August 03, 2008, 04:01:58 pm
Correct me if I'm incorrect, but wouldn't compression result in decreased performance?
Title: Re: ZIP format support
Post by: Herra Tohtori on August 03, 2008, 04:02:58 pm
Only during loading/caching the data I think... :nervous:
Title: Re: ZIP format support
Post by: blowfish on August 03, 2008, 04:03:44 pm
Well, that's bound to happen with INFASA since we're not using .VPs.

Umm ... why? :wtf:
Title: Re: ZIP format support
Post by: Stormkeeper on August 03, 2008, 09:02:47 pm
Well, that too. But I was thinking of something like "Oh! These are zips. I know that file format!", followed by the opening of the zips, some tinkering, failure and a new thread at HLP saying "freespace doesnt start up you most make an update".
Ah... I remember that guy.

Umm ... why? :wtf:
Methinks he's joking.
Title: Re: ZIP format support
Post by: WMCoolmon on August 03, 2008, 10:06:40 pm
Well, that too. But I was thinking of something like "Oh! These are zips. I know that file format!", followed by the opening of the zips, some tinkering, failure and a new thread at HLP saying "freespace doesnt start up you most make an update".

I always hated that scene in Jurassic Park since I learned what Unix was...

there are ways around that, like making an FS2Open-only config file that you could put in the ZIP. Possibly then the Launcher could read it and use it as a mod.ini

But I think the main reason is that it would slow down loading times egregiously, for not a whole lot of gain in disk space. You should already be compressing graphics files if you can, and text files (eg missions, readmes, tables) don't take up a whole lot of room by themselves. Models take up a fair bit. Sound is usually compressed, and should be compressed, as putting it an OGG from a WAV greatly decreases filesize without a whole lot of compression artifacts, especially with voice.

So basically all you're gaining is the space from compressing models, but you're having to decompress every single file in the ZIP file.

Compressed VP format is on the way to address some of the recompressing issues, but it's still a ways from being implemented into the engine. It will actually have the possibility of better compression than ZIP, as well as certain error-detection algorithms to prevent corruption issues.
Title: Re: ZIP format support
Post by: Mr_Maniac on August 04, 2008, 12:35:40 am
Well... For the "longer loading": Depends on what level of compression you choose.
In Quake 3, for example, you can recompress the pk3s with a higher rate. And up to a certain rate, the decompressing isn't taking much longer than reading from a "store only" pk3.
And for the "not much smaller"-Argument: Well... Different VPs could be compressed at different rates. And, at least, compression IS making a difference here. Okay. With todays HDDs, this difference seems very small, but I'm still a "fan" of "keeping as much free as possible" ;)
Title: Re: ZIP format support
Post by: Jeff Vader on August 04, 2008, 05:56:35 am
Well, that too. But I was thinking of something like "Oh! These are zips. I know that file format!", followed by the opening of the zips, some tinkering, failure and a new thread at HLP saying "freespace doesnt start up you most make an update".
Ah... I remember that guy.
Incidentally, I'm now quite sure that the issue he had back then was very much related to the (absence of) chipset drivers. Too bad he doesn't seem to be online anymore.
Title: Re: ZIP format support
Post by: Stormkeeper on August 04, 2008, 08:12:15 am
Where was that thread of his, anyway? I like to read dumb threads like that once in a while, remind myself of the stupidity humans are capable of.
Title: Re: ZIP format support
Post by: Jeff Vader on August 04, 2008, 08:16:09 am
http://www.hard-light.net/forums/index.php/topic,51250.0.html

Agreed. I also do that from time to time. My personal favourites are fs2freak88's rants about declaring ST:R dead.
Title: Re: ZIP format support
Post by: Stormkeeper on August 04, 2008, 08:26:24 am
Indeed. Those and Darth hojape(sp?) random rants around HLP.
Title: Re: ZIP format support
Post by: Snail on August 04, 2008, 09:22:18 am
Umm ... why? :wtf:
I was against the idea too, but it makes patching things one effing hell of a lot easier. Before that we had to download the entire mother****ing .VP (excuse my French) all over again just to change one or to things (remember, we've gone standalone now, which compounded the problem further). In future I think we'd probably add all of the stuff into small .VP files to make it (slightly) easier.
Title: Re: ZIP format support
Post by: karajorma on August 04, 2008, 10:18:21 am
You shouldn't have to be changing things to the VP file though. If you're doing that you've already ****ed up.

As far as I'm concerned you only compile a VP file after you've finished beta testing (or mostly finished at least). If you're compiling one before that then yes, you're going to end up having to re-upload just cause you needed to change one or two files. But the sheer stupidity involved in doing that doesn't justify even greater stupidity when it comes to distributing a mod with files the users can easily delete or change and then complain at you about. Just cause VPs will cause a hassle if you are foolish enough to use them during development doesn't mean that you should release that way.

Multiple smaller VPs (The way :v: did it basically) has always been my preference.
Title: Re: ZIP format support
Post by: Herra Tohtori on August 04, 2008, 10:46:41 am
Umm ... why? :wtf:
I was against the idea too, but it makes patching things one effing hell of a lot easier. Before that we had to download the entire mother****ing .VP (excuse my French) all over again just to change one or to things (remember, we've gone standalone now, which compounded the problem further). In future I think we'd probably add all of the stuff into small .VP files to make it (slightly) easier.

Can't you include the to-be-replaced and new content in a small patch VP and make sure it gets loaded before the older VP, so that what's inside it takes precedence over older content with similar filenames...? :nervous:
Title: Re: ZIP format support
Post by: Jeff Vader on August 04, 2008, 10:49:56 am
I was just about to say the same thing. Once released, if something is broken or missing, release a patch vp that is read before the actual vp. And, if necessary, update the original vp as well, including mentionings about the (un)necessity of the patch depending on the time of downloading the original.
Title: Re: ZIP format support
Post by: karajorma on August 04, 2008, 12:20:17 pm
Can't you include the to-be-replaced and new content in a small patch VP and make sure it gets loaded before the older VP, so that what's inside it takes precedence over older content with similar filenames...? :nervous:

By and large, yes.

There are a few data related things you can't do that for (for instance if you added DDS textures in the original VP you can't patch them with TGAs in the new one since the game first checks for DDS files in any VPs in the main folder before checking the same VPs for TGAs) But such cases are few and far between and usually would suffer from the same problem as loose files (You'd have to tell the end user to manually delete the DDS files and add TGAs for instance).


There's a why the BtRL VPs began with a Z. :D
Title: Re: ZIP format support
Post by: WMCoolmon on August 04, 2008, 06:17:51 pm
Yeah, kara and I were just talking last night, and even though the latest version of Maja Express features a lockout against people overwriting their retail VP files, I've still seen people trying to do that or overwriting their mediaVP files.

Every time you try and make something foolproof, somebody goes and makes a bigger fool. :p