As those of you who have SCP SVN access may know, I'm currently working on a rewrite of the FSO Installer. One of the high priority feature requests is 7zip support.
Unfortunately, the 7zip SDK is atrocious. No in-line source comments, very little documentation, and all you get is a Decoder and an Encoder class. That only works if you have a simple stream of bytes you want to compress using LZMA, and it's useless if you have a 7zip container with a bunch of files.
That's not the subject of the rant though, because some kind and benevolent soul figured out how to use the JNI (Java Native Interface) to wrap the actual 7zip application libraries using Java classes. It's here at 7-Zip-JBinding
, and it has bindings available for all the major platforms - Windows, OSX, and Linux. Which is A-1 SUPAR.
The problem I have is that 7-zip puts the table of contents AT THE END OF THE FILE. Good old vanilla Zip puts it at the beginning. This means that you have to seek to the end of the file before you know what the file contains.
Turey was clever enough to figure out that Java allows you to unzip regular Zip files on-the-fly, which is to say that you're downloading and extracting in the same operation. And
you're only download the out-of-date files. This is extremely handy because downloading a stream of bytes from the internet is a strictly sequential operation.
Guess what... 7zip was apparently designed on the assumption that random-access would be the norm. So not only does 7zip require you to seek to the end of the file, it seeks back and forth a lot once it's there. Unfortunately, in a forward-only stream, seeking backwards means that you have to start reading all over again and seek forward to that location from the beginning.
Using predictive buffering and some smart seeking strategies, I was able to smooth everything out so that only one seek, from beginning to end, is required to get all the necessary information. Unfortunately, that one seek is necessary due to the inherent design of 7zip; it's as if you have to download the file before you can download the file. On the 7.4 MB file I used for testing, that takes about 12 seconds.
So that's the current story. There are several paths I can take from here, and I'd like to know what people would prefer:
1) Drop 7zip support, and enforce Zip only. (I'm not seriously considering this option. First, I've already done a lot of work on it; second, 7zip allows the extraction of not only .7z files but also .rar, .bzip, .gzip, and a bunch of others.)
2) Leave the behavior as it is, and tell people they'll just have to live with the delay that occurs prior to downloading.
3) Change the behavior so that the installer will always download 7zip files before checking whether they need to be extracted. I'm not sure that this will save any time because in both cases the installer has to first move across the internet connection from start to finish, and then (optionally) extract something from it. The only significant difference is that in the current case the extraction happens over an internet connection, whereas in this case it would occur on the local hard drive, saving bandwidth. However, the user would have to live with a potentially large, potentially unnecessary file being downloaded to, and immediately deleted from, a temporary folder.