Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re: Zlib is stopping me leaving work early and going to the pub...

by Malkavian (Friar)
on Apr 06, 2001 at 19:58 UTC ( #70503=note: print w/replies, xml ) Need Help??


in reply to Zlib is stopping me leaving work early and going to the pub...

Another couple of gotchas that I've found along the way:

  • Maximum filesize of the operating system: On linux 2.2.x, this is a 2GB file limit. Anything above that will give you errors and a truncated file. Other OSes have other filesize limits. Check on this.
  • Errors because of archive creation: A common way to create a large archive is simply concatenate gzips together (*NIX style: cat archive1.gz >> archive2.gz). If erroneous data gets thrown in here in one of the concatenation steps, it could break your unzip stream.
  • Errors becuase of double zip in concatenated archive: If the above concatenation is used to create the archive, it's possible that at some point, a zip is then itself zipped. This actually increases the zipped zip's size by a few bytes for the headers, but unzipping it would then produce a zip file. This could explain your 'partially unzipped file'.
The ports of call are thus to check out your maximum os filesize (shouldn't really be a problem for AIX), your version of Zlib, and the way the archive was created.
Also make sure the partition you're unzipping on can take the data. The zip size itself shouldn't be a problem (a fair wack of my work deals with pretty big log files. On average, the gzips I work with are 170-230MB in size. And they work fine.

Hopefully this helps some,

Malk
  • Comment on Re: Zlib is stopping me leaving work early and going to the pub...

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://70503]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (6)
As of 2020-04-04 09:48 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    The most amusing oxymoron is:
















    Results (32 votes). Check out past polls.

    Notices?