How to test if a file is gzipped or not

[Special_K]

I have a script that needs to be able to operate on either a plain text file or a gzipped version of that file. If the file is not gzipped, the file will be opened as follows:

open(FOO, $foo) || die("ERROR: Unable to open file $foo for read: $!")
+;

If the file is gzipped, the file will be opened as follows:

open(FOO, "zcat $foo |") || die("ERROR: Unable to open gzipped file $f
+oo for read: $!");

How do I write a check to determine whether the file is gzipped or not? zcat errors out when called on non-gzipped files.

[sectokia]

IO::Uncompress::AnyInflate will gunzip data, and has a Transparent option. The option is useful as you can select if you want non-gzipped data to just pass through and be returned as if it was the uncompressed result, or if you want it to throw an error.

[soonix]

TIL IO::Uncompress::AnyInflate exists.

I don't know if it's important for Special_K, but I read under "Transparent":

In addition, if the input file/buffer does contain compressed data and there is non-compressed data immediately following it, setting this option will make this module treat the whole file/buffer as a single data stream.

This might pose a security risk, if one isn't aware of it.

[LanX]

Why don't you just catch the error from gunzipping and try proceeding otherwise?

If that's not enough, Gzip files start with a header including a magic number (1f 8b)

You could also look if available uncompress tools have a check option to validate if it's a legit format.°

Edit

°) See documentation for gunzip

• -t --test
  Test. Check the compressed file integrity.

[Tux]

Instead of opening an extra process (zcat), one can also use modules as already mentioned. Like PerlIO::gzip. For inconsistency gzip is also supported in PerlIO::via::gzip, the two other perlio compress/decompress supported algorithms in that namespace are PerlIO::via::Bzip2 and PerlIO::via::xz.

On CPAN you can find a plethora of decompressors in the IO::Uncompress namespace where IO::Uncompress::Gunzip is part of (CORE since 5.9.4)

[eyepopslikeamosquito]

Instead of using zcat you might consider using core Perl or CPAN modules.

Core Perl:

• IO::Compress::FAQ
• IO::Compress::Gzip
• IO::Uncompress::Gunzip
• Re: PerlIO::gzip can't handle appended stuff by haukex - sample code using core Perl IO::Uncompress::Gunzip

CPAN:

• PerlIO::via::gzip
• PerlIO::gzip

See also:

• Re: Searching a gzip file by Corion - sample code reading via open my $fh, "gzip -cd $file |" and while (<$fh>) { ...
• Difference between gzip -d and zcat (stackexchange) - zcat is equivalent to gzip -dc

[jwkrahn]

If you have a magic file that should tell what signature a gzip has in order to determine that it is a gzip file.

[Happy-the-monk]

Testing for a file type is called "magic" in linux/unix, hence jwkrahn above mentions the magic file that facilitates translating file signatures to file types.

In a linux/unix you may test with the file command: file $fileyouwanttotest

CPAN has File::MimeInfo (and also File::MimeInfo:Magic, but the name is a bit misleading)
and it works like this:

use File::MimeInfo;
my $mime_type = mimetype('perl-5.38.0.tar.gz');
print("$mime_type\n");

Cheers, Sören

[choroba]

There's also File::MMagic and File::Type, and many more.

[Anonymous Monk]

If zcat throws an error then you know it's not gzipped:

open(FOO, "zcat $foo |") || open(FOO, $foo) || die("ERROR: Unable to o
+pen file $foo for read: $!");