http://www.perlmonks.org?node_id=1004571

So I use my laptop for all CPAN development and testing before I release anything to the production machines or to CPAN. I test my releases on Linux, HP-UX and AIX (and sometimes when I am near one, on Windows with Strawberry perl), but even that - and the invaluable cpantesters, do not reveal upcoming problems :(

I just noted by accident that CPANTS turned mostly red for my latest upload of Text::CSV_XS. As I know how many others depend on that, I rushed to analyse why it got red and did not find any clue. All the "fail"s did not show on my box, so I reverted to IRC on #perl-qa to ask what this distro did on their boxes and we found horror!

On my machine(s), that distro looks like:

$ gzip -d <Text-CSV_XS-0.92.tgz | tar tvf -
tar: Record size = 16 blocks
drwxr-xr-x merijn/users      0 2012-11-13 10:28 Text-CSV_XS-0.92/
drwxr-xr-x merijn/users      0 2012-11-13 10:28 Text-CSV_XS-0.92/examp
+les/
-rwxr-xr-x merijn/users   1966 2012-01-11 09:38 Text-CSV_XS-0.92/examp
+les/parser-xs.pl
-rw-r--r-- merijn/users   2282 2012-05-30 09:04 Text-CSV_XS-0.92/examp
+les/speed.pl
-rwxr-xr-x merijn/users   5846 2012-01-11 09:40 Text-CSV_XS-0.92/examp
+les/csv2xls
-rwxr-xr-x merijn/users   3733 2012-03-08 12:27 Text-CSV_XS-0.92/examp
+les/csvdiff
-rwxr-xr-x merijn/users   4043 2012-03-20 10:09 Text-CSV_XS-0.92/examp
+les/csv-check
-rw-r--r-- merijn/users  22087 2012-11-13 10:17 Text-CSV_XS-0.92/Chang
+eLog
-rw-r--r-- merijn/users    881 2012-11-13 10:28 Text-CSV_XS-0.92/META.
+yml
-rw-r--r-- merijn/users   1731 2012-11-13 10:28 Text-CSV_XS-0.92/MANIF
+EST
-rw-r--r-- merijn/users 180260 2012-05-28 11:45 Text-CSV_XS-0.92/pppor
+t.h
-rw-r--r-- merijn/users   3115 2012-06-15 16:33 Text-CSV_XS-0.92/Makef
+ile.PL
-rw-r--r-- merijn/users  43306 2012-11-13 10:17 Text-CSV_XS-0.92/CSV_X
+S.xs
drwxr-xr-x merijn/users      0 2012-11-13 10:28 Text-CSV_XS-0.92/files
+/
-rw-r--r-- merijn/users    931 2008-12-20 10:24 Text-CSV_XS-0.92/files
+/macosx.csv
-rw-r--r-- merijn/users     12 2008-06-04 12:52 Text-CSV_XS-0.92/files
+/utf8.csv
-rw-r--r-- merijn/users   1706 2012-05-28 12:53 Text-CSV_XS-0.92/READM
+E
-rw-r--r-- merijn/users   1463 2012-11-13 10:28 Text-CSV_XS-0.92/META.
+json
drwxr-xr-x merijn/users      0 2012-11-13 10:28 Text-CSV_XS-0.92/t/
-rw-r--r-- merijn/users   5379 2012-06-16 10:36 Text-CSV_XS-0.92/t/46_
+eol_si.t
-rw-r--r-- merijn/users   4341 2012-06-16 19:55 Text-CSV_XS-0.92/t/20_
+file.t
-rw-r--r-- merijn/users   1925 2012-06-14 20:13 Text-CSV_XS-0.92/t/76_
+magic.t
-rw-r--r-- merijn/users    544 2012-06-07 15:08 Text-CSV_XS-0.92/t/81_
+subclass.t
-rw-r--r-- merijn/users    489 2007-06-09 12:39 Text-CSV_XS-0.92/t/uti
+l.pl
-rw-r--r-- merijn/users   1705 2012-06-07 15:07 Text-CSV_XS-0.92/t/40_
+misc.t
-rw-r--r-- merijn/users  25550 2012-11-13 10:17 Text-CSV_XS-0.92/t/70_
+rt.t
-rw-r--r-- merijn/users   4924 2012-08-27 13:39 Text-CSV_XS-0.92/t/22_
+scalario.t
-rw-r--r-- merijn/users   4374 2012-06-16 19:56 Text-CSV_XS-0.92/t/21_
+lexicalio.t
-rw-r--r-- merijn/users   1709 2012-06-16 19:53 Text-CSV_XS-0.92/t/77_
+getall.t
-rw-r--r-- merijn/users   5609 2012-06-07 15:07 Text-CSV_XS-0.92/t/12_
+acc.t
-rw-r--r-- merijn/users    175 2012-06-07 15:07 Text-CSV_XS-0.92/t/00_
+pod.t
-rw-r--r-- merijn/users    782 2012-06-07 15:08 Text-CSV_XS-0.92/t/60_
+samples.t
-rw-r--r-- merijn/users    245 2012-06-07 15:07 Text-CSV_XS-0.92/t/01_
+pod.t
-rw-r--r-- merijn/users   4404 2012-06-16 19:52 Text-CSV_XS-0.92/t/75_
+hashref.t
-rw-r--r-- merijn/users  13126 2012-06-16 10:50 Text-CSV_XS-0.92/t/65_
+allow.t
-rw-r--r-- merijn/users   4244 2012-06-16 20:09 Text-CSV_XS-0.92/t/50_
+utf8.t
-rw-r--r-- merijn/users   2022 2012-06-16 10:30 Text-CSV_XS-0.92/t/41_
+null.t
-rw-r--r-- merijn/users   1702 2012-06-15 15:57 Text-CSV_XS-0.92/t/30_
+types.t
-rw-r--r-- merijn/users   3274 2012-06-07 15:07 Text-CSV_XS-0.92/t/51_
+utf8.t
-rw-r--r-- merijn/users   3920 2012-06-15 15:56 Text-CSV_XS-0.92/t/10_
+base.t
-rw-r--r-- merijn/users   2333 2012-06-07 15:08 Text-CSV_XS-0.92/t/55_
+combi.t
-rw-r--r-- merijn/users   4597 2012-06-16 10:22 Text-CSV_XS-0.92/t/80_
+diag.t
-rw-r--r-- merijn/users   7711 2012-06-16 10:30 Text-CSV_XS-0.92/t/45_
+eol.t
-rw-r--r-- merijn/users  10320 2012-06-07 15:07 Text-CSV_XS-0.92/t/15_
+flags.t
-rw-r--r-- merijn/users  56767 2012-08-27 14:25 Text-CSV_XS-0.92/CSV_X
+S.pm

[download]

Whereas on someone else's laptop it showed as:

$ gzip -d <Text-CSV_XS-0.92.tgz | tar tvf -
rw-r--r-- 203/100     85 Nov 13 10:28 2012 ./PaxHeaders.23664/Text-CSV
+_XS-0.92
rwxr-xr-x 203/20       0 Nov 13 10:28 2012 Text-CSV_XS-0.92/
rw-r--r-- 203/100     90 Nov 13 10:28 2012 Text-CSV_XS-0.92/PaxHeaders
+.23664/examples
rwxr-xr-x 203/20       0 Nov 13 10:28 2012 Text-CSV_XS-0.92/examples/
rw-r--r-- 203/100     89 Jan 11 09:38 2012 Text-CSV_XS-0.92/examples/P
+axHeaders.23664/parser-xs.pl
rwxr-xr-x 203/20    1966 Jan 11 09:38 2012 Text-CSV_XS-0.92/examples/p
+arser-xs.pl
rw-r--r-- 203/100     89 May 30 09:04 2012 Text-CSV_XS-0.92/examples/P
+axHeaders.23664/speed.pl
rw-r--r-- 203/20    2282 May 30 09:04 2012 Text-CSV_XS-0.92/examples/s
+peed.pl
rw-r--r-- 203/100     89 Jan 11 09:40 2012 Text-CSV_XS-0.92/examples/P
+axHeaders.23664/csv2xls
rwxr-xr-x 203/20    5846 Jan 11 09:40 2012 Text-CSV_XS-0.92/examples/c
+sv2xls
rw-r--r-- 203/100     89 Mar  8 12:27 2012 Text-CSV_XS-0.92/examples/P
+axHeaders.23664/csvdiff
rwxr-xr-x 203/20    3733 Mar  8 12:27 2012 Text-CSV_XS-0.92/examples/c
+svdiff
rw-r--r-- 203/100     89 Mar 20 10:09 2012 Text-CSV_XS-0.92/examples/P
+axHeaders.23664/csv-check
rwxr-xr-x 203/20    4043 Mar 20 10:09 2012 Text-CSV_XS-0.92/examples/c
+sv-check
rw-r--r-- 203/100     89 Nov 13 10:17 2012 Text-CSV_XS-0.92/PaxHeaders
+.23664/ChangeLog
rw-r--r-- 203/20   22087 Nov 13 10:17 2012 Text-CSV_XS-0.92/ChangeLog
rw-r--r-- 203/100     84 Nov 13 10:28 2012 Text-CSV_XS-0.92/PaxHeaders
+.23664/META.yml
rw-r--r-- 203/20     881 Nov 13 10:28 2012 Text-CSV_XS-0.92/META.yml
rw-r--r-- 203/100     90 Nov 13 10:28 2012 Text-CSV_XS-0.92/PaxHeaders
+.23664/MANIFEST
rw-r--r-- 203/20    1731 Nov 13 10:28 2012 Text-CSV_XS-0.92/MANIFEST
rw-r--r-- 203/100     89 May 28 11:45 2012 Text-CSV_XS-0.92/PaxHeaders
+.23664/ppport.h
rw-r--r-- 203/20  180260 May 28 11:45 2012 Text-CSV_XS-0.92/ppport.h
rw-r--r-- 203/100     87 Jun 15 16:33 2012 Text-CSV_XS-0.92/PaxHeaders
+.23664/Makefile.PL
rw-r--r-- 203/20    3115 Jun 15 16:33 2012 Text-CSV_XS-0.92/Makefile.P
+L
rw-r--r-- 203/100     89 Nov 13 10:17 2012 Text-CSV_XS-0.92/PaxHeaders
+.23664/CSV_XS.xs
rw-r--r-- 203/20   43306 Nov 13 10:17 2012 Text-CSV_XS-0.92/CSV_XS.xs
rw-r--r-- 203/100     90 Nov 13 10:28 2012 Text-CSV_XS-0.92/PaxHeaders
+.23664/files
rwxr-xr-x 203/20       0 Nov 13 10:28 2012 Text-CSV_XS-0.92/files/
rw-r--r-- 203/100     59 Dec 20 10:24 2008 Text-CSV_XS-0.92/files/PaxH
+eaders.23664/macosx.csv
rw-r--r-- 203/20     931 Dec 20 10:24 2008 Text-CSV_XS-0.92/files/maco
+sx.csv
rw-r--r-- 203/100     59 Jun  4 12:52 2008 Text-CSV_XS-0.92/files/PaxH
+eaders.23664/utf8.csv
rw-r--r-- 203/20      12 Jun  4 12:52 2008 Text-CSV_XS-0.92/files/utf8
+.csv
rw-r--r-- 203/100     89 May 28 12:53 2012 Text-CSV_XS-0.92/PaxHeaders
+.23664/README
rw-r--r-- 203/20    1706 May 28 12:53 2012 Text-CSV_XS-0.92/README
rw-r--r-- 203/100     84 Nov 13 10:28 2012 Text-CSV_XS-0.92/PaxHeaders
+.23664/META.json
rw-r--r-- 203/20    1463 Nov 13 10:28 2012 Text-CSV_XS-0.92/META.json
rw-r--r-- 203/100     90 Nov 13 10:28 2012 Text-CSV_XS-0.92/PaxHeaders
+.23664/t
rwxr-xr-x 203/20       0 Nov 13 10:28 2012 Text-CSV_XS-0.92/t/
rw-r--r-- 203/100     89 Jun 16 10:36 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/46_eol_si.t
rw-r--r-- 203/20    5379 Jun 16 10:36 2012 Text-CSV_XS-0.92/t/46_eol_s
+i.t
rw-r--r-- 203/100     89 Jun 16 19:55 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/20_file.t
rw-r--r-- 203/20    4341 Jun 16 19:55 2012 Text-CSV_XS-0.92/t/20_file.
+t
rw-r--r-- 203/100     89 Jun 14 20:13 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/76_magic.t
rw-r--r-- 203/20    1925 Jun 14 20:13 2012 Text-CSV_XS-0.92/t/76_magic
+.t
rw-r--r-- 203/100     89 Jun  7 15:08 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/81_subclass.t
rw-r--r-- 203/20     544 Jun  7 15:08 2012 Text-CSV_XS-0.92/t/81_subcl
+ass.t
rw-r--r-- 203/100     59 Jun  9 12:39 2007 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/util.pl
rw-r--r-- 203/20     489 Jun  9 12:39 2007 Text-CSV_XS-0.92/t/util.pl
rw-r--r-- 203/100     89 Jun  7 15:07 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/40_misc.t
rw-r--r-- 203/20    1705 Jun  7 15:07 2012 Text-CSV_XS-0.92/t/40_misc.
+t
rw-r--r-- 203/100     89 Nov 13 10:17 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/70_rt.t
rw-r--r-- 203/20   25550 Nov 13 10:17 2012 Text-CSV_XS-0.92/t/70_rt.t
rw-r--r-- 203/100     89 Aug 27 13:39 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/22_scalario.t
rw-r--r-- 203/20    4924 Aug 27 13:39 2012 Text-CSV_XS-0.92/t/22_scala
+rio.t
rw-r--r-- 203/100     87 Jun 16 19:56 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/21_lexicalio.t
rw-r--r-- 203/20    4374 Jun 16 19:56 2012 Text-CSV_XS-0.92/t/21_lexic
+alio.t
rw-r--r-- 203/100     89 Jun 16 19:53 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/77_getall.t
rw-r--r-- 203/20    1709 Jun 16 19:53 2012 Text-CSV_XS-0.92/t/77_getal
+l.t
rw-r--r-- 203/100     89 Jun  7 15:07 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/12_acc.t
rw-r--r-- 203/20    5609 Jun  7 15:07 2012 Text-CSV_XS-0.92/t/12_acc.t
rw-r--r-- 203/100     89 Jun  7 15:07 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/00_pod.t
rw-r--r-- 203/20     175 Jun  7 15:07 2012 Text-CSV_XS-0.92/t/00_pod.t
rw-r--r-- 203/100     89 Jun  7 15:08 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/60_samples.t
rw-r--r-- 203/20     782 Jun  7 15:08 2012 Text-CSV_XS-0.92/t/60_sampl
+es.t
rw-r--r-- 203/100     89 Jun  7 15:07 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/01_pod.t
rw-r--r-- 203/20     245 Jun  7 15:07 2012 Text-CSV_XS-0.92/t/01_pod.t
rw-r--r-- 203/100     89 Jun 16 19:52 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/75_hashref.t
rw-r--r-- 203/20    4404 Jun 16 19:52 2012 Text-CSV_XS-0.92/t/75_hashr
+ef.t
rw-r--r-- 203/100     89 Jun 16 10:50 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/65_allow.t
rw-r--r-- 203/20   13126 Jun 16 10:50 2012 Text-CSV_XS-0.92/t/65_allow
+.t
rw-r--r-- 203/100     89 Jun 16 20:09 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/50_utf8.t
rw-r--r-- 203/20    4244 Jun 16 20:09 2012 Text-CSV_XS-0.92/t/50_utf8.
+t
rw-r--r-- 203/100     89 Jun 16 10:30 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/41_null.t
rw-r--r-- 203/20    2022 Jun 16 10:30 2012 Text-CSV_XS-0.92/t/41_null.
+t
rw-r--r-- 203/100     89 Jun 15 15:57 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/30_types.t
rw-r--r-- 203/20    1702 Jun 15 15:57 2012 Text-CSV_XS-0.92/t/30_types
+.t
rw-r--r-- 203/100     88 Jun  7 15:07 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/51_utf8.t
rw-r--r-- 203/20    3274 Jun  7 15:07 2012 Text-CSV_XS-0.92/t/51_utf8.
+t
rw-r--r-- 203/100     89 Jun 15 15:56 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/10_base.t
rw-r--r-- 203/20    3920 Jun 15 15:56 2012 Text-CSV_XS-0.92/t/10_base.
+t
rw-r--r-- 203/100     89 Jun  7 15:08 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/55_combi.t
rw-r--r-- 203/20    2333 Jun  7 15:08 2012 Text-CSV_XS-0.92/t/55_combi
+.t
rw-r--r-- 203/100     89 Jun 16 10:22 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/80_diag.t
rw-r--r-- 203/20    4597 Jun 16 10:22 2012 Text-CSV_XS-0.92/t/80_diag.
+t
rw-r--r-- 203/100     89 Jun 16 10:30 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/45_eol.t
rw-r--r-- 203/20    7711 Jun 16 10:30 2012 Text-CSV_XS-0.92/t/45_eol.t
rw-r--r-- 203/100     89 Jun  7 15:07 2012 Text-CSV_XS-0.92/t/PaxHeade
+rs.23664/15_flags.t
rw-r--r-- 203/20   10320 Jun  7 15:07 2012 Text-CSV_XS-0.92/t/15_flags
+.t
rw-r--r-- 203/100     87 Aug 27 14:25 2012 Text-CSV_XS-0.92/PaxHeaders
+.23664/CSV_XS.pm
rw-r--r-- 203/20   56767 Aug 27 14:25 2012 Text-CSV_XS-0.92/CSV_XS.pm

[download]

That does not make me happy. I used GNU tar version 1.26, which is default on my OpenSUSE 12.2 and without it being documented, it produces the latest POSIX 1003.1-2001 (pax) format by default.

You won't notice if you have a "modern" tar (or ptar), but you will with tar versions that only support the older POSIX 1003.1-1988 (ustar) format.

With the generation of distribution well hidden behind ExtUtils::MakeMaker, Module::Build or whatever tool one prefers to use, what is the "best" way to attack this problem?

My choice might not be the best, but for me it is the simplest: set TARFLAGS in my Makefile.PL to force the older format like this (stripped for clarity):

use ExtUtils::MakeMaker;
WriteMakefile (
    ABSTRACT     => "Comma-Separated Values manipulation routines",
    VERSION_FROM => "CSV_XS.pm",
    macro        => { TARFLAGS   => "--format=ustar -c -v -f",
                    },
    );

I just hope this post will enlighten those that are bound to hit the same.


Enjoy, Have FUN! H.Merijn

Replies are listed 'Best First'.
Re: Living on the edge …
by Anonymous Monk on Nov 19, 2012 at 16:40 UTC

    With the generation of distribution well hidden behind ExtUtils::MakeMaker, Module::Build or whatever tool one prefers to use, what is the "best" way to attack this problem?

    A patch to the relevant module (like MakeMaker). If they can lower my MAXLINELENGTH they can add some ustar flags.

[reply]

      I know the maintainer and signaled him about this, but I don't know about all/other builder modules. So I thought spreading this experience here might be useful to all.


      Enjoy, Have FUN! H.Merijn
[reply]
Re: Living on the edge …
by tobyink (Canon) on Nov 20, 2012 at 13:04 UTC

    Another option would be to set the TAR_OPTIONS environment variable. The tar man page doesn't mention this possibility (at least it doesn't on my system) but it seems to work fairly universally.

    If you're using something like Module::Install which buries TARFLAGS under many layers, then a little line at the top of Makefile.PL may be use:

    BEGIN { $ENV{TAR_OPTIONS} = q{--format=ustar} };

    [download]
    perl -E'sub Monkey::do{say$_,for@_,do{($monkey=[caller(0)]->[3])=~s{::}{ }and$monkey}}"Monkey say"->Monkey::do'
[reply]
[d/l]
[select]
Re: Living on the edge …
by mithaldu (Monk) on Oct 09, 2013 at 14:29 UTC
    So, i tried out downloading the files in various ways from these links:

    CPAN: http://search.cpan.org/CPAN/authors/id/H/HM/HMBRAND/Text-CSV_XS-1.01.tgz
    BackPAN: http://backpan.perl.org/authors/id/H/HM/HMBRAND/Text-CSV_XS-1.01.tgz

    Notably different browsers and sources resulted not only in different sizes, but also different filenames. The platform used for all cases is Windows 7. The results are as follows:

    Browser CPAN Name CPAN Size BPAN Name BPAN Size
    wget Text-CSV_XS-1.01.tgz 123288 Text-CSV_XS-1.01.tgz 123288
    Internet Explorer 10 Text-CSV_XS-1.01.gz 123298 (+10) Text-CSV_XS-1.01.tar 123288
    Opera 12 Text-CSV_XS-1.01.tgz 123288 Text-CSV_XS-1.01.tgz 123288
    Firefox 24 Text-CSV_XS-1.01.tgz 123298 (+10) Text-CSV_XS-1.01.tgz 123288
[reply]
      The platform used for all cases is Windows 7

      You don't have to be on Windows to be bitten by this browser mismanagement.
      Iceweasel on Debian wheezy and Firefox on Ubuntu-12.04LTS also throw in the extra 10 bytes when downloading Text-CSV_XS-1.01.tgz from CPAN.

      (Sorry - I don't know the version numbers of those 2 browsers - and couldn't quickly find out how to get them.)

      Cheers,
      Rob
[reply]

Back to Meditations