Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Living on the edge

by Tux (Monsignor)
on Nov 19, 2012 at 16:20 UTC ( #1004571=perlmeditation: print w/ replies, xml ) Need Help??

So I use my laptop for all CPAN development and testing before I release anything to the production machines or to CPAN. I test my releases on Linux, HP-UX and AIX (and sometimes when I am near one, on Windows with Strawberry perl), but even that - and the invaluable cpantesters, do not reveal upcoming problems :(

I just noted by accident that CPANTS turned mostly red for my latest upload of Text::CSV_XS. As I know how many others depend on that, I rushed to analyse why it got red and did not find any clue. All the "fail"s did not show on my box, so I reverted to IRC on #perl-qa to ask what this distro did on their boxes and we found horror!

On my machine(s), that distro looks like:

$ gzip -d <Text-CSV_XS-0.92.tgz | tar tvf - tar: Record size = 16 blocks drwxr-xr-x merijn/users 0 2012-11-13 10:28 Text-CSV_XS-0.92/ drwxr-xr-x merijn/users 0 2012-11-13 10:28 Text-CSV_XS-0.92/examp +les/ -rwxr-xr-x merijn/users 1966 2012-01-11 09:38 Text-CSV_XS-0.92/examp +les/parser-xs.pl -rw-r--r-- merijn/users 2282 2012-05-30 09:04 Text-CSV_XS-0.92/examp +les/speed.pl -rwxr-xr-x merijn/users 5846 2012-01-11 09:40 Text-CSV_XS-0.92/examp +les/csv2xls -rwxr-xr-x merijn/users 3733 2012-03-08 12:27 Text-CSV_XS-0.92/examp +les/csvdiff -rwxr-xr-x merijn/users 4043 2012-03-20 10:09 Text-CSV_XS-0.92/examp +les/csv-check -rw-r--r-- merijn/users 22087 2012-11-13 10:17 Text-CSV_XS-0.92/Chang +eLog -rw-r--r-- merijn/users 881 2012-11-13 10:28 Text-CSV_XS-0.92/META. +yml -rw-r--r-- merijn/users 1731 2012-11-13 10:28 Text-CSV_XS-0.92/MANIF +EST -rw-r--r-- merijn/users 180260 2012-05-28 11:45 Text-CSV_XS-0.92/pppor +t.h -rw-r--r-- merijn/users 3115 2012-06-15 16:33 Text-CSV_XS-0.92/Makef +ile.PL -rw-r--r-- merijn/users 43306 2012-11-13 10:17 Text-CSV_XS-0.92/CSV_X +S.xs drwxr-xr-x merijn/users 0 2012-11-13 10:28 Text-CSV_XS-0.92/files +/ -rw-r--r-- merijn/users 931 2008-12-20 10:24 Text-CSV_XS-0.92/files +/macosx.csv -rw-r--r-- merijn/users 12 2008-06-04 12:52 Text-CSV_XS-0.92/files +/utf8.csv -rw-r--r-- merijn/users 1706 2012-05-28 12:53 Text-CSV_XS-0.92/READM +E -rw-r--r-- merijn/users 1463 2012-11-13 10:28 Text-CSV_XS-0.92/META. +json drwxr-xr-x merijn/users 0 2012-11-13 10:28 Text-CSV_XS-0.92/t/ -rw-r--r-- merijn/users 5379 2012-06-16 10:36 Text-CSV_XS-0.92/t/46_ +eol_si.t -rw-r--r-- merijn/users 4341 2012-06-16 19:55 Text-CSV_XS-0.92/t/20_ +file.t -rw-r--r-- merijn/users 1925 2012-06-14 20:13 Text-CSV_XS-0.92/t/76_ +magic.t -rw-r--r-- merijn/users 544 2012-06-07 15:08 Text-CSV_XS-0.92/t/81_ +subclass.t -rw-r--r-- merijn/users 489 2007-06-09 12:39 Text-CSV_XS-0.92/t/uti +l.pl -rw-r--r-- merijn/users 1705 2012-06-07 15:07 Text-CSV_XS-0.92/t/40_ +misc.t -rw-r--r-- merijn/users 25550 2012-11-13 10:17 Text-CSV_XS-0.92/t/70_ +rt.t -rw-r--r-- merijn/users 4924 2012-08-27 13:39 Text-CSV_XS-0.92/t/22_ +scalario.t -rw-r--r-- merijn/users 4374 2012-06-16 19:56 Text-CSV_XS-0.92/t/21_ +lexicalio.t -rw-r--r-- merijn/users 1709 2012-06-16 19:53 Text-CSV_XS-0.92/t/77_ +getall.t -rw-r--r-- merijn/users 5609 2012-06-07 15:07 Text-CSV_XS-0.92/t/12_ +acc.t -rw-r--r-- merijn/users 175 2012-06-07 15:07 Text-CSV_XS-0.92/t/00_ +pod.t -rw-r--r-- merijn/users 782 2012-06-07 15:08 Text-CSV_XS-0.92/t/60_ +samples.t -rw-r--r-- merijn/users 245 2012-06-07 15:07 Text-CSV_XS-0.92/t/01_ +pod.t -rw-r--r-- merijn/users 4404 2012-06-16 19:52 Text-CSV_XS-0.92/t/75_ +hashref.t -rw-r--r-- merijn/users 13126 2012-06-16 10:50 Text-CSV_XS-0.92/t/65_ +allow.t -rw-r--r-- merijn/users 4244 2012-06-16 20:09 Text-CSV_XS-0.92/t/50_ +utf8.t -rw-r--r-- merijn/users 2022 2012-06-16 10:30 Text-CSV_XS-0.92/t/41_ +null.t -rw-r--r-- merijn/users 1702 2012-06-15 15:57 Text-CSV_XS-0.92/t/30_ +types.t -rw-r--r-- merijn/users 3274 2012-06-07 15:07 Text-CSV_XS-0.92/t/51_ +utf8.t -rw-r--r-- merijn/users 3920 2012-06-15 15:56 Text-CSV_XS-0.92/t/10_ +base.t -rw-r--r-- merijn/users 2333 2012-06-07 15:08 Text-CSV_XS-0.92/t/55_ +combi.t -rw-r--r-- merijn/users 4597 2012-06-16 10:22 Text-CSV_XS-0.92/t/80_ +diag.t -rw-r--r-- merijn/users 7711 2012-06-16 10:30 Text-CSV_XS-0.92/t/45_ +eol.t -rw-r--r-- merijn/users 10320 2012-06-07 15:07 Text-CSV_XS-0.92/t/15_ +flags.t -rw-r--r-- merijn/users 56767 2012-08-27 14:25 Text-CSV_XS-0.92/CSV_X +S.pm

Whereas on someone else's laptop it showed as:

$ gzip -d <Text-CSV_XS-0.92.tgz | tar tvf - rw-r--r-- 203/100 85 Nov 13 10:28 2012 ./PaxHeaders.23664/Text-CSV +_XS-0.92 rwxr-xr-x 203/20 0 Nov 13 10:28 2012 Text-CSV_XS-0.92/ rw-r--r-- 203/100 90 Nov 13 10:28 2012 Text-CSV_XS-0.92/PaxHeaders +.23664/examples rwxr-xr-x 203/20 0 Nov 13 10:28 2012 Text-CSV_XS-0.92/examples/ rw-r--r-- 203/100 89 Jan 11 09:38 2012 Text-CSV_XS-0.92/examples/P +axHeaders.23664/parser-xs.pl rwxr-xr-x 203/20 1966 Jan 11 09:38 2012 Text-CSV_XS-0.92/examples/p +arser-xs.pl rw-r--r-- 203/100 89 May 30 09:04 2012 Text-CSV_XS-0.92/examples/P +axHeaders.23664/speed.pl rw-r--r-- 203/20 2282 May 30 09:04 2012 Text-CSV_XS-0.92/examples/s +peed.pl rw-r--r-- 203/100 89 Jan 11 09:40 2012 Text-CSV_XS-0.92/examples/P +axHeaders.23664/csv2xls rwxr-xr-x 203/20 5846 Jan 11 09:40 2012 Text-CSV_XS-0.92/examples/c +sv2xls rw-r--r-- 203/100 89 Mar 8 12:27 2012 Text-CSV_XS-0.92/examples/P +axHeaders.23664/csvdiff rwxr-xr-x 203/20 3733 Mar 8 12:27 2012 Text-CSV_XS-0.92/examples/c +svdiff rw-r--r-- 203/100 89 Mar 20 10:09 2012 Text-CSV_XS-0.92/examples/P +axHeaders.23664/csv-check rwxr-xr-x 203/20 4043 Mar 20 10:09 2012 Text-CSV_XS-0.92/examples/c +sv-check rw-r--r-- 203/100 89 Nov 13 10:17 2012 Text-CSV_XS-0.92/PaxHeaders +.23664/ChangeLog rw-r--r-- 203/20 22087 Nov 13 10:17 2012 Text-CSV_XS-0.92/ChangeLog rw-r--r-- 203/100 84 Nov 13 10:28 2012 Text-CSV_XS-0.92/PaxHeaders +.23664/META.yml rw-r--r-- 203/20 881 Nov 13 10:28 2012 Text-CSV_XS-0.92/META.yml rw-r--r-- 203/100 90 Nov 13 10:28 2012 Text-CSV_XS-0.92/PaxHeaders +.23664/MANIFEST rw-r--r-- 203/20 1731 Nov 13 10:28 2012 Text-CSV_XS-0.92/MANIFEST rw-r--r-- 203/100 89 May 28 11:45 2012 Text-CSV_XS-0.92/PaxHeaders +.23664/ppport.h rw-r--r-- 203/20 180260 May 28 11:45 2012 Text-CSV_XS-0.92/ppport.h rw-r--r-- 203/100 87 Jun 15 16:33 2012 Text-CSV_XS-0.92/PaxHeaders +.23664/Makefile.PL rw-r--r-- 203/20 3115 Jun 15 16:33 2012 Text-CSV_XS-0.92/Makefile.P +L rw-r--r-- 203/100 89 Nov 13 10:17 2012 Text-CSV_XS-0.92/PaxHeaders +.23664/CSV_XS.xs rw-r--r-- 203/20 43306 Nov 13 10:17 2012 Text-CSV_XS-0.92/CSV_XS.xs rw-r--r-- 203/100 90 Nov 13 10:28 2012 Text-CSV_XS-0.92/PaxHeaders +.23664/files rwxr-xr-x 203/20 0 Nov 13 10:28 2012 Text-CSV_XS-0.92/files/ rw-r--r-- 203/100 59 Dec 20 10:24 2008 Text-CSV_XS-0.92/files/PaxH +eaders.23664/macosx.csv rw-r--r-- 203/20 931 Dec 20 10:24 2008 Text-CSV_XS-0.92/files/maco +sx.csv rw-r--r-- 203/100 59 Jun 4 12:52 2008 Text-CSV_XS-0.92/files/PaxH +eaders.23664/utf8.csv rw-r--r-- 203/20 12 Jun 4 12:52 2008 Text-CSV_XS-0.92/files/utf8 +.csv rw-r--r-- 203/100 89 May 28 12:53 2012 Text-CSV_XS-0.92/PaxHeaders +.23664/README rw-r--r-- 203/20 1706 May 28 12:53 2012 Text-CSV_XS-0.92/README rw-r--r-- 203/100 84 Nov 13 10:28 2012 Text-CSV_XS-0.92/PaxHeaders +.23664/META.json rw-r--r-- 203/20 1463 Nov 13 10:28 2012 Text-CSV_XS-0.92/META.json rw-r--r-- 203/100 90 Nov 13 10:28 2012 Text-CSV_XS-0.92/PaxHeaders +.23664/t rwxr-xr-x 203/20 0 Nov 13 10:28 2012 Text-CSV_XS-0.92/t/ rw-r--r-- 203/100 89 Jun 16 10:36 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/46_eol_si.t rw-r--r-- 203/20 5379 Jun 16 10:36 2012 Text-CSV_XS-0.92/t/46_eol_s +i.t rw-r--r-- 203/100 89 Jun 16 19:55 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/20_file.t rw-r--r-- 203/20 4341 Jun 16 19:55 2012 Text-CSV_XS-0.92/t/20_file. +t rw-r--r-- 203/100 89 Jun 14 20:13 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/76_magic.t rw-r--r-- 203/20 1925 Jun 14 20:13 2012 Text-CSV_XS-0.92/t/76_magic +.t rw-r--r-- 203/100 89 Jun 7 15:08 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/81_subclass.t rw-r--r-- 203/20 544 Jun 7 15:08 2012 Text-CSV_XS-0.92/t/81_subcl +ass.t rw-r--r-- 203/100 59 Jun 9 12:39 2007 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/util.pl rw-r--r-- 203/20 489 Jun 9 12:39 2007 Text-CSV_XS-0.92/t/util.pl rw-r--r-- 203/100 89 Jun 7 15:07 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/40_misc.t rw-r--r-- 203/20 1705 Jun 7 15:07 2012 Text-CSV_XS-0.92/t/40_misc. +t rw-r--r-- 203/100 89 Nov 13 10:17 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/70_rt.t rw-r--r-- 203/20 25550 Nov 13 10:17 2012 Text-CSV_XS-0.92/t/70_rt.t rw-r--r-- 203/100 89 Aug 27 13:39 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/22_scalario.t rw-r--r-- 203/20 4924 Aug 27 13:39 2012 Text-CSV_XS-0.92/t/22_scala +rio.t rw-r--r-- 203/100 87 Jun 16 19:56 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/21_lexicalio.t rw-r--r-- 203/20 4374 Jun 16 19:56 2012 Text-CSV_XS-0.92/t/21_lexic +alio.t rw-r--r-- 203/100 89 Jun 16 19:53 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/77_getall.t rw-r--r-- 203/20 1709 Jun 16 19:53 2012 Text-CSV_XS-0.92/t/77_getal +l.t rw-r--r-- 203/100 89 Jun 7 15:07 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/12_acc.t rw-r--r-- 203/20 5609 Jun 7 15:07 2012 Text-CSV_XS-0.92/t/12_acc.t rw-r--r-- 203/100 89 Jun 7 15:07 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/00_pod.t rw-r--r-- 203/20 175 Jun 7 15:07 2012 Text-CSV_XS-0.92/t/00_pod.t rw-r--r-- 203/100 89 Jun 7 15:08 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/60_samples.t rw-r--r-- 203/20 782 Jun 7 15:08 2012 Text-CSV_XS-0.92/t/60_sampl +es.t rw-r--r-- 203/100 89 Jun 7 15:07 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/01_pod.t rw-r--r-- 203/20 245 Jun 7 15:07 2012 Text-CSV_XS-0.92/t/01_pod.t rw-r--r-- 203/100 89 Jun 16 19:52 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/75_hashref.t rw-r--r-- 203/20 4404 Jun 16 19:52 2012 Text-CSV_XS-0.92/t/75_hashr +ef.t rw-r--r-- 203/100 89 Jun 16 10:50 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/65_allow.t rw-r--r-- 203/20 13126 Jun 16 10:50 2012 Text-CSV_XS-0.92/t/65_allow +.t rw-r--r-- 203/100 89 Jun 16 20:09 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/50_utf8.t rw-r--r-- 203/20 4244 Jun 16 20:09 2012 Text-CSV_XS-0.92/t/50_utf8. +t rw-r--r-- 203/100 89 Jun 16 10:30 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/41_null.t rw-r--r-- 203/20 2022 Jun 16 10:30 2012 Text-CSV_XS-0.92/t/41_null. +t rw-r--r-- 203/100 89 Jun 15 15:57 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/30_types.t rw-r--r-- 203/20 1702 Jun 15 15:57 2012 Text-CSV_XS-0.92/t/30_types +.t rw-r--r-- 203/100 88 Jun 7 15:07 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/51_utf8.t rw-r--r-- 203/20 3274 Jun 7 15:07 2012 Text-CSV_XS-0.92/t/51_utf8. +t rw-r--r-- 203/100 89 Jun 15 15:56 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/10_base.t rw-r--r-- 203/20 3920 Jun 15 15:56 2012 Text-CSV_XS-0.92/t/10_base. +t rw-r--r-- 203/100 89 Jun 7 15:08 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/55_combi.t rw-r--r-- 203/20 2333 Jun 7 15:08 2012 Text-CSV_XS-0.92/t/55_combi +.t rw-r--r-- 203/100 89 Jun 16 10:22 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/80_diag.t rw-r--r-- 203/20 4597 Jun 16 10:22 2012 Text-CSV_XS-0.92/t/80_diag. +t rw-r--r-- 203/100 89 Jun 16 10:30 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/45_eol.t rw-r--r-- 203/20 7711 Jun 16 10:30 2012 Text-CSV_XS-0.92/t/45_eol.t rw-r--r-- 203/100 89 Jun 7 15:07 2012 Text-CSV_XS-0.92/t/PaxHeade +rs.23664/15_flags.t rw-r--r-- 203/20 10320 Jun 7 15:07 2012 Text-CSV_XS-0.92/t/15_flags +.t rw-r--r-- 203/100 87 Aug 27 14:25 2012 Text-CSV_XS-0.92/PaxHeaders +.23664/CSV_XS.pm rw-r--r-- 203/20 56767 Aug 27 14:25 2012 Text-CSV_XS-0.92/CSV_XS.pm

That does not make me happy. I used GNU tar version 1.26, which is default on my OpenSUSE 12.2 and without it being documented, it produces the latest POSIX 1003.1-2001 (pax) format by default.

You won't notice if you have a "modern" tar (or ptar), but you will with tar versions that only support the older POSIX 1003.1-1988 (ustar) format.

With the generation of distribution well hidden behind ExtUtils::MakeMaker, Module::Build or whatever tool one prefers to use, what is the "best" way to attack this problem?

My choice might not be the best, but for me it is the simplest: set TARFLAGS in my Makefile.PL to force the older format like this (stripped for clarity):

use ExtUtils::MakeMaker;
WriteMakefile (
    ABSTRACT     => "Comma-Separated Values manipulation routines",
    VERSION_FROM => "CSV_XS.pm",
    macro        => { TARFLAGS   => "--format=ustar -c -v -f",
                    },
    );

I just hope this post will enlighten those that are bound to hit the same.


Enjoy, Have FUN! H.Merijn

Comment on Living on the edge
Select or Download Code
Re: Living on the edge
by Anonymous Monk on Nov 19, 2012 at 16:40 UTC

    With the generation of distribution well hidden behind ExtUtils::MakeMaker, Module::Build or whatever tool one prefers to use, what is the "best" way to attack this problem?

    A patch to the relevant module (like MakeMaker). If they can lower my MAXLINELENGTH they can add some ustar flags.

      I know the maintainer and signaled him about this, but I don't know about all/other builder modules. So I thought spreading this experience here might be useful to all.


      Enjoy, Have FUN! H.Merijn
Re: Living on the edge
by tobyink (Abbot) on Nov 20, 2012 at 13:04 UTC

    Another option would be to set the TAR_OPTIONS environment variable. The tar man page doesn't mention this possibility (at least it doesn't on my system) but it seems to work fairly universally.

    If you're using something like Module::Install which buries TARFLAGS under many layers, then a little line at the top of Makefile.PL may be use:

    BEGIN { $ENV{TAR_OPTIONS} = q{--format=ustar} };
    perl -E'sub Monkey::do{say$_,for@_,do{($monkey=[caller(0)]->[3])=~s{::}{ }and$monkey}}"Monkey say"->Monkey::do'
Re: Living on the edge
by mithaldu (Monk) on Oct 09, 2013 at 14:29 UTC
    So, i tried out downloading the files in various ways from these links:

    CPAN: http://search.cpan.org/CPAN/authors/id/H/HM/HMBRAND/Text-CSV_XS-1.01.tgz
    BackPAN: http://backpan.perl.org/authors/id/H/HM/HMBRAND/Text-CSV_XS-1.01.tgz

    Notably different browsers and sources resulted not only in different sizes, but also different filenames. The platform used for all cases is Windows 7. The results are as follows:

    Browser CPAN Name CPAN Size BPAN Name BPAN Size
    wget Text-CSV_XS-1.01.tgz 123288 Text-CSV_XS-1.01.tgz 123288
    Internet Explorer 10 Text-CSV_XS-1.01.gz 123298 (+10) Text-CSV_XS-1.01.tar 123288
    Opera 12 Text-CSV_XS-1.01.tgz 123288 Text-CSV_XS-1.01.tgz 123288
    Firefox 24 Text-CSV_XS-1.01.tgz 123298 (+10) Text-CSV_XS-1.01.tgz 123288
      The platform used for all cases is Windows 7

      You don't have to be on Windows to be bitten by this browser mismanagement.
      Iceweasel on Debian wheezy and Firefox on Ubuntu-12.04LTS also throw in the extra 10 bytes when downloading Text-CSV_XS-1.01.tgz from CPAN.

      (Sorry - I don't know the version numbers of those 2 browsers - and couldn't quickly find out how to get them.)

      Cheers,
      Rob

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://1004571]
Approved by marto
Front-paged by Arunbear
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (13)
As of 2014-07-29 14:10 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (217 votes), past polls