Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Pure Perl or the toolkit?

by Abigail-II (Bishop)
on Aug 20, 2003 at 14:15 UTC ( [id://285202]=perlmeditation: print w/replies, xml ) Need Help??

Every now and then, there is a discussion whether to shell out to do something, or do it all in Perl. Many purists argue that if it's possible to do it in Perl, you should do it in Perl. Portability and such. And specially if the functionality comes in a standard module.

I got bitten by that today. I was working on a few programs that, in combination with make create a floppy disk to install a small Linux system on a specialized box. Part of the process involves copying executables to the root file system to be. Simple, isn't? Just do

use File::Copy; umask 022 or die "Failed to set umask: $!"; copy $src => $dest or die "Failed to copy: $!";

Well, that was what I did. Later, I needed to copy a directory structure. According to the manual page of File::Copy, that module just deals with copying files, so I shelled out:

system cp => '-a', $src, $dest; die "..." if $?;

But back to File::Copy and copy. The floppy disk was created, I insert it in the box, reboot, and after some time, it fails. /sbin/loader is not executable.

It turns out that File::Copy doesn't respect execution bits. Urg. Out went File::Copy and back came shelling out to the UNIX toolkit.

Here's a program that shows the unwanted behaviour of File::Copy:

#!/usr/bin/perl use strict; use warnings; use File::Copy; umask 022 or die "umask: $!"; my $one = "/tmp/one"; my $two = "/tmp/two"; my $three = "/tmp/three"; foreach my $file ($one, $two, $three) { ! -f $file or unlink $file or die "unlink $file: $!" } # Create first file. open my $fh, "> $one" or die "one: $!"; close $fh; # Make it executable. chmod 0755 => $one or die "chmod: $!"; # Copy it with File::Copy. copy $one, $two or die "copy: $!"; # Copy using the UNIX toolkit. system cp => $one, $three; die "cp: ", $? >> 8 if $?; # And check. print "$one is executable\n" if -x $one; print "$two is executable\n" if -x $two; print "$three is executable\n" if -x $three; __END__ /tmp/one is executable /tmp/three is executable

Abigail

Replies are listed 'Best First'.
Re: Pure Perl or the toolkit?
by arthas (Hermit) on Aug 20, 2003 at 14:33 UTC

    Hi!

    It seems that under Unix it's not possible to preserve file attributes using File::Copy. The documentation somehow explains that:

    File::Copy also provides the "syscopy" routine, which copies the file specified in the first parameter to the file specified in the second parameter, preserving OS-specific attributes and file structure. For Unix systems, this is equivalent to the simple "copy" routine, which doesn't preserve OS-specific attributes. For VMS systems, this calls the "rmscopy" routine (see below). For OS/2 systems, this calls the "syscopy" XSUB directly. For Win32 systems, this calls "Win32::CopyFile".

    So, there's syscopy, but it won't do anything different from "copy" useful under Unix. So, it might not be possible to go with the pure perl solution this time...

    Michele.

Re: Pure Perl or the toolkit?
by dbwiz (Curate) on Aug 20, 2003 at 15:36 UTC

    So File::Copy does not preserve the attributes of the copied file. Can't we fix it?

    package NewCopy; use strict; use warnings; use File::Copy; require Exporter; our @ISA = qw(File::Copy); our @EXPORT = qw(copy move); our @EXPORT_OK = qw(cp mv); sub cp; sub mv; *cp = \© *mv = \&move; no warnings; eval { sub copy{ my ($attr) = (stat( $_[0] ))[2] or return -1; return (File::Copy::copy(@_) && (chmod ($attr, $_[1]) ); } sub move { return File::Copy::move(@_) } }; 1;

    Replace "use File::Copy" with "use NewCopy" in your script and it will give

    /tmp/one is executable /tmp/two is executable /tmp/three is executable
      So File::Copy does not preserve the attributes of the copied file. Can't we fix it?

      This is a band-aid that more closely approximates the desired behavior, but it will always be a chasing game of approximation (until the Unix/Linux systems provide a basic syscopy feature).

      Say you implement this. Then you want to copy files which have ACLs (Access Control Lists) attached. Now you have to fix File::Copy again to copy the ACLs too. Tomorrow, what other file metadata will be employed at the filesystem level?

      --
      [ e d @ h a l l e y . c c ]

        Say you implement this. Then you want to copy files which have ACLs (Access Control Lists) attached. Now you have to fix File::Copy again to copy the ACLs too. Tomorrow, what other file metadata will be employed at the filesystem level?

        IMHO, it doesn't make sense to worry about tomorrow problems if you don't know what they will be (well, at least with a fair amount of probability). <AFAIK>ACLsare still different from OS to OS, and somewhere they can be considered experimental;</AFAIK> it is a moving target, that one can plan to address only where it is stable enough.

        By the way, UNIX file permissions are there from the start, on every flavour of UNIX, and that problem on File::Copy should be addressed...

        Ciao!
        --bronto


        The very nature of Perl to be like natural language--inconsistant and full of dwim and special cases--makes it impossible to know it all without simply memorizing the documentation (which is not complete or totally correct anyway).
        --John M. Dlugosz
      And after that, we will make a NewerCopy.pm that also take umask into consideration (which is what cp does)? And after that, we make a EverNewerCopy which supports the -p option?

      My point wasn't that File::Copy needs fixing. My point is that the choice between shelling out to do something and using a Perl solution isn't as simple as it is sometimes suggested, and that people should be trigger happy and reply "Why don't you use Perl for that" if someone shows code where shell commands are being used.

      Abigail

Re: Pure Perl or the toolkit?
by dragonchild (Archbishop) on Aug 20, 2003 at 14:30 UTC
    From reading the source, it looks like it didn't take permissions into account at all, depending completely on the way open sets the permissions. Has the author been contacted?

    ------
    We are the carpenters and bricklayers of the Information Age.

    The idea is a little like C++ templates, except not quite so brain-meltingly complicated. -- TheDamian, Exegesis 6

    Please remember that I'm crufty and crochety. All opinions are purely mine and all code is untested, unless otherwise specified.

Re: Pure Perl or the toolkit?
by chunlou (Curate) on Aug 20, 2003 at 19:02 UTC

    Everyone could get caught off guard on things he's not aware of once a while, be it pure or toolkit. Regex bites people on a regular basis.

    Whether one writes in pure Perl for the sake of portability and such is a bit like a quality control or risk management question.

    Using your case, File::Copy might seem less advicable to a sysadmin working on a multi-OS environment than someone who just uses it for productivity tool on his own desktop. The former is more likely to get bitten than the latter and therefore "pure Perl" will be a more immediate concern.

    For the latter, who rarely works on other OSs or even other machines, worrying about portablity and such is like buying a typhoon damage insurance when he's living in Canada.

    VB programmers apparently don't concern themselves with portability and such issues, as they apply themselves to a restricted enough environment. On the other hand, a Java programmer may need to be concerned about the issues.

    Whether someone should be writing in pure Perl partly depends on the risk for him for not doing so. If the risk is low, one might feel, why spend the extra time on the "5%" when he has covered the "95%"; it might seem counterproductive to spend the extra time to deal with things that almost never occurs to him. However, when the risk is high and someone ignores it, it's reckless.

    So, cliche as it sounds, pure Perl vs toolkit is not a binary choice but more like a calculated bet. If a toolkit save someone more time than the time he fixing it, why not use a toolkit? But if the reverse is true for him, long live pure Perl. (Of course, sure many other people will look at the issue from other perspectives as well. It's a healthy thing.)

      Are you suggesting I'm wasting my money on typhoon insurance??? EXCELLENT!!! I can drop that and afford to keep my-suddenly-expensive blackout policy....
Re: Pure Perl or the toolkit?
by liz (Monsignor) on Aug 20, 2003 at 15:18 UTC
    Hmmm...File::Copy says:
    File::Copy also provides the syscopy routine, which copies the file specified in the first parameter to the file specified in the second parameter, preserving OS-specific attributes and file structure. For Unix systems, this is equivalent to the simple copy routine, which doesn't preserve OS-specific attributes (ed: emphasis added). For VMS systems, this calls the rmscopy routine (see below). For OS/2 systems, this calls the syscopy XSUB directly. For Win32 systems, this calls Win32::CopyFile.

    At least it was put in the small print ;-(

    Liz

Re: Pure Perl or the toolkit?
by bluto (Curate) on Aug 20, 2003 at 17:35 UTC
    I've ran into so many problems copying files (esp during a 'make') that I now do the equivalent of the following for each executable ...
    rm -f dest cp src dest chmod ugo+rx dest
    Which handles cases where dest already exists, src and/or dest are owned by someone else and may already have wrong permissions set, or dest is already in use (and some OSs complain if you try to overwrite it). This isn't perfect (esp for NFS) so YMMV.

    The problem I have with using pure perl to replace common Unix commands is that I need to spend the time and test to make sure my assumptions are correct about how they work and I need to be able to spend the time to pursue a fix/workaround if they don't work like I expect. If the module is not in Perl's core I need to go to additional trouble (e.g. Do I really want to spend the effort on a module at version 0.01 which hasn't been upgraded in 3 years?).

    bluto

Re: Pure Perl or the toolkit?
by bm (Hermit) on Aug 21, 2003 at 10:17 UTC
    ...shell out to do something, or do it all in Perl. Many purists argue that if it's possible to do it in Perl, you should do it in Perl. Portability and such.

    I am not sure it is possible to argue that one should *generally* use a pure Perl solution, or *generally* shell out.
    It's on a case by case basis.

    For me, I suppose the deciding factor is portability requirements.

    If the code needs to run in many different environments, such as CPAN module, then I would use pure Perl as much as possible. Definitely if it is possible in the core, less likey if a non-core modules is required. The result is probably quicker execution time.

    If my production environment is limited to a certain type, as in your specific example, where the environment may be assumed, then I see nothing wrong with using the toolkit. The result is probably quicker implementation time.
    --
    bm

Re: Pure Perl or the toolkit?
by crouchingpenguin (Priest) on Aug 21, 2003 at 13:24 UTC

    Since you're already using make, why not use install as well from within the Makefile? install should be available on most any system that has some form of make avaible. That seems to satisfy the required simplicity as well as being fairly portable. Instead of worrying about whats missing in File::Copy, just add an install call to your make target like so:

    FILES=/tmp/one /tmp/two /tmp/three DEST=/tmp/mounted_image install: install -m 0755 $(FILES) $(DEST)

    cp
    ----
    "Never be afraid to try something new. Remember, amateurs built the ark. Professionals built the Titanic."
      Because it has nothing to do with installing files on a final destination. It happens in the middle of a script that creates an image, creates a file system on the image, mounts it, creates device files, creates and copies other files, creates links, finds and installs kernel modules installs busybox, does a handful of other things, and then umounts the file system and compresses the image. By the time the script is finished, no destination to copy to exists.

      Sure, I could drive the entire thing with make, and lots of things are already done in pure make. But then I would have to start parsing configuration files as well. Not impossible, but sometimes, it's easier to use Perl.

      Abigail

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlmeditation [id://285202]
Approved by broquaint
Front-paged by jeffa
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others cooling their heels in the Monastery: (3)
As of 2024-03-19 11:14 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found