Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask
 
PerlMonks  

Re: Re: system, pipes, shell, quoting

by superpete (Beadle)
on Nov 13, 2002 at 05:22 UTC ( #212492=note: print w/replies, xml ) Need Help??


in reply to Re: system, pipes, shell, quoting
in thread system, pipes, shell, quoting

$file1 = s/(\W)/\\$1/; # and similarly for other filenames

unfortunately, this doesn't seem to work. Here is an example :-(

foreach $d ( @dirs ) { -e $d or die; # works -d $d or die; # works $d =~ s/(\W)/\\$1/g; system("touch $d"); # touch can't find it, it prints an error!!! }

"open" will most likely do the same thing. The problem is the basic act of quoting arbitrary unprintable characters to the shell (nevermind security for the time being). FYI, the shell is /bin/sh on FreeBSD 4.1, which I assume is pretty solid.

Replies are listed 'Best First'.
Re: Re: Re: system, pipes, shell, quoting
by graff (Chancellor) on Nov 13, 2002 at 06:14 UTC
    "open" will most likely do the same thing

    (ahem) You mean you won't even try it? Pull down the "shloop" utility I referred to earlier, and try this (I just did):

    echo '?*;&'
    (just to convince you that using single-quotes around a nasty string will help for some cases -- so long as the string doesn't include any single-quote characters) And then try this:
    mkdir /tmp/junk echo '?*;&' | shloop -e "touch '/tmp/junk/\i'" ls -l /tmp/junk
    If that doesn't work for you, please let me know. I don't have access to a FreeBSD system, and I'd be interested to learn how different it could be from linux & solaris (where this works).

    BTW, if your list of potential file names does happen to involve cases that include one or more quote-like characters, then you're right -- the above will not work. In such cases, let me suggest that you should only be facing this issue with existing file names that are created by other processes and are to be used as input to a given pipeline -- don't ever create such file names yourself for output. To handle these, open the the nasty-named file for input in perl (get the file name via "readdir" so you don't need to present such a name as a command-line arg -- who knows, maybe a file name could include a "\n"!), then open your pipeline as a file handle, and pass file data to the pipeline that way. E.g.:

    opendir( DIR, "some_dir" ); @files = grep /[^.]/, readdir( DIR ); # skips "." and ".." # (assumes you never need to look for files named "...") foreach $file ( @files ) { next unless ( -f $file ); $outfile = "something_sane"; open( PIPE, "| $prog1 | $prog2 -x -y | $prog3 > $outfile" ); open( IN, "<", $file ); while (<IN>) { print PIPE; } close PIPE; close IN; # and/or unlink or rename that input file so it's less of a bother hen +ceforth }
    I haven't tried that yet (but I think you should try it yourself before you say it won't work... ;^).
      BTW, if your list of potential file names does happen to involve cases that include one or more quote-like characters, then you're right -- the above will not work. In such cases, let me suggest that you should only be facing this issue with existing file names that are created by other processes and are to be used as input to a given pipeline -- don't ever create such file names yourself for output. To handle these, open the the nasty-named file for input in perl (get the file name via "readdir" so you don't need to present such a name as a command-line arg -- who knows, maybe a file name could include a "\n"!), then open your pipeline as a file handle, and pass file data to the pipeline that way.

      Good point... This is probably the strategy I will adopt. It still bothers me a little bit that you are so completely at the mercy of the shell in the general case of this situation.

        To clarify, when I wrote

        It still bothers me a little bit that you are so completely at the mercy of the shell...

        I meant "you" in the general sense, meaning me :-)
        I didnt mean to be offensive in any way.

      This is offtopic so i'll reply to it first :-)

      I don't have access to a FreeBSD system, and I'd be interested to learn how different it could be from linux & solaris

      Because bourne shell is so important, the behavior should be identical (POSIX), but I can't swear to it. I've encountered /bin/sh scripts containing some bash-specific syntax (i.e., poorly written shell scripts). These worked on linux but not FreeBSD because FreeBSD's /bin/sh was more anal about POSIX correctness.

      Just so you know, my test data really does contain EVERY conceivable character. For testing, the filenames are a random mix of the characters:
      chr(rand(128))
      except for "/" and "\0"
        my test data really does contain EVERY conceivable character

        That's swell. So you need to cope with the residue of processes that create files whose names consist of randomly selected byte values? (Why stop at 128?) I hope that your execution path doesn't include directories or programs with such liberal names -- that could be a very awkward system to work with...

        Given such an environment, I'd be inclined to focus on a means to rename files that have troublesome characters in their names -- e.g. using readdir, locate each data file whose name would match /[^-\w\$\@\%\#.,:+~=]/, invent a new name for each such file using only sensible characters, perhaps create a suitable table that documents the original "name" and the newly-assigned name, and rename the file before you do anything else with it. Creating and assigning distinct, usable names is easy. Don't even start to worry about how to run a shell command on file names that contain control characters and whatnot.

        update: If renaming nasty files is not an option (due to permissions or politics), you could use the "symlink()" function instead, creating a nice name for accessing the file without altering the original directory entry. The symlink wouldn't need to be in the same place as the file (could even be on /tmp, and last only as long as needed to run system("long | pipe | command")

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://212492]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (3)
As of 2021-02-26 01:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?