Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

why IPC::Open3 can't execute MS-DOS "dir" command?

by kiinoo (Novice)
on Dec 04, 2009 at 11:26 UTC ( #811057=perlquestion: print w/ replies, xml ) Need Help??
kiinoo has asked for the wisdom of the Perl Monks concerning the following question:

I'm using open3 method to run a dos command 'DIR', it stops here:
open3($rdr, \*POut, \*PErr, $cmd);
Any one knows what's the problem? Thanks very much!
#!/usr/bin/perl -w use strict; use File::Find; use File::Copy; use Time::Local; use IPC::Open3; my $log="TestOpen3.log"; open FHO, ">$log"; select FHO; &executeCmd("dir * \/b\/s"); close FHO; exit; sub executeCmd{ my ($cmd) = @_; print "Execute Command: $cmd\n"; use Symbol 'gensym'; my $rdr = gensym; my $pid = open3($rdr, \*POut, \*PErr, $cmd); waitpid( $pid, 0 ); if((my $returnCode = $? >> 8) !=0){ print "[ERROR] ($returnCode) : "; while(<PErr>) { print; } close PErr; die $returnCode; } while(<POut>) { print "\t".$_; } close POut; }

Comment on why IPC::Open3 can't execute MS-DOS "dir" command?
Select or Download Code
Re: why IPC::Open3 can't execute MS-DOS "dir" command?
by moritz (Cardinal) on Dec 04, 2009 at 11:40 UTC
    I'm not a Windows user, so forgive my stupid question: Is there a dir.exe or dir.com binary? If not, it might be a shell builtin, and thus not be available for launching externally.

    Anyway, for file listings you can also use glob or opendir, readdir, closedir or File::Find (or File::Find::Rule, which has the more convenient interface for newcomers).

      nah, system tries to use cmd /c if it doesn't work without.
        Curious, is that documented somewhere?

        system mentions usage of the shell only for the case when there is a shell meta character in the command.

        perlport mentions that there might be commands which are invokable but don't exists as files, but doesn't mention that perl makes extra effort to makes them invocable.

Re: why IPC::Open3 can't execute MS-DOS "dir" command?
by bingos (Vicar) on Dec 04, 2009 at 11:44 UTC

    Very likely because dir isn't actually a command in and of itself, but is a built-in command of cmd.exe

    Change "dir * \/b\/s" to be "cmd /C dir * \/b\/s"

    You also shouldn't call subroutines with the & prefix, unless you know the behaviour that implies

      Change "dir * \/b\/s" to be "cmd /C dir * \/b\/s"
      Seems it can't work, got the following error:
      Execute Command: cmd /c dir * /b/s [ERROR] (1) : Parameter format not correct - "s"".
      And if I remove the parameters "/b/s" it stopped again.
Re: why IPC::Open3 can't execute MS-DOS "dir" command?
by ikegami (Pope) on Dec 04, 2009 at 13:07 UTC

    You have a race condition.

    The child is waiting for the pipe to empty so it can print out more.

    The parent is waiting for the child to end before emptying the pipe.

      That might be the problem, do you know how can I solve it? thanks
        Yes, don't launch a program to get a directory listing. Two other posts in this thread discuss alternatives: moritz's and afoken's
Re: why IPC::Open3 can't execute MS-DOS "dir" command?
by zentara (Archbishop) on Dec 04, 2009 at 13:14 UTC
    Any one knows what's the problem?

    ... yes...since you are trying to run the msdos dir command, i assume you are on win32 and not an msdos emulator under linux;-)

    IPC::Open3 dosn't work on win32... see IPC::Open3 failure on Win32 for alternatives.....IIRC win32 dosn't have an easy select on pipes..... and IPC::Open3 requires that..... but maybe windows7 will correct that shortcoming


    I'm not really a human, but I play one on earth.
    Old Perl Programmer Haiku

      If passed the ends of socket pairs instead of unopened handled, you could use open3 on Windows. I think the following will allow you to use select on Windows too:

      use Socket qw( AF_INET SOCK_STREAM PF_UNSPEC ); use IPC::Open3 qw( open3 ); sub _pipe { socketpair($_[0], $_[1], AF_INET, SOCK_STREAM, PF_UNSPEC) or return undef; shutdown($_[0], 1); # No more writing for reader shutdown($_[1], 0); # No more reading for writer return 1; } sub _open3 { local (*TO_CHLD_R, *TO_CHLD_W); local (*FR_CHLD_R, *FR_CHLD_W); local (*FR_CHLD_ERR_R, *FR_CHLD_ERR_W); if ($^O =~ /Win32/) { _pipe(*TO_CHLD_R, *TO_CHLD_W ) or die $!; _pipe(*FR_CHLD_R, *FR_CHLD_W ) or die $!; _pipe(*FR_CHLD_ERR_R, *FR_CHLD_ERR_W) or die $!; } else { pipe(*TO_CHLD_R, *TO_CHLD_W ) or die $!; pipe(*FR_CHLD_R, *FR_CHLD_W ) or die $!; pipe(*FR_CHLD_ERR_R, *FR_CHLD_ERR_W) or die $!; } my $pid = open3('>&TO_CHLD_R', '<&FR_CHLD_W', '<&FR_CHLD_ERR_W', @ +_); return ( $pid, *TO_CHLD_W, *FR_CHLD_R, *FR_CHLD_ERR_R ); } my ($pid, $to_chld, $fr_chld, $fr_chld_err) = _open3(...);

        See also Win32::Socketpair


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
        Got an error like this:
        Unknown error at C:\811150.pl line 18
        use Socket qw( AF_INET SOCK_STREAM PF_UNSPEC ); use IPC::Open3 qw( open3 ); sub _pipe { socketpair($_[0], $_[1], AF_INET, SOCK_STREAM, PF_UNSPEC) or return undef; shutdown($_[0], 1); # No more writing for reader shutdown($_[1], 0); # No more reading for writer return 1; } sub _open3 { local (*TO_CHLD_R, *TO_CHLD_W); local (*FR_CHLD_R, *FR_CHLD_W); local (*FR_CHLD_ERR_R, *FR_CHLD_ERR_W); if ($^O =~ /Win32/) { _pipe(*TO_CHLD_R, *TO_CHLD_W ) or die $!; _pipe(*FR_CHLD_R, *FR_CHLD_W ) or die $!; _pipe(*FR_CHLD_ERR_R, *FR_CHLD_ERR_W) or die $!; } else { pipe(*TO_CHLD_R, *TO_CHLD_W ) or die $!; pipe(*FR_CHLD_R, *FR_CHLD_W ) or die $!; pipe(*FR_CHLD_ERR_R, *FR_CHLD_ERR_W) or die $!; } my $pid = open3('>&TO_CHLD_R', '<&FR_CHLD_W', '<&FR_CHLD_ERR_W', @ +_); return ( $pid, *TO_CHLD_W, *FR_CHLD_R, *FR_CHLD_ERR_R ); } my $log="TestOpen3.log"; open FHO, ">$log"; select FHO; my ($pid, $to_chld, $fr_chld, $fr_chld_err) = _open3("cmd /c dir * \/b +\/s"); if((my $returnCode = $? >> 8) !=0){ print "[ERROR] ($returnCode) : "; while(<$fr_chld_err>) { print; } close $fr_chld_err; die $returnCode; } while(<$fr_chld>) { print "\t".$_; } close $fr_chld; close FHO;
      Yes, I'm on windows xp 32-bit.
Re: why IPC::Open3 can't execute MS-DOS "dir" command?
by afoken (Parson) on Dec 05, 2009 at 13:50 UTC

    Why do you take the pain to run an external program just to get a directory listing? And why do you add extra pain by using Microsofts bug-compatible command interpreter for that task? "If it hurts, stop doing it!"

    Perl has opendir, readdir, closedir, stat, and the -X functions that easily allow re-implementing any directory lister in perl. Without the overhead of parsing output that was designed to please the human eye, not to be parsed easily. Without the overhead of creating an external process. Without the problems finding the right external helper program and make it return the expected result.

    If you want to work with the files you wanted to get from running the dir builtin (no, it's not a standalone program, it's burried deep inside cmd.exe and another implementation with different behaviour is again burried deep inside command.com), you better have a look at File::Find::Rule, File::Find, and perhaps some other CPAN modules like File::Spec and File::stat.

    (Oh, and by the way: Unless you are still running Windows 9x or ME, there is no thing as "DOS" or even "MS-DOS". No NT-based Windows has an underlying DOS.)

    Alexander

    --
    Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)
      Thanks, Alexander. Actually I want to write an utility to run "DOS" commands on windows xp/vista, "dir" command is one of them. I would use other commands in the future, such as xcopy, ftp etc. Currently I'm truely going to output the files recursively in current folder.

        OK, then the first step is to learn about the different types of "DOS" commands. DOS and Windows have a long and ugly history, and you need to learn a little bit of that history if you want to do more than just pushing the mouse over the desktop. My ancient DOS manuals distinguished between three types of DOS commands:

        1. Internal commands, meaning commands built into command.com
        2. External COM commands, i.e. commands compiled and linked into the COM format (no header, fixed start offset, one shared segment for code and data, limited to max. 64k-256 bytes)
        3. External EXE commands, i.e. commands compiled and linked into the EXE format (header, variable start offset, multiple segments for code and data, nearly unlimited size)

        One day, the DOS developers decided to look at the file data instead of the file extension when running external commands, so it was possible to rename EXE files to *.COM and vice versa. This allowed them to compile and link commands as EXE files while still using a COM file extension for backwards compatibility.

        So, you really need to distinguish between internal and external commands.

        With Windows 2000, XP, and newer, things become a little bit complicated, they inherited a very similar, but still different command shell named cmd.exe from the NT line, which behaves different and has different build-in commands. Of course, each Windows version has its own version of cmd.exe, all with different features. cmd.exe is the default shell on 2000 and newer, but command.com is still there.

        dir, cls, echo and a few others are internal commands, i.e. you have to invoke them using command.com or cmd.exe. Typically, you prefix the desired command with command.com /c or cmd.exe /c, depending on the OS version and on which implementation of the command you want to execute.

        The majority of commands are external commands, most of then are EXE files, a few are renamed to *.COM for backwards compatibility. These commands can be executed directly, just like you would execute any other application (because that's what they are).

        The next DOS / Windows history lession is the command line. Unix shells expand, interpret and split the command line and pass an array of parameters to the invoked program. If you type foo *.txt *.asc into a Unix shell, the foo program is typically invoked with a list of all txt and asc files in the current directory. DOS and Windows shells do not expand or split the command line, instead, they pass it as a single string to the invoked program. If you type foo *.txt *.asc into a DOS or Windows shell, the foo program gets *.txt *.asc as a single command line string, and has to expand and split it in its very own way.

        Most DOS and Windows programs delegate that to the C runtime library (or equivalent), and so most DOS and Windows programs expand the command line in a very similar way. But there is no guarantee for that.

        Under DOS, the command line was limited to 126 bytes due to the COM file format, and sometimes, you still tun into this limit.

        The DOS / Windows shells still process the command line, e.g. they replace environment variables or remove redirections from the command line, so the command line is not passed completely unmodified to the invoked program. The rules to prevent this from happening (i.e. quoting) differ from shell to shell, and from OS version to OS version

        So invoking the DOS shell really begs for trouble. (Not that unix shells are paradise, their quoting rules are a little bit simplyer, but still differ from shell to shell.) Obviously, avoiding the shell as much as possible prevents problems. For exec and system, this means to use the multi-argument form. When you call exec or system with a single argument, perl may pass that argument to a random shell, leaving you with a lot of problems.

        On Windows, things become again problematic here, as there is no way to pass an array of parameters to the invoked program. Perl has to generate a single command line string from your arguments. And you have to hope that perls algorithm to construct the string is compatible with the invoked program's algorithm to split it again into an array.

        I/O redirection is once again problematic on Windows. On Unix, you fork, modify I/O and environment as needed in the forked copy of the current process, and then execute the real child, which inherits I/O redirection, environment and so on. On Windows, you have to do all this when creating a new process using the CreateProcess API function, and for more tricks, you need the CreateProcessEx API function. (I wonder when Microsoft will reach CreateProcessExExExEx ;-)) Don't worry, perl.exe will handle most of this for you. But it is not perfect.

        So, your really best bet on Windows is NOT to use external programs at all if you can avoid it.

        You do not need dir, as I've explained before. copy and xcopy can be replaced by File::Copy and File::Copy::Recursive. For ftp, you have a load of CPAN modules, like Net::FTP, LWP::Simple, LWP::UserAgent. cd won't work at all, but you can use chdir and Cwd.

        For many other commands, you will find superiour perl replacements, either build-in (e.g., the DOS find command is a crippled grep) or on CPAN. Only for very low-level access (fdisk, format, network config), you need either an external command or access to the Windows API. Often, Win32::API can be helpfull, sometimes DBD::WMI works, and if you are lucky, someone has already written a module and published it on CPAN, typically named Win32::Something.

        Alexander

        --
        Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://811057]
Approved by bingos
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (6)
As of 2014-12-28 05:17 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (178 votes), past polls