http://www.perlmonks.org?node_id=291299

Mr. Muskrat has asked for the wisdom of the Perl Monks concerning the following question:

I am attempting to redirect STDERR to a scalar. However, I'm using ActiveState Perl v5.6.1 (MSWin32-x86-multi-thread ) build 635. The only way that I found to work (so far anyway) is as follows:

  1. Save a copy of STDERR.
  2. Redirect it to a temporary file.
  3. Run the process that only outputs on STDERR.
  4. Restore the original STDERR.
  5. Slurp the contents of the temporary file into a scalar.

I have stripped down my code greatly to produce the following example script. Both &oldway and &newway work fine but which one is the best way of doing this?

use File::Temp 'tempfile'; print "Trying oldway...\n"; my $oldway = oldway(); # should not display anything print "Finished with oldway.\n"; print $oldway; print "Testing STDERR...\n"; print STDERR "(STDERR) can you hear me?\n"; print "Trying newway...\n"; my $newway = newway(); # should not display anything print "Finished with newway.\n"; print $newway; print "Testing STDERR...\n"; print STDERR "(STDERR) can you hear me now?\n"; sub oldway { my ($fh, $filename) = tempfile(UNLINK => 1); open(OLDERR, ">&STDERR") || die "Can't save STDERR, $!"; # save STDE +RR open(STDERR, '>', $filename) || die "Can't redirect STDERR to the te +mp file, $!"; select(STDERR); $| = 1; print STDERR "oldway says 'hi'\n"; open(STDERR, ">&OLDERR"); # restore the original STDERR select(STDOUT); my $oldway = slurp_file($filename); return $oldway; } sub newway { my ($fh, $filename) = tempfile(UNLINK => 1); *oldfh = select STDERR; # save STDERR open(STDERR, '>', $filename) || die "Can't redirect STDERR to the te +mp file, $!"; select(STDERR); $| = 1; print STDERR "newway says 'hi'\n"; open(STDERR, ">&oldfh"); # restore STDERR select(STDOUT); my $newway = slurp_file($filename); return $newway; } sub slurp_file { my $filename = shift; my $slurp; open(TXT, '<', $filename) || die "Can't read the file: $filename, $! +"; { local $/ = undef; $slurp = <TXT>; # slurp! } close(TXT); return $slurp; } __DATA__ Trying oldway... Finished with oldway. oldway says 'hi' Testing STDERR... (STDERR) can you hear me? Trying newway... Finished with newway. newway says 'hi' Testing STDERR... (STDERR) can you hear me now?

Replies are listed 'Best First'.
Re: Best Way to Redirect STDERR to a Scalar
by davido (Cardinal) on Sep 13, 2003 at 22:35 UTC
    The first method I would propose is based on a technique I read in the Perl 5.8.0 perldoc perlfunc manpage for open. The trick discussed therein is to close the old STDERR before reopening it, redirected to a scalar.


    UPDATE:
    I'm sorry to say that this proposed method works only under 5.8.0 or later, as expressed in the perldoc perldelta manpage, "File handles can be opened to "in memory" files held in Perl scalars via open($fh,'>',\$variable) || ...."

    However, all is not lost. Read on to my second update to see how to do it in any version using the module Tie::STDERR.


    my $variable; # First, save away STDERR open SAVEERR, ">&STDERR"; close STDERR; open STDERR, ">", \$variable or die "What the hell?\n"; # Now print something to STDERR, redirected to $variable print STDERR "This is a test.\n"; # Now close and restore STDERR to original condition. close STDERR; open STDERR, ">&SAVEERR"; # Now test to see if $variable actually received the text. print "Now for the real test.\n"; print $variable;

    That ought to do the trick. ...at least under 5.8.0+. It gets even more fun when you use it to redirect STDOUT to a scalar. It's too bad it didn't exist prior to 5.8.0.


    UPDATE:

    Under pre-5.8.0 versions of Perl (or post-5.8.0 too if you wish, though it becomes unnecessary), you could use the module: Tie::STDERR

    use Tie::STDERR \$append_to_scalar;

    From that point on, STDERR will be routed to the scalar. The module can also be used to redirect STDERR to email, or to a function that is called upon program exit. If you look into the module's internals you see that 'tie' is being used to tie *STDERR to the package named Tie::STDERR, and then the module is holding onto anything that would have been writen to STDERR and appending it to the scalar you supply in the use Tie::STDERR .....; line.


    I hope this helps for you too.

    Dave

    "If I had my life to do over again, I'd be a plumber." -- Albert Einstein

      Unfortunately Tie::STDERR won't do what's needed.   There's this note in the POD:
      However, if you run external command (system, ``), stderr output from that process won't be caught.   It is because we tie Perl's STDERR fileglob, not the external filehandle.
      (grumble - sounded wonderful tho)
        I don't know that it's correct or safe to jump to the conclusion that Tie::STDERR won't do what's needed, as the OP didn't specify in his original post that he was trying to catch errors that come from executing external commands.

        Tie::STDERR will allow you to redirect any errors that come from Perl or Perl commands, but not from external (such as system() or `` backticks). It will even catch Perl's compiler errors, though as the POD states, the errors that it reports from the compiler aren't what you might expect to see.

        So if you're trying to redirect the output from any Perl-generated errors, or errors that you generate internally in your script, Tie::STDOUT works great. But if you want to redirect external command errors, you'll have to dig deeper.

        Dave

        "If I had my life to do over again, I'd be a plumber." -- Albert Einstein

Re: Best Way to Redirect STDERR to a Scalar
by Fletch (Bishop) on Sep 13, 2003 at 22:27 UTC

      I should have mentioned that I tried IO::Scalar. The following code causes a page fault in perl56.dll.

      use IO::Scalar; print "Trying ioscalar...\n"; my $io = ioscalar(); print "Finished with ioscalar.\n"; print $io; print "Testing STDERR...\n"; print STDERR "(STDERR) can you hear me now?\n"; sub ioscalar { my $data; *oldfh = select STDERR; # save STDERR print "STDERR saved.\n"; my $SH = new IO::Scalar \$data; print "\$SH created.\n"; open(STDERR, '>', $SH) || die "Can't redirect STDERR to the IO::Scal +ar, $!"; print "STDERR redirected to \$SH\n"; select(STDERR); $| = 1; print STDERR "ioscalar says 'hi'\n"; close(STDERR); open(STDERR, ">&oldfh"); # restore STDERR select(STDOUT); return $data; } __DATA__ Trying ioscalar... STDERR saved. $SH created.

        I am curious to see if the following works for you or not:
        use strict; use warnings; use IO::Scalar; my $str; my $err = tie *STDERR, 'IO::Scalar', \$str; print STDERR "captured\n"; undef $err; untie *STDERR; print STDERR "not captured\n";
        UPDATE:
        whoops, this is exactly what shenme already said. My apologies.

        jeffa

        L-LL-L--L-LL-L--L-LL-L--
        -R--R-RR-R--R-RR-R--R-RR
        B--B--B--B--B--B--B--B--
        H---H---H---H---H---H---
        (the triplet paradiddle with high-hat)
        
Re: Best Way to Redirect STDERR to a Scalar
by shenme (Priest) on Sep 14, 2003 at 01:17 UTC
    Perhaps the way you were testing redirecting STDERR confused other responders.   (Then again you may have confused me - see davido's response.)   If you want to capture the STDERR output of another program that you start it gets differently complicated than just playing with the STDERR of your own program.

    The various methods of TIE'ing STDERR won't work as they don't affect the STDERR filehandle (or more particularly the file descriptor ala fileno(STDERR)).   And you can play with open's using '>&STDERR' but that's still playing only with the running program.

    On an *nix you could use fork and capturing output from child processes or any of the neat ways of doing the same using the open forms.   But since you mention ActiveState we should probably try for less complicated ways of doing things.   Would the code below work for you?   Namely, can you execute the program using the backtick operator?

    # see qx/STRING/ in perlop, $? in perlvar, perlvar#error indicators my( @output, $status ); ($status, @output) = do_this_cmd( 'net use' ); ($status, @output) = do_this_cmd( 'net arf' ); ($status, @output) = do_this_cmd( 'netz arf' ); sub do_this_cmd { my $the_cmd = shift; my( @the_output, $error_status ); @the_output = qx($the_cmd 2>&1); $error_status = $? / 256; # Peek at arguments and returned values while debugging if(1){ printf "\nExecuted command '%s'\n", $the_cmd; printf "Process returned error status '%d'\n", $error_status if $error_status; if( @the_output ) { foreach (@the_output) { print " >>> ", $_; } } else { print "Process didn't return any output!\n"; } } ( $error_status, @the_output ); }
    The outputs returned from the tests (on my system) were:
    
    Executed command 'net use'
      >>> New connections will not be remembered.
      >>>
      >>>
      >>> Status       Local     Remote                    Network
      >>>
      >>> -------------------------------------------------------------------------------
      >>> Disconnected Z:        \\tls2ka\c$               Microsoft Windows Network
      >>> The command completed successfully.
      >>>
    
    Executed command 'net arf'
    Process returned error status '1'
      >>> The syntax of this command is:
      >>>
      >>>
      >>> NET [ ACCOUNTS | COMPUTER | CONFIG | CONTINUE | FILE | GROUP | HELP |
      >>>       HELPMSG | LOCALGROUP | NAME | PAUSE | PRINT | SEND | SESSION |
      >>>       SHARE | START | STATISTICS | STOP | TIME | USE | USER | VIEW ]
      >>>
    
    Executed command 'netz arf'
    Process returned error status '1'
      >>> 'netz' is not recognized as an internal or external command,
      >>> operable program or batch file.
    
    I used the net command as it prints normal output to STDOUT (first test) but error output to STDERR (second test) as does the OS (third test).
Re: Best Way to Redirect STDERR to a Scalar
by mandog (Curate) on Sep 13, 2003 at 22:44 UTC

    I know you are using Perl 5.6.1 but if you were using Perl 5.8...

    perldoc -f open #(snip) File handles can be opened to "in memory" files held in Perl scalars via: open($fh, '>', \$variable) or die "$!"; Though if you try to re-open "STDOUT" or "STDERR" as an "in memory" file, you have to close it first: close STDOUT; open STDOUT, '>', \$variable or die "Can't open STDOUT: $!";


    email: mandog
Re: Best Way to Redirect STDERR to a Scalar
by demerphq (Chancellor) on Sep 13, 2003 at 23:15 UTC

    Tie STDERR to a scalar using any one of a number of modules. However if you really are trying to trap warnings then you may want to trap $SIG{__WARN__} or overide warn. Personally I would just trap $SIG{__WARN__} unless I knew that the code I was dealing with also used $SIG{__WARN__}.


    ---
    demerphq

    <Elian> And I do take a kind of perverse pleasure in having an OO assembly language...
Re: Best Way to Redirect STDERR to a Scalar
by Mr. Muskrat (Canon) on Sep 14, 2003 at 01:35 UTC

    I'm trying to capture the output of Devel::Peek's Dump function which outputs to STDERR. The methods outlined in my original post work fine.

    I just want to know if there is a better way.

    I've tried IO::Scalar (see above).

    I've tried Tie::Handle::Scalar. It works but using it results in a warning 'Permission denied at C:/Perl/site/lib/Tie/Handle/Scalar.pm line 114.' I looked and that is the line that deletes the temporary file. I read further and I see that File::Temp provides a much safer means of using temporary files so T::H::S is out.

    Tie::STDERR is out mainly because I don't want to capture all messages sent to STDERR, only those generated by D::P's Dump function as mentioned above.

    I'm going to be trying IO::String & IO::Stringy later this evening.

    Any suggestions would be appreciated. Thank you.

      Okey-dokey, I can understand frustration with packages that don't allow you to (easily) capture their debugging output.   Please try the following which works on ActiveState 5.6.1 build 635 running on Windows XP.   Amazing, no open's, no close's, but all TIE'd up prettily anyway.
      my $data; print STDERR " STDERR test (1)\n"; tie *STDERR, 'IO::Scalar', \$data; print STDERR " STDERR test (2)\n"; untie *STDERR; print STDERR " STDERR test (3)\n"; printf " Scalar string has length %d\n", length($data); printf " and contains '%s'\n", $data;
      Outputs
       STDERR test (1)
       STDERR test (3)
        Scalar string has length 17
          and contains ' STDERR test (2)
      '
      

      Update: Drat, it works wonderfully until you hit submit, then you think of the case that doesn't work.   If I throw in a warn which presumably uses the filehandle more directly:

      tie *STDERR, 'IO::Scalar', \$data; print STDERR " STDERR test (2)\n"; warn " STDERR test (2a)"; untie *STDERR;
      I then see
       STDERR test (1)
       STDERR test (2a) at mrmskrat3.pl line 14.
       STDERR test (3)
        Scalar string has length 17
          and contains ' STDERR test (2)
      '
      
      showing that some output to 'STDERR' isn't going to be captured this way.   So it will very much depend on the way the package outputs to STDERR whether this will work.   Rats!

        I really thought this would do it but it did not. :-(

        use IO::Scalar; use Devel::Peek; print STDERR " STDERR test (1)\n"; my $io = ioscalar(); print STDERR " STDERR test (3)\n"; printf " Scalar string has length %d\n", length($io); printf " and contains '%s'\n", $io; print STDERR " STDERR test (4)\n"; my $peek = peek(); print STDERR " STDERR test (5)\n"; printf " Scalar string has length %d\n", length($peek); printf " and contains '%s'\n", $peek; sub ioscalar { my $data; tie *STDERR, 'IO::Scalar', \$data; print STDERR " STDERR test (2)\n"; untie *STDERR; return $data; } sub peek { my $data; my $test = 'string'; tie *STDERR, 'IO::Scalar', \$data; Dump($test); untie *STDERR; return $data; } __DATA__ STDERR test (1) STDERR test (3) Scalar string has length 17 and contains ' STDERR test (2) ' STDERR test (4) SV = PV(0x177eb58) at 0x1cbc540 REFCNT = 1 FLAGS = (PADBUSY,PADMY,POK,pPOK) PV = 0x1773c74 "string"\0 CUR = 6 LEN = 7 STDERR test (5) Scalar string has length 0 and contains ''

Re: Best Way to Redirect STDERR to a Scalar
by Roger (Parson) on Sep 14, 2003 at 03:05 UTC
    The IO::Scalar module is one way to redirect STDERR to a scalar. There is another method for redirecting STDERR by using the IO::Capture module, and more specifically, by using the IO::Capture::Stderr module.

    use IO::Capture::Stderr; my $capture = new IO::Capture::Stderr; $capture->start(); # ... print anything to STDERR ... print STDERR "Test 1\n"; print STDERR "Test 2\n"; $capture->stop(); # To map everything into an array -- my @messages = $capture->read;
    The CPAN site offers detailed description of this module.
      While attractive, this is just one more way of tieing the STDERR glob.   Because this nor any of the other tie methods 'really' hook into the filehandle used by STDERR they don't help capture the low-level references to STDERR (aka fileno(STDERR)==2) done by code like Devel::Peek.

      Tested with the following code it fails the "warn test" and testing versus Mr. Muskrat's target Devel::Peek:

      use IO::Capture::Stderr; use Devel::Peek; my $data; print STDERR " STDERR test (1)\n"; my $capture = IO::Capture::Stderr->new(); $capture->start(); print STDERR " STDERR test (2)\n"; warn " STDERR test (2a)"; my $test = 'string'; Dump($test); $capture->stop(); print STDERR " STDERR test (3)\n"; my @the_output = $capture->read; if( @the_output ) { foreach (@the_output) { print " >>> ", $_; } }
      produces output
       STDERR test (1)
       STDERR test (2a) at mrmskrat4.pl line 15.
      SV = PV(0x183ec0c) at 0x197138c
        REFCNT = 1
        FLAGS = (PADBUSY,PADMY,POK,pPOK)
        PV = 0x196deb4 "string"\0
        CUR = 6
        LEN = 7
       STDERR test (3)
        >>>  STDERR test (2)
      
Re: Best Way to Redirect STDERR to a Scalar
by BrowserUk (Patriarch) on Sep 14, 2003 at 04:22 UTC

    I *think* the problem is that Devel::Peek is writing directly to the C file descriptor 2 (see perlfuc:fileno) rather than via a perl file handle, which basicaly means that any attempt to capture its output via tie is simply bypassed. Ie. The tie magic only works when you use the filehandle, which the perl internals don't.

    Update: Forget the rest of this, it's bo..er..wrong! It couldn't work.

    I have a feeling that it may be possible to capture the output by dupping the file descriptor. POSIX::dup2 might be one way, and re-opening files(dups) using the "&=2" version of the syntax might be another. I have had a play with the latter, without success, but I didn't spend much time on it.


    Examine what is said, not who speaks.
    "Efficiency is intelligent laziness." -David Dunham
    "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller
    If I understand your problem, I can solve it! Of course, the same can be said for you.