Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Wrap around an interactive python script

by Kirsle (Pilgrim)
on Jul 20, 2010 at 23:41 UTC ( #850535=perlquestion: print w/replies, xml ) Need Help??

Kirsle has asked for the wisdom of the Perl Monks concerning the following question:

Fellow monks, I have need of your wisdom.

I'm trying to write a Perl script that wraps around an interactive Python script (it's the "upload.py" for Google's code review service). My script needs to get the output given to the user so that it can parse out the issue URL generated by upload.py.

I was thinking I would just do something like this:

my $output = `/script/upload.py @ARGV`;

I was expecting that this would run upload.py and print what upload.py prints to the user's terminal so they can see it, and when they're prompted to fill out information, it would pass their input through to upload.py directly.

But the backticks don't let STDOUT sneak through to the terminal, so then I tried using system() with "tee" to copy the STDOUT to a file which my script could read after:

system("/script/upload.py @ARGV | tee temp.tee"); my $output = `cat temp.tee`; # not really, did a read() instead

But this didn't work either. No output from upload.py was given on my terminal; it was obviously hanging, waiting for input, but didn't give any output unless I made the upload.py exit normally (i.e. answering "no" when it prompted me about svn files that aren't managed by svn at the beginning). It may be a line buffering issue in upload.py; how could I force autoflush on it if this is the case?

Replies are listed 'Best First'.
Re: Wrap around an interactive python script
by ahmad (Hermit) on Jul 21, 2010 at 02:17 UTC

    You can do this through open pipe, but you might run into some buffering problems.

    open(LOG,">>/path/to/log"); open(SYS,"/path/to/python/script.py |"); while (<SYS>) { print $_; print LOG $_; } close(SYS); close(LOG);

    This way you can print to screen and to your log file

      I've recently started using IPC::Open3::open3 for InterProcess Communication type purposes and haven't looked back. Makes it easy to interact with another application by just reading and writing to filehandles, very slick and more intuitive (at least for me, anyway).

      But I'm still quite the amateur and may be missing some subtleties (or outright facts).

        While open3 does a lot (incl propagating exec errors to the parent), it's still a low level module. It's doesn't help you read from the child's STDOUT and its STDERR without combining them, a situation prone to deadlocks. IPC::Run3 and IPC::Run are higher-level solutions.
Re: Wrap around an interactive python script
by ikegami (Pope) on Jul 21, 2010 at 16:04 UTC
    Expect is designed to work with interactive programs. One of the things it does is use a pseudo tty. This "fools" most programs into disabling output buffering.
Re: Wrap around an interactive python script
by cdarke (Prior) on Jul 21, 2010 at 07:54 UTC
    It may be a line buffering issue in upload.py; how could I force autoflush on it if this is the case?

    If you set $| = 1; in Perl, and assuming that STDOUT is the default file handle, then that should be inherited by the python program. Otherwise (forgive me monks for I have sinned) in Python you can
    sys.sysout.flush()
    although I admit that is not an autoflush.

      I don't think so. Autoflushing output happens in the C runtime library (libc/glibc) or even above that, and the auto flush flag is also a thing of the C runtime library or the interpreter or application above that. It is not part of the O/S information about the file handle, and so it does not survive the exec() system call. You have to enable autoflushing in Python (or any other application you start).

      The flag does survive a fork(), like any other simple variable does.

      Alexander

      --
      Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://850535]
Approved by ahmad
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others meditating upon the Monastery: (4)
As of 2021-09-24 06:26 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?