Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Re: FileHandles and threads

by BrowserUk (Patriarch)
on Sep 21, 2005 at 14:08 UTC ( #493801=note: print w/replies, xml ) Need Help??


in reply to FileHandles and threads

It is possible to share a filehandle from multiple threads (but there may be caveats). The problem is how to pass the handle (an object in Perl's terms) to the thread that is going to perform the further processing as iTthreads won't allow you to share objects.

The solution is to pass the fileno associated with the filehandle and then "dup" the handle within the thread using the syntax  open my $newhandle, "<&=$fileno" ... Note:That is "<&=$fileno" (an alias file descriptor) not "<&$fileno" (a duplicate file descriptor) (see perlopentut for details). Note: Although this uses the open built-in syntax, it is not reopening the file, mearly duplicating the internal control structures required for accessing the existing open filehandle.

This incomplete demo, open files in the main thread, prints the first 10 lines before passing the fileno for that filehandle to a newly created thread that prints the rest of the file, closes it and dies:

#! perl -slw use strict; use threads; use threads::shared; sub thread{ my( $fileno ) = @_; open my $fh, "<&=$fileno" or warn $! and die; printf "%d:%s", threads->self->tid, $_ while defined( $_ = <$fh> ) +; close $fh; } for my $file ( map{ glob $_ } @ARGV ) { open my $fh, '<', $file or warn "$file : $!" and next; printf "From main: %s", scalar <$fh> for 1 .. 10; printf "Fileno:%d\n", fileno $fh; threads->create( \&thread, fileno( $fh ) )->detach; printf 'paused:';<STDIN>; }

This seems to work for simple filehandles and sockets, but I see no reason (but have not verified) why it should not also work for those created using Filehandle, just use the fileno method.

The one caveat I am aware of is that I have occasionally seen strange effects when trying this with files opened for read-write access, but it is transient and usually "goes away". That is to say, I've only seen it occasionally and it has always appeared to fix itself when other bugs where resolved, but I have done very limited testing with this. If you wish to pursue this, you will be a pioneer. Feedback welcomed.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.

Replies are listed 'Best First'.
Re^2: FileHandles and threads
by sodul (Initiate) on Sep 22, 2005 at 00:10 UTC

    Thanks a lot this technique is working very well.

    I just need to be careful that the original $fh is not destroyed, which require to use a "global" variable.

    You made my day.

    - Stephane

Re^2: FileHandles and threads
by zentara (Archbishop) on Sep 22, 2005 at 18:21 UTC
    Thanks for that code example! Passing a filehandle through a thread shared variable was something I've been searching for for awhile. Just for demonstration purposes, it can go the other way too.....create the filehandle in the thread, and read it in main.

    I'm not really a human, but I play one on earth. flash japh

      I first played with this a couple of years ago. I first mentioned it here about a year ago in Re^3: Passing globs between threads.

      By using a pool of threads and passing the filenos via a queue, it is possible to efficiently emulate all the usual forms of forking server architecture.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
      "Science is about questioning the status quo. Questioning authority".
      The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://493801]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (3)
As of 2023-02-06 08:46 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    I prefer not to run the latest version of Perl because:







    Results (33 votes). Check out past polls.

    Notices?