Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Re^5: multi threading DBI

by BrowserUk (Pope)
on Oct 30, 2013 at 20:08 UTC ( #1060433=note: print w/ replies, xml ) Need Help??


in reply to Re^4: multi threading DBI
in thread multi threading DBI

Hm. That's not a "working, cut-down version" ...

So this isn't working either, but it ought to be close once you've filled in the blanks:

#! perl -slw use strict; use threads; use Threads::Queue; use LWP::Simple; use DBI; sub fetchNParse { my( $Qin, $Qout ) = shift; while( my $uname = $Qin->dequeue ) { my $content = get $uname; my @bits = $content =~ m[....]; $Qout->enqueue( join $;, $uname, @bits ); } $qOut->enqueue( undef ); } our $THREADS //= 16; my $Qunames = new Thread::Queue; my $Qrets = new Thread::Queue; threads->create( \&fetchNParse, $Qunames, $Qrets )->detach for 1 .. $T +HREADS; my $dbh = DBI->connect( ... ); my $unames = $dbh->selectcol_arrayref("select uname from event"); $Qunames->enqueue( @$unames ); for( 1 .. $THREADS ) { while( my $ret = $Qrets->dequeue ) { my( $uname, @bits ) = split $;, $ret. $dbh->do( insert stuff ); } } $dbh->disconnect;

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.


Comment on Re^5: multi threading DBI
Download Code
Re^6: multi threading DBI
by Anonymous Monk on Oct 31, 2013 at 06:56 UTC
    Thank you, but I have a bunch of termination errors in "fetchNParse": Thread 1 terminated abnormally: Can't call method "enqueue" on an undefined value at test.cgi line 16. seems $Qout is causing problem?

      I'm not psychic. Post the (exact) code that is giving you the errors.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
        Yes, Sir! To isolate the error I disabled the DBI and even the LWP... so this script do nothing but printing ...
        #! perl use strict; use threads; use Thread::Queue; use LWP::Simple; use DBI; sub fetchNParse { my( $Qin, $Qout ) = shift; while( my $uname = $Qin->dequeue ) { #my $content = get $uname; #my @bits = $content =~ m[....]; my @bits=qw(pizza2 steak2 chicken2 burgers2); print " $uname 1\n"; #$Qout->enqueue( join $;, $uname, @bits ); $Qout->enqueue( $uname ); } $Qout->enqueue( undef ); } our $THREADS //= 16; my $Qunames = new Thread::Queue; my $Qrets = new Thread::Queue; threads->create( \&fetchNParse, $Qunames,$Qrets )->detach for 1 .. $TH +READS; #my $dbh = DBI->connect( ... ); #my $unames = $dbh->selectcol_arrayref("select uname from event"); my @unames = qw(pizza steak chicken burgers); print "a\n"; $Qunames->enqueue( @unames ); print "b\n"; for( 1 .. $THREADS ) { while( my $ret = $Qrets->dequeue ) { #my( $uname, @bits ) = split $;, $ret. #$dbh->do( insert stuff ); print "2\n"; } } #$dbh->disconnect;

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1060433]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (18)
As of 2014-10-22 15:46 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    For retirement, I am banking on:










    Results (119 votes), past polls