Beefy Boxes and Bandwidth Generously Provided by pair Networks vroom
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Re^9: [threads] Open a file in one thread and allow others to write to it

by BrowserUk (Pope)
on Nov 16, 2009 at 15:57 UTC ( #807497=note: print w/ replies, xml ) Need Help??


in reply to Re^8: [threads] Open a file in one thread and allow others to write to it
in thread [threads] Open a file in one thread and allow others to write to it

your bogus code.

Hm. You've provided a precise definition for your own tagline: "... a lack of valid arguments".


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.


Comment on Re^9: [threads] Open a file in one thread and allow others to write to it
Re^10: [threads] Open a file in one thread and allow others to write to it
by gulden (Monk) on Nov 16, 2009 at 16:07 UTC
    The below output file, generated by your code is not enough to clarify you?
    1: The time is 1258377733.481062 1: The time is 1258377733.481164 1: The time is 1258377733.481337 1: The time is 1258377733.4 3: The time is 1258377733.473267 3: The time is 1258377733.473595 3: The time is 1258377733.473749 3: The time is 1258377733.473885 3: The time is 1258377733.474008 3: The time is 1258377733.474110 3: The time is 1258377733.474212 3: The time is 1258377733.474313

    Your code without the sleep()'s:

    #! perl -slw use strict; use Time::HiRes qw[ time sleep ]; use threads; use threads::shared; our $THREADS ||= 4; my $sem :shared; open LOG, '>', 'log.txt' or die $!; sub logit { lock $sem; return printf LOG @_; } sub thread { my $tid = threads->self->tid; my $stop = shift; warn $tid, ": starting:", time(), " Stoptime: $stop\n"; while( time() < $stop ) { logit "%2d: The time is %f\n", $tid, time; } warn $tid, ": stopping at ", time(), "\n"; } my @threads = map threads->create( \&thread, time() + int( rand 10 ) ), 1 .. $THREADS; warn "threads started; waiting\n"; $_->join for @threads; warn "threads done\n"; open LOG, '<', 'log.txt' or die $!; print while <LOG>; close LOG;

    My position in this forum is to share ideas,opinions,knowledge,... and of course, "to learn", with the help of all the monks. And only joining strengths it will be possible to achieve real knowledge. Sometimes this forum is used to humiliate members in order to praise to himself, which is a pity.

    A contentious debate is always associated with a lack of valid arguments.
      The below output file, generated by your code is not enough to clarify you?

      It is, now you've posted it. Which is what I asked you to do 3 posts ago!

      And the system you are running this on?


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
        We have had a similar thread in the past 764973.

      There's no race condition there. Notice the timestamps aren't in order? You never ask Perl to write to disk, so it writes in 4k chunks, whether the chunk ends in the middle of a line or not.

      Simplest solution: Change

      open LOG, '>', 'log.txt' or die $!;
      to
      use IO::Handle qw( ); open LOG, '>', 'log.txt' or die $!; LOG->autoflush(1);

      If you're still having jumbled output after fixing the above problem, there might be a problem keeping file pointers in sync. Use the following before the print:

      use Fcntl qw( :seek ); seek(LOG, 0, SEEK_END);

      I don't think it'll come to that, though.

        The AUTOFLUSH does not solve the problem

        The SEEK works !!! and it works without the lock in $sem and the AUTOFLUSH in LOG. However, IMHO the lock in $sem must be used

        In overall i've understand your tip, but it's not clear for me how all the file pointers are syncked in a thread concurrency level, without conflicts. With the lock in $sem it's obvious , because only one thread can SEEK the Filehandle and write to it. However, I'm getting the log file syncked without using the lock in $sem. But these are other discussions

        Thanks for the tip it will be very useful for me...

      Hum, "buffered IO" and "buffer-fulls"

      Remember what I was saying about buffering?

      Your modification of my code, mildy modified:

      #! perl -slw use strict; use Time::HiRes qw[ time sleep ]; use threads; use threads::shared; our $THREADS ||= 4; my $sem :shared; open LOG, '>', 'log.txt' or die $!; my $old = select LOG; $|++; select $old; sub logit { lock $sem; return printf LOG @_; } sub thread { my $tid = threads->self->tid; my $stop = shift; warn $tid, ": starting:", time(), " Stoptime: $stop\n"; while( time() < $stop ) { logit "%2d: The time is %f\n", $tid, time; } warn $tid, ": stopping at ", time(), "\n"; } my @threads = map threads->create( \&thread, time() + int( rand 10 ) ), 1 .. $THREADS; warn "threads started; waiting\n"; $_->join for @threads; warn "threads done\n";

      When I run that under *nix:

      mehere@mehere-desktop:~/perl$ perl -v This is perl, v5.10.0 built for x86_64-linux-gnu-thread-multi ... mehere@mehere-desktop:~/perl$ wc -l log.txt 173746 log.txt

      It created a 173000 line file that is flawless!


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
        This is the solution 807525, until proven otherwise.
        A contentious debate is always associated with a lack of valid arguments.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://807497]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (6)
As of 2014-04-20 09:43 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    April first is:







    Results (485 votes), past polls