http://www.perlmonks.org?node_id=807500


in reply to Re^9: [threads] Open a file in one thread and allow others to write to it
in thread [threads] Open a file in one thread and allow others to write to it

This node falls below the community's threshold of quality. You may see it by logging in.

Replies are listed 'Best First'.
Re^11: [threads] Open a file in one thread and allow others to write to it
by ikegami (Patriarch) on Nov 16, 2009 at 17:45 UTC

    There's no race condition there. Notice the timestamps aren't in order? You never ask Perl to write to disk, so it writes in 4k chunks, whether the chunk ends in the middle of a line or not.

    Simplest solution: Change 

    open LOG, '>', 'log.txt' or die $!;

    [download]
    to 
    use IO::Handle qw( );
open LOG, '>', 'log.txt' or die $!;
LOG->autoflush(1);

    [download]

    If you're still having jumbled output after fixing the above problem, there might be a problem keeping file pointers in sync. Use the following before the print: 

    use Fcntl qw( :seek );
seek(LOG, 0, SEEK_END);

    [download]

    I don't think it'll come to that, though.

[reply]
[d/l]
[select]

      The AUTOFLUSH does not solve the problem

      The SEEK works !!! and it works without the lock in $sem and the AUTOFLUSH in LOG. However, IMHO the lock in $sem must be used

      In overall i've understand your tip, but it's not clear for me how all the file pointers are syncked in a thread concurrency level, without conflicts. With the lock in $sem it's obvious , because only one thread can SEEK the Filehandle and write to it. However, I'm getting the log file syncked without using the lock in $sem. But these are other discussions

      Thanks for the tip it will be very useful for me...

[reply]

        and it works without the AUTOFLUSH in LOG.

        seek apparently forces a flush. Assuming it's safe to rely on that, it's not good enough. The catch is that it's forcing the flush at the wrong time. You want to flush immediately after the print, not at the start of the thread's next print.

        and it works without the lock in $sem

        You're relying on your OS to do writes atomically. That might be reliable for small writes, but why chance it. If you do find a performance bottleneck, you might be better off using a separate log file for each thread and merge them later.

[reply]
[d/l]
[select]
Re^11: [threads] Open a file in one thread and allow others to write to it
by BrowserUk (Patriarch) on Nov 16, 2009 at 19:21 UTC
    Hum, "buffered IO" and "buffer-fulls"

    Remember what I was saying about buffering?

    Your modification of my code, mildy modified: 

    #! perl -slw
use strict;
use Time::HiRes qw[ time sleep ];
use threads;
use threads::shared;

our $THREADS ||= 4;

my $sem :shared;
open LOG, '>', 'log.txt' or die $!;
my $old = select LOG;
$|++;
select $old;


sub logit {
    lock $sem;
    return printf LOG @_;
}

sub thread {
    my $tid = threads->self->tid;
    my $stop = shift;
    warn $tid, ": starting:", time(), " Stoptime: $stop\n";
    while( time() < $stop ) {
        logit "%2d: The time is %f\n", $tid, time;
    }
    warn $tid, ": stopping at ", time(), "\n";
}

my @threads = map
    threads->create( \&thread, time() + int( rand 10 ) ),
    1 .. $THREADS;

warn "threads started; waiting\n";
$_->join for @threads;

warn "threads done\n";

    [download]

    When I run that under *nix: 

    mehere@mehere-desktop:~/perl$ perl -v
This is perl, v5.10.0 built for x86_64-linux-gnu-thread-multi
...

mehere@mehere-desktop:~/perl$ wc -l log.txt
173746 log.txt

    [download]

    It created a 173000 line file that is flawless!

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
[d/l]
[select]
    A reply falls below the community's threshold of quality. You may see it by logging in.
Re^11: [threads] Open a file in one thread and allow others to write to it
by BrowserUk (Patriarch) on Nov 16, 2009 at 16:43 UTC
    The below output file, generated by your code is not enough to clarify you?

    It is, now you've posted it. Which is what I asked you to do 3 posts ago!

    And the system you are running this on?

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
      We have had a similar thread in the past 764973.
[reply]

In Section Seekers of Perl Wisdom