Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Re: matching datetimestamps and concatenating data where timestamps match from multiple large datafiles

by hdb (Parson)
on Sep 22, 2013 at 12:09 UTC ( #1055179=note: print w/ replies, xml ) Need Help??


in reply to matching datetimestamps and concatenating data where timestamps match from multiple large datafiles

UPDATE: As CountZero points out below, there is a problem with my do {...} blocks, so please check his proposal Re: matching datetimestamps and concatenating data where timestamps match from multiple large datafiles which should run as intended.

It sounds like your data is sorted ascending with respect to time. Under this assumption I propose to use the following idea of a script:

use strict; use warnings; open my $fileA, ... or die; open my $fileB, ... or die; # same with the other files my $lineB = ''; while( my $lineA = <$fileA> ) { chomp $lineA; next unless $lineA =~ m/^\d\.+(\d{2}):(\d{2}):(\d{2})\.+(\d{2})(\d{2 +})(\d{2})\d{2}.abc/; my $dateA = ... ; # construct date from $1, $2 etc do { chomp $lineB; next unless $lineB =~ m/^\d\.+(\d{2}):(\d{2}):(\d{2})\.+(\d{2})(\d +{2})(\d{2})\d{2}.abc/; my $dateB = ... ; # construct date ... next if $dateB < $dateA; # use some date and time library for thes +e comparisons last if $dateB > $dateA; # ... $lineA .= $lineB; } while( $lineB = <$fileB> ); # same with fileC etc... print "$lineA\n"; }


Comment on Re: matching datetimestamps and concatenating data where timestamps match from multiple large datafiles
Select or Download Code
Re^2: matching datetimestamps and concatenating data where timestamps match from multiple large datafiles
by CountZero (Bishop) on Sep 22, 2013 at 13:57 UTC
    Did you try your code?

    One cannot use next or last inside a do { ... } block. As the docs say : "contrary to popular belief do{} blocks, which do NOT count as loops".

    It is not a loop, so you cannot use loop-control commands. You can double the curly braces and put a bare block inside the do { ... } block. Bare blocks are loops that execute once. The loop-control commands will work ... somewhat, but last will not do as you might expect: it will exit the inner (bare) block, only to fall into the outer (do) block and hence has the same effect as next.

    CountZero

    A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

    My blog: Imperial Deltronics

      I did not try my code as it is only a sketch of how to start.

      Thanks for pointing out this flaw, I was not aware of it. I did put the while at end to avoid the duplicate checking of the times. You have solved this directly in your code by adding another check.

        Yes, the duplicate checks do not make me very happy, but I cannot see another easy way out.

        CountZero

        A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

        My blog: Imperial Deltronics

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1055179]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others musing on the Monastery: (16)
As of 2014-08-22 14:36 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (158 votes), past polls