Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Re: matching datetimestamps and concatenating data where timestamps match from multiple large datafiles

by RichardK (Vicar)
on Sep 22, 2013 at 12:19 UTC ( #1055181=note: print w/ replies, xml ) Need Help??


in reply to matching datetimestamps and concatenating data where timestamps match from multiple large datafiles

I assume that your files must be already sorted as they're log files, then what you are trying to do is a type of Merge Sort

You can then process you files one line at a time. Read the current line for fileA and decide if you want to write the current lines for fileB and fileC, and when you write a line from a file get the next one. Keep going until you run out of lines.


Comment on Re: matching datetimestamps and concatenating data where timestamps match from multiple large datafiles

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1055181]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (8)
As of 2015-07-03 00:08 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (47 votes), past polls