Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Re: Re: What's the most efficient way to write out many lines of data?

by hagen (Friar)
on Jul 10, 2003 at 04:43 UTC ( #272890=note: print w/ replies, xml ) Need Help??


in reply to Re: What's the most efficient way to write out many lines of data?
in thread What's the most efficient way to write out many lines of data?

Thanks for the interesting example... I had to try it as the times you quoted seemed very fast. But on my P4 with 512Mb RAM, and <who knows?> disc speed your program creates the file in around 3.5 secs!

However the "converted file" only contained "\n"s.

I'm no guru, but I worked out that (I think) you need a split before the join, or you won't have a list that join requires.

print OUTFILE join (",",  (split /(XXXXXXXXXX)/)), "\n";

worked, sort of, for me - I couldn't get your /(.{,10})/ pattern to work, although I think I understand what it's trying to match - any 10 chars exactly and reflect those in the stream as well as the other characters - which in this case aren't any. The resulting file simply had "\n"s as the split didn't seem to find a match.

This then took around 20 seconds...

$ perl file.pl Creating file took 3.40489602088928 seconds Splitting file using regexp took 19.5581229925156 seconds

This results in lines containing collections of 10 sets of the following ",XXXXXXXXXX," so that 2 commas appear between adjacent groups of X's and at the beginning and end of each line.


Comment on Re: Re: What's the most efficient way to write out many lines of data?
Select or Download Code
Re: Re: Re: What's the most efficient way to write out many lines of data?
by flounder99 (Friar) on Jul 10, 2003 at 11:27 UTC
    D'oh!

    You'd think I'd at least look at the output file! I changed the regex line to

    print OUTFILE join (",", /(.{1,10})/g), "\n"
    and I got the results:
    Creating file took 9.90625 seconds Splitting file using regexp took 12 seconds
    a lot slower but nowhere near 16 minutes.

    --

    flounder

      Yup! That did it.

      However this is an idiom I'm not familiar with - I've only got the Camel's head here at work (Perl in a Nutshell) and the join entry doesn't expand on it. It's a great example of DWIM!

      With my use of the explicit split, as I said, the split file had lines like:-

      ,XXXXXXXXX,,XXXXXXXXXX,,XXXXXXXXXX,,...

      however, when I changed it to your use - even using my beginner's pattern of /(XXXXXXXXXX)/g - it DWIM'd!

      If you (or another brother) have time I'd appreciate an "expanded" Perl baby-talk explanation - i.e. put in all the bits that aren't necessary so we can see what's been left out.

      I (think I) understand the default $_ as being the input, but it's the use of a pattern to generate list context that's new to me.

      Thanks in advance

      hagen

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://272890]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (16)
As of 2014-09-23 14:32 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (223 votes), past polls