Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister

Re: Re: What's the most efficient way to write out many lines of data?

by hagen (Friar)
on Jul 10, 2003 at 04:43 UTC ( #272890=note: print w/replies, xml ) Need Help??

in reply to Re: What's the most efficient way to write out many lines of data?
in thread What's the most efficient way to write out many lines of data?

Thanks for the interesting example... I had to try it as the times you quoted seemed very fast. But on my P4 with 512Mb RAM, and <who knows?> disc speed your program creates the file in around 3.5 secs!

However the "converted file" only contained "\n"s.

I'm no guru, but I worked out that (I think) you need a split before the join, or you won't have a list that join requires.

print OUTFILE join (",",  (split /(XXXXXXXXXX)/)), "\n";

worked, sort of, for me - I couldn't get your /(.{,10})/ pattern to work, although I think I understand what it's trying to match - any 10 chars exactly and reflect those in the stream as well as the other characters - which in this case aren't any. The resulting file simply had "\n"s as the split didn't seem to find a match.

This then took around 20 seconds...

$ perl Creating file took 3.40489602088928 seconds Splitting file using regexp took 19.5581229925156 seconds

This results in lines containing collections of 10 sets of the following ",XXXXXXXXXX," so that 2 commas appear between adjacent groups of X's and at the beginning and end of each line.

Replies are listed 'Best First'.
Re: Re: Re: What's the most efficient way to write out many lines of data?
by flounder99 (Friar) on Jul 10, 2003 at 11:27 UTC

    You'd think I'd at least look at the output file! I changed the regex line to

    print OUTFILE join (",", /(.{1,10})/g), "\n"
    and I got the results:
    Creating file took 9.90625 seconds Splitting file using regexp took 12 seconds
    a lot slower but nowhere near 16 minutes.



      Yup! That did it.

      However this is an idiom I'm not familiar with - I've only got the Camel's head here at work (Perl in a Nutshell) and the join entry doesn't expand on it. It's a great example of DWIM!

      With my use of the explicit split, as I said, the split file had lines like:-


      however, when I changed it to your use - even using my beginner's pattern of /(XXXXXXXXXX)/g - it DWIM'd!

      If you (or another brother) have time I'd appreciate an "expanded" Perl baby-talk explanation - i.e. put in all the bits that aren't necessary so we can see what's been left out.

      I (think I) understand the default $_ as being the input, but it's the use of a pattern to generate list context that's new to me.

      Thanks in advance


Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://272890]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chilling in the Monastery: (4)
As of 2016-10-23 15:01 GMT
Find Nodes?
    Voting Booth?
    How many different varieties (color, size, etc) of socks do you have in your sock drawer?

    Results (301 votes). Check out past polls.