Beefy Boxes and Bandwidth Generously Provided by pair Networks
good chemistry is complicated,
and a little bit messy -LW
 
PerlMonks  

Re: Re: What's the most efficient way to write out many lines of data?

by hagen (Friar)
on Jul 10, 2003 at 04:43 UTC ( #272890=note: print w/ replies, xml ) Need Help??


in reply to Re: What's the most efficient way to write out many lines of data?
in thread What's the most efficient way to write out many lines of data?

Thanks for the interesting example... I had to try it as the times you quoted seemed very fast. But on my P4 with 512Mb RAM, and <who knows?> disc speed your program creates the file in around 3.5 secs!

However the "converted file" only contained "\n"s.

I'm no guru, but I worked out that (I think) you need a split before the join, or you won't have a list that join requires.

print OUTFILE join (",",  (split /(XXXXXXXXXX)/)), "\n";

worked, sort of, for me - I couldn't get your /(.{,10})/ pattern to work, although I think I understand what it's trying to match - any 10 chars exactly and reflect those in the stream as well as the other characters - which in this case aren't any. The resulting file simply had "\n"s as the split didn't seem to find a match.

This then took around 20 seconds...

$ perl file.pl Creating file took 3.40489602088928 seconds Splitting file using regexp took 19.5581229925156 seconds

This results in lines containing collections of 10 sets of the following ",XXXXXXXXXX," so that 2 commas appear between adjacent groups of X's and at the beginning and end of each line.


Comment on Re: Re: What's the most efficient way to write out many lines of data?
Select or Download Code
Re: Re: Re: What's the most efficient way to write out many lines of data?
by flounder99 (Friar) on Jul 10, 2003 at 11:27 UTC
    D'oh!

    You'd think I'd at least look at the output file! I changed the regex line to

    print OUTFILE join (",", /(.{1,10})/g), "\n"
    and I got the results:
    Creating file took 9.90625 seconds Splitting file using regexp took 12 seconds
    a lot slower but nowhere near 16 minutes.

    --

    flounder

      Yup! That did it.

      However this is an idiom I'm not familiar with - I've only got the Camel's head here at work (Perl in a Nutshell) and the join entry doesn't expand on it. It's a great example of DWIM!

      With my use of the explicit split, as I said, the split file had lines like:-

      ,XXXXXXXXX,,XXXXXXXXXX,,XXXXXXXXXX,,...

      however, when I changed it to your use - even using my beginner's pattern of /(XXXXXXXXXX)/g - it DWIM'd!

      If you (or another brother) have time I'd appreciate an "expanded" Perl baby-talk explanation - i.e. put in all the bits that aren't necessary so we can see what's been left out.

      I (think I) understand the default $_ as being the input, but it's the use of a pattern to generate list context that's new to me.

      Thanks in advance

      hagen

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://272890]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (20)
As of 2015-07-06 20:33 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (83 votes), past polls