Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

Re^2: Reading a huge input line in parts

by kroach (Pilgrim)
on May 04, 2015 at 15:53 UTC ( #1125585=note: print w/replies, xml ) Need Help??


in reply to Re: Reading a huge input line in parts
in thread Reading a huge input line in parts

I thought about using read, however, since the numbers are not constant length, a single number could be split between two chunks. This would introduce additional complexity to detect and merge such split numbers. I should've included such examples in the sample input from the start, I've updated the question.
  • Comment on Re^2: Reading a huge input line in parts

Replies are listed 'Best First'.
Re^3: Reading a huge input line in parts
by Laurent_R (Canon) on May 04, 2015 at 18:05 UTC
    It should not be to costly in terms of resources and performance to check if you have a space at the beginning and at the end of each chunk of data before splitting it, and reconstruct the boundary numbers accordingly, especially if your read data chunks are relatively large.

    Je suis Charlie.
Re^3: Reading a huge input line in parts
by aaron_baugher (Curate) on May 04, 2015 at 22:36 UTC

    In that case, I'd check the end of the buffer for digits, and if there are any, trim them off and save them to prepend to the next buffer that you read in. But you don't want to do that if it's the final 0 in the file, so I have some if statements in here. There's probably a more elegant way to do some of this, but I think this will handle it correctly:

    #!/usr/bin/env perl use 5.010; use strict; use warnings; my $l; # chunk of a line my $tiny_buffer = 8; # tiny buffer for testing my $leftover = ''; # leftover, possibly partial number at end of buf +fer while ( read DATA, $l, $tiny_buffer ) { $l = $leftover . $l; say " ;$l;"; $leftover = ''; if( $l =~ s/(\d+)$//g ){ if( $1 == 0 ){ $l .= '0'; $leftover = ''; } else { $leftover = $1; } } for (split ' ', $l) { if ( $_ == 0 ) { say 'Reached a zero'; } else { say "; $_ ;"; # process a number } } } __DATA__ 1 2 3 4 5 6 7 8 99 1 2 3 4 5 6 7 8 9 0 1 22 3 4 5 6 7 8 99 1 2 3 4 5 6 77 8 9 0

    Aaron B.
    Available for small or large Perl jobs and *nix system administration; see my home node.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1125585]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (6)
As of 2020-10-30 19:41 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My favourite web site is:












    Results (284 votes). Check out past polls.

    Notices?