Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Re^2: Displaying/buffering huge text files

by Anonymous Monk
on Dec 22, 2011 at 17:28 UTC ( #944816=note: print w/ replies, xml ) Need Help??


in reply to Re: Displaying/buffering huge text files
in thread Displaying/buffering huge text files

Hey! I was looking for a way to handle big files and this looks awesome. I already tried and runs so cool. However, I am kind newbie on Perl. I was trying to modify in order to get secuencial access, not random. I haven't good luck on this. I just tried to remove the int rand( length( $index )/8) and let this with length of $index. Didn't work, the seek doesn't move the pointer in the file. Could someone can give me a hint? Thank you in advance!


Comment on Re^2: Displaying/buffering huge text files
Re^3: Displaying/buffering huge text files
by Anonymous Monk on Dec 23, 2011 at 01:03 UTC
    Us N instead of non-portable J.
    my $index = pack 'N', 0; $index .= pack 'N', tell FILE while <FILE>; for( my $line_idx = $s; $line_idx < $fileLastLine; $line_idx++ ) { my $line = unpack( 'N', substr($index, $line_idx*4, 4)); seek FILE, $line, 0; chomp( $_ = <FILE> ); my $line_str = $_;
      In case it isn't clear, $s is for start, the line to start on. For $fileLastLine I used:
      my $fileLastLine = `wc -l ${filename}`; chomp $fileLastLine; $fileLastLine = int ($fileLastLine);
      Us N instead of non-portable J.

      'J' may be non-portable, but why oh why would you build an index on one machine and the export it to another? (Hint:You wouldn't unless you are naive!)

      The reason for using 'J' in preference to 'N', is that it is 'native'. That is, whatever architecture of machine you use it on it uses the format that is natural, and therefore most efficient, for that machine.


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

      The start of some sanity?

Re^3: Displaying/buffering huge text files
by BrowserUk (Pope) on Dec 23, 2011 at 01:37 UTC
    I was trying to modify in order to get secuencial access, not random.

    Sorry, but that makes no sense at all. You have to read the entire file sequentially once in order to build an index.

    If you want to process the file sequentially, just read it sequentially. There is nothing to be gained by using an index unless you need random access.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    The start of some sanity?

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://944816]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (14)
As of 2014-08-22 15:27 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (159 votes), past polls