Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

Offset Reading - two files

by gio001 (Acolyte)
on Dec 14, 2012 at 14:01 UTC ( #1008840=perlquestion: print w/ replies, xml ) Need Help??
gio001 has asked for the wisdom of the Perl Monks concerning the following question:

Hello everyone, I am a newbie and hope you forgive if this question is simple to your eyes. I have been looking at this code for some time:

perl -sple"BEGIN{open BIG};($undef,$l)=split;read(BIG,$_,$l)" -- -BIG=bigfile.dat index.dat >outfile.dat

The index file contains two columns, one for the offset and the second for a number of characters to read. The code works fine but it seems to be processing the bigfile sequentially, by this I mean it starts at 0 reads the first value in column 2 of the index file and writes out that many characters, next it continues from there and writes out the next group of x characters as identified by the next value in column 2 of the index file.

How can I force a reposition of the read inside the big file by using the value in column 1 of the index file anytime a new read has to occur?

Can the logic be added to the one liner above?

Additionally how do I add a CR to each record that is going to the output file?

Thanks to all.

Comment on Offset Reading - two files
Download Code
Re: Offset Reading - two files
by Anonymous Monk on Dec 14, 2012 at 14:08 UTC
Re: Offset Reading - two files
by BrowserUk (Pope) on Dec 14, 2012 at 14:13 UTC

    Let's see a little history:

    1. In 2008, you asked; I replied.
    2. Two and a half years later you re-posted my code and asked.
    3. And now nearly two years on, you do the same again.

    You haven't explained what you are trying to do; shown examples of the files in question; or explained why it should be a one-liner.

    You seem to have unlimited time to solve the problem; but also seem to have made no progress in the intervening 5 years.

    What are you actually trying to solve?


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.

    RIP Neil Armstrong

      OK, yes I understand your valid points. Here is a little history: when I originally asked you the question I was facing a problem on retrieving data from a large file whose record structure information was stored in an index file. At that time the structure of the index file was sequential that is the next record was always starting right after the end of the prior record, so no offset was actually involved. The code you supplied was perfectly doing the retrieval by sequentially picking up chunks from the big file. Now the structure of the big file has been changed by the vendor and there is extraneous data in between each record for which I have to make use of the offset information each time before I can get the record I am interested in. I understand your surprise and the disappointment for my ignorance, I can only say I wish I had more time to dedicate to the discovery of Perl abilities ..... Thanks for all you did
        Now the structure of the big file has been changed by the vendor and there is extraneous data in between each record for which I have to make use of the offset information each time before I can get the record I am interested in.... I wish I had more time to dedicate to the discovery of Perl abilities

        That change appears to have happened 20 months ago. How have you been handling the problem in the interim?

        Have you heard of seek?


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

        RIP Neil Armstrong

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1008840]
Approved by marto
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others having an uproarious good time at the Monastery: (4)
As of 2014-09-20 15:14 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (160 votes), past polls