Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris
 
PerlMonks  

Re: Loading only portions of a file at any given time

by kcott (Chancellor)
on Dec 01, 2017 at 05:54 UTC ( #1204643=note: print w/replies, xml ) Need Help??


in reply to Loading only portions of a file at any given time

G'day TJCooper,

I've provided an example script below which shows some techniques that you might use for this. I've assumed your IN2 file just contains headers (your description wasn't 100% clear on that). If you're going to be reusing IN2, you may want to consider running the first part of the script separately, and storing the @all_order data: a database, Storable, or other serialisation methods are all possible candidates (depends on your requirements and what you have available).

#!/usr/bin/env perl use strict; use warnings; use autodie; use Data::Dump; my @all_order; { open my $in2_fh, '<', 'pm_1204589_IN2'; while (<$in2_fh>) { chomp; push @all_order, $_; } } for my $range ([0..0], [0..2], [1..3], [3..3]) { my %wanted = map { $_ => 1 } @all_order[@$range]; { open my $in1_fh, '<', 'pm_1204589_IN1'; local $/ = "\n>"; while (<$in1_fh>) { chomp; substr $_, 0, 1, '' if $. == 1; my ($head, $seq) = split; $wanted{$head} = $seq if $wanted{$head}; } } print "Range: @$range\n"; dd \%wanted; }

With these dummy files (yes, I know, the sequences are totally bogus — just used for demonstration purposes):

$ cat pm_1204589_IN1 >Z ABC >Y DEF >X GHI >W JKL
$ cat pm_1204589_IN2 W X Y Z

I get this output:

Range: 0 { W => "JKL" } Range: 0 1 2 { W => "JKL", X => "GHI", Y => "DEF" } Range: 1 2 3 { X => "GHI", Y => "DEF", Z => "ABC" } Range: 3 { Z => "ABC" }

See also: Data::Dump.

— Ken

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1204643]
help
Chatterbox?
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (1)
As of 2018-02-23 05:36 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    When it is dark outside I am happiest to see ...














    Results (300 votes). Check out past polls.

    Notices?