Re: Can you help profile this?

[Tanktalus]

First thing's first. If you have sets embedded in your list, keep them as sets. What you're doing is nearly the same thing as pointer arithmetic in C, and is bad for many of the same reasons. ww mentioned an AoA, but as a question. I'll make it a statement: it should be an array of references. Personally, I'm a bit more partial to AoH, as it allows me to use names for keys to make sense of the inner data. An AoA would look like:

@in = (
  [ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14 ],
  [ 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34 ],
  # ...
);

but we still don't know what any of the numbers mean without more context. An AoH usually would take up a lot more memory, but would give the context implicitly:

@in = (
  { user => 'Fighter2', id => 936570, xp => 34, writeups => 11, ... },
);

You can get hashes to take a lot less memory if you can define "default" values that cover most cases, reducing most of the hashes down to only unique values, at a slight cost to code complexity. I'll assume that's not an option (it rarely is, in my experience).

Once you do this, you would have $in[$i][12] or $in[$i]{attn} instead of $in[$i+12] Your outer for loop would be a simple for my $i (0..$#in) (you would want to go through the indexes, and the last one is the last index, which is what $# gives you). You wouldn't be creating an @Attn the length of your original array, but 1/14th the size, which would help both memory usage and speed. Also, your inner arrays would be arbitrary lengths - so if you regularly have less than 13 items in a set, you simply would not allocate the extra memory. And, should you need more than 14 items in a set, you can now handle that, kind of.

Here's a first crack:

#!/usr/bin/perl

#Initialization
my @in=(
        [...],
        [...],
       );
my $MaxAgc0 = "1.15503129132678e-005"; # MaxAgc0 is a constant

#Interpolation
if (@in) {
    for my $i (0..$#in)
    {
        my $ptr = $in[$i]; # convenience.

        # we don't need the first item to be a count of sectors anymor
+e.
        # we also don't need this loop at all.
        #for my $j (0..$#$ptr)
        #{
            # so we also reduce the numbers here.
            my $num = $ptr->[11] * $MaxAgc0;
            push @Attn, $num / $ptr->[9];
        #}
    }

    for my $i (0..$#in)
    {
        push @powers, ($in[$i][$_] - $Attn[$i])/100 for (1..8);
    }
} else { # @in is empty
    print "\n No data to map for Panda machine. Check if input logfile
+ is empty";
    kill;
}

Now, what I'd do from here is pull apart the sets even more. Say something like this:

(
 {
    attn => 5, # attenuation?
    agc  => 6, # the MaxAgc0 multiplier
    sectors => [ all the sectors for this data point in this array ],
    # etc.
  },
  # repeat for each item.
)

I'd also change the output to match everything done above: return a list of arrays:

#!/usr/bin/perl

#Initialization
my @in=(
        {...},
        {...},
       );
my $MaxAgc0 = "1.15503129132678e-005"; # MaxAgc0 is a constant

use List::MoreUtils qw(pairwise);

#Interpolation
if (@in) {
    my @Attn = map {
        my $num = $_->{agc} * $MaxAgc0;

        $num / $_->{attn};
    } @in;

    @powers = pairwise {
        [ map { ( $a->{sectors}[$_] - $b ) / 100 } for @{$a->{sectors}
+} ]
    } @in, @Attn;
} else { # @in is empty
    print "\n No data to map for Panda machine. Check if input logfile
+ is empty";
    kill;
}

Now the return will keep the sectors grouped, there will be no extra data in the output. We can still improve this by getting rid of @Attn altogether:

#Initialization
my @in=(
        {...},
        {...},
       );
my $MaxAgc0 = "1.15503129132678e-005"; # MaxAgc0 is a constant

#Interpolation
if (@in) {
    @powers = map {
        my $attn = $_->{agc} * $MaxAgc0 / $_->{attn};
        [ map { ( $_->{sectors}[$_] - $attn ) / 100 } for @{$_->{secto
+rs}} ]
    } @in;
} else { # @in is empty
    print "\n No data to map for Panda machine. Check if input logfile
+ is empty";
    kill;
}

We got rid of a huge amount of wasted memory. That @Attn list was humongous in your code, and we've pared it down to 1/14th and now to a single scalar (which we calculate as we need it, then discard it once no longer needed). And we got rid of unnecessary items in your sets, though, in the interests of readability, I increased the memory usage of the input by making it an AoH's. That's probably a wash, if not slightly bigger now than it was.

Hope that helps.