I think roboticus nailed the biggest one. You're doing two things in that one line that are killers to performance.

The first, which is probably the smaller one, is shelling out. The fact that you're forking a second process (overhead) and execing something else (reinitialising the C library, parsing parameters, etc., more overhead), well, that's a lot of overhead. Especially for things that Perl can do internally nearly as quickly, but without the overhead.

The second, and larger, problem is that this subprocess is scanning your 2-million-record file from the beginning each time. That is, it opens the file (overhead), and then reads each record one at a time, looking at each byte for record-ending text (the \n character), counting them up to find the specific record you need.

The minor issue is the way you get rid of leading/ending spaces (see this stackoverflow question though I'm sure we've covered it here, too).

As for the first problem, 2 million records doesn't seem that bad. Say 500 bytes per line on average, that's 1GB. Perl adds some overhead, but if you were to load each record into an array, you're looking at less than 1.25GB of RAM, which is fairly minor for most (but definitely not all) systems. It'd be a bit of a strain on many systems, but not beyond the realm of reason, especially if it can save you 20+ minutes of processing time (I'm hesitant to say how much you will actually save, but I'm guessing more than that).

If you're actually on a system where this is an issue, even Tie::File can save a ton of time just because it will cache file offsets for records so you'll only scan through the file a handful of times.

Hope that helps.