Well, you definitely don't want to die in the middle of a file - that'll make things too complicated - so you probably don't want to just kill and restart it. What you -could- do is check memory usage after each file and then exit if you've built up more than a certain percentage of memory usage. This works on my system:
use strict;
use warnings;
my ($data, $mem);
my $kb = ' ' x 1024**2;
do {
$data .= $kb;
$_ = `ps -p $$ -o %mem`;
($mem) = m/(\d+\.\d+)/;
}
while ($mem < 50);
You could also have it check for a file named a certain thing (like exit.txt) and exit if it sees that as well, deleting before exit so you're prepped for the next run.