perl cgi shared hosting memory limit problem

[scorpio17]

I manage a perl-based website in a shared-hosting environment. It's been online for several years, with no problem. The traffic is fairly low (several hundred hits per day - maybe 4000 hits per month, max.) My web hosting company recently upgraded the server to FreeBSD 8 (not sure what it was previously - FreeBSD 7 I guess?). With this upgrade, I went from perl version 5.8.8 to 5.12.2. Ever since this upgrade, I've been noticing intermittent failures. I'll occasionally get a 'server error', but if I click 'reload' the page loads normally. This happens maybe 1% of the time. It seems random, and is nearly impossible to reproduce. There's nothing in the error logs. However, it appears that I may be exceeding the host's per-process memory limits. But I don't understand why a page would load okay sometimes, but fail other times. For a given page, the memory requirement should be the same every time, right? Is there any tool available that might help debug which part of the script is using the most memory? I use a lot of cpan modules. Maybe there's a bug in one of them specific to 5.12.2, but how can I find/fix it? Has anyone else out there ever had a similar problem?

[daxim]

Memory Profiling

[Anonymous Monk]

What tells you its the memory limit? Are you getting closer and closer to the limit on each refresh?

[scorpio17]

The web host has an 'account control center' web page. From there, I can see what they call 'resource logs'. It's a table that shows every time a process (in this case, my cgi script) gets killed because of some resource violation. All the violations are of type 'memory hogs user'. For the past week, I'm averaging about 10 violations per day. I can't see data for more than the past 7 days.

[BrowserUk]

Try logging the script parameters at the start of the script and a 'goodbye' message at the very end.

For runs that get killed you should have the first message logged but not the last; so that'll perhaps give you an insight as to what parameters are involved when you get killed.

Of course, if this is a mod_perl or fcgi type persistent environment, it may be the accumulation of many runs that causes death; in which case you would need to monitor and log the size(s) of any globals you use.

[Anonymous Monk]

My (former) host lowered the limit four times in four months.

[sundialsvc4]

#define FLAME ON ... :-D

First, you have to find a hosting company that doesn’t presume that the letter “P” in “Perl” doesn’t stand for “PHP.”   (Tip:   if they spell it PERL, i.e. uppercase-only, run like hell ...)

I had a site running for many years on (Omitted) without trouble, even though their Perl was ancient.   But one day it “unexpectedly” just stopped working, and when I logged-in to check it, the ulimit was set so small that the cpan command wouldn’t run!   (Fortunately, cpanm still did.)   The number of Apache-workers running on that ridiculously-tiny VM was ridiculously-large, and I am quite sure that every site except mine was running PHP.   (They had a Big Phat PHP with all the options turned-on.)   So, I loaded-up my site and moved it to a much nicer home.   They never bothered to announce any of the changes that they had obviously been making over time.

Mind you, it’s not that I have anything against PHP, because I don’t.   (“If it on-ly had a brain ...”)   ;-)   But, shared hosting companies know which side their bread is buttered-on, and they squeeze a server until it hurts.