Re: Perl v5.6.1 looking for .pmc files
by fglock (Vicar) on Oct 22, 2002 at 17:55 UTC
|
| [reply] [Watch: Dir/Any] |
|
Ha! Many thanks for digging that one out of the archives. Nick's comment that "The required stat()s on a sick NFS system are significant!" summed up my feelings exactly :-)
By the sound of the subsequent mail chain, there should be a workaround in v5.8.1. Until then, I guess I will have to look for improvements elsewhere.
Cheers, Kevin
| [reply] [Watch: Dir/Any] |
|
| [reply] [Watch: Dir/Any] |
|
Sooo... instead you could speed things up by compiling the modules to bytecode using Module::Compile (okay, in theory).
| [reply] [Watch: Dir/Any] |
|
yes! except that the feature is more used nowadays to preprocess complex code into plain perl5, instead of bytecode. v6, for example, process perl6 source code into plain perl5 "pmcs".
| [reply] [Watch: Dir/Any] [d/l] |
Re: Perl v5.6.1 looking for .pmc files
by Fastolfe (Vicar) on Oct 22, 2002 at 16:17 UTC
|
10 seconds to start up sounds about right if you have 150 modules you're loading and compiling also on modest hardware. Consider using Autoloader to load functions on demand instead of at startup to avoid some of that penalty. Also consider using Devel::DProf (run Perl with -d:DProf) and dprofpp to profile your code and see where perl is spending most of its time.
A 'stat' call is common if Perl is using 'stat' to look through @INC trying to search for your modules. A given module will rarely be found in the very first place perl looks, so you're going to get a lot of these failed calls.
Though I'm not really sure why it'd be looking for a .pmc and not .pm. I haven't heard anything about that. But still, as another poster mentioned, these calls should take almost no time whatsoever. I doubt these calls (failing or otherwise) are what's causing your performance woes.
| [reply] [Watch: Dir/Any] |
|
Hi
Re Autoloader, I did consider it, but all of the modules are required, so I have to take the hit at some point. Admittedly, there is something to be said for spreading the pain a bit :-)
Re DProf, I find it very useful for general tuning, but do not think it will help me diagnose compile time performance issues. I find strace more helpful for that.
Re stat calls, I already set up @INC so that only the required dirs are on the path and the dirs are in decreasing order of used modules, in order to obtain the lowest number of open failures.
Perl does not actually issue stat calls when looking for .pm files. It just issues open calls and gets ENOENT if they are not there. stat calls are issued for .pmc, .so and .bs files (there may be others). In my example, the vast majority of stat calls are for .pmc files.
Re the stat calls not being the problem, I can see from the strace output that they are taking around three seconds, which is a significant part of the startup time.
Cheers, Kevin
| [reply] [Watch: Dir/Any] |
|
I do not like part of my previous reply. It is not true to say that DProf is not useful for diagnosing compile time performance issues in general. I just do not think it is helpful to my particular issue.
| [reply] [Watch: Dir/Any] |
Re: Perl v5.6.1 looking for .pmc files
by bluto (Curate) on Oct 22, 2002 at 16:03 UTC
|
Check your filesystem. 2-3 seconds for 600 stats sounds
awfully slow to me. On my AIX box I get around 1000 in
about a tenth of a second. If you are running over NFS, your general
performance could be slow.
Also, it might be nice to see how
quick your script starts up the *second* time (i.e. after
filesystem info is cached).
bluto | [reply] [Watch: Dir/Any] |
|
Yes, it is very slow, thanks to a very slow network. That is why I am particularly concerned to reduce the number of such calls :-)
Re running the program a second time, I did that. Should have mentioned this. No difference to number of stat64 calls, but then I would not expect any.
My interest is really why Perl is looking for .pmc files in the first place.
Cheers, Kevin
| [reply] [Watch: Dir/Any] |
|
I guess my point was that it
seemed like you were trying to solve a problem related to a broken hardware environment by changing the way Perl does it's job. I don't envy you since it sounds
like you can't install everything on
a local disk, or use something like NFS mounts with cachefs.
FWIW, the number of stat64 calls would not decrease, but even NFS
should (I think) cache the results of these calls rather than going
over the network, but I digress...
bluto
| [reply] [Watch: Dir/Any] |