Re^2: Perl: friend or foe ?

[merlyn]

You mean: should Perl somehow secretly cache the startup steps, so that it performs those only once? Sure, but then we run into problems specifying exactly what to cache and where to cache that translation step.

And hence, what then happens (in other programming languages) is that we make the users specify exactly when and how to cache the translation from the programming language to the intermediate language. And that would make Perl distinctly less "human friendly" for me. I'd hate having to develop a "makefile" for every Perl program I write. Those systems typically also separate the "compilation" environment from the "execution" enviroment, losing some of the meta information (like the names of all methods in this class), and losing the ability to "compile" while "executing" for some neat tricks.

No, I like the current system. When I don't need caching of the translation (which is what your "compilation" seeks to do), I can use Perl by simply saying "go". When I've decided that caching helps, I can do that explicitly using the mechanisms I gave earlier.

To force caching on all users is a premature optimization, which is an evil step.

For example, I'm developing a few applications for clients right now in CGI::Prototype. The eventual application will likely be executed in a mod_perl environment, but I'm running it as CGI because I don't want any caching to interfere with my clean-slate testing, especially as I tweak various parts that will eventually be cached. The fact that Perl lets me do this makes my development time shorter, not longer. (And in fact, some parts of CGI::Prototype are possible only because I can blur the lines between "compilation" and "execution", so I have a richer framework to do my work, even if I use those features only indirectly.)

[skyknight]

I'm worried that if I mention Python one more time this weekend I might get dragged off by a lynch mob, but here goes anyway... When you import a module in Python, say module Foo.py, it writes to disk the byte code of the compilation in a file called Foo.pyc. The next time you run, it goes to do the same thing, but checks the time stamps on the .py file and .pyc file, only recompiling the .pyc file if it is stale, and otherwise just reading it in and ignoring the .py file. There are no Makefiles involved.

What is wrong with this idiom? I guess one tricky issue is where you put the .pyc files. If you don't have permission to the directory in which the .py files are stored, and the .pyc files aren't already there, then I suppose you have to figure out where to put the .pyc files you create, though I don't think this would be too terrible. You could have a python cache directory in your home directory, or something like that.

Why wouldn't such an idiom work for Perl?

[merlyn]

To be able to do that in Python, you have to give up source filters and prototypes in Perl. If you want no source filter or no prototypes, you know where to find Python, but I prefer that they stay in Perl.

If you want to promise never to use prototypes or source filters (or any module that might use those), you can certainly use B::Bytecode to create compiled ".pmc" files. In fact, I think the current "use" operator already probes for .pmc as well as .pm files.

---

Update: regarding .pmc files: seen in perldoc -tf require:

            Now that you understand how "require" looks for files in t
+he
            case of a bareword argument, there is a little extra
            functionality going on behind the scenes. Before "require"
+ looks
            for a ".pm" extension, it will first look for a filename w
+ith a
            ".pmc" extension. A file with this extension is assumed to
+ be
            Perl bytecode generated by B::Bytecode. If this file is fo
+und,
            and it's modification time is newer than a coinciding ".pm
+"
            non-compiled file, it will be loaded in place of that
            non-compiled file ending in a ".pm" extension.

[perrin]

I think the bottom line is that mod_perl and similar daemonizing tools (PPerl) makes all this work totally unnecessary. If those didn't exist, people would likely be a lot more interested in saving the in-between steps.

[MonkPaul]

Ah, now we have hit the nail on the head.

If pyton looks at the previous code and its time stamp etc, and other languages complie their code also, then surely this is an advantage if the program code is big and has lots of intergrated snipptes from other modules.
Yet, with perl, its runtime compilation makes it far quicker than the 'compliers' for small programs.

Can a middle man language be concieved, where, for instance it has the option of compiling the code again if its a large program, or performing a runtime execution.

[bart]

I seem to recall of some tests that tried this approach for perl, and that the resulting binary was so huge, that it turned out that recompiling perl from source was actually faster, than loading this huge binary from disk.

Can anybody confirm or deny this anecdote?

[merlyn]

I can confirm that it's an anecdote. {grin}

But I do also recall the same thing, somewhere back in the early perl5 days.

[chromatic]

Timestamps fail too often for Makefiles. If there's another comparison possibility, such as hashing the original source file and storing that in a header of the compiled file, it could be useful.

[Tanktalus]

To force caching on all users is a premature optimization, which is an evil step.

I disagree. First, I disagree that just because perl could cache the compiled bytecode that it must do so for everyone. There could be a commandline arg, or maybe a pragma ("use cachedbytecode;") which would enable it. Of course, if there is no downside to the caching (it's smart enough to deal with non-writable filesystems in a reasonable, unobtrusive manner, i.e., disable bytecode caching automatically), then I see no reason not to enable it by default.

Second, I disagree that it's premature optimisation. It's optimisation, yes. But premature? We already know the overhead of compiling code each time. We can compare that to the caching speed. If the cache is slower than recompiling, then we throw away that code, don't commit it to the main trunk, and document it. If the cache is faster than recompiling each time, especially in small programs (where compilation is a larger percentage of the runtime), then it's a proven optimisation.

Finally, I mostly disagree with the OP that this is really even needed. I laugh at my cow-orkers that work in Java. By the time that their code has finished recompiling, my code is almost done executing. And we have similar numbers of lines of code to work with. They won't even have their code loaded into memory by the time mine is finished running, the JVM load time is so slow.

My cow-orkers working in C/C++ are somewhere in the middle .. except that they're all writing JNI code, which again relies on that JVM load time :-)

[adrianh]

And hence, what then happens (in other programming languages) is that we make the users specify exactly when and how to cache the translation from the programming language to the intermediate language.

Image-based environments like the Smalltalks and Poplog would be another approach (which has its own benefits and problems of course :-)