High Performance Perl

Replies are listed 'Best First'.
Re: High Performance Perl by thundergnat (Deacon) on May 24, 2005 at 14:42 UTC
There is an article on perl.com by MJD (locally known as Dominus,) that explains this pretty well. (far better than I could myself.) What it boils down to is, it isn't that perl is so particularly slow, it just has a lot of overhead from the built in memory managment and flexible data structures. The very things that make perl so powerful and flexible, make it run somewhat slower than C/C++.	[reply]
Re^2: High Performance Perl by willyyam (Priest) on May 25, 2005 at 12:35 UTC
Thank you, that was an interesting article, and goes a long way to explaining things for me. I was asking because I get this sensation, more like the barest, subtlest hint, that people like writing code in Perl, but there are things that Perl isn't good for. So I thought to my self: 'Self, if all code of any language is just CPU math in the end anyway, why can't you write kernels in Perl - just use the derived math on bare metal, rather than parse-interpret-compile-execute?' I'm beginning to see that there is more to it than that - mostly due to memory issues - as I understand it, all memory allocations must be defined in size and type for code to be safe, and Perl does that for us (which is one reason to love it) but that added complexity precludes Perl's use on bare metal. Very cool. These computer thingies are complicated.	[reply]
Re: High Performance Perl by duff (Parson) on May 24, 2005 at 15:04 UTC
Perl is a highly dynamic language. A compiler can't optimize much because the programmer is allowed to change the world. Optimizations can only happen when you have guarantees. Perl 5 doesn't have much in the way of letting the programmer tell the compiler, "hey, this is an array of ints and I'll never ever put anything but ints in it" (for instance). Perl 6 has much more capability in this regard. I'm curious though about why/where people would say that perl doesn't have great performance. Of course perl isn't going to be as fast as C for most tasks because perl is interpretted bytecode. But where Perl really wins is in programmer productivity. I don't have an online reference handy, but in the book "The Practice of Programming" Kernighan and Pike use several languages to implement the same task. A table on page 81 of that book shows the run time and number of lines of code for each language. C had the fastest runtime at .3 sec but also had the most lines of code at 150. Perl on the other hand came in second in runtime at 1 second but was only 18 lines of code. Sure, the perl code is probably denser, but I'd still be willing to bet that it took far less time to write and debug those 18 lines than it did to write and debug those 150 lines. And the actual run time for the perl implementation was fast enough and that's often what matters. duff	[reply]
Re: High Performance Perl by jonadab (Parson) on May 25, 2005 at 12:30 UTC
Perl 5 doesn't have much in the way of letting the programmer tell the compiler, "hey, this is an array of ints and I'll never ever put anything but ints in it" (for instance) And if Perl5 did have that, it would be a neverending source of problems. I was gravely disappointed when I heard Perl6 will have some of this sort of misguided nonsense. The problem with having those sorts of optimization misfeatures is simple: many programmers don't know better than to use them. Read more... (2 kB) I'm curious though about why/where people would say that perl doesn't have great performance. People who don't know Perl use this as an excuse to avoid learning it. It's a meme that took hold eons ago, and it was probably even true at one time, in the pre-5 era. But in fact, Perl5 today has astonishingly great performance, considering everything that it does. If you try to duplicate all of Perl5's features, or even the major ones, using C libraries, it's not likely to be faster. (Granted, some programs don't need all those features.) And the actual run time for the perl implementation was fast enough and that's often what matters. Here it is: the only times I've had Perl be notably slow were times when I got careless and did something that would be slow in any language, such as store several hundred megabytes of data in an enormous nested hash on a system with 32MB of physical RAM (to process a year-end report; since it ran once a year, I could afford to let it run for a while), or the time I inadvertently coded an O(N!) algorithm that used RAM proportional to runtime. (I wasn't concerned about efficiency because I knew N was never going to exceed 20 or so; O(N^2) would have been fine and dandy, but O(N!) was too much, obviously. Actually, it was O((N^2)!); it worked fine for N=2, but when I tried to run it on N=3 the swapfile grew until I ran out of drive space.) No language feature can make that sort of thing efficient. (There are things that are noticeably slower in Perl than in C; I've just never written code that demonstrates this. I've been writing Perl since 2000 May. In summary, it's not an issue that comes up very often in practice, so don't sweat it. On the rare occasion that you run into one of those situations, you can treat the Perl version as a prototype.) "In adjectives, with the addition of inflectional endings, a changeable long vowel (Qamets or Tsere) in an open, propretonic syllable will reduce to Vocal Shewa. This type of change occurs when the open, pretonic syllable of the masculine singular adjective becomes propretonic with the addition of inflectional endings." — Pratico & Van Pelt, BBHG, p68	[reply]
Re^2: High Performance Perl by audreyt (Hermit) on May 25, 2005 at 14:25 UTC
I do agree that blindly using unboxed types (like Perl 6's `int`) is a Bad Idea. Actually, all uses of unboxed types, except in very tightly contained and verified places, are potential disasters waiting to happen. I'd much rather the Perl 6 compiler being able to infer when it is safe to use unboxed types, and only apply unboxing optimisations when it is sound to do so. At least, that is my goal in my work on Perl 6 to Parrot codegen as part of Pugs.	[reply]
Re: High Performance Perl by jonadab (Parson) on May 26, 2005 at 10:54 UTC
Re^2: High Performance Perl by audreyt (Hermit) on May 26, 2005 at 14:35 UTC
Re: High Performance Perl by jeffa (Bishop) on May 24, 2005 at 14:24 UTC
It's really quite simple. Money. Developer time costs far more money than hardware. If you really need "high perforance," then use C. But, if you want to get a product out the door in a much more timely manner that is still correct, robust, and maintable, then you can do so with Perl -- and other similar scripting languages, but Perl has the CPAN. jeffa L-LL-L--L-LL-L--L-LL-L-- -R--R-RR-R--R-RR-R--R-RR B--B--B--B--B--B--B--B-- H---H---H---H---H---H--- (the triplet paradiddle with high-hat)	[reply]
Re: High Performance Perl by dragonchild (Archbishop) on May 24, 2005 at 14:37 UTC
Here's something else to consider - 90% of all applications will run "fast enough" when written in Perl/Python/Ruby/PHP/VB/etc. Most people can't distinguish time intervals less than 1/20th of a second. If you're already there, you don't need to go faster. Plus, there's the problem that people expect certain tasks to take a certain amount of time. If the application goes too fast, the user may not believe the task was actually accomplished. I've actually seen applications with a slow-down built in, just to appease user expectations. In general, if you think something isn't in Perl, try it out, because it usually is. :-) "What is the sound of Perl? Is it not the sound of a wall that people have stopped banging their heads against?"	[reply]
Re: High Performance Perl by hardburn (Abbot) on May 24, 2005 at 16:19 UTC
Perl's current implementation doesn't really have bytecode, at least not in the sense that Java or Python does. Rather, it takes the tree generated by the parser and starts interpreting it directly. In Java or Python, there is another step where the tree is translated to bytecode and then the bytecode is interpreted. The advantage of interpreting the tree is that the compile phase is fast--you're pretty much done as soon as the parser is finished. However, it also makes it difficult to write the compiler's output to disk--abstract syntax trees are a lot messier than simple bytecode. Future Perls will be generating bytecode for Parrot. "There is no shame in being self-taught, only in not trying to learn in the first place." -- Atrus, Myst: The Book of D'ni.	[reply]
Re^2: High Performance Perl by Juerd (Abbot) on May 24, 2005 at 17:50 UTC
Perl's current implementation doesn't really have bytecode, at least not in the sense that Java or Python does. Hmmm. Then what do B::Bytecode and ByteLoader do? Juerd # { site => 'juerd.nl', plp_site => 'plp.juerd.nl', do_not_use => 'spamtrap' }	[reply]
Re^3: High Performance Perl by hardburn (Abbot) on May 24, 2005 at 18:12 UTC
Take the tree Perl normally uses, make it into bytecode, and then load it back into a tree for Perl to use later. They don't have anything to do with how Perl runs the program internally. Notice that it doesn't work very well for any non-trivial code, and you aren't gaining anything except shaving off the compile-time. "There is no shame in being self-taught, only in not trying to learn in the first place." -- Atrus, Myst: The Book of D'ni.	[reply]
Re: High Performance Perl by gaal (Parson) on May 24, 2005 at 14:31 UTC
Almost anything that runs on a virtual machine — for example, Perl — will almost always be slower than something that runs on bare metal — for example, c — especially if the low-level code has been lovingly optimized and tuned, but under proviso that the author of the low-level code hadn't run out of tuits making the code both fast and correct. If you have one hour to code something that uses associative arrays, chances are if you use Perl for the job you'll finish sooner and have fewer bugs than if you use c; plus the code might actually run faster (because perl's associative arratys have already been lovingly optimized and tuned for so many years). Of course, "one hour" and "something" are just arbitrary values; perhaps for you hashes in c is a trivial job (or maybe you're already familiar with an excellent c toolkit). Keep in mind that you can Inline code from other languages (from c to Perl 6) in your Perl code, and thus make the hot spots much happier.	[reply]
Re: High Performance Perl by brian_d_foy (Abbot) on May 24, 2005 at 20:56 UTC
Besides the issues with bytecode, interpreters, and all that other good stuff, I've found the the most serious slow down in any of my programs has been my (bad) design decisions. I find that bottleneck, research the problem, and usually come up with something that does it orders of magnitude faster. I guess it would be nice to pass around bytecode, but not being able to hasn't really stopped me from doing anything. Although a few other people have had other experiences, I've found most of my performance suckers happen at runtime, so the compilation time wasn't that big of a problem. The problem I dread, however, is that the same bytecode acts differently with different versions of a virtual machine (or the other way around). I experienced that situation in the early days of Java, althougth that was mostly because they were changing the names of a lot of things. I feel pretty good about passing the source around and letting the latest parser-interpreter combination figure it out. -- brian d foy <brian@stonehenge.com>	[reply]
Re: High Performance Perl by thundergnat (Deacon) on May 24, 2005 at 19:15 UTC
UPDATE: I've been unclear. Allow me to try again: As I understand it, Perl must be interpretted, rendered as bytecode, and executed. The question is this: how good is that bytecode, and similar/different is it from a compiled binary? I am asking because if the bytecode generated is good, why can't we distribute the bytecode rather than the source - skipping a run-time step and improving the performance along the way? This is sounding more and more like a FAQ. Check out: `perldoc -q compile` [download] Particularly the second item, in perlfaq3; Key quote: In general, the compiler will do nothing to make a Perl program smaller, faster, more portable, or more secure. In fact, it can make your situation worse. The executable will be bigger, your VM system may take longer to load the whole thing, the binary is fragile and hard to fix, and compilation never stopped software piracy in the form of crackers, viruses, or bootleggers. The real advantage of the compiler is merely packaging, and once you see the size of what it makes (well, unless you use a shared libperl.so), you'll probably want a complete Perl install anyway. It is worth noteing that perl6 should minimize many of those issues. My understanding is that the parrot VM executes byte code, and doesn't care whether it comes directly from the perl parser or a saved file. In that way it becomes more like Java. As long as a virtual machine is available, pre-parsed (compiled) scripts will run on it. Perl5 doesn't have a separate virtual machine for each OS, (well, it does but it is closely tied to the parser for each platform,) so the output from the parser is tailored to run on a platform specific VM; a portability nightmare.	[reply] [d/l]
Re: High Performance Perl by zentara (Archbishop) on May 25, 2005 at 10:47 UTC
After my years of hacking Perl, and trying to learn and compare it's speed to C and C++, this is what I have concluded. 1. The "script compilation phase" of running a Perl script is NOT the cause of it's slowness, compared to C. What makes Perl run a bit slower than C, is that Perl stores all variables in structures, which hold the variable, along with it's attributes, like type of variable, number of references, etc. This slows Perl down, when running long computation intensive scripts. However, this very same thing also makes Perl easier write and use. Try doing some simple tasks in C or C++, and you end up spending half your programming time trying to figure out "casts" and tracking what form digital data is currently in. It is why so many C, C++ programmers switch to using Perl, (or other scripted language). There is also the "buffer overflow" problem, which plagues C, which is just a "distant memory" for Perl programmers. 2. If you need more speed, it is far more economical just to buy a faster motherboard and harddrive. A Perl script running on a 2 Ghz machine with SCSI harddrives, will generally outperform a C program running on a 500 Mhz machine with IDE drives. So you can upgrade hardware for $300, instead of paying a C programmer $500 a week to boost your performance by 500 milliseconds per run. Of course there is always the place for C, in the underlying libraries, etc., but in general, Perl performance is very fast just the way it is, (assuming you write scripts correctly). :-) I'm not really a human, but I play one on earth. flash japh	[reply]
Re: High Performance Perl by TedPride (Priest) on May 24, 2005 at 17:48 UTC
Perl is quite efficient for an interpreted language, and it's much easier to write short, powerful code in. You can write your whole application in a day or two, make adjustments as necessary over the next few weeks, then when traffic starts to get high and efficiency becomes a concern, rewrite the least efficient parts of your application in a different language such as C++. Writing the whole thing from scratch in C++ would be hell. Or you can just buy a block or two of memory, which is probably cheaper than rewriting the code in C++. Hardly any applications will need more efficiency than Perl / Mod_Perl can dish out, anyhow.	[reply]
Re: High Performance Perl by Anonymous Monk on May 24, 2005 at 18:39 UTC
UPDATE: I've been unclear. Allow me to try again: As I understand it, Perl must be interpretted, rendered as bytecode, and executed. The question is this: how good is that bytecode, and similar/different is it from a compiled binary? I am asking because if the bytecode generated is good, why can't we distribute the bytecode rather than the source - skipping a run-time step and improving the performance along the way? Well, there's been efforts to create a perl compiler that does pretty much what you suggest, in an efficient way. The UNIX effort was called "perlcc", and I believe there's a Windows attempt as well. It's a hard problem, and hasn't been very successful (ie. after over five years work, the results are still marked "experimental"; perlcc now compiles a sample "hello,world" program, but still crashed on me (as of last year) trying to compile a program of reasonable size (3,000 lines or so, plus modules). -- Ytrew	[reply]
Re: High Performance Perl by casiano (Pilgrim) on May 22, 2008 at 12:51 UTC
In this tutorial: GRID::Machine::perlparintro I try to prove that computing performance can be improved without implying a overwhelming programming effort using a synergetic combination of C and Perl Hope it Helps Casiano	[reply]


Just another Perl shrine
	PerlMonks