Re: Are monks hibernating?

Replies are listed 'Best First'.
Re^2: Are monks hibernating? by BrowserUk (Patriarch) on Feb 14, 2007 at 03:02 UTC
It really does look like Perl has seen it's best years on the basis of those stats. Unless people are finding some other source of support and assistance. One wonders what it would take to re-invigorate interest? Will 5.10 have sufficient new features or improvements to rekindle the flame? Would an installable version of Perl 6 with sufficent performance to at least equal Perl 5 do the trick? I've been looking to find an alternative to Perl in order to take many of my interesting projects further. The single biggest reason is sub/method call performance that means that every layer of abstraction you add, the slower the code runs. I've stalled on several projects because I've initially started out with nicely structured code only to find myself de-structuring it to try and recover some performance. Perl OO is just so slow. I've investigated writing OO using Inline::C, but even then the wrapping process that mates C to XS to Perl just sucks away most of the performance gains. Writing XS directly can save some of that, but then you're in a whole other world of pain. I'd love the idea of moving on to Perl 6, but it's current levels of performance preclude that. There are plenty of alternatives, but none of them have the flexiblity of Perl. And each seems to have some quirk or caveat that niggles me enough that I end up back in Perl. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply]
Re^3: Are monks hibernating? by davebaker (Pilgrim) on Feb 14, 2007 at 05:31 UTC
For me, Perl is every bit as powerful and fun to use this month as it was last month. Even more, actually; new modules or improvements to existing modules are posted to CPAN every single day. Even if it's true that the level of participation has gone down, the level of participation is still enormous. As always, that participation includes people who are new to the language. Many of them are in the process of becoming regular Perl users who will make valuable contributions to the community or even to the Perl code itself. What does it matter if Perl isn't as "popular" as it was last month or last year? Sheer popularity matters when electing a Prom King and Queen at high school (although I've never seen that achievement actually listed on a resume). Perl is popular enough, meaning it continues to attract enough bright, interesting new people to push the language forward. "Re-invigorate interest"? That implies there isn't any interest or that there isn't enough interest. There continues to be an enormous amount of interest. It's more than enough interest. "Re-kindle the flame"? That implies the flame has gone out. No way, man. What we need to do on PerlMonks and in other public venues is to make the new people feel good so they'll stay a while. Some of them are on their first date. Others are going out for the second or third time. They are trying to decide whether they'd like to take the time and effort to get to know this person and possibly make a commitment. You're out to eat at a nice restaurant with your date (the "PerlMonks Cafe") but when you go into the restroom you overhear some guy talking about your date: "Yeah, I've gone out with her and she still gets dates but not as many as she used to ... so she's seen her best years." You return to your table a bit discouraged. Do those other guys know something you don't know, or are they just fascinated with the new girl in town who wears tight sweaters (we'll call her "Ruby")? If you let that kind of popularity be your guide and end your date now, you might well miss the opportunity to get to know the one who would be your loyal soulmate for life.	[reply]
Re^4: Are monks hibernating? by BrowserUk (Patriarch) on Feb 14, 2007 at 06:19 UTC
Hmm. You sound like me. But still I stick by my comments in the post above. There is, on the evidence of the statistics referenced it the post it replies to, a long term decline in the questions being asked about Perl. In as much as that reflects to some degree, the levels of interest and use of Perl, whilst neither critical nor worrying for those of us that use Perl, it is interesting. On that basis alone, I don't think it is unreasonable to wonder what it would take to curtail the decline, or even reverse it. As I mentioned above, I'm far from abandoning Perl. Even when I find reasons to code something in some other language, I invariably reach for Perl to prototype it first. And whenever I take one of my frequent forays off into exploring another new (to me) language, I invariably find myself returning to Perl for anything I wish to solve, rather than play at. I'm far from being one of the "fashion sheep" I refer to elsewhere. Still, Perl has it's limitations, and ignoring them or pretending they do not exist benefits nobody. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply]
Re^4: Are monks hibernating? by j3 (Friar) on Feb 15, 2007 at 18:36 UTC
... just fascinated with the new girl in town who wears tight sweaters (we'll call her "Ruby") I dated Ruby for a while. We hung out at the student center a few times. Caught a few on-campus movies. She's nice, but I'd prefer to meet with Perl behind the stacks any day. ;)	[reply]
Re^3: Are monks hibernating? by talexb (Chancellor) on Feb 14, 2007 at 03:56 UTC
Interesting -- there are good reasons to use Perl -- fast to prototype, works easily on many platforms, all those yummy modules on CPAN -- yet speed, while very good, is not Perl's best feature. For speed you do have to go to C or assembler -- that's something I know from experience. So at that point the question becomes, How much do you do in Perl, and how much in C? Should you bother to do any of the project in Perl? I haven't written a large project in C in quite some time -- about ten years -- but when I have time, I'd love to do it again. With enough planning, the right structure and a great development environment, it would be lots of fun. Alex / talexb / Toronto "Groklaw is the open-source mentality applied to legal research" ~ Linus Torvalds	[reply]
Re^4: Are monks hibernating? by BrowserUk (Patriarch) on Feb 14, 2007 at 05:16 UTC
For speed you do have to go to C or assembler You certainly don't have to drop that far. I certainly wouldn't entertain doing much of anything in a language that doesn't support automatic memory management. And there are plenty of languages that do that can match the performance of C. Or even exceed it for some applications. Nip over to The Great Computer Language Shootout to see what I mean. Take a look at Perl's lousy showing in the recursive Benchmark for what I mean about subroutine/method performance. The lack of tail recursion optimisation accounts for some, but by no means all of the problem. There is a substantial overhead in simple calling a subroutine, which effectively penalises the use of subroutines for abstraction unless they do a fairly substantial amount of work to amortise the overheads. Once you move to using OO, you add another bunch of overheads in class lookup; dereferencing instance vars; inheritance resolution; et al. And OO encourages the use of small methods. If you implement a nicely architected solution, it runs like a dog. Maybe that's a necessary penalty for the benefits of using a dynamic language, but even without jit, Java seems to do a whole lot better. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply]
Re^5: Are monks hibernating? by shmem (Chancellor) on Feb 15, 2007 at 15:27 UTC
Re^6: Are monks hibernating? by BrowserUk (Patriarch) on Feb 17, 2007 at 01:28 UTC
Re^5: Are monks hibernating? by talexb (Chancellor) on Feb 14, 2007 at 15:26 UTC
Re^6: Are monks hibernating? by BrowserUk (Patriarch) on Feb 14, 2007 at 16:38 UTC
Re^5: Are monks hibernating? by bart (Canon) on Feb 16, 2007 at 11:20 UTC
Re^6: Are monks hibernating? by BrowserUk (Patriarch) on Feb 16, 2007 at 11:46 UTC
Some notes below your chosen depth have not been shown here
Re^3: Are monks hibernating? by demerphq (Chancellor) on Feb 14, 2007 at 11:44 UTC
One wonders what it would take to re-invigorate interest? Will 5.10 have sufficient new features or improvements to rekindle the flame? Im hoping it will raise the excitement level somewhat. Ive had a modest amount of feedback about the new regex features that would indicate that there is some enthusiasm for a new release. Would an installable version of Perl 6 with sufficent performance to at least equal Perl 5 do the trick? I'm sure it would. Although I'm sure the current perl6 folks would object to the implication that the current perl6 isn't installable. :-) --- $world=~s/war/peace/g	[reply]
Re^4: Are monks hibernating? by BrowserUk (Patriarch) on Feb 14, 2007 at 12:49 UTC
perl6 folks would object to the implication that the current perl6 isn't installable. I wasn't having a dig. I just mean that a 300+mb dependancy (ghc) is hardly conducive to mass distribution. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply]
Re^4: Are monks hibernating? by blazar (Canon) on Feb 17, 2007 at 12:35 UTC
Im hoping it will raise the excitement level somewhat. Ive had a modest amount of feedback about the new regex features that would indicate that there is some enthusiasm for a new release. I've seen an impressive example in a recent clpmisc Abigail's post, also available from Google Groups.	[reply]
Re^5: Are monks hibernating? by demerphq (Chancellor) on Feb 17, 2007 at 21:53 UTC
Re^6: Are monks hibernating? by blazar (Canon) on Feb 18, 2007 at 09:10 UTC
Re^3: Are monks hibernating? by zentara (Archbishop) on Feb 14, 2007 at 14:50 UTC
Yeah, I noticed a falloff in posts around Christmas, lasting until the SuperBowl was over. Maybe people are too dulled from partying to be thinking about code? Actually today was the first day in a long time, that there seemed to be alot of posts. Not that anyone would notice my lack of posts, but I've been spending my time trying to learn enough c to do GLib . To me, it is a nice attempt to fix some c problems, like c strings, arrays ,hashs and automatic memory allocation. Its also a breeze to do threads with GLib, much easier than Perl, because it lets you use shared values and subs without declaring them as shared. I've noticed the C++ people have done alot with Boost , which has libraries for handling text that almost make it as convenient as Perl, but its about 200 Megs of libraries which you need to learn. I'll make an attempt toward Glib, but leave Boost for the next generation. :-) So it seems that the compiled languages c and c++ are making great strides toward better usablity...... but there are still those fcing Makefiles, which I don't ever think I will ever grasp. But maybe the lack of Perl questions, is a symptom of it's success? It has made almost all common problems easy to solve, and most questions have been repeatedly answered before, and can be found by a quick google search. I'm not really a human, but I play one on earth. Cogito ergo sum a bum	[reply]
Re^4: Are monks hibernating? by BrowserUk (Patriarch) on Feb 14, 2007 at 16:47 UTC
maybe the lack of Perl questions, is a symptom of it's success? It has made almost all common problems easy to solve, and most questions have been repeatedly answered I doubt that is the case. Newbies are newbies. Their first problem is always that they don't know what they are looking for. This place has become vastly more tolorant of the newbie over the last few years than any other similar on-line venue. If the newbies aren't coming here to ask their inevitable questions, then they probably don't exist. Besides, the most interesting questions (to me) usually come from the old hands. New twists on old themes. Rehashing of old arguments. Extended discussions on alternative approaches to familar problems. It's these type of questions that seem to be most lacking. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply]
Re^3: Are monks hibernating? by syphilis (Archbishop) on Feb 14, 2007 at 12:30 UTC
I've investigated writing OO using Inline::C, but even then the wrapping process that mates C to XS to Perl just sucks away most of the performance gains. Writing XS directly can save some of that ... I see (from time to time) vague allusions to the notion that XS can achieve something that Inline::C cannot - and the above quote is just one more example of that. But then ... the allusions are so vague that I'm never sure that I've understood correctly :-) Are you saying that there's some performance improvement to be had by writing XS directly (that can't be achieved with Inline::C) ? If so, could you (or anyone) give an example - more for my own edification, rather than for any other reason. Given that Inline::C merely autogenerates an XS file, I find it hard to comprehend that the one has any advantage over the other (re performance). I'll go back to sleep, now :-) Cheers, Rob	[reply]
Re^4: Are monks hibernating? by BrowserUk (Patriarch) on Feb 14, 2007 at 12:58 UTC
At the very least, using Inline::C adds another layer of subroutine call around the c function. Depending upon what parameters the sub takes and whether you declare them using C types or XS types, the translations in and out can add conversions and casts that can be unnecessary. Try comparing the post processed C with your original C function. It can be quite an eye opener. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply]
Re^3: Are monks hibernating? by ambrus (Abbot) on Feb 14, 2007 at 21:20 UTC
It really does look like Perl has seen it's best years on the basis of those stats. What? You think people are coming here for perl?	[reply]
Re^3: Are monks hibernating? by Withigo (Friar) on Feb 14, 2007 at 19:12 UTC
I've been regularly reading on here for years now and this is the first time I've heard that the major bottle neck in Perl is that sub/methods calls are slow. It's common knowledge that Perl is significantly slower than straight C, but are you saying that the bulk of that slow down occurs due to sub/method calls being slow? I've recently been poking around the perlguts, writing some XS/Inline code, and reading Perl's C source, so I'd love to get a peek at the underlying reasons for the slowdown you mention--even just a cursory peek which is probably all I could comprehend at this point.	[reply]
Re^4: Are monks hibernating? by BrowserUk (Patriarch) on Feb 17, 2007 at 01:25 UTC
...the first time I've heard that the major bottle neck in Perl is that sub/methods calls are slow It's common knowledge that Perl is significantly slower than straight C, but are you saying that the bulk of that slow down occurs due to sub/method calls being slow? I never said "the major bottleneck". And of course it is slower than straight C. It could not be otherwise. However, here is a partial breakdown of the costs of subroutine and method calls, and various forms of OO: Rate OO_7 OO_8 OO_10 OO_9 OO_6 OO_5 OO_4 OO_3 OO_2 OO_1 sub2 s +ub1 inline OO_7 70.4/s -- -0% -43% -44% -69% -70% -81% -88% -89% -91% -92% - +95% -99% OO_8 70.5/s 0% -- -43% -44% -69% -70% -81% -88% -89% -91% -92% - +95% -99% OO_10 124/s 76% 76% -- -1% -45% -46% -66% -79% -81% -84% -85% - +91% -98% OO_9 126/s 79% 79% 1% -- -44% -46% -66% -79% -81% -84% -85% - +91% -98% OO_6 227/s 222% 222% 83% 80% -- -2% -38% -62% -65% -71% -73% - +84% -96% OO_5 232/s 229% 229% 87% 84% 2% -- -37% -61% -65% -70% -73% - +83% -96% OO_4 367/s 421% 420% 195% 191% 62% 58% -- -39% -44% -53% -57% - +74% -93% OO_3 600/s 752% 752% 383% 376% 165% 159% 64% -- -9% -23% -29% - +57% -88% OO_2 656/s 832% 831% 428% 421% 189% 183% 79% 9% -- -16% -23% - +53% -87% OO_1 782/s1010% 1009% 529% 520% 244% 237% 113% 30% 19% -- -8% - +44% -85% sub2 847/s1103% 1102% 582% 572% 273% 266% 131% 41% 29% 8% -- - +39% -84% sub1 1388/s1871% 1869%1017%1001% 512% 499% 278% 131% 112% 78% 64% + -- -73% inline5179/s7252% 7246%4067%4009%2182%2134%1312% 763% 689% 562% 511% 2 +73% -- [download] Note the big step changes between `inline` code and `sub1`; the code runs nearly 3 times more slowly. The 'body' of the code, a single integer increment is not worth a sub, but it is designed to emphasis the cost of the calls, and OO support code. `sub1` & `sub2`; name the parameters and your code runs 5 times slower. `OO_3` & `OO_4`; use getters and setters and your code is now 13 times slower. `OO_6` & `OO_9`; use Best Practices and your code is 40 times slower. `OO_10` & `OO_8`; admittedly far better than traditional InsideOut objects which result in code that runs 72 times more slowly. Of course this is 'worst case'. If your application is IO-bound, or DB-bound, or only creates a few big objects, with a few large methods and only runs for a few seconds; then none of this is of the slightest concern to you. But if your applications are running cpu-bound over large volumes of data. If you create lots of small objects with small methods and call them lots of times. If your program runs for a few hours when coded inline, or as a few large subroutines. Then if you properly OO-ize it using Class::Std, it could end up running for a day or more for every hour the original ran. Or to look at it another way, if you take an app that uses properly structured OO approach using Class::Std and you bastardize it to use simple subs and inline code blocks. If the original ran for 6 hours, the destructured version might end up taking 10 minutes! The point is that even a few percent improvement in the costs of function calling would benefit every Perl application, and could have profound affect on the performance for some types of applications. Benchmark code Read more... (8 kB) Another way of looking at it is to take a heavily recursive function (Ackermann,) and re-write it to use exactly the same algorithm--repeating all the same motions, stacking and unstacking the same amount of data; building and unwinding the same number of code block frames--but do it without calling a subroutine. Ie. A recursive algorithm coded manually using a loop and a stack. This shows that using the identical 'recursive' algorithm, but avoiding the sub calls, `Ack( 3, 0 )` runs twice as fast. `Ack( 3, 8 )` runs 4 times as fast. And by the time you get to `Ack( 3, 9 )` it's a whopping 12 1/2 times faster. Note: There are no tricks here. No memoizations, or reduced recursion levels. When the truely recursive code stacks a parameter, this also stack a parameter. When it pops a parameter, this pops one. When it builds a block (sub) frame, this builds a block (while) frame. In as far as it is possible to emulate recursion through iteration this does so. It really shouldn't be possible to emulate what Perl does when it calls a subroutine, in unoptimized, Perl code, and beat it quite so emphatically. `c:\test>b-recursive-nosub.pl Ack(3,0): 5 Ack(3,0): 5 1 trial of Recursive Ack 3,0 (227us total) 1 trial of Iterative Ack 3,0 (111us total) c:\test>b-recursive-nosub 8 Ack(3,8): 2045 Ack(3,8): 2045 1 trial of Recursive Ack 3,8 (13.698s total) 1 trial of Iterative Ack 3,8 (3.328s total) c:\test>b-recursive-nosub 9 Ack(3,9): 4093 Ack(3,9): 4093 1 trial of Recursive Ack 3,9 (162.104s total) 1 trial of Iterative Ack 3,9 (13.234s total)` [download] The benchmark Read more... (1295 Bytes) At this point I had hoped to try and produce an annotated assembler level trace of the transitions into and out of a subroutine. I got part way (a couple of hours), through producing that and had finger trouble and managed to throw it all away. Forgive me for not starting over but it is incredibly tedious and requires a lot of concentration, and I'm simply not ready to face that again just yet. What I will say is, based on my inspection (on my Win32/AS 811 combination), of the disassembled C that is run when a Perl level subroutine is called, there appears to be considerable scope for optimisation. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply] [d/l] [select]
Re^2: Are monks hibernating? by xdg (Monsignor) on Feb 15, 2007 at 12:10 UTC
Interesting. Though from the first graph my initial question is what happened on Perl Monks, in the Perl community or in the programming world at large in November 2005? That seem to be a turning point of sorts. Update: For fun I trolled through Super Search. Hmmm... I used to be a saint and A Level Playing Field -- this is kind of interesting, actually Are we a dying breed? -- a general trend? (not this post specifically) Attack on Perl or Perl's need better PR (again) "Perl is the Cobol of the WWW" Some things I don't think are responsible :-) (Sometimes picked for title more than content) New Module Announcement: Object::InsideOut Seeking Programmers. Many-to-many relationships in databases : SOLVED OT: Ruby On Rails - your thoughts? Tim Vroooom ! when is the comeback ? Teaching Children How to Program TheDamian gets a mention on the Beeb PERL as shibboleth and the Perl community Can I please have simple modules? The Accessor Heresy What is Enterprise Software? Benchmarks aren't everything -xdg Code written by xdg and posted on PerlMonks is public domain. It is provided as is with no warranties, express or implied, of any kind. Posted code may not have been tested. Use of posted code is at your own risk.	[reply]
Re^2: Are monks hibernating? by jbert (Priest) on Feb 14, 2007 at 08:10 UTC
c.l.p.m could also be suffering from a (supposed) more general decline in usenet usage (or usenet usage for programming-related questions). It would be interesting to graph number of new monks per month. I wonder how easy it would be to calculate that?	[reply]