Beefy Boxes and Bandwidth Generously Provided by pair Networks
good chemistry is complicated,
and a little bit messy -LW

How good is Perl for AI?

by PetaMem (Priest)
on Nov 06, 2001 at 19:19 UTC ( #123590=perlmeditation: print w/replies, xml ) Need Help??


I would like to know your opinion about the suitability of Perl for AI tasks. Iīm currently writing a bigger NLP application in Perl and came to a short discussion with tilly in the chatterbox where he/she stated, that Perl is unsuited for AI.

Now tilly knows Perl most probably much better than me, but perhaps I know AI much better than tilly and I have a pretty different opinion about the issue.

Sure: Perl hasnīt made much ado in the AI world. At least not as much as Common Lisp or Scheme have. But on the other side: havenīt the results AI has produced so far been worth a laugh?

I will try not to make this node an advocacy for Perls AI fitness. Lets just state, that there are people thinking perl is good for solving AI problems and there are people thinking perl is not suited.

I have stumbled across some links that connect perl to AI in one form or another and I would like to present them here. None of those shows significant usage of perl, that couldnīt be done with almost any othe programming language also. But in my opinion that could be because of the lacking perl KnowHow amongst AI scientists in this special CS discipline.

If you search for NLP (Natural Language Processing) on Perlmonks, youīll find exactly one node asking for some references. There are some references, many of them outdated, others working but very rough, like David MacKays AI perl tidbits

The Perl @ Cannon Research Centre link holds some interesting code for crawler and metatext applications, but nothing that couldnīt be found on CPAN also.

If you search directly for AI, you find the homenode of the user AI who never logged in. All in all it seems, like Perl hasnīt made it into the AI world. Until you search Google - that is: Then youīll find General Problem Solver, Chatterbots,... and you also find there some emulations of Lisp-Functionality in Perl.

Unfortunatedly, the Perl for artificial Intelligence Mailinglist is not working. I will try to find out if it died or just moved.

One reason why I would like to talk about this issue here is, that I read the Slashdot discussion about Lisp or Perl and thereīs an interesting article there, titled: "couldn't have picked a worse forum to ask in"

But perhaps all this talk about which language suits better is complete nonsense and one should program in that language he feels most comfortable with. Perhaps - as Iīve even read somewhere at perlmonks - the language itself is not so important as the libraries it comes with. And most probably the used language for a given task is not half as important as the skills of the person using it.

Give me your input.


Replies are listed 'Best First'.
Re: How good is Perl for AI?
by footpad (Monsignor) on Nov 06, 2001 at 19:54 UTC
    But perhaps all this talk about which language suits better is complete nonsense and one should program in that language he feels most comfortable with. Perhaps - as Iīve even read somewhere at perlmonks - the language itself is not so important as the libraries it comes with. And most probably the used language for a given task is not half as important as the skills of the person using it.

    I think these points are the most relevant, for while I do not know much about AI programming (and certainly know less about Perl than tilly1), I believe that computer languages are just tools. The most important thing is learning to use them well.

    Personally, I try to avoid "which language is better for whatever" (especially ones that show up here). Frequently, such discussions devolve into flame wars. While it's educational to learn what others think of a language and how they use it, each of us has different preferences and find different things appealing. Thus, such discussions will inevitably require a certain amount of "agreement to disagree" about various points. (I think it's safe to assume that most of us know how easy it is to find that flexibility on the other site you mentioned.)

    I do think that certain languages are more suited for certain tasks. For example, I probably wouldn't use a Perl CGI script to play Solitaire. It could be done, but that's not my point. Most tasks can be distilled into things that each major language does decently, if not well. Specifically, you have data that needs to be stored, processed, and reported on. Frequently, the challenge is determining how best to assemble the language's syntactic tools into a workable solution for the task at hand. Once you know how to build certain things in one, you can certainly build similar things in others. The trick is learning each language's strengths for doing so.

    For example, it's trivial to build database applications in many languages, however, nearly all such applications require certain skills, such as normalization2 and/or SQL of some form. Thus, time spent learning these skills is well invested, even if you plan to eventually replace your original implementation with another written in a different language.

    Since AI programming is, at its heart, the art of automated decision making, perhaps it would be best to use the language you're most comfortable with to prototype a rule-processing core that works for the project you have in mind. I know I'd love to see something where I could define a series of conditions, add in some "fudge factor" that lets the software do more than I originally envisioned, and then let it loose on various annoyances. For example, I'd love to see something I could run against my mailboxes that would automatically--and intelligently--separate the SPAM from the real messages.3

    Having said all that, Prolog was once touted as good AI development tool. Perhaps that's worth a look.


    1 - Questions regarding a monk's personal details should be directed to the monk in question, either via email or private /msg. Personally, I don't see how a monk's gender is germane to the discussion. The information is not going to be more valid if they're male, female, or whatever. Not trying to be too militant about this, but it's something I'm sensitive to. Some monks consider such details private and I don't believe it's up to the others to second-guess those choices. Besides, we've already lost members over similar discussions and I'd rather not see that recur.

    2 - Ever notice how few people seem to know how to normalize databases? Or, for that matter, how to separate business rules from presentation? Yet, these two skill sets can be applied to most projects developed for business use.

    3 - I actually have this on my personal project "to do" list, along with a dozen other things that I plan to get to "soon." (Yes, I know there are things out there already, but I keep hoping that someone will come up with the definitive version before I end up teaching myself the idiosyncracies of all the cruft I get in my daily dose.)

Re: How good is Perl for AI?
by dragonchild (Archbishop) on Nov 06, 2001 at 20:04 UTC
    Just as in every single large project, there are several levels to working on an AI program. I personally think Perl is eminently suited to certain aspects of AI development.

    I, for one, am working on a program to play Go. I'm using pure Perl. Why? Because it's the easiest language I know to do effective proof-of-concepts in. It gives the highest-level datastructures I've ever seen, a ton of good modules that make the hard stuff easy and the impossible stuff merely hard, and pretty decent speed to boot! (So, Perl's a RAM hog. RAM's cheaper than my time!)

    I'm working on the design for an intelligent MMRPG world. I will definitely be using Perl for the prototyping of the AI subroutines. I have some very different ideas on how to get (at least the simulation of) intelligent behavior. They don't fit very easily into my understanding of rules-based languages. Therefore, I will use the language I know best to do the initial stages.

    Now, does this mean you have to release in Perl? Of course not! Go to the language(s) you feel are appropriate for the needs of the release. Do you need to squeeze RAM? CPU cycles? Different syntax needs? Maybe the team chooses another language? Are you getting subsidies from someone? There's all sorts of reasons to pick one language over another, and there's usually not enough discussion on which to choose.


    We are the carpenters and bricklayers of the Information Age.

    Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.

Re (tilly) 1: How good is Perl for AI?
by tilly (Archbishop) on Nov 06, 2001 at 22:07 UTC
    First of all, it shouldn't matter what my gender is. It is no secret, if you browse my nodes long enough you can find a fuller name than just tilly, and then it should be dead obvious what my gender is. But if you wanted to know, I would strongly prefer that you asked me directly rather than posting the question to the winds.

    As for the actual question, well there are a lot of things called "AI". For some of them Perl is a good fit. For others it is not. When Perl is a fit, it is a fit for the same reason that it is good for other things. You can do complex stuff easily, you have a lot of high-level constructs, you have useful stuff on CPAN. I am sure everyone here can find other things to put on the list of good things about Perl.

    But Perl also has the following well-known disadvantages:

    1. Excessive memory use. If your program winds up working with and correlating a lot of data, Perl may fall over before you expect it.
    2. Poor performance. Many tasks that we call AI involve a lot of computation, backtracking, etc. Perl is good for a scripting language, but is is no speed demon. You do not, for instance, find good ray-tracers or chess programs written in Perl, for good reason.
    3. No true garbage collection. People argue back and forth whether or not it is better in general to have reference counting or true garbage collecting. But there is no question that if you want to manage recursive data structures with self-references, then it is much more convenient to have true garbage collection. The WeakRef module helps, but not enough if you are doing something complex.
    Now when you came by in chatter, your problem was that you were running out of RAM on your cluster of rather nice machines. Now after discussion, some of that didn't seem to really be Perl's fault. A lot of it appeared to be the recopying of data inside of Parse::RecDescent. (I have heard multiple times now that TheDamian is working on a replacement that uses pos manipulation rather than recopying. That likely would solve your problem.)

    But still when you are bottlenecking on known issues in the language, that should make you wonder at the suitability of the tool. So in this instance you would try to get the memory usage down and performance up. If you cannot get them within acceptable parameters, switching to Lisp, Dylan, Haskell, or OCaml would be perfectly reasonable.

    But if it works for you, then it works for you.

Re: How good is Perl for AI?
by stefan k (Curate) on Nov 06, 2001 at 21:26 UTC
    Fellow Monks,
    strange thing to happen: I'm just reading this very PM posting when I am very unhappy with my current AI project on the other desktop and hit the reload buttons to calm my mind....

    Well, the last time I started a new project (for my PhD) I chose ruby. Until today I felt quite content with it. Many of the things I wanted to do were implemented quite fast and in a way it's almost as much fun coding in ruby as it is coding in perl.

    Today, though, I set some internal variables of my machine to real values (like setting the size of a population from 3 for testing purposes to 100 to start something like a real world problem) and BOOM it crashes on my head segfaulting at various places throughout the programm (depending on the settings of my variables).

    Consulting the comp.lang.ruby newsgroup produced a similar backtrace to mine but no real solution. I'm now hoping for an email of a poster on that group.

    Why I'm telling all this crap? Well, first off just to get it off my brain (thank you for listening - it already feels a bit better now :-). The second point is that perl has got a huge code base plus an almost as huge coder base plus the PerlMonks community to call for help, perl has a long history in which it was used in many projects of different sizes (thus it is probable that it works with lots of data just as it does with small amounts) plus there are already a few AI modules available.

    Taking into account that many AI problems have been adressed with C (or C++) it should be possible to do just that in perl. I once taught a pupil some first steps in programming (she didn't know anything!) and because we had just a week or so I chose perl. At the end we wrote hello_world as a small and simple genetic algorithm. The next week I released the module to PM. That was AI (GA), it was easy and it worked. I think it should be possible to do other things in perl, too.

    As a matter of fact just today I regret having started ruby the other day (but that may change tomorrow when I maybe get a solution, who knows?)

    Uh, and you got eval for GP *grin*

    Regards... Stefan
    you begin bashing the string with a +42 regexp of confusion

      stefan- just out of curiosity, have you evaluated the library i mentioned in this node? i was just thinking the other day that i should play with it a little.
Re: How good is Perl for AI?
by toma (Vicar) on Nov 06, 2001 at 21:09 UTC
    One of problems with this question is that there is rarely consensus on whether or not a problem or a solution is 'AI.' In my experience most AI work is figuring out the right algorithm, not so much coding.

    On my last project, I worked with a PhD AI specialist at our corporate labs. He had programmed AI systems in LISP for years. His part of the problem was NLP of PDF documents. He used perl exclusively on our project. I had no influence on this decision since he had already started the project when I learned of it.

    Part of the project also required image segmentation. I did that part in C for speed, but I fed the results to a perl program.

    Another part of the project required table decomposition. That part of the project was coded in Java.

    I was able to attend the lab reviews for this fairly large corporate labs research group. The solutions being sought tended to require natural language processing, visualization, image processing, statistics, and graph theory. The only languages in use were perl, C, and Java. I never heard anyone discuss LISP except as a story from long ago.

    It should work perfectly the first time! - toma

Re: How good is Perl for AI?
by frag (Hermit) on Nov 06, 2001 at 20:31 UTC
    The AI List moved to, and is also available at Activestate.

    As for Perl's suitability for AI -- it's irrelevant to me, personally. I prefer programming in Perl, I'm interested in AI, purely on my own and not for work; therefore, I'll use Perl for AI.

    -- Frag.
    "Just remember what ol' Jack Burton does when the earth quakes, the poison arrows fall from the sky, and the pillars of Heaven shake. Yeah, Jack Burton just looks that big old storm right in the eye and says, "Give me your best shot. I can take it."

Re: How good is Perl for AI?
by blakem (Monsignor) on Nov 06, 2001 at 23:40 UTC
    If you search directly for AI, you find the homenode of the user AI

    When this happens, you should use super search or try this little interface trick: Add a gibberish second term to your query to prevent an "exact match". A search on 'AI asdf' will bring up a list of relevant AI results, rather than jumping directly to node AI.


      Some of the most useful things are often burried in larger discussions :-)
Re: How good is Perl for AI?
by Hanamaki (Chaplain) on Nov 07, 2001 at 01:17 UTC
    I am not sure what AI is -- or to say it in other words -- whether something like AI really exists. Anyway it seems much better to ask whether Perl is suitable for a given task inside AI or NLP.

    Okay, I have to admit I never have done AI, and if you ask me I will tell you that I have no clue about AI, I am just a dumb Natural Language Engineering specialist who calls himself sometimes Computer Linguist or Consulting Linguist .

    And yes for my NLP work I almost exclusivly use Perl, because its perfect for Data munging. While you will not write an embedded speech recognition engine in Perl, more than 90% of the actual work to produce such a beast can be done in Perl. And its done in Perl.

    For Corpus Linguistics -- very often an important starting point for Natural Language Engineering -- Perl is a pretty good tool, not only for prototyping.

    Some lightweight parser you could realize in Perl as well, but if you have large dictionaries etc. you probably will need some XS or C programms for speed.

Re: How good is Perl for AI?
by FoxtrotUniform (Prior) on Nov 06, 2001 at 22:05 UTC

    I know SFA about NLP, but I'd imagine that Perl would be quite suitable: it has excellent text segmentation support, which I'd guess would be useful. (Although I'd think semantic structure would be more important than text chunking... argh. Too much image processing in brain.)

    As far as Perl's "suitability" goes: Perl's really good for hashing out proof-of-concept code and dealing with text, which would (likely) make it good for some aspects of AI programming. By the same token, Lisp is good for, say, genetic programming (since its code/data separation is even thinner than Perl's), Prolog's good for planning/reasoning (because its logical syntax makes encoding knowledge easier), and C's good for search-based and general number-crunching (neural nets, anyone?), because, well, you can write some bloody fast C code.

    In my (limited) experience, a lot of AI involves a lot of number crunching, which Perl isn't so good at. (Not that it can't crunch numbers, but it can't crunch them fast enough. And let's face it: you're going to be a happier camper if your program takes one second to run than if it takes five minutes, especially when you're tweaking parameters.) I'd be most likely to write a prototype in Perl (provided Lisp, Prolog, and Haskell are unsuited for the job), make sure that I'm on the right track, then recode in C for the efficiency win. As always, YMMV.

      On of my pet projects is a word segmenter, meaning a programm which gets sentences and spits out words. Easy to get good results for the English language, but pretty hard for languages like Japanese were you won't find any spaces between words. A pretty though task were academic and commercial research aren't that advanced.

      I have to say for this task Perl is the perfect language. I don't care on speed but only on results. If I get satisfied by my project I may implement it in C, or may not. Call it prototyping, call it research, call it whatever, I won't try it with Prolog or Lisp.

by Fletch (Chancellor) on Nov 06, 2001 at 23:10 UTC
Re: How good is Perl for AI?
by elusion (Curate) on Nov 07, 2001 at 03:23 UTC
    Good question/discussion. Here are my thoughts on the topic.

    TMTOWTDI, but there is probably only one or two right ways to do it. It really depends on what area of AI you're focusing on. AI is such a general topic that for choosing a langauge you need to split it into different catagories. You also need to decide what your objectives are.

    For a task such as NLP, Perl is a very appropriate language. It provides good (read effecient) methods for this task. You have regexes, data structures, and weak type. Those things are all good for something like this (at least I think so).

    However, NLP is just a small portion of AI. Artificial Intelligence is defined as the ability to make intelligent decisions. Most often, this involves interacting with the world, or at least a psuedo-world. Let's see, interacting with the world, bring any specific type of programming to mind? Oh yes, Object-Oriented programming. I'd imagine that'd be good for something like this.

    Perl has OO (object oriented). It's not the best OO, but it's OO none the less. So do I support using PerlOO for real world tasks in AI? I didn't say that. Other languages provide much cleaner and simpler OO than Perl. My personal favorite is Ruby. It's clean, it works. It is slow though. This could be a reason not to use it. Maybe Java or some other language would be better suited.

    I would do most of my AI programming in Ruby. I like it because:
    1. It's OO -- good for simulating real world problems
    2. It has regexes, just like Perl does
    3. The ability to create methods such as [] and + delight me

    You may want to take this with a grain of salt. I've done very little AI programming. I've read a couple books. I'm currently trying to do some NLP in Ruby. So far it's going good. It is a memory hog, way more than Perl is, and it's slow, but I think it's worth it. Just my $.02

    elusion :

Re: How good is Perl for AI?
by Chmrr (Vicar) on Nov 07, 2001 at 08:47 UTC

    First off, as others have noted, the Perl AI mailing list is indeed functionning -- but it currently very low traffic. The following headers may help:

    List-Post: <> List-Help: <> List-Unsubscribe: <> List-Subscribe: <>

    As for using Perl for AI -- I have fooled around with the idea before. I found Perl's speed to be the only major limiting factor. I answered the question "why Perl?" with the following points:

    • Because I know it, and am comfortable with it. At that point, I was unwilling to leard Scheme or Lisp just to do AI.
    • Because I learn more writing my own system. There are innumerable more implementations out there, some more well-known than others. If I take it off the shelf and use it, I gain no deeper understanding of how it works.
    • Because I can tailor it to my needs. I was planning on making some fairly strange tweaks to how basic GP worked -- and to jump right into someone else's code, which might not have been built with such extensibility in mind, is a scary thought.
    • Because it's mostly cross-platform. Yes, so is Java and C, and lisp is mostly. But so is Perl -- why not?
    • Because I want to be crazy. Nobody had ever written a complete GP system in Perl. I didn't want that to be a possible point of leverage against a language I liked so much. ;>
    • Because I want to give something back. This goes along with writing my own, as well as being crazy. I was hoping to come out of the project with something that might be mildly useful to others. The chances of me doing that had I chosen another language were basically nil. I'm still trying to get myself to put it on CPAN, though. It'll get there Real Soon Now, I promise. I also got pings from the person who is writing a new genetic algorithms library. Unfortunatly, I've been lax in getting my implementation working with his system.

    Anyways, just some pseudo-random thoughts.

    perl -pe '"I lo*`+$^X$\"$]!$/"=~m%(.*)%s;$_=$1;y^+*`^ ve^#$&V"+@( NO CARRIER'

Re: How good is Perl for AI?
by poqui (Deacon) on Nov 06, 2001 at 22:08 UTC
    Aw shoot!

    When I first started reading this post, I saw NLP and thought "Kewl! Neuro Linguistic Programming AI in Perl! Wow!" But then I read on and saw it was "Natural Langage Processing" instead... But AI in Perl is still pretty cool.

    I agree that your facility with a General Purpose language can turn out to be more important for any task, than the specific suitability of a language for a task. If the language is powerful enough, you can always mock up your own modules to handle the stuff the language doesnt do naturally.

    Good luck and keep us posted!

    "That that is, is... for what is that but that? and is but is?" Shakespeare, Twelfth Night, Act IV, Scene 2

    "Yet THAT which is not neither is nor is not That which is!" Frater Perdurabo (pseud. Aleister Crowley), Liber CCCXXXIII, The Book of Lies
Re: How good is Perl for AI?
by tonyday (Scribe) on Nov 07, 2001 at 12:26 UTC
    If and when the AI robots emerge from their non-conscious electric swamp they will undoubtably be powered by perl or a descendent. Why, because our definition of intelligent is based on ourselves and humans are a weed-like generalist species able to survive in many environments. Sounds just like perl to me. Perl will be like the English language; mutating rapidly, consuming the best ideas from competitors.

    Heinlein - "Specialisation is for insects"

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://123590]
Approved by root
[LanX]: unusual that "Pimp my Bride" never made it into TV (?)
LanX sighs ... no idea stupid enough
[choroba]: "Smack My Bitch Up" did, though
[Eily]: that's the internet for you LanX
[Eily]: and I'm pretty sure there's a rule somewhere that states that the stupider the idea, the more likely it is to already have been done
zentara is thinking of moving to Mars to esacpe it all
[LanX]: Zentara eq Dr Manhatten ?

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (14)
As of 2017-03-27 15:20 GMT
Find Nodes?
    Voting Booth?
    Should Pluto Get Its Planethood Back?

    Results (320 votes). Check out past polls.