Favourite modules March 2004

[Juerd]

Dear it in more than one way doing friends,

After Favourite modules March 2002 and Favourite modules April 2003, I think it is time to list my favourite modules for March 2004.

These lists are a very good way to learn about modules you didn't know before. This and other things have influenced my list of favourite modules. The most important reason for the big difference with last year's list is that I used sort to get the modules in a more accurate order.

sort {
    print "A: $a\nB: $b\n> ";
    { A => -1, B => 1 }->{uc substr <>, 0, 1}
} @foo;

For each choice between modules, I chose the one I liked better. I found that the one I like better isn't always the one that I use more often. According to Perl, this is my list for this year: (As before, excluding pragmata and my own modules)

1. LWP
2. B::Deparse
3. Carp
4. Template
5. Benchmark
6. DBI
7. DBD::SQLite
8. WWW::Mechanize
9. File::Slurp
10. File::Find::Rule

The modules that were in @foo but didn't make it are (in alphabetical order): Class::DBI, HTML::TreeBuilder, IPC::Run, MIME::Lite, Regexp::Common, Text::Reform.

I'll try to explain the difference with my list of April 2003:

• I haven't used IPC::Run a lot. I think I loved it mostly because I was used to open2/open3.
• LWP apparently is cooler than I realised. WWW::Mechanize is new and I really love it. I think it influenced my thoughts on LWP.
• To my surprise, Regexp::Common is not as useful as it seemed. It has many very useful regexes, but in a year my own regex skills improved. For most of the situations in which I was about to use a regex from %RE, that regex was slightly different from what I had in mind. I think I used only the IP and hostname matching. The enormous number of tests also made me like it a little less: it takes too long to install.
• Having done less email related programming, I almost even forgot about MIME::Lite.
• Template Toolkit is new in this list. It has simplified working together with non-programmers and forces me to code in a more structured manner.
• I have always liked File::Slurp, but with Uri Guttman's improvements it now deserves a place in the top 10.

The question for you is the same as always:

What are your favourite modules?

[valdez]

My list is:


• CGI::Application with CGI::Session, HTML::Template and Data::FormValidator
• Config::General with Class::Singleton
• DBI, mostly with DBD::mysql and DBD::Oracle, plus SQL::Abstract and a modified version of Data::Page
• Parse::FixedLength, Text::CSV_XS and Spreadsheet::ParseExcel to import data
• LWP, URI, Cache::Cache, XML::Simple, HTML::TokeParser and HTML::TableExtract to interface to this site
• Mail::Sender
• Math::Currency
• Regexp::Common

update: I think WWW::Mechanize is too powerful for my needs; I don't need an automated browser, only a way to retrieve nodes and possibly have a different cache policy for them. Does this satisfy your curiosity? :)

[petdance]

LWP, URI, Cache::Cache, XML::Simple, HTML::TokeParser and HTML::TableExtract to interface to this site

Have you looked at WWW::Mechanize and WWW::Mechanize::Cached? Are you doing something with your stuff that Mechanize isn't? I'm curious as to what it is, and what we can add to Mech.

[petdance]

update: I think WWW::Mechanize is too powerful for my needs; I don't need an automated browser, only a way to retrieve nodes and possibly have a different cache policy for them. Does this satisfy your curiosity? :)

That's fine. I'm just curious as to what's coming out of the parsing that you're doing that Mech wouldn't do. Also, WWW::Mechanize::Cached does all the caching for you. It uses Cache::Cache, just like you are.

[flyingmoose]

Lately, for me:
• File::Finder -- filter chaining all so much more elegant than File::Find
• Proc::Background -- makes me able to forget the differences of process execution between Win32 and other OS's
• File::Repl -- kind of cool non-local rsync-like module
• DBD::Pg -- the want for it never goes away :)
• WWW:Mechanize -- just played with it shortly, but definitely funky in a good way
• YAML -- actual concise hand-editable serialization! Yes!
• XML::Simple -- when folks aren't cool enough to use YAML :)
• Tk -- looks aside, you can't beat a clean GUI API
• MIDI::Simple -- has some fun examples, though my Computer Music for-fun projects aren't really going anywhere
• PAR -- solid gold for program distribution. Convert perl to binary, "essentially".

[larsen]

Instead of listing my favourite modules, I'll mention the modules I'd like to become more familiar with or I'd like to study. In a sense, this should be my Favourite modules March 2005 :). In no particular order:
• POE
• Parse::RecDescent
• CGI::Application::ValidateRM
• Test::* (and, mainly, the techniques to correctly test code)
• Class::DBI
• WWW::Mechanize
• Maypole
• SPOPS, Alzabo... (probably is better doing some research considering even different tools first)

[toma]

• YAML because I've gotten tired of hand-editing XML
• Storable for fast storage
• Spreadsheet::WriteExcel for delivering complicated data to non-perl people
• Audio::Data is fast, use sox to convert to wav
• GraphViz to draw networks
• File::Slurp for extraordinary laziness
• File::Flat because I'm just as lazy about directories as files
• HTTP::Daemon when I need to add a web server to a program, instead of the other way around.
These two seem to make simple things hard and hard things simple:
• CGI::Application
• HTML::Template

[Anonymous Monk]

File::Slurp and File::Flat? File::Flat has a slurp method, but neither it nor File::Slurp's use binmode (on purpose -- dumb).

[TVSET]

• Class::DBI is currently number one in my list. It brought back the job of programming.
• Date::Calc makes me hate current date/time system in use a little less. Sometimes I still wonder though, why not devide a year into a proper number like a 100 or 128. :)
• Net::LDAP does not need any comment.
• Everything else is a more or less one-time per task use, so no more names.

[greenFox]

++ for pointing me towards File::Slurp, I'd like to see this one added to the core along with an implementation of tilly's idea, perhaps r/R?

Favourite modules (picked because I use them a lot and listed alphabetically):

• CGI
• File::Copy
• File::Find
• File::Path
• Getopt::Std
• IPC::Open3
• MLDBM
• Net::LDAP
• POSIX - (mostly for strftime)
• Sys::Syslog

Modules to learn this year:

• Template-Toolkit
• Bryar

[TheDamian]

I'd like to see [slurp] added to the core

Then take a look at Perl6::Slurp for a Perl 5 implementation of our current notion of Perl 6's slurp built-in.

What? You meant "added to the Perl 5 core"? Sorry, can't help there... ;-)

[Anonymous Monk]

Why in CORE? Besides, module is broken.

[greenFox]

Because I use perl on a large number and variety of machines where installing modules is not always practical or possible, if it is in core I don't have to worry.

I understand there are moves afoot to "slim down" core and I don't object to that but personally I think both of the modules I mentioned are small and useful enough to warrant including them... not that I expect my opinion to carry any weight :)

[PodMaster]

I use lot of modules {and I like a lot of those I use}, but favorite status is very hard to achieve, which is why I only have a single favorite this time {I am excited}: Test::Pod::Coverage.

These I like, they're kind of new (some just to me)(:

• CPAN::DistnameInfo - about time :)
• Plucene
• Music::Image::Chord
• SQL::Translator

[castaway]

Favourite/Most used modules currently:
(no particular order)

• DBI - databases++
• DBD::SQLite - best little db around
• DBI::Shell - instant dbi
• Time::ParseDate - favourite date/time parser
• XML::LibXML - everything XML
• Storable - when no DBs are in sight..

Used but not much re-used:


• MailTools - everything mailish
• Class::DBI - abstract SQL/DB stuff
• Net::OSCAR - DIY AIM/ICQ client
• Term::Screen - term i/o
• HTML::Tree - HTML parsing

New kids on the block:


• Data::Dump::Streamer - compacter output than Data::Dumper
• Geo::IP::PurePerl - where is that IP?

Looked cool but failed to install itself+dependencies:
CPANPLUS

Module list courtesy of: perl -MExtUtils::Installed -le'print join("\n", ExtUtils::Installed->new->modules)' (only been using this machine since December)

im2 uses a whole bunch daily, including 5 date/time modules, damn I wish there was one that "did it all".

C.

[phenom]

I don't have a Top 10 List prepared, but I've been playing with File::Scan lately. I primarily use Linux - but when people send me viruses I like to see if this module can catch it. Most times it does, or at least labels them as "suspicious".

[dragonchild]

The enormous number of tests also made me like it a little less: it takes too long to install.

This statement made me pause. Why are a large number of tests a "Bad Thing"(tm)? I would think that more tests would imply better code. (I am not naive enough to think it ensures better good, but if two pieces of code were in front of me and 1 passed 100 unique tests and the other 1000 unique tests, I'd be more confident initially with the latter ...)

Plus, don't you install each distribution only once in a while?

[Juerd]

Why are a large number of tests a "Bad Thing"(tm)?

They become a bad thing when you have 157732 tests for 2245 lines of code. That is 70 tests per line of code, while of course a lot of lines don't even contain any logic. make test for Regexp::Common takes too long. I've already met a sysadmin who refused to install it because of that and I'm slowly becoming one myself.

if two pieces of code were in front of me and 1 passed 100 unique tests and the other 1000 unique tests, I'd be more confident initially with the latter

Of course, but do you really need thousands of tests for zipcodes? Are these thirty six thousand tests really needed for a palindrome regex?

Plus, don't you install each distribution only once in a while?

Yes, but I dislike this test suite that takes minutes to run for the same reason I dislike compiling from source. I am lazy and impatient and that is exactly why I like Perl.

There's only one thing regarding testing that I dislike more than an exaggerated test suite like Regexp::Common's and that is not having tests at all. I think R::C could do with only, say, a tenth or maybe a hundredth of its current test suite. For development, a huger one might be nice, but 1.5e5 tests on installation is more than I'd like.

[dragonchild]

I'm still confused. What's wrong with kicking off an installation and going to get a cigarette or a cup of coffee? Kick it off, then go to lunch. Or, kick it off at home and go watch the latest SG-1 episode ... That's what I do when I'm installing OSes or Apache, and those take over an hour each. *shrugs* I guess that it seems a little too impatient, in my book.

And, frankly, I don't think that thousands of tests for regexes is ridiculous. It's extremely easy to get them every-so-slightly wrong. Abigail-II is right, imho, to provide that kind of coverage. Those regexes are going to be depended on for business decisions. It might be a small part, but it's nice to know that "It Will Just Work".

And, the test suite on installation isn't always just for installation. I've played with making changes to many modules. I use their test suite for making sure my changes are backwards-compatible.

Now, maybe it might be useful for "make test" to have a "make test TEST_TYPE=dev" vs. "make test TEST_TYPE=install" (which would be the default). Then, the tester would indicate a subset of tests which are regression-only and which ones should always run.

Plus, you can install the distribution without running the test suite ...

[autarch]

Hmm, I just installed it and it only took about 60-90 seconds or so. That's not a big problem AFAIC.

[jonadab]

Plus, don't you install each distribution only once in a while?

Define "once in a while". I install CPAN modules less often than I use them, but I don't very often go a week without installing some modules off of CPAN onto one or more of the various computers that I use or administer at home or at work. Long install times are annoying, annoying enough that I have developed this little habbit of doing screen -R CPAN, starting the install, detaching, and doing something else for a while. But I can't continue to work on whatever I was doing until the install finishes. I wish more of the modules on CPAN were pure Perl so they wouldn't take so long to install. It is my sincere hope that in Perl6 XS will be unnecessary and heavily deprecated.

[exussum0]

SpreadSheet::Excel.. how i love AND hate thee. excel files in windows excel load up rather quickly, but it is such a bear for doing so with this terrible little module. But alas, it's so terrible, but works so well.

While I love LWP, I wish it were named something a bit.. nicer. When someone asked me on an interview once what it was, Light Weight Process was my immediate answer. Of course, they chuckled as I was told that I did very well on their test, 'cept they meant perl's LWP. (Doh)

[dragonchild]

Have you taken a look at Excel::Template? It simplifies the creation of Excel files, and uses the same data structure that HTML::Template uses, making Excel-ification of HTML reports quite ... simple. :-)

[exussum0]

Well, for converting from Excel to CSV, it's simple as they give you the example. :) Can't get simpler than that.

[Anonymous Monk]

While I love LWP, I wish it were named something a bit.. nicer.

How about something like the "World-Wide Web library for Perl". Then we could shorten it to libwww-perl, or even LWP ;)

Seriously though, those are two (three if you count both Spreadsheet::ParseExcel and Spreadsheet::WriteExcel) of my favorites - also:

• DBI
• DBD::Excel
• DBD::SQLite

[lestrrat]

Modules I use all the time:

• XML::LibXML
• Storable
• DateTime (<shameless plug>My DateTime Modules</shameless plug>
• YAML
• Params::Validate

Modules I'd like to know more about:

• POE
• Class::DBI
• Alzabo

[jonadab]

Here's the short list of modules I don't like to be caught without. These are the ones that I don't just find convenient from time to time but rely on constantly.

• CPAN. Perl isn't properly installed until this is set up, as far as I'm concerned.
• DateTime (I might die without this one.) Also certain DateTime::Format modules to go with it.
• DBI and one of the DBD modules. (I've been using DBD::mysql, but one of the others will probably do just as well. You want the corresponding DateTime::Format module, e.g., DateTime::Format::MySQL, to go with it.) I also have my own little wrapper module with functions for my most common queries (add (takes a table name and a hashref), update (ditto), and so on), but if I need to do something I don't have a function for I can always hand-build a query on the db handle.
• WWW::Mechanize saves a lot of time. If you do a lot of this, you might also want HTML::Tree as a companion. The look_down method is pretty keen.
• Archive::Zip is spectacularly useful, especially for dealing with OpenOffice documents. (Next question: will I eventually pick an XML module similar to HTML::Tree and use that in conjunction with Archive::Zip for parsing, modifying, and saving my documents? So far I'm still dealing with the XML by hand...)
• Data::Dumper is a really useful debugging aid. There's also Data::Dumper::Streamer
• Clone::PP is helpful. The most common things I clone are DateTime objects.
• Finally, I have a handful of my own include files that I wouldn't want to be without; I copy them to every system I use Perl on. These aren't modules per se, as I 'require' rather than 'use' them.

[autarch]

Clone::PP is helpful. The most common things I clone are DateTime objects.

But DateTime has a clone() method! Why not use that?

[jonadab]

But DateTime has a clone() method! Why not use that?

Oh, so it does. Cool. Well, mark that as one less use for Clone::PP from now on (though of course it'll still be useful for other stuff) :-)

[tsee]

The modules I use most in my stand-alone scripts are: (list was generated by a script, strict and friends removed)
• Data::Dumper
• Pod::Usage
• File::Find
• Imager or GD (using Imager exclusively now)
• Parse::RecDescent (for fancy input parsing)
• Getopt::Long
• DBI
• DB_File (for hash-persistance)
• Config::IniFiles
• CGI
• HTML::Template
• File::Stream
• Tie::Hash::Abbrev (for shell like interfaces)
• File::Copy
In my modules (http://search.cpan.org/~smueller), I tend to require other modules: (hand-selected list)
• Carp
• Data::Dumper
• Test::More, Test::Distribution (staring to use this), Pod::Coverage
• Parse::RecDescent
• Imager, GD
• many modules in the Math::* section since I'm developing Math::* modules myself pretty actively. Examples: Math::Complex, Math::Approx, Math::Spline, Math::Big*, Math::MatrixReal, etc.
• perltidy (or Perl::Tidy), well, at least indirectly.
• and many more...

Steffen

[coldhawk]

I don't currently use that many modules, because i fancy security and many modules that i would needed for this and that weren't that secure, and most cases just as lazy person i am, i wrote the whole code part myself. only in last resource i start fixing or changing thigs to way i like them on ready made modules, when i can't find or there isn't enough time/lazyness to start learning that. but what i have just browsed i have noticed there are lot of good modules out there, but i just haven't got time to start checking them out.

here are currently what i have used for this/last year often.

CGI = only used it for multipart forms, until i got some time to read more about their used format.
Digest::MD5 # always for passwords.
File::Copy # just to keep me from doing it without copy/unlink style.