Unwritten Perl Books: 2007 version

[lin0]

Greetings Fellow Monks,

Some days ago I was reading the thread on Unwritten Perl Books. It made me wonder: are the comments on the thread still valid now in 2007 (the node was written in 2004)? On my side, I would love to see a book on the use of Perl for Data Mining from Open APIs. In particular, I would love to see a book in which the authors show how to

1. gather the data
2. modify the data to facilitate the analysis
3. visualize the data
4. use some numerical capabilities such as those in the PDL for analysing the data
5. visualize the results

This is what I had in my wish list. What do you have in yours?

Cheers,

lin0

[Ovid]

My top choices:

AI Exploration in Perl

Perl is too slow for AI programming in general, but it's a great way to explore basic concepts and perhaps whet people's appetites.

Perl for Students

Build simple dynamic Web pages. Download music. Post to your blog. Tell the RIAA to go stuff themselves.

And while I don't know that it would make an entire book, more information about using chromatic's P5NCI would be great. It would make it much easier to use a lot of existing C or C++ code for the traditionally slow bits of Perl.

[Muggins]

I agree about P5NCI, would be nice to see more on this. There's also a similar looking thing called FFI on CPAN. I wonder how these compare/contrast?

Sadly, I couldn't get these to install on Windows XP, (a while back so I can't remember what problems were holding me up) but there is Win32::API. Also the Inline and InlineX modules are great of course.

Of course there's Extending and Embedding Perl which covers XS, a bit of Inline::C and PDL, and embedding Perl in C. But things have moved on a little.

[Jenda]

I wonder how does it differ from FFI. And which one if more likely to stay around and supported :-|

[lin0]

Hi Ovid,

Perl is too slow for AI programming in general, but it's a great way to explore basic concepts

I completely agree with this comment. However, we should not forget that using wrappers to well established C/C++ or Fortran routines can provide the benefits of an easy to use application without affecting the processing speed (see for instance PDL)

Build simple dynamic Web pages. Download music. Post to your blog...

This one would be a very interesting book, indeed

By the way, thanks for the reference to P5NCI, it looks like a very useful and interesting module I did not know about

Cheers,

lin0

[Zaxo]

I think the tie interface, PerlIO, and the fragmentary attributes syntax all three deserve a good solid treatment in print. I wouldn't write a book about any of those explicitly, but one about problems solvable by them would work.

The tie interface is the only subject adequately treated by the camel book, but the examples there only scratch the surface of what is possible with it.

[brian_d_foy]

Mastering Perl has a chapter on tie. I think it's one of the longer chapters in the book.

One of my goals in Mastering Perl is to deal with some advanced topics, but also show the programmer that you're not going to be able to get a book on everything you want to do. At some point, you have to stop being merely a consumer and start integrating information on your own.

A book is too much for some of these topics. It's a waste of time for almost everyone involved because the sales are so little. I just spent a year and a half on Mastering Perl, and if the trail off from Learning Perl to Intermediate Perl continues to Mastering Perl, it means that I'll make almost no money from book sales for the tremendous amount of effort for the latest book. Other Perl books that would seem to have a larger natural audience don't even sell out their first printing. The potential market for advanced Perl books is large, but the actual market is pretty small. Lots of people claim to like books, but very few actually like to buy them. :)

O'Reilly and some other publishers have been experimenting with PDF-only publications, but that really only saves on the printing and distribution costs (which can be significant). It doesn't really save on author or editor time if they want to maintain the same level of quality. That is, it doesn't make it any more profitable for the author, the biggest limiting factor in the amount of information out there.

Better than books are directed articles for something like Perl.com, The Perl Review, or even a blog entry. I'd be happy to publish articles on most of the topics listed in this thread. Someone just has to write them.

Don't wait for other people to write articles though. Nothing gets done by waiting for the people who know the subject to explain it. Someone gets to step up, learn the subject, and write about it. It's a great way to learn new things.

[dragonchild]

Read the code for DBM::Deep if you want an education in the advanced use of tie, particularly with bless.

[TGI]

I would like to see a really good book on POE.

I know the latest Advanced Perl Programming has a chapter on POE. But I'd like more.

Actually, a book on event driven programming with perl would be even more interesting. That way it could tie together common concepts from GUI programming (Tk, Tkx, etc.), POE, and Event (I'm sure there are many others).

I'd also like to see books on Wx and GTK2 programming. The biggest problem with these libraries is the lack of docs.

[zentara]

I would like to see a giant ~3 cd set of working Perl5 example scripts, all catagorized, and commented inline. Sure everything is already out there on Google and various websites, and in certain monk's collections; but it would be nice for beginners to have instant access to all of the working knowledge. (But I guess that would make programming too easy :-) )

[jhourcle]

As anonymous mentioned, one of the important things is in making sure that you're getting good examples. (and then there's the issue that what's good for one level of coder might not be good for others)

For some examples of code, you can try krugle or Google codesearch (I've been known to search for 'use PackageName' to find examples, even on search engines that aren't code-specific)

For the best examples ... I'm not sure what's really 'beginner' quality these days ... for more advanced stuff, you might take a look at the NMS work by london.pm, As it's been produced by a group, I find it has better commenting than many solo projects.

[Anonymous Monk]

Yeah, there should be some sort of archive that has everything in one place. We could call it the Complete Perl Examples Library. It could have a search engine so people could find things, and a module so people could download everything to their machine. We can even set up a group of people to automatically test everything in ComPEL so we know newbies are getting good stuff. We could set up a site where other people could comment on the examples and rate them, too. That would be really cool. I wish Perl had something like that.

[derby]

Well ... not to open a huge can of worms but I would like to see Effective Perl6 Programming or the Perl6 Cookbook.

[samizdat]

I like that thought, lin0. I think I'd like to see one on building web systems for data analysis, with a discussion of graphing and visualizing on the fly. I built a lot of such systems at Sandia Labs, but I wouldn't say my code was all that elegant although the users were sure happy. I'm sure there are Mo Betta Ways To Do It. :D

[lin0]

Hi samizdat,

I think I'd like to see one on building web systems for data analysis, with a discussion of graphing and visualizing on the fly.

me too :-)

And since we are talking about web systems, I would love to read more about the building of Yahoo Pipes

By the way samizdat, can you tell us a little bit more about the systems you are developing at Sandia Labs? Something like which modules you use and why, how you deal with large amount of data and response time of your system, etc.

Cheers,

lin0

[samizdat]

Sure, lin0 -

We had a boatload of data parsed from ASCII log files of chip testers. Each test had its own syntax for responses and some were pass-fail, others were binned. I had a bunch of log parsers (straight Perl scripts, although with the parsing driven by files of regexen and hints for each test), and they stuffed a big MySQL database.

The overall app/tool structure was that I used cron jobs to seek out new log files and parse them into the (MySql 4) DB. All the tables were very simple (read, FLAT), no special joins or exotic relationalism. My (Apache 1.3.33) web pages were built with Embperl (1.6, not 2), which did the data presentation. GD-based modules were used (GD::Graph::bars3d, IIRC) to generate pix on the fly. These were dumped into a temporary directory which was housecleaned of all files older than 3 hours. Another option was to build spreadsheets (Spreadsheet::WriteExcel::Big) or formatted RTF's (RTF::Writer, IIRC) from the data. Likewise, these were disposable.

All the web pages were dynamic Embperl constructs, and they used the directory structure to allow me to use the same pages for many different test sets. Embperl's really good at that, because the environment persists as you go deeper into the directory hierarchy unless you overwrite it locally. Very trick. Embperl also has a lot of automatic HTML table-generation functionality which just blows PHP away. Dunno about the other web-ready Perl templating systems, but I was able to do a lot with a tiny bit of Embperl coding.

We didn't have a lot of traffic, but we did have a pretty hefty storehouse of data. Even so, the queries returned full pages (with graphics and color-coded data tables) in just a few seconds. I could have split out the data into separate data servers, but there was no real need. Truth is, Open Source tools are plenty fast enough for most applications without needing threading, multi-processing, or multiple machines.

[hangon]

How about a book about modeling and simulation using Perl? Even if Perl isn't the best language for this, like the AI book suggested by Ovid it could still be used to explore the basic concepts.

[dokkeldepper]

The Book of JAPH -- Collection of nice and inspiring JAPHs
The Obfuscated Perl Book -- Collection of beautiful obfuscations user friendly explained
Perl Fun Book -- A book of Mind-, Math- and other Hacks with Perl

[astroboy]

I'd be interested in a book on a Perl Framework like Catalyst with DBIx::Class and TT. It's easy to learn syntax, but it's more useful to learn how to build an application properly. Perl web books kind of stopped at how to write CGI script or write funky mod_perl handlers, but think it would be great to have at least one Perl book that dealt with modern MVC design with Perl's mighty answer to Ruby on Rails

[Muggins]

Well I don't know Maypole & Catalyst that well, but I'm pretty sure at least Maypole (and possibly both) _predated_ RoR. (?)

Re Maypole: this and this could be of interest - being the author's own very readable intro. Theres also some in the chapter on Databases in APP2.