What happened to FEAR::API?

[spazm]

While pondering some web-scraping today, I found some circa 2006 references to FEAR::API. I checked CPAN and it is no longer there. The author, Xern, is still active. Does anyone know what happened to it?

Does it still exist, only now with a new name? Has it been abandonded? Removed? other? Replaced by something cooler? (if so, what is it?)

DESCRIPTION:
FEAR::API is a tool that helps reduce your time creating site scraping scripts and help you do it in a much more elegant way. FEAR::API combines many strong and powerful features from various CPAN modules, such as LWP::UserAgent, WWW::Mechanize, Template::Extract, Encode, HTML::Parser, etc. and digests them into a deeper Zen.

2006 perl.com article on FEAR::API by the module author:

http://www.perl.com/pub/a/2006/06/01/fear-api.html

Author's page at cpan:

http://search.cpan.org/~xern/

Backpan:

http://backpan.perl.org/authors/id/X/XE/XERN/

Fresh Ports package:

http://www.freshports.org/www/p5-FEAR-API/

Author's Twitter stream

http://twitter.com/xern

Update:
Xern replied:

@spazm it was pulled off long time ago. bad design. need to come up with a better one.

[perrin]

You're better off using Web::Scraper or just WWW::Mechanize directly with an HTML parser.

[spazm]

Thanks for the pointer to Web::Scraper!

[syphilis]

http://search.cpan.org/~xern/Lingua-EN-GeniaTagger-0.01/ ?

The description of that module says "There's no fear with this elegant site scraper" ... which might be a cryptic hint ...

Cheers,
Rob

[Burak]

You can get a version of it from the glorious BackPAN However, I'd not waste my time on an abandoned module :p

Edit: Khen1950fx, you beat me to it :)

[spazm]

One less mystery in the universe, Xern replied to my tweet:

@spazm it was pulled off long time ago. bad design. need to come up with a better one.

And now a new question: Why was the design bad, what can we learn from it to move forward? I liked the premise of having a reusable component for site scraping, I feel like I'm so often doing one-off design for scraping.

My most recent scrapers have used WWW::Mechanize::TreeBuilder and HTML::TreeBuilder::XPath. Today I helped a friend put together one with Template::Extract, which was cute but seemed like a step backwards since it generates regexp's instead of using the tree nature of HTML.

Maybe I should add HTML::Selector::XPath to help me generate my XPath expressions. Tuning those is often my biggest pain with an XPath solution. This node and thread has some nice XPath tips: Re: HTML and Xpath.

[xern]

Hi,

There's actually a redesign of it and some not-so-practical features, such as operator overloading, are taken out.

Please see here

Though there is no example or documents on it at the moment. The design is better than what I did before, though it's still quite experimental and the code also uses some cleanup. In this framework, scrapers can be inherited and you don't necessarily need to create a scraper from scratch for any new projects.

If you have questions or suggestions, welcome to shoot me an email, which is a better way to contact me.

Thanks :-)
Yung-chung Lin

[Khen1950fx]

Just for fun, I downloaded and installed it. It installed ok, but it missed one of the prereqs, Parallel::ForkManager. It failed 10/44 test programs and 5/157 subtests failed. I ran the programs in the bin---fear, fear-recorder, and fearperl---none of them worked. The problem was "could not  vf_load '/fear' at /path/to/Fear/API.pm line 889".

What I don't understand is why the author would put all that time into the module and then not iron out the bugs? I can see why it was BACKPANed.