Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris

Net::Google replacement?

by dsheroh (Parson)
on Sep 11, 2007 at 17:09 UTC ( #638364=perlquestion: print w/ replies, xml ) Need Help??
dsheroh has asked for the wisdom of the Perl Monks concerning the following question:

I'm working on a project which involves pulling some data from Google search results. Since I know Google doesn't like people scraping their search result pages, it was off to CPAN to see what might be available there. DBD::Google looked particularly promising and it (like most other Googly modules) is based on Net::Google, which in turn uses the Google SOAP API. So, next stop, get a Google SOAP API key.

And then I came to a screeching halt. "As of December 5, 2006, we are no longer issuing new API keys for the SOAP Search API... Depending on your application, the AJAX Search API may be a better choice for you instead. It tends to be better suited for search-based web applications..."

Well, I'm not doing a search-based web application, I'm looking to grab some document titles/URLs and stuff them into an email message. A quick review of the AJAX Search API examples looks an awful lot like it doesn't do anything that's particularly useful to me - I probably could work up a way to use it, but I'd just be scraping those results instead of the results from the normal search form.

Is there a new Google API which replaces the functionality of the apparently-discontinued SOAP API or has Google basically decided to say "screw you" to anyone who wants to use search results in ways other than displaying them on a web page? If there is a new API, do any Perl modules exist to provide an interface to it (yet)?

In the end, I can scrape the standard page if that's what's needed, but I would, of course, prefer to avoid incurring the wrath of Google...

Comment on Net::Google replacement?
Re: Net::Google replacement?
by pemungkah (Priest) on Sep 12, 2007 at 02:00 UTC
    Well, there's always Yahoo:

    Disclaimer: I work there. Then again, that means you know who to complain to if you don't like the Perl interface, or if you have patches or suggestions...

Reaped: Microsoft
by NodeReaper (Curate) on Oct 11, 2007 at 13:57 UTC

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://638364]
Approved by naikonta
Front-paged by grinder
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others studying the Monastery: (7)
As of 2014-07-10 10:59 GMT
Find Nodes?
    Voting Booth?

    When choosing user names for websites, I prefer to use:

    Results (207 votes), past polls