Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling

Net::Google replacement?

by dsheroh (Monsignor)
on Sep 11, 2007 at 17:09 UTC ( #638364=perlquestion: print w/replies, xml ) Need Help??

dsheroh has asked for the wisdom of the Perl Monks concerning the following question:

I'm working on a project which involves pulling some data from Google search results. Since I know Google doesn't like people scraping their search result pages, it was off to CPAN to see what might be available there. DBD::Google looked particularly promising and it (like most other Googly modules) is based on Net::Google, which in turn uses the Google SOAP API. So, next stop, get a Google SOAP API key.

And then I came to a screeching halt. "As of December 5, 2006, we are no longer issuing new API keys for the SOAP Search API... Depending on your application, the AJAX Search API may be a better choice for you instead. It tends to be better suited for search-based web applications..."

Well, I'm not doing a search-based web application, I'm looking to grab some document titles/URLs and stuff them into an email message. A quick review of the AJAX Search API examples looks an awful lot like it doesn't do anything that's particularly useful to me - I probably could work up a way to use it, but I'd just be scraping those results instead of the results from the normal search form.

Is there a new Google API which replaces the functionality of the apparently-discontinued SOAP API or has Google basically decided to say "screw you" to anyone who wants to use search results in ways other than displaying them on a web page? If there is a new API, do any Perl modules exist to provide an interface to it (yet)?

In the end, I can scrape the standard page if that's what's needed, but I would, of course, prefer to avoid incurring the wrath of Google...

Replies are listed 'Best First'.
Re: Net::Google replacement?
by pemungkah (Priest) on Sep 12, 2007 at 02:00 UTC
    Well, there's always Yahoo:

    Disclaimer: I work there. Then again, that means you know who to complain to if you don't like the Perl interface, or if you have patches or suggestions...

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://638364]
Approved by naikonta
Front-paged by grinder
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (4)
As of 2020-12-02 07:01 GMT
Find Nodes?
    Voting Booth?
    How often do you use taint mode?

    Results (35 votes). Check out past polls.