|Problems? Is your data what you think it is?|
Net::Google replacement?by dsheroh (Parson)
|on Sep 11, 2007 at 17:09 UTC||Need Help??|
dsheroh has asked for the
wisdom of the Perl Monks concerning the following question:
I'm working on a project which involves pulling some data from Google search results. Since I know Google doesn't like people scraping their search result pages, it was off to CPAN to see what might be available there. DBD::Google looked particularly promising and it (like most other Googly modules) is based on Net::Google, which in turn uses the Google SOAP API. So, next stop, get a Google SOAP API key.
And then I came to a screeching halt. "As of December 5, 2006, we are no longer issuing new API keys for the SOAP Search API... Depending on your application, the AJAX Search API may be a better choice for you instead. It tends to be better suited for search-based web applications..."
Well, I'm not doing a search-based web application, I'm looking to grab some document titles/URLs and stuff them into an email message. A quick review of the AJAX Search API examples looks an awful lot like it doesn't do anything that's particularly useful to me - I probably could work up a way to use it, but I'd just be scraping those results instead of the results from the normal search form.
Is there a new Google API which replaces the functionality of the apparently-discontinued SOAP API or has Google basically decided to say "screw you" to anyone who wants to use search results in ways other than displaying them on a web page? If there is a new API, do any Perl modules exist to provide an interface to it (yet)?
In the end, I can scrape the standard page if that's what's needed, but I would, of course, prefer to avoid incurring the wrath of Google...