Re^2: Rotating IP Addresses for Scraping

in reply to Re: Rotating IP Addresses for Scraping
in thread Rotating IP Addresses for Scraping

Oh, I'm sorry. I didn't know this wasn't kosher. It's not a free trial issue although its similar; I need the longitude and latitude of each zip code and this is the only place I've been able to find this data and they only allow you to submit thirty zip codes a day. This would take nearly three years.

The project has to do with the race-perception effects of inducing slave ownership in the 1850s. If someone is "induced" to own slaves by owning at the time of the cotton gin's penetration optimal cotton land, does this create in the family a self-serving bias in the form of racism. The longitude and latitude data are useful for giving distance to the nearest port which is also a factor for how optimal cotton production is in any particular zip code as transportation costs are theoretically proportional to distance.

Will you consider helping me understand what to do?

Comment on Re^2: Rotating IP Addresses for Scraping

Replies are listed 'Best First'.
Re^3: Rotating IP Addresses for Scraping by Fletch (Bishop) on May 18, 2011 at 17:17 UTC
I need the longitude and latitude of each zip code and this is the only place I've been able to find this data ... ORLY? Free Zip Code Latitude and Longitude Database The cake is a lie. The cake is a lie. The cake is a lie.	[reply]
Re^4: Rotating IP Addresses for Scraping by jandrewc (Initiate) on May 18, 2011 at 17:29 UTC
Thanks for the find, Fletch. This will do if I can't figure this out. These datasets tended to be less complete and accurate.	[reply]
Re^3: Rotating IP Addresses for Scraping by ww (Archbishop) on May 18, 2011 at 17:46 UTC
Your source offers US (and Canadian) postal code data as of May 2011, including lat/lon for USD79.95. The project description sounds like a graduate level (PhD dissertation?) project. If this is indeed that, or, in fact, other than a self-funded project, one might think that a reasonable budget would include that kind of allowance for data acquisition. If it doesn't, /methinks one would still * NOT * be entitled to steal data.	[reply]
Re^3: Rotating IP Addresses for Scraping by GrandFather (Saint) on May 19, 2011 at 01:06 UTC
This sounds rather an XY problem to me. What you actually seem to want are travel distances between various pairs of locations unrelated to zip codes (I doubt in the 1850s zip codes were much in vogue) and actually unrelated (directly) to lat/long values. Calculating a distance between two points doesn't actually provide a very good indicator of the travel distance or travel difficulty between the two points, but is at best an approximate proxy for the data you really want. Using zip codes to specify locations is just another layer of inexact proxies. Maybe there are better ways of determining the information you actually want than "stealing" data to provide and approximation of and approximation to the data you really want? True laziness is hard work	[reply]
Re^4: Rotating IP Addresses for Scraping by hominid (Priest) on May 19, 2011 at 20:38 UTC
Agreed. The United States Postal Service began using zip codes in 1963. In densely populated regions, a zip code's area may be less than a square mile. In sparsely populated regions, a zip code's area may be several hundred square miles (check out West Texas, for example).	[reply]

In Section Seekers of Perl Wisdom