Beefy Boxes and Bandwidth Generously Provided by pair Networks
XP is just a number
 
PerlMonks  

Re^2: Rotating IP Addresses for Scraping

by jandrewc (Initiate)
on May 18, 2011 at 16:49 UTC ( #905511=note: print w/ replies, xml ) Need Help??


in reply to Re: Rotating IP Addresses for Scraping
in thread Rotating IP Addresses for Scraping

Oh, I'm sorry. I didn't know this wasn't kosher. It's not a free trial issue although its similar; I need the longitude and latitude of each zip code and this is the only place I've been able to find this data and they only allow you to submit thirty zip codes a day. This would take nearly three years.

The project has to do with the race-perception effects of inducing slave ownership in the 1850s. If someone is "induced" to own slaves by owning at the time of the cotton gin's penetration optimal cotton land, does this create in the family a self-serving bias in the form of racism. The longitude and latitude data are useful for giving distance to the nearest port which is also a factor for how optimal cotton production is in any particular zip code as transportation costs are theoretically proportional to distance.

Will you consider helping me understand what to do?


Comment on Re^2: Rotating IP Addresses for Scraping
Re^3: Rotating IP Addresses for Scraping
by Fletch (Chancellor) on May 18, 2011 at 17:17 UTC
    I need the longitude and latitude of each zip code and this is the only place I've been able to find this data ...

    ORLY? Free Zip Code Latitude and Longitude Database

    The cake is a lie.
    The cake is a lie.
    The cake is a lie.

      Thanks for the find, Fletch. This will do if I can't figure this out. These datasets tended to be less complete and accurate.
Re^3: Rotating IP Addresses for Scraping
by ww (Bishop) on May 18, 2011 at 17:46 UTC
    Your source offers US (and Canadian) postal code data as of May 2011, including lat/lon for USD79.95.

    The project description sounds like a graduate level (PhD dissertation?) project. If this is indeed that, or, in fact, other than a self-funded project, one might think that a reasonable budget would include that kind of allowance for data acquisition.

    If it doesn't, /methinks one would still * NOT * be entitled to steal data.

Re^3: Rotating IP Addresses for Scraping
by GrandFather (Cardinal) on May 19, 2011 at 01:06 UTC

    This sounds rather an XY problem to me. What you actually seem to want are travel distances between various pairs of locations unrelated to zip codes (I doubt in the 1850s zip codes were much in vogue) and actually unrelated (directly) to lat/long values. Calculating a distance between two points doesn't actually provide a very good indicator of the travel distance or travel difficulty between the two points, but is at best an approximate proxy for the data you really want. Using zip codes to specify locations is just another layer of inexact proxies.

    Maybe there are better ways of determining the information you actually want than "stealing" data to provide and approximation of and approximation to the data you really want?

    True laziness is hard work

      Agreed.

      The United States Postal Service began using zip codes in 1963.

      In densely populated regions, a zip code's area may be less than a square mile. In sparsely populated regions, a zip code's area may be several hundred square miles (check out West Texas, for example).

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://905511]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (3)
As of 2014-09-19 23:39 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (151 votes), past polls