http://www.perlmonks.org?node_id=994757


in reply to Scrape Yahoo Financial Historical- Process Dataset - format and create dynamic page

Ha, Big Blue Marble has a project like this.

The web front end is a really simple database-backed site that does none of this interesting processing. When someone requests information on a stock that's not already in the database, the site inserts a stub entry for that stock.

Every few minutes, a cron job on the server looks for stub entries and runs the gamut of historical and daily analysis on those stubs. The live site updates what it displays for that stock as new information comes in. So far no one has objected to waiting up to five minutes for intrinsic value calculations. (We discussed running the cron job more frequently, but I haven't made that happen yet.)

I ported a couple of formulas from Excel, but if you already understand things like a present value calculation, or could do very basic calculus on the back of an envelope (did that too!), you can reproduce those formulas easily in Perl.

With that all said...

I use excel functions to operate on the historical information to predict the next trading day's behavior...

... my best advice is to pray a lot. (I'm a value investor.)

  • Comment on Re: Scrape Yahoo Financial Historical- Process Dataset - format and create dynamic page

Replies are listed 'Best First'.
Re^2: Scrape Yahoo Financial Historical- Process Dataset - format and create dynamic page
by tbone654 (Beadle) on Sep 21, 2012 at 18:34 UTC

    I used Finance::QuoteHist::Yahoo to pull the historical data I need to operate on the data. So I will set it up to be dynamically called by a form, and print the output right back to the same webpage.

    The problem right now is that yahoo doesn't support the ::Yahoo perl module I need to pull the data directly from the form. I can run it on my laptop and push the output to the yahoo server, but that's not sexy. I guess I have three options here:

  • Move it all to a web server somewhere where perl supports the module.
  • Build my own LAMP server and pay for a static IP, then I control the perl modules (I think?)
  • Use the standard modules which Yahoo supports, plus a few more, and write the excruciatingly painful scraper functions from what's available. (yuck)
  • Run some kind of simple http web server from my laptop that lets me map the perl script from my laptop to a public URL. eg. www.somesmallwebserver.com/3666698/tmp/perlscript.pl Which maps to my /tmp filesystem on my laptop.

    • I am sort of going down the path of the last option for now, just to get it going. I found one that allows downloading files only (brick something) so I was able to download a file from my laptop just fine. So I imagine there are some that just act as a webserver through a similar public ip address schema?

    Thank you for your reply. Good luck with that value investing thing, I'm an action junkie, so I need to move pretty fast and must have fairly live data.

[reply]

      The problem right now is that yahoo doesn't support the ::Yahoo perl module

      You can, like, install the module

[reply]
        How do you get Yahoo web hosting to install the module?
[reply]

In Section Seekers of Perl Wisdom