tbone654 has asked for the wisdom of the Perl Monks concerning the following question:
CURRENT STATE
I currently download Yahoo finance historical data to a CSV and cut and paste into excel. I use excel functions to operate on the historical information to predict the next trading day's behavior, and backtest different setups. A worksheet in the workbook creates the html in column A, which I cut and paste into notepad++. Then I use filezilla to put the page to my yahoo served website. eg. http://www.aztecura.com/php/data/test.php
very manual, very timeconsuming, very outdated information almost immediately...
New solution inspired by: http://www.stockta.com/cgi-bin/analysis.pl?symb=AAPL&cobrand=&mode=stock
PSEUDO-CODE
From a webpage form, enter a Yahoo financial symbol and fetch the symbol.
Use perl modules "xxx" to scrape Yahoo historical data for fetched symbol.
Use perl modules "xxx" to perform analysis (similar to excel) functions on the scraped dataset and push additional values to the end of the array/hash.
Format the output and print below the original form used to fetch the symbol (as shown on the inspiration webpage).
The setup works for stocks, options, indexes, etc. alike... Some better than others... I would obviously like to automate this currently labor intensive activity and provide an output of the "top 10" into a dashboard or something... Also, I love excel, but I would like to be able to run this directly from the web, from my phone, tablet, someone else's computer, etc (lots of things that may not have excel). This is not intended to be a commercial product. I'm just planning on making it mobile and dynamic for my own use.
I am looking for guidance from anyone having an opinion on tips, tricks, warnings, better ideas, etc. I've written programs in "C" for testing craps, blackjack, dogs and trading strategies for many years where needed as a hobby. I do a lot of one-liner perl and awk for managing enterprise storage and performing automated data migrations. I will figure this out as I always do, but I'm trying not to re-invent everything from scratch as I often also tend to do. Note: Perl makes more sense than C for this project , mostly because yahoo doesn't like to serve compiled C. I can't seem to even run "hello world" from a yahoo site with gcc. I would also like to avoid setting up LAMP on my own box and getting a static IP from my service provider just to do this.
I've been playing with LWP::Simple and a few others to get some pages, but I think there must be better modules for the type of work described above.
Thanks in advance for any advice.