Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation

Comment on

( #3333=superdoc: print w/replies, xml ) Need Help??
Think of this as the current state, then the pseudo-code for a new project to move it to a dynamic webpage...


I currently download Yahoo finance historical data to a CSV and cut and paste into excel. I use excel functions to operate on the historical information to predict the next trading day's behavior, and backtest different setups. A worksheet in the workbook creates the html in column A, which I cut and paste into notepad++. Then I use filezilla to put the page to my yahoo served website. eg.

very manual, very timeconsuming, very outdated information almost immediately...

New solution inspired by:


From a webpage form, enter a Yahoo financial symbol and fetch the symbol.

Use perl modules "xxx" to scrape Yahoo historical data for fetched symbol.

Use perl modules "xxx" to perform analysis (similar to excel) functions on the scraped dataset and push additional values to the end of the array/hash.

Format the output and print below the original form used to fetch the symbol (as shown on the inspiration webpage).

The setup works for stocks, options, indexes, etc. alike... Some better than others... I would obviously like to automate this currently labor intensive activity and provide an output of the "top 10" into a dashboard or something... Also, I love excel, but I would like to be able to run this directly from the web, from my phone, tablet, someone else's computer, etc (lots of things that may not have excel). This is not intended to be a commercial product. I'm just planning on making it mobile and dynamic for my own use.

I am looking for guidance from anyone having an opinion on tips, tricks, warnings, better ideas, etc. I've written programs in "C" for testing craps, blackjack, dogs and trading strategies for many years where needed as a hobby. I do a lot of one-liner perl and awk for managing enterprise storage and performing automated data migrations. I will figure this out as I always do, but I'm trying not to re-invent everything from scratch as I often also tend to do. Note: Perl makes more sense than C for this project , mostly because yahoo doesn't like to serve compiled C. I can't seem to even run "hello world" from a yahoo site with gcc. I would also like to avoid setting up LAMP on my own box and getting a static IP from my service provider just to do this.

I've been playing with LWP::Simple and a few others to get some pages, but I think there must be better modules for the type of work described above.

Thanks in advance for any advice.

In reply to Scrape Yahoo Financial Historical- Process Dataset - format and create dynamic page by tbone654

Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":

  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?

    What's my password?
    Create A New User
    and all is quiet...

    How do I use this? | Other CB clients
    Other Users?
    Others studying the Monastery: (4)
    As of 2018-04-24 01:19 GMT
    Find Nodes?
      Voting Booth?