Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

Stocks, Perl & Gambling Game

by chunlou (Curate)
on Jul 22, 2003 at 04:09 UTC ( #276598=CUFP: print w/ replies, xml ) Need Help??

Regular wife: "Honey, did you buy Viagra?"
Smart wife: "Honey, did you buy Pfizer?"

Script to fetch stock quotes from Yahoo, based on stock symbol supplied, and generate four files--e.g. if 'ibm' symbol used, files written to drive will be ibm.html, ibmlog.jpg, ibmdiff.jpg and ibm.csv. Sample html can be viewed here.

It's customary for data analysts to analyze stock prices (yt) on log scale, log(yt). Notice the first differences of the log prices, log(yt) - log(yt-1), are approximately equal to the (small) percentage changes of the stock per time period (which can be shown by Taylor expansion).

(As you see in the second chart on the sample html page, the daily changes of the stock roughly center around zero. That's why stock market sometimes is considered nothing short of sheer gambling.)


___________________
Caution. Beware when you fetch data automatically via the Internet. "Large" volumn might result in your IP being blocked by the website you make request to. Some sites did that for security reason. They might do so if they believe you're a "robot" or "worm."

use strict; use warnings; # - - - - - - - - - - - - - - - - - - - - - - - - - - # e.g. if 'ibm' queried, script writes to files: # ibm.html ibmdiff.jpg ibmlog.jpg ibm.csv # - - - - - - - - - - - - - - - - - - - - - - - - - - # stock symbol lookup: http://finance.yahoo.com/l # e.g. aapl ko ibm msft rhat sunw my $stocksymbol = "rhat"; my $startdate = "4-15-2002"; # since it's for historical date, my $enddate = "6-30-2003"; # latest data may not be available my $interval = "d"; # d: daily, w: weekly, m: monthly my $agent = "Mozilla/4.0"; # your id sent to Yahoo Web server, enter + something my $q = quotes::get($stocksymbol, $startdate, $enddate, $interval, $ag +ent); my ($ma, $diff) = (20, 1); # lags for MA & differencing open OUT, ">$stocksymbol.html"; print OUT chart::html($stocksymbol, $q, $ma, $diff); # expecting hea +ders: Date,Open,High,Low,Close,Volume close OUT; print "done: $stocksymbol.html generated.\n"; {package quotes; use LWP::UserAgent; sub get { my ($symbol, $startdate, $enddate, $agent) = @_; print "fetching data...\n"; my $dat = _fetch($symbol, $startdate, $enddate, $agent); # c +sv file, 1st row = header my @q = split /\n/, $dat; my @header = split /,/, shift @q; my %quotes = map { $_ => [] } @header; for my $q (@q) { my @val = split ',', $q; unshift @{$quotes{$header[$_]}}, $val[$_] for 0..$#val; +# unshift instead of push if data listed latest 1st & oldest last } open OUT, ">$symbol.csv"; print OUT $dat; close OUT; print "data written to $symbol.csv.\n"; return \%quotes; } sub _fetch { my ($symbol, $startdate, $enddate, $interval, $agent) = @_; my $url = "http://chart.yahoo.com/table.csv?"; my $freq = "g=$interval"; # d: daily, w: weekly, m: monthly my $stock = "s=$symbol"; my @start = split '-', $startdate; my @end = split '-', $enddate; $startdate = "a=" . ($start[0]-1) . "&b=$start[1]&c=$start[2]" +; $enddate = "d=" . ($end[0]-1) . "&e=$end[1]&f=$end[2]"; $url .= "$startdate&$enddate&$stock&y=0&$freq&ignore=.csv"; my $ua = new LWP::UserAgent(agent=>$agent); my $request = new HTTP::Request('GET',$url); my $response = $ua->request($request); if ($response->is_success) { return $response->content; } else { warn "Cannot fetch $url (status ", $response->code, " ", $ +response->message, ")\n"; return 0; } } } {package chart; use GD::Graph::lines; # my @headers = qw/ Date Open High Low Close Volume /; hardcoded i +n _tbl() # $q->{Close} assumed exists in plotlog() & plotdiff() sub html { my ($stock, $q, $ma, $diff) = @_; print "generating html...\n"; my $str = ""; $str .= "<html><head><title>$stock</title></head><body bgcolor +=\"#00000\" text=\"ffffff\"><center>\n"; $str .= "<p><img src=\"" . plotlog($stock, $q, $ma) . "\"></p> +\n"; $str .= "<p><img src=\"" . plotdiff($stock, $q, $ma, $diff) . +"\"></p>\n"; $str .= _tbl($stock, $q); $str .= "</center></body></html>\n"; return $str; } sub plotlog { my ($stock, $q, $diff) = @_; my $img = $stock . "log.jpg"; print "generating $img...\n"; my ($s, $lines) = ([],[]); my $y_format = sub { sprintf " \$%.2f", exp $_[0] }; $s = ts::logs($q->{Close}); $lines->[0] = { name => 'Log of Closing Price', color => 'm +arine', data => $s }; $lines->[1] = { name => "MA($diff) (Moving Avg)", color => +'cyan', data => ts::ma($lines->[0]->{data}, $diff) }; plotlines($img, $stock, $q->{Date}, $lines, $y_format); return $img; } sub plotdiff { my ($stock, $q, $lag, $diff) = @_; my $img = $stock . "diff.jpg"; print "generating $img...\n"; my ($s, $lines) = ([],[]); my $y_format = sub { sprintf " %.2f", $_[0] }; $s = ts::logs($q->{Close}); $lines->[0] = { name => "Diff($diff)", color => 'marine', d +ata => ts::diff($s, $diff) }; $lines->[1] = { name => "MA($lag) (Moving Avg)", color => ' +cyan', data => ts::ma($lines->[0]->{data}, $lag) }; $s = ts::stdev($lines->[0]->{data}, $lag); $s = ts::nstdev_ma($s, $lines->[1]->{data}, 2); $lines->[2] = { name => 'MA + 2 Std Dev', color => 'lred', +data => $s->[0] }; $lines->[3] = { name => 'MA - 2 Std Dev', color => 'lred', +data => $s->[1] }; plotlines($img, $stock, $q->{Date}, $lines, $y_format); return $img; } sub plotlines { my ($file, $stock, $x, $lines, $y_format) = @_; my @legend; my ($data, $colors) = ([], []); $data->[0] = $x; # x-axis labels for (0..$#{$lines}) { $data->[(1+$_)] = $lines->[$_]->{data}; $colors->[$_] = $lines->[$_]->{color}; $legend[$_] = $lines->[$_]->{name}; } my $graph = GD::Graph::lines->new(740,420); $graph->set (dclrs => $colors) or die $graph->error; $graph->set_legend(@legend) or die $graph->error; $graph->set (legend_placement => 'BC') or die $graph->error; $graph->set(y_number_format => $y_format) if $y_format; $graph->set ( title => "stock: $stock", boxclr => 'black', bgclr => 'dgray', axislabelclr => 'white', legendclr => 'white', textclr => 'white', r_margin => 20, tick_length => -4, y_long_ticks => 1, axis_space => 10, x_labels_vertical => 1, x_label_skip => int(0.2*scalar(@{$data->[0]})) ) or die $graph->error; my $gd = $graph->plot($data) or die $graph->error; open (IMG, ">$file") or die $!; binmode IMG; print IMG $gd->jpeg(90); return 1; } sub _tbl { my ($stock, $q) = @_; my $str = ""; my @headers = qw/ Date Open High Low Close Volume /; my $tr_start = "<tr align=\"center\">\n"; $str .= "<table border=\"1\" cellpadding=\"3\" cellspacing=\"0 +\">\n"; $str .= $tr_start . "<td colspan=\"" . scalar @headers . "\">" +; $str .= "<b>Stock: $stock</b></td></tr>\n"; $str .= $tr_start; $str .= "<td><b>" . $headers[$_] . "</b></td>\n" for 0..$#head +ers; $str .= "</tr>\n"; for my $i (reverse 0..$#{$q->{Date}}) { $str .= $tr_start; $str .= "<td>" . $q->{$headers[$_]}->[$i] . "</td>\n" for +0..$#headers; $str .= "</tr>\n"; } $str .= "</table>\n"; return $str; } } {package ts; sub logs { my $s = shift; return [ map {log} @{$s}[0..$#{$s}] ]; } sub diff { my ($series, $lag) = @_; my @diff = map {undef} 1..$lag; push @diff, $series->[$_] - $series->[$_-$lag] for ( $lag..$#{ +$series} ); return \@diff; } sub ma { my ($series, $lag) = @_; my @ma = map {undef} 1..$lag; for(@{$series}){unless($_){push @ma,undef}else{last}} my $sum = 0; for my $i ($#ma..$#{$series}) { $sum += $series->[$i-$_] for (0..($lag-1)); push @ma, $sum/($lag); $sum = 0; } return \@ma; } sub stdev { my ($series, $lag) = @_; my @stdev = map {undef} 1..$lag; for(@{$series}){unless($_){push @stdev,undef}else{last}} my ($sum, $sum2) = (0, 0); for my $i ($#stdev..$#{$series}) { for (0..($lag-1)) { $sum2 += ($series->[$i-$_])**2; $sum += $series->[$i-$_] ; } push @stdev, ($sum2/$lag - ($sum/$lag)**2)**0.5; ($sum, $sum2) = (0, 0); } return \@stdev; } sub nstdev_ma{ my ($sd, $ma, $n) = @_; my $ans=[[],[]]; for (0..$#{$sd}) { my $yn = defined $sd->[$_] && defined $ma->[$_]; $ans->[0][$_] = $yn ? $ma->[$_] + $n*($sd->[$_]) : undef; $ans->[1][$_] = $yn ? $ma->[$_] - $n*($sd->[$_]) : undef; + } return $ans; } }

Comment on Stocks, Perl & Gambling Game
Download Code
Re: Stocks, Perl & Gambling Game
by wufnik (Friar) on Jul 23, 2003 at 16:29 UTC
    another valuable posting imho. great example of gd use plus LWP. can't figure out if you think gambling a good idea or not tho;-) would be nice to correlate return movement with offical research calls to rate how good/bad analysts were...

    wufnik

    in the world of the mules there are no rules
Re: Stocks, Perl & Gambling Game
by djbiv (Scribe) on Jul 24, 2003 at 20:15 UTC
    Great Concept, I however encountered what one might consider a `small` problem. at runtime, I recieve a perl.exe application error??? I'm running "This is perl, v5.8.0 built for MSWin32-x86-multi-thread" Any help would be great... I would love to get this working.

      What error message did you receive?

        it's a dialog popup perl.exe - Application Error
        The instruction at "0x01d6a113" referenced memory at "0x00000006". The memory could not be "read". ???

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: CUFP [id://276598]
Approved by valdez
Front-paged by mojotoad
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others pondering the Monastery: (13)
As of 2014-07-31 20:29 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite superfluous repetitious redundant duplicative phrase is:









    Results (253 votes), past polls