This is a production script that I've been working on, and it's at a point where it can serve as an sscce for a pm thread. I've documented a lot of it with comments, but let me state that this attempts to get the weather details for a specific location as well as the relative position of the sun. I'll take any and all suggestions on how to smooth down the rough parts or ecstatic to replace it with something better. For example, I'd love to see what marto can do with mojo....
Let me roll out the log first. I unlink it and recreate it with every time the script runs. These data are only typical:
2020/05/11 14:23:44 INFO ./6.weather.pl
2020/05/11 14:23:45 INFO $VAR1 = [
{
'base' => {
'unitCode' => 'unit:m',
'value' => 910
},
'amount' => 'FEW'
},
{
'base' => {
'value' => 3050,
'unitCode' => 'unit:m'
},
'amount' => 'BKN'
},
{
'base' => {
'unitCode' => 'unit:m',
'value' => 4880
},
'amount' => 'BKN'
},
{
'amount' => 'OVC',
'base' => {
'value' => 7620,
'unitCode' => 'unit:m'
}
}
];
2020/05/11 14:23:45 INFO $VAR1 = undef;
2020/05/11 14:23:45 INFO $VAR1 = '2020-05-11T20:53:00+00:00';
2020/05/11 14:23:45 INFO td is 2020-05-11 20:53:00
2020/05/11 14:23:45 INFO julian day is 2020-05-11 20:53:00
2020/05/11 14:23:46 INFO row sun is Sun 3h 16m 43s +18° 9.7' 1.010 61.
+305 24.656 Up
2020/05/11 14:23:46 INFO name: Sun
2020/05/11 14:23:46 INFO altitude: 61.305
2020/05/11 14:23:46 INFO azimuth: 24.656
2020/05/11 14:23:46 INFO visible?: Up
First of all, wow, right? We've got 3 different identified cloud layers. This makes me want to have this capability on my phone. Let's take one more look now that I've taken the time to write this up, so this is hot off the press:
2020/05/11 17:40:07 INFO ./6.weather.pl
2020/05/11 17:40:08 INFO $VAR1 = [
{
'base' => {
'unitCode' => 'unit:m',
'value' => 1220
},
'amount' => 'FEW'
},
{
'amount' => 'OVC',
'base' => {
'value' => 1680,
'unitCode' => 'unit:m'
}
}
];
2020/05/11 17:40:08 INFO $VAR1 = undef;
2020/05/11 17:40:08 INFO $VAR1 = '2020-05-11T23:53:00+00:00';
2020/05/11 17:40:08 INFO td is 2020-05-11 23:53:00
2020/05/11 17:40:08 INFO julian day is 2020-05-11 23:53:00
2020/05/11 17:40:09 INFO row sun is Sun 3h 17m 12s +18° 11.6' 1.010 35
+.801 79.815 Up
2020/05/11 17:40:09 INFO name: Sun
2020/05/11 17:40:09 INFO altitude: 35.801
2020/05/11 17:40:09 INFO azimuth: 79.815
2020/05/11 17:40:09 INFO visible?: Up
It feels like the weather has changed since then. It feels like sunset to me, though there may be another ten degrees for the mathematical-idealized data I'm receiving. Here's the script:
#!/usr/bin/env perl
use strict;
use warnings;
use DateTime::Format::Strptime;
use REST::Client;
use Data::Roundtrip qw/:all/;
use 5.016;
use Log::Log4perl;
use Data::Dump;
# get rid of old log
my $file = '/home/hogan/Documents/hogan/logs/4.log4perl.txt';
unlink $file or warn "Could not unlink $file: $!";
my $log_conf4 = "/home/hogan/Documents/hogan/logs/conf_files/4.conf";
Log::Log4perl::init($log_conf4); #info
my $logger = Log::Log4perl->get_logger();
$logger->info($0);
# this is our fetcher, similar to LWP::UserAgent
# but better suited for this kind of web service: REST
my $rest = REST::Client->new() or die "failed to construct client";
# set the host
$rest->setHost('https://api.weather.gov');
# note to self: KTTD is troutdale
my $query = "stations/KPDX/observations/latest";
# make the request and check the response code, 200 is good
my $response = $rest->GET($query) or die "failed to GET($query)";
if ( $rest->responseCode() != 200 ) {
die "failed to GET("
. $rest->getHost()
. "/$query) with "
. $rest->responseCode();
}
# we get back JSON
my $jsonstr = $response->responseContent();
# convert JSON string to a perl variable
my $pv = json2perl($jsonstr);
if ( !defined $pv ) {
die "something wrong with this alleged json : '$jsonstr'";
}
# go to the interesting part
my $var1 = $pv->{'properties'}->{'cloudLayers'};
$logger->info(perl2dump($var1));
## failure
#my %cl = %$var1;
#my $cl1 = $cl{'amount'};
#say "cl 1 is $cl1";
# try heat index
# try to pluck out the reference:
my $var2 = $pv->{'properties'}->{'heatIndex'};
my %cl2 = %$var2;
my $str2 = $cl2{'value'};
say "str 2 is $str2";
$logger->info(perl2dump($str2)); # this ain't right
# let's get the timestamp
my $var3 = $pv->{'properties'}->{'timestamp'};
say "var3 is $var3";
print perl2dump($var3);
$logger->info(perl2dump($var3));
## from bliako on same host (hope these are similar)
# we have some dates in the data in ISO8601 format
# this is a parser to convert that date to a DateTime
# object which we can query about things (like seconds-unix-epoch)
my $dateparser = DateTime::Format::Strptime->new(
# parses 2020-04-26T06:00:00-05:00,
# %F then literal T then %T then timezone offset
pattern => '%FT%T%z'
) or die "failed to DateTime::Format::Strptime->new()";
my $time_dateobj = $dateparser->parse_datetime($var3)
or die "parsing date failed";
my $td = $time_dateobj->strftime('%Y-%m-%d %H:%M:%S');
say "td is $td";
$logger->info("td is $td");
## check to see what julian day it is
use DateTime;
my $julian=$time_dateobj->jd(); # this ain't right
$logger->info("julian day is $td");
### use retrieved time to figure out where sun is concurrently
# use a perl automation tool that works for the $site* in question,
# *which is written kinda funny
use WWW::Mechanize;
use HTML::TableExtract qw(tree);
use open ':std', OUT => ':utf8';
my $site = 'http://www.fourmilab.ch/yoursky/cities.html';
my $mech = WWW::Mechanize->new;
$mech->get($site);
$mech->follow_link( text => 'Portland OR' );
$mech->set_fields(qw 'date 1');
$mech->set_fields( utc => $td );
$mech->click_button( value => "Update" );
my $te = 'HTML::TableExtract'->new;
$te->parse( $mech->content );
my $table = ( $te->tables )[3];
my $table_tree = $table->tree;
my $row = 2; # the sun
my @sun = $table_tree->row($row)->as_text;
$logger->info("row sun is @sun");
my $name = $table_tree->cell($row,0)->as_text;
$logger->info("name: $name");
my $altitude = $table_tree->cell($row,4)->as_text;
$logger->info("altitude: $altitude");
my $azimuth = $table_tree->cell($row,5)->as_text;
$logger->info("azimuth: $azimuth");
my $visible = $table_tree->cell($row,6)->as_text;
$logger->info("visible?: $visible");
$te->delete;
__END__
Meanwhile, STDOUT is clogged like this in unscalable screenfuls:
Use of uninitialized value in hash element at /usr/share/perl5/HTML/El
+ement.pm line 1674.
Use of uninitialized value in hash element at /usr/share/perl5/HTML/El
+ement.pm line 1674.
Use of uninitialized value in hash element at /usr/share/perl5/HTML/El
+ement.pm line 1674.
Use of uninitialized value in hash element at /usr/share/perl5/HTML/El
+ement.pm line 1674.
Use of uninitialized value $ptag in string eq at /usr/share/perl5/HTML
+/TreeBuilder.pm line 1071.
Use of uninitialized value in hash element at /usr/share/perl5/HTML/El
+ement.pm line 1674.
Use of uninitialized value $ptag in string eq at /usr/share/perl5/HTML
+/TreeBuilder.pm line 1071.
$
Q1) How do I clear my terminal from all these warnings?
- subquestion 1a) Can I use Log4Perl to intern these warnings in a file, or would it just crush my pretty log?
- subquestion 1b) Can I use the command line to direct diagnostics to a file?
## failure
#my %cl = %$var1;
#my $cl1 = $cl{'amount'};
#say "cl 1 is $cl1";
Q2) How do I tease values out of this json structure more reliably than hunting and pecking?
Where am I going with this? Essentially in the direction that bliako suggested here, Re: using a stochastic matrix to simulate weather conditions:
You still need to figure out how to convert the received data to machine-readable format, re: "sunny", and, most importantly, figure out how to store received data for easy access over time (SQLite database?, csv files?, json files? perl variable files which you can later eval? - in order of my preference) - what search keys, what identifies a unique record? etc.
I can create whatever container I want to in the my_data folder that is defined by Path::Tiny in the script. Q3) Is it possible to create a human-readable json file with the cloud data, relevant values like temperature and windspeed, as well as the values retrieved by the WM part of the script?
Ultimately, I'd like to determine what conditions attain that will kill the exposed virus on fomites, with further considerations given for how they affect the mini-oven which is dashboard of my auto: fomites in a car.
I *still* have not seen data for how well this virus fares on surfaces as a function of temperature, moisture, exposure to wind, sunlight, although I have seen them in foreign language sources, particularly russian. What veracity these glimpses have is what I am exploring here.
Other than that, I've tried to come up with the julian date of the timestamp I get from the api, and that is not attaining yet...
w.r.t. the above:
Q2) How do I tease values out of this json structure more reliably than hunting and pecking?
Q3) Is it possible to create a human-readable json file with the cloud data, relevant values like temperature and windspeed, as well as the values retrieved by the WM part of the script?
Q5) I'm gonna put in q5 as a placeholder, because I have feel like I'm missing something and will want to come back to with an update.
We always practice social distance in the monastery...I feel closer to the people here than the patriotic, unscientific, insouciant orcs I have to avoid in real life.
