johnfl68 has asked for the wisdom of the Perl Monks concerning the following question:
Hello, and as always thank you for help and support.
I have been using LWP::UserAgent with the NWS API using mirror to save API data for several locations once an hour.
Lately the API has been having issues (many other people having issues as well, not just me), but I think part of the issues I am having has to do with me using mirror, and occasionally I will get an older data file from NWS, but it will put a future date for the file date for some reason (July 27, 2020 for example), and then it does not update anymore because the file on the server is newer (but old data).
I am supposed to send User Agent with API requests to NWS. I could go back to just doing GetStore, but don't want to have to change to a UserAgent solution again at some point in the future if I get blocked for not sending it.
I have looked around and there are dozens of different ways to get and store a file from http, but some may be better suited for this case then others. So before I recode this to do some other get store type option instead of doing a mirror type option (I don't really need to check if I already have the same data at this point), are there any other ways/modules that you would recommend? I am not getting webpages, just JSON and XML data sets.</o>
Stick with LWP::UserAgent and use 'get' and then just output the response to a file? HTTP::Tiny? Mechanize? Something completely different?
Thank you!