Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"

LWP get Cookies for API

by abs52 (Initiate)
on Jun 11, 2019 at 11:54 UTC ( #11101228=perlquestion: print w/replies, xml ) Need Help??

abs52 has asked for the wisdom of the Perl Monks concerning the following question:

Hi All,
I am trying to use LWP ( i dont mind using something else ) to do some posts and get from a website API

But i am failing at the first step.

According to their docs you need to login with your broswer, get the cookie AUTH_TOKEN and get the and then use that output for the curl command authorization. Is there way to do this in perl ?
The curl would look like

curl -X GET "<URL> -H "accept: application/json" -H "authorization:<to +ken>"
and my LWP looks like
my $cookies = HTTP::Cookies->new(); my $ua = LWP::UserAgent->new( cookie_jar => {} ); $req -> authorization_basic ( 'cloud-bespoke' ); my $rc = REST::Client->new( { useragent => $ua } ); my $headers = {Content-type => 'application/json'}; my $client = REST::Client->new( { useragent => $ua }); my $res = $client->POST('URL', '{"username": "username, "password":"password"}', {"Content-type" => ' +application/json'}); chkerr($client->responseCode()); print $client->responseContent(); print "\n" . $cookies->as_string;
Thanks a

Replies are listed 'Best First'.
Re: LWP get Cookies for API
by Corion (Pope) on Jun 11, 2019 at 12:17 UTC

    You don't tell us where your approach fails and how.

    Simply converting your curl command line to LWP::UserAgent using curl2lwp gives:

    #!perl use strict; use warnings; use WWW::Mechanize; use HTTP::Request; my $ua = WWW::Mechanize->new(); my $r = HTTP::Request->new( 'GET' => '', [ 'Accept' => '*/*', 'Accept' => 'application/json', 'Authorization' => '<token>', 'Host' => '', 'User-Agent' => 'curl/7.55.1', ], ); my $res = $ua->request( $r, ); __END__ Created from curl command line curl -X GET "" -H "accept: application/json" -H "au +thorization:<token>"

    If your question is how to extract the cookie from a browser request, have a look at the response you see in the browser.

    If your question is how to extract the cookie automatically without involving a browser, first you need to find out what request you have to make to which URL, and also whether sending the username and password as a JSON string in a POST request is the correct approach there. If that is correct, then instead of inspecting the responseContent, inspect the response headers using $response->headers->as_string().

    You already mention a variable suggestively named $cookies, but you never pass it anywhere. See the ->cookie_jar method of LWP::UserAgent on how to set a cookie jar for LWP::UserAgent.

Re: LWP get Cookies for API
by karlgoethebier (Monsignor) on Jun 11, 2019 at 13:08 UTC
    "...i dont mind using something else..."

    It looks like you are a curl user. You might take a look at WWW::Curl::Easy (don't know if this helps) and Re: How to get html code from a secure (https:\\) page? (slightly OT). Best regards, Karl

    «The Crux of the Biscuit is the Apostrophe»

    perl -MCrypt::CBC -E 'say Crypt::CBC->new(-key=>'kgb',-cipher=>"Blowfish")->decrypt_hex($ENV{KARL});'Help

Re: LWP get Cookies for API
by abs52 (Initiate) on Jun 11, 2019 at 13:11 UTC
    Thanks for the replies I don't think i was that clear.

    To use the API i have to login (username and password) using a browser to the url ( and then (in chrome) show the cookies and look for the one called AUTH_TOKEN and then have use that token in the curl command. I am trying to find a way to grab that token so i can use it in my perl

    Hope that clears it up
      my $cookies = HTTP::Cookies->new(); my $ua = LWP::UserAgent->new( cookie_jar => {} );

      ICBW but it looks to me like you are setting up a new cookie jar in $cookies but then failing to associate that with your user agent. The docs for LWP::UserAgent also suggest using a HTTP::CookieJar::LWP instead. Try replacing these lines with

      my $cookies = HTTP::CookieJar::LWP->new; my $ua = LWP::UserAgent->new( cookie_jar => $cookies );

      and see if that gets you any further.

      You're already using Rest::Client to fetch the initial cookie. You should inspect $client->responseHeaders to see where the cookie you want lives. Most likely, it is something like:

      $client->POST('URL', ... credentials ...); my $token = $client->responseHeader('Set-Cookie'); ...

      A better approach is likely to use $ua for fetching the token instead, as that gives you far better control over the headers, and the $cookies cookie jar, if you properly associate it with LWP::UserAgent:

      my $response = $ua->post('URL', Content => ... credentials ...); my $token = $response->headers->{'Set-Cookie'}; # if there is only one + cookie line my $token = $cookies->get_cookies( 'URL', 'AUTH_TOKEN' ); ...

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://11101228]
Approved by LanX
Front-paged by haukex
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (6)
As of 2019-06-25 19:40 GMT
Find Nodes?
    Voting Booth?
    Is there a future for codeless software?

    Results (107 votes). Check out past polls.