http://www.perlmonks.org?node_id=221012

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks,

I am hoping someone can help me out with a problem retrieving and storing cookies that are written to an HTML page.

My Perl script uses LWP to retrieve remote web pages and then stores cookies that are returned in the HTTP header with no problems.

Several pages I am seeing lately use javascript to write their cookies within the HTML, and my script fails to recognize these cookies.

Here is a snippet of my code...

$ua = new LWP::UserAgent; $cookie_jar = HTTP::Cookies->new(file => "cookies.dat", autosave = +> 1, ignore_discard => 1); $ua->cookie_jar($cookie_jar); $header = new HTTP::Headers (Accept => 'text/html', Referer => $referer); $ua->agent($browser); $request = HTTP::Request->new(GET => $link, $header); $response = $ua->request($request); if ($response->is_success) { ...do stuff }

My question is how can I access and store a cookie that is written within the HTML that looks something like this...

<SCRIPT language=JavaScript> <!-- document.cookie = "ref=" +top.document.referrer + "; ; PATH=/; ;" // --> </SCRIPT>

Replies are listed 'Best First'.
Re: (nrd) Retrieve and Store JavaScript cookies
by newrisedesigns (Curate) on Dec 19, 2002 at 02:50 UTC

    Check out the JavaScript module that's on CPAN. Perhaps you can print/writeln out the document.cookie, and evaluate it with the module, which would return the cookie.

    I'm sure you could also do this with HTML::TokeParser or with some regexps. Hope this helps.

    John J Reiser
    newrisedesigns.com

Re: Retrieve and Store JavaScript cookies
by CountZero (Bishop) on Dec 19, 2002 at 06:33 UTC

    As there are probably more ways to make a cookie in JavaScript can we all care to think about, other than by somehow 'running' the JavaScript, it will be impossible to know

    1. if there is a cookie;
    2. what the content of the cookie is.

    This means that you must execute all JavaScript code on the webpage, to be sure that the correct value of the cookie is calculated.

    CountZero

    "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law

Re: Retrieve and Store JavaScript cookies
by perrin (Chancellor) on Dec 19, 2002 at 14:34 UTC
    If the JavaScript is written correctly, this will work just fine. Cookies are cookies, no matter how they were set. There may be something wrong with the JavaScript code that is preventing the cookies from being written.