Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"

WWW::Mechanize and javascript

by rhxk (Beadle)
on Mar 23, 2004 at 21:53 UTC ( #339222=perlquestion: print w/ replies, xml ) Need Help??
rhxk has asked for the wisdom of the Perl Monks concerning the following question:

My objective is to write a perl script I can put in my cron to do my timecard daily & submit when necessary...all automated.

I wrote a perl script that utilizes the WWW::Mechanize module where it basically screen-scrapes the webpage, follows a few links, and logs me in.

However, I get to a page that has the link I'd like to go to:
<a HREF="javascript:SetOkCode('CATW')" onMouseOver="rollOver('A1', 't +ime.gif')" onMouseOut="rollOver('A1', 'time.gif')" ><IMG SRC="right.g +if" BORDER=0></A>
But, WWW::Mechanize doesn't support javascript... has anyone done anything like it?


Comment on WWW::Mechanize and javascript
Download Code
Replies are listed 'Best First'.
Re: WWW::Mechanize and javascript
by saintmike (Vicar) on Mar 23, 2004 at 22:16 UTC
    Even if you can run JavaScript within your script via something like JavaScript::SpiderMonkey, it will still be a challenge to understand the browser's DOM completely. JavaScript itself doesn't know what an "onClick" event is, it's just a language. Only in combination with the browser's DOM you get this powerful browser-scripting ability. I heard that leira is working on something like that.

    Until then, I'd recommend using a logging proxy like Just start it on the command line like


    and when it says Server listening at port 8017, point your browser to it via Preferences-Advanced-Proxies, where you click on Manual Proxy Configuration and set the HTTP proxy entry to localhost and port 8017.

    With this enabled, the logging proxy will tell you exactly what he browser sends over the wire when you're surfing, and you can simply wrap its output into a WWW::Mechanize script.

      I ran the in regular http:// requests and saw the very nice output. I can certainly use it when I want to know what mozilla is doing & redirect it to a perl script...very nice..thanx...

      However, for this task, that webpage I need to access resides on the https protocol & didn't see STDOUT the https requests.

      should I add localhost:8017 in the mozilla proxy or ?????

        Unfortunately, the proxy approach ( or HTTP::Recorder) works only on plain HTTP, not in combination with SSL.

        If it's just a single request you're trying to examine, could you possibly wing it by saving the page locally and modify the URL it sends the data to, as described here?


        Instead of the proxy hassle, you may also want to look at Live HTTP Headers (since you're already seem to use Mozilla).


        All code is usually tested, but rarely trusted.
        how do i get the log file
        I tried using the method defined.I executed
        On doing this, it said Server listening at port 8017
        then as per instructions, i directed the browser's proxy settings to loacalhost with port 8017
        What i did not understand is how do i obtain the log file.
        Please help
Re: WWW::Mechanize and javascript
by bbfu (Curate) on Mar 23, 2004 at 22:03 UTC

    There's a Javascript module on CPAN, but it interfaces to libjs, so you'll need that.

    In my experience, however, it's usually easier to simply look at the javascript code in the page, and use perl to emulate whatever calculations or changes the javascript would accomplish. Usually it's little more than setting a particular form value.

    Black flowers blossom
    Fearless on my breath

Re: WWW::Mechanize and javascript
by Roy Johnson (Monsignor) on Mar 23, 2004 at 22:31 UTC
    See this thread.

    Basically, you need to figure out what the javascript does -- look at the SetOkCode function, see what it does, and keep following along -- to figure out where you need to go next.

    The PerlMonk tr/// Advocate
Re: WWW::Mechanize and javascript
by Vautrin (Hermit) on Mar 23, 2004 at 22:28 UTC
    If you post the javascript for the rollOver function I can try and help you understand it and translate it into Perl. Also, try downloading Mozilla and install the Venkman Javascript debugger. That will let you step through the javascript on a page, so even if it isn't quite obvious at first glance of the code what is going on you can find it out. Also, there are facilities to view all the functions, etc. of the javascript like most good IDEs.

    Want to support the EFF and FSF by buying cool stuff? Click here.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://339222]
Approved by kvale
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (8)
As of 2015-10-06 20:42 GMT
Find Nodes?
    Voting Booth?

    Does Humor Belong in Programming?

    Results (160 votes), past polls