Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

Need to pull a javascript generated webpage

by miriamherald (Initiate)
on Feb 03, 2010 at 17:22 UTC ( #821224=perlquestion: print w/replies, xml ) Need Help??
miriamherald has asked for the wisdom of the Perl Monks concerning the following question:

I have a webpage that is generated almost entirely using javascript:
<div class="teamleaders"> <script LANGUAGE="JavaScript" SRC="http://www.mywebpage.com"></script>
Pulling this page just reveals all the javascript code rather than the html generated by the page. All I want to do is 'get' the file using an LWP agent and print the html to a new static file - I do not want the javascript code, I want rather the html that it generates. How can I do this?

Replies are listed 'Best First'.
Re: Need to pull a javascript generated webpage
by Corion (Pope) on Feb 03, 2010 at 17:28 UTC
      Thanks Corion- how do you suggest I go about replicating it? I've used the Perl::Mechanize module before, but not Mechanize::Firefox. Can you give me a few more tips on how to implement this? I still feel lost.
Re: Need to pull a javascript generated webpage
by pileofrogs (Priest) on Feb 03, 2010 at 19:04 UTC

      Hm, are you sure that lynx supports Javascript (to render the page)?

      Maybe you are thinking of ELinks, which does have some limited JavaScript support?

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://821224]
Approved by marto
help
Chatterbox?
Corion had a meeting with some startup today. They have a very interesting DB proxy product, but their tech stack is really, really weird. They use the Pg wire protocol but not the Pg libraries to handle it. They support Pg SQL syntax, but don't use ...
[Corion]: ... the Pg parser (or so they claim).
[Corion]: Also, they rolled their own user management instead of supporting LDAP for user/role management, but that just shows that they're new in the enterprise market :)
Corion also just now realizes they didn't leave business cards.

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (7)
As of 2018-04-19 12:13 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Notices?