Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

In a Web Page forms are displayed as empty

by jesuashok (Curate)
on Mar 30, 2007 at 10:27 UTC ( #607429=perlquestion: print w/ replies, xml ) Need Help??
jesuashok has asked for the wisdom of the Perl Monks concerning the following question:

monks,

I am using WWW::Mechanize for my inhouse project. Earlier I asked some question in WWW::Mechanize - Could not connect to a server and I got clarification from that node replies. Also there is a recent node which discusses about web forms in Submit HTML Forms with WWW:Mechanize. I picked up some code from there and used in my code, but I did not get any from names.

#!/usr/bin/perl use WWW::Mechanize ; use LWP::Debug qw(+); use strict; use warnings; use Data::Dumper; #use LWP::UserAgent; my $mech = WWW::Mechanize->new ; #my $ua = LWP::UserAgent->new; $mech->proxy(['http','ftp'], 'http://proxy_ip_dress:port_number/'); $mech ->get("http://application_ip_address:8082/root/Login") ; if ( $mech ->success ) { print "Connection Success ", $mech ->res->st +atus_line, "\n"; } else { print "Status :" , $mech ->res->status_line +, "\n"; } my @webforms = $mech->forms(); print Dumper [ @webforms ]; # @webforms is empty foreach my $form (@webforms) { my @inputfields = $form->param; foreach my $inputfield (@inputfields) { print Dumper [ $inputfield ]; } print Dumper [ $form ]; }
If I try to get google page I am getting value in the webforms arrary. If I use my application URL I am not getting that. Am I missing still something in the code ? will there be any web page without web forms ? Just curious to know.



hmmm ....let me think what did I said

Comment on In a Web Page forms are displayed as empty
Download Code
Re: In a Web Page forms are displayed as empty
by Preceptor (Chaplain) on Mar 30, 2007 at 10:52 UTC
    Well, if you're not getting any errors, then ... yes, you should have some content.

    If you're looking at just checking it works, is there a reason you're not using $mech -> content to see what you're actually fetching? I'd consider it more likely that something's up with your forms processing, than something trivial like a web page fetch...

      Hi Preceptor,

      Sorry for the delayed comment.
      As you suggested I used $mech -> content. I got the following output.

      #!/usr/bin/perl use WWW::Mechanize ; use LWP::Debug qw(+); use strict; use warnings; use Data::Dumper; my $mech = WWW::Mechanize->new ; $mech->proxy(['http','ftp'], 'http://63.117.9.150:8080/'); #Fetch URL or Die Tryin' $mech ->get("http://<application_ip>:8082/metrica_root/Login") ; if ( $mech ->success ) { print "Connection Success ", $mech ->res->sta +tus_line, "\n"; } else { print "Status :" , $mech ->res->status_line , + "\n"; } my @webforms = $mech->forms(); print Dumper [ @webforms ]; print "Trying to print the content ......\n"; $mech->content; foreach my $form (@webforms) { my @inputfields = $form->param; foreach my $inputfield (@inputfields) { print Dumper [ $inputfield ]; } print Dumper [ $form ]; }
      Output :- LWP::UserAgent::new: () LWP::UserAgent::proxy: ARRAY(0x1bf0e28) http://63.117.9.150:8080/ LWP::UserAgent::proxy: http http://63.117.9.150:8080/ LWP::UserAgent::proxy: ftp http://63.117.9.150:8080/ LWP::UserAgent::request: () HTTP::Cookies::add_cookie_header: Checking 155.226.230.173 for cookies HTTP::Cookies::add_cookie_header: Checking .226.230.173 for cookies HTTP::Cookies::add_cookie_header: Checking 226.230.173 for cookies HTTP::Cookies::add_cookie_header: Checking .230.173 for cookies HTTP::Cookies::add_cookie_header: Checking 230.173 for cookies HTTP::Cookies::add_cookie_header: Checking .173 for cookies LWP::UserAgent::send_request: GET http://155.226.230.173:8082/metrica_ +root/Login LWP::UserAgent::_need_proxy: Proxied to http://63.117.9.150:8080/ LWP::Protocol::http::request: () LWP::Protocol::collect: read 266 bytes LWP::Protocol::collect: read 301 bytes HTTP::Cookies::extract_cookies: Set cookie JSESSIONID => 0000000000000 +01a49444c3 a687474702f52657150726f636573736f723a312e30000000000000010000000000000 +0ac0001020 0000000103135352e3232362e3233302e31373300c39300000000004900504d4300000 +0000000001 a49444c3a687474702f52657150726f636573736f723a312e300020200000000974635 +f696e73743 1002020200000000d2f746f6d636174345f706f6100000000000000035649530300000 +0050005070 17f0000000000000000000008000000005649530000000001000000140000000000010 +0010000000 00001010900000000_2925080BF3F637D984812BF731798941 LWP::UserAgent::request: Simple response: OK Connection Success 200 OK $VAR1 = []; Trying to print the content ......
      I don't have any clues to move further on this. I am expecting your valid inputs on the same.
        Hi jesuashok,

        I had a hunch that the pages in question are generated by JavaScript using 'document.write', so I /msg'd you to confirm this. I have seen some pages that use document.write to output html. Let us take for a basic example the following html page:
        <html> <head> <title>Bad Example</title> </head> <body> <script> document.write('<H1>JavaScript made me do it!</H1>') </script> </body> </html>
        Now the following Perl code:
        #!/usr/bin/perl use strict; use warnings; use WWW::Mechanize; my $mech = WWW::Mechanize->new; $mech->get('http://yourserverhere/test.html'); print $mech->content( format => "text" );
        Which outputs: "Bad Example". Why? Because "Please note that Mech does NOT support JavaScript. Please check the FAQ in WWW::Mechanize::FAQ for more.".

        You can use Win32::IE::Mechanize or Mozilla::Mechanize or alike to control a browser which does know how to deal with JavaScript. This is another reason to always refer to the documentation.

        Update: Excuse the lack of error checking in the Perl script above, I wanted to get this post to you ASAP and I have to attend a project meeting.

        Hope this helps

        Martin
Re: In a Web Page forms are displayed as empty
by MonkE (Hermit) on Mar 30, 2007 at 13:06 UTC
    A related technique I've used before is to put the content in a file with $mech->save_content('filename.html'). Then have a look at it with a browser using a file:// URL.
      Hi

      #!/usr/bin/perl use WWW::Mechanize ; use LWP::Debug qw(+); use strict; use warnings; use Data::Dumper; #use LWP::UserAgent; my $mech = WWW::Mechanize->new ; #my $ua = LWP::UserAgent->new; $mech->proxy(['http','ftp'], 'http://proxy_ip_dress:port_number/'); $mech ->get("http://application_ip_address:8082/root/Login") ; if ( $mech ->success ) { print "Connection Success ", $mech ->res->st +atus_line, "\n"; } else { print "Status :" , $mech ->res->status_line +, "\n"; } $mech->save_content('sample.html');
      I have executed the above code and saved the content of a page in "sample.html". when I opened that file in Browser I could a see page saying that Page Cannot be displyed.
      any other points would be appreciated.



      hmmm ....let me think what did I said

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://607429]
Approved by GrandFather
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (7)
As of 2014-08-29 08:18 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (277 votes), past polls