http://www.perlmonks.org?node_id=934489

gizmo_mathboy has asked for the wisdom of the Perl Monks concerning the following question:

I'm doing some web scraping and falling short.
#!/usr/local/bin/perl use strict; use warnings; use WWW::Mechanize::Firefox; use 5.010; my $mech = WWW::Mechanize::Firefox->new( autoclose => 0, ); my $url = q(https://selfservice.mypurdue.purdue.edu/prod/bwckctlg.p_disp_dyn_ctl +g?); $mech->get($url); say q(got url); say q(choose term); $mech->form_number(1); $mech->field('cat_term_in', '201220'); $mech->submit(); say q(choose subject); $mech->form_number(1); $mech->select('sel_subj', ['AAE']); $mech->submit(); say $mech->content();
This will generate.
got url choose term choose subject 2 elements found for './/*[(local-name(.)="input" or local-name(.)="se +lect" or local-name(.)="textarea") and @name="sel_subj"]' at ./get_co +urse_descriptions2.pl line 28
I think this is because the html for the page has:
<FORM ACTION="/prod/bwckctlg.p_display_courses" METHOD="POST"> <INPUT TYPE="hidden" NAME="term_in" VALUE="201220"> <INPUT TYPE="hidden" NAME="sel_subj" VALUE="dummy"> ^^^^^ Relavant Hidden Input Type <SNIPPED HTML> vvvvvvvvv Relevant Select Name <SELECT NAME="sel_subj" SIZE="3" MULTIPLE ID="subj_id"> <OPTION VALUE="AAE">AAE-Aero & Astro Engineering
How the do I get it to properly select a value? I think the problem lies in that there is a hidden input type of the same name as the select. The select has a label and id defined but I'm not sure how to use those. I think this script might work in plain WWW::Mechanize but it just returns the same page without giving me this error. Thanks, gizmo