I am processing a webpage with Perl using IO::All and the io($url) function.
The pages I am processing ostensibly have well formed URLs. e.g.
http://www.forbes.com/billionaires/list/#page:15_sort:0_direction:asc_ +search:_filter:All%20industries_filter:All%20countries_filter:All%20s +tates
Note the #page:15_ portion of the url
When I view the source or print out the source return fromm io() I see names that appear on page 1 of the website, e.g.,
<!-- Start: list_row --> <tr> <td class="rank">1</td> <td class="company"> <a href="/profile/carlos-slim-helu/"> <img src="http://i.forbesimg.com/media/lists/people/carlos-slim- +helu_50x50.jpg" alt=""> <h3>Carlos Slim Helu & family</h3> </a> </td> <td class="worth">$73 B</td> <td>73</td> <td>telecom</td> <td>Mexico</td> </tr>
However, if I open the page up in developer tools in the browser and look through the object model (HTML tab in IE)I see the entries in the list for the people I expect to see on page 15 e.g.,
<TR> <TD class=rank>1342</TD> <TD class=company><A href="/profile/park-hyeon-joo/"> <IMG alt="" src="http://i.forbesimg.com/media/lists/people/park-hy +eon-joo_50x50.jpg"> <H3>Park Hyeon-Joo</H3></A> </TD> <TD class=worth>$1 B</TD> <TD>54</TD> <TD>Mirae</TD> <TD>South Korea</TD> </TR>
Can I get Perl to open the page and give me the right contents?
Someone suggested looking at Mechanize. I have looked at pQuery and Web::Query. The ->text methods just return the data from page 1, not page 15.
Is there a way to keep the url "alive" until it refeshes with the correct content?
Thanks in advance