To try to give you a little more direction on this ... in the aforementioned perldoc, there are two things to zero in on: (1) WWW::Mechanize doesn’t support JavaScript, and (2) JavaScript is critical to what this web page is doing. The page is sending AJAX requests to a host, and, based on the results obtained, it (JavaScript) is populating the box. Furthermore, only it knows how to do it, and when it has finished.
If you simply used UserAgent to “fetch the page,” it would be kinda like exhuming a lifeless corpse. You would have the HTML and maybe even all the JavaScript, but no way to bring the thing to life, and unless it is “alive,” it’s useless to you.
At the bottom of the perldoc section is a paragraph entitled, Which modules work like Mechanize and have JavaScript support?“ The one I’m most familiar with is WWW::Selenium, which actually is an interface to the Selenium web-testing framework. Selenium spawns an actual web-browser process (of your choice ...) and uses it to access the site. The browser is controlled and monitored by Selenium, but it is an actual browser and therefore able to execute scripts that are part of the page. Perl, in turn, is able to control Selenium.
