note Sinistral <p>The most likely candidate NPM seems like it might be <a href="http://search.npmjs.org/#/jscrape">jscrape</a>, which combines <a href="http://search.npmjs.org/#/jsdom">jsdom</a>, <a href="https://new.npmjs.org/package/request">request</a>, and jquery. The reason I recommended Javascript / Node as an option is your own language:</p> <blockquote><p>This works more-or-less the way I intended, there are two problems though - since the list of links is dynamic, and partly created using javascript, I had to use the browser to create that list.</p> <p>I need a way of parsing web page, and getting a list of all its component, and this is my first problem.</blockquote> </p> <p>If you are dealing with pages that use Javascript to dynamically load resources, then you have to have something that can interpret that Javascript as a browser would.</p> <p>As something completely different, you might want to check out <a href="http://seleniumhq.org/">Selenium</a>.</p> 981041 981391