Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options

ASP's with lwp

by (Initiate)
on Apr 09, 2004 at 15:48 UTC ( #343923=perlquestion: print w/replies, xml ) Need Help?? has asked for the wisdom of the Perl Monks concerning the following question:

Active Perl 5.8.2, Win 2K Server,
I'm bringing web pages to extract data using lwp::simple. Most pages retrieve perfectly. But when I access an active server page (, I'm getting the following error:

<h2>There was an error loading the header for this page.</h2> <p> An error has occurred. You may contact your system administrator w +ith the following information: </p> <p>Location : /mktdata/mktstat.asp</p> <p>Source : Microsoft VBScript runtime error</p> <p>Code : 5</p> <p>Description : Invalid procedure call or argument</p> <br></br>

What do I need to do to access asp pages properly?


20040409 Edit by Corion: Fixed formatting

Replies are listed 'Best First'.
Re: ASP's with lwp
by Jenda (Abbot) on Apr 09, 2004 at 20:05 UTC
    What do I need to do to access asp pages properly?

    Access only pages without bugs. They aparently have a bug in the page. Seems like they try to emit different output based on the user's browser and the code is broken. If I download the page with Opera set to indentify itself as MSIE everything is cool, if I change it to identify as Opera I get exactly the same error as you.

    1. Contact the site and inform them of the problem. (It might be better to talk about Opera than about your script. That way they can test it themselves and cannot say it's your bug.)
    2. Ask LWP to identify itself as MSIE.
    use LWP::Simple qw($ua); $ua->agent('Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CL +R 1.0.3705; .NET CLR 1.1.4322)'); ...

    HTH, Jenda
    Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.
       -- Rick Osborne

    Edit by castaway: Closed small tag in signature

Re: ASP's with lwp
by esskar (Deacon) on Apr 09, 2004 at 16:24 UTC
    try a different user-agent; on like the MS Internet Explorer is using. just a guess!
Re: ASP's with lwp
by eXile (Priest) on Apr 09, 2004 at 16:26 UTC
    Hi, hard to tell what you need from the error message you got back. I think the ASP expects a certain line to be in your request. For instance if I access a page with Internet Explorer the header looks something like:
    GET / HTTP/1.1 Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, applicati +on/msword, application/, application/, a +pplication/x-shockwave-flash, */* Accept-Language: nl Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1) Host: Connection: Keep-Alive
    when I do the same in LWP it looks like:
    GET / HTTP/1.1 TE: deflate,gzip;q=0.3 Connection: TE, close Host: User-Agent: libwww-perl/5.69
    You can make dumps like this with a packetsniffer (tcpdump, ethereal). My guess is you have to add/change your LWP request so it more closely resembles the Internet Explorer one. An obvious first try is to change the User-Agent header using the agent method of a LWP::UserAgent. Hope this helps.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://343923]
Approved by Corion
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (3)
As of 2023-06-03 18:57 GMT
Find Nodes?
    Voting Booth?
    How often do you go to conferences?

    Results (16 votes). Check out past polls.