Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

LWP HTTP Request POST being sent in two packets

by Anonymous Monk
on Oct 10, 2002 at 16:00 UTC ( #204211=perlquestion: print w/ replies, xml ) Need Help??
Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to automate some data submission to a web page. There are 3 different forms I need to use, 2 submit via GET and the 3rd via POST. I've got the code for the GET forms working fine, however the POST is unsuccesful.

If I change the 3rd to a GET, it is partially successful. The first half of the fields are properly submitted, but the latter half is lost.

I realize that this is because GET requests can consist only of header info and are limited to 512 byes. This 3rd post is approaching 600 bytes, and the latter fields are not received/processed because the data for them falls beyond that limit.

If I submit the form as a POST I get a "200 Created" status line, but the web server does not process the transmission. This is not Apache/IIS/Netscape. It is a proprietary wwww server for a networking monitoring tool, WhatsUp from IpSwitch.

I think the problem with the POST is that Perl breaks the headers and the content for the POST into 2 different packets. The same submission from IE is a single packet. I realize this probably means that the web server is not completely HTTP compliant, but I'm stuck with it and have no other options.

I have tried using LWP "send_request" and "simple_request" because the docs says "these methods dispatch a single request" whereas "request" which says "this method may actually send several different requests" but they do not perform any differently. All three methods make the transmission as 2 packets, one for headers and another for content.

Looking for any suggestions on how to submit a POST from within Perl that only consists of a single packet.

Thanks

Comment on LWP HTTP Request POST being sent in two packets
Re: LWP HTTP Request POST being sent in two packets
by RMGir (Prior) on Oct 10, 2002 at 16:23 UTC
    Interesting problem...

    I don't know whether you can convince LWP to do this the way you want or not, but did you look at Net::HTTP? It's labeled "experimental" in the version I have, but I think it'll do what you need; its write_request method seems to print the request in one shot to the socket.
    --
    Mike

Re: LWP HTTP Request POST being sent in two packets
by Mr. Muskrat (Abbot) on Oct 10, 2002 at 16:34 UTC
    Can you post the code so that we can at least try to see if there is something wrong there?
Re: LWP HTTP Request POST being sent in two packets
by dws (Chancellor) on Oct 10, 2002 at 17:40 UTC
    I realize that this is because GET requests can consist only of header info and are limited to 512 bytes.

    It's actually the length of the URL that's the limit, and it's unequally enforced.

    I think the problem with the POST is that Perl breaks the headers and the content for the POST into 2 different packets. The same submission from IE is a single packet.

    This might be a case where you'll need to keep buffering enabled. If you find a $|++; in your code, comment it out.

    To get the entire header + form out in one packet, you might need to dive in a level lower than LWP, and deal with the raw socket. Here is some info that'll get you started.

Re: LWP HTTP Request POST being sent in two packets
by Anonymous Monk on Oct 10, 2002 at 17:48 UTC
    This is the code that is sending the headers & content in 2 different packets.
    I've also tried a different version using query_form() from the URI module. With that it is all send as a single packet, but the parameters are then part of the URI and not the content. Can someone confirm if query_form should only be used with GETs?
    use HTTP::Request::Common; use LWP::UserAgent; $ua = LWP::UserAgent->new; $ua->request(POST 'http://user:password@z.y.z.com/devicesave.asp', Content_Type => 'application/x-www-form-urlencoded', Referer => 'http://x.y.z.com/newdevice.asp?map=CHI.wu +p', Content => [ 'map' => 'CHI.wup', 'CreateNewDevice' => 'true', 'Title' => 'node', 'HostName' => 'node.acme.com', 'IP' => '127.0.0.1', 'PollType' => 'ICMP', 'HostType' => 'Server', 'Info1' => 'info1', 'Info2' => 'info2', 'Snmp' => '1', 'ReadComm' => 'read', 'WriteComm' => 'write', 'OID' => '', 'Logging' => '1', 'Monitoring' => '1', 'StartTime' => '0000', 'EndTime' => '2400', 'Sun' => '1', 'Mon' => '1', 'Tues' => '1', 'Wedn' => '1', 'Thur' => '1', 'Fri' => '1', 'Sat' => '1', 'PollFreq' => '1', 'TimeOut' => '5000', 'DownDep' => '0', 'UpDep' => '0', 'Notes' => '', 'end' => 'end' ] );
Re: LWP HTTP Request POST being sent in two packets
by NodeReaper (Curate) on Oct 10, 2002 at 17:48 UTC

    Reason: BrowserUk Appears to be a total duplicate of 204255

    For more information on this node visit: this

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://204211]
Approved by RMGir
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others cooling their heels in the Monastery: (5)
As of 2014-12-20 13:24 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (95 votes), past polls