LWP HTTP Request POST being sent in two packets

[Anonymous Monk]

I'm trying to automate some data submission to a web page. There are 3 different forms I need to use, 2 submit via GET and the 3rd via POST. I've got the code for the GET forms working fine, however the POST is unsuccesful.

If I change the 3rd to a GET, it is partially successful. The first half of the fields are properly submitted, but the latter half is lost.

I realize that this is because GET requests can consist only of header info and are limited to 512 byes. This 3rd post is approaching 600 bytes, and the latter fields are not received/processed because the data for them falls beyond that limit.

If I submit the form as a POST I get a "200 Created" status line, but the web server does not process the transmission. This is not Apache/IIS/Netscape. It is a proprietary wwww server for a networking monitoring tool, WhatsUp from IpSwitch.

I think the problem with the POST is that Perl breaks the headers and the content for the POST into 2 different packets. The same submission from IE is a single packet. I realize this probably means that the web server is not completely HTTP compliant, but I'm stuck with it and have no other options.

I have tried using LWP "send_request" and "simple_request" because the docs says "these methods dispatch a single request" whereas "request" which says "this method may actually send several different requests" but they do not perform any differently. All three methods make the transmission as 2 packets, one for headers and another for content.

Looking for any suggestions on how to submit a POST from within Perl that only consists of a single packet.

Thanks

[RMGir]

Interesting problem...

I don't know whether you can convince LWP to do this the way you want or not, but did you look at Net::HTTP? It's labeled "experimental" in the version I have, but I think it'll do what you need; its write_request method seems to print the request in one shot to the socket.
--
Mike

[dws]

I realize that this is because GET requests can consist only of header info and are limited to 512 bytes.

It's actually the length of the URL that's the limit, and it's unequally enforced.

I think the problem with the POST is that Perl breaks the headers and the content for the POST into 2 different packets. The same submission from IE is a single packet.

This might be a case where you'll need to keep buffering enabled. If you find a $|++; in your code, comment it out.

To get the entire header + form out in one packet, you might need to dive in a level lower than LWP, and deal with the raw socket. Here is some info that'll get you started.

[Mr. Muskrat]

Can you post the code so that we can at least try to see if there is something wrong there?

[Anonymous Monk]

This is the code that is sending the headers & content in 2 different packets.
I've also tried a different version using query_form() from the URI module. With that it is all send as a single packet, but the parameters are then part of the URI and not the content. Can someone confirm if query_form should only be used with GETs?

use HTTP::Request::Common;
use LWP::UserAgent;

$ua = LWP::UserAgent->new;
$ua->request(POST 'http://user:[e-mail]/devicesave.asp',
         Content_Type    => 'application/x-www-form-urlencoded',
          Referer        => 'http://x.y.z.com/newdevice.asp?map=CHI.wu
+p',
         Content        => [
                            'map'                => 'CHI.wup',
                            'CreateNewDevice'    => 'true',
                            'Title'                => 'node',
                            'HostName'            => 'node.acme.com',
                            'IP'                => '127.0.0.1',
                            'PollType'            => 'ICMP',
                            'HostType'            => 'Server',
                            'Info1'                => 'info1',
                            'Info2'                => 'info2',
                            'Snmp'                => '1',
                            'ReadComm'            => 'read',
                            'WriteComm'            => 'write',
                            'OID'                => '',
                            'Logging'            => '1',
                            'Monitoring'        => '1',
                            'StartTime'            => '0000',
                            'EndTime'            => '2400',
                            'Sun'                => '1',
                            'Mon'                => '1',
                            'Tues'                => '1',
                            'Wedn'                => '1',
                            'Thur'                => '1',
                            'Fri'                => '1',
                            'Sat'                => '1',
                            'PollFreq'            => '1',
                            'TimeOut'            => '5000',
                            'DownDep'            => '0',
                            'UpDep'                => '0',
                            'Notes'                => '',
                            'end'                => 'end'
                            ]
                );