Debugging a POST that won't

[BernieC]

I can't get a POST to work and I can't figure out why not. I can get and post well enough to get authenticated and otherwise navigate around the site, but this post just won't work. I've tried comparing my _request with the request I see in a WebDeveloper capture of submitting the same POST and as far as I can tell they are identical. I submit all of the same fields that Firefox does {collected from the WD capture - I did't trust that if I just looked at the "view page source" I wouldn't miss one}. The site doesn't give me a hint as to what's wrong, of course, I just get to the appropriate reponse page but with my 'submit' having not happened. Any ideas/tricks for how to figure out what's wrong? It's https:// so I can't go beyond WD to do sniffing of the of actual network data. There aren't any redirects or other oddities: WD says when I click "submit" firefox does a single POST that, as I mentioned, looks pretty much identical to my UserAgeng POST. I have the security tokens correct {it uses two: a security_token POST paramter and a csrf cookie}. I don't know where else to look or what else to try. <p<Just to make it more concrete, the code involved is dead simple:

    @fields = (current_step => "New" ,
                    next_step => "NewFinal" ,
  [etc ... all the identical fields from both scraping the HTML and lo
+oking at WD]            );
                    );
     unshift @fields, csrf_token => $csrf_token if $csrf_token ;
    unshift @fields, security_key => $security_key if $security_key ;
    $res = $ua->request(POST $_[0], \ @fields) ;

What should I try next?

[haj]

Two ideas:
• An intermediate step between a direct POST and using Firefox would be automating your Firefox with WWW::Mechanize::Firefox. This might give a hint whether there's some Javascript happening in Firefox which you didn't mimic yet.
• Also, some websites don't like robots, so you might carefully set your user agent's identification to that of your Firefox.
Good luck!

[markong]

Are you sure that the entire HTTP dialog is the same? With the help of LWP::ConsoleLogger::Easy you can inspect deeply the entire LWP dialog with the server to double check.

Also be sure that Firefox web dev tools (you are looking at the net tab I guess) have the "preserve session" option checked (I can't recall the exact name at the moment) to prevent a clean up in between redirects.

Also check out that the "common" header values are the same e.g.: which UserAgent are you sending with LWP ?

[Anonymous Monk]

You say everything is identical. It is not. Check the headers