Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

LWP / HTPP Post not working

by amitsq (Beadle)
on Jul 24, 2017 at 14:43 UTC ( [id://1195869]=perlquestion: print w/replies, xml ) Need Help??

amitsq has asked for the wisdom of the Perl Monks concerning the following question:

I am currently using Perl to write a program which can automatically submit information to some websites with LWP, getting results, which i print in a table on my own website. For most of the pages, it works well, but for one it doesn't: After submit, I get a redirection to a captcha (no need of any complex solving her, after extracting the captcha, i want the user to solve it and enter the code). However once i want to post the reply to reach to the result page, the session get reloaded and I can not fetch the result page. Whats going wrong?
#extracted captcha picture before, and catched user's reply in $captch +a #trying to submit captcha code: $param="numIg=&numDgc=&numIgc=&numIp=&pageSize=3&strWhere='.$mySearch. +'&numDg=&numIpc=&numUgc=&numUgd=&strLicenseCode=&numIgd=&showType=1&n +umSortMethod=&strSources=&numUg=&pageNow=1&vct='.$captcha.'&Submit=%E +7%BB%A7%E7%BB%AD"; $Search_url ="one website"; $request = HTTP::Request->new('POST', $Search_url); $request->header('Accept' => 'text/html,application/xhtml+xml,applicat +ion/xml;q=0.9,*/*;q=0.8'); #$request->header('Accept-Language' => 'de,en-US;q=0.7,en;q=0.3'); $request->header( 'Referer' => "one webiste"); $request->header('Connection' => 'keep-alive'); $request->content($param); $response = $ua->request($request); $page = $response->decoded_content(); print $page;
If i visit the html page, i can enter the captcha code and get to the result page directly. To compare, the request observed by the firefox plugin looks as following:
Request Header Host: website User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/2010010 +1 Firefox/52.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0. +8 Accept-Language: de,en-US;q=0.7,en;q=0.3 Accept-Encoding: gzip, deflate Referer: website Cookie: _gscu_1718069323=94938064vupi3q75; _gscu_7281245=960585782o5hw +t18; fp_ver=4.2.3; BSFIT_CLICK_ARRAY=; _va_id=6af9c5bacce82b34.149725 +6276.9.1500561808.1500561808.; _va_ref=%5B%22%22%2C%22%22%2C150056180 +8%2C%22http%3A%2F%2Fepub.sipo.gov.cn%2Fpatentoutline.action%22%5D; BS +FIT_EXPIRATION=1499900863032; BSFIT_OkLJUJ=FCPai-v72cPAKxWwXtmw7Q5V23 +KeTUXF; BSFIT_DEVICEID=DJtgNB-72bmDzc9ryT1Cgn-fF8rxZOW3kAgDU2FyxVGjzC +7zxS49h52Qp1ACtmLH80ib8FDb4CmAPGjJLNEZapA7Wp83mJtoq2ZD4U1hXHNjGYkHEss +9Rd21Z7bHaoRKU4o5N6CCTPsB36cPMQwXSxGrZuTY7XRd; br_access_code=ZrbCHtq +r/h8AAAAATpJHWQAAAABmB0vG; JSESSIONID=7F350E89175BADCBEEDEA44B6F12F4E +A; WEB=20111116; _gscbrs_7281245=1; _gscs_7281245=t009042585zmvlf21|p +v:5 Connection: keep-alive Upgrade-Insecure-Requests: 1
Request Body ontent-Type: application/x-www-form-urlencoded Content-Length: 267 numIg=&numDgc=&numIgc=&numIp=&pageSize=20&strWhere=%27.%E6%97%A0%E5%88 +%B7%E7%94%B5%E6%9C%BA.%27&numDg=&numIpc=&numUgc=&numUgd=&strLicenseCo +de=&numIgd=&showType=1&numSortMethod=&Submit=%E7%BB%A7%E7%BB%AD&strSo +urces=&numUg=&pageNow=1&vct=y23m4&Submit=%E7%BB%A7%E7%BB%AD

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1195869]
Approved by ww
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (4)
As of 2024-04-25 07:58 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found