Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

LWP::UserAgent throwing 1010 error

by throwaway (Novice)
on Feb 13, 2021 at 02:11 UTC ( #11128318=perlquestion: print w/replies, xml ) Need Help??

throwaway has asked for the wisdom of the Perl Monks concerning the following question:

using LWP user agent and gettting error 1010 for every website i try besides setup api json endpoints, this is the error seen in terminal

HTTP/1.1 403 Forbidden Cache-Control: private, max-age=0, no-store, no-cache, must-revalidate +, post-check=0, pre-check=0 Connection: close Date: Sat, 13 Feb 2021 02:04:56 GMT Server: cloudflare Content-Length: 16 Content-Type: text/plain; charset=UTF-8 Expires: Thu, 01 Jan 1970 00:00:01 GMT CF-RAY: 620afb475bb5f242-ORD Cf-Request-Id: 083abb60980000f242471b1000000001 Client-Date: Sat, 13 Feb 2021 02:04:56 GMT Client-Peer: 104.19.128.108:443 Client-Response-Num: 1 Client-SSL-Cert-Issuer: /C=US/O=Cloudflare, Inc./CN=Cloudflare Inc ECC + CA-3 Client-SSL-Cert-Subject: /C=US/ST=CA/L=San Francisco/O=Cloudflare, Inc +./CN=sni.cloudflaressl.com Client-SSL-Cipher: TLS_AES_256_GCM_SHA384 Client-SSL-Socket-Class: IO::Socket::SSL Client-SSL-Version: TLSv1_3 Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.c +om/cdn-cgi/beacon/expect-ct" Set-Cookie: __cfduid=d8ad08b4932ed14004949bb070cb4f43f1613181896; expi +res=Mon, 15-Mar-21 02:04:56 GMT; path=/; domain=.4chan.org; HttpOnly; + SameSite=Lax; Secure X-Frame-Options: SAMEORIGIN error code: 1010

This is the exact code im using except url

use LWP::UserAgent; # specify a CA path my $ua = LWP::UserAgent->new(); my $response = HTTP::Request->new(GET => 'https://example.com/'); my $res = $ua->request($response)->as_string; print $res

Replies are listed 'Best First'.
Re: LWP::UserAgent throwing 1010 error
by jcb (Parson) on Feb 13, 2021 at 03:02 UTC

    You are using a website "Proudly Broken by CloudFlare" and the error you are getting is intentional.

    In fact, details in your post indicate that the site in question is 4chan. Why are you pointing LWP::UserAgent at 4chan?

      "Why are you pointing LWP::UserAgent at 4chan?"

      For the lulz I'm guessing...

Re: LWP::UserAgent throwing 1010 error
by throwaway (Novice) on Feb 13, 2021 at 02:16 UTC
    seems like something to do with cloudfare? maybe some authorization that im not configuring?
Re: LWP::UserAgent throwing 1010 error
by throwaway (Novice) on Feb 13, 2021 at 14:09 UTC
    why not its simple html to use when im learning lol but i was using wrong urls!!!! i guess the endpoints were changing from what I would use in the browser to what was actually having a connection in the end :/
Re: LWP::UserAgent throwing 1010 error
by throwaway (Novice) on Feb 13, 2021 at 14:55 UTC
    any advice on matching nested structures like html would also be appreciated
        thank you those are great, I think I will try MOJO

      "any advice on matching nested structures like html would also be appreciated "

      Here you go: Never send a regexp to do a parser's work.

      Hope this helps!


      The way forward always starts with a minimal test.
        ahhh true it does, thanks for that
Re: LWP::UserAgent throwing 1010 error
by throwaway (Novice) on Feb 13, 2021 at 14:20 UTC
    4chan was becoming 4channel in this case lmao

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://11128318]
Approved by jcb
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (3)
As of 2021-05-15 18:56 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Perl 7 will be out ...





    Results (150 votes). Check out past polls.

    Notices?