Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris
 
PerlMonks  

Connection and Fetching

by freak
on May 06, 2004 at 10:30 UTC ( #351049=perlquestion: print w/replies, xml ) Need Help??

freak has asked for the wisdom of the Perl Monks concerning the following question:

If i connect to a website with LWP::UserAgent and fetch the whole website, does it count as if i've "entered" it in a browser ?

Replies are listed 'Best First'.
Re: Connection and Fetching
by inman (Curate) on May 06, 2004 at 10:40 UTC
    Have you read this node regarding a number of recent posting that you have made to this site? You will find that the monks are a generous and helpful bunch of people who are eager to help anyone with a Perl related problem. It would appear that a number of monks have tried to contact you in an effort to get you to explain what you are trying to do. Can you take a look at the discussion thread and answer a few of the questions?

    Regards

    inman

Re: Connection and Fetching
by castaway (Parson) on May 06, 2004 at 10:37 UTC
    Since browser and LWP send exactly the same commands to the web server, yes, of course it does. There's no magic to LWP, it just simulates what a browser would do, using perl.

    C.

Re: Connection and Fetching
by edan (Curate) on May 06, 2004 at 11:24 UTC

    No. Websites are like T.V. stations - data is constantly being 'broadcast' live over the Internet. A Web Browser is sort of like a T.V. It 'tunes in' to the right 'frequency' to get the website. The frequency is technically called an 'Address', and is specified by the 'URL' that you type into the browser.

    I understand from other posts of yours that you are interested in the technology known as 'web-counters' . This works in the same way that T.V. 'viewer ratings' (or 'Nielsen Ratings') work for Television, to measure how many people are viewing the website. An independent company provides a piece of hardware that attaches to the user's browser, and records what websites he is tuned into. The values are submitted to the WebCounter headquarters, and 'ratings' are generated, which are then delivered to the 'web-count' box that you see on the web page.

    Since LWP::UserAgent does not support this special hardware that attaches to the browser and records what websites you are tuned in to, you cannot simulate viewing a website with LWP::UserAgent, any more than you could increase the T.V. ratings of the hit show 'Friends' by popping a tape into your VCR and watching a taped version of last weeks show.

    I sincerely hope this helps you.

    --
    edan

    A reply falls below the community's threshold of quality. You may see it by logging in.
Re: Connection and Fetching
by matija (Priest) on May 06, 2004 at 10:42 UTC
    Not exactly. A browser doesn't usually fetch the whole website, it just downloads a page, and the img links for that page.

    Downloading the whole website will tend to make you conspicous fairly quickly.

    A reply falls below the community's threshold of quality. You may see it by logging in.
Re: Connection and Fetching
by dave_the_m (Monsignor) on May 06, 2004 at 11:14 UTC
    If i connect to a website with LWP::UserAgent and fetch the whole website, does it count as if i've "entered" it in a browser ?
    Depends. If the website contains swallows, then the browser may download the European variety, while LWP::UserAgent will get the African ones. To get round this, you need to specify this in the setup stage:
    $ua->type('European'); $ua->using('Coconuts'); $ua->dorsal_fin->tie();
    Of course this still won't help if your connection is lighter than a duck.
Re: Connection and Fetching
by blue_cowdawg (Monsignor) on May 06, 2004 at 13:27 UTC

    Iff yuoo cunnect tu a vebseete-a feea [cpun://LVP::UserEgent] it cuoonts in rumun noomerels und iff yuoor inter zee deener thruoogh a brooser it uses und ebecoos. Um gesh dee bork, bork! Su noo zeey dun't qooeete-a cuoont zee seme-a. Boot zeen it meeght depend oon zee brooser. Hurty flurty schnipp schnipp! Iff yuoo use-a Oopera it meeght use-a a cuoonteeng steeck insteed. Bork bork bork! Boot seence-a zee munooel ves vreettee in Svedeesh Cheff (bork! bork!) it ves herd fur me-a tu reed. Bork bork bork!

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://351049]
Approved by inman
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others meditating upon the Monastery: (6)
As of 2021-01-25 22:33 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Notices?