Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change

Re: Re: suck on www::mechanize question

by smackdab (Pilgrim)
on Mar 01, 2004 at 07:10 UTC ( #332809=note: print w/replies, xml ) Need Help??

in reply to Re: suck on www::mechanize question
in thread can't get www::mechanize to work on a web site

Thanks for trying it!
The connection refused error might be the problem, but I can get it to work in the browser, not www::mechanize. I'll look into HTTP::Recorder and see what it does...

Did you get it to work??????

I tried switching the artist/song to others w/o change. I also tried the ->follow_link having the artist and song as part of the search string, but no difference.

The original $mech->get() has artist and song, if you know of any artist/song that works, maybe that would give me a hint...(or maybe not...)
  • Comment on Re: Re: suck on www::mechanize question

Replies are listed 'Best First'.
Re: Re: Re: suck on www::mechanize question
by leira (Monk) on Mar 01, 2004 at 07:33 UTC
    OK, further investigation suggests that it's not one bad link, but that the server just sometimes returns a 500 error. Other times it succeeds. If I run my script several times, it will sometimes succeed and sometimes fail -- and I get the same results if I try to follow the link several times in my browser.

    I was able to get around it like this:

    my $maxtries = 10; my $i = 1; while ($i <= $maxtries) { $mech->follow_link(text => "Hey Jude (lennon/mccartney)", n => "1" +); last if $mech->success; $mech->back(); $i++; } $mech->success() or die "Can't find song page\n";

    Of course, you can set $maxtries to whatever you think is prudent, and you can put in a sleep() in the loop if you think that might help.


      Thanks again for the help

      I tried your string 200 times w/o success and it works every time in the browser...Don't know what I could be doing wrong...

      I don't think it is the lyrics web site as the first ->get() works, it is just the 2nd one that fails...

      The link that failes in www:mechanize is:

      This URL works in my browser always...

      Here is my complete test program incase someone else wants to give it a try ;-)

      use WWW::Mechanize; use URI::URL; use strict; use warnings; my $artist = 'The Beatles'; my $title = 'Hey Jude'; my $mech = WWW::Mechanize->new(autocheck=>1); #$mech->agent_alias('Windows IE 6'); my $search = join("+", split(/ /, $artist)) . "+" . join("+", split(/ +/, $title)); print "search=" . "$search" . + "\n"; $mech->get("$search"); $mech->success() or die "Can't get the search page\n"; $mech->follow_link(text=>"Hey Jude"); #$mech->follow_link(n=>6); $mech->success() or die "Can't find lyrics page\n";
        Your script works for me as often as following the like works for me in a browser.

        I'd suggest that you try to get some more information about what's going on. Instead of just die-ing after a failure, you might try printing out $mech->status and/or $mech->content to see if that sheds any light on what's going on.

        Sorry I can't be of more help.


Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://332809]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others imbibing at the Monastery: (7)
As of 2018-06-19 16:58 GMT
Find Nodes?
    Voting Booth?
    Should cpanminus be part of the standard Perl release?

    Results (114 votes). Check out past polls.