Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation
 
PerlMonks  

How to extract links from a webpage and store them in a mysql database

by syedahmed.uos (Novice)
on Nov 27, 2006 at 18:59 UTC ( [id://586312]=perlquestion: print w/replies, xml ) Need Help??

syedahmed.uos has asked for the wisdom of the Perl Monks concerning the following question:

This node falls below the community's threshold of quality. You may see it by logging in.
  • Comment on How to extract links from a webpage and store them in a mysql database
  • Download Code

Replies are listed 'Best First'.
Re: How to extract links from a webpage and store them in a mysql database
by g0n (Priest) on Dec 05, 2006 at 12:53 UTC
    Hi syedahmed.uos,

    Running your code and passing it a single URL, I get a single row in the table, with the base in column 'webpage', and a space separated set of fully qualified links in the 'links' column.

    You say that you want to have each link in a separate row of the table. To do that, you'll need to iterate over the list of links. Something like this:

    for my $link (@a) { $dbh->do("INSERT INTO htmllinks VALUES ('$base',' $link')"); }

    Just making that change won't work because you've set 'webpage' to be the primary key, so you'll get an error because the value for 'webpage' is the same for every row. If you take the 'primary key' out of your table create, and put the loop in as above, you'll get something like this:

    +---------------------------------------------------------+ | webpage | htmllinks | +---------------------------------------------------------+ |http://myurl.com | http://myurl.com/link_one/ | |http://myurl.com | http://myurl.com/link_two/ | +---------------------------------------------------------+

    Is that what you're looking for?

    --------------------------------------------------------------

    "If there is such a phenomenon as absolute evil, it consists in treating another human being as a thing."
    John Brunner, "The Shockwave Rider".

    A reply falls below the community's threshold of quality. You may see it by logging in.
Re: How to extract links from a webpage and store them in a mysql database
by Melly (Chaplain) on Dec 05, 2006 at 12:07 UTC

    Hi syedahmed.uos - you'd be better off starting your own question, rather than hijacking another.

    This isn't a moral judgement, btw, it's just that not many people will notice your question buried away in here.

    map{$a=1-$_/10;map{$d=$a;$e=$b=$_/20-2;map{($d,$e)=(2*$d*$e+$a,$e**2 -$d**2+$b);$c=$d**2+$e**2>4?$d=8:_}1..50;print$c}0..59;print$/}0..20
    Tom Melly, pm@tomandlu.co.uk
      u r right, i didnt know, this was my first posting and after i looked at this post it matched nearly for what i was looking for and hence i have written down my question in reply to it. it would be so kind of you if you could tell me how to post a question on my own without hijacking others A......

        Go to Seekers of Perl Wisdom and scroll to the bottom of the page - you should find an "Add your own question" section.

        BTW welcome to perlmonks...

        map{$a=1-$_/10;map{$d=$a;$e=$b=$_/20-2;map{($d,$e)=(2*$d*$e+$a,$e**2 -$d**2+$b);$c=$d**2+$e**2>4?$d=8:_}1..50;print$c}0..59;print$/}0..20
        Tom Melly, pm@tomandlu.co.uk

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://586312]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others learning in the Monastery: (4)
As of 2024-04-24 12:42 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found