Re: How to extract links from a webpage and store them in a mysql database

[g0n]

Hi syedahmed.uos,

Running your code and passing it a single URL, I get a single row in the table, with the base in column 'webpage', and a space separated set of fully qualified links in the 'links' column.

You say that you want to have each link in a separate row of the table. To do that, you'll need to iterate over the list of links. Something like this:

for my $link (@a)
{
  $dbh->do("INSERT INTO htmllinks VALUES ('$base',' $link')");
}

Just making that change won't work because you've set 'webpage' to be the primary key, so you'll get an error because the value for 'webpage' is the same for every row. If you take the 'primary key' out of your table create, and put the loop in as above, you'll get something like this:

+---------------------------------------------------------+
| webpage               | htmllinks                       |
+---------------------------------------------------------+
|http://myurl.com       |   http://myurl.com/link_one/    |
|http://myurl.com       |   http://myurl.com/link_two/    |
+---------------------------------------------------------+

Is that what you're looking for?