We don't bite newbies here... much PerlMonks

### Script for a URL that constantly changes

by semrich (Initiate)
 on Oct 19, 2011 at 21:52 UTC Need Help??
semrich has asked for the wisdom of the Perl Monks concerning the following question:

Hey everybody, I'm new at Perl and I am trying to use this URL:

https://www.vwrsp.com/catalog/product/index.cgi?partnumber=$sku&sim_code=2.9&catalog_number=PAL1001&inE=1&highlight=PAL1001&reference_type=2 Where it says$sku, that will change constantly via a database, but I need a script that will change the "PAL1001" (catalog_number) each time $sku changes, so the$sku and catalog_number matches according to the site. Here is the script I used to insert the SKU I want from my database:

my @sku = ('CODE TO THE DATABASE')

Hopefully this makes sense. Thanks in advance!

Replies are listed 'Best First'.
Re: Script for a URL that constantly changes
by anneli (Pilgrim) on Oct 19, 2011 at 23:03 UTC

This is what I have so far:


use strict;
$|++; use Test::WWW::Mechanize; use WWW::Mechanize::Sleepy; use File::Basename; use WWW::Mechanize; use WWW::Mechanize::Image; use Storable; use HTTP::Cookies; use HTML::SimpleParse; my @sku = ('NUMBER'); for my$sku (@sku) {

# sleep between 5 and 20 seconds between requests
my $mech = WWW::Mechanize::Sleepy->new( sleep => '1..3' ); my$URL ="https://www.vwrsp.com/psearch/ControllerServlet.do?D=$sku&sp +age=header&CurSel=Ntt&Nty=1&Ntx=mode%2bmatchpartialmax&cntry=us&Ntk=A +ll&N=0&Ntt=$sku";

$mech->get($URL ); $mech->success or die$mech->response->status_line
+;
$mech->success or die "post failed: "; my$url1= $mech->uri(); print "$url1\n";

my @links = $mech->find_all_links( url_regex => qr/\/catalog\/product/ +i); for my$links (@links) {

$mech->get($links->url() );
$mech->success or die$mech->response->status_line;
$mech->success or die "post failed: "; my$pike = "|";
open (price_file, "FILE NAME") || die "can't open price.txt: $!\n"; my$some_html = $mech-> content(); my$p = new HTML::SimpleParse ($mech); print$p;
[download]

What happens on the site is, you go to the URL listed above to get a list that matches the SKU fed to it, then from there each listing has it's own page where it holds all the information you see on the initial page (where the list is). When you go to that second page, in the URL there is a catalog number, that changes depending on which listing you clicked on (which is why I am not using that URL to parse), and I need to somehow capture that catalog number each time the SKU changes so I can get the correct information from the "second" page. I am not sure if that made any more sense. It is sort of complicated to explain.

Re: Script for a URL that constantly changes
by pvaldes (Chaplain) on Oct 19, 2011 at 23:57 UTC
I'm trying to use this URL

Define use...

Where it says $sku, that will change constantly via a database Then you probably want to interact with the database in any way: use DBI my @sku = ('CODE TO THE DATABASE') Good try... my %code = 'CODE TO SOLVE YOUR PROBLEM HERE'; ;-) Aha, question updated with some real code, let's see... looks like a work for a hash... my @sku = ('NUMBER') is not what you want probably; first of all because @ mean a list of elements, not a single number, and also because a %hash is much better here. If you define @sku as an array you could have by mistake the same number for two products (and this is a BIG problem), but a hash will not permit you to do this. The number$sku is the key (unique) and the catalog ref is the value.

so the first thing that you should try to fix is these two lines.

my @sku = ('NUMBER');
for my $sku (@sku) {... [download] Instead connect to the database and prepare something like: select field2 for mytable where field1 = ? and fill the gap with the correct number sku with execute($sku). Read the manual of DBI for the details

finally when you have your hash you could write something like this

https...partnumber=$sku...catalog_number=$hash{$sku}&inE=1&highlight... I have it connecting to the database like this: use DBI; use DBD::ODBC; my$conn_string = "driver={SQL Server};Server=SERVER NAME;Database=DB
+NAME";
my $dbh = DBI->connect( "DBI:ODBC:$conn_string" ) or die $DBI::errstr; # get each SKU my ($sth, $stmt ); my$count=0;
$stmt = qq { select sku from TABLE NAME };$sth = $dbh->prepare ($stmt);
\$sth->execute ();
[download]
I believe this is working, because I did a trial run of it. Do you have a suggestion as to where I can look up how to write a hash? Thank you for all your help!

Create A New User
Node Status?
node history
Node Type: perlquestion [id://932516]
Approved by keszler
help
Chatterbox?
 [choroba]: needed everywhere, available rarely [LanX]: Question: starting a script from an icon in Windows, but after C-c the window closes ... what I want is to restart in the cmd.exe. Recommendations? [Corion]: LanX: What do you mean by "restart in the cmd.exe" ? [Corion]: Do you want to launch a script and keep the command prompt/console window open? [Corion]: Do you want to wait for a key press before closing the window? [LanX]: I want the command line in the history [tye]: -Mouse [Corion]: Option a) would mean launching cmd.exe /k c:\path\to\ batchfile- launching-perl- script.cmd. Option b) would be to add pause as the last line of said batch file.

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (12)
As of 2017-03-27 15:42 GMT
Sections?
Information?
Find Nodes?
Leftovers?
Voting Booth?
Should Pluto Get Its Planethood Back?

Results (320 votes). Check out past polls.