Re: Perl/CGI Performance for a Shopping Cart
by gav^ (Curate) on Jul 30, 2002 at 10:42 UTC
|
This might sound like a harsh reply and it is meant to be.
What the world does not need is yet another PERL/CGI (sic) programmer
churning out yet another poorly designed shopping cart script. It is
obvious to me that from Sasquire's post that he does not understand
some of the important issues and will be doing his clients a disservice.
A shopping cart is not a simple thing. It definatly isn't something that
a beginning perl programmer should be attempting in any other context
than a learning one. There are some big issues at stake here:
Performance: How does the app scale? Can it handle peak loads?
Talking about '$15p/m' hosting worries me. You won't nearly have the
level of control you need.
Security: Is it something I can trust? How are credit card details stored?
Can something go wrong and they end up web accessible? What about
other customer data? Does the script make any of the obvious mistakes
like putting prices in hidden fields?
Stability: Is it going to be up 24x7? Is there any degree of
redundancy? Is there a plan for failure (ISP going under, hardware dying, etc)?
Are backups done? How secure are these?
Customization: How easy is it to respond to requirement changes?
Can the client modify templates? What happens when they want to add
features like cross selling? Is the tool powerful enough to handle changes?
For these reasons I would never recommend that a small client bases their
business on something written by a somebody who isn't an 'expert'. This
is why a platform such as Yahoo Store
is a win-win situation for somebody who doesn't want to spend a lot of money.
For more advanced stores there are tools such as such as
Interchange and
Intershop.
gav^ | [reply] [Watch: Dir/Any] |
Re: Perl/CGI Performance for a Shopping Cart
by dws (Chancellor) on Jul 30, 2002 at 07:58 UTC
|
So I was wondering...
You ask a number of questions to which the proper answer is some variation on "it depends". For high volumes of traffic, you don't want to have to start up a new process external to the web server for each hit. That rules out CGI, though mod_perl is certainly a candidate. Or, you could arrange to run a separate shopping cart process that a thin web servlet (ASP, PHP, mod_perl, whatever) could delegate to via socket. I've seen setups like that handle high data rates, though they're more complicated to construct.
If your database is 500K, you could benefit from either keeping it in RAM, or loading it into MySQL. (Loading data into MySQL is left as a (simple) exercise.)
If you've already designed the shopping cart as a CGI, you have a good basis for experimentation. See how it scales against a 500K CSV on risk, then try moving it to RAM, then load it into MySQL. With a bit of stress testing, you should be able to make predictions on how the cart will scale.
If you're going to be renting your cart out, you'll also need to consider availablity, which leads to load balancing, which leads to more hardware and a bit more complexity in the software.
perrin has a good writeup on how eToys handled huge loads. See Building a Large-scale E-commerce Site with Apache and mod_perl.
| [reply] [Watch: Dir/Any] |
Re: Perl/CGI Performance
by ajt (Prior) on Jul 30, 2002 at 08:30 UTC
|
Tough question....
You have several things mixed up, and I think it's best to separate them out.
- CGI
- Common Gateway Interface is a standard way that web servers can run programs, collecting input from web-forms, and returning results to the browser. It is slow but stable, and almost all servers support it. Most people use Perl when using CGI, but you do not have to, you can use virtually anything you want, e.g. BASIC, shell or c.
- In-Process / embedded architectures
- There are several embbedded technologies for web servers. Here the language interpretor is embedded into the web server, which makes it much faster, but it's more complex to use. Examples include ASP (where you can use one of several languages, e.g. Visual BASIC or Perl), and some with only one language, e.g. PHP/PHP, JSP/Java, mod_Perl/Perl.
- Perl
- Perl is a high-level language which is fast to develop in, and can run as a CGI script on a wide range of web servers and a wide range of operating systems. It can also be made to run much faster by using ASP or the mod_Perl interface. The CPAN library of pre-written code contains code to do most things you could think off - no other language has anything on this scale...
Returning to your questions. CGI/Perl is not the fastest way of doing things, but it's pretty good for low to medium volumes. If you have written the code well, it can probably be converted from CGI to mod_Perl, in which case it will run much faster.
There are many advantages of using a DB for storing data, and it should be easy to export data using Perl into any simple output fromat. Proper database systems have all the advantages of file-locking, multi-user capability, and potentially higher speeds. Examples include: MySQL, PostgreSQL or even DBD::SQLite all should be easy to use from Perl.
A basic Linux/Apache/Perl box with a 1GHz+ processor and 1Gb RAM, will run very quickly, and be adequate for most needs, but cost more than USD15/month. You may find that at your suggested price range your options are more limited.
One word of caution, you say you are a novice, in which case I would urge extreme caution if you are going to be moving money about. Web security is easy to get wrong, and you may be better off using code that someone else has written, and has been scrutinised by the public in general. I don't claim to know of such software.
Additional links to read:
Update: Extra Links added, minior typographical changes and corrections.
ajt
| [reply] [Watch: Dir/Any] |
Re: Perl/CGI Performance
by IlyaM (Parson) on Jul 30, 2002 at 08:22 UTC
|
| [reply] [Watch: Dir/Any] |
Re: PERL/CGI Performance
by Ryszard (Priest) on Jul 30, 2002 at 08:25 UTC
|
The performance of your site isnt *only* related to perl. We're talking network latency, machine speed, ram, IO, disk speed et al. Much of the overhead with perl CGI's is related to brining up the perl interpreter. (one instance per CGI). Check out mod_perl, it is apache with the perl binary build into it at compile time (Only have to load perl when you start apache).
DONT USE FLAT FILES!, I'm thinking of several reasons:
- Integrity - you may trash your "database" via a fatfingers (or late night) typo, very easy to do if you delete where you should have inserted.
- Normalisation - much easier to do in an RDBMS than using flat files. If you're unsure as to what normalisation is, i reckon it could be a good research topic.. :-)
- Scalability - It is much easier to preserve normalisation and extend your database using SQL than flatfiles.
- Data integrity - Foreign key constraints. Enforcement of primary key, data typing, all native to an RDBMS you "should" use..
Why use perl over ASP? Microsoft.. :-) completly irrational argument. TBH both may do the job, both platforms have quirks and the performance is highly dependant on application design. I'd choose perl as its easy, lots of support (you posted the question here didnt you?), and portable, (you can run it on solaris, linux, windows...). Also means your choice of host provider is not as limited.
What makes MySQL better than others? Not much IMO. The latest stable release doesnt support transactions (Postgres, Oracle does). Both Postgres and MySQL are free, both have heaps of support... If youre leaning toward more of an "Enterprise" solution, Postgres may be the way to go as it is transaction based. If you're not particularly worried about this feature, MySQL might be the way to go.
Of course if you have lots of cash, i'd say Oracle, which would probably be complete overkill for a small(ish) application.. :-)
Anyways, good luck.
Update: Whoops, i stand corrected on my ASP comments after reading IlyaM's post.
| [reply] [Watch: Dir/Any] |
|
Integrity - you may trash your "database" via a fatfingers (or late night) typo, very easy to do if you delete where you should have inserted.
I've heard so many horror stories about late night mistakes like forgetting about WHERE part of UPDATE or DELETE queries that I don't think that SQL database is any safer than filesystem in this respect. In reallity no matter what database backend you use backups are must have for important data.
The latest stable release doesnt support transactions
Not true (for quite long time) with InnoDB table type which is part of source MySQL distro and part of MySQL-Max binary distro.
--
Ilya Martynov
(http://martynov.org/)
| [reply] [Watch: Dir/Any] |
|
I've heard so many horror stories about late night mistakes like forgetting about WHERE part of UPDATE or DELETE queries that I don't think that SQL database is any safer than filesystem in this respect. In reallity no matter what database backend you use backups are must have for important data.
Jepp! Two years ago I damaged all ongoing projects of my company from home by forgetting a where-clause; the backups were not @home :-)
pure luck I wasn't killed in a car-crash that day...while we hurried to the companys rooms
So while typos aren't an argument for DBMS :-), an rdbms is the way to go if scaleabilty matters, and it matters for every project not typed at the console-prompt (and somtimes even there...)
regards,
tomte
| [reply] [Watch: Dir/Any] |
|
Re: Perl/CGI Performance for a Shopping Cart
by anolte (Novice) on Jul 30, 2002 at 16:03 UTC
|
What really draws on performance, if you spawn a new perl interpreter for every request - so you should use s.th. like mod_perl under apache for this.
If you want to recycle some code: look at a toolkit like openinteract ( sourceforge.net/projects/openinteract ), embperl or mason etc. Here you usually also find pointers to mod_perl etc.
Andreas | [reply] [Watch: Dir/Any] |
Re: Perl/CGI Performance for a Shopping Cart
by Spenser (Friar) on Aug 04, 2002 at 17:58 UTC
|
I used to use Perl with flat database files because I thought it was the simplest and best way to go. Then I discovered mySQL--this reads like a laundry detergent commercial. Anyway, mySQL has worked out much better for me. It does take a bit of an effort to get started, but it's not too bad with the proper books and some patience. As for its performance compared to other databases, it ranks as high as Oracle these days, according to e-week.
Regarding your concerns about exporting to delimited text files, that's a breeze with mySQL and Perl and the Perl DBI module. As with everything else with Perl, there are many ways of doing it, but below is a simple overview of an export program. This export program is assuming that you've already set up a database and table in mySQL, of course.
#!/usr/bin/perl -w
use DBI;
# Connect to the mySQL server and
# get today's orders.
my $dbh = DBI->connect("DBI:mysql:orders:localhost","user","passwd")
|| die "Could not connect:".DBI->errstr;
my $sql_stmnt = "SELECT order_id, item_no, qty, price
FROM orders WHERE order_date=CURRENT_DATE'";
my $sth = $dbh->prepare($sql_stmnt);
$sth->execute();
while(@orders = $sth->fetchrow_array()) {
$order_id = $orders[0];
$item_no = $orders[1];
$qty = $orders[2];
$price = $orders[3];
# Create record with bars as delimiters
# and push record into array for exporting.
$order = $order_id . "|" . $item_no . "|" . $qty . "|" . $price . "
+\n";
push(@export, $order);
}
$sth->finish();
$dbh->disconnect();
# Now loop through array of orders
# and save to text file.
open(EXP, ">client-orders.txt")
|| die "Could not open export file: $!";
foreach(@export){ print EXP; }
close(EXP);
This bit of code is a lot wordier than it needs to be and can be consolidated into few lines. However, when starting out I think its better to spread the code out to keep your sanity and to make learning and trouble-shooting easier.
-Spenser
That's Spenser, with an "s" like the detective. | [reply] [Watch: Dir/Any] [d/l] |