A new challenge: game rankings

[Anonymous Monk]

Hey there... I am a beginner Perl programmer, but have been told that i have make a rankings list for an online game. So heres the problem. Theres ~40,000 players in this game and they are currently sorted by team, not by score, on a different cgi page for each team. This page of each is of the form http://blahblah.com/scores.cgi&teampage=X where x is the number of the team (integer). Then on each of these pages is a list of the team members in no particular order. Each listing consists of their nickname, then 3 different statistics about the player in the form:
http://blahblah.com/scores.cgi&teampage=1
Name A        B          C
John 1000 175320 950
Mary 2300 425321 1550
Bob 1324 211123 850

http://blahblah.com/scores.cgi&teampage=2
Name A        B          C
Steve 1855 310123 1100
Dave 3271 460456 1300
Joan 1999 375789 1000

(Those were numbers I made up, but thats just how it is)
Now, I have already worked out that i want to have the last team page as a constant then run an loop to increment the team page num until it = the last one. This way it can be modified if theres a different number of teams next round. Then get the contents of each page and store it somewhere. Then what needs to be done is the stuff sorted into order and stored somewhere long term (like a database). I originally thought of just a hash table but then realised that it would be VERY slow. Anyway, once sorted, the results need to be printed to a webpage (easy). Also, users have to be able to search for anyone in the game by name, stat A, stat B or stat C. For example of someone wanted to search all players that have statistic A between 1000 and 1200. It would only display these ppl. Any ideas?

[repson]

You need to first break this problem into steps:
• Get the data: For this you probably want to look at LWP::Simple and then use loops and regex to parse each page.
• Store it: Use DBI for any database you like (under unix mysql is a decent and free system).
• Receive querys: CGI.pm is the way :)
• Look at the data: Go back to your database in step 2, you may need to learn some more SQL if you don't know it.
• Display results: Something like HTML::Template would be nice, but there are many other solutions for this step.
See what you can work out for those and then maybe come asking for furthur help on specific items.

[BBQ]

Yes, a few ideas:
• use DBI;
• and MySQL
• do some reading
and come back with real questions... So far it seems you have the right idea, but no code will get you nowhere around here. You need to show some effort as well.

#!/home/bbq/bin/perl
# Trust no1!

[gharris]

Doesn't the CGI for the teamscores get its results from somewhere?
I would think it would be easier (and more accurate) to get the data from the source, rather than having to parse an interpretation of the data.
On the other hand, I know that that isn't always possible, in which case I would follow repson's approach.

--Glenn

[Grizzly]

No, Glenn, it doesnt. Basically the team pages are the src. They retrieve the score from the player info, something i don't have access too.

Thanks, repson, for ur response. I will look further into those things and let ya know...

BBQ: I didnt want to have code, I wanted a discussion of ideas, so i can write up my own code, not steal other ppl's. I pretty much wanted to know what i had to learn before i could advance any further. Repson did this.