Re^2: Finer points of Class::DBI

[dba]

You have not mentioned what is your database. If it is a RDBMS, then you could join the tables and order by areas. It will be much more simple and faster.

[jreades]

Yeah, some more details on the db architecture would be very useful since this seems like more of a query issue than anything else.

While I can envision a number of ways that an area would have a 1..n relationship with the statistics, I'm just going to have to guess at the field layout (note, I'm not even getting into normalisation):

--------------------  --------------------
| Areas            |  | Stats            |
--------------------  --------------------
| PersonId         |  | PersonId         |
| Person           |  | Person           |
| Group            |  | Stat             |
--------------------  | MonthKey         |
                      --------------------

Assuming that your setup looks anything like this, then there are several ways to query this table for an ordered list of stats.

If you wanted everything ordered by date (so no rollup) it would be:

SELECT monthKey, group, person, stat
FROM areas a, stats s
WHERE
    a.personId=s.personId
ORDER BY
    monthKey, group, person, stat
;

If you wanted to get a date range it would be:

SELECT monthKey, group, person, stat
FROM areas a, stats s
WHERE
    a.personId=s.personId
AND
    s.monthKey > 200401
AND
    s.monthKey < 200406
ORDER BY
    monthKey, group, person, stat
;

And so on.

The output from this query would be quite easy to work with in Perl because you can run through the output without worrying about order:

my $sth = $dbh->prepare($query);
$sth->execute();

my $current_month;
my $current_group;
my $current_person;

while (my $rv = $sth->fetchrow_arrayref());
   if ($rv[0] ne $current_monthkey) {
      # print out new month header
      $current_month = $rv[0];
      undef $current_group;
      undef $current_person;
   }
   if ($rv[1] ne $current_group) {
      # print out new group header
      $current_group = $rv[0];
      undef $current_person;
   }
   if ($rv[2] ne $current_person) {
      # print out new person header
      $current_person = $rv[1];
   }
   # print out stat
}

$sth->finish();
$dbh->disconnect();

Hope that helps -- it was a bit of a speculative jump based on the available information, but dbs are designed for querying, sorting, and ordering in clever ways that require you to be a true programming god to reproduce in Perl with anything like comparable performance.

[rlb3]

Hello thanks to all who helped. I guess I skipped over the section on limiting has_many in the docs. That was what I needed. Some of you have asked for the make up of my tables and which database I'm using.

Database: mysql 4.0.21 Tables:

CREATE TABLE areas (
  id int(11) NOT NULL auto_increment,
  short_name varchar(10) NOT NULL default '',
  long_name varchar(200) NOT NULL default '',
  PRIMARY KEY  (id),
  KEY short_name (short_name)
) TYPE=MyISAM;

CREATE TABLE stats (
  id bigint(20) unsigned NOT NULL auto_increment,
  aid int(11) NOT NULL default '0',
  uid int(11) NOT NULL default '0',
  login_name varchar(15) NOT NULL default '',
  full_name varchar(50) NOT NULL default '',
  cpu int(11) NOT NULL default '0',
  mem int(11) NOT NULL default '0',
  conn int(11) NOT NULL default '0',
  disk_blocks bigint(20) NOT NULL default '0',
  procs int(11) NOT NULL default '0',
  sess int(11) NOT NULL default '0',
  month int(11) NOT NULL default '0',
  year year(4) NOT NULL default '2004',
  PRIMARY KEY  (id)
) TYPE=MyISAM;


Again, thank you for your help.

rlb3

[cleverett]

Then he'd have to use Data::Grouper, or something like that to build his hierarchical format. Yes, using CDBI results in N+1 queries to the RDBMS, where N = the number of records returned by the initial Areas->retrieve_all(). However, the real glory of CDBI has to do with the amazing code reduction it achieves (especially in conjuction with Template Toolkit for example), not the performance.

Moral: use CDBI where reusability, portability and ease of coding take a priority over performance (almost everywhere). Use hand coded SQL and DBI when you need to sacrifice everything for speed.