is worth a look, for design inspiration if not for direct use. It's use of column classes, particularly the Essential class, is aimed at solving some of the problems you're concerned with. You designate a set of columns as "essential" and they're the ones that are loaded every time you load an object from that class. Other fields, like the big ones that are edited rarely, are demand-loaded only when you access them.
That said, Class::DBI is not a very efficient database user, so if performance is something you need it's not likely to be a good fit. I've heard good things about Rose::DB, particularly in terms of performance, although I can't speak from experience.