Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Questions on Optimisation (DateTime::SpanSet and RRDs.pm)

by smullis (Pilgrim)
on Jan 26, 2005 at 13:28 UTC ( #425201=perlquestion: print w/ replies, xml ) Need Help??
smullis has asked for the wisdom of the Perl Monks concerning the following question:

Hello All...


I have a simple script that calculates the average values from a number of RRD files. I would now like to discard all data returned by RRDs::fetch that does not fall within working hours.


So, after some reading - and some trial and error - the following works:
use strict; use warnings; use DateTime::Event::Recurrence; use DateTime::SpanSet; use RRDs; # # Set up paths, config data etc... # foreach $file ( @array_of_RRD_filenames ) { my ( $metric, $interval, $begin, $end ) = ( 'AVERAGE', '1800', '-800h', 'now' ); my $start_span = DateTime::Event::Recurrence->weekly( days => [ 1 .. 5 ], hours = +> 8 ); my $end_span = DateTime::Event::Recurrence->weekly( days => [ 1 .. 5 ], hours = +> 18 ); my $span_set = DateTime::SpanSet->from_sets( start_set => $start_s +pan, end_set => $end_spa +n ); push @opts, $file; push @opts, $metric; push @opts, "-r $interval"; push @opts, "-s $begin"; push @opts, "-e $end"; my ( $start, $step, $names, $data ) = RRDs::fetch @opts;
For those unfamiliar with RRDs.pm, $data is an AoA ref.
my ( $intotal, $outtotal, $i ); foreach my $value ( @{ $data } ) { $start += $step; my ( $minute, $hour, $day, $month, $year ) = ( localtime( $start ) )[ 1, 2, 3, 4, 5 ]; $month += 1; # Why does localtime give month -1? my $dt = DateTime->new( year => $year, month => $month, day => $day, hour => $hour, minute => $minute ); next unless $span_set->contains( $dt ); $i++; $in_total += $value->[ 0 ]; $out_total += $value->[ 1 ]; } # Now go off and work out averages / percentages etc... # Build a HoH and exit the for loop. } # Output some purty HTML.

The problem is, this is incredibly slow! I've qualitatively isolated the major slowdown to the "next unless $span_set->contains ( $dt )" -- this slows things down by a factor of 5.

dprofpp just segfaults when I try to profile the script.

Anyone have any suggestions as to how to optimise this? Is there another way rather than using Date::Time?


Many thanks in advance for any pointers / help / critisicism / suggestions.

Cheers

SM

Comment on Questions on Optimisation (DateTime::SpanSet and RRDs.pm)
Select or Download Code
Re: Questions on Optimisation (DateTime::SpanSet and RRDs.pm)
by edoc (Chaplain) on Jan 26, 2005 at 13:48 UTC

    Does this do what you need? If so, I'm guessing it should do a bit less work than DateTime would..

    my ( $minute, $hour, $day, $month, $year, $wday ) = ( localtime( $start ) )[ 1, 2, 3, 4, 5, 6 ]; next unless (($wday >= 1) && ($wday <= 5) && ($hour >= 8) && ($hour < 18));

    cheers,

    J

      Thankyou!

      This is exactly the kind of answer I was looking for...

      Cheers

      SM

Re: Questions on Optimisation (DateTime::SpanSet and RRDs.pm)
by fglock (Vicar) on Jan 27, 2005 at 11:49 UTC
    $span_set->contains( $dt );

    Actually this method was not optimized for operating on simple datetimes. The sentence is executed as:

    $span_set->contains( DateTime::Set->from_datetimes( dates => [ $dt ] ) );

    I'm working in the optimization.

    update: DateTime::Event::Recurrence was uploaded to CPAN as version 0.19_01. If you benchmark it, please let me know.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://425201]
Approved by neniro
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others chanting in the Monastery: (7)
As of 2014-12-25 22:17 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Is guessing a good strategy for surviving in the IT business?





    Results (163 votes), past polls