http://www.perlmonks.org?node_id=883070

tsk1979 has asked for the wisdom of the Perl Monks concerning the following question:

I have already looked at Date:Manip, and its very powerful, as well as slow. MY requirement is a bit simple. I am writing a gpx file, and every trkpnt entry requires a date time in the following format
<time>2011-01-01T00:00:03Z</time>
I will be writing track points inside a loop, and in every entry I will be incrementing the time by 1 hour 2 minutes and 3 seconds. I am planning to write over a 1000 trackpoints. Since I do not require all the complications of Date:Manip, is there something simpler I can use A typical algo I will do
for (i from 1 to 1000) print <header> print "$date"; $date = $date + "01:02:33"; endfor

Replies are listed 'Best First'.
Re: Simple date and time manipulation
by Corion (Patriarch) on Jan 19, 2011 at 09:31 UTC

    I would look at Time::Local as an alternative - but I haven't benchmarked any of the relevant modules. Time::Local talks about caching, so maybe it is faster, or maybe that:s just an indication of how slow it was in the beginning.

    To generate the initial timestamp, use Time::Local::timelocal to convert your date into a unix epoch timestamp. Then use localtime to (re)create the timestamp. Add the equivalent of 1 hour, 2 minutes and 3 seconds in seconds to your timestamp and redo. At least if you don't have to worry about daylight savings time adjustments, this will work. If you cross such adjustment boundaries, you will need to think long and hard about how they will affect your output.

      The UNIX epoch timestamp, being the number of seconds elapsed since midnight Coordinated Universal Time (UTC) of January 1, 1970, not counting leap seconds has no problems with daylight savings time adjustments. That is something you will have to look into only at the level of printing "human readable" output for a particular time-zone.

      The internal format is therefore best left and stored as a timestamp and converted into whatever you need upon preparing the output.

      The docs at DateTime say:

      use UTC for all calculations

      If you do care about time zones (particularly DST) or leap seconds, try to use non-UTC time zones for presentation and user input only. Convert to UTC immediately and convert back to the local time zone for presentation:

      Arguably DateTime is the best for making such conversions, but yes it is a heavy and relatively slow module, but you pay that price only once at start-up.

      CountZero

      A program should be light and agile, its subroutines connected like a string of pearls. The spirit and intent of the program should be retained throughout. There should be neither too little or too much, neither needless loops nor useless variables, neither lack of structure nor overwhelming rigidity." - The Tao of Programming, 4.1 - Geoffrey James

      Ok, I went through the replies just now, and did some google also. For my need I actually do not need to use any modules. I have a very simple requirement! Here is the code I wrote. In this I have put incrtime as 1 sec, but I can put any number of seconds I want, so no issues there!
      #!/usr/bin/perl use strict; my $lat = 23.438; my $slon= 68.000; my $elon= 90.000; my $clon = $slon; my $incrlon= 0.01; my $time = time; my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime($ +time) ; my $incrtime=1; my $printmon; my $printyear; open OUTFILE,">tropic_of_cancer.gpx" or die \"Cannot open file for wri +ting\n"; print \"Printing Header...\n"; print OUTFILE "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"n +o\"?>\n"; print OUTFILE "<gpx version=\"1.1\" creator=\"MakeTropic\" xmlns=\"htt +p://www.topografix.com/GPX/1/1\" xmlns:xsi=\"http://www.w3.org/2001/X +MLSchema-instance\" xsi:schemaLocation=\"http://www.topografix.com/GP +X/1/1 http://www.topografix.com/GPX/1/1/gpx.xsd\">\n"; print OUTFILE "<trk>\n"; print OUTFILE " <name>Tropic of Cancer</name>\n"; while ($clon < $elon) { $printmon = $mon+1; $printyear = $year+1900; print OUTFILE " <trkseg>\n"; printf OUTFILE (" <trkpt lat=\"%.3f\" lon=\"%.3f\">\n", +$lat,$clon); print OUTFILE " <time>$printyear-$printmon-$mday"." +T"."$hour:$min:$sec"."Z"."</time>\n"; print OUTFILE " </trkpt>\n"; $clon=$clon+$incrlon; $time=$time+$incrtime; ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime( +$time) ; } print OUTFILE " </trkseg>\n"; print OUTFILE " </trk>\n"; print OUTFILE "</gpx>"; close OUTFILE;
      It works fast and is tiny.
        It works fast and is tiny.
        I beg to differ. It is both slow and verbose. DateTime is 10 times faster, and Date::Calc is 700 times faster! It is a good habit to Benchmark your code before making bold claims about its performance.
        --
        No matter how great and destructive your problems may seem now, remember, you've probably only seen the tip of them. [1]
Re: Simple date and time manipulation
by andreas1234567 (Vicar) on Jan 19, 2011 at 09:57 UTC
    It depends on your definition of "slow". I can create 10000 timestamps in ~3 seconds (Increased from 1000 to 10000 for more reliable count) with DateTime, which is also considered "slow" by some.
    use strict; use warnings; use DateTime; use Benchmark qw(timethis); my $dt = DateTime->new( year => 2011, month => 1, day => 1, hour => 0, minute => 0, second => 0, ); timethis( 10000, sub { print $dt->ymd() . ' ' . $dt->hms; $dt->add(hours => 1, minutes => 2, seconds => 3); } ); timethis( 10000, sub { $dt->add(hours => 1, minutes => 2, seconds => 3); } ); __END__ # Calc + print timethis 10000: 3 wallclock secs ( 3.09 usr + 0.06 sys = 3.15 CPU) +@ 3174.60/s (n=10000) # Just calc: timethis 10000: 3 wallclock secs ( 2.95 usr + 0.00 sys = 2.95 CPU) +@ 3389.83/s (n=10000)
    If you are really in a hurry, the general consensus is to use Date::Calc. However, with it you would have to do more of the sprintf() formatting yourself.
    use strict; use warnings; use DateTime; use Date::Calc qw (Add_Delta_DHMS); use Benchmark qw(cmpthese); my $dt = DateTime->new( year => 2011, month => 1, day => 1, hour => 0, minute => 0, second => 0, ); my ($year, $month, $day, $hour, $min, $sec) = (2011, 1, 1, 0, 0, 0); cmpthese( -1, { 'DateTime' => sub { print $dt->ymd() . ' ' . $dt->hms; $dt->add(hours => 1, minutes => 2, seconds => 3); }, 'Date::Calc' => sub { print $year . '-' . $month . '-' . $day . ' ' . $hour . '-' . $m +in . '-' . $sec; ($year, $month, $day, $hour, $min, $sec) = Add_Delta_DHMS($year, $month, $day, $hour, $min, $sec, 0, 1, 2 +, 3); }, } ); __END__ $ perl -l 883070.pl | tail -5 2011-07-18 10:20:15 2011-07-18 11:22:18 Rate DateTime Date::Calc DateTime 3258/s -- -99% Date::Calc 481882/s 14690% -- $
    Updated Wed Jan 19 11:12:58 CET 2011 with Date::Calc example.
    --
    No matter how great and destructive your problems may seem now, remember, you've probably only seen the tip of them. [1]
Re: Simple date and time manipulation
by Utilitarian (Vicar) on Jan 19, 2011 at 09:57 UTC
    I would suggest keeping the time in epoch time and incrementing by (((60 * 1) + 22 ) *60 ) + 33) seconds each iteration. you could then write an ISO_date_format function to return the value of date in the preferred time format, eg.
    sub ISO_date_format{ my $epoch_date = shift; my @localtime = localtime($epoch_date); my $iso_date = sprintf ("%4d-%0.2d-%0.2d", $localtime[5] + 1900, $localtime[4]+ 1, $localtime[3]); return ($iso_date); }
    This could of course be shortened for less clarity

    print "Good ",qw(night morning afternoon evening)[(localtime)[2]/6]," fellow monks."
      Since you're using UTC, you don't have to worry about DST, but leap seconds may still be a consideration. You don't say if this is a one-shot program, nor whether the times may be in the future. With leap seconds, future times cannot be accurately predicted. If you're not worried about leap-seconds, just adding the appropriate offset as Utilitarian suggests is probably your best bet.
Re: Simple date and time manipulation
by sundialsvc4 (Abbot) on Jan 19, 2011 at 13:17 UTC

    I would cordially argue that your reasoning (that it is “slow”) is quite un-sustainable.   And it would remain so, even if you had a billion trackpoints to write.

    This program is, and always will be, I/O bound.   Its completion rate will always be wholly and completely driven by the speed and efficiency of the input/output hardware.   Meanwhile, I am quite sure that it could be demonstrated that the CPU could complete one billion date/time calculations in far less than one second’s time ... and I am being generous in my estimation.

    Write the computational code in the simplest, clearest, and most-general way that you can.

    “Don’t diddle the code to make it faster.”
    – The Elements of Programming Style

      To expand a bit on BrowserUk's terse reply:

      Our target is to execute 1 billion such calculations in one second.

      Let's assume that "the CPU" can execute 1 date/time calculation per time step.

      We will need to execute 1 billion CPU time steps within one second. That is 1_000_000_000_000 time steps per second (burst, not sustained, mind you). So our CPU needs to run at 1000 GHz to meet that, never mind the overhead of fetching/storing the program or results.

      As 1000 GHz is still three magnitudes away and chip makers seem to have given up on the race of raw GHz, likely due to physical limits, maybe we can assume that "the CPU" has more than one core. "Mainline" CPUs (AMD or Intel) have something like 8 or 16 execution units per die, so these would need to run at 1000/16 GHz, still a bit far off. GPUs, albeit very specialized vector machines, are claimed to have "up to 512" CPUs (well, stream processors) in their marketing brochures (NVidia Geforce 500), but they only run at 750 MHz, or only 75% of what we need to come close to that claim.

      If you look at these numbers, and the simplifications I've made, I think you'll concede that your formulation of "one billion date/time calculations in far less than one second" is still a bit away in the future.

        Indeed. And that's if you wrote in machine code.

        But my proof is a little simpler, and more germane to the OPs situation. 10e7 takes three seconds, and that's just returning a constant, albeit not an optimised away one.

        C:\test>perl -MTime::HiRes=time -E"sub t{1}; $t=time; $x=t()for 1..1e7; printf qq[%f\n], time()-$t" 3.305000

        Not hard to verify.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.
      I am quite sure that it could be demonstrated that the CPU could complete one billion date/time calculations in far less than one second’s time

      Prove it!.