Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

How do I synchronize data to common timestamp

by lewnewby (Novice)
on May 23, 2013 at 22:47 UTC ( #1035037=perlquestion: print w/ replies, xml ) Need Help??
lewnewby has asked for the wisdom of the Perl Monks concerning the following question:

Let me get this over up front, I am a relatively new perl user (Starting over after 15 years).

I have been working on extracting data from a management server via an API supplied by the vendor. So far that is all working great, I am able to get the data. My problem arises in that when I receive the data it will have some odd offset in the time stamps.

I make a call requesting data at the default sample rate of 60 seconds over a period of 1 day. The API returns the data in a sequence of Key:Value pairings for each counter I am retrieving. The first counter may have X secs:Value and proceed through at 60 second increments. Ok great so far, the second and forth counters may return to the top with the first and X is the time stamp but the third counter returns to the top and may be X+6 secs instead of X.

The data ends up looking like this:

2013-05-21 18:46:44,221.667,9.041,92.133,4.008,84.450,6.220,8.983,2.57 +0,5.917,6.710,87.067,6.133 2013-05-21 18:46:45,7.519,5.069 2013-05-21 18:47:44,33.617,2.817,70.900,2.931,161.000,9.811,8.233,2.28 +0,5.617,6.010,86.117,8.108,6.067,5.304 2013-05-21 18:48:44,26.417,1.758,38.933,2.550,42.900,3.526,5.750,1.515 +,4.867,6.132,86.433,6.067 2013-05-21 18:48:45,7.562,5.068 2013-05-21 18:49:44,42.683,2.308,33.633,2.208,42.983,3.807,6.033,1.751 +,4.900,6.042,87.300,8.118,5.983,5.207 2013-05-21 18:50:44,86.983,8.791,103.417,7.851,62.900,8.295,15.500,4.4 +21,8.300,9.368,87.800,7.697,6.050,5.202 2013-05-21 18:51:44,50.033,2.840,63.350,3.071,36.817,3.134,6.217,1.480 +,4.783,5.444,88.433,6.150 2013-05-21 18:51:45,7.725,5.275 2013-05-21 18:52:44,50.017,3.071,61.033,2.538,45.317,4.027,6.233,1.769 +,4.767,5.653,88.383,8.086,5.550,4.966 2013-05-21 18:53:44,38.850,2.265,43.100,2.051,42.683,3.795,5.883,1.403 +,4.850,5.667,89.700,6.000 2013-05-21 18:53:45,7.164,5.061 2013-05-21 18:54:44,22.000,1.741,42.817,2.352,41.700,3.430,5.700,1.543 +,5.567,5.756,87.517,7.939,6.300,5.188 2013-05-21 18:55:44,62.700,6.359,78.267,6.949,58.883,6.804,12.350,3.60 +1,8.200,9.379,87.550,7.349,6.017,5.114

I would like it to look like this:

2013-05-21 18:47:00,30.400,2.157,9.900,1.315,39.917,4.201,6.383,1.798, +5.167,7.250,129.950,7.965,5.667,4.892 2013-05-21 18:48:00,33.917,2.276,7.017,1.052,40.150,3.489,5.233,1.440, +4.900,6.393,129.850,8.257,6.033,5.191 2013-05-21 18:49:00,31.867,2.567,10.367,1.337,45.467,4.138,6.417,1.796 +,5.267,7.148,124.867,8.340,5.983,4.897 2013-05-21 18:50:00,57.533,2.943,8.800,1.063,41.867,3.701,5.083,1.445, +4.900,6.176,129.517,7.715,5.867,4.925 2013-05-21 18:51:00,72.900,10.128,103.833,6.560,66.317,6.734,14.933,4. +398,11.267,11.092,119.767,7.753,6.100,5.121 2013-05-21 18:52:00,23.950,1.803,7.217,1.460,50.667,3.845,5.183,1.492, +4.583,5.481,124.500,9.369,6.017,5.030 2013-05-21 18:53:00,25.500,1.910,9.767,1.754,49.467,4.112,6.133,1.772, +23.117,17.196,127.583,8.158,6.067,5.076 2013-05-21 18:54:00,47.350,2.344,8.833,1.419,47.283,3.875,5.400,1.409, +4.867,6.035,148.567,8.428,5.883,4.940 2013-05-21 18:55:00,29.067,2.338,10.400,1.789,45.067,3.250,6.233,1.795 +,5.100,5.866,126.517,7.990,6.133,5.219

Here is the code I am using to do this:

my ($list, $obj, $server_ctx) = @_; my @objList = @{ $list }; my (@counter_arr,$objName,@perf_cnt_data,%perfhash); my $gen_time_arr = 1; my $gen_hash_hdr = 1; my $obj_i = 0; my ($rec,$tmpHDR,$tmpVAL); if ($obj eq "system") { $obj = "host"; } $perfhash{"Time"} = ""; foreach my $perf_out ( @objList ){ my $instance = $perf_out->child_get("perf-instances"); my @instances = $instance->children_get("perf-instance-counter +-data"); foreach $rec (@instances){ my $obj_id = $rec->child_get_string("object-id"); my $IterStart = NaElement->new("$obj" . "-list-info-iter-s +tart"); $IterStart->child_add_string("object-name-or-id","$obj_id" +); my $output = $server_ctx->invoke_elem($IterStart); if ( $output -> results_status()=~/failed/ ) { print "\n\n\tFailed to get list of $obj: Error: ".$out +put->results_reason(). "\n\n"; } else { my $records = $output->child_get_int("records"); my $tag = $output->child_get_string("tag"); $output = $server_ctx->invoke("$obj" . "-list-info-ite +r-next", "maximum", $records, "tag", $tag); if ( $output -> results_status()=~/failed/ ) { print "\n\n\tFailed to iterate list of $obj : Erro +r: ".$output->results_reason(). "\n\n"; } } my $aggregates = $output->child_get("$obj" . "s"); my @aggregateArr = $aggregates->children_get(); foreach my $aggr (@aggregateArr) { $objName = $aggr->child_get_string("$obj" . "-name"); $objName =~ s/\///g; } my $counters = $rec->child_get("counters"); @perf_cnt_data = $counters->children_get("perf-counter-dat +a"); my $rec1; # TODO If the counter is aggregate:pa_max_disk_busy and returns nothin +g get the max of disk:disk_busy for the aggregate foreach $rec1 (@perf_cnt_data) { my $counter_name = $rec1->child_get_string("counter-na +me"); my $counter_str = $rec1->child_get_string("counter-dat +a"); my @counter_arr = split (',', $counter_str); my $colName = $objName . ":" . $counter_name; chomp($colName); $tmpHDR = $perfhash{"Time"}; $perfhash{"Time"} = join( ',' , $tmpHDR , $colName); for (@counter_arr) { my ($time,$value) = split(':',$_); my @timestamp = ( localtime($time) )[0..5]; $time = POSIX::strftime( '%F %H:%M:%S', @timestamp +); if ($gen_time_arr == 1) { $perfhash{$time} = join('' , "," , $value); $gen_time_arr = 0; }else { if ($perfhash{$time}) { $tmpVAL = $perfhash{$time}; } else { $tmpVAL = ""; } $perfhash{$time} = join( ',' , $tmpVAL , $valu +e); } } } } } return(\%perfhash); }

Thanks in advance

Lew

Comment on How do I synchronize data to common timestamp
Select or Download Code
Re: How do I synchronize data to common timestamp (mystery)
by Anonymous Monk on May 24, 2013 at 02:24 UTC

    Its impossible to guess :)

    Too many unknown moving parts

    Try again please, for example like this

    my $input = [ ... ]; my $want = { ... }; my $result = synch_that_time( $before ); require Test::More; Test::More::is( $input, $want, ' synch_that_time works as designed', );

    Where you generate $input with liberal sprinkling of Data::Dump in the above script

Re: How do I synchronize data to common timestamp
by hdb (Prior) on May 24, 2013 at 09:03 UTC

    There is very little resemblence between your input and the desired output but the timestamps rounded to next full minute (or previous?). The remaining numbers bear no obvious relation between input and output. Your explanation does not add to the understanding either.

Re: How do I synchronize data to common timestamp
by lewnewby (Novice) on May 24, 2013 at 17:25 UTC

    The input isn't displayed in my message. What I provided were examples of the output and it's format. This is performance data being collected from a NetApp Storage Array. The first data example is the output generated if I were to execute a command line request on the management server such as:

    dfm perf data retrieve -R -x TImeIndexed -C aggregate:total_transfers -C aggregate:pa_max_disk_busy -o <filerA> -o <filerB> -d 86400 | sed -e 's/\t/,/g' | sed -e 's/\s//2g'

    The -R indicates to "roll-up" the -x TimeIndexed indicates to step through at the sample rate (default is 60 seconds) the -C is an actual counter name -d is the duration in seconds going back from current time

    The second output example which comes from making an external API call to the same management server with the same -C options over the same period of time there is no option in the API for the -R or -x options so I have to figure a way to replicate it.

    The API output is a stream of timestamp:value pairs for each 60 second period in the duration per counter per aggregate per filer. Below is an example of the XML output from the API call.

    <?xml version='1.0' encoding='UTF-8' ?> <netapp version='1.0' xmlns='http://www.netapp.com/filer/admin'> <!-- Output of perf-get-counter-data, ... [Execution Time: 3921 ms +] --> <results status='passed'> <perf-instances> <perf-instance-counter-data> <instance-name>aggr_32_01</instance-name> <object-id>35441</object-id> <counters> <perf-counter-data> <object-type>aggregate</object-type> <counter-name>total_transfers</counter-name> <label-names/> <unit>per_sec</unit> <counter-data>1368985652:0.000,1368985712:0.00 +0,1368985832:0.000,1368985892:0.000,1368985952:0.000,1368986012:0.000 +,1368986132:0.000,1368986192:0.051,1368986312:613.593,1368986372:1341 +.300,1368986492:1468.627,1368986552:1425.617,1368986672:1420.000,1368 +986792:1015.066,1368986852:939.333,1368986912:1040.610,1368987032:105 +4.492,1368987092:1133.881,1368987212:989.932,1368987332:688.426,13689 +87392:613.932,1368987512:616.705,1368987572:451.250,1368987632:0.000, +1368987692:0.000,1368987752:0.034,1368987812:0.017,1368987932:0.000,1 +368988052:0.000,1368988172:0.051,1368988232:0.000,1368988352:0.000,13 +68988412:0.000,1368988532:0.000,1368988592:0.000,1368988652:0.050,136 +8988772:0.000,1368988832:0.000,1368988892:0.000,1368988952:0.000,1368 +989012:0.000,1368989072:0.333,1368989132:0.050,1368989192:0.000,13689 +89252:0.000,1368989372:0.000,1368989432:0.000,1368989492:0.000,136898 +9552:0.050,1368989672:0.000,1368989792:0.000,1368989852:0.000,1368989 +972:1239.186,1368990032:1248.700,1368990152:1381.902,1368990212:1361. +033,1368990272:1285.650,1368990332:1326.833,1368990392:956.542,136899 +0512:980.864,1368990572:1120.033,1368990632:1129.833,1368990752:891.0 +66,1368990812:798.550,1368990872:1005.833,1368990932:622.186,13689910 +52:552.068,1368991112:54.333,1368991232:0.000,1368991292:0.000,136899 +1352:0.017,1368991412:0.117,1368991532:0.328,1368991592:0.000,1368991 +712:0.000,1368991772:0.000,1368991832:0.050,1368991952:0.000,13689920 +12:0.051,1368992072:0.000,1368992132:0.000,1368992192:0.000,136899225 +2:0.000,1368992312:0.050,1368992432:0.000,1368992492:0.000,1368992552 +:0.000,1368992612:0.000,1368992672:0.000,1368992732:0.333,1368992792: +0.050,1368992852:0.000,1368992912:0.000,1368992972:0.000,1368993032:0 +.000,1368993092:0.000,1368993212:0.000,1368993272:0.050,1368993332:0. +000,1368993392:0.000,1368993452:0.000,1368993512:0.000,1368993572:125 +5.949,1368993632:1239.650,1368993692:1386.733,1368993812:1366.738,136 +8993872:1279.233,1368993932:948.983,1368993992:967.867,1368994052:104 +4.333,1368994112:983.217,1368994172:1142.067,1368994232:1138.017,1368 +994292:883.850,1368994352:804.833,1368994412:1003.717,1368994472:656. +983,1368994532:609.900,1368994592:608.617,1368994652:543.600,13689947 +12:7.433,1368994772:0.000,1368994832:0.000,1368994892:0.000,136899495 +2:0.000,1368995012:0.033,1368995072:0.350,1368995132:0.000,1368995252 +:0.000,1368995312:0.000,1368995372:0.000,1368995432:0.000,1368995492: +0.017,1368995612:0.000,1368995672:0.000,1368995792:0.000,1368995852:0 +.000,1368995912:0.000,1368995972:0.017,1368996092:0.000,1368996152:0. +000,1368996212:0.000,1368996332:0.017,1368996392:0.017,1368996452:0.0 +00,1368996512:0.000,1368996572:0.000,1368996632:0.000,1368996752:0.00 +0,1368996812:0.000,1368996872:0.000,1368996932:0.000,1368996992:0.000 +,1368997052:0.000,1368997112:654.000,1368997172:1337.850,1368997232:1 +314.450,1368997292:1252.817,1368997352:1456.383,1368997412:1222.100,1 +368997472:1356.483,1368997532:1186.017,1368997652:994.864,1368997712: +1006.300,1368997772:1071.833,1368997832:1041.433,1368997892:1145.750, +1368997952:779.767,1368998012:980.967,1368998072:815.250,1368998132:6 +26.333,1368998192:590.567,1368998252:610.217,1368998312:387.433,13689 +98372:0.000,1368998432:0.000,1368998492:0.000,1368998552:0.000,136899 +8612:0.033,1368998732:0.017,1368998792:0.000,1368998852:0.000,1368998 +912:0.000,1368998972:0.000,1368999032:0.000,1368999092:0.000,13689991 +52:0.017,1368999212:0.000,1368999272:0.000,1368999332:0.000,136899939 +2:0.000,1368999452:0.000,1368999512:0.000,1368999572:0.000,1368999692 +:0.000,1368999752:0.000,1368999812:0.000,1368999872:0.333,1368999932: +0.000,1368999992:0.000,1369000052:0.000,1369000172:0.000,1369000232:0 +.000,1369000292:0.000,1369000412:0.000,1369000472:0.000,1369000532:0. +000,1369000592:0.000,1369000652:0.000,1369000712:666.633,1369000772:1 +267.417,1369000832:1238.767,1369000892:1403.200,1369000952:1365.433,1 +369001072:1347.254,1369001132:934.767,1369001252:1033.373,1369001312: +976.483,1369001432:1150.770,1369001492:894.949,1369001552:797.550,136 +9001612:985.583,1369001672:654.283,1369001792:603.237,1369001852:539. +117,1369001912:33.183,1369002032:0.000,1369002092:0.000,1369002152:0. +000,1369002212:0.033,1369002272:0.333,1369002332:0.017,1369002392:0.0 +00,1369002512:0.000,1369002632:0.000,1369002692:0.000,1369002752:0.00 +0,1369002812:0.000,1369002872:0.017,1369002932:0.000,1369002992:0.000 +,1369003052:0.000,1369003112:0.000,1369003172:0.000,1369003292:0.000, +1369003352:0.000,1369003472:0.339,1369003592:0.000,1369003652:0.000,1 +369003712:0.000,1369003772:0.000,1369003832:0.000,1369003892:0.000,13 +69004012:0.000,1369004072:0.000,1369004132:0.000,1369004252:0.017,136 +9004312:670.583,1369004372:1223.883,1369004432:1243.667,1369004492:14 +16.550,1369004552:1353.983,1369004612:1287.050,1369004672:1343.050,13 +69004732:936.217,1369004792:961.667,1369004852:1023.717,1369004912:97 +5.433,1369004972:1148.533,1369005032:1090.950,1369005092:920.617,1369 +005152:786.583,1369005212:974.000,1369005272:644.800,1369005332:617.0 +17,1369005392:611.900,1369005452:536.800,1369005512:76.367,1369005572 +:0.000,1369005632:0.017,1369005692:0.000,1369005752:0.000,1369005812: +0.033,1369005872:0.333,1369005932:0.017,1369005992:0.000,1369006052:0 +.017,1369006112:0.000,1369006172:0.000,1369006232:0.000,1369006292:0. +000,1369006352:0.000,1369006412:0.000,1369006472:0.000,1369006592:0.0 +16,1369006652:0.000,1369006712:0.000,1369006772:0.000,1369006832:0.00 +0,1369006892:0.000,1369006952:0.000,1369007012:0.000,1369007072:0.016 +,1369007132:0.333,1369007192:0.017,1369007252:0.000,1369007312:0.000, +1369007372:0.000,1369007432:0.000,1369007492:0.017,1369007552:0.000,1 +369007612:0.000,1369007672:0.000,1369007732:0.000,1369007792:0.000,13 +69007852:0.000,1369007912:0.000,1369007972:701.443,1369008032:1253.88 +3,1369008092:1254.917,1369008152:1400.267,1369008212:1421.583,1369008 +272:1290.424,1369008332:1343.148,1369008392:1018.800,1369008452:955.5 +17,1369008512:1043.300,1369008572:1070.424,1369008632:1075.967,136900 +8692:1129.117,1369008752:1059.067,1369008812:766.017,1369008872:1065. +800,1369008932:770.102,1369008992:605.705,1369009052:614.542,13690091 +12:597.607,1369009172:256.033,1369009232:0.000,1369009292:0.017,13690 +09352:0.016,1369009412:0.000,1369009472:0.017,1369009532:0.339,136900 +9592:0.000,1369009652:0.000,1369009712:0.000,1369009772:0.000,1369009 +832:0.000,1369009892:0.000,1369009952:0.000,1369010012:0.000,13690100 +72:0.000,1369010132:0.000,1369010192:0.000,1369010252:0.000,136901031 +2:0.000,1369010372:1.271,1369010432:0.000,1369010492:0.000,1369010552 +:0.000,1369010612:0.000,1369010672:0.033,1369010732:0.328,1369010792: +0.000,1369010852:0.000,1369010912:0.000,1369010972:0.000,1369011032:0 +.000,1369011092:0.000,1369011152:0.000,1369011212:0.000,1369011272:0. +000,1369011332:0.000,1369011392:0.000,1369011452:0.000,1369011512:0.0 +00,1369011572:751.183,1369011632:1321.328,1369011692:1277.373,1369011 +752:1343.377,1369011812:1493.233,1369011872:1244.250,1369011932:1445. +900,1369011992:1160.383,1369012052:996.667,1369012112:1044.458,136901 +2172:1054.410,1369012232:1043.883,1369012292:1150.186,1369012352:1150 +.787,1369012412:1097.881,1369012472:816.885,1369012532:1092.583,13690 +12592:903.417,1369012652:722.667,1369012712:709.283,1369012772:713.56 +7,1369012832:638.917,1369012892:311.167,1369012952:189.483,1369013012 +:189.033,1369013072:149.517,1369013132:112.483,1369013192:117.417,136 +9013252:119.283,1369013312:113.836,1369013372:112.167,1369013432:114. +367,1369013492:138.967,1369013552:120.417,1369013612:114.317,13690136 +72:117.433,1369013732:114.033,1369013792:119.983,1369013852:110.467,1 +369013912:110.067,1369013972:115.700,1369014032:116.567,1369014092:11 +6.383,1369014152:114.183,1369014212:119.410,1369014272:110.633,136901 +4332:111.667,1369014392:114.517,1369014452:113.400,1369014512:121.483 +,1369014572:108.233,1369014632:114.733,1369014692:115.333,1369014752: +119.200,1369014812:118.800,1369014872:112.900,1369014932:123.797,1369 +014992:118.836,1369015052:111.288,1369015112:114.951,1369015172:835.6 +10,1369015232:1391.295,1369015292:1401.483,1369015352:1280.283,136901 +5412:1404.966,1369015472:1410.770,1369015532:1331.833,1369015592:1405 +.033,1369015652:1046.767,1369015712:954.317,1369015772:1055.700,13690 +15832:1047.767,1369015892:1027.583,1369015952:1218.136,1369016012:107 +1.833,1369016072:1041.738,1369016132:841.583,1369016192:1125.333,1369 +016252:759.183,1369016312:692.117,1369016372:697.153,1369016432:674.9 +84,1369016492:627.898,1369016552:343.459,1369016612:164.983,136901667 +2:163.344,1369016732:172.433,1369016792:161.283,1369016852:171.800,13 +69016912:164.700,1369016972:170.150,1369017032:168.717,1369017092:167 +.983,1369017152:168.576,1369017212:165.934,1369017272:162.717,1369017 +332:165.333,1369017392:163.817,1369017452:163.750,1369017512:165.917, +1369017572:172.475,1369017632:163.262,1369017692:167.067,1369017752:1 +66.153,1369017812:162.148,1369017872:165.650,1369017932:170.576,13690 +17992:166.467,1369018052:162.115,1369018112:167.300,1369018172:165.39 +0,1369018232:164.016,1369018292:167.867,1369018352:160.424,1369018412 +:158.213,1369018472:157.017,1369018532:159.767,1369018592:160.633,136 +9018652:158.667,1369018712:156.627,1369018772:847.250,1369018832:1327 +.066,1369018892:1296.167,1369018952:1295.356,1369019012:1475.500,1369 +019072:1300.683,1369019132:1371.410,1369019192:1291.407,1369019252:10 +04.933,1369019312:982.317,1369019372:1021.508,1369019432:1098.661,136 +9019492:1075.133,1369019552:1102.607,1369019612:1162.254,1369019672:1 +020.541,1369019732:835.695,1369019792:1027.667,1369019852:794.098,136 +9019912:691.450,1369019972:693.814,1369020032:701.233,1369020092:630. +017,1369020152:199.705,1369020212:195.034,1369020272:185.983,13690203 +32:194.400,1369020392:190.295,1369020452:189.317,1369020512:192.339,1 +369020572:189.770,1369020632:190.533,1369020692:187.492,1369020752:18 +3.867,1369020812:188.623,1369020872:214.034,1369020932:185.885,136902 +0992:185.650,1369021052:185.550,1369021112:190.450,1369021172:186.533 +,1369021232:182.850,1369021292:191.983,1369021352:183.361,1369021412: +196.915,1369021472:191.550,1369021532:196.083,1369021592:194.033,1369 +021652:192.983,1369021712:199.917,1369021772:190.700,1369021832:200.2 +50,1369021892:190.983,1369021952:196.817,1369022012:193.917,136902207 +2:201.967,1369022132:189.650,1369022192:190.800,1369022252:194.550,13 +69022312:195.800,1369022372:821.100,1369022432:1330.317,1369022492:12 +58.867,1369022552:1363.333,1369022612:1499.717,1369022672:1245.917,13 +69022732:1431.633,1369022792:1230.950,1369022852:1010.233,1369022912: +1011.000,1369022972:1060.917,1369023032:1095.183,1369023092:1131.541, +1369023152:1098.373,1369023212:1189.917,1369023272:927.869,1369023332 +:945.847,1369023392:1028.000,1369023452:745.197,1369023512:718.186,13 +69023572:696.900,1369023632:707.083,1369023692:559.017,1369023752:211 +.983,1369023812:213.800,1369023872:215.617,1369023932:224.517,1369023 +992:212.717,1369024052:215.550,1369024112:224.183,1369024172:195.344, +1369024232:195.661,1369024292:192.783,1369024352:194.433,1369024412:1 +95.233,1369024472:196.717,1369024532:214.333,1369024592:210.967,13690 +24652:243.517,1369024712:216.817,1369024772:210.633,1369024832:224.55 +0,1369024892:208.689,1369024952:223.373,1369025012:212.800,1369025072 +:216.150,1369025132:217.934,1369025192:216.831,1369025252:242.300,136 +9025312:213.083,1369025372:222.633,1369025432:216.867,1369025492:221. +700,1369025552:222.550,1369025612:212.967,1369025672:208.633,13690257 +32:209.533,1369025792:207.333,1369025852:212.850,1369025912:214.350,1 +369025972:741.550,1369026032:1111.800,1369026092:1130.983,1369026152: +1235.133,1369026212:1105.639,1369026272:1156.050,1369026332:1143.733, +1369026392:767.678,1369026452:1049.950,1369026512:1042.733,1369026572 +:1143.066,1369026632:1207.407,1369026692:1154.820,1369026752:883.083, +1369026812:793.492,1369026872:990.833,1369026932:1055.600,1369026992: +968.600,1369027052:996.883,1369027112:1034.400,1369027172:989.417,136 +9027232:998.050,1369027292:964.333,1369027352:506.317,1369027412:210. +050,1369027472:210.950,1369027532:204.933,1369027592:214.633,13690276 +52:166.467,1369027712:159.600,1369027772:163.300,1369027832:167.333,1 +369027892:165.467,1369027952:162.377,1369028012:160.746,1369028072:15 +9.250,1369028132:157.733,1369028192:158.164,1369028252:157.600,136902 +8312:159.898,1369028372:155.033,1369028432:209.333,1369028492:216.169 +,1369028552:215.317,1369028612:216.583,1369028672:208.367,1369028732: +215.459,1369028792:217.542,1369028852:217.017,1369028912:213.717,1369 +028972:214.017,1369029032:222.917,1369029092:217.133,1369029152:231.0 +82,1369029212:213.085,1369029272:213.150,1369029332:218.367,136902939 +2:223.333,1369029452:213.098,1369029512:213.712,1369029572:761.885,13 +69029632:1301.288,1369029692:1288.000,1369029752:1364.317,1369029812: +1448.548,1369029872:1315.034,1369029932:1405.016,1369029992:1109.847, +1369030052:971.623,1369030112:1061.983,1369030172:1059.967,1369030232 +:1061.607,1369030292:1169.383,1369030352:1140.300,1369030412:1053.533 +,1369030472:857.153,1369030532:1054.377,1369030592:1094.458,136903065 +2:728.967,1369030712:754.117,1369030772:736.983,1369030832:691.049,13 +69030892:531.583,1369030952:207.559,1369031012:204.689,1369031072:210 +.500,1369031132:210.167,1369031192:204.083,1369031252:186.517,1369031 +312:156.883,1369031372:156.483,1369031432:157.183,1369031492:157.133, +1369031552:155.500,1369031612:156.650,1369031672:158.627,1369031732:1 +55.328,1369031792:156.567,1369031852:157.983,1369031912:156.898,13690 +31972:156.951,1369032032:158.814,1369032092:155.443,1369032152:159.44 +1,1369032212:154.033,1369032272:160.533,1369032332:160.067,1369032392 +:158.750,1369032452:159.492,1369032512:155.000,1369032572:158.067,136 +9032632:159.933,1369032692:157.700,1369032752:154.533,1369032812:160. +695,1369032872:156.967,1369032932:154.262,1369032992:158.733,13690330 +52:159.085,1369033112:155.656,1369033172:842.050,1369033232:1644.833, +1369033292:1963.217,1369033352:1970.067,1369033412:1878.267,136903347 +2:1427.283,1369033532:1286.350,1369033592:1385.102,1369033652:1243.60 +0,1369033712:953.770,1369033772:969.441,1369033832:1028.067,136903389 +2:1065.934,1369033952:1091.050,1369034012:1047.600,1369034072:1185.78 +0,1369034132:887.721,1369034192:890.450,1369034252:1003.542,136903431 +2:701.066,1369034372:688.167,1369034432:656.517,1369034492:662.217,13 +69034552:469.667,1369034612:162.050,1369034672:157.881,1369034732:157 +.459,1369034792:157.746,1369034852:156.754,1369034912:158.458,1369034 +972:155.475,1369035032:158.068,1369035092:154.836,1369035152:155.333, +1369035212:158.186,1369035272:158.033,1369035332:154.754,1369035392:1 +57.717,1369035452:158.525,1369035512:156.590,1369035572:155.983,13690 +35632:156.317,1369035692:157.600,1369035752:156.983,1369035812:156.95 +0,1369035872:155.850,1369035932:157.467,1369035992:157.600,1369036052 +:158.517,1369036112:154.770,1369036172:159.881,1369036232:154.721,136 +9036292:163.034,1369036352:159.500,1369036412:156.230,1369036472:158. +833,1369036532:159.933,1369036592:161.183,1369036652:161.500,13690367 +12:160.559,1369036772:805.733,1369036832:1264.883,1369036892:1252.633 +,1369036952:1336.350,1369037012:1444.067,1369037072:1264.850,13690371 +32:1410.459,1369037192:1092.203,1369037252:946.410,1369037312:1035.28 +8,1369037372:1049.049,1369037432:1019.322,1369037492:1123.250,1369037 +552:1128.279,1369037612:1082.000,1369037672:818.467,1369037732:1076.4 +83,1369037792:839.583,1369037852:706.966,1369037912:681.754,136903797 +2:664.200,1369038032:653.627,1369038092:389.180,1369038152:167.407,13 +69038212:163.393,1369038272:163.383,1369038332:163.983,1369038392:165 +.500,1369038452:167.550,1369038512:164.983,1369038572:165.712,1369038 +632:163.984,1369038692:165.717,1369038752:161.600,1369038812:164.150, +1369038872:166.450,1369038932:164.467,1369038992:166.033,1369039052:1 +62.750,1369039112:217.450,1369039172:184.678,1369039232:167.098,13690 +39292:175.583,1369039352:165.450,1369039412:169.700,1369039472:182.20 +0,1369039532:168.167,1369039592:165.763,1369039652:161.443,1369039712 +:159.500,1369039772:163.797,1369039832:162.583,1369039892:165.689,136 +9039952:171.136,1369040012:166.983,1369040072:163.700,1369040132:164. +617,1369040192:164.754,1369040252:166.220,1369040312:221.883,13690403 +72:783.883,1369040432:1263.492,1369040492:1269.051,1369040552:1328.11 +5,1369040612:1469.712,1369040672:1239.180,1369040732:1420.183,1369040 +792:1129.441,1369040852:954.770,1369040912:1023.483,1369040972:1057.6 +83,1369041032:1017.983,1369041092:1100.295,1369041152:1139.051,136904 +1212:1078.250,1369041272:795.733,1369041332:1066.567,1369041392:867.9 +67,1369041452:685.230,1369041512:696.850,1369041572:678.150,136904163 +2:633.356,1369041692:342.350,1369041752:168.783,1369041812:165.933,13 +69041872:165.770,1369041932:168.831,1369041992:165.541,1369042052:177 +.356,1369042112:174.800,1369042172:180.500,1369042232:176.867,1369042 +292:176.917,1369042352:173.883,1369042412:177.100,1369042472:176.317, +1369042532:171.817,1369042592:180.246,1369042652:176.153,1369042712:1 +77.533,1369042772:172.443,1369042832:171.550,1369042892:175.576,13690 +42952:172.300,1369043012:176.150,1369043072:172.475,1369043132:176.11 +9,1369043192:173.623,1369043252:171.119,1369043312:174.300,1369043372 +:172.115,1369043432:177.475,1369043492:174.017,1369043552:173.333,136 +9043612:175.300,1369043672:175.733,1369043732:174.770,1369043792:177. +683,1369043852:175.373,1369043912:177.754,1369043972:947.119,13690440 +32:1297.525,1369056992:543.863,1369057052:174.117,1369057112:163.900, +1369057172:193.458,1369057232:164.150,1369057292:165.250,1369057352:1 +65.033,1369057412:164.083,1369057472:166.016,1369057532:166.000,13690 +57592:170.574,1369057652:170.610,1369057712:166.783,1369057772:163.95 +0,1369057832:168.850,1369057892:233.426,1369057952:166.983,1369058012 +:166.217,1369058072:173.767,1369058132:166.400,1369058192:165.033,136 +9058252:165.783,1369058312:166.150,1369058372:1141.500,1369058432:122 +9.217,1369058492:1257.333,1369058552:1349.967,1369058612:1418.633,136 +9058672:1278.500,1369058732:1407.783,1369058792:1074.817,1369058852:9 +55.450,1369058912:1021.633,1369058972:1029.417,1369059032:1012.067,13 +69059092:1159.867,1369059152:1090.787,1369059212:1047.983,1369059272: +813.450,1369059332:1063.180,1369059392:852.169,1369059452:699.246,136 +9059512:704.150,1369059572:672.508,1369059632:645.656,1369059692:417. +898,1369059752:169.067,1369059812:167.867,1369059872:167.717,13690599 +32:180.574,1369059992:175.678,1369060052:169.983,1369060112:168.000,1 +369060172:172.850,1369060232:173.934,1369060292:181.850,1369060352:18 +0.100,1369060412:180.017,1369060472:176.717,1369060532:181.400,136906 +0592:175.623,1369060652:183.153,1369060712:179.617,1369060772:176.017 +,1369060832:174.934,1369060892:178.898,1369060952:177.344,1369061012: +182.237,1369061072:176.317,1369061132:170.950,1369061192:183.783,1369 +061252:295.283,1369061312:225.267,1369061372:222.433,1369061432:213.8 +17,1369061492:213.934,1369061552:219.517,1369061612:217.153,136906167 +2:223.283,1369061732:216.883,1369061792:210.500,1369061852:212.617,13 +69061912:204.967,1369061972:876.000,1369062032:1241.683,1369062092:12 +67.237,1369062152:1365.100,1369062212:1409.417,1369062272:1270.377,13 +69062332:1418.390,1369062392:1048.783,1369062452:958.557,1369062512:1 +042.847,1369062572:1041.717,1369062632:1022.367,1369062692:1157.492,1 +369062752:1107.733,1369062812:1005.356,1369062872:802.700,1369062932: +1146.617,1369062992:783.867,1369063052:708.250,1369063112:726.233,136 +9063172:712.033,1369063232:666.467,1369063292:294.783,1369063352:227. +800,1369063412:218.500,1369063472:218.800,1369063532:228.483,13690635 +92:221.150,1369063652:212.950,1369063712:210.500,1369063772:213.650,1 +369063832:213.417,1369063892:210.067,1369063952:211.700,1369064012:20 +9.917,1369064072:219.383,1369064132:245.300,1369064192:219.867,136906 +4252:219.817,1369064312:221.459,1369064372:210.966,1369064432:224.850 +,1369064492:212.100,1369064552:217.787,1369064612:215.750,1369064672: +222.966,1369064732:217.933,1369064792:216.750,1369064852:227.400,1369 +064912:232.567,1369064972:221.133,1369065032:210.600,1369065092:218.7 +83,1369065152:214.967,1369065212:220.517,1369065272:219.814,136906533 +2:222.467,1369065392:210.083,1369065452:217.367,1369065512:209.950,13 +69065572:1542.817,1369065632:1982.098,1369065692:1990.797,1369065752: +1766.967,1369065812:1381.417,1369065872:1442.017,1369065932:1353.115, +1369065992:1376.847,1369066052:1052.017,1369066112:965.820,1369066172 +:1055.085,1369066232:1076.367,1369066292:1061.164,1369066352:1156.850 +,1369066412:1174.917,1369066472:1002.881,1369066532:805.117,136906659 +2:1119.250,1369066652:917.689,1369066712:719.881,1369066772:727.550,1 +369066832:708.049,1369066892:647.869,1369066952:343.233,1369067012:21 +3.450,1369067072:287.450,1369067132:214.767,1369067192:212.683,136906 +7252:227.305,1369067312:221.100,1369067372:217.721,1369067432:213.833 +,1369067492:218.800,1369067552:222.017,1369067612:218.283,1369067672: +219.150,1369067732:212.133,1369067792:221.983,1369067852:236.733,1369 +067912:219.950,1369067972:215.600,1369068032:219.667,1369068092:211.3 +00,1369068152:219.600,1369068212:211.917,1369068272:211.000,136906833 +2:213.100,1369068392:218.695,1369068452:209.295,1369068512:215.150,13 +69068572:223.650,1369068632:232.763,1369068692:212.377,1369068752:206 +.617,1369068812:216.800,1369068872:216.220,1369068932:218.082,1369068 +992:214.383,1369069052:214.949,1369069112:210.049,1369069172:1210.267 +,1369069232:1283.117,1369069292:1324.083,1369069352:1405.567,13690694 +12:1451.517,1369069472:1319.800,1369069532:1421.283,1369069592:1101.6 +33,1369069652:1011.600,1369069712:1052.233,1369069772:1071.917,136906 +9832:1043.717,1369069892:1185.900,1369069952:1137.683,1369070012:1084 +.167,1369070072:840.600,1369070132:1092.167,1369070192:910.390,136907 +0252:731.933,1369070312:719.525,1369070372:704.183,1369070432:669.633 +,1369070492:417.050,1369070552:216.867,1369070612:283.400,1369070672: +215.633,1369070732:214.051,1369070792:222.311,1369070852:217.300,1369 +070912:218.883,1369070972:222.814,1369071032:215.607,1369071092:213.4 +67,1369071152:223.373,1369071212:213.410,1369071272:213.949,136907133 +2:219.200,1369071392:235.295,1369071452:219.450,1369071512:218.627,13 +69071572:218.633,1369071632:212.639,1369071692:217.317,1369071752:212 +.617,1369071775:211.400</counter-data> </perf-counter-data> <perf-counter-data> <object-type>aggregate</object-type> <counter-name>pa_max_disk_busy</counter-name> <label-names/> <unit>percent</unit> <counter-data>1368985661:0.000,1368985721:0.00 +0,1368985781:0.000,1368985841:0.000,1368985901:0.000,1368985961:0.000 +,1368986021:0.000,1368986081:0.000,1368986141:0.000,1368986201:0.039, +1368986261:0.000,1368986321:42.458,1368986381:69.015,1368986441:70.61 +5,1368986501:67.124,1368986561:67.129,1368986621:65.561,1368986681:66 +.102,1368986741:57.295,1368986801:52.614,1368986861:51.757,1368986921 +:52.960,1368986981:50.856,1368987041:50.917,1368987101:49.481,1368987 +161:35.832,1368987221:35.905,1368987281:31.044,1368987341:31.759,1368 +987401:31.608,1368987461:32.737,1368987521:22.967,1368987581:0.000,13 +68987641:0.000,1368987701:0.023,1368987821:0.071,1368987881:0.042,136 +8987941:0.000,1368988001:0.000,1368988061:0.000,1368988121:0.000,1368 +988181:0.037,1368988301:0.000,1368988361:0.000,1368988421:0.000,13689 +88481:0.000,1368988541:0.000,1368988661:0.034,1368988721:0.000,136898 +8781:0.000,1368988841:0.000,1368988901:0.000,1368989021:0.000,1368989 +081:0.064,1368989141:0.000,1368989201:0.000,1368989261:0.000,13689893 +81:0.000,1368989441:0.000,1368989501:0.000,1368989621:0.000,136898968 +1:0.000,1368989801:0.000,1368989861:0.000,1368989921:43.899,136898998 +1:65.945,1368990041:68.528,1368990101:61.271,1368990161:66.873,136899 +0221:63.030,1368990281:65.144,1368990341:53.065,1368990401:51.917,136 +8990461:51.024,1368990521:52.074,1368990581:49.263,1368990641:49.656, +1368990701:40.882,1368990761:37.504,1368990821:35.788,1368990881:31.0 +68,1368990941:29.926,1368991001:33.985,1368991061:29.209,1368991121:0 +.383,1368991181:0.000,1368991241:0.000,1368991301:0.000,1368991361:0. +050,1368991421:0.078,1368991481:0.037,1368991541:0.000,1368991601:0.0 +00,1368991661:0.000,1368991721:0.000,1368991781:0.000,1368991841:0.03 +5,1368991901:0.000,1368991961:0.000,1368992021:0.038,1368992081:0.000 +,1368992141:0.000,1368992201:0.000,1368992261:0.000,1368992321:0.035, +1368992381:0.000,1368992441:0.000,1368992501:0.000,1368992561:0.000,1 +368992621:0.000,1368992681:0.038,1368992741:0.043,1368992801:0.000,13 +68992861:0.000,1368992921:0.000,1368992981:0.000,1368993041:0.000,136 +8993101:0.000,1368993161:0.000,1368993221:0.043,1368993281:0.000,1368 +993341:0.000,1368993401:0.000,1368993461:0.000,1368993521:45.660,1368 +993581:67.076,1368993641:67.535,1368993701:61.504,1368993761:67.336,1 +368993821:63.695,1368993881:65.982,1368993941:53.623,1368994001:51.90 +5,1368994061:51.198,1368994121:52.071,1368994181:49.637,1368994241:49 +.850,1368994301:40.183,1368994361:38.837,1368994421:35.799,1368994481 +:31.390,1368994541:29.605,1368994601:33.586,1368994661:26.949,1368994 +721:0.000,1368994781:0.000,1368994841:0.000,1368994901:0.000,13689949 +61:0.068,1368995021:0.027,1368995081:0.047,1368995141:0.000,136899520 +1:0.000,1368995261:0.000,1368995321:0.000,1368995381:0.000,1368995441 +:0.000,1368995501:0.020,1368995561:0.000,1368995621:0.000,1368995681: +0.000,1368995741:0.000,1368995861:0.000,1368995921:0.000,1368995981:0 +.000,1368996041:0.032,1368996101:0.000,1368996161:0.000,1368996221:0. +000,1368996281:0.000,1368996341:0.040,1368996401:0.028,1368996461:0.0 +28,1368996521:0.000,1368996581:0.000,1368996641:0.000,1368996701:0.00 +0,1368996761:0.000,1368996821:0.000,1368996881:0.000,1368996941:0.000 +,1368997001:0.000,1368997061:0.000,1368997121:0.000,1368997181:45.691 +,1368997241:77.386,1368997301:77.254,1368997361:66.737,1368997421:62. +048,1368997481:65.512,1368997541:63.123,1368997601:60.531,1368997661: +53.054,1368997721:51.032,1368997781:52.308,1368997841:49.961,13689979 +01:50.585,1368997961:48.129,1368998021:34.504,1368998081:42.353,13689 +98141:31.766,1368998201:31.199,1368998261:32.336,1368998321:31.942,13 +68998381:18.150,1368998441:0.000,1368998501:0.000,1368998561:0.000,13 +68998621:0.000,1368998681:0.082,1368998741:0.038,1368998801:0.023,136 +8998861:0.000,1368998921:0.000,1368998981:0.000,1368999041:0.000,1368 +999101:0.000,1368999161:0.000,1368999221:0.023,1368999281:0.000,13689 +99341:0.000,1368999401:0.000,1368999461:0.000,1368999521:0.000,136899 +9581:0.000,1368999641:0.000,1368999701:0.000,1368999761:0.000,1368999 +821:0.000,1368999881:0.000,1368999941:0.038,1369000001:0.000,13690000 +61:0.000,1369000121:0.000,1369000181:0.000,1369000241:0.000,136900030 +1:0.000,1369000361:0.000,1369000421:0.000,1369000481:0.000,1369000541 +:0.000,1369000601:0.000,1369000661:0.000,1369000721:0.000,1369000781: +45.715,1369000841:68.178,1369000901:68.189,1369000961:62.106,13690010 +21:65.990,1369001081:63.405,1369001141:65.130,1369001201:53.136,13690 +01261:51.410,1369001321:51.595,1369001381:52.055,1369001441:50.318,13 +69001501:49.415,1369001561:40.548,1369001621:38.810,1369001681:34.804 +,1369001741:31.528,1369001801:29.909,1369001861:33.130,1369001921:27. +502,1369001981:0.023,1369002041:0.000,1369002101:0.000,1369002161:0.0 +00,1369002221:0.000,1369002281:0.051,1369002341:0.040,1369002401:0.02 +5,1369002461:0.000,1369002521:0.000,1369002581:0.000,1369002641:0.000 +,1369002701:0.000,1369002761:0.000,1369002821:0.000,1369002881:0.000, +1369002941:0.025,1369003001:0.000,1369003061:0.000,1369003121:0.000,1 +369003181:0.000,1369003241:0.000,1369003301:0.000,1369003361:0.022,13 +69003421:0.000,1369003481:0.000,1369003541:0.038,1369003601:0.031,136 +9003661:0.000,1369003721:0.000,1369003781:0.000,1369003841:0.000,1369 +003901:0.000,1369003961:0.000,1369004021:0.000,1369004081:0.000,13690 +04141:0.000,1369004201:0.000,1369004261:0.000,1369004321:0.032,136900 +4381:45.850,1369004441:65.682,1369004501:67.157,1369004561:62.136,136 +9004621:66.570,1369004681:64.346,1369004741:65.213,1369004801:52.641, +1369004861:50.749,1369004921:50.655,1369004981:52.055,1369005041:49.4 +76,1369005101:49.375,1369005161:40.568,1369005221:37.165,1369005281:3 +4.433,1369005341:30.954,1369005401:94.142,1369005461:100.000,13690055 +21:100.000,1369005581:5.090,1369005641:0.000,1369005701:0.026,1369005 +761:0.000,1369005821:0.000,1369005881:0.060,1369005941:0.038,13690060 +01:0.022,1369006061:0.000,1369006121:0.032,1369006181:0.000,136900624 +1:0.000,1369006301:0.000,1369006361:0.000,1369006421:0.000,1369006481 +:0.000,1369006541:0.000,1369006601:0.025,1369006661:0.000,1369006721: +0.000,1369006781:0.000,1369006841:0.000,1369006901:0.000,1369006961:0 +.000,1369007021:0.020,1369007081:0.000,1369007141:0.040,1369007201:0. +027,1369007261:0.000,1369007321:0.000,1369007381:0.000,1369007441:0.0 +00,1369007501:0.023,1369007561:0.000,1369007621:0.000,1369007681:0.00 +0,1369007741:0.000,1369007801:0.000,1369007861:0.000,1369007921:0.000 +,1369007981:46.144,1369008041:68.744,1369008101:69.333,1369008161:64. +462,1369008221:69.039,1369008281:66.032,1369008341:68.454,1369008401: +57.520,1369008461:55.274,1369008521:53.926,1369008581:54.065,13690086 +41:51.144,1369008701:51.112,1369008761:45.365,1369008821:34.230,13690 +08881:42.095,1369008941:31.917,1369009001:30.555,1369009061:33.000,13 +69009121:31.179,1369009181:11.403,1369009241:0.000,1369009301:0.042,1 +369009361:0.023,1369009421:0.000,1369009481:0.053,1369009541:0.037,13 +69009601:0.000,1369009661:0.000,1369009721:0.000,1369009781:0.000,136 +9009841:0.000,1369009901:0.000,1369009961:0.000,1369010021:0.000,1369 +010081:0.000,1369010141:0.000,1369010201:0.000,1369010261:0.000,13690 +10321:0.000,1369010381:0.419,1369010441:0.000,1369010501:0.000,136901 +0561:0.000,1369010621:0.000,1369010681:0.059,1369010741:0.039,1369010 +801:0.000,1369010861:0.000,1369010921:0.000,1369010981:0.000,13690110 +41:0.000,1369011101:0.000,1369011161:0.000,1369011221:0.000,136901128 +1:0.000,1369011341:0.000,1369011401:0.000,1369011461:0.000,1369011521 +:0.000,1369011581:50.162,1369011641:70.843,1369011701:71.312,13690117 +61:69.594,1369011821:66.802,1369011881:67.853,1369011941:67.083,13690 +12001:63.972,1369012061:58.410,1369012121:57.296,1369012181:57.423,13 +69012241:57.377,1369012301:55.254,1369012361:56.078,1369012421:51.719 +,1369012481:43.779,1369012541:52.385,1369012601:40.907,1369012661:42. +651,1369012721:38.613,1369012781:42.914,1369012841:40.655,1369012901: +18.674,1369012961:14.807,1369013021:15.368,1369013081:9.136,136901314 +1:7.809,1369013201:8.267,1369013261:7.909,1369013321:8.308,1369013381 +:7.770,1369013441:8.366,1369013501:7.806,1369013561:8.252,1369013621: +7.805,1369013681:8.113,1369013741:8.103,1369013801:8.276,1369013861:8 +.904,1369013921:8.174,1369013981:8.189,1369014041:8.432,1369014101:8. +200,1369014161:8.004,1369014221:8.223,1369014281:8.636,1369014341:8.0 +93,1369014401:7.897,1369014461:7.956,1369014521:8.691,1369014581:8.44 +1,1369014641:8.249,1369014701:8.962,1369014761:7.781,1369014821:7.536 +,1369014881:7.641,1369014941:8.618,1369015001:8.484,1369015061:8.313, +1369015121:8.288,1369015181:54.812,1369015241:79.756,1369015301:78.89 +7,1369015361:74.065,1369015421:67.833,1369015481:68.273,1369015541:67 +.348,1369015601:67.994,1369015661:59.998,1369015721:57.094,1369015781 +:57.560,1369015841:56.983,1369015901:57.311,1369015961:55.411,1369016 +021:56.299,1369016081:52.860,1369016141:42.914,1369016201:49.362,1369 +016261:40.253,1369016321:39.449,1369016381:38.703,1369016441:41.213,1 +369016501:40.439,1369016561:18.562,1369016621:13.575,1369016681:15.12 +7,1369016741:14.688,1369016801:14.475,1369016861:14.705,1369016921:14 +.493,1369016981:14.608,1369017041:14.245,1369017101:14.633,1369017161 +:13.870,1369017221:14.097,1369017281:14.048,1369017341:13.542,1369017 +401:13.113,1369017461:13.752,1369017521:13.840,1369017581:14.364,1369 +017641:14.966,1369017701:14.550,1369017761:14.194,1369017821:14.197,1 +369017881:14.466,1369017941:14.788,1369018001:14.347,1369018061:13.60 +5,1369018121:14.768,1369018181:13.994,1369018241:14.058,1369018301:14 +.068,1369018361:14.039,1369018421:13.992,1369018481:14.115,1369018541 +:14.078,1369018601:14.210,1369018661:14.082,1369018721:13.887,1369018 +781:56.029,1369018841:73.193,1369018901:74.110,1369018961:72.618,1369 +019021:68.100,1369019081:70.158,1369019141:67.784,1369019201:67.007,1 +369019261:60.307,1369019321:58.829,1369019381:57.367,1369019441:58.72 +2,1369019501:55.139,1369019561:54.956,1369019621:54.230,1369019681:49 +.843,1369019741:44.123,1369019801:45.293,1369019861:41.558,1369019921 +:40.208,1369019981:40.447,1369020041:39.953,1369020101:37.493,1369020 +161:14.805,1369020221:14.833,1369020281:14.886,1369020341:14.813,1369 +020401:15.320,1369020461:14.860,1369020521:14.807,1369020581:14.418,1 +369020641:15.034,1369020701:15.161,1369020761:15.451,1369020821:14.76 +1,1369020881:14.746,1369020941:13.928,1369021001:14.794,1369021061:14 +.670,1369021121:14.905,1369021181:14.694,1369021241:14.555,1369021301 +:15.465,1369021361:15.344,1369021421:14.571,1369021481:15.175,1369021 +541:18.841,1369021601:15.266,1369021661:15.051,1369021721:14.973,1369 +021781:15.821,1369021841:15.823,1369021901:14.839,1369021961:15.701,1 +369022021:14.992,1369022081:15.721,1369022141:15.234,1369022201:17.99 +0,1369022261:15.529,1369022321:15.576,1369022381:53.499,1369022441:72 +.465,1369022501:71.081,1369022561:70.492,1369022621:70.287,1369022681 +:69.455,1369022741:68.175,1369022801:66.478,1369022861:59.129,1369022 +921:57.495,1369022981:57.307,1369023041:59.198,1369023101:56.182,1369 +023161:57.259,1369023221:55.569,1369023281:44.988,1369023341:49.146,1 +369023401:44.225,1369023461:40.668,1369023521:39.680,1369023581:40.84 +0,1369023641:41.128,1369023701:33.911,1369023761:15.858,1369023821:15 +.301,1369023881:15.921,1369023941:15.887,1369024001:15.236,1369024061 +:15.356,1369024121:16.159,1369024181:15.694,1369024241:15.506,1369024 +301:14.959,1369024361:14.356,1369024421:14.422,1369024481:14.879,1369 +024541:15.128,1369024601:15.357,1369024661:15.057,1369024721:14.844,1 +369024781:15.656,1369024841:15.718,1369024901:16.429,1369024961:15.62 +0,1369025021:15.532,1369025081:15.645,1369025141:15.974,1369025201:15 +.581,1369025261:15.293,1369025321:15.102,1369025381:15.792,1369025441 +:15.710,1369025501:15.305,1369025561:15.412,1369025621:15.248,1369025 +681:15.198,1369025741:15.011,1369025801:14.904,1369025861:16.343,1369 +025921:15.004,1369025981:46.071,1369026041:60.117,1369026101:59.839,1 +369026161:55.953,1369026221:57.616,1369026281:53.518,1369026341:51.32 +1,1369026401:46.204,1369026461:56.909,1369026521:56.997,1369026581:53 +.717,1369026641:56.204,1369026701:57.932,1369026761:49.209,1369026821 +:51.839,1369026881:57.782,1369026941:58.856,1369027001:59.142,1369027 +061:57.656,1369027121:59.013,1369027181:57.694,1369027241:56.671,1369 +027301:46.674,1369027361:18.804,1369027421:14.895,1369027481:15.651,1 +369027541:15.361,1369027601:16.659,1369027661:14.767,1369027721:14.82 +4,1369027781:14.480,1369027841:14.243,1369027901:13.665,1369027961:14 +.133,1369028021:13.673,1369028081:14.162,1369028141:14.713,1369028201 +:13.963,1369028261:13.905,1369028321:15.109,1369028381:14.931,1369028 +441:15.133,1369028501:15.277,1369028561:16.890,1369028621:15.999,1369 +028681:15.814,1369028741:16.592,1369028801:16.104,1369028861:15.367,1 +369028921:15.076,1369028981:15.272,1369029041:15.452,1369029101:15.92 +9,1369029161:16.287,1369029221:15.362,1369029281:15.528,1369029341:15 +.369,1369029401:15.673,1369029461:15.370,1369029521:16.334,1369029581 +:48.708,1369029641:70.165,1369029701:70.893,1369029761:68.219,1369029 +821:67.975,1369029881:66.601,1369029941:68.169,1369030001:62.023,1369 +030061:58.993,1369030121:57.311,1369030181:57.316,1369030241:58.187,1 +369030301:56.045,1369030361:57.366,1369030421:53.866,1369030481:46.96 +0,1369030541:56.333,1369030601:46.475,1369030661:43.288,1369030721:42 +.608,1369030781:45.736,1369030841:41.591,1369030901:31.786,1369030961 +:15.499,1369031021:15.558,1369031081:15.068,1369031141:15.030,1369031 +201:15.070,1369031261:14.832,1369031321:14.071,1369031381:14.407,1369 +031441:13.775,1369031501:14.416,1369031561:13.880,1369031621:13.546,1 +369031681:14.026,1369031741:14.028,1369031801:13.786,1369031861:14.67 +9,1369031921:14.675,1369031981:14.748,1369032041:14.684,1369032101:14 +.819,1369032161:13.963,1369032221:14.748,1369032281:14.654,1369032341 +:14.552,1369032401:13.910,1369032461:14.252,1369032521:14.548,1369032 +581:13.596,1369032641:13.500,1369032701:14.097,1369032761:14.138,1369 +032821:13.933,1369032881:14.463,1369032941:13.880,1369033001:13.767,1 +369033061:14.091,1369033121:14.506,1369033181:55.105,1369033241:77.63 +8,1369033301:83.765,1369033361:81.829,1369033421:72.341,1369033481:66 +.173,1369033541:68.379,1369033601:66.550,1369033661:66.077,1369033721 +:58.707,1369033781:57.122,1369033841:56.844,1369033901:58.719,1369033 +961:55.067,1369034021:55.854,1369034081:55.910,1369034141:45.594,1369 +034201:47.046,1369034261:42.591,1369034321:39.168,1369034381:38.360,1 +369034441:39.604,1369034501:39.770,1369034561:29.871,1369034621:14.39 +8,1369034681:13.794,1369034741:13.681,1369034801:13.778,1369034861:13 +.505,1369034921:13.547,1369034981:13.439,1369035041:13.779,1369035101 +:14.100,1369035161:13.855,1369035221:13.931,1369035281:13.745,1369035 +341:14.076,1369035401:14.047,1369035461:13.646,1369035521:14.372,1369 +035581:14.652,1369035641:13.895,1369035701:14.405,1369035761:14.007,1 +369035821:13.716,1369035881:13.977,1369035941:13.766,1369036001:14.05 +1,1369036061:14.580,1369036121:14.230,1369036181:13.726,1369036241:13 +.537,1369036301:14.422,1369036361:14.169,1369036421:13.992,1369036481 +:13.968,1369036541:14.315,1369036601:14.132,1369036661:14.408,1369036 +721:14.829,1369036781:52.669,1369036841:70.333,1369036901:70.492,1369 +036961:68.005,1369037021:68.389,1369037081:67.737,1369037141:68.729,1 +369037201:62.845,1369037261:58.456,1369037321:57.772,1369037381:57.09 +6,1369037441:58.909,1369037501:55.878,1369037561:56.278,1369037621:54 +.403,1369037681:44.911,1369037741:51.638,1369037801:42.755,1369037861 +:39.530,1369037921:38.493,1369037981:43.135,1369038041:39.832,1369038 +101:25.812,1369038161:14.220,1369038221:15.885,1369038281:15.476,1369 +038341:15.244,1369038401:15.062,1369038461:14.688,1369038521:15.168,1 +369038581:14.319,1369038641:14.755,1369038701:14.932,1369038761:14.87 +4,1369038821:14.439,1369038881:14.102,1369038941:18.789,1369039001:14 +.903,1369039061:14.821,1369039121:18.126,1369039181:15.419,1369039241 +:14.261,1369039301:14.813,1369039361:14.443,1369039421:14.278,1369039 +481:14.677,1369039541:15.120,1369039601:14.854,1369039661:14.810,1369 +039721:14.132,1369039781:13.780,1369039841:14.223,1369039901:14.153,1 +369039961:14.257,1369040021:13.852,1369040081:14.338,1369040141:14.26 +0,1369040201:14.804,1369040261:15.070,1369040321:15.185,1369040381:49 +.579,1369040441:70.095,1369040501:71.189,1369040561:69.298,1369040621 +:68.140,1369040681:67.566,1369040741:66.812,1369040801:64.008,1369040 +861:58.261,1369040921:56.311,1369040981:56.421,1369041041:58.461,1369 +041101:54.623,1369041161:55.943,1369041221:54.111,1369041281:43.992,1 +369041341:50.495,1369041401:41.068,1369041461:40.422,1369041521:37.30 +0,1369041581:42.561,1369041641:39.030,1369041701:22.212,1369041761:14 +.956,1369041821:15.394,1369041881:14.986,1369041941:15.174,1369042001 +:14.771,1369042061:15.212,1369042121:14.845,1369042181:14.719,1369042 +241:14.819,1369042301:14.288,1369042361:14.947,1369042421:15.005,1369 +042481:15.151,1369042541:15.090,1369042601:14.851,1369042661:18.641,1 +369042721:15.149,1369042781:14.404,1369042841:15.294,1369042901:16.31 +1,1369042961:15.848,1369043021:14.311,1369043081:15.149,1369043141:14 +.756,1369043201:14.668,1369043261:14.537,1369043321:14.132,1369043381 +:14.363,1369043441:15.085,1369043501:14.834,1369043561:14.396,1369043 +621:14.815,1369043681:14.609,1369043741:14.427,1369043801:14.261,1369 +043861:14.693,1369043921:14.963,1369043981:60.078,1369044041:72.379,1 +369057001:30.920,1369057061:13.895,1369057121:14.537,1369057181:14.80 +9,1369057241:14.049,1369057301:14.135,1369057361:14.348,1369057421:14 +.367,1369057481:14.814,1369057541:14.421,1369057601:14.967,1369057661 +:13.854,1369057721:14.221,1369057781:13.689,1369057841:14.220,1369057 +901:14.334,1369057961:13.924,1369058021:14.689,1369058081:14.682,1369 +058141:14.045,1369058201:13.904,1369058261:14.040,1369058321:14.141,1 +369058381:66.762,1369058441:73.110,1369058501:72.215,1369058561:69.17 +6,1369058621:68.512,1369058681:69.094,1369058741:68.546,1369058801:62 +.561,1369058861:58.377,1369058921:56.169,1369058981:56.042,1369059041 +:59.492,1369059101:55.988,1369059161:56.316,1369059221:52.758,1369059 +281:44.112,1369059341:51.951,1369059401:44.943,1369059461:41.088,1369 +059521:38.413,1369059581:43.005,1369059641:40.294,1369059701:28.510,1 +369059761:14.894,1369059821:14.942,1369059881:15.271,1369059941:14.90 +5,1369060001:14.808,1369060061:14.104,1369060121:14.660,1369060181:15 +.761,1369060241:14.501,1369060301:14.398,1369060361:14.238,1369060421 +:14.459,1369060481:14.238,1369060541:15.095,1369060601:14.375,1369060 +661:14.686,1369060721:14.594,1369060781:14.531,1369060841:14.326,1369 +060901:14.704,1369060961:15.038,1369061021:14.911,1369061081:14.501,1 +369061141:14.623,1369061201:15.288,1369061261:16.899,1369061321:15.99 +9,1369061381:15.038,1369061441:15.132,1369061501:16.103,1369061561:15 +.455,1369061621:15.189,1369061681:15.364,1369061741:15.640,1369061801 +:15.305,1369061861:15.264,1369061921:15.059,1369061981:50.819,1369062 +041:69.295,1369062101:70.318,1369062161:67.842,1369062221:68.617,1369 +062281:67.026,1369062341:68.732,1369062401:61.366,1369062461:57.574,1 +369062521:56.607,1369062581:56.938,1369062641:57.131,1369062701:54.70 +3,1369062761:55.190,1369062821:51.926,1369062881:42.845,1369062941:50 +.557,1369063001:39.261,1369063061:40.198,1369063121:39.238,1369063181 +:41.696,1369063241:41.303,1369063301:18.791,1369063361:16.841,1369063 +421:16.031,1369063481:15.589,1369063541:17.072,1369063601:15.761,1369 +063661:15.639,1369063721:15.153,1369063781:15.616,1369063841:14.895,1 +369063901:15.743,1369063961:14.856,1369064021:14.954,1369064081:15.19 +3,1369064141:16.026,1369064201:15.029,1369064261:14.980,1369064321:15 +.749,1369064381:15.543,1369064441:15.820,1369064501:15.071,1369064561 +:16.616,1369064621:17.064,1369064681:16.043,1369064741:15.370,1369064 +801:14.786,1369064861:15.860,1369064921:15.867,1369064981:14.991,1369 +065041:14.497,1369065101:15.930,1369065161:15.560,1369065221:15.599,1 +369065281:15.670,1369065341:15.346,1369065401:15.323,1369065461:15.11 +4,1369065521:15.384,1369065581:68.885,1369065641:83.585,1369065701:81 +.906,1369065761:73.466,1369065821:67.976,1369065881:67.424,1369065941 +:66.611,1369066001:67.619,1369066061:61.462,1369066121:57.989,1369066 +181:57.020,1369066241:57.223,1369066301:57.573,1369066361:55.437,1369 +066421:55.352,1369066481:54.009,1369066541:44.091,1369066601:52.213,1 +369066661:42.415,1369066721:39.611,1369066781:39.044,1369066841:42.53 +6,1369066901:40.011,1369066961:22.282,1369067021:16.403,1369067081:16 +.099,1369067141:15.336,1369067201:15.770,1369067261:16.230,1369067321 +:15.664,1369067381:15.580,1369067441:14.955,1369067501:15.521,1369067 +561:15.234,1369067621:15.995,1369067681:15.207,1369067741:15.334,1369 +067801:15.110,1369067861:14.998,1369067921:15.148,1369067981:14.895,1 +369068041:14.885,1369068101:15.320,1369068161:15.773,1369068221:15.57 +6,1369068281:15.958,1369068341:15.922,1369068401:16.010,1369068461:14 +.796,1369068521:14.445,1369068581:15.768,1369068641:15.976,1369068701 +:15.175,1369068761:14.558,1369068821:15.737,1369068881:16.408,1369068 +941:15.897,1369069001:15.356,1369069061:15.939,1369069121:15.230,1369 +069181:64.849,1369069241:72.844,1369069301:73.322,1369069361:70.824,1 +369069421:69.694,1369069481:68.373,1369069541:68.061,1369069601:65.06 +6,1369069661:59.042,1369069721:56.941,1369069781:57.079,1369069841:59 +.631,1369069901:56.832,1369069961:55.916,1369070021:54.187,1369070081 +:43.520,1369070141:49.716,1369070201:43.522,1369070261:41.314,1369070 +321:39.311,1369070381:43.091,1369070441:40.524,1369070501:27.812,1369 +070561:16.213,1369070621:16.770,1369070681:15.385,1369070741:15.057,1 +369070801:16.063,1369070861:15.595,1369070921:15.404,1369070981:15.22 +4,1369071041:15.245,1369071101:15.615,1369071161:15.581,1369071221:15 +.537,1369071281:14.934,1369071341:15.243,1369071401:15.140,1369071461 +:15.029,1369071521:15.257,1369071581:14.728,1369071641:15.341,1369071 +701:15.986,1369071716:15.027</counter-data> </perf-counter-data> </counters> </perf-instance-counter-data>

    Output is abbreviated for size.

      :) Dude , we're still left to interpret your code with our minds , change it with our minds, in hopes to correlate to your wanted output -- we can't run/test the code -- or start rewriting until we get something we can run (duplication) -- thats too much work

      Here is a start towards that, separates the data from $server_ctx .... so you can focus on synchronization

      Tip, use the download link on the side

      #!/usr/bin/perl -- use strict; use warnings; use Data::Dump; use Text::CSV; my $object = { 'instance-name' => 'aggr_32_01', 'object-id' => '35441', 'counters' => [
      ], }; dd $object; $_->{'counter-data'} = uncsv($_->{'counter-data'}) for @{$object->{cou +nters}}; dd $object; $_->{'counter-data'} = synch_that_time( $_->{'counter-data'} ) for @{$ +object->{counters}}; dd $object; sub uncsv { ## TODO Text::CSV maybe [ split ',', $_[0] ]; } __END__
Re: How do I synchronize data to common timestamp
by naChoZ (Curate) on May 27, 2013 at 03:10 UTC

    By no means an answer, but just food for thought. One thing you might consider is to just keep the timestamps as they are not worry so much about normalizing them up front.

    I was coming up with a way of modeling stats for authenticated smtp spam and was trying to get a breakdown of hourly counts per relay ip and per customer email address. It ended up being much simpler to just store the raw values (epoch timestamp, ip, email) in a mysql table and then put some of the heavy lifting on the sql side.

    The date handling you can use in mysql is quite good. I used statements like below to give me a very simple-to-process array of data on which to report.

    So, to give myself the last the day's worth of data broken down by hour, I would do something like this:

    SELECT timestamp, CONVERT_TZ(FROM_UNIXTIME(timestamp - timestamp % 3600 ), '+00:00', + '-5:00') AS Hourstamp, CONVERT_TZ(FROM_UNIXTIME(timestamp), '+00:00', '-5:00') AS 'Datest +ampEDT', FROM_UNIXTIME(timestamp) AS 'Datestamp', relay, email FROM relay_report WHERE FROM_UNIXTIME(timestamp) >= DATE_SUB(NOW(), INTERVAL 23 HOUR)

    The imporant bit is the modulus math. This would give me the rows with Hourstamp being the time at the top of the current row's hour, the raw datestamp in Eastern time (yes, I eventually got around to installing the time zones in mysql and fixed this...), the raw datestamp (all our infrastructure uses GMT), and then the relay ip and email address. I could then easily group things by that hour stamp and have a nice breakdown of hourly rates. I had other things I wanted to do with the data while I was processing it which is why I didn't just do a GROUP BY right in the sql statement.

    My primary goal was to come up with a way of detecting compromised customer accounts that were being used to relay spam via authenticated smtp, but I also was giving other departments a way to look at these stats. Doing it this way proved ridiculously easy to just run Excel with the odbc connector to pull data directly, because it was then trivial to create a quick pivot table and a few graphs and charts that were good for a few oohs and aahs on the conference room projector. (Plus date and time stamp handling in Excel is so effing difficult I half expected the Excel elites in there to hold me down with a blanket and pummel me with thick manuals because I'd handled the timestamps perfectly before the data ever got to Excel.)

    Given your raw sample data, I'd probably do something similar. Start stuffing your epoch timestamp, key, and value into db rows as-is. Then select the info you want using a similar method to what I demonstrated. Play with the modulus math if you want to group by smaller increments, 60 seconds or whatever.

    --
    Andy

      Ok thanks for the thoughts guys.

      I could probably upload my scripts and associated modules but I can't upload the SDK here without all sorts of legal hoop issues so unless you have that and a NetApp environment it would be hard to test. I'll keep plodding away and see what I can manage with Data::Dumper

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1035037]
Approved by marto
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (11)
As of 2014-10-30 23:06 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    For retirement, I am banking on:










    Results (211 votes), past polls