Re^5: Log Parsing

[tybalt89]

That should all fit in memory as one string easily.

[piyushmnnit06]

this is what I got as error message :"Complex regular subexpression recursion limit (32766) exceeded at kafka3.pl line 12, <DATA> line 1."

[tybalt89]

Try with ?'s like this:

#!/usr/bin/perl -l

# http://perlmonks.org/?node_id=1186713

use strict;
use warnings;

$_ = do { local $/; <DATA> };

print "topic,start_time,Endtime";

print join ',', $3,  $2, $4 while
  /\b(\d+)-SUCCESSFUL
  (?=
    (?:.*\n)*?
    \d+\ (\S+).*Started.*\b(\d+_\1_\d+_IN_0)
    (?:.*\n)*?
    \d+\ (\S+).*Done.*\b\3
  )/gx;

__DATA__
0317 09:53:14.865+0000 {12772} INFO  [pm-worker-exec slot-Task:id=8274
+,env=12772,type=11][c.s.w.t.f.s.PostExecutionStage  ] Loaded {child r
+unId vs completion type}: {8286-SUCCESSFUL}{8287-SUCCESSFUL}{8288-SUC
+CESSFUL}{8289-SUCCESSFUL}{8290-SUCCESSFUL}{8291-SUCCESSFUL}{8292-SUCC
+ESSFUL}{8293-SUCCESSFUL}{8294-SUCCESSFUL}{8295-SUCCESSFUL}{8296-SUCCE
+SSFUL}
0317 09:54:12.498+0000 {12772} INFO  [pm-worker-exec slot-Task:id=8273
+,env=12772,type=55][edProcessInputBatchKafkaProducer] Started produci
+ng records on topic 12772_8286_20170317_IN_0
0317 09:54:13.428+0000 {12772} INFO  [pm-worker-exec slot-Task:id=8273
+,env=12772,type=55][edProcessInputBatchKafkaProducer] Started produci
+ng records on topic 12772_8287_20170317_IN_0
0317 09:55:13.027+0000 {12772} INFO  [pm-worker-exec slot-Task:id=8273
+,env=12772,type=55][edProcessInputBatchKafkaProducer] Done with produ
+cing records on topic 12772_8286_20170317_IN_0
0317 09:55:15.027+0000 {12772} INFO  [pm-worker-exec slot-Task:id=8273
+,env=12772,type=55][edProcessInputBatchKafkaProducer] Done with produ
+cing records on topic 12772_8287_20170317_IN_0