Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Parsing log file with regular-expression match nested brackets [RESOLVED]

by thanos1983 (Parson)
on Feb 28, 2017 at 15:48 UTC ( [id://1183152]=perlquestion: print w/replies, xml ) Need Help??

thanos1983 has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks,

I am trying to parse multiple log files where eventually will be split and stored on an array of arrays. Each file will have an array for it self and the secondary level of each array will have all lines split based on the log file input.

For example imagine:

$VAR1 = [
          [
            'line1',
            'line2',
            'linen',
          ],
          [
            'line1',
            'line2',
            'linen',
          ]
        ];

[download]
But let's take it one step at a time. I have big log files (4000 lines) that contain multiple uneven (smaller or bigger) data such as: 
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx: UDT{
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine2: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine3: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine4: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine5: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine6: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine7: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne8: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne9: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne10 data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne11: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne12: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne13: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne14: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne15: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne16: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne17: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne18: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne19: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne 20 {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine21 data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            lin
+e22 data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            lin
+e23 data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            lin
+e24: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            lin
+e25: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line26 = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line27: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line29 data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line30: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line32: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line33: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line34: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line35: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line36: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line38: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line39: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line40: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line42 =    {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line43: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line44: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line45: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line46: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line47: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line48: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line49: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line50: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line51: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line52: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line53: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line55: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line56: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line57: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line58: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line59: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line60 = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line61: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line63: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line65 = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line66 = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line67: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line69: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line70: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line72 = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line73: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line74: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line75: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line76: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line77: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line78: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line79: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line80: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line81: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line82: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line84: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line85: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line86: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line87: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line88: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line89: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line90: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line91: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line92: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line93: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line94: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line95: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line98: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line99: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line100: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line104: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line105: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line106: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line107: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            lin
+e110: data

[download]

The common reference point of the data is that all start with ([     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx: UDT{ and end } after multiple open and close curly brackets in and out).

I am at the beginning of writing the script and I am just parsing the file and splitting the files into array elements based on each matching ([     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx: UDT{ and end } after multiple open and close curly brackets in and out).

It seems to be working fine and yet perfect, because I am really bad with perlre. I manage to found a similar case of my problem from Perl regular expression: match nested brackets , where I used the my @matches = $pdus =~ /[^{}\s]+ | \{ (?: (?R) | [^{}]+ )+ \} /gx;. It seems to working fine apart that it can process the beginning of the log file properly and the end.

The log file contains multiple instances of the sample given above that always start with [     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx: UDT{ and end with } but in between multiple brackets.

Sample of script:

#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;

die "Please provide minimum one file for parsing!\n" if @ARGV < 1;

sub fileProcessing {
    my ( @files ) = @_;
    foreach my $file ( @files ) {
    open(my $fh, '<' , $file)
        or die "Could not open file '$file' $!";

    my $pdus = do { local $/; <$fh> };

    close $fh
        or die "Could not close file '$file' $!";

    chomp( $pdus );
    # my @matches = $pdus =~ /[^{}\s]+ | \{ (?: (?R) | [^{}]+ )+ \} /g
+x;
        my @matches = $pdus =~ /[^{}]+ | \{ (?: (?R) | [^{}]+ )+ \} /g
+x;

    print Dumper \@matches;
    }
    return;
}

fileProcessing( @ARGV );

[download]

Sample of the output:

$VAR1 = [
          '[',
          '11]',
          '25/2/2017-19:02:06.980',
          'proces_name',
          'thanos-Rx:',
          'UDT',
          '{
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine2: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine3: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine4: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine5: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine6: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine7: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne8: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne9: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne10 data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne11: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne12: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne13: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne14: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne15: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne16: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne17: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne18: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne19: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne 20 {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine21 data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            lin
+e22 data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            lin
+e23 data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            lin
+e24: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            lin
+e25: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line26 = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line27: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line29 data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line30: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line32: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line33: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line34: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line35: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line36: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line38: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line39: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line40: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line42 =    {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line43: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line44: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line45: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line46: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line47: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line48: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line49: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line50: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line51: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line52: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line53: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line55: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line56: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line57: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line58: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line59: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line60 = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line61: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line63: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line65 = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line66 = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line67: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line69: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line70: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line72 = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line73: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line74: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line75: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line76: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line77: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line78: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line79: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line80: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line81: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line82: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line84: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line85: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line86: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line87: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line88: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line89: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line90: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line91: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line92: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line93: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line94: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line95: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line98: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line99: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line100: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line104: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line105: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line106: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line107: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            lin
+e110: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            }'
        ];

[download]

Sample of the updated output:

$VAR1 = [
          '[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx: UDT
+',
          '{
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine2: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine3: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine4: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine5: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine6: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine7: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne8: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne9: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne10 data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne11: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne12: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne13: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne14: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne15: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne16: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne17: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne18: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne19: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne 20 {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine21 data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            lin
+e22 data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            lin
+e23 data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            lin
+e24: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            lin
+e25: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line26 = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line27: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line29 data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line30: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line32: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line33: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line34: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line35: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line36: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line38: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line39: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line40: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line42 =    {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line43: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line44: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line45: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line46: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line47: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line48: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line49: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line50: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line51: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line52: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line53: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line55: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line56: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line57: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line58: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line59: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line60 = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line61: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line63: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line65 = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line66 = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line67: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line69: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line70: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line72 = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line73: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line74: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line75: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line76: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line77: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line78: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line79: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line80: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line81: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line82: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line84: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line85: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line86: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line87: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line88: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line89: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line90: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line91: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line92: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line93: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line94: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line95: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line98: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line99: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line100: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line104: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line105: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line106: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line107: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            lin
+e110: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            }'
        ];

[download]

Sample of desired output:

$VAR1 = [
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx: UDT{
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine2: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine3: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine4: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine5: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine6: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine7: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne8: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne9: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne10 data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne11: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne12: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne13: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne14: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne15: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne16: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne17: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne18: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne19: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:             li
+ne 20 {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine21 data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            lin
+e22 data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            lin
+e23 data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            lin
+e24: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            lin
+e25: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line26 = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line27: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line29 data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line30: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line32: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line33: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line34: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line35: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line36: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line38: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line39: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line40: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line42 =    {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line43: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line44: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line45: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line46: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line47: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line48: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line49: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line50: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line51: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line52: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line53: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     line55: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line56: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line57: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line58: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line59: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line60 = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line61: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line63: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line65 = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line66 = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line67: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line69: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line70: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line72 = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line73: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line74: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line75: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line76: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line77: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line78: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line79: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line80: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line81: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line82: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line84: data = {
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line85: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line86: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line87: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line88: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line89: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line90: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line91: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line92: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line93: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line94: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+             line95: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line98: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line99: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+         line100: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+     }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line104: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line105: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line106: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ line107: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:               
+ }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            }
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            lin
+e110: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:            }'
        ];

[download]

To execute the script:

perl test.pl sampleFile

[download]

If someone could help me to figure out where I am going wrong it would be perfect.

Update: Removing last "unmatched" curly bracket from file input and also modifying printable output.

Update2: Adding modified regex my @matches = $pdus =~ /[^{}]+ | \{ (?: (?R) | [^{}]+ )+ \} /gx;. Updating output.

Thanks in advance for everyone time and effort.

Seeking for Perl wisdom...on the process of learning...not there...yet!

Replies are listed 'Best First'.
Re: Parsing log file with regular-expression match nested brackets
by 1nickt (Canon) on Feb 28, 2017 at 15:57 UTC

    Hi thanos1983,

    Your creativity is boundless! ++ for that. However this seems like a good time to bring up the concept of the XY Problem.

    If you could describe what you are really trying to do with those log files, I'd bet you dollars to donuts that you'll be offered a direct solution.

    Hope this helps!


    The way forward always starts with a minimal test.
[reply]

      Hello 1nickt,

      Well my first step is to read the log file into a string as I am doing. Then as a second step is to split complete entities from log file into separate array elements that start with ([     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx: UDT{) and end with (}).

      As a third step I want them to spit each array element into separate lines based on new line characters (\n) into separate element on sub array. The reason that I want to do that is to have concrete element reference so when I will be using loops with regular expressions rules to search through the array elements it will match.

      My ultimate goal is to match entities "first level array elements" with each other based on similarities with in the lines.

      It sounds a bit of complicated so I will try to simplify it.

      Let say that I have a log file:

      [     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx: UDT{
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine2: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine3: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine4: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine5: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine6: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine7: data
}
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Tx: UDT{
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Tx:              l
+ine10: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Tx:              l
+ine11: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Tx:              l
+ine12: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Tx:              l
+ine13: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Tx:              l
+ine14: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Tx:              l
+ine15: data
}

      [download]

      For simplicity reasons I kept the sample of log file without any complexities.

      So lets assume that: [     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              line7: data contains some data that I want to fine exactly the same match on the log file e.g.: [     11] 25/2/2017-19:02:06.980 proces_name thanos-Tx:              line11: data

      .

      it would be very simple if we wanted to catch the single line but I need to return the whole array containing all the lines that starts [     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx: UDT{ and ends with }.

      So the expected and desired result would be a hash:

      $VAR1 = {
          'Test1' => {
                          '[     11] 25/2/2017-19:02:06.980 proces_nam
+e thanos-Rx: UDT{
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine2: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine3: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine4: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine5: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine6: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Rx:              l
+ine7: data
}'
                        },
          'Test2' => {
                         '[     11] 25/2/2017-19:02:06.980 proces_name
+ thanos-Tx: UDT{
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Tx:              l
+ine10: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Tx:              l
+ine11: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Tx:              l
+ine12: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Tx:              l
+ine13: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Tx:              l
+ine14: data
[     11] 25/2/2017-19:02:06.980 proces_name thanos-Tx:              l
+ine15: data
}'
                       }
        };

      [download]

      I hope my explanation did not confused more it helped you to understand my approach to my problem.

      If you have any further questions please do not hesitate to ask.

      Seeking for Perl wisdom...on the process of learning...not there...yet!
[reply]
[d/l]
[select]

        Well, hm, did you read the node I linked to? You basically restated the content of your first post. What I was suggesting was to state instead a higher-level overview of your task. Because from what I can see, for example, you would be much better off logging into a database, whereupon you select groups of records using an SQL query. But I am guessing, because you haven't yet said something like (guessing again):

        "I'm working on a logging system for my application, and I want to collate the debug output from the client and the server for the same transaction. I'm thinking of parsing the text log file of each using a regexp, but I'm wondering if there's a better way?"

        That's the difference between discussing "X" and discussing "Y".

        Hope this helps!


        The way forward always starts with a minimal test.
[reply]
Re: Parsing log file with regular-expression match nested brackets
by BillKSmith (Monsignor) on Feb 28, 2017 at 19:49 UTC
    If it really is necessary to match nested brackets with a regex, you should use the module Regexp::Common::balanced.
    Bill
[reply]

      Hello BillKSmith,

      The module sounds very promising, but still I have not figure it out the syntax of my regex.

      Thank you for your time and effort reading and replying to my question.

      Seeking for Perl wisdom...on the process of learning...not there...yet!
[reply]
[d/l]
[select]
Re: Parsing log file with regular-expression match nested brackets
by thanos1983 (Parson) on Mar 02, 2017 at 14:27 UTC

    Finally I came up with a solution that works for me.

    I do not know if it will help anyone in future I am always posting my solutions just in case someone would be benefited by it.

    So I observed based on experimentation that regex:

    Read more... (17 kB)

    Also regex:

    Read more... (17 kB)

    The difference between the two regex is that one parses the string into pieces when the curly bracket begins and the second one is parsing the string by ignoring anything before the curly bracket.

    Thanks to poj for his interesting idea to use the first part of the string before the curly bracket as a key it actually was really useful for me. I am using timestamps in the string which helps me to differentiate the keys of a hash and sort them.

    So the final solution to my problem is:

    Read more... (2 kB)

    The final code output is with a test including multiple entries similar to the one that I posted on the question. Due to many lines output I am posting the whole file, but it can be replicated by copying and pasting the same file multiple times and then you need to alter the time-stamp slightly to differentiate them.

    Hope this helps someone else also in the future.

    Seeking for Perl wisdom...on the process of learning...not there...yet!
[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?

www.com | www.net | www.org
Node Status?
node history
Node Type: perlquestion [id://1183152]
Approved by 1nickt
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? • Last hour • Other CB clients
Other Users?
Others studying the Monastery: (5)
As of 2024-04-23 07:35 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found