Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Re^4: Out of memory error!!!

by sathya83aa (Acolyte)
on Mar 28, 2013 at 08:59 UTC ( #1025908=note: print w/ replies, xml ) Need Help??


in reply to Re^3: Out of memory error!!!
in thread Out of memory error!!!

Ok Let me make it more clear..Im fetching those variables from a flat file and printing it in the XML file. The order of printing of the variable will differ from how we are getting the variable.

#!/usr/bin/perl

open (my $INFO, "<", "$File") or die("Cannot open the flat file $!");

open(my $LOG,">>LOG_file") or die("Cannot open $!");

open(my $XML,">xml_file") or die("Cannot open $!");

while (defined(my $line2 = $INFO)) {

    $line2 =~ s/&/&amp;/g;

    print "$line2\n";

    my $segment = substr($line2,0,2);

    if($segment eq "00")

    {

    my $Var1 = trim(substr($line2,40,8));

    }

    if($segment eq "A0")

    {

    $Var2 = trim(substr($line2,2,9));

    $Var3 = trim(substr($line2,14,3));

    $Var4 = trim(substr($line2,20,42));

    $Var5 = trim(substr($line2,62,30));

and so on

}

    print XML "<Header>\n";

    print XML "<AAA>VAR1<\/AAA>\n";

    print XML "<BBB>VAR2<\/BBB>\n";

    print XML "<CCC>VAR3<\/CCC>\n";

  print XML "<DDD>VAR4<\/DDD>\n";

  print XML "<EEE>VAR5<\/EEE>\n";

    print XML "<Header>\n";

close $LOG;

close $XML;

close $INFO;

sub trim($)

{

   my $string = shift;

    $string =~ s/^\s+//;

   $string =~ s/\s+$//;

   return $string;

}


Comment on Re^4: Out of memory error!!!
Select or Download Code
Re^5: Out of memory error!!!
by McA (Curate) on Mar 28, 2013 at 10:23 UTC

    How much data do you have to collect to decide in which order you can generate the xml? Do you have to collect all records of the input file? Do you only have to collect a sequence of segments to create a - let's call it - xml record? Sorry, but your excerpt didn't help?

    How big is the input file?

    McA

      I have to collect data from all the lines in the input file and then print it to the XML file.

      Actually the input file is huge,..but now for testing we are using a input file with 30 lines for which we are getting the error.

        ... while (defined(my $line2 = $INFO)) { ...

        ... you need to use the diamond operator to read from a file line by line:

        ... while (defined(my $line2 = <$INFO>)) { ...

        First of all. 30 lines mean nothing when you don't know how long the lines are. We never talked about that. Probably you have a unusual encoding where the line ending is not the same as with unix. Do you really have LF (0x0a) for a line ending?

        Is there some weird preallocation which is unrelated to the count of line which blows your memory resources?

        Can't you give us a little complete running example showing us the error message? Otherwise I don't know how to help you any further.

        McA

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1025908]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others examining the Monastery: (3)
As of 2014-08-02 05:25 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    Who would be the most fun to work for?















    Results (54 votes), past polls