Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Large file processing

by arivu198314 (Sexton)
on Oct 01, 2010 at 03:44 UTC ( #862898=perlquestion: print w/replies, xml ) Need Help??
arivu198314 has asked for the wisdom of the Perl Monks concerning the following question:

I have a file with 50,000 lines of find and replace string.

for example
/Test/Sample/
/A/X/
...

Now i want to process the file with input file.

I have tried with usual method, it takes more than 1 hour.

Please advice.

|Arivu|

Replies are listed 'Best First'.
Re: Large file processing
by GrandFather (Sage) on Oct 01, 2010 at 04:06 UTC

    Show us your code. For a standard sort of "read the file line by line, edit each line and write it out" I would expect a modest size file such as you imply to be processed in a matter of seconds.

    True laziness is hard work
      Indeed, and even faster if the entire file is loaded into memory at once, processed, and dumped.
        My code is here.
        #Usage: <Support-File> <Input-File> undef $/; open(TEXT, $ARGV[0]) or die $!; my $text = <TEXT>; close TEXT; open(XML, $ARGV[1]) or die $!; my $xml = <XML>; close XML; while($text =~ m/\/(.+)\/([^\/]+)\/[^\/]+$/mgi) { my $FindWord = $1; my $ReplaceWord = $2; $xml =~ s/(>[^>]*\b)\Q$FindWord\E(\b[^>]*<)/$1$ReplaceWord$2/gi; } open(OUTXML, ">$ARGV[1]") or die $!; print OUTXML $xml; close OUTXML;

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://862898]
Approved by ikegami
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (4)
As of 2018-10-18 03:05 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    When I need money for a bigger acquisition, I usually ...














    Results (99 votes). Check out past polls.

    Notices?