Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Perl script to split a file and process then concatenate based on size and a string

by sharp859 (Initiate)
on Feb 02, 2013 at 06:49 UTC ( [id://1016656]=perlquestion: print w/replies, xml ) Need Help??

sharp859 has asked for the wisdom of the Perl Monks concerning the following question:

Could somebody help me to get some logic to following in perl I am using windows 7.

C:\script>perl split_concatenate.pl large_file a or b (Input would be large file and value a or b to process it later).

Check the file if it is greater than 40KB (some size), and choice is "a" , if not run a command

command -i large_file.txt -o large_file_new -a

else if the choice is b

command -i large_file.txt -o large_file_new -b

else

if it is greater say 40KB+, split the file for each 40KB arround (will be part1,) and append a first "particular string" which will be in the file_part2 to the part1 save it for processing, if there are multiple "particular string" then create subsequent files which should end with next "particular string" in the following part. ("Particular String" starts with some String but ends in different value). So script should search if there are more "Particular string", in the part2 or so and append first available one, if there is only one available no need to anything just split. As file always should end with a particular string.

Then process same command

command -i filepart1.txt -o filepart1.dat -a command -i filepart2.txt - o filepart2.dat (if needed) -a

or

command -i filepart1.txt -o filepart1.dat -b command -i filepart2.txt - o filepart2.dat (if needed) -b

After this needs to be concatenated.

Concatenate filepart1.dat + filepart2.dat + filepartN =large_file.dat

I started to find the size first using below code,

 #!/usr/bin/perl  use strict; use warnings;  use File::stat;   my $filesize = stat("Full_File.txt")->size; print "Size: $filesize\n"; exit 0;

It will be great if some one help so I can learn. If this is not possible, then @ each 500th line the file reaches to 40KB, so I think this would be easier, every 500th line append the Next available "Particular String", and split and process above command, if file is less than 1000 lines then only 2 split and no need to append in the part2 as already it has one in its eof. May be easier?

Thanks a lot

Replies are listed 'Best First'.
Re: Perl script to split a file and process then concatenate based on size and a string (@ARGV)
by Anonymous Monk on Feb 02, 2013 at 07:59 UTC
      I am reading it, but it would take some time any idea now.

        I am reading it, but it would take some time any idea now.

        My idea is for you to read the links I gave you, start with the first links , the book covers the same material

Re: Perl script to split a file and process then concatenate based on size and a string
by Anonymous Monk on Feb 03, 2013 at 10:26 UTC

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1016656]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others having an uproarious good time at the Monastery: (6)
As of 2024-04-23 14:19 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found