Sorting files you Have read

brusimm has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: Sorting files you Have read by GrandFather (Saint) on Nov 16, 2006 at 21:16 UTC
That's a bit like asking "How should I drink?". It depends a great deal on what it is you are drinking! First up: What does your data look like? How big is the file? How often does it need to be done? Is sort time important? Will the sorted data be reused in some fashion? Actually, many of those are related to each other. Because you were good and supplied some code, I'll show you some: `my @contents = sort <SOMELIST>; print @contents;` [download] and because I'm kind I'll give you some hints: Check through the Tutorials section Read up about sort Study Schwartzian Transform and the replies to What is "Schwarzian Transform" (aka Schwartzian) Super Search "sort column" in SoPW and read some of the wealth of material already posted DWIM is Perl's answer to Gödel	[reply] [d/l]
Re^2: Sorting files you Have read by brusimm (Pilgrim) on Nov 16, 2006 at 21:30 UTC
My file is comma delimited about 50 lines of text How often - for my initial purposes, to run and make it happen. At the moment, due to the small file size, sort time is unimportant. At some point, when I get to that stage, the sorted data will be reused. I tried your code, and it seems quite simple, hence, efective, BUT It runs with no errors, but nothing prints Neither to the screen or to a file. Thank you - I have read through various tutorials, and sort and it's various ways of handling data, i saw the Schwartzian Transform, but it made my brain hurt at this point in time.. remember, newbie here and either I did not operate it right, or there is little on the simpler process, but super search did not turn up anything I could make sense of... again, newbie, injured brain, etc, etc. Thanks.	[reply]
Re^3: Sorting files you Have read by GrandFather (Saint) on Nov 16, 2006 at 21:56 UTC
Ok, lets give that "simple effective" sample code some data: `use strict; use warnings; my @contents = sort <DATA>; print @contents; __DATA__ At the moment, due to the small file size, sort time is unimportant. At some point, when I get to that stage, the sorted data will be reuse +d. I tried your code, and it seems quite simple, hence, efective. BUT, It runs with no errors, but nothing prints.` [download] Prints: `At some point, when I get to that stage, the sorted data will be reuse +d. At the moment, due to the small file size, sort time is unimportant. BUT, It runs with no errors, but nothing prints. I tried your code, and it seems quite simple, hence, efective.` [download] which is sorted on the whole line. It works, but ain't what you want. So lets add in some brain hurty code to sort by the "second column": :) use strict; use warnings; my @contents = map { $_->[0] } sort { $a->[1] cmp $b->[1] } map { [$_, extractColumn (1, $_)] } <DATA>; print @contents; sub extractColumn { my ($columnIndex, $line) = @_; my ($key) = $line =~ /(?:[^,],){$columnIndex}([^,])/; return $key; } __DATA__ At the moment, due to the small file size, sort time is unimportant. At some point, when I get to that stage, the sorted data will be reuse +d. I tried your code, and it seems quite simple, hence, efective. BUT, It runs with no errors, but nothing prints. [download] Prints: `BUT, It runs with no errors, but nothing prints. I tried your code, and it seems quite simple, hence, efective. At the moment, due to the small file size, sort time is unimportant. At some point, when I get to that stage, the sorted data will be reuse +d.` [download] However if you are dealing with csv (comma separated variable) data then you really want to be using a module such as Text::CSV to read the file. You may like to check out a few nodes that have asked the "sort CSV" question before (Super Search SoPW remember): Sorting a CSV file and sorting CSV files may help too. DWIM is Perl's answer to Gödel	[reply] [d/l] [select]
Re^4: Sorting files you Have read by brusimm (Pilgrim) on Nov 16, 2006 at 23:09 UTC
Re^3: Sorting files you Have read by markh (Scribe) on Nov 16, 2006 at 22:25 UTC
I'll modify your example code to help you do some simple sorting. Now, this is likely not the most efficient way to do it, but since you have a small dataset and you are new to perl, this can help you get started. `use strict; use warnings; open (SOMELIST, "somelist") or die "Cannot open file $!\n"; my %sort_data; while (my $record = <SOMELIST>) { my @one_line = split(/,/, $record); while (exists $sort_data{$one_line[1]}) { $one_line[1] = "$one_line[1]" . " "; #add a blank for + uniqueness } $sort_data{$one_line[1]} = $record; #store it by 2nd column } close (SOMELIST); foreach my $line (sort {$a cmp $b} keys %sort_data) { print "$sort_data{$line}"; }` [download] Like I said, this isn't the most efficient or even best method, but it is simple enough that you can hopefully see what is going on. (2006-11-18 17:21 GMT) Edited my perl code to remove a couple of syntax errors. - Thanks Grandfather for pointing them out.	[reply] [d/l]
Re^4: Sorting files you Have read by GrandFather (Saint) on Nov 16, 2006 at 23:07 UTC
Re: Sorting files you Have read by swampyankee (Parson) on Nov 16, 2006 at 21:24 UTC
First suggestion: use your system's sort routine. While I think the ix sort is better than the Windows sort, both are reliable, and quite fast. As for sorting a list in Perl: `use strict; # a good idea use warnings; # and another my $input = 'myinput.txt'; open(my $in, "<", $input) or die "Could not open $input because $!\n"; @unsorted = <$in>; chomp(@unsorted); # get rid of end-of-record markers close($in); @sorted = sort @unsorted;` [download] will sort the @unsorted array, based on the entire string, with the output going to @sort. Note that this requires a copy of the entire* file be in memory, which may be a Bad Idea. There are several tutorials, for example, here and here. And of, course, here. emc At that time [1909] the chief engineer was almost always the chief test pilot as well. That had the fortunate result of eliminating poor engineering early in aviation. —Igor Sikorsky, reported in AOPA Pilot magazine February 2003.	[reply] [d/l]
Re^2: Sorting files you Have read by GrandFather (Saint) on Nov 16, 2006 at 23:16 UTC
If you are going to `use strict; use warnings;` (and it is a really really good idea), then you really really need to declare your variables: ;) `use strict; # a good idea use warnings; # and another my $input = 'myinput.txt'; open(my $in, "<", $input) or die "Could not open $input because $!\n"; my @unsorted = <$in>; chomp (@unsorted); # get rid of end-of-record markers close ($in); my @sorted = sort @unsorted;` [download] DWIM is Perl's answer to Gödel	[reply] [d/l] [select]
Re^3: Sorting files you Have read by brusimm (Pilgrim) on Nov 16, 2006 at 23:34 UTC
Thank you, everyone This points me in the right direction I used to do IDL and Fortran (About 7 years ago..) but this seems harder. Sheesh.. is it me being out of the programming loop so long, or is see-nellity setting in? Maybe you shouldn't answer that!! Take care.	[reply]
Re^4: Sorting files you Have read by robot_tourist (Hermit) on Nov 17, 2006 at 16:48 UTC
Re^2: Sorting files you Have read by brusimm (Pilgrim) on Nov 16, 2006 at 23:07 UTC
Hey swampyankee, thank you, but when I run the script, I get the following Global symbol "@unsorted" requires explicit package name at one-c.pl line 6. Global symbol "@unsorted" requires explicit package name at one-c.pl line 7. Global symbol "@sorted" requires explicit package name at one-c.pl line 10. Global symbol "@unsorted" requires explicit package name at one-c.pl line 10. Execution of one-c.pl aborted due to compilation errors. I am on an XP machine, if that has any bearing, but would there be something else amiss with my installation? The reason i ask is my first responder suggested some code and that looked like it ran successfully, but did not translate any result to the screen or a file. Now your code returns errors... hmm Any other thoughts?	[reply]
Re^3: Sorting files you Have read by swampyankee (Parson) on Nov 16, 2006 at 23:36 UTC
I cleverly forgot to declare @unsorted and @sorted. This is why one should (a) test code before posting and (b) use strict; and use warnings;. They do wonders to prevent things like: `@array = (1,2,3,4,5,6,7,8,9); $sum+= $_ foreach @array; $mean = $sum/scalar(@aray);` [download] (Yes, I know scalar(@aray) is 0.) emc At that time [1909] the chief engineer was almost always the chief test pilot as well. That had the fortunate result of eliminating poor engineering early in aviation. —Igor Sikorsky, reported in AOPA Pilot magazine February 2003.	[reply] [d/l]
Re: Sorting files you Have read by Fletch (Bishop) on Nov 16, 2006 at 21:22 UTC
Also consider that TMTOWTDI which don't involve perl. See the manual page for the `sort` utility if you're on some flavour of *NIX (or can get a hold of Cygwin in Wintendo land).	[reply]


more useful options
	PerlMonks