Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses

File comparison: not diff

by rschuler (Beadle)
on Apr 09, 2002 at 16:16 UTC ( #157760=categorized question: print w/replies, xml ) Need Help??
Contributed by rschuler on Apr 09, 2002 at 16:16 UTC
Q&A  > files


I need to compare two files. These files have a key and a value on each line. I need to note when a key is in one file, and not in the other. I also need to note when the keys match, but the values are different. ex. given:
left-file right-file 1,a 1,a 2,b 3,c 4,f 4,g 5,h 5,h
I need 2 absent from right-file, 3 absent from left-file and 4 is different. diff works so long as difference are separated by one or mores lines with a matching key and value. With the above input files diff produces:
< 2,b < 4,f --- > 3,c > 4,g
Which is not quite right. -Bass has the beginnings of the shell of what I need. Being lazy, I was hoping the CPAN would have the algorithm I need, but I have found nothing. Note that the files are big, 1/2 million lines or so and the key and value are actually multiple fields.

Answer: File comparison: not diff
contributed by thelenm

Here's a little program I whipped up that should do about what you want. Hopefully it should be easy to modify so that you can plug it into whatever your project is. I tested it on the input you gave, but not extensively.

#/usr/bin/perl use strict; use warnings; @ARGV == 2 or die "Must specify 2 files!\n"; my $afile = shift; my $bfile = shift; my $ahash = make_hash($afile); my $bhash = make_hash($bfile); sub make_hash { my $file = shift; my %hash = (); open IN, "<$file" or die "Can't open '$file': $!\n"; while (<IN>) { chomp; my ($key,$val) = split(/,/,$_); $hash{$key} = $val; } return \%hash; } print "In A hash but not B hash:\n", map {"$_\n"} grep {not exists $bhash->{$_}} keys %$ahash; print "In B hash but not A hash:\n", map {"$_\n"} grep {not exists $ahash->{$_}} keys %$bhash; print "In A hash and B hash but different:\n", map {"$_\n"} grep {exists $bhash->{$_} and $ahash->{$_} ne $bhash->{$_ +}} keys %$ahash;
Answer: File comparison: not diff
contributed by graff

I just uploaded a utility I wrote some time ago to handle this sort of thing on the command line. Look for "cmpcol" by graff in the code catacombs (new posting).

It probably doesn't do exactly what you want in your particular case, but it does tend to be very handy for a lot of things...

Here are some sample usages, given your left and right files as input:

# intersection (-i) of key fields: # (default delimiter is \s+, so use "-d ," here) $ cmpcol -i -d , left right 1 4 5 # print those records with the combined line # content from both files: $ cmpcol -i -d , -lb left right 1,a:<>:1,a 4,f:<>:4,g 5,h:<>:5,h # keys only in "left" file, which is "#1" on the # command line (-x1), showing full line (-l1): $ cmpcol -x1 -l1 -d , left right 2,b # lines with identical content (don't use # the comma delimiter in this case): $ cmpcol -i left right 1,a 5,h # all lines that are not identical (this # always shows which file contained each line): $ cmpcol -x left right 2,b <1 3,c <2 4,f <1 4,g <2 # and so on. I think the "usage" message is # the best part (at least I hope so): $ cmpcol Usage: cmpcol {-i|-u|-us|-x|-x1|-x2} [options] file1[:col#] file2[:col +#] Comparison modes: -i : produce intersection of file1[:col#] and file2[:col#] -u(s) : produce union (and identify sources) -x : produce exclusive-or, identifying sources -x1 (-x2) : produce items unique to file1 (or file2) Options: -l{1,2,b} : print whole lines from file1,file2 or both (default: pri +nt column) -g(v) ptn : grep (-v) -- only compare lines that (don't) contain /pt +n/ -c cchar : ignore material following cchar -d delim : use /delim/ as column separator (default is white-space) use "-d tab" for tab-delimited, "-d dot" for period-deli +mited file1 or file2 may be 'stdin' default ':col#' for comparison is first column from each file, which +is ':1'

Please (register and) log in if you wish to add an answer

  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?

    What's my password?
    Create A New User
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others musing on the Monastery: (1)
    As of 2021-02-25 06:03 GMT
    Find Nodes?
      Voting Booth?

      No recent polls found