Re: Remove duplicate from the same line..

Are you getting this duplication only on the Company Name line? If so, wrap the solution from Athanasius in an if test when you read the line from your file. Otherwise you can damage any address information as warned by Athanasius. Can you get a look at the system that is causing the problem in the first place? That might be your better long term solution..

Comment on Re: Remove duplicate from the same line..

Replies are listed 'Best First'.
Re^2: Remove duplicate from the same line.. by Anonymous Monk on Jun 01, 2013 at 15:36 UTC
Thx. Athanasius... That was good. but now I got 1 more problem, I have a company name like "Goldman Sachs Group, ... Goldman Sachs Group, Inc." Here I want only 'Goldman Sachs Group'. Is there any option for that? Basically If a word appear second time in a single line delete the rest of the line including that word and trim out if any , or . is present!. Is that possible? & Thx rpnoble, I will do it like that... :) Thx.	[reply]
Re^3: Remove duplicate from the same line.. by gam3 (Curate) on Jun 01, 2013 at 18:51 UTC
As a counter example I submit: 'Smith Smith & Feeley LLP' -- gam3 A picture is worth a thousand words, but takes 200K.	[reply]
Re^4: Remove duplicate from the same line.. by hdb (Monsignor) on Jun 02, 2013 at 06:12 UTC
This is a very good objection to the whole exercise. There is not really any way to know whether `Smith Smith, Inc` is a duplication or a valid company name. Without some real world knowledge I cannot see a way to distinguish between the two. As a remediation one could write all replacements into a log file for review and build a list of exceptions.	[reply] [d/l]
Re^5: Remove duplicate from the same line.. by sundialsvc4 (Abbot) on Jun 02, 2013 at 17:05 UTC
Re^3: Remove duplicate from the same line.. by hdb (Monsignor) on Jun 01, 2013 at 17:00 UTC
Adding `.*` after `\1` in Atanasius' solution should do the trick as it matches everything up to the trailing `\n`.	[reply] [d/l] [select]
Re^4: Remove duplicate from the same line.. by Anonymous Monk on Jun 01, 2013 at 17:28 UTC
No, it didnt worked!	[reply]
Re^5: Remove duplicate from the same line.. by hdb (Monsignor) on Jun 01, 2013 at 18:26 UTC


Problems? Is your data what you think it is?
	PerlMonks