Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery

Re: stripped punctuation

by ambrus (Abbot)
on Oct 06, 2005 at 20:58 UTC ( #498050=note: print w/replies, xml ) Need Help??

in reply to stripped punctuation

Basically, you have to take whitespace-separated words, removing non-alphanumeric (i.e. punctation) characters from the beginning or the end. Punctation is allowed inside a word. What makes this a bit more complicated is that in some languages, the space os omitted around em dashes, so you have to consider em-dashes as whitespace too. Thus, it's easier to define what separates words than what counts as a word. Here's a simple solution.

#!perl use warnings; use strict; my # I hate to say this, as it's not mine $text = q{ "'My Dear Mr. Sherlock Holmes:--"Lord Backwater tells me that I may place implicit reliance upon your judgement and discretion. I have determined, therefore, to call upon you and to consult you in referenc +e to the very painful event which has occurred in connection with my wedding. Mr. Lestrade, of Scotland Yard, is acting already in the matt +er, but he assures me that he sees no objection to your co-operation, and that he even thinks that it might be of some assistance. I will call at four o'clock in the afternoon, and, should you have any other engagement at that time, I hope that you will postpone it, as this matter is of paramount importance. Yours faithfully, ST. SIMON.' -- Sir Arthur Conan Doyle, The Adventures of Sherlock Holmes. 1892. }; use locale; my $stuff_between_words = qr{ [^[:alnum:][:space:]]* (?: [[:space:]]+ | --+ | ^ ) [^[:alnum:]]* }x; my @words = split $stuff_between_words, $text; shift @words; # remove the empty word from the beginning print join(" ", map "<<$_>>", @words), "\n"; __END__

However, if I'd really need a fast solution, I'd use flex.

Update: on second thought, it'd be better to use just -- instead of --+.

Replies are listed 'Best First'.
Re^2: stripped punctuation
by thealienz1 (Pilgrim) on Oct 06, 2005 at 21:32 UTC
    Thanks I will take a look at flex.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://498050]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others browsing the Monastery: (9)
As of 2019-11-22 18:41 GMT
Find Nodes?
    Voting Booth?
    Strict and warnings: which comes first?

    Results (113 votes). Check out past polls.