That silently ignores whitespace (read up on <skip>).
Also, P::RD is rather slow. I'd even say inexcusably slow if you're just using it as a tokenizer. May I suggest a much faster tokenizer?
use strict;
use warnings;
sub process_token {
my ($token) = @_;
print("Token: $token\n");
}
{
my $str = "ABC[GHI]XY[Z]1A";
for ($str) {
/\G \[ ([A-Z]*) \] /xgcs && do {
process_token("$1");
redo
};
/\G ([A-Z]) /xgcs && do {
process_token("$1");
redo
};
/\G (.) /xgcs && do {
printf("Unexpected '%s' at pos %d\n", $1, pos()-length($1));
redo
};
}
}