Debugger line numbering anomaly

[lzipin]

O monks, I've found anomalous behavior with the Perl debugger, and I'd like a sanity check.

Here's a simple script that demonstrates the issue — at least on my laptop with a recent Xubuntu, Perl 5.26.1 and perl5db.pl v 1.51 (my Win7 machine with Strawberry Perl 5.28.2 doesn't have this problem). The script does a global regex replace on its first arg (word).

#!/bin/perl -w
use strict;
$_ = shift // '';  # line 3
s{a([bc])}         # 4
 {d$1}g;           # 5
print;             # 6

Open this in the debugger, l(ist) the script, and the 'print;' line is tagged line 5, not 6. The debugger doesn't find the script's line 5 ... although the script runs fine (e.g., the single arg 'abacadbd' yields 'dbdcadbd'). The debugger applies the commands 'c 5' and 'b 5' to the 'print;' line.

With one or more multiline replace statements in the middle of a sizable script, this gets aggravating. Further, it's a tough problem to Google — a possible bug in the debugger, etc.

Any resonance from No-Mind?

[haukex]

Bisecting this was a bit tricky, but the results are that the regression was accidentally introduced in 6432a58, reported as RT#131930, and fixed in 823ba44 - the bug should therefore only exist in Perl releases v5.26.*.

Read more... bisecting script (2 kB)

[LanX]

> reported as RT#131930

Thanks, the bug report shows that __LINE__ gets confused while parsing and compiling the code, which is not the fault of the debugger.

Of course a warn in the test script would have demonstrated the wrong __LINE__ too.

update

> Bisecting this was a bit tricky,

well, with help of the warn statements hinting to __LINE__, it would have been much easier.

update

turns out that the git on my win-box runs under v5.26, and it's easily demonstrated there:

$ perl -v

This is perl 5, version 26, subversion 2 (v5.26.2) built for x86_64-ms
+ys-thread-multi


$ perl
warn "before";
s(x)
(y);
warn "after";
__END__
before at - line 1.
after at - line 3.

$ perl
warn "before";
s/x/y/;
warn "after";
__END__
before at - line 1.
after at - line 3.

$ perl -MO=Concise
warn "before";
s(x)
(y);
warn "after";
__END__
d  <@> leave[1 ref] vKP/REFC ->(end)
1     <0> enter ->2
2     <;> nextstate(main 1 -:1) v:{ ->3
5     <@> warn[t1] vK/1 ->6
3        <0> pushmark s ->4
4        <$> const[PV "before"] s ->5
6     <;> nextstate(main 1 -:2) v:{ ->7
8     </> subst(/"x"/) vK ->9
7        <$> const[PV "y"] s ->8
9     <;> nextstate(main 1 -:3) v:{ ->a         # should be :4 not :3
c     <@> warn[t2] vK/1 ->d
a        <0> pushmark s ->b
b        <$> const[PV "after"] s ->c
- syntax OK

[LanX]

Looks good to me

DB<3> l 1-6
1       #!/bin/perl -w
2:      use strict;
3==>    $_ = shift // '';  # line 3
4:      s{a([bc])}         # 4
5        {d$1}g;           # 5
6:      print;             # 6
  DB<4>

You can't break at 5 though, because the multi-line statement starts at 4.

Semicolons matter!

Edit

Please try recreating the file and test again.

$ cat db_tst.pl
#!/bin/perl -w
use strict;
$_ = shift // '';  # line 3
s{a([bc])}         # 4
 {d$1}g;           # 5
print;             # 6

Do you probably have a weird line ending (from windows?) after the shebang?

[lzipin]

In the test code, lines 4 and 5 comprise a substitution operator. With delimiter pairs for the match and replace clauses, the operator allows whitespace between the clauses (with the single ';' terminating the operator on line 5). And, in fact, the code runs in the debugger (and without the debugger) just fine. It's just the debugger that, it seems, isn't parsing the line, and mis-numbering subsequent lines.

I tried again on a Debian Linux, with Perl 5.28.1 and perl5db.pl 1.5.3. It doesn't show the problem. Maybe that's a clue: I can compare perl5db.pl code from the two machines. I'll report back.

[LanX]

again:
• is the problem reproducible if you copy the code from this thread?
• did you include warn to see if the compilation recorded the right line-numbers?
and
• can you narrow down which code-line is starting the problem?

update

There are internal arrays which hold all parsed lines of the debugger, you could check there if the debugger got it right.

perldebguts :

Each array @{"_<$filename"} holds the lines of $filename for a file compiled by Perl.

[lzipin]

My grateful thanks to you all, particularly Rolf for your suggestions and prodding, and haukex for confirming the bug in Perl v5.26.*.

haukex, what did you search to find the bug references you cited?

BillKSmith, perltidy is new to me. But I tried it on my sample code; it did nothing with it (short of changing the whitespace). It didn't flag any issue with it.

Meanwhile, I've punted. The actual code was this:

  s{\s(")?(\w+):(\w)((([^"\n]*)")|([^\s\n]*)([\s\n]))}
   { " <a>$2:$3" . ( ($1 && $5) ? "$6</a>" : "$7</a>$8" ) }eg;

That finds phrases like 'source:tag' and 'source:some tag' and wraps each in HTML anchor tags. I rewrote it like this:

  my $p = qr{\s(")?(\w+):(\w)((([^"\n]*)")|([^\s\n]*)([\s\n]))};
  s|$p| " <a>$2:$3" . ( ($1 && $5) ? "$6</a>" : "$7</a>$8" ) |eg;

So the full search-and-replace operation fits fine on one line.

Thanks again. Enjoy what's left of your holiday, wherever you are.

--Lee

[haukex]

what did you search to find the bug references you cited?

In this case, I didn't directly search for the bug, I ran the bisect, and the second of the two commits referenced the bug.

[BillKSmith]

At one time, I had this problem with code that did not conform to all the specifications in perlstyle. In this case:

•Long lines broken after an operator (except and and or).

I do not know if this issue has been fixed because I now routinely use perltidy. Even if this does not fix your issue, it is a good practice. The book "Perl Best Practices" recommends "break before operator". I have never had a problem with this exception.

[LanX]

The debugger doesn't parse the code, it simply assigns every line found to an internal array, according to line-endings "\n" and shows them in accordance to the compiled line-numbers.

Now if the Perl compiler got confused by multi-lines (very unlikely), this could be demonstrated by including a warn message.

[Anonymous Monk]

line number wrong, Re: Why is the error thrown from close(FH) when the error is the missing <> on while(FH)? ( line number mismatch, look above, look before )(Bareword found in conditional)