I've pondered this argument a long time because something in it didn't sit right with me. I agree with you, but then sauoq is correct as well.
The problem is we're talking about two different levels here.
One is the representation of an image. Even capturing it on any kind of media is already a lossy step, because nothing short of an exact replica of of the entire scenery (that is, in meatspace, with the exact same materials and everything used) can ever record that image absolutely faithfully. Here indeed PNG is lossy as well - by definition, in fact, since there is no such thing as a lossless image capture media.
The other is the representation of a finite stream of bits. On this level it is certainly very possible to achieve an entirely faithful capture of the input. I don't think we need to argue this any further.
I agree with you insofar as people do tend to confuse these issues. However, you failed to point out the distinction as well. I knew both positions in the argument where right even though they seemed to conflict - and even though I'd consider myself at least somewhat knowledgable of information theory, and it is so very obvious in hindsight, it is obviously easy to confuse these.
Also note that repeated reserialization of an image into a bit stream as performed by JPEG will reduce the accuracy of the reconstructed input. JPEG being "lossy" in both sense thus is a downside if you're going to manipulate the image further. It is a mighty fine final storage format, of course.
Makeshifts last the longest.