Re^6: Can't use an undefined value as a HASH reference

[Monk::Thomas]

Firstly, no where in my post did I suggest: "automagically initialize a hash reference"; so that's a pure strawman.

Sorry, I did not want to imply you were suggesting this. It is simply an example in how one could apply the robustness principle to this particular case.

Regarding the rest of your posting:

First I'd like to define my understanding of 'liberal to accept': It would mean to try as hard as possible to make some sense out of the provided data. In order to do so you may need to apply some guesswork / heuristics or ignore contradictions. (I think of it as a blacklist approach - some kind of data is definitely wrong and will be refused, everything else is going to yield some kind of result.) On the other hand being 'conservative to accept' would require to establish specific rules what is acceptable and then refuse everything else. (a whitelist approach) These rules can be quite extensive - conservative does not imply a narrow focus - it just means the rules are enforced.

I can argue exactly to the contrary by saying this whole 'be liberal in what you accept' is throwing us back 20 years. 2 Examples: Never accepting invalid input would have prevented Stagefright (if a movie file contains strange data then simply reject to play it, don't attempt to automatically fix the file and generate a heap overflow) and an Android-packaging trojan horse (forgot the name, basically you pack two different archives into the same package, Google's package check would only look at the first one and deem it safe - the second contains the malware and is the one that's actually installed on the phone) because the tools would not need to bother about 'properly' handling invalid data and trying to guess what the intention was. (As can seen from the second example: The problem can be as simple as guessing differently.)

I don't really care if you want to name it 'malformed', 'invalid', 'corrupt' or whatever. It's not an XML-problem. It's a networking/environmental problem.

'be liberal in what you accept' is quite nice if you're working in a rather isolated place where you can control where stuff is coming from and what kind of stuff that may be - or where everyone is nice and friendly and just tries to get a long.

But today's internet isn't such a nice and friendly place. It's openly hostile. As soon as you connect something interesting to the net then there's a pretty good chance somebody is interested in it enough to take a very close look and try to figure out how to exploit it. And that's why I'm also concluding it's throwing us back 20 years - it's throwing us back into a time when we were less connected and less easy to exploit. (As long as I don't insert any media I am perfectly safe. USB was some newfangled toy which probably does not last long anyway.) Networks were mostly isolated. Yes, BBS and Internet did exist back then. But for most people it was not relevant at all. That's the reason why I picked the 2 Android exploits - they are relevant to roughly a billion devices (~700 million people?).

'malformed', 'invalid', 'corrupt' is not something that accidentally happens any more. Someone will manipulate the data to make sure it triggers exactly the right spots and delivers its payload. Someone wanted to overload AntiVirus-Scanners and carefully crafted 42.zip - 42 kBytes which extract into multiple Petabytes. That was more of a roundhouse kick but even if it's a very specific exploits that triggers in very specific conditions - with today's billions of connected devices you are pretty much guaranteed to find exploitable targets.

I don't know what a suitable approach for your XML-example is. In a different situation it may be a problem to discard the malformed data. If you have someone who is responsible for the data quality then it could be a good time to fetch the cluebat and apply some data-quality improvement lessons. If you have no one who is accountable then the 'ignore crap' approach seems reasonable. Maybe your example is actually a good example why 'be liberal' is actually a bad thing. Maybe the one who's generating your XML is relying on you to be liberal and therefore him/her being able to get away with producing garbage.

P.S.: Automated cars are going to be very liberal in what they need to accept as suitable road conditions and it scares the hell out of me. Especially since car maker don't even understand the concept of proper separation between different security zones yet.

[BrowserUk]

It is simply an example in how one could apply the robustness principle to this particular case.

Sorry, but that is garbage; as I said, a strawman.

The robustness principle does not suggest being stupid; and I reject all of your attempts to imply that it does.

I do not recognise any of your "examples" of stupidity as valid examples of applying the Robustness Principle as it is intended; or as it is correctly used.

The rest of your post reads as a hastily concocted justifiction by someone who knows his bluff has been called; but continues to think that he can get away with it if he can throw enough other strawmen, misdirection, smoke screens and specious arguments into the mix. You can't.

The Robustness Principle simply means that you don't die in the face of unexpected or incorrect input; you log a warning, ignore it; and then carry on doing so until you either get good input or EOF or timeout. And that's it. It could equally have been termed the "Don't be fragile or pedantic or a prima donna" Principle.

It does not mean that you attempt to fix bad input; nor fail to apply common sense; nor do any of the other stupid actions you claim are attributable to it. They aren't.

[Monk::Thomas]

You said: The Robustness Principle simply means that you don't die in the face of unexpected or incorrect input; you log a warning, ignore it; and then carry on doing so until you either get good input or EOF or timeout. And that's it. It could equally have been termed the "Don't be fragile or pedantic or a prima donna" Principle.

RFC 761 says: 2.10: TCP implementations should follow a general principle of robustness: be conservative in what you do, be liberal in what you accept from others. (It then continues with part 3 "FUNCTIONAL SPECIFICATION".)

I do not see a reference on how to handle data that falls outside of what's deemed 'liberal' and acceptable. Additionally you seem to be reading something into this posting that is not there - I did not say anything about a particular way to react to an error (e.g. terminate). What I did say was to refuse invalid input. I did not say how to refuse it. (I even gave an example on how to be more forgiving and prevent die() in Re^4: Can't use an undefined value as a HASH reference!)

But maybe I misunderstood the meaning of these few words. In an attempt to rectify that I was looking around a bit and stumbled upon:

The Robustness Principle was formulated in an Internet of cooperators. The world has changed a lot since then. Everything, even services that you may think you control, is suspect. It's not just user input that needs to be checked—attackers can potentially include arbitrary data in DNS (Domain Name System) results, database query results, HTTP reply codes, you name it.
[...]
The atmosphere of the Internet has changed so much that the Robustness Principle has to be severely reinterpreted. Being liberal in what you accept can contribute to security problems. Sometimes interoperability and security are at odds with each other. In today's climate they are both essential. Some balance must be drawn.
-- http://cacm.acm.org/magazines/2011/8/114933-the-robustness-principle-reconsidered/fulltext

This is precisely the point I'm trying to make. There must be a line where you say enough is enough and refuse to process the provided data any further. (Again: I do not say how to refuse it.) To me it seems like we favored interoperability way to much and now we're paying the price. Adversaries deliberately exploit corner cases. Being more conservative in what to accept would NOT magically make these exploits go away. But it would reduce the attack surface because there are less conditions to consider. Less cases -> less code -> less bugs -> less exploits. (Hopefully.)

[BrowserUk]

And now you have strayed so far from where this started it would be meaningless to continue to respond to you.

Nothing in the OP suggests that he is dealing with data received from untrusted sources. NOTHING!

And everything you have said that is founded upon that premise is meaningless in the context of the OP and my response.