I have no idea where UDP came into the conversation since SMTP is a TCP protocol. I agree that there is no way to be sure unless you can deliver mail to a recipient and get a DSN or, preferred, a response. Seeing if the server is listening on port 25 and is responding in a manner consistent with a SMTP server and not giving a 521 status code is a pretty safe bet though.
Depends how many servers you are looking at and how good the IDS is on the site. If it was enough of a problem, I could see using that fingerprint to tag a site to watch behavior more closely. ISTR discussions about this many years ago on SPAM-L. Perhaps browsing the archives for doorknocking, testing addresses, and some similar search terms might be useful.