"I don't know what Diagnostics does to your code performance."

If the messages are about problems that cause your program to abort, it obviously doesn't matter. For warning messages, you'd probably want to deal with them before code performance is an issue (i.e. prior to volume testing); so, again, it's unlikely to matter.

You can always use Benchmark but, if you do, just be aware that generating a warning multiple times will only generate the verbose diagnostics message once. Furthermore, the time taken to get the message to your output device (I/O) is likely to far exceed the time taken to generate the message (CPU). All in all, testing diagnostics-related performance is not as straightforward as it might, at first glance, seem to be.

"For my learning time, what is $DEBUG_MODE best used for?"

I made up the $DEBUG_MODE variable merely to indicate whether or not the code was running in debug mode. Applications have their own ways of defining this, including: command-line options, configuration settings, variables.