I would add far more checks/tracing to DebuggerInit. Maybe ->GetActiveObject() does differ in behaviour for the debugger, or creating the new object fails if the debugger application is not already running. Output the functions called and the results in DebuggerInit.

Your choice of setting up a call to exit() in the object destructor:

$debugger = new Win32::OLE( 'Excel.Application', \&main::QuitApp );
[download]

... strikes me as weird. I use the call to properly tear down the OLE object, and not to exit the Perl script in a very hard way. Maybe the debugger application does not like that method of improper cleanup. As a first start, I would not try to quit the OLE application at all upon script exit. This will likely leave you with headless processes accumulating, at least it does with Excel. After trying that, I would look at finding the proper way to quit the debugger application. Maybe you need to do nothing, or you need to do the same thing that the Win32::OLE documentation suggests, except in the appropriate way for your debugger:

    $ex = Win32::OLE->new('Excel.Application', sub {$_[0]->Quit;})
    # calls $ex->Quit() when the Perl program ends or $ex goes out of 
+scope
[download]

As a personal style option, I would not cache the $debugger in a file-global lexical variable. In most cases, using a proper global variable in the packages is better for me, because it allows me to explicitly undef such variables for debugging purposes.