So you didn't eliminate the sleep() call? Think about it, why would you still need the sleep call when you know the time the first ctrl-c happened. Without the sleep call the ctrl-c subroutine should be finished almost immediately.
PS: Using something like sleep() in an interrupt routine is bad design. Interrupt routines should never have a long running time because the code they are interrupting might have expectations of things occuring at a certain speed without seconds-long freezing. For example the user interface