Ah INSERT MODE backspace key works for me and will backspace you to the begining of the file if you hold it down long enough
It'll backspace over line breaks if you "set backspace=eol".
I have backspace set to "indent, eol, start" in my .vimrc. indent allows me to backspace over autoindented lines, eol allows backspacing over line breaks, and start allows backspacing past the starting insertion point.
<sarcasm>A list of all available options can be found with the obvious and intuitive command ":he Q_op"</sarcasm>, in case a vi/vim beginner is looking.