I could make just as strong arguments that Forth (using little more than postfix notation, stacks, and a library of symbols) is the only way to start, because it balances mathematical elegance with performance and the theories that are actually behind modern mainstream hardware implementations.
What's so great about Forth? It abstracts just what needs to be abstracted and keeps its implementation and language very close to what the hardware offers, resulting in a higher-level language with very good speed of execution and lightweight memory requirements. It is easy to move both down to assembly or up to more abstract languages. It encourages code reuse at every level, and many implementations have certain library routines that directly wrap the OS's libraries. It doesn't require explicit heap management (although it is all about the stack). It runs in very tight environments, like embedded hardware.
At the same time, Forth ties one to a model of programming pretty tightly (although writing an interpreter or translator in Forth isn't that complicated). Lisp has that drawback, too, though. Perl does not.