The "cafeteria with spring-loaded trays" model is exactly right for stacks (LIFOs - Last in, First out) which are what push and pop apply to. And yes, you absolutely would both push and "pop" trays - push to add em in the first place,and pop to get one to use.
Perl extends the concept to include access to the "bottom" of the "stack" which allows a few more operational models. In particular the combination of push and shift (or unshift and pop) provide a queue (FIFO - First in, First out). Shift parallels an operation common in handling arguments in some OS batch/scripting languages. Unshift is a neat complement to shift for "putting back" an argument after taking a peek at it. At times there is an advantage to using a queue where you can add and remove elements to/from either end - can't think of an example right now in the software world, but it often happens in supermarket checkout queues!
Nomenclature adjusted for northern hemisphere usage. In the southern hemisphere stacks are the other way up. Queues can be either way and may be affected by the Coriolis effect.
DWIM is Perl's answer to Gödel