You consider that making it easy?
Here are a few issues to think through.
- Any decent GUI program needs to handle interrupts well. It was doing X, the user decides not to and interrupts it. The API that you are describing offers no good way to pass interrupts along.
- The system that you describe doesn't seem to have a lot of debugging hooks when things go wrong.
- A huge problem with Expect is that unusual, unexpected or unanticipated input causes programs to diverge. When the GUI and internal program diverge on what they think is being displayed, you can readily get into deadlocks.
- What you are describing sounds like a lot of overhead at all ends.