Some contracts allow it, some forbid it, some are ambiguous, and some don't mention it.
I would be interested in keeping copies of 2 kinds of code: one on an operational level, such as whether split, m//, or substr is better given a certain context; and one on a conceptual level, such as how to send high bandwidth data over balanced multiple connections without drawing attention.
For the former, writing notes to yourself, about something you could easily duplicate given the time and inclination, having almost no relevance to the task you were hired for, seems reasonable on the face of it (though it may or may not pass muster legally). For the latter, you can bet that someone will be curious about your solution, especially if they've paid for a competitive advantage that no longer is.
But if you can't keep copies of your work to prove what you did (or didn't) do, then it's your word against theirs, and the outcome may depend on who has deeper pockets. Though if your copies "escape", you can end up liable for that.
In the end it may be better to have a sharp memory and a sharp mind. Anything you've created once, should be easier the second time, right? [This is where certain people would bring up "Eve after Adam" :]
Quantum Mechanics: The dreams stuff is made of