Making a reloadable module, allowing "live" edits

[pryrt]

I was recently watching this youtube video on a simple PID controller implemented in Python (it was in a watchlist of math-related videos, though his PID was more math-adjacent than math-focused). But instead of focusing on the controller algorithm, I was intrigued by his demo environment, where he was able to update his python code live, and have the demo program automatically incorporate the changes immediately, without reloading the demo program.

I followed his link to the repo for his demo, where he explains his reloadable.py module and how that portion works.

"That should be doable in Perl," said I. "And I might even be able to do it." And indeed, after some effort, I could.

He basically used a class variable to store the state of the module-under-development ("mod"), then used a loop in his demo program that every loop would check the timestamp on the mod's file, and if it was newer, he would store the state, then reload the mod and return a new instance of the mod object initialized to the stored state.

In my example, which I will replicate in the spoiler, I did something similar, but I just stored the state in the instance of my reloadable object, and had the loop read the state from there and pass it as an option when creating the instance from the reloaded package. (My implementation isn't clean enough for CPAN or anything like that, but as toy, I thought it was a pretty cool usage of Perl, and is good enough for a proof-of-concept.)

My implementation maps his parent class in reloadable.py to Reloadable.pm; his intersin.py is the example wrapper that contains the loop calling the function(s) in his module under development, which maps to my tryReload.pl; his module under development, which is a subclass of Reloadable, was sinecalc.py, which I have mapped to SineCalculatorReloadable.pm.

Reloadable.pm

<Reveal this spoiler or all spoilers in this node or all in this thread>

SineCalculatorReloadable.pm

<Reveal this spoiler>

tryReload.pl

<Reveal this spoiler>

The funny thing is, a few days after I implemented it, as I was finishing up debug of something or other, where I was reloading my program quite frequently, I realized just as I was finishing up, "that would have been a perfect time to use Reloadoadable. DOH!".

If this has piqued your interest, I would love to see, and learn from, how some of the other monks would implement this. You don't have to use the sine calculator as your example; I just thought it was a simple enough example for the proof of concept. If you wanted to do all the graphics to replicate his PID ship controller instead, by all means... ;-)

[cavac]

I played with reloading modules at runtime a few years back. But there are quite a few problems that really can mess things up in subtle ways:

• If you change your variable initialization, like for example adding a variable or changing a default, a saved state becomes invalid.
• If your saved state doesn't become invalid in reload, it doesn't mean your code will still work after a clean restart. The saved state may be valid, but the initialization function you just tinkered with might produce garbage.
• When doing a larger edit, you'd normally save regularly with unfinished code, just to make sure an editor or system crash doesn't loose you a lot of work.
• Autosave is a thing in some editors.
• The first error in your code can either crash the program or (much worse) mess up the in-memory state without you knowing it.
• If you working on a forking application (like a webserver or similar), you have to choose between responsiveness of changes and reducing Disk IO. Hundred or more processes hitting the disk every half second can be "not fun". Plus, if you have NOT disabled "file access time" (Linux: noatime), this is a surefire way of reducing your SSD lifespan.

I'm not discouraging the use of automatic reloads. Just saying while automatic reloads might be cool and solve some problems, they certainly introduce a set of their own. After a lot of extra gray hair, i have pretty much abandoned the concept of live-reloading code-in-development.

[Fletch]

Pre-caffeine random early morning musing: there's almost a continuum from "normal" C style development (every change necessitates a complete rebuild and restart of the "program") to Smalltalk-y / lisp-y tinkering with the code of the running instance live. There's neat stuff you can do on the latter end of that scale but (as you mention) there's also caveats you've got to start thinking of and unless the language / compiler / runtime provides support you may be out on your own limb.

(As a specific (maybe) example, emacs has just now added support in 28.1 so that if you're interactively evaluating a defvar declaration it will now actually override an existing declaration (where previously it'd be ignored in subsequent evaluations same as if you'd re-eval'd the statement in the code (and any similarity of this paragraph and lisp is purely coincidental (probably)))

Anyhoo, agree you've got to be aware of the limitations (and capabilities) of where you're trying to work because if you're in a strange land things may trip you up.

[Anonymous Monk]

Module::Reload?

[pryrt]

I thought there might be such, but didn't want to go looking for it before I made my implementation. Thanks for pointing it out.

Based on what I see there, it looks to be more generic, but as a result, just reloads the module, without propagating state from one load to the next. So examples like the sine calculator, or the ship controller, would reset to time 0 every reload, which might be undesirable. TIMTOWTDI

[etj]

An alternative approach: a Jupyter notebook together with Devel::IPerl?