Re: Retrieve changes in remote dataset

[soonix]

Neither CPAN, not even Perl, probably only for low-change (e.g. configuration) databases

• https://www.viget.com/articles/backup-your-database-in-git
• https://ongardie.net/blog/sqlite-in-git/

Then, there is Sqitch, in CPAN: sqitch, however, that seems to concentrate on database structure, not the data itself.

Disclaimer: I didn't use any of these, just have them on my "explore later" list.

[hippo]

I had considered the possibility of using git (or at least some VCS) fetching the changes, extracting the diff, processing that and then merging. It's a possibility for sure but the work involved might be even more than rolling our own RESTful updater. Thanks for the thought.

We looked at sqitch for another project a while back. I think it can do data changes OK but it is seriously overkill for this particular task (and the dependency tree is on the heavy side too).

[1nickt]

I seem to remember that we used rsync for this some time in the Jurassic. Worked well with MySQL ISAM tables IIRC.