Data lost from private scratchpad

[Athanasius]

For most of my time at the Monastery (nearly 4½ years now) I’ve been slowly accumulating information in my private scratchpad. Today I discovered that the majority of that data is gone — apparently just truncated below a certain point. I assume that means I’ve exceeded some pre-set limit? If so, I have three further questions:

1. Is there any way I can recover the data?
2. Is the limit local to the private scratchpad, or global across my home node, public scratchpad, and messages?
3. How can I monitor my data usage in the future to prevent this from re-occurring?

Thanks,

[Corion]

Your scratchpads are subject to the 64k text limit. I think we have a warning displayed if you save your scratchpad and it is close to the 64k limit, but that warning might be restrited to wiki pages instead, where the problem is much more prevalent.

As for recovery, if you've appended new data at the end, it never reached the database, so it can't be recovered. If you prepended the new data, maybe the last week can be recovered but not more.

I'm sorry for your information loss :-(

[Athanasius]

Hello Corion,

I’m sure I didn’t get any warning message. :-(

I’m still not clear whether the 64k text limit applies to messages, scratchpads, and home node separately or all together. The reason I ask is that I’m not sure whether reducing the contents of, say, my messages folder or public scratchpad would alleviate the problem with my private scratchpad?

As for the new data: the only change I’ve made to my private scratchpad recently was to click on the “Add to private pad” link (at the bottom of my Personal Nodelet) for a single node I wanted to remember. But the data truncated from the scratchpad runs to (probably) more than a hundred lines — far more than I could have added.

So if the data can be recovered, that will be great! :-)

Thanks for your help,

[Co-Rion]

After looking through the backups, the scratchpad contents seem to have fallen prey to file corruption and were truncated. Unfortunately, our backups don't reach back far enough in time to make recovery possible.

I've recovered the data in the same database storage page which was not picked up by the database anymore but which was clearly at one time part of your private scratchpad.

Again, I'm sorry for not detecting the corruption and for the loss of your data, even if some of it could be recovered.

[Discipulus]

You have my solidariety, me too I'm pretty sure there are no warnings on such cases. I've noticed only when i tried to upload a big (~100Kb) script and was silently truncated.

I wonder if it is not possible to modify a bit such limit to something more modern, like 512Kb or possibly more.

L*

[Your Mother]

I’m certainly sorry about the loss. I’ve lost plenty in the past which is why everything I care about is in some RCS now. Bitbucket has free private repos so there is no need to share stuff you don’t want to.

[stevieb]

"Bitbucket has free private repos..."

I don't think they did when I switched over to github mid-2015 (I could be wrong though). That's very handy, as github doesn't provide free private repos. I have use for that for sure, so it's time I dust off the ol' account over there.

[Your Mother]

I’ve been using bitbucket since 2014 with private repos. Don’t know if that’s when they first offered them or not but it was the reason I started. I generally use github for public stuff because of its better integrations, and API, and bitbucket for things I don’t want in the wild due to it being too sloppy/shameful, me-specific, or one of my many million dollar ideas that so far made me nothing but a little more poor in time and money. :P

[planetscape]

I dimly recall this happening to me a few yrs back...

What I began doing (very sloppily) was copying the source HTML from the edit pane to an editor and saving it locally as text somewhere in my personal VCS.

IIRC, around this time, Corion suggested TiddlyWiki as a scratchpad alternative. It works well for many things, though of course little local hacks like adding nodes to your personal nodelet or public/private pads do not work.

As someone who backs up to a minimum of three local drives and also offsite, you have my sincere condolences. I know your pain.