Automagically Losing Work. A Lost Art

It's been a long time since I've written a post (almost a month in fact), although not as long as it seems.

The oddest thing happened while I was writing a blog post, which I'm just not used to these days. I've got a process for writing in Markdown using Emacs. I also write the post and, believe it or not, come back to edit it later. That part of the process includes sourcing images etc.

What?!? You edit this stuff?

So, I got up early, wrote the post as a brain dump and vowed to come back to it later that day. I didn't come back to it for two days, and in between my Mac had crashed. No problem, all my apps had reopened and any documents I had been working on reappeared.

Except Emacs! I didn't really notice to begin with, but I went to open my Markdown file and I couldn't find it. It was nowhere on my hard drive. "Odd", I thought to myself, "let's check out Time Machine". Nothing there either. The post wasn't there, gone, kaput! I checked temporary files, autosave directories, everything I could think of and it was nowhere to be seen.

This felt strange for a couple of reasons. Firstly, for this to happen I must have not saved the file once. I am extremely trigger happy with saving files, mainly because I come from a past era of computer crashes where that meant you lost work. That leads me on to the second odd feeling I got. Losing work! I don't think I've experienced that feeling for years.

During college I despised working on a Mac. It just wanted to do everything for me and it felt impossible to get under the hood. For some reason the project we did on a Mac was not creative and used Excel! Macro programming in Excel I believe. The combination of buggy Microsoft software on a platform that crashed with a handy bomb icon, forcing you to restart the machine, did nothing for my blood pressure.

In fact, the whole experience scarred me to the point that I still reflexively hit whatever key combination saves my document, far more than I need to.

For a brief stint, before going to University, I worked as a computer technician at the same college. One of the jobs was recovering work from, usually crying, students who had just seen the Mac bomb or blue screen of death. The process involved checking floppy discs with Norton Disk Doctor as well as searching through temporary files on the machine they'd been working on.

A Norton Disc Doctor screen I spent far too much time staring at!

Knowing that awful feeling of losing work, it felt all the more rewarding if I managed to bring their 20,000 word dissertation back. It must be similar to how a vet feels reviving a family pet, with the accompanying emotional conversations if there's nothing they can do to save it.

The Positives

When hit with a negative experience like this you have to focus on the positives.

I've struggled to spend time rewriting the post. That proves that it needed some work! If it's not exciting enough to write then it's definitely not going to be exciting enough for anyone to read.

Reflecting on when I've lost work before, I realised that when I've rewritten the document, recreated the design, or reimplemented the code, it's usually turned out better. There's some people who believe that a rewrite of a code base is inevitable and necessary. In the Mythical Man Month Fred Brooks encourages programmers to "throw the first version away and then build a second one". However, it's something people find hard to do. Perhaps we should bring back the ability for people to lose work to force their hand?

Ernest Hemingway practised rewriting every day

I always rewrite each day up to the point where I stopped. When it is all finished, naturally you go over it. You get another chance to correct and rewrite when someone else types it, and you see it clean in type. The last chance is in the proofs. You're grateful for these different chances.

— Ernest Hemingway

You forget sometimes that great work doesn't just splurge out of someone's brain first time.

The Solution

The other positive was that it forced me to teach myself some more about Emacs Lisp and configuring the editor. So how could I have ensured I didn't lose my work?

Emacs auto-save. There was me cursing Emacs for not having auto-save on by default. Emacs comes from a different era where having multiple versions of your document saved to the cloud for access on any device was something of science fiction. It probably also thinks that anyone dumb enough to write 1500 words without saving deserves whatever is coming to them!

And then I noticed this line in the configuration I'd got from "Clojure for the Brave and True":

(setq auto-save-default nil)

It's not commented but looks suspiciously like it's disabling auto-save. After a little bit of searching I had my suspicions confirmed.

Highlighting one of the perils of trusting code, especially that configures your main work tool, and not reviewing what it does. There's a great article on this I came across, that goes a step further and suggests the way most developers install customisations in their editor is as bad as some naive internet user installing search bar malware in their browser. It's an interesting point, and I've definitely been guilty of including elisp scripts I've downloaded from around the web without vetting them first.

Venturing into Emacs Lisp

I've been looking forward to doing a bit more customisation of Emacs given it's so famous for it. Since I've been teaching myself Clojure, learning Emacs Lisp is more exciting than daunting now, and small tweaks allow me to learn it incrementally.

The previous line in the configuration I'm using was:

(setq backup-directory-alist `(("." . ,(concat user-emacs-directory
                                               "backups"))))

After some research and discussion with a helpful Clojurian I realised how this works.

The variable backup-directory-alist is an associative list (think map with key value pairs separated with a .) of regular expressions and values. If the regular expression matches then it's associated value becomes the directory for backups.

There's a special case here where we want to match all files. The documentation states this regular expression key should be ".". So this bit of lisp is associating ~/.emacs.d/backups to the catch all regular expression. Straightforward really!

Emacs determines the auto-save location using a different variable auto-save-file-name-transforms. This is similar to backup-directory-alist except it's a list of lists with three elements. The first two are a regular expression and the replacement, with the third being a flag as to whether Emacs should 'uniqueify' the filename. That process involves concatenating the folders and buffer name with !. The documentation explains all of this, and researching the default value a bit I also discovered that t in Emacs Lisp is the preferred way to represent true - told you I was still learning!

This led me to swap out the two lines turning auto-save off (grrrrr...) and setting the backup directory with:

(defconst emacs-backup-dir (concat user-emacs-directory "backups/"))
 
(setq backup-directory-alist `(("." . ,emacs-backup-dir)))
 
(setq auto-save-file-name-transforms
  `((".*" ,emacs-backup-dir t)))

Attempting to be DRY I thought I would define a constant to store the backup directory and then use it for both values. I'm not sure if you could/should use a single . rather than .* for the auto-save transforms, but if it ain't broke...

The End Of That Feeling

So, now I should never feel that annoyance of having lost hours of work. The question is should I turn auto-save off occasionally? It could be a benefit to lose work and force myself to rewrite it. When I have some time I'll rewrite a post pretending I lost the original and compare the results.

When was the last time you lost some work? Do you think the standard improved when you recreated it? I'd love to hear about your experiences because if nothing else it would help to know it isn't just me this still happens to!