Failed at backup... (data recovery part 1)

So everywhere can you read about how important it is to backup your data. Are you actually doing it? I don’t do it, although I have a private git server where I store some of my projects. But I’m not good at starting a new git project either, usually I write code for some days or even weeks before I actually start a git project.

So what did I do

I was running my virtual Debian machine and thinking for myself that I’m not going back to Manjaro. I did not like that distro, to buggy. So I just rm -rf the folder with all the data for my Manjaro virtual machine. Well what is the problem with removing my Manjaro virtual machine then? If I’m running a Debian virtual machine?

QCow2 snapshots & backing file

Well, I really was smart and used a feature in the QCow2 file format where you can create a snapshot file as an overlay on top of an existing QCow2 file.

All new data will be written to the new QCow2 file and old files will be read from the old QCow2 file until the data is changed and written to the new QCow2 file. So I used this feature to snapshot my /home data drive, this way I would get all my files over to the Debian VM. And if I wanted to go back to Manjaro after testing Debian for a while I could just delete the snapshot version and everything in Manjaro would be unchanged.

But well, Debian was so stable and everything just worked so good so I kept using Debian and I forgot that I was still using a snapshot QCow2 that was linked to my Manjaro qcow2.

So when I deleted the Manjaro VM folder and all the QCow2 files in it, I deleted the backing file of my Debian virtual machine /home drive.

So now Qemu can’t read the data in the QCow2 file without the linked backing file. The virtual machine didn’t even start beacuse the file system on my /home drive was broken. I was forced to create a new QCow2 drive file to use as my /home drive. But this one is brand new and empty and all my files was gone.

Trying to recover data from QCow2

At first I didn’t think this was a problem, mostly because I knew I had used Debian for a while and I knew most of the data I wanted back would exist in the Snapshot QCow2 file I still had and not in the backing file.

So first I tried to just re-create a QCow2 file with the same size and file system and rename it and place it at the same location as the original backing file would be. But this did not work, although I actually didn’t think it would work either.

Well I’m not defeated yet. So I looked into the QCow2 tools bundled with Qemu, I tried to convert the file to a plain IMG file and I tried to mount it as an filesystem. But everything failed because the tools could not open the backing file and I could not find any parameters to force the tools to ignore the missing backing file.

Finally I began reading up a bit on the file format, I got enough information so I could hack into the file and remove the information about a backing file. I thought this would force the qemu-img tools to just ignore the backing file and write everything to a IMG file. But I was wrong, now the tools failed because the file was corrupt.

The next step

I know my data is still in there and I know there has to be some way to get it out. After reading a bit about the QCow2 file format to find out about the backing file and remove it. I got a lot more interested in the file format, so in my next part I will dive into how the QCow2 file format works.

QCow2 File format (data recovery part 2)