Optical Recovery

Data preservation is increasingly threatened in the digital age. That almost sounds backwards, until you dig into modern architecture.

Once upon a time, the greatest threat to information preservation was nature and memory retrieval errors. The digital age introduced denser, longer-lasting forms of data storage. As we progressed from punch card, to tape, to 8 inch floppies, to flash and optical, and ever denser forms of all, we got better at preventing data loss. Quality control, error correction techniques bundled with file systems, and redundancy (parity) in storage virtualizations, like RAID.

But in the modern age, the Enshittocene¹, some would call it, consumers are ever strongarmed into corporate-controlled cloud storage. With that, comes a lack of ownership.

Now, normally that's the lead-up to advocating for homelabbing, FOSS, and de-Googling. However, in this instance, it's the lead up to optical media preservation. Why? Because optical media remains a great storage medium today. Even if you don't believe that, many old games in their original form, or other preserved media, are still only available on discs.

However, discs are subject to natural degradation through scratches, delamination, and disc rot. Especially as the quality control on these discs dwindles. Thus, there must be additional work put into caring for this medium.

RAID arrays are great as alternatives too, as they already have parity, ddrescue, and error-correcting filesystems available. The problem is that the arrays themselves are not. At least not right now. Once again, partly thanks to AI datacentres demanding obscene amounts of resources, the cost of hard drives has gone up significantly since the pandemic.

With that, we segue into my latest project, temporarily named Kramer. I know I did some research to come up with that name, but I genuinely don't recall who it was in reference to. In fact, having tried tracking down who it's in reference to, I may have messed up when I picked the name. My best guess is that it's in reference to Kane Kramer, but he helped design the Digital Audio Player, not optical media.

Regardless, it's always been a working name while I figure out if I can finish the project.

So, what is Kramer?

Kramer is a recovery utility. Conceptually, it's meant to be akin to ddrescue², but optimized for recovering data from optical media.

There are two main problems I've experienced while trying to use ddrescue with optical drives, and both stem from the drive's firmware. Now, I don't know if me using the --reset-slow flag is contributing to these issues or not, but I experience more problems without it.

If you've ever tried working with optical media in Linux, you may be familiar with The Sleep Bug. Normally, this is when a drive's firmware decides to sleep after two minutes, and never wakes. In the case of ddrescue, I've noticed that the drive often sleeps after read errors. When this happens, I can't kill ddrescue, and it just hangs. Even if I forcefully disconnect the drive, it just hangs, awaiting a response from the block device.

For the record, Kramer still has this issue. I've only recently gotten mapping working alongside directIO to get around the Linux kernel's read poisoning.

Keep in mind that I know very little C, and I don't play this low in the system much yet. So, I don't know why, but without the O_DIRECT flag to bypass the kernel's buffer (and whatever else directIO technically bypasses), after a single IO error, the kernel will refuse to return anything but that same IO error until the parent process is restarted. I tried doing the C calls directly through FFI, since poisoning is something that Rust employs in some places for safety. That didn't work, so I tried spawning a child process to execute the read, but that also didn't work. Thankfully, I know a guy, and with his help I was able to track down not only how to implement directIO, but also without using FFI to call posix_memalign.

Back on track, the other issue is the spin-down. As ddrescue reads smaller clusters and/or starts hitting a particularly poor section of the disc, some drives' firmware is too quick to spin down, causing the spindle to constantly speed up and slow down, increasing wear.

---

1. I would recommend reading Enshittification by Cory Doctorow.
2. As I'm writing Kramer in Rust, I also have to stress the point that this project is not intended as a ddrescue rewrite in rust, for the sake of a rewrite in rust. I'm not trying to clone or replace ddrescue. Kramer has a different target, and ddrescue is a resource for me to learn from and use as a guide while I dig deeper into the system than I ever have.