Duply, Duplicity, and The Best Laid Plans

I was going to go to bed at a reasonable hour tonight. Honest.

The background is too long to explain, but in short I was trying to configure Bitwarden_RS, an open-source Rust implementation of the Bitwarden platform. I’m using an OpenVZ VPS, on an older OS image, so docker deployments of even the stripped down Bitwarden_RS was out. So I grabbed their code, built it, and struggled through spinning up a service-to-service nginx configuration. I got everything working. But before I took a dependency on a new service, I wanted two things: self-starting (system.d to the rescue) and backed up.

Wait, whoops.

It turns out that around January, Microsoft turned off the legacy OneDrive API. Apparently, my backup alerts never fired, but I haven’t had a successful backup since January. Well, that’s not good. So over the last twenty-four and change hours, I’ve been trying to figure out what’s going wrong.

Problem 1: 410 response from OneDrive API (“gone forever”).

A bit of investigation, and it appears that someone implemented the updated OneDrive API for duplicity! It’s been in since.. right after the last stable release.

Problem 2: Daily builds from Duplicity are still targeting the last stable release (0.7.19). I needed 0.8.0.

The dev channel builds don’t appear to be working. So I grabbed the source code.

Problem 3: The duplicity source code is under-documented.

No make file — but if you fumble around a bit, you discover the “setup.py” script actually does incredibly useful things — like building. And installing. I had to fumble through a few dependencies, but… yay.

Problem 4: The apt-get package for duply has a dependency on duplicity, and the duplicity install from a local build isn’t recognized by apt-get.

Problem 5: … and can’t be suppressed.

Okay, so I worked around it. The “equivs” package lets you create a stub package to fulfill requirements. So I used “apt-cache showpkg duply” to find out which version of duplicity was required, modified the faux duplicity package configuration, and used “dpkg -i duplicity.deb” to install it.

Problem 6: I couldn’t install it because duplicity 0.4.whatever would break other things.

FINE. duplicity.0.8.0 it is. “apt-get install duply” worked fine afterwards.

Problem 7: duplicity did NOT work fine.

So I tried rebuilding it, reinstalling it. Installing pip. Updating pip. Checking that I had python3 installed. Installing python3 pip. Rebuilding duplicity. Learning “find . -name ‘_libsyncr.so'”.

Actual problem? setup.py, which I alluded to earlier, installs all of duplicity to /usr/local/lib/python2.7/dist-packages/duplicity/, but /usr/local/bin/duplicity starts with “!env python3” — note the version. Change duplicity to use python2.7 — and bam, everything’s working!

Problem 8: No, not really. Duplicity (and duply) report an error trying to reach the Microsoft Graph (the new API) — invalid scope. There’s exactly one blog post I could find that talks about how to setup the OAuth flow for duplicity to OneDrive, and I got the same error when I tried to follow it’s steps.

The solution, in hindsight, is painfully obvious. Delete the file ~/.duplicity_onedrive_oauthtoken.json , and the OAuth flow will be re-run! Unfortunately, you have to remember the file is there. It’s not documented that I could find. New OAuth flow -> permissions -> everything works!

Problem 9: No, not really. “duply blogs status” reports no backups!

duply/duplicity actually stores files in a OneDrive folder based on “username” — maybe there’s a setting to change that default naming, but that’s what it is. So the various files are stored in OneDrive:[email protected]@serverbackups/whatever/*. Except, they’re not. They were stored in OneDrive:user%40office-monkey.com@serverbackups/whatever. Apparently, at some point in the last few months, they stopped URL encoding the username. How did I find this out? I started a full backup of the largest data set! It backed up over a gig of data that had already been backed up. I moved all my existing backups over into the new path, and now all the backup patterns can see them!

Problem 10: I’m still up at 2:27am.

Solution: Good night!