Arq 5.8.5 for Mac Fixes a Bad Bug
I hate writing blog posts like this.
In Arq 5.7.9 we changed the way Arq caches the list of objects at the destination for AWS, Google Drive and Google Cloud Storage. For those 3 destination types, all non-packed objects were stored in one “objects” directory. If you had years of backups and/or terabytes of data, refreshing Arq’s cache could take a very long time, and if you rebooted in the meantime, it would have to start over. So we changed Arq to only query for 1/256th of the list at a time, on demand, by “translating” the actual object paths like /objects/8a3a1fcac5dd03050dc91f8231fee8959339d68d into /objects/8a/3a1fcac5dd03050dc91f8231fee8959339d68d for use in Arq.
Unfortunately, we didn’t properly test the effect this would have on folks who had old caches. Very unfortunately, the effect was that during the periodic object-cleanup process, Arq would mistake existing objects as not being referenced by any backup record, and would delete them.
I sincerely apologize for this. I don’t know if it’s the smartest business move to write this blog post, but I’d rather honestly explain what happened than try to cover up the issue. That way people can understand what’s happening with their backups.
If you installed Arq 5.7.9, 5.8.1 or 5.8.2 sometime between April 3 and May 21, and Arq’s periodic budget enforcement (or “object cleanup” if you didn’t set a budget) process happened during that time, and you’re backing up to AWS, Google Drive or Google Cloud Storage, then you were likely affected by this issue. Please run the latest installer. Arq will upload what it needs to upload going forward. But your older backup records are likely missing objects.
I’m working on putting more tests in place so this doesn’t happen before we ship another update.