I hate writing blog posts like this. <\/p>\n
In Arq 5.7.9 we changed the way Arq caches the list of objects at the destination for AWS, Google Drive and Google Cloud Storage. For those 3 destination types, all non-packed objects were stored in one \u201cobjects\u201d directory. If you had years of backups and\/or terabytes of data, refreshing Arq\u2019s cache could take a very long time, and if you rebooted in the meantime, it would have to start over. So we changed Arq to only query for 1\/256th of the list at a time, on demand, by \u201ctranslating\u201d the actual object paths like \/objects\/8a3a1fcac5dd03050dc91f8231fee8959339d68d into \/objects\/8a\/3a1fcac5dd03050dc91f8231fee8959339d68d for use in Arq.<\/p>\n
Unfortunately, we didn\u2019t properly test the effect this would have on folks who had old caches. Very unfortunately, the effect was that during the periodic object-cleanup process, Arq would mistake existing objects as not being referenced by any backup record, and would delete them.<\/p>\n
I sincerely apologize for this. I don\u2019t know if it\u2019s the smartest business move to write this blog post, but I\u2019d rather honestly explain what happened than try to cover up the issue. That way people can understand what\u2019s happening with their backups.<\/p>\n