Merge or transaction-style begin/commit multiple backup calls for one host

[mirabilos]

Output of restic version

restic 0.16.2 compiled with go1.21.3 on linux/amd64

What should restic do differently? Which functionality do you think we should add?

I would like to have a way to either merge multiple backups into one or to have a transaction-like system, where I would first do restic backup-transactional --start which returns a token to me, then I do restic backup --transaction=TOKEN multiple times, and then a restic backup-transactional --commit=TOKEN to finish it, so all the parts of the backup end up in one backup.

What are you trying to do? What problem would this solve?

I have a host where I have to do (and two more hosts where I would like to do) separate database backups (pg_dump | restic backup --stdin --stdin-filename=/var/backups/db/$dbname.sql) and filesystem backups (where the database is backed up online, i.e. possibly recoverable but maybe not). I also like the SQL backups so I have the ability to restore individual rows, and I know they’re consistent.

Doing so however completely prevents me from using --group host in restic prune because that leads to accidents like #3415 where I lost basically all of my old DB backups and a few filesystem backups, and even without this, prune can decide to, say, today delete the fs backup of 2024-04-28 but the DB backup of 2024-04-28 and 2024-04-29 or something.

I would prefer the DB backups (or any backups from stdin) to be made able to land in a filesystem backup as if they were there as files at backup time. (Optionally, of course; I’m sure this is not always necessary.)

For backup integrity with rest-server, I think the transactional form is better (where the backupping host can control how much and what goes into one snapshot but cannot later change anything) than a form in which multiple backups are merged later (which also means someone would have to figure out which one(s) to merge).

So I could do something like:

token=$(restic backup-transactional --start)
trap "restic backup-transactional --rollback=$token" EXIT
token=$token /usr/local/libexec/bkup-db.sh
restic backup --transaction=$token /
restic backup-transactional --commit=$token
trap - EXIT

Did restic help you today? Did it make you happy in any way?

Unfortunately not today, but thanks anyway.