Apparently that’s something that everybody is rolling on their own.
It’s 2019 and Netapp still rightfully has market share because it’s the only vendor that has snapmirror technology.
Common scenario: you have to sites, and you want to have both data volume snapshots and replication between two sites.
Apparently ZFS can readily do both as it makes easy to both create snapshots and send them around.
BUT: if you propagate the snapshots one by one, as you create them, then it’s fine.
Otherwise, there’s (apparently) no way of bulk-sending all the snapshots.
I’ve tried messing up with the -i (for incremental) option of zfs send, but it can only send the immediately-following snapshot of a snapshot. If like me you have 30-40 snapshots of a single dataset, you’re out of luck.
In this post, thus, I will be describing what I came up with to automate the sending of all the snapshots from a machine to another, using zfs. Make sure to make the adequate alterations and customization when/where needed.
Legend of hosnames and stuff
From here on, the following names will have the associated meaning
- source: the already existing machine that currently hosts the zfs pool, the zfs dataset and all the volumes
- destination: the new machine that has a new zpool with an empty dataset and no snapshot
Short version:
First things first, receive the first snapshot (basically most of the dataset) onto the destination:
[root@destination ~]# ssh root@source \
"zfs send storage/mail@201907191100" | \
zfs recv storage/mail2
Then on the source, create a list of commands to send a snapshot as incremental of the previous one:
[root@source ~]# zfs list -t snapshot | \
grep -Po 'storage/mail@\d+' | \
awk 'BEGIN { prev="storage/mail@201907191000" }\
{ printf("zfs send -i %s %s\n", prev, $0) ; prev=$0 }' \
> commands.txt
This will create a chain of commands sending each snapshot as an increment over the previous one.
Brief extract from the commands.txt file:
zfs send -i storage/mail@201907191000 storage/mail@201907191000 zfs send -i storage/mail@201907191000 storage/mail@201907191100 zfs send -i storage/mail@201907191100 storage/mail@201907191200 zfs send -i storage/mail@201907191200 storage/mail@201907191300 zfs send -i storage/mail@201907191300 storage/mail@201907191400 zfs send -i storage/mail@201907191400 storage/mail@201907191500
Make sure to drop the first line from commands.txt, then copy commands.txt from the source to the destionation machine. Scp or sftp will do.
On the destination now, it is time to wrap each command from commands.txt into a remote call, catch its output and redirect it to zfs receive.
[root@destination ~]# xargs -L1 -I "{}" \
bash -c 'ssh -p 2222 root@source "{}" | \
zfs recv storage/mail < commands.txt
Long version
To be written. Some more details will be added.
Znpy 