Since and b/c we are currently on a maintenance- and backup-spree, trouble arose when dealing w/ the gluster distributed-replicated setup involving four rootservers. After ranting a bit about the incompatibility of gluster e.g. in wheezy and jessy, we received a tip (thanks rpw!) and decided to start using git-annex from now on.
Note: To run a really fresh version (5.2), using the neurodebian repo is a good option.
Create Backup Data
As with every backup, it is ideal to have a spare b0x to host all the initial backup data and to function as an initial place to sort and clean-up stuff. As w/ every backup, it is best practice to always keep local backups until we really have a fully functioning, redundant backup. So, in our case, we get most of the backup data via rsync from two other hosts, and some more via scp. It is also a good idea to create a listing of all included files for later comparison, e.g. by doing
tree -ah | tee -a TREE
Initialize local repo
When we have all the data we want in place, we can continue by
git init Initialized empty Git repository in /home/d1g/ANNEX/.git/ git annex init "NWS4" init NWS4 ok (recording state in git...) git annex direct
Now, files should be added:
git annex add . (takes quite some time to create the initial checksums) git commit -m "Initial git-annex NWSx REPO"
Initialize remote repo
On all of the remote nodes, the preparation should be something like
mkdir ANNEX cd ANNEX git init git annex init "NWS3" git annex direct
Add remote repo
git remote add nws4 ssh://nws4/home/d1g/ANNEX
Now we can start to synchronize the data in between all involved repos, first copying from the remote location:
git annex sync --content
or, if we want to be more specific:
git annex copy VMB2015 --from-nws4
Keep data up to date
git annex watch
If annex watch is not running, and If now one host sends a new file via
git annex add . git annex sync --content
we can use nws4 to “spread” it to all the other hosts automatically, if we want – after having added them all as remotes – by issuing the exact same command there.
For common things that could possibly go wrong, this page may contain some useful information.