1

I'm trying to make an automated backup of my /home directory. In theory this is simple by running this in a cronjob:

cd /home
git add .
git commit -m "nightly backup"
git push

where push will push the changes to a remote backup server.

The problem I have is if there is a git repo in the /home directory e.g. /home/tom/projects/myproject/.git. The backup task runs and pushes all changes to the server. However, any directories that contain repositories are now blank on the server.

I realise this could be handled with submodules. However, this has several downsides:

  • Each submodule must be registered with the parent repo (This could be automated but it would be a bit of a pain, recursing into every directory and adding the submodule each time the cron runs)

  • Non-committed changes from the submodule will not be backed up, this is not desirable behaviour for a backup, I want a live snapshot of all the files as they are at the moment, whether or not they're tracked elsewhere

I realise there are other tools such as rsync to do this, however it would be nice to have git track the changes over time. Is there any way to have git treat other repositories as if they are just normal files (including their .git directory!)

A workaround for this would be running something like this in the cron:

  • Recurse into all subirectories
  • Look for a .git folder
  • Rename .git to something else
  • Then run the standard add/commit/push

This will work but will require renaming all the .git folders back during any restoration process. Is there any way to achieve this without this overhead?

migrated from serverfault.com Apr 19 '16 at 12:43

This question came from our site for system and network administrators.

  • What about simply adding the nested repos to your .gitignore? – duenni Apr 19 '16 at 12:26
  • I want them backed up to the remote server. I want an entire snapshot of the /home directory including other git repos. Effectively like copying/pasting the directory manually, but I want it tracked via git. If I ignore them, then the files will not be copied to the backup server – Tom B Apr 19 '16 at 12:29
  • The best way i see is adding the indexes of all sublevel repositories into gitignore, similar to duennis suggestion. the expression for this should be '\.git/**' – gerhard d. Apr 19 '16 at 13:13
  • 1
    Trying to use Git as a general backup too is not going to work well. It's a version control system, not a backup tool. Try using software that is actually designed for backups. I've recently been recommending Borg Backup, which is a young tool (a fork of attic) but which looks very promising. – Faheem Mitha Apr 19 '16 at 16:37
  • 1
    Are you going to branch your home directory? If not, git is the wrong tool (in the same sense that while you can hammer in a nail with a screwdriver, that's not the right tool for the job). Keeping history without branching is a different problem that rsync or tools built on it such as rsnapshot handle well. – Gilles Apr 19 '16 at 22:47
1

I know my comment is not a quite straight answer, but... I would recommend re-organizing your backup methodology in following ways (either / or):

  • Having single "major" repo with all subprojects inside it: one repo for /home I mean. That may look crazy at a first glance, but it depends on complexity of your sub-projects (subdirectories) and may work for something simple and which is not actively developed.
  • Discard your idea and maintain complete backup with special backup solution (rsync is only one of them and not the panacea). That make sense because "why do you need to backup all the git-related stuff?". It's better to look at this as on a higher level of your architecture that needs to be maintained in other way that git (git is something inner). Here I mean that If your main reason of "however it would be nice to have git track the changes over time" is the ability to roll-back the changes to some exact point, then Git is an overhead and not a solution you need. Differential/incremental + full backups with special software (where you can exclude all ".git" folders, btw) is a proper way.
  • Thanks for pointing me the the right direction. Having seen the responses here and from discussions elsewhere I can see that git is not the tool for this job. I'm currently looking at rsnapshot which should be able to do what I want and in a more sane way. – Tom B Apr 23 '16 at 20:54
2

Git over git is just a little crazy :)

  1. Try to add to gitignore dir with your repositories.
  2. Add in your cron script command to archive your repositories before commit.
  3. (Optioal) Setup auto commit/push to all your included repository.

IMHO, 1 & 2 will be good for you.

cd /home
tar -zcvf my-gits-backup.tar.gz path/to/git/projects
git add .
git commit -m "nightly backup"
git push
2

I also have a bunch of git repos and back them up upon logout, reboot or shutdown with rsync to my cloud. Its the simplest solution you will find out there.
I also heard about bup(https://github.com/bup/bup), but never used it, so I cannot tell you how well it works.
Myself, I wouldn't use git to backup multiple git repos (as I don't need a history of backups) The only history im interested in, is already inside the git repositories!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.