Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I have for many years had my entire $HOME directory checked into subversion. This has included all my dotfiles and application profiles, many scripts, tools and hacks, my preferred basic home directory structure, not a few oddball projects and a warehouse worth of random data. This was a good thing. While it lasted.

But it's gotten out of hand. The basic checkout is the same across dozens of systems, but not all that stuff is appropriate for all my machines. It doesn't even all play nicely with different distros.

I'm in the process of cleaning house -- separating the data out where it belongs, splitting out some scripts as separate projects, fixing some broken links in stuff that should be automated, etc.

My intent is to replace subversion with git for the toplevel checkout of $HOME, but I'd like to pare this down to just the things I'd like to have on ALL my systems, meaning dotfiles, a few directories and some basic custom scripts.

In reading up online a lot of people seem to be doing this using the symlink approach: clone into a subdirectory then create symlinks from $HOME into the repository. Having had my $HOME under full version control for over a decade, I don't like the idea of this aproach and I can't figure out why people seem so averse to the straight checkout method. Are there pitfalls I need to know about specific to git as a top level checkout for $HOME?

P.S. Partly as an exercise in good coding, I'm also planning on making my root checkout public on github. It's scary how much security sensitive information I've allowed to collect in files that aught to be sharable without a second thought! WiFi password, un-passphrased RSA keys, etc. Eeek!

share|improve this question
4  
Curious what lead to the belief that $HOME should be sharable without a second thought‽ Even encrypted RSA private keys shouldn't be shared. –  derobert Aug 28 '12 at 19:16
2  
if you are in fact talking about putting the contents of your home directory into git just note: it is difficult (but not impossible) to dig through git history and carefully remove sensitive items permanently (git is designed to help prevent losing things), and also remember that when you switch branches or checkout an earlier revision git will change the permissions of your files to 644 after checkout which is bad for things like private ssh keys. however etckeeper is a solution for using git with permissions for /etc/ –  cwd Aug 28 '12 at 19:29
    
@derobert: I'm well aware of that. I wasn't talking about making $HOME public, just dotfiles and convenience scripts. Those are where I've been finding stuff that doesn't belong. And yes, I should be able to share my .zshrc, .vimrc and similar things without having to sanitize them first! –  Caleb Aug 28 '12 at 19:53
3  
If you haven't seen it, see the vcs-home wiki and mailing lists, which is basically people discussing exactly this -- how to keep your $HOME under revision control. –  Jim Paris Aug 28 '12 at 20:02
    
I don't know how much you can change the behaviour of git, but at least the way it works out-of-the-debian-repository it is quite greedy when it comes to searching for tracked/untracked/modified files and it automatically feels responsible for every file. mrb already stated this. Sometimes I get annoyed by this greedy behaviour even in relatively small projects I wouldn't want that in my home directory. Why do want to use git? I'm also using a versioning system to sync my configuration files across hosts and I'm quite happy with CVS because it's so simple! Git is very (too!) powerfull for that –  Bananguin Aug 29 '12 at 7:54

6 Answers 6

I wouldn't want my entire home directory checked into version control simply because it means every subdirectory I go into would have the version-control context of my home dir. Commands like git checkout would have an actual action in that case, causing issues if I accidentally run something from the wrong directory, whether that something is git itself or a script that calls git.

It also makes it more likely to add something to the repo that you don't want, which would not have been an issue when you had everything checked in, but now becomes a problem. What if you accidentally add a private key file (perhaps out of habit) and push it to github?

Having said that, I think the primary disadvantages are not really technical — just wanting to save me from myself.

As for symlinking: You could clone your repo into a subdirectory, and have a script which updates any symlinks that need to be updated. The amount of maintenance required for this script might outweigh the benefits of having it at all, though; symlinking might turn out to be less work.

With symlinks, you can also easily make distro-specific (or even host-specific) additions that get checked into git. Your symlink-update script will ignore files intended for incompatible platforms or different hosts, and only update the appropriate ones.

Something like:

HOMEREPO=$HOME/homerepo
HOST=$(hostname)
UNAME=$(uname)

for dotfile in $HOMEREPO/shared/* $HOMEREPO/host-$HOST/* $HOMEREPO/uname-$UNAME/*
do
    target=$HOME/$(basename $dotfile)
    [ ! -r $target ] && ln -s $dotfile $target
done

Personally: I use symlinks, and I don't symlink directories; only the files within. This gives me some flexibility to make site-local changes in those directories (ie. add/remove files). Setting up my account on a new system is tedious because I have to recreate all the symlinks by hand.

share|improve this answer
    
Any git commands I run would either be for the home directory itself or would be buried at least one deep in a NON commited directory. Using svn this one folder isolation is quite effective and hasn't caused me any trouble in a decade. Your first paragraph indicates something else. Is this a actually a difference in the way git works? –  Caleb Aug 28 '12 at 19:57
    
Also my configs and scripts already have conditional logic for different hosts and platforms built in, so using a script to setup different links as conditionals doesn't seem like much of a gain over git's easy to mange branches. Still something I'm missing or is this going to come down to preference? –  Caleb Aug 28 '12 at 20:01
3  
One-folder isolation doesn't really isolate in git — not sure about svn — but for an example, git init foo && mkdir -p foo/bar/baz/spam && cd foo/bar/baz/spam && git status (or other git commands) show that you're still in foo's version control context. –  mrb Aug 28 '12 at 21:26
    
Configs & scripts: Not all dotfiles support conditionals, which is why I suggested the alternate approach. These are all reasons that I think people prefer not to use version control for $HOME — and versioning isn't really valuable for dotfiles imo — but ultimately it's your home directory, so if you prefer to use git and these aren't problems for you, go for it! –  mrb Aug 28 '12 at 21:30
    
Thanks for the info. Actually your comment about git not allowing for isolation is the most useful bit. You might work that prominently into your answer. Subversion behaves very differently on that point and its significant for this use case. –  Caleb Aug 31 '12 at 18:25
up vote 6 down vote accepted

Yes, there is at least one major pitfall when considering git to manage a home directory that is not a concern with subversion.

Git is both greedy and recursive by default.

Subversion will naively ignore anything it doesn't know about and it stops processing folders either up or down from your checkout when it reaches one that it doesn't know about (or that belongs to a different repository). Git, on the other hand, keeps recursing into all child directories making nested checkouts very complicated due to namespace issues. Since your home directory is likely also the place where you checkout and work on various other git repositories, having your home directory in git is almost certainly going to make your life an impossible mess.

As it turns out, this is the main reason people checkout their dotfiles into an isolated folder and then symlink into it. It keeps git out of the way when doing anything else in any child directory of your $HOME. While this is purely a matter of preference if checking your home into subversion, it becomes a matter of necessity if using git.

However, there is an alternate solution. Git allows for something called a "fake root" where all the repository machinery is hidden in an alternate folder that can be physically separated from the checkout working directory. The result is that the git toolkit won't get confused: it won't even SEE your repository, only the working copy. By setting a couple environment variables you can tip off git where to find the goods for those moments when you are managing your home directory. Without the environment variables set nobody is the wiser and your home looks like it's classic file-y self.

To make this trick flow a little smoother, there are some great tools out there. The vcs-home mailing list seems like the defacto place to start, and the about page has a convenient wrap up of howtos and people's experiences. Along the way are some nifty little tools like vcsh, mr. If you want to keep your home directory directly in git, vcsh is almost a must have tool. If you end up splitting your home directory into several repostories behind the scenes, combine vcsh with mr for quick and not very dirty way to manage it all at once.

share|improve this answer
1  
but why not just add '*' to your .gitignore file? That way git will ignore everything except files that are already in the repository, and you can add new files with git add -f <file>. –  ALiX Aug 31 '12 at 21:59
    
@ALiX: Because the git tools would still consider you to be working on your home dir repo even if you were in some subdirectory that was a separate git repo for some project. That solution would render your entire home directory off limits to all other git work. –  Caleb Aug 31 '12 at 22:05
3  
but a '*' in your .gitignore means that all files not in your home-dir repo are ignored. and when you check out a new git repo in some subdirectory, everything should still work as expected (i think). As far as I know, the git tools will look for the first .git directory while moving up the directory hierarchy. So, when working in the subdirectory, the correct git repository will be used. Of course, if you are using git's environment variables, I guess things could get messy. But otherwise, I don't see why this wouldn't work. –  ALiX Aug 31 '12 at 22:17
    
@ALiX is right. Nested git repos appear to work fine as long as you gitignore them in the parent repo. I wonder what are the drawbacks of this very simple approach, aside from possible problems with git's environment variables. –  evanrmurphy Oct 17 '13 at 5:19
    
Been experimenting with this today. I think /* works better than * because it still ignores everything by default but makes it much easier to add directories. Instead of git add -f I'm using !-prefixed patterns like !/.vimrc and !/.gitignore (for the .gitignore file itself) to explicitly include things in the repo. –  evanrmurphy Oct 17 '13 at 21:34

I've tried both, and preferred the symlink approach in the end:

  • Check out to wherever
  • make install
  • Log out and in again to load the X settings

Disadvantages:

  • Have to move files to the repo before adding them
  • Have to maintain the list of symbolic links in the Makefile

Advantages:

  • No need for a massive .gitignore (I have 133 dotfiles in ~ on my humble Ubuntu box)
  • Can keep maintenance scripts and other ~-related stuff (such as Makefile and cleanup.sh) out of the way
  • Can version control personal and public settings separately

Restrictions:

  • Unlike @mrb, I only create symlinks in ~. That keeps the symlinking simple, and makes it trivial to notice new files in for example ~/.vim, at the cost of some very rare .gitignore maintenance.

That last two advantages tipped the scales in my case - I don't want to clutter the home directory, and I want to keep private and public content clearly separate.

The only application I know of which has (or at least had) problems with handling symlinks was Pidgin - It kept overwriting my symlinks with ordinary files.

share|improve this answer
    
Thanks for your input on the pro's and cons of each approach. In my follow up I've discovered there is a third aproach that might make the best of both worlds if you're ok setting up the extra wiring to get started. –  Caleb Aug 31 '12 at 18:43

To give another point of view: I have my $HOME under git since sometime now and didn't find any drawbacks. I obviously do not sync this git repo to github; I use a service which have private repos. I also do not put any media files or downloads or packages under git control.

  • git status is a kind of "to do, to clean" checklist.

  • I have a ~/tmp for temporary things, which is gitignored.

  • I like to see in git status anything that a recently installed software dare to add to my $HOME, and often delete these files, or even uninstall the culprits.

  • I add manually the really useful local files and dirs to .gitignore, which has a 'know what you do when installing things' benefit.

  • If I build a new VM or install a new PC, I just clone my remote home to $HOME and have immediately everything I need under hands.

  • Things like vundle for vim plugins are not necessary anymore.

I dislike complexity. When I tweak any rcfile, I just do it, commit and push. Then, as a reflex, I git pull in $HOME every other day, and have always the latest config. It is that simple.

Machines currently under this regimen: Home laptop, work PC, work VM, plus 3 or 4 remote servers.

share|improve this answer
    
Do you have any other git checkouts nested inside your home? –  Caleb Sep 4 '12 at 7:56
    
No, I put other things in a /work directory and do not clone little tools like vim pugins. –  gb. Sep 5 '12 at 10:49
    
I have work inside a ~/Sites and do this approach as well, there's no problem with nested git repos –  philfreo Nov 2 '12 at 15:49

The warnings in some of the answers here are heavily exaggerated - like this one:

having your home directory in git is almost certainly going to make your life an impossible mess

I have been having my $HOME folder as a git repo for a long while and it works without any problems, as long as you know how git works it's going to be fine.

See it in action here: https://github.com/thnee/linux-profile

I also check out lots of work repos in sub directories of $HOME. When you cd into a sub directory, git will use that repo, and not the one higher up.

The only thing you have to look out for, is not adding those sub dirs to your $HOME repo. But you should already be having full control of what you're adding, if you are using any VCS.

To help with this I have a $HOME/.gitignore with an asterisk (*) in it, and add files with git add -f.

It can also be useful to know that git supports adding other repos under a repo, this is called submodule: http://git-scm.com/docs/git-submodule

share|improve this answer

My major problem with using Git on the home directory is that Git does not store file attributes such as file permissions and timestamps. For me it is important to know when certain files were created, that may or may not be the case for you. Furthermore, loosing permissions to files and directories such as .ssh is problematic. I understand that you plan on keeping .ssh out of Git, but there will be other places where permissions might matter (such as uncompressed website backups).

share|improve this answer
    
This is misleading if not factually wrong. Git by default preserves many file attributes including permissions. I've been keeping .ssh in git for some time now without an issue, proper secure permissions are preserved. What it doesn't do in the base configuration is preserve ownership or timestamps; yet if either of these this is an issue for a specific use case there are plugins that can make handling these additional properties part of the regular workflow (see metastore or git-cache-meta). –  Caleb Sep 21 '13 at 9:00
    
Even if it doesn't store them, how is that any worse than just having a home directory not in vcs? git is not going to actively overwrite mtimes unless you ask it to change a file. –  poolie Apr 9 at 23:51

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.