I’m a big fan of chezmoi (https://www.chezmoi.io/) which is a very capable dotfile manager. Chezmoi supports some useful advanced capabilities like work/home profiles and secrets manager integration.
Same for me. I'd done the same thing as the author with various methods like stow, symlink farms, etc. over the years. Chezmoi is good enough that I'm willing to let someone else handle maintaining all logic.
Yup, I tried a number of dotfile managers. I think yadm was the first one I started with and then ended up with chezmoi.
The main reason was because I discovered the power of templating. With Yadm it required an external dependency, envptl, then j2cli, and both of these became unmaintained, while chezmoi used the text/template standard library. After the task of converting my jinja2 templates to gotmpl I never looked back.
One of the other things I like about chezmoi is I significantly cut down any "scripts" to just a few as most of the logic became "deterministic", ie I would set conditions based on the host in chezmoi.toml.tmpl and then that would define how everything under that would run across multiple hosts, and devices.
I migrated to chezmoi recently my only gripe is `chezmoi cd` opening in a new shell but `chezmoi git` usually is what I need. The age [0] integration is nice.
I added an alias `cm='cd $(chezmoi source-path)'` to my shell config to cd to the chezmoi directory (without opening a new shell) so I can use all the usual commands (e.g. git) without need the chezmoi prefix. The alias is in a chezmoi-managed file, naturally.
Hey, I had never heard about chezmoi before reading your comment, but I just installed it. Took less than 10 minutes to set up from start to finish. I noticed that if you choose to use it to manage your `~/.ssh/config/`, by default chezmoi sets it up as `private_dot_ssh/` and so if your dotfiles are public it doesn't expose sensitive data like private key files such as `~/.ssh/id_rsa`. Smart!
The private_ only applies to file permissions so in this case it makes the .ssh directory only readable by the owner. This is checked for by openssh and the config will be ignored if it's readable by the group or all.
If you make your dotfiles repo publicly accessible, you will leak your private keys unless you use other features in chezmoi to protect them.
also a big fan of it because the templating feature makes it very easy to handle dotfiles with different locations on multiple machines and if you use multiple operating systems. Really not that many tools around that have good windows support.
What happens when you need to link a file that does not support comments like that? For example, something which stores its config as plain JSON.
Or how about when you want to symlink an entire directory? For example something like neovim, considering that you may want to split config into separate files for organization. My neovim configuration has an "autoload setup" so any lua files inside the config directory are automatically required.
Lastly, this approach does not appear to support running commands. My dotfile install script ensures that tmux plugins are installed, the terminal font I use is available, and some other stuff that you need to invoke a command or script to achieve.
I like that the approach is simple, but I do not think it can support even relatively common use cases very well.
I'm not using a dotfiles manager, i track my ~/.config in git and have a script that globs for ~/.config/*/dot.* files to create symlinks for them. Like ~/.config/bash/dot.bashrc . Works with directories.
I prefer using ONE symlinked ~/.zshrc -> ~/dotfiles/etc/zsh/.zshrc and then using ENV-Variables within the .zshrc to specifiy other config file locations:
So one symlink is enough :-) Something similar can be done for ssh config (this file can not be symlinked for security reasons, so be careful working around this "feature"):
# contents of $HOME/.ssh/config
Include ~/dotfiles/etc/ssh/hosts.d/*
Shouldn't you at least put those environment variables in ~/.profile? You'd want to make sure applications like VS Code and whatnot pick up the configurations. ~/.zshrc would only be used for interactive zsh sessions, which not all applications would respect/use.
Every time I see these tools for “managing” dotfiles, or something for “managing” notes I get a bit perplexed as to what the use is, but then I am reminded of and impressed with how different people’s brains are and how we work and think.
I was getting by with bare git and then YADM until I grew tired of managing the logic to keep everything in sync across multiple machines and types or versions of OS, on top of managing my configs already.
Or if you follow just the right amount. Apart from the implementation issues, it's a great idea. MacOS essentially has a similar thing available through defaults.
What I realized after 25 years is that configuration comes in three parts:
1/ the defaults, either built in or read from /etc;
2/ my defaults, included in each file (or with ssh, at the bottom) with that particular config’s native version of #include; and
3/ local specifics that are rarely if ever used anywhere else, or trivially short as to be copy-paste-able.
Almost everything I want to customize goes into (2) so I wrote a single Python function that manages a block at the top (or with ssh, at the bottom) of each config file:
# BEGIN my foo stuff
include = /my/repo/foo/config
# END my foo stuff
That way foo starts out with (1) the system defaults; then adds (2) my personal foo defaults as defined in a working copy at /my/repo; (3) anything else I insert in the file after that which isn’t centrally managed and that’s ok.
I haven’t ever needed anything more complicated. I do not have any work specific configs that I need to gate. I no longer have to manage different configs based on whether I am using Debian, Debian (old), Debian (very old), SunOS (very very old), or AIX (very very very old) because those days are behind me.
If you do still need to manage slightly different but ethereally different configs on different hosts then I’m sorry to hear that. Rationalising my computing life so that I use the latest version of some Linux distribution everywhere has been very helpful!
IMO you don't need a special tool to manage your home directory / dotfiles. Git is the tool. Your home directory is a repo with a .git directory like any other repo. No other tools; no symlinks; nothing else. Commit what you want and gitignore the rest. I've done this since 2008.
That's what I do as well. Since you can .gitignore entire directories, that makes it easy. And one major advantage is having `git status` tell you if new things show up, so you can decide whether to track or ignore them (or change how/whether they're created in the first place).
I just put dotfiles directly where they are and majr a .git directory in my $HOME. .gitignore everything and git add -f files when I need to. no symlinking or anything.
Some responders here almost seem offended you wouldn't use chezmoi. Kind of strange. I couldn't care less about someone using chezmoi/tool x - big deal - but I'm always interested to read how people approach automation of their workflows, the tradeoffs, any cool tricks they employed that I might not have heard of before. Are engineers discouraged from problem solving now? It's not like youre doing this on company time; you are allowed to do things for fun/interest's sake..
In my case, I have things like Github Personal Access Tokens stored in my password manager and then use chezmoi's templates to populate them in my dotfiles, for example:
This means that:
1. My secrets are safely stored in my password manager so I can share my dotfiles.
2. When I update the secret in my password manager it automatically gets updated in my dotfiles when I run `chezmoi apply`.
I know a lot of folks are talking about what they do, or what dot file managers they use, but there's something to be said for building a workflow that works for you. Pretty clever setup!
> what they do, or what dot file managers they use, but there's something to be said for building a workflow that works for you
Like consuming time. If there is a tool which does what you need like chezmoi then you should use it, so that you don't have to spend maintaining something bespoke which consumes your time that could be better spent on other things.
Stupid argument. With this logic, nothing new is ever explored, nothing is learned, no insights gained, or shared. This logic even questions the need for chezmoi - stow existed for decades before chezmoi. You think chezmoi sprang into existence with all its features? High chance it was someone's toy project because they didn't want to use stow.
stow's approach of using symlinks is extremely limiting. For example, it means you can't have templates (for small machine-to-machine differences and secrets) or encrypted files. chezmoi does have a symlink mode, like stow, but using symlinks has multiple downsides: https://www.chezmoi.io/user-guide/frequently-asked-questions...
chezmoi was actually inspired by Puppet, not stow.
Especially given the fact that things are moving (too slowly) towards the XDG specification, my dotfiles repo is just my ~/.config directory, with a policy of basically ignoring everything except those things that I want to track.
I then have a directory under .config, .config/dotfiles, where I have all of my unfixable dotfiles without the leading ., so to install them I have a script that just does `ln -snf ./$x ~/.$x` instead of messing with sed scripts.
This is both self-contained and allows me to manage both XDG-style config and traditional dotfiles.
I do exactly the same: I have a Git repo that is cloned into ~/.config and that covers most of the terminal apps that I use.
For the holdouts that don’t yet support that config directory, I have a short Makefile that sets up the required symlinks. So running “make” makes the links I’ll most likely need on a new server, while e.g. “make ssh” makes only the links required for that specific program.
Now that tmux supports ~/.config, and vim just added support as well, that Makefile is shrinking.
It's interesting and a fun hacking challenge for someone so inclined but it also seems slightly ... I don't know ... extraneous? But the community that needs dotfiles is diverse enough to support everything from Linux From Scratch devotees to chezmoi users, so that's cool.
Nice tricks, though in a sane world this would be metadata on the config files and not in-band data.
Of course, (ab)using comment syntax for structured machine directives is something many programming languages end up doing too. Here's a recent example: https://peps.python.org/pep-0723/#example. Surrounded with a pair of "# ///"? It must be something to do with Adidas.
But not all things you might do with a dotfile (or, more generally, per-user customization) are just replacing files. Things like cronjobs, brew installs, `defaults` in MacOS, etc. Viewing dotfile-based customization as strictly files to obliterate with pre-existing files is needlessly myopic.
For this broader problem, there are other more complete solutions that are more robust and flexible. Personally I like dotbot (https://github.com/anishathalye/dotbot) as a balance between power and simplicity, particularly when managing files across multiple OS homedirs (e.g. linux server, macos laptop).
That's provisioning, not dotfiles management. My dotfiles only includes config files. I'd just use the package manager to install packages and I'd just use the relevant program to enable stuff. As I use stow, I just create different configurations for different OS if they differ too much. At most, a handful of scripts to customize my user account.
Dotfiles are just a component, but not the whole story, of your personal compute environment. Your environment also includes things like:
* ~/bin scripts (etc)
* programming language stuff - e.g. go, rust, python, ruby etc have tooling for per-user package management, language version, etc.
* various forms of password/key/auth stuff like ssh allow lists, encrypted password stores, etc.
And the biggest one: Type of machine - work, daily driver, server, etc
The type of machine may require different dotfiles or different parts of dotfiles (e.g. what basrc includes from `. .config/bash/my_local_funcs`), and having some scripting around this makes life easier.
Similarly OS packages are great, and I use them heavily, but work and personal servers and personal desktop all use a different OS, so its useful to have provision scripts for the type of machine, and i keep all that together with my dotfiles (etc) in my "personal environment repo" (it's name is dots, and when i talk about dotfiles I really mean "personal environment". I suspect other share this view, which leads to this "pure dotfiles" vs "dotfiles+parts of provisioning" viewpoint difference even though they largely have the same set of problems and tooling.
The majority of my computing happens at my workstation (desktop). That is what I consider my personal environment, and I would script its setup, but I can't find the motivation to do so (and I like to do ad-hoc changes). Permanent configuration (related to my usage, not the computer. My core utilities, I can say) get added to my dotfiles. As for server and works, their intersection and my personal stuff are minimal (mostly bash, vim, emacs?) I'd rather have a different system/project to manage them.
This is why I use Nix + home-manager to manage my CLI, programming environment, and system configuration across Linux, macOS and WSL using one GitHub repo. It also handles differences across machine types well.
A dot file management system is only part of the picture.
To spin up a new machine is a 30 minute job, and then it feels like “home”.
I imagine that things like provisioning are essential to people that switch computers often. So it's not a dotfile-specific problem, but more of a dotfile-adjacent problem.
There's so many interesting edge-cases that affect UX even when distro-hopping between Debian-based distros... especially if you used it for several years and had plenty of custom scripts in your ~/.local/bin folder.
I may yet need to learn or (re)discover some best practices of how to get up to a working development environment faster. I'm thinking of using Guix for that... but I digress.
So far, my workflow goes like this (on a newly-installed distro):
1. Configure environment variables that affect package-specific file locations (/etc/security/pam_env.conf and a custom /etc/profile.d/xdg_std_home.sh script that creates and assigns correct permissions for required directories).
2. Provision packages
3. Deploy config files (using stow).
What I've yet to figure out (haven't really researched it yet), how do you handle app-specific configs (think Firefox add-ons, add-on configs, Thunderbird accounts, etc.)?
"Switch computers often" can also apply to "switch computers with little notice". Even if 95% of my time is spent on one computer, it's nice to know my config is safely squirreled away and, uh, trivially unsquirrelable if something terrible happens to this hardware and I have to get another computer. Seems like a relatively low probability event, but my child has already destroyed two ThinkPads (both were very old and very disposable--still an accomplishment).
As to your last question, nix+home manager gets you there, but that's a whole other Thing.
(n)vim for example: my dotfiles don't vendor the handful of plugins i use, they just include the directives to install those with plugin manager.
I generally use a makefile + stow to handle my dotfiles and home-dir setup. Each program has an entry in this Makefile - most of them are very simple, I keep a list of programs who's dots need to be in ~, and another for ~/.config/ and using make's variable expansion they just get a stow target.
This also allows me to not just copy preference, but provision a bunch of stuff that's invariant across machines (e.g. what i have installed via rustup, go install, etc).
Well, you can't have different configs for different hosts. Other than that, I can't quickly recall what other limitations are, I see none. I really like the simplicity of the "pure git" approach.
My dotfiles repo dates back to 2018, I'm happy user of this git one-liner for the past 6 years.
Might be niche but for me - I have config files outside $HOME, I use a number of `.gitignore`-aware tools (tree, fuzzy finder), and I just don't like `git status` telling me I'm in a repo in any subdir of $HOME.
Glad someone else already posted it. I understand if Nix is too much for folks, but it repeatedly is absolutely end-game for stuff like this. I have centralized, unified dotfiles, with all of the power of Nix to have one-off config flexed in. No extra templating or hacky interpolation.
- Make a new system setup script, run on other machine:
# install git; cd ~
# I use a read-only token, optional:
git clone "https://x-token-auth:${TOKEN}@bitbucket.org/you/dot_repo.git" dot_repo
# move into $HOME
cp -afv dot_repo/. .
rm -rf dot_repo
git config --local status.showUntrackedFiles no
Later on, if I want to write to the repo on this machine I run ssh-keygen, copy the public key to the remote and remove the token from .git/config and use ssh access instead.
I used to have everything excluded in .gitignore and force add files to the repo, but prefer status.showUntrackedFiles instead. There are still a few edge cases but they don't bother you everyday like having to force every operation.
Some of my scripts have things like, if dist fedora, do this, else debian, do this, else Mac, do that, when they differ.
I like the storing of the ln command in the file itself. I think I might implement something like that myself with my own dotfiles. I've been keeping the ln commands in a readme but I never really liked having to update that readme each time I add or remove a file to my dotfiles repo (and using a dotfile manager like those mentioned in other comments here does not appeal to me at all). This is a nice solution I hadn't considered. Now I can just replace the list of ln commands in my readme with a one-liner to run that'll show the ln commands from the files. (I don't actually want to automate the actual running of the commands, though.)
>fta: first-line comments of the form <comment delimiter> ln <link name> in files which needed to be symlinked.
if you're talking about symlinks, you ought to say "ln -s" shouldn't you?
(also, I think this is the tip of another iceberg: learn to say "link" when you mean "hardlink" because that's what a link is in unix filesystem. not saying "stamp out hardlink", I'm saying if you feel comfortable using it correctly yourself, it will help you help other people to disambiguate what they say and not get progressively sloppier. you may not like what these words mean, but that ship is well at sea)
I disagree with the latter. You're correct, but I think it overlooks human factors. If I can say or write one more syllable and remove all shadow of a doubt that I do, in fact, mean a hard link, then that's a syllable well spent. Until we can get 100% of people to immediately think of the hard link concept when someone says "link" in a Unix context, not spelling it out leaves too much room for misinterpretation.
you are overlooking the human factor I pointed out: you have no idea if the other person has removed the ambiguity from what they said. My point was to be comfortable with the correct language yourself so you'll more easily spot slip-ups, not to mention be able to read docs. I specifically said not trying to stamp out hardlink.
>Until we can get 100% of people to immediately think of the hard link concept
that's not the goal (being impossible). my goal is to get OP to be cleaner in his doc, and for anybody who cares to be cleaner in their usage.
Not using "link" to mean "symlink" is certainly a reasonable improvement. But using "link" to mean "hardlink" (outside of a context like "the link system call") creates the possibility of ambiguity.
You're suggesting "being comfortable with the correct language", but if it is not already the case that almost everyone means "hardlink" when they say "link", then it's not "the correct language". Your mental mappings should reflect actual language as actually used, so that you can understand and be understood. Those mappings should include knowledge of the ambiguity.
"hardlink" unambiguously means "hardlink" (modulo rare mistakes)
"symlink" unambiguously means "symlink" (modulo rare mistakes)
"link" means "probably hardlink, but possibly symlink being referred to sloppily, or possibly referring to the category that includes both; generally ambiguous without further contextual information".
the position you are advocating leads precisely to TFA which has it precisely wrong, that's why I went to the trouble to write a comment that makes people think. I don't need to hear a detailed counter argument for the status quo ante, just write "we'll just thoughtlessly do it the same shitty way you're arguing against"
No, as mentioned I am not suggesting using "link" to mean "symlink". I am suggesting being aware that not everyone (speaker or listener) will have the word "link" mapped unambiguously to "hardlink", and that if you hear "hardlink" it's likely the speaker means "hardlink", while if you hear "link" you can't be quite as sure.
In other words: I am advocating against deleting error-correcting bits from language, because language is a noisy channel and error-correcting bits help reduce errors.
unlinking is what you are doing: deletion only takes place when you unlink the last link, so yes, great idea, that's why the syscall is called unlink() https://www.man7.org/linux/man-pages/man2/unlink.2.html and the command is not called delete, but rm=remove
I was tempted to the the same thing as OP, but decided to go with stow, because its concept is more sound than whatever I may concoct. Currently, my dotfiles are packages (folder) which targets $HOME. I clone the repo to $HOME/dotfiles, then I can use `stow <package>` inside the folder to install each package easily. I treat it like dpkg|rpm|... but for my configuration. I'm trying to make each package its own thing, like mail, sway, i3, etc,...
I don't have any experience with chezmoi. I've managed my dotfiles manually and with the bare git repo trick, but I've looked at chezmoi's docs and it looks like ansible, but for your $HOME. I only have a few computers I want my dotfiles on and stow has been perfect for having my dotfiles in a single folder and linking them where I want to be. Only a few of these configs I'd want somewhere else and I'd either create a new repo (work computers) or copy the files (random place)
Big +1 to forgetting about dotfile managers, and scripting it out.
I just keep my dotfiles repo in the same tree structure as the home directory, and loop over the tree to create symlinks. Plus some miscellaneous commands to set some other things up.
> I just keep my dotfiles repo in the same tree structure as the home directory, and loop over the tree to create symlinks.
Depending on the use case, the `/etc/skel` directory (and equivalents depending on the distro/OS) might be useful. When creating a user, the files in $HOME are copied from such "skeleton" directory, and there's usually a way to tell that command to use a different skeleton directory.
So a different way (not better, just different) would be to have a directory already setup with symlinks and all, and use that directory as the skeleton when creating the user, so its $HOME gets created all ready with symlinks and all.
In my dotfiles I have a couple folders which includes rcfiles and configs. rcfiles includes things like {bash,zsh}rc, .vim{,rc}, tmux.conf, and so on, as well as folders like zsh that include things I import like aliases I have for specific linux machines (e.g. ubuntu has batcat instead of bat...) or osx. Then in config I have folders that contains all the things I would have under ~/.config (starship.toml, ipython_config.py, wezterm/<only lunatics have a single config if you have more than 50 total lines>, and so on. Then I just
I mean I have other folders too like scripts, skels, templates, systemd configs, notes (notes in dotfiles is underappreciated!), and so on. What are you all using these managers for? Are they replacements for bash scripting? And also, find is super powerful and I think under appreciated. It really is worth learning. If you jump into the deepend I think you can get good at it in an afternoon.
The problem isn't you. It's the way Unix-inspired systems do things that are entirely wrong. It's an insane mess of all of the things you mentioned because all the programs could do whatever they wanted. Bad conventions emerged and were propagated throughout the ecosystem.
Here's a fun exercise: try to change your home directory's physical location on a *nix system and see if anything works afterwards. It won't because every config thinks your files are in /home/me while you've changed your user name/home directory to /home/new-me. Windows (eventually) actually got this (approximately) right with a 'virtual directory' for 'my home directory' regardless of where exactly it is on disk. Programs refer to that virtual location.
The current status quo on any *nix is absurd.
(Just FTR, I only use Linux systems, personally. My criticism is borne from a place of aspiration/hope.)
Yeah I've never really had a problem either. Even with using skels.
Besides, install scripts are the norm for me, so it's typically easy to fix when things do go wrong.
Pro tip: "${HOME%/}/" will always result in /home/godelski even if I include my last name and even if I accidentally modified it to have a / at the end. Some simply variable substitutions go a long way.
I’m a big fan of chezmoi (https://www.chezmoi.io/) which is a very capable dotfile manager. Chezmoi supports some useful advanced capabilities like work/home profiles and secrets manager integration.
Same for me. I'd done the same thing as the author with various methods like stow, symlink farms, etc. over the years. Chezmoi is good enough that I'm willing to let someone else handle maintaining all logic.
Yup, I tried a number of dotfile managers. I think yadm was the first one I started with and then ended up with chezmoi.
The main reason was because I discovered the power of templating. With Yadm it required an external dependency, envptl, then j2cli, and both of these became unmaintained, while chezmoi used the text/template standard library. After the task of converting my jinja2 templates to gotmpl I never looked back.
One of the other things I like about chezmoi is I significantly cut down any "scripts" to just a few as most of the logic became "deterministic", ie I would set conditions based on the host in chezmoi.toml.tmpl and then that would define how everything under that would run across multiple hosts, and devices.
I migrated to chezmoi recently my only gripe is `chezmoi cd` opening in a new shell but `chezmoi git` usually is what I need. The age [0] integration is nice.
[0] - https://github.com/FiloSottile/age
I added an alias `cm='cd $(chezmoi source-path)'` to my shell config to cd to the chezmoi directory (without opening a new shell) so I can use all the usual commands (e.g. git) without need the chezmoi prefix. The alias is in a chezmoi-managed file, naturally.
Unfortunately there's no other way of doing this, except for adding a shell alias. See https://www.chezmoi.io/user-guide/frequently-asked-questions....
Makes sense. Thank you for the project.
Chezmoi is amazing. I dabbled with Stow, but Chezmoi is the way to go.
Hey, I had never heard about chezmoi before reading your comment, but I just installed it. Took less than 10 minutes to set up from start to finish. I noticed that if you choose to use it to manage your `~/.ssh/config/`, by default chezmoi sets it up as `private_dot_ssh/` and so if your dotfiles are public it doesn't expose sensitive data like private key files such as `~/.ssh/id_rsa`. Smart!
The private_ only applies to file permissions so in this case it makes the .ssh directory only readable by the owner. This is checked for by openssh and the config will be ignored if it's readable by the group or all.
If you make your dotfiles repo publicly accessible, you will leak your private keys unless you use other features in chezmoi to protect them.
also a big fan of it because the templating feature makes it very easy to handle dotfiles with different locations on multiple machines and if you use multiple operating systems. Really not that many tools around that have good windows support.
What happens when you need to link a file that does not support comments like that? For example, something which stores its config as plain JSON.
Or how about when you want to symlink an entire directory? For example something like neovim, considering that you may want to split config into separate files for organization. My neovim configuration has an "autoload setup" so any lua files inside the config directory are automatically required.
Lastly, this approach does not appear to support running commands. My dotfile install script ensures that tmux plugins are installed, the terminal font I use is available, and some other stuff that you need to invoke a command or script to achieve.
I like that the approach is simple, but I do not think it can support even relatively common use cases very well.
I'm not using a dotfiles manager, i track my ~/.config in git and have a script that globs for ~/.config/*/dot.* files to create symlinks for them. Like ~/.config/bash/dot.bashrc . Works with directories.
I prefer using ONE symlinked ~/.zshrc -> ~/dotfiles/etc/zsh/.zshrc and then using ENV-Variables within the .zshrc to specifiy other config file locations:
So one symlink is enough :-) Something similar can be done for ssh config (this file can not be symlinked for security reasons, so be careful working around this "feature"):Shouldn't you at least put those environment variables in ~/.profile? You'd want to make sure applications like VS Code and whatnot pick up the configurations. ~/.zshrc would only be used for interactive zsh sessions, which not all applications would respect/use.
Yeah I should, but since it only contains tools I tend to start from zsh, I did not care too much :-) thanks for the hint though, almost forgot that.
Consult the documentation for your programs whether they already look in ~/.config and thus don't need a environment variable.
I don't want them in `.config` though :-) I'd like to specify the location to a personal git repository, that is not named `.config` :-)
The JSON hiccup occured to me - I don't have any JSON dotfiles but I guess I'd store it as hjson (https://hjson.github.io/) and add a conversion step.
Symlinking a directory - admittedly didn't come up for my dotfiles, maybe `.ln` files with a similar format in dir roots.
Commands - yes, I still keep a set of shell scripts alongside my config files.
Huh. Seems like an interesting alternative to JSONC.
Sounds like a job for a personal Ansible playbook.
Every time I see these tools for “managing” dotfiles, or something for “managing” notes I get a bit perplexed as to what the use is, but then I am reminded of and impressed with how different people’s brains are and how we work and think.
I was getting by with bare git and then YADM until I grew tired of managing the logic to keep everything in sync across multiple machines and types or versions of OS, on top of managing my configs already.
https://www.chezmoi.io/user-guide/manage-machine-to-machine-...
I always think "managing dotfiles" is the path that leads to Windows Registry if you follow it too far.
Or if you follow just the right amount. Apart from the implementation issues, it's a great idea. MacOS essentially has a similar thing available through defaults.
What I realized after 25 years is that configuration comes in three parts:
1/ the defaults, either built in or read from /etc;
2/ my defaults, included in each file (or with ssh, at the bottom) with that particular config’s native version of #include; and
3/ local specifics that are rarely if ever used anywhere else, or trivially short as to be copy-paste-able.
Almost everything I want to customize goes into (2) so I wrote a single Python function that manages a block at the top (or with ssh, at the bottom) of each config file:
That way foo starts out with (1) the system defaults; then adds (2) my personal foo defaults as defined in a working copy at /my/repo; (3) anything else I insert in the file after that which isn’t centrally managed and that’s ok.I haven’t ever needed anything more complicated. I do not have any work specific configs that I need to gate. I no longer have to manage different configs based on whether I am using Debian, Debian (old), Debian (very old), SunOS (very very old), or AIX (very very very old) because those days are behind me.
If you do still need to manage slightly different but ethereally different configs on different hosts then I’m sorry to hear that. Rationalising my computing life so that I use the latest version of some Linux distribution everywhere has been very helpful!
I use Gary Bernhardt's (non)-method, quoted here:
IMO you don't need a special tool to manage your home directory / dotfiles. Git is the tool. Your home directory is a repo with a .git directory like any other repo. No other tools; no symlinks; nothing else. Commit what you want and gitignore the rest. I've done this since 2008.
Never had a problem with it.
That's what I do as well. Since you can .gitignore entire directories, that makes it easy. And one major advantage is having `git status` tell you if new things show up, so you can decide whether to track or ignore them (or change how/whether they're created in the first place).
One better, put a '*' in your .gitignore and then !.vimrc the files you do want to track.
yadm implements this model, but also allows for machine-specific branching and encrypted secrets (no secrets should be directly committed in git).
One version of this is to use a bare repo, as described in https://news.ycombinator.com/item?id=11070797
I switched to this method after using symlinks for years. This guide was helpful: https://queensidecastle.com/guides/tracking-your-home-direct...
I just put dotfiles directly where they are and majr a .git directory in my $HOME. .gitignore everything and git add -f files when I need to. no symlinking or anything.
For those who are convinced and want a slightly more convenient tool for doing this: try yadm.
As someone who used yadm, and switched to chezmoi, the main reason was templating was so much better in chezmoi, no external dependencies.
Also less need on any scripts to set things up, due to chezmoi being more "deterministic" which means applying everything was a lot faster too.
If you want to avoid force:
(OP) appreciate all the feedback. I tried stow and chezmoi, neither felt right to me. Evidently that puts me in a small minority!
On the off chance there's anyone else who sees this as worth exploring here's a <1min demo video - https://www.reddit.com/r/unixporn/comments/1f9u1xk/oc_better...
Some responders here almost seem offended you wouldn't use chezmoi. Kind of strange. I couldn't care less about someone using chezmoi/tool x - big deal - but I'm always interested to read how people approach automation of their workflows, the tradeoffs, any cool tricks they employed that I might not have heard of before. Are engineers discouraged from problem solving now? It's not like youre doing this on company time; you are allowed to do things for fun/interest's sake..
I just do this. Easy
https://www.atlassian.com/git/tutorials/dotfiles
Same! Nothing better than git to store a bunch of text files!
How do you handle differences between machines? How do you handle secrets? How do you run scripts?
Scripts check environment to see where they are.
What secrets? I generally don’t copy them, but you could manually. For example I use the same ssh keypair on two machines, but I generally don’t.
In my case, I have things like Github Personal Access Tokens stored in my password manager and then use chezmoi's templates to populate them in my dotfiles, for example:
https://github.com/twpayne/dotfiles/blob/master/home/dot_zsh...
This means that: 1. My secrets are safely stored in my password manager so I can share my dotfiles. 2. When I update the secret in my password manager it automatically gets updated in my dotfiles when I run `chezmoi apply`.
Hmm, I don’t keep any secrets in dot files and have never thought to. Seems simpler that way.
To clarify, .ssh/* are “dot files” and have a private key but I dont track them in the repo.
I have one machine at any time
I store them in Bitwarden not in dotfiles
I… am not sure what that has to do with scripts?
I know a lot of folks are talking about what they do, or what dot file managers they use, but there's something to be said for building a workflow that works for you. Pretty clever setup!
> what they do, or what dot file managers they use, but there's something to be said for building a workflow that works for you
Like consuming time. If there is a tool which does what you need like chezmoi then you should use it, so that you don't have to spend maintaining something bespoke which consumes your time that could be better spent on other things.
Stupid argument. With this logic, nothing new is ever explored, nothing is learned, no insights gained, or shared. This logic even questions the need for chezmoi - stow existed for decades before chezmoi. You think chezmoi sprang into existence with all its features? High chance it was someone's toy project because they didn't want to use stow.
stow's approach of using symlinks is extremely limiting. For example, it means you can't have templates (for small machine-to-machine differences and secrets) or encrypted files. chezmoi does have a symlink mode, like stow, but using symlinks has multiple downsides: https://www.chezmoi.io/user-guide/frequently-asked-questions...
chezmoi was actually inspired by Puppet, not stow.
Source: me, I'm the author of chezmoi.
Especially given the fact that things are moving (too slowly) towards the XDG specification, my dotfiles repo is just my ~/.config directory, with a policy of basically ignoring everything except those things that I want to track.
So my .gitignore looks like
I then have a directory under .config, .config/dotfiles, where I have all of my unfixable dotfiles without the leading ., so to install them I have a script that just does `ln -snf ./$x ~/.$x` instead of messing with sed scripts.This is both self-contained and allows me to manage both XDG-style config and traditional dotfiles.
I do exactly the same: I have a Git repo that is cloned into ~/.config and that covers most of the terminal apps that I use.
For the holdouts that don’t yet support that config directory, I have a short Makefile that sets up the required symlinks. So running “make” makes the links I’ll most likely need on a new server, while e.g. “make ssh” makes only the links required for that specific program.
Now that tmux supports ~/.config, and vim just added support as well, that Makefile is shrinking.
https://chezmoi.io/
This is so much easier and more full featured.
> I avoided superfluous dotfile managers and the limitations of the "version control $HOME" method by instead including first-line comments
ok...
> The files can be scanned and symlinks can be created with some awk magic:
wut.
this is just bespoke file management scheme. we've all been there, but there are better tools for this, as other comments have mentioned.
It's interesting and a fun hacking challenge for someone so inclined but it also seems slightly ... I don't know ... extraneous? But the community that needs dotfiles is diverse enough to support everything from Linux From Scratch devotees to chezmoi users, so that's cool.
Nice tricks, though in a sane world this would be metadata on the config files and not in-band data.
Of course, (ab)using comment syntax for structured machine directives is something many programming languages end up doing too. Here's a recent example: https://peps.python.org/pep-0723/#example. Surrounded with a pair of "# ///"? It must be something to do with Adidas.
But not all things you might do with a dotfile (or, more generally, per-user customization) are just replacing files. Things like cronjobs, brew installs, `defaults` in MacOS, etc. Viewing dotfile-based customization as strictly files to obliterate with pre-existing files is needlessly myopic.
For this broader problem, there are other more complete solutions that are more robust and flexible. Personally I like dotbot (https://github.com/anishathalye/dotbot) as a balance between power and simplicity, particularly when managing files across multiple OS homedirs (e.g. linux server, macos laptop).
I'm not suggesting you do this (and I certainly don't) but arguably you could still manage that with just files on Linux boxes:
1. Cronjobs replaced with systemd user timers
2. User packages (i.e. brew install or $HOME/bin) with systemd user services and distrobox manifest files
3. I don't think there's a `defaults` equivalent on Linux or at least not one that isn't file based (and thus manageable through dotfiles)
So maybe that's just an OSX concern.
That's provisioning, not dotfiles management. My dotfiles only includes config files. I'd just use the package manager to install packages and I'd just use the relevant program to enable stuff. As I use stow, I just create different configurations for different OS if they differ too much. At most, a handful of scripts to customize my user account.
A different view worth considering:
Dotfiles are just a component, but not the whole story, of your personal compute environment. Your environment also includes things like:
* ~/bin scripts (etc)
* programming language stuff - e.g. go, rust, python, ruby etc have tooling for per-user package management, language version, etc.
* various forms of password/key/auth stuff like ssh allow lists, encrypted password stores, etc.
And the biggest one: Type of machine - work, daily driver, server, etc
The type of machine may require different dotfiles or different parts of dotfiles (e.g. what basrc includes from `. .config/bash/my_local_funcs`), and having some scripting around this makes life easier.
Similarly OS packages are great, and I use them heavily, but work and personal servers and personal desktop all use a different OS, so its useful to have provision scripts for the type of machine, and i keep all that together with my dotfiles (etc) in my "personal environment repo" (it's name is dots, and when i talk about dotfiles I really mean "personal environment". I suspect other share this view, which leads to this "pure dotfiles" vs "dotfiles+parts of provisioning" viewpoint difference even though they largely have the same set of problems and tooling.
The majority of my computing happens at my workstation (desktop). That is what I consider my personal environment, and I would script its setup, but I can't find the motivation to do so (and I like to do ad-hoc changes). Permanent configuration (related to my usage, not the computer. My core utilities, I can say) get added to my dotfiles. As for server and works, their intersection and my personal stuff are minimal (mostly bash, vim, emacs?) I'd rather have a different system/project to manage them.
This is why I use Nix + home-manager to manage my CLI, programming environment, and system configuration across Linux, macOS and WSL using one GitHub repo. It also handles differences across machine types well.
A dot file management system is only part of the picture.
To spin up a new machine is a 30 minute job, and then it feels like “home”.
What are doing with a dotfile that needs to install a package?
I imagine that things like provisioning are essential to people that switch computers often. So it's not a dotfile-specific problem, but more of a dotfile-adjacent problem.
There's so many interesting edge-cases that affect UX even when distro-hopping between Debian-based distros... especially if you used it for several years and had plenty of custom scripts in your ~/.local/bin folder.
I may yet need to learn or (re)discover some best practices of how to get up to a working development environment faster. I'm thinking of using Guix for that... but I digress.
So far, my workflow goes like this (on a newly-installed distro):
1. Configure environment variables that affect package-specific file locations (/etc/security/pam_env.conf and a custom /etc/profile.d/xdg_std_home.sh script that creates and assigns correct permissions for required directories).
2. Provision packages
3. Deploy config files (using stow).
What I've yet to figure out (haven't really researched it yet), how do you handle app-specific configs (think Firefox add-ons, add-on configs, Thunderbird accounts, etc.)?
"Switch computers often" can also apply to "switch computers with little notice". Even if 95% of my time is spent on one computer, it's nice to know my config is safely squirreled away and, uh, trivially unsquirrelable if something terrible happens to this hardware and I have to get another computer. Seems like a relatively low probability event, but my child has already destroyed two ThinkPads (both were very old and very disposable--still an accomplishment).
As to your last question, nix+home manager gets you there, but that's a whole other Thing.
(n)vim for example: my dotfiles don't vendor the handful of plugins i use, they just include the directives to install those with plugin manager.
I generally use a makefile + stow to handle my dotfiles and home-dir setup. Each program has an entry in this Makefile - most of them are very simple, I keep a list of programs who's dots need to be in ~, and another for ~/.config/ and using make's variable expansion they just get a stow target.
For things like the above example (nvim):
This also allows me to not just copy preference, but provision a bunch of stuff that's invariant across machines (e.g. what i have installed via rustup, go install, etc).I just use one-liner git command that I put into a script
+1 I've been using this for almost ten years. It's straightforward, easy to add files, and there are no symlinks to deal with.
When people ask me which tool I use to manage my dotfiles, I tell them about this little-known tool called `git`. ;)
Side note: I use an shell alias instead, but it's pretty much the same.
`alias home="git --work-tree=$HOME --git-dir=$HOME/.files.git"`
OP's first sentence refers to limitations of "version control $HOME" ... What are the limitations?
Ah, that is what OP was referring to.
Well, you can't have different configs for different hosts. Other than that, I can't quickly recall what other limitations are, I see none. I really like the simplicity of the "pure git" approach.
My dotfiles repo dates back to 2018, I'm happy user of this git one-liner for the past 6 years.
> you can't have different configs for different hosts
You can? Git is a DVCS, that's kinda the point of the Distributed part (although most people use it as a centralized VCS.
I have not only host specific configs, but also (host specific) work configs. It's really easy to merge/cherry pick changes between them.
> Well, you can't have different configs for different hosts
Though you can use git branches, but I prefer not to
Might be niche but for me - I have config files outside $HOME, I use a number of `.gitignore`-aware tools (tree, fuzzy finder), and I just don't like `git status` telling me I'm in a repo in any subdir of $HOME.
That's why I use --git-dir=$HOME/.dotfiles
I rename .git directory to .dotfiles in my $HOME
or use home-manager via nix!
Glad someone else already posted it. I understand if Nix is too much for folks, but it repeatedly is absolutely end-game for stuff like this. I have centralized, unified dotfiles, with all of the power of Nix to have one-off config flexed in. No extra templating or hacky interpolation.
I use git and and a hub site:
- Create repo in ~ with a few dot files, push
- Make a new system setup script, run on other machine:
Later on, if I want to write to the repo on this machine I run ssh-keygen, copy the public key to the remote and remove the token from .git/config and use ssh access instead.I used to have everything excluded in .gitignore and force add files to the repo, but prefer status.showUntrackedFiles instead. There are still a few edge cases but they don't bother you everyday like having to force every operation.
Some of my scripts have things like, if dist fedora, do this, else debian, do this, else Mac, do that, when they differ.
I like the storing of the ln command in the file itself. I think I might implement something like that myself with my own dotfiles. I've been keeping the ln commands in a readme but I never really liked having to update that readme each time I add or remove a file to my dotfiles repo (and using a dotfile manager like those mentioned in other comments here does not appeal to me at all). This is a nice solution I hadn't considered. Now I can just replace the list of ln commands in my readme with a one-liner to run that'll show the ln commands from the files. (I don't actually want to automate the actual running of the commands, though.)
>fta: first-line comments of the form <comment delimiter> ln <link name> in files which needed to be symlinked.
if you're talking about symlinks, you ought to say "ln -s" shouldn't you?
(also, I think this is the tip of another iceberg: learn to say "link" when you mean "hardlink" because that's what a link is in unix filesystem. not saying "stamp out hardlink", I'm saying if you feel comfortable using it correctly yourself, it will help you help other people to disambiguate what they say and not get progressively sloppier. you may not like what these words mean, but that ship is well at sea)
I disagree with the latter. You're correct, but I think it overlooks human factors. If I can say or write one more syllable and remove all shadow of a doubt that I do, in fact, mean a hard link, then that's a syllable well spent. Until we can get 100% of people to immediately think of the hard link concept when someone says "link" in a Unix context, not spelling it out leaves too much room for misinterpretation.
you misunderstand.
you are overlooking the human factor I pointed out: you have no idea if the other person has removed the ambiguity from what they said. My point was to be comfortable with the correct language yourself so you'll more easily spot slip-ups, not to mention be able to read docs. I specifically said not trying to stamp out hardlink.
>Until we can get 100% of people to immediately think of the hard link concept
that's not the goal (being impossible). my goal is to get OP to be cleaner in his doc, and for anybody who cares to be cleaner in their usage.
Not using "link" to mean "symlink" is certainly a reasonable improvement. But using "link" to mean "hardlink" (outside of a context like "the link system call") creates the possibility of ambiguity.
You're suggesting "being comfortable with the correct language", but if it is not already the case that almost everyone means "hardlink" when they say "link", then it's not "the correct language". Your mental mappings should reflect actual language as actually used, so that you can understand and be understood. Those mappings should include knowledge of the ambiguity.
"hardlink" unambiguously means "hardlink" (modulo rare mistakes)
"symlink" unambiguously means "symlink" (modulo rare mistakes)
"link" means "probably hardlink, but possibly symlink being referred to sloppily, or possibly referring to the category that includes both; generally ambiguous without further contextual information".
the position you are advocating leads precisely to TFA which has it precisely wrong, that's why I went to the trouble to write a comment that makes people think. I don't need to hear a detailed counter argument for the status quo ante, just write "we'll just thoughtlessly do it the same shitty way you're arguing against"
No, as mentioned I am not suggesting using "link" to mean "symlink". I am suggesting being aware that not everyone (speaker or listener) will have the word "link" mapped unambiguously to "hardlink", and that if you hear "hardlink" it's likely the speaker means "hardlink", while if you hear "link" you can't be quite as sure.
In other words: I am advocating against deleting error-correcting bits from language, because language is a noisy channel and error-correcting bits help reduce errors.
thank you for describe the shitty status quo way again, you've finally convinced me my ideas are pointless
Should we also say unlink instead of delete?
unlinking is what you are doing: deletion only takes place when you unlink the last link, so yes, great idea, that's why the syscall is called unlink() https://www.man7.org/linux/man-pages/man2/unlink.2.html and the command is not called delete, but rm=remove
What are the advantages of this over stow(1)?
I was tempted to the the same thing as OP, but decided to go with stow, because its concept is more sound than whatever I may concoct. Currently, my dotfiles are packages (folder) which targets $HOME. I clone the repo to $HOME/dotfiles, then I can use `stow <package>` inside the folder to install each package easily. I treat it like dpkg|rpm|... but for my configuration. I'm trying to make each package its own thing, like mail, sway, i3, etc,...
> but decided to go with stow, because its concept is more sound
Could you elaborate on what makes stow's concept more sound than chezmoi?
I don't have any experience with chezmoi. I've managed my dotfiles manually and with the bare git repo trick, but I've looked at chezmoi's docs and it looks like ansible, but for your $HOME. I only have a few computers I want my dotfiles on and stow has been perfect for having my dotfiles in a single folder and linking them where I want to be. Only a few of these configs I'd want somewhere else and I'd either create a new repo (work computers) or copy the files (random place)
I keep mine in a github repository and wrote a little script to sync files in both directions:
https://github.com/llimllib/personal_code/blob/6e86441d992ca...
I've had very little issue making a ~/dotfiles directory and just manually adding to my `~/.zshrc`:
This is how I manage different environments. There are work/home/remote specific things in each domain's `~/.zshrc` files!Clever, but doesn't work for directories.
I just have a small shell script which symlinks everything into my home dir - also serves as a reference as to what actually gets put where.
well, you can do even better by taking all this parsing and complexity out of the comment sections.
just add a makefile, add your mappings. done!
example:
you can even add something like and just have 1:1 mapping...I've tried something similar, and finally I've choose to avoid dotfiles, tangling them from org-mode notes. I keep notes, not their tangled output.
Curious. What config do you mess with aside from .bashrc and .profile (or i3 config if applicable)? Do people sync their.ssh?
This is just a custom dotfile manager implemented in shell/awk. Nothing inherently "better" about it, AFAICT.
Big +1 to forgetting about dotfile managers, and scripting it out.
I just keep my dotfiles repo in the same tree structure as the home directory, and loop over the tree to create symlinks. Plus some miscellaneous commands to set some other things up.
https://github.com/cbarrick/dotfiles
> I just keep my dotfiles repo in the same tree structure as the home directory, and loop over the tree to create symlinks.
Depending on the use case, the `/etc/skel` directory (and equivalents depending on the distro/OS) might be useful. When creating a user, the files in $HOME are copied from such "skeleton" directory, and there's usually a way to tell that command to use a different skeleton directory.
So a different way (not better, just different) would be to have a directory already setup with symlinks and all, and use that directory as the skeleton when creating the user, so its $HOME gets created all ready with symlinks and all.
I think I'm doing my dotfiles wrong.
In my dotfiles I have a couple folders which includes rcfiles and configs. rcfiles includes things like {bash,zsh}rc, .vim{,rc}, tmux.conf, and so on, as well as folders like zsh that include things I import like aliases I have for specific linux machines (e.g. ubuntu has batcat instead of bat...) or osx. Then in config I have folders that contains all the things I would have under ~/.config (starship.toml, ipython_config.py, wezterm/<only lunatics have a single config if you have more than 50 total lines>, and so on. Then I just
I mean I have other folders too like scripts, skels, templates, systemd configs, notes (notes in dotfiles is underappreciated!), and so on. What are you all using these managers for? Are they replacements for bash scripting? And also, find is super powerful and I think under appreciated. It really is worth learning. If you jump into the deepend I think you can get good at it in an afternoon.> I think I'm doing my dotfiles wrong.
The problem isn't you. It's the way Unix-inspired systems do things that are entirely wrong. It's an insane mess of all of the things you mentioned because all the programs could do whatever they wanted. Bad conventions emerged and were propagated throughout the ecosystem.
Here's a fun exercise: try to change your home directory's physical location on a *nix system and see if anything works afterwards. It won't because every config thinks your files are in /home/me while you've changed your user name/home directory to /home/new-me. Windows (eventually) actually got this (approximately) right with a 'virtual directory' for 'my home directory' regardless of where exactly it is on disk. Programs refer to that virtual location.
The current status quo on any *nix is absurd.
(Just FTR, I only use Linux systems, personally. My criticism is borne from a place of aspiration/hope.)
I've done this and it almost worked, ha ha, except for all the paths that don't use $HOME
Yeah I've never really had a problem either. Even with using skels.
Besides, install scripts are the norm for me, so it's typically easy to fix when things do go wrong.
Pro tip: "${HOME%/}/" will always result in /home/godelski even if I include my last name and even if I accidentally modified it to have a / at the end. Some simply variable substitutions go a long way.
Why do it this way instead of just using GNU stow?
[flagged]
Ha forgive me, yes, British schooling, I put at least a full minute's deliberation into my first word for this very reason! Evidently chose wrong.
I'm a Texan and I use "whilst". So I'm confused.
Then again, I can't stand using the word "learnt" instead of "learned". It just sounds wrong. But it doesn't make me stop reading someone's blog.
[flagged]