Mostly Textfile Directories: How to Sync?

1

I am asking for advice about a problem that is likely familiar to many people on this forum. First, I have four large-ish directories of mostly plain text (well, emacs org-mode) files, each of which constitutes a separate “notebook” with a different focus: personal, gaming, computer, and work. Second, I have too many computers that are sporadically and haphazardly re-imaged with Debian variants, and I want to be able to edit these files with emacs and keep them in sync without too much suffering.

My current solution is four bare git repositories behind SSH at an IP address and distinct physical servers in my home. I configure each computer with the appropriate remotes and ssh keys. It is critical to partition these directories, so that, e.g., my work computer cannot read the personal (and gaming) notebooks at all.

This setup actually has all the features I want — simple, self-hosted, free software, partitioned — but is fairly high-suffering in terms of setting up a repository on each new computer and making many “noisy” meaningless commits just to sync. Questions follow:

  • I am looking at the unison program with some envy. Does anyone have experience with it? Seems like I could continue to use SSH as transport and it is better tailored to this use-case than git.
  • Is there a better (more automated?) way to abuse git as a content-addressable sync backend than using or scripting the porcelain?
  • Am I missing some amazing sync daemon? I have had poor experiences with syncthing in the past and prefer manually resolving conflicts and hosting my own tools. But maybe it is great today! So, recommendations and thoughts about solving this problem with tools besides git and unison are quite welcome.

Also I have pretty much given up running operating systems that aren’t Debian or at least Debian-based and am (slowly) moving my entire collection of personal computers to MNT hardware, so maybe that helps narrow recommendations down somewhat.

1 Like
2

What you describe sounds like what git annex provides out of the box, no?

3

Oh, very interesting. It seems I should seriously consider using the git annex sync command as the “automatic” centerpiece of this workflow. Thank you!

4

I had exactly this issue (to sync work files, mostly text/docments) between my different Reforms and I use unison paired with a personal server that I use as the centralized repository and that has backup tasks. I have a crontab with:

* * * * * flock -n /tmp/unison.lock unison -batch minute

In ~/.unison/minute.prf I have:

root = /home/minute/Documents
root = ssh://my-server-out-there//home/minute/Documents

ignore = Name .*
ignore = Name *~
ignore = Name *.tmp
ignore = Name *.bak

It mostly works fine, just rarely I had to clean up some local unison “archive” files that might have been corrupted (written with 0 size) during a power loss, not sure how to automate that yet.

5

What you describe is exactly my setup as well. Six repositories (for work, private and passwordmanager) holding mostly emacs org files, git synced among multiple laptops and workstations. I switch between these systems daily and it is so convenient to have my passwords and documents follow me.

I run the sync every 30 minutes with a systemd user timer. The flow of the script:

    test -d <repository> || git clone <shareuser>@<server> <repository> --branch main
    git pull -X theirs
    git add .
    git commit --no-gpg-sign -am "$(git status -s)"
    git push origin main

I’ll follow this thread closely as I’m looking for something similar (but not with git) for my family.