Distributed locking using git tags

Hacky way to get distributed locking. I needed parallel runners to claim tasks without two of them starting work on the same one. The task store has no locking, and I didn’t want to host an extra service like Redis. My runners already talk to GitHub, and I can get this locking behavior using an empty repo (creatively called “locks”).

The hack uses Git’s built-in atomic operation: creating a ref. If the name exists, the push fails. So a tag named lock-<task-id> is the lock, and whoever creates it holds it.

A small detail is that if two runners push the same commit hash to the same tag, git sees nothing to change and exits 0, so both runners think they won. Each claim therefore makes its own commit, with the runner name and time in the message (which results in a unique commit hash):

claim()   { git push -q origin "$(git commit-tree "$(git mktree </dev/null)" -m "$2 $(date +%s)")":refs/tags/lock-$1; }
release() { git push -q origin :refs/tags/lock-$1; }

# e.g.
# claim task_123 runner-1
# release task_123

This creates an empty tree, and a unique commit on it (due to a unique commit message).

Setup:

The commands mentioned above need to run in a throwaway local repo with one remote set. No need to clone or fetch. That repo just needs to track a central “origin” repo, e.g. “github.com/foo/locks.git”

Commands to set up the throwaway local repo:

git init
git remote add origin "git@github.com:foo/locks.git"

To free a stale lock, delete the tag in the host’s UI, or via git tag -d <tag> (on the remote repo, not local), or run the release command.

Nothing here is GitHub-specific. Any git host works, including GitLab or a repo in another folder on your PC, as long as every runner can push to the designated “origin” repo.

This suits a few tasks a day, not heavy contention. It’s pretty slow, but works just fine for my needs.