147 lines
7.1 KiB
Markdown
147 lines
7.1 KiB
Markdown
# AGENTS.md
|
|
|
|
Notes for anyone (human or agent) changing this repository.
|
|
User-facing docs live in [README.md](README.md); this file is about the code.
|
|
|
|
## What this is
|
|
|
|
A daemon that mirrors git repositories from `src` to `dst` on a timer, driven by a
|
|
hot-reloadable TOML file. Go 1.24, one third-party dependency
|
|
(`github.com/BurntSushi/toml`), ~1500 lines of non-test code.
|
|
|
|
Design goals, in priority order: **do not corrupt or delete anything on dst**,
|
|
stay small in memory, stay simple. When a change trades any of these for
|
|
convenience, it is probably the wrong change.
|
|
|
|
## Commands
|
|
|
|
```bash
|
|
go build ./...
|
|
go test ./... # includes end-to-end tests that shell out to real git
|
|
go test -race -count=1 ./... # what CI runs
|
|
go vet ./...
|
|
gofmt -l . # must print nothing; CI fails otherwise
|
|
go run . -config config.example.toml -check # needs SRC_TOKEN set to anything
|
|
```
|
|
|
|
The tests need a `git` binary on `PATH`. They create real repositories under
|
|
`t.TempDir()` and never touch the network.
|
|
|
|
## Layout
|
|
|
|
```
|
|
main.go flags, logger, signal handling, config file watcher
|
|
signal_unix.go/_other.go build-tagged SIGHUP wiring (no-op on non-unix)
|
|
internal/config/ TOML parsing, validation, defaulting → []Job
|
|
internal/gitx/ hermetic wrapper around the git CLI
|
|
proc_unix.go/_other.go build-tagged process-group creation and kill
|
|
internal/syncer/ one sync cycle for one repository
|
|
internal/manager/ per-repo goroutines, reload diffing, backoff
|
|
http.go /healthz /readyz /status /metrics
|
|
```
|
|
|
|
Dependency direction is strictly `main → manager → syncer → gitx → config`.
|
|
Nothing under `internal/` imports `manager`.
|
|
|
|
## Invariants
|
|
|
|
These are load-bearing. Breaking one is a bug even if the tests still pass.
|
|
|
|
1. **`config.Job` is fully resolved.** All defaulting, env expansion and
|
|
validation happen in `config.resolve`. Downstream code never asks "was this
|
|
field set?" — it reads the value. This is also what makes reload diffing work:
|
|
`Apply` decides whether a repo changed with `reflect.DeepEqual` on `Job`, so a
|
|
`Job` must contain no pointers, maps, funcs, or timestamps, and must be
|
|
deterministic for identical input.
|
|
|
|
2. **Repository URLs never reach disk.** They are passed as command-line
|
|
arguments to `git fetch` / `git push` / `git ls-remote`, never written into
|
|
`.git/config` via `git remote add`. URLs may embed tokens; mirror directories
|
|
may outlive the process.
|
|
|
|
3. **Secrets are scrubbed from every string that escapes.** `gitx.Run` runs both
|
|
its log lines and git's output through `scrub` using `Options.Secrets`, and
|
|
URLs through `RedactURL`. Any new code path that logs or wraps git output must
|
|
do the same. There is a test asserting a token never appears in an error.
|
|
|
|
4. **Every git invocation goes through `gitx.Run`.** It supplies the isolated
|
|
`HOME`, disables system/global gitconfig, drops inherited `GIT_SSH_COMMAND`
|
|
and friends, sets `GIT_TERMINAL_PROMPT=0`, and puts the child in its own
|
|
process group so a timeout kills `ssh` too. Calling `exec.Command("git", ...)`
|
|
directly anywhere else reintroduces all of those problems.
|
|
|
|
5. **The syncer holds no state between cycles.** Every cycle re-derives truth
|
|
from `ls-remote` on both ends. Do not add a state file, a "last synced SHA"
|
|
cache, or an in-memory `map[repo]refs` shortcut — the self-healing behaviour
|
|
(recovering from external force-pushes and partial pushes) comes entirely from
|
|
not trusting anything remembered.
|
|
|
|
6. **An empty source never empties a destination** unless `allow_empty = true`.
|
|
The guard is in `syncer.Sync`; keep it before the push, not inside it.
|
|
|
|
7. **One goroutine per mirror directory, ever.** `Apply` cancels the goroutine
|
|
being replaced and passes its `done` channel to the successor as `waitFor`, so
|
|
the successor blocks until the predecessor has exited. Two goroutines running
|
|
`git` in the same bare repo will corrupt it.
|
|
|
|
8. **`Apply` must not block.** It is called from the config watcher; a repo may
|
|
be 25 minutes into a 30-minute sync. Hence the `waitFor` handoff above rather
|
|
than waiting inline.
|
|
|
|
## Gotchas discovered the hard way
|
|
|
|
- **`limitedWriter.Write` must return `len(p)`**, not the number of bytes it
|
|
chose to keep. `os/exec` treats a short write as an error and would abort an
|
|
otherwise healthy git run once output exceeded the cap.
|
|
|
|
- **`signal.Notify` with an empty slice subscribes to every signal.** The
|
|
`reloadSignals()` call site is guarded with a length check for the non-unix
|
|
build where the slice is empty.
|
|
|
|
- **Git refuses to delete the branch that HEAD points at** (`receive.denyDeleteCurrent`).
|
|
This shows up when pruning a renamed default branch; GitHub behaves the same
|
|
way. It is server-side behaviour, not something to work around in the client.
|
|
The syncer test fixture sets `receive.denyDeleteCurrent ignore` on its bare dst
|
|
for exactly this reason. Documented in the README's troubleshooting section.
|
|
|
|
- **`url.UserPassword` percent-encodes.** `RedactURL` uses the literal string
|
|
`redacted` as the placeholder; `***` came back as `%2A%2A%2A`.
|
|
|
|
- **Deploy keys mounted read-only are usually 0644** and ssh rejects them.
|
|
`gitx.usableKey` stages a 0600 copy in the per-sync temp dir. Do not "fix" this
|
|
by chmod'ing the original — the operator often cannot make it writable.
|
|
|
|
- **`IdentitiesOnly=yes` is required.** Without it ssh offers every key the agent
|
|
knows about and GitHub closes the connection with "too many authentication
|
|
failures" before reaching the right one.
|
|
|
|
- **`expandEnv` deliberately only understands `${VAR}`.** Bare `$VAR` is left
|
|
alone because `$` is common in passwords, and an undefined variable is a hard
|
|
error rather than an empty expansion, which would otherwise produce a URL that
|
|
fails in a confusing way much later.
|
|
|
|
## Testing conventions
|
|
|
|
`internal/syncer/syncer_test.go` has a harness that builds real source and bare
|
|
destination repositories and drives `Sync` against them. New sync behaviour
|
|
belongs there rather than in a mock — the interesting bugs in this program are
|
|
all in how git actually behaves. Existing cases cover first sync, no-op cycles,
|
|
new commits, prune, force-push after a rewrite, repairing drift on dst, the
|
|
empty-source refusal, ref filtering, unreachable sources, and timeouts.
|
|
|
|
Manager tests assert reload semantics: unchanged repos keep running, changed
|
|
repos restart, removed repos stop. They run under `-race`; the `wg` in `Manager`
|
|
exists because a goroutine replaced by a reload could otherwise outlive `Stop`
|
|
and race `t.TempDir()` cleanup.
|
|
|
|
## When adding a config field
|
|
|
|
1. Add it to `Repo` (and `Global` if it should be inheritable) in `config.go`.
|
|
2. Resolve it in `resolveJob` — with an explicit default, never a zero value that
|
|
downstream code has to interpret.
|
|
3. Add it to `Job`, keeping the type comparable (see invariant 1).
|
|
4. Thread it through `syncer.Sync` / `gitx`.
|
|
5. Document it in the README table and, if it is interesting, `config.example.toml`.
|
|
6. Add a case to `config_test.go`. `unknownKeys` rejects unrecognised keys, so a
|
|
field that is parsed but not registered will make previously valid configs fail.
|