Syncthing service on Milner

tepichord commented

2026-09-30 09:39:03 -07:00

Owner

Closes #21

The agent's ~/.pi becomes one folder in three places: dima, fedora, and
an always-on pod. The pod is Flux-owned and holds one credential — its own
device identity, which comes from Vault like every other in-cluster credential
(AGENTS.md, Vault-first).

What is in the change

Piece	File
Deployment (Recreate, `node-role: worker`), the seed init container that copies the identity and the ignore policy out of read-only mounts	`gitops/clusters/milner/apps/annia/syncthing.yaml`
`pi-data` — 10Gi, `persistent`, ReadWriteOnce (also milner#25's claim: its pod mounts it at `/root/.pi`)	`syncthing-pvc.yaml`
GUI (ClusterIP 8384) and sync (tailnet LoadBalancer `svc:syncthing`, 22000)	`syncthing-services.yaml`
The ignore policy — the three lines, hashed into the ConfigMap name so editing it rolls the pod	`stignore`
`VaultStaticSecret syncthing-config` → `Secret annia/syncthing-config`, `rolloutRestartTargets` so a rotation is a rollout (Syncthing reads its config once)	`secrets/annia-secrets.yaml`
The producer: one `vault kv put` of the three files, plus the printed root-only `annia-read` half	`apps/annia/set-syncthing-credential.sh`
Story 24, the §19 runbook, the OPERATIONS.md row, the fourth `svc:` approval in docs/tailscale.md	`USER_STORIES.md`, `docs/syncthing.md`, `OPERATIONS.md`, `docs/tailscale.md`

The fix commit in this PR — 9315a23

--no-console is gone. It does not exist in the pinned 2.1.5 (syncthing: error: unknown flag --no-console, measured on the image), and serve refuses
an unknown flag by exiting — so it was a CrashLoopBackOff on the pod the whole
apps layer waits on (wait: true), not a warning.
re.S → re.DOTALL in the two new regexes (ruff FURB167), which is what
pipeline 143's test step failed on.
The stignore assertion now requires the hashed name kustomize generates
instead of a startswith, which passed for the unrewritten one.

State

CI: pipeline 145 (this head) — all 13 steps green, including L2 dry-run,
the suite and L3 preview.
Vault: secret/syncthing/config holds config.xml, cert.pem, key.pem
(written before the merge on purpose — apps.yaml is wait: true, so a pod
whose Secret does not exist yet reddens the layer).
Still outstanding, root-only, before the merge: the annia-read policy
needs the exact path secret/data/syncthing/config —
set-syncthing-credential.sh prints the block. Measured from a pod using the
SA the role is bound to: login succeeds and secret/data/annia/rabbitmq reads
200, while secret/data/syncthing/config reads 403, so this is the one
step between the merge and a synced Secret.

After the merge (docs/syncthing.md §19d): PVC Bound → pod Ready → approve
svc:syncthing in the Tailscale console (once) → add
tcp://syncthing.faun-snapper.ts.net:22000 to the pod's device entry on dima
and fedora and approve the pod's device once → set-syncthing-credential.sh verify.

Closes #21 The agent's `~/.pi` becomes **one folder in three places**: `dima`, `fedora`, and an always-on pod. The pod is Flux-owned and holds one credential — its own device identity, which comes from Vault like every other in-cluster credential (AGENTS.md, Vault-first). **What is in the change** | Piece | File | |---|---| | Deployment (Recreate, `node-role: worker`), the seed init container that copies the identity and the ignore policy out of read-only mounts | `gitops/clusters/milner/apps/annia/syncthing.yaml` | | `pi-data` — 10Gi, `persistent`, ReadWriteOnce (also milner#25's claim: its pod mounts it at `/root/.pi`) | `syncthing-pvc.yaml` | | GUI (ClusterIP 8384) and sync (tailnet LoadBalancer `svc:syncthing`, 22000) | `syncthing-services.yaml` | | The ignore policy — the three lines, hashed into the ConfigMap name so editing it rolls the pod | `stignore` | | `VaultStaticSecret syncthing-config` → `Secret annia/syncthing-config`, `rolloutRestartTargets` so a rotation is a rollout (Syncthing reads its config once) | `secrets/annia-secrets.yaml` | | The producer: one `vault kv put` of the three files, plus the printed root-only `annia-read` half | `apps/annia/set-syncthing-credential.sh` | | Story 24, the §19 runbook, the OPERATIONS.md row, the fourth `svc:` approval in docs/tailscale.md | `USER_STORIES.md`, `docs/syncthing.md`, `OPERATIONS.md`, `docs/tailscale.md` | **The fix commit in this PR — `9315a23`** * `--no-console` is gone. It does not exist in the pinned 2.1.5 (`syncthing: error: unknown flag --no-console`, measured on the image), and `serve` refuses an unknown flag by exiting — so it was a CrashLoopBackOff on the pod the whole `apps` layer waits on (`wait: true`), not a warning. * `re.S` → `re.DOTALL` in the two new regexes (ruff FURB167), which is what pipeline 143's `test` step failed on. * The stignore assertion now requires the *hashed* name kustomize generates instead of a `startswith`, which passed for the unrewritten one. **State** * CI: pipeline **145** (this head) — all 13 steps green, including L2 dry-run, the suite and L3 preview. * Vault: `secret/syncthing/config` holds `config.xml`, `cert.pem`, `key.pem` (written before the merge on purpose — `apps.yaml` is `wait: true`, so a pod whose Secret does not exist yet reddens the layer). * **Still outstanding, root-only, before the merge:** the `annia-read` policy needs the exact path `secret/data/syncthing/config` — `set-syncthing-credential.sh` prints the block. Measured from a pod using the SA the role is bound to: login succeeds and `secret/data/annia/rabbitmq` reads 200, while `secret/data/syncthing/config` reads **403**, so this is the one step between the merge and a synced Secret. **After the merge** (docs/syncthing.md §19d): PVC Bound → pod Ready → approve `svc:syncthing` in the Tailscale console (once) → add `tcp://syncthing.faun-snapper.ts.net:22000` to the pod's device entry on `dima` and `fedora` and approve the pod's device once → `set-syncthing-credential.sh verify`.

tepichord added 9 commits

2026-09-30 09:39:03 -07:00

feat(annia): syncthing service — the agent's folder, Vault-seeded and single-writer 5ebefbad04

An always-on Syncthing pod (pinned syncthing/syncthing:2.1.5) in annia on
a ReadWriteOnce `persistent` claim holding the operator's ~/.pi: the seed
initContainer copies the Vault-synced identity (config.xml, cert.pem,
key.pem) into an emptyDir on every start so the device ID survives a
restart, the three-line ignore policy lands at /pi/.stignore before the
first sync, a ClusterIP serves the in-cluster GUI only, and a Tailscale
LoadBalancer carries 22000/TCP for dima and fedora.

Identity is Vault-first: VaultStaticSecret syncthing-config reads
secret/syncthing/config, set-syncthing-credential.sh writes it (values on
stdin, verdicts in verify) and prints the annia-read half that gains the
one exact path in docs/vault.md §4c-ter — the VSS, the policy, the
producer and both count tripwires (14→15, (9,14)→(9,15)) land together so
no intermediate commit has a path with no producer. The runbook
(docs/syncthing.md §19) and test_syncthing_is_recreate_rwo_and_vault_seeded
come with the manifests they describe.

Refs: #21

docs(ops): index the syncthing credential and workload 3b798350be

A §1 row beside IRC's — Vault-first producer, Flux-owned app/claim/Services,
one admin-console approval for the tailnet name — plus the domain index row
so every docs/ runbook stays reachable from OPERATIONS.md.

Refs: #21

docs(tailscale): auto-approve svc:syncthing once a21af8a259

The autoApprovers snippet gains "svc:syncthing": ["tag:k8s"] with the
note for syncthing.faun-snapper.ts.net: one admin-console approval (§2.4)
when the apps layer reconciles, same as svc:irc before it.

Refs: #21

docs(stories): story 24 — the agent's folder is one folder d9a7e86f25

Appended after 23 so the numbering stays one ordered sequence
(test_user_stories_are_the_spec_and_only_stories); the plan's exact text.

Refs: #21

docs(gitops): trim repeated seed and image-pin rationale in syncthing.yaml e0445975e4

Refs: #21

fix(annia): let verify report a missing syncthing field f72d060377

A bare `val="$(vault_field …)"` under `set -e` exits the script the
moment Vault has no such field — the FAIL branch for "has no <field>"
was unreachable, the cluster checks never ran, and the run died with
exit 2 ("unusable environment") after printing one ok line.
`|| true` keeps the verdict with the summary, the way set-credential.sh
chains the same read into an `if` condition.

Refs: #21

fix(annia): verify says when the syncthing pod is not ready eabb342e84

Every check verify makes above holds while the Deployment crashloops —
a config.xml the daemon refuses to load leaves Vault, the Secret and the
VaultStaticSecret all green, and the run ended with "every check held"
over zero ready replicas. Same check set-credential.sh's verify makes
for ergo, same last-line verdict (docs/syncthing.md §19e).

Refs: #21

fix(annia): reject unknown options in set-syncthing-credential.sh d1e7b5cc4b

The argument loop took any unmatched word as the directory, so a
sibling's `--rotate` (set-credential.sh's flag) became DIR and failed
as "…/--rotate/config.xml is missing" — a missing file, not the
unknown option it is. Options now say so, as both set-credential.sh
scripts do.

Refs: #21

fix(annia): carry the syncthing path in every writer of annia-read

ci/woodpecker/pr/gates Pipeline failed

Details

4c48e4782a

`policy write` replaces the policy, and annia-read now has two writers:
docs/vault.md §4c-ter and set-syncthing-credential.sh both gained the
exact `secret/data/syncthing/config` path, but apps/annia/set-credential.sh
— the script an operator re-runs on a broker rotation — still wrote a
block with only `secret/data/annia/*`. Its next root run would have
dropped the Syncthing pod's read with no error at the write site, and
the VaultStaticSecret reading it fails at the next 1h refresh.

The tripwire test compares both scripts' POLICY_HCL and the docs block
against each other, so the three copies cannot drift apart again.

Refs: #21

tepichord added the

pr

reviewing

label

2026-09-30 09:39:03 -07:00

tepichord added this to the MVP project

2026-09-30 15:32:32 -07:00

tepichord added 1 commit

2026-09-30 15:47:29 -07:00

fix(annia): drop --no-console, which the pinned syncthing does not have

ci/woodpecker/pr/gates Pipeline was successful

Details

ci/woodpecker/pull_request_closed/cleanup Pipeline was successful

Details

9315a2343f

`serve` is a kong CLI and refuses a flag it does not know; measured on the
pinned image, `syncthing serve --no-console -H /tmp/h` prints
`syncthing: error: unknown flag --no-console` and exits. 1.x had the flag,
2.1.5 has no console toggle at all (`serve --help` lists `--no-browser`,
`--no-port-probing`, no console-related flag but `--log-file`), so the pod
would have CrashLooped — and it is the readiness gate the whole `apps` layer
waits on (`wait: true`). The comment now says why the flag is absent, and
`--no-browser`/`--no-port-probing` are kept as the two that exist.

Also in the same test's commit:

* the two `re.search(..., re.S)` calls are `re.DOTALL`: ruff's FURB167 fails
  `test_python_lint` on the alias, which is what turned pipeline 143's `test`
  step red (`324 passed, 1 failed` locally, the same one).
* the stignore volume assertion now requires the *hashed* name kustomize
  generates (`stignore-<hash>`) rather than any name starting with `stignore`
  — the old form passed for the unrewritten name too, which is the failure
  its own message describes.

Verified against the image and the cluster, not the docs: the tag exists and
runs (v2.1.5, go1.27.1, linux-arm64, `[noupgrade]`), the entrypoint is
`/bin/entrypoint.sh /bin/syncthing` and `STGUIADDRESS=0.0.0.0:8384` with
`STHOMEDIR=/var/syncthing/config` are the image's own env (which is why the
emptyDir at `/var/syncthing` and the seed's copy into
`/var/syncthing/config` are the right paths). Suite: 325 passed.

tepichord merged commit 2c213b2021 into main

2026-09-30 18:01:33 -07:00

tepichord deleted branch feat/21-syncthing-service-milner

2026-09-30 18:01:33 -07:00

tepichord referenced this pull request from a commit

2026-09-30 18:01:35 -07:00

Merge pull request 'Syncthing service on Milner' (#113) from feat/21-syncthing-service-milner into main

tepichord removed the

pr

reviewing

label

2026-10-01 18:48:58 -07:00

Rows
Columns

Syncthing service on Milner #113