# Operating simplepages `pages-server` is a static-site host built around one guarantee: **a request either sees the whole old deployment or the whole new one, never a mixture.** Everything below either explains how to run it or tells you where that guarantee stops. Read [Security model and its limits](#security-model-and-its-limits) before you put this on a network. Three of the items there are not bugs to be fixed later — they are properties of the design, and one of them (same-origin) decides whether this server is suitable for your workload at all. --- ## 1. What it does ``` CI job pages-server visitor ------ ------------ ------- pages deploy ./dist ──► upload only the changed blobs assemble a directory tree ┌─────────────────────────┐ │ ONE atomic pointer swap │ ◄── GET /~demo/ └─────────────────────────┘ repoint $WEBROOT/~demo ``` Unlike `rsync`, no file of the new version is visible until every file of it is present and verified. The switch itself is a single `atomic.Pointer` store, so a request that started before the swap keeps reading the old snapshot to completion — including a large file already halfway down the wire. Rollback is the same operation pointed at an older deployment id, and costs the same single store. --- ## 2. Installation Build both binaries. The server needs SQLite (pure Go, so no cgo and no C toolchain); the CLI deliberately shares no server code at all. ```sh CGO_ENABLED=0 go build -o /usr/local/bin/pages-server ./cmd/pages-server CGO_ENABLED=0 go build -o /usr/local/bin/pages ./cmd/pages ``` `pages` is what goes into CI runners. Its dependency graph is enforced by a test (`go test ./cmd/pages -run TestCLIImportGraph`), so it can never accidentally start dragging in the database driver. ### Directories | Path | Contents | Who writes it | |---|---|---| | `$DATA_DIR/pages.db` | projects, deployments, keys, blob refcounts | server only | | `$DATA_DIR/cas/` | content-addressed blobs, `0444` | server only | | `$DATA_DIR/deployments///` | assembled trees | server only | | `$WEBROOT/~` | symlink to the active deployment | server only | `$WEBROOT` and `$DATA_DIR` must not overlap; the server refuses to start if they do. ### systemd ```ini [Unit] Description=simplepages static site server After=network-online.target [Service] Type=exec ExecStartPre=/usr/local/bin/pages-server --config /etc/pages-server/config.toml --check-config ExecStart=/usr/local/bin/pages-server --config /etc/pages-server/config.toml Restart=on-failure StateDirectory=pages-server ReadWritePaths=/var/lib/pages-server /srv/www ProtectSystem=strict PrivateTmp=yes NoNewPrivileges=yes UMask=0022 [Install] WantedBy=multi-user.target ``` `--check-config` validates and exits without binding a port, which is why it is safe in `ExecStartPre`. **Do not use `DynamicUser=yes`.** The UID it allocates rotates, and the assembled trees are hardlinks whose ownership must stay stable across restarts. --- ## 3. Configuration Precedence is **flag > `PAGES_*` environment variable > config file > default**. ```toml # /etc/pages-server/config.toml data_dir = "/var/lib/pages-server" webroot = "/srv/www" listen = ":8080" # static content api_listen = "127.0.0.1:8081" # management API site_url = "https://pages.example.com" trusted_proxy_cidrs = ["127.0.0.1/32", "::1/128"] assemble_mode = "auto" # auto | hardlink | copy | none log_level = "info" log_format = "json" gc_interval = "15m" reconcile_interval = "5m" shutdown_grace = "30s" read_header_timeout = "10s" read_timeout = "5m" idle_timeout = "120s" [limits] max_file_bytes = 268435456 # 256 MiB max_manifest_files = 50000 max_concurrent_uploads = 32 max_manifest_bytes = 67108864 # 64 MiB of manifest JSON max_json_bytes = 1048576 # 1 MiB for ordinary API bodies ``` Every key above has a matching `PAGES_*` variable (`PAGES_DATA_DIR`, `PAGES_API_LISTEN`, `PAGES_ASSEMBLE_MODE`, `PAGES_RECONCILE_INTERVAL`, …). The ones worth overriding on the command line also have flags: `--data-dir`, `--webroot`, `--listen`, `--api-listen`, `--site-url`, `--assemble-mode`, `--log-level`, `--log-format`, `--gc-interval`, `--shutdown-grace`, plus `--config`, `--check-config` and `--version`. Run `pages-server -h` for the authoritative list. `listen` and `api_listen` must differ, and `webroot` may be neither equal to nor nested inside `data_dir` (in either direction) — the reconciler prunes symlinks from one and the GC deletes trees from the other, so overlapping them would let each destroy the other's state. ### The two listeners are separate on purpose `listen` serves nothing but site content. `api_listen` serves nothing but the management API, and defaults to loopback. Neither answers for the other — there is a test that asserts exactly this. Expose the API through your reverse proxy on its own hostname, or not at all; nothing forces a management surface onto the origin your sites are served from. TLS is the reverse proxy's job in both cases. ### `site_url` Purely cosmetic: it is the origin API responses quote back to a CI job so the job can print where its build landed. Behind a proxy the server cannot learn this name, so you have to tell it. Leaving it empty just omits the `url` field. ### `assemble_mode` - `auto` (default) — probe for hardlink support at startup, fall back to copy. - `hardlink` — require hardlinks; fail loudly if the filesystem refuses. - `copy` — always copy. Correct everywhere, uses disk proportional to content. - `none` — skip on-disk assembly entirely. Content is served straight from the CAS and **`$WEBROOT` symlinks are not maintained**. Choose this only if nothing outside `pages-server` reads the files. On overlayfs (Docker's default upper layer, among others) cross-layer `link()` either triggers a copy-up or fails outright, so `auto` may silently land on copy. Check the startup log line reporting the detected mode if disk usage surprises you. ### `trusted_proxy_cidrs` `X-Forwarded-For` is honoured **only** when the direct peer address falls inside one of these prefixes. The client IP is used for the failed-authentication rate limiter, so getting this wrong either lets one proxy IP absorb everyone's budget (too narrow) or lets a client forge its own identity (too wide). Set it to your proxy's address and nothing else. --- ## 4. First run On first start, if no unrevoked admin key exists, the server mints one and writes the token to `$DATA_DIR/bootstrap-token` with mode `0600`, logging a warning. **This is the only place the server ever writes a token to disk.** ```fish set -x PAGES_SERVER http://127.0.0.1:8081 set -x PAGES_TOKEN (cat /var/lib/pages-server/bootstrap-token) pages project create demo pages key create --project demo --name github-actions -o json # token shown ONCE ``` Then delete the bootstrap file. It is not recreated as long as one unrevoked admin key exists. Give CI a **project-scoped** key, never the admin one. A project key can create, upload, finalize and activate deployments in its own project and read that project's settings — and nothing else, in any other project. --- ## 5. Deploying ```sh pages deploy ./dist --project demo ``` What happens, and what each step buys you: 1. **Scan.** Walk the directory, hash every file with SHA-256 in parallel. Symlinks are refused by default (`--follow-symlinks` opts in, and still refuses targets outside the tree). Devices, sockets and FIFOs are always refused. 2. **Create.** A `pending` deployment row. No filesystem is touched. 3. **Manifest.** Send every path + digest + size; the server answers with the digests it does *not* already have. This is where the incremental win shows up — a rebuild that changes one file uploads one blob, regardless of how big the site is. The CLI prints the saving. 4. **Upload.** Only the missing blobs, bounded concurrency (`--concurrency`), exponential backoff honouring `Retry-After` (`--retries`). 5. **Finalize.** The server verifies nothing is missing, then assembles the tree into `…/.staging`, fsyncs it, and renames it into place. 6. **Activate** (unless `--activate=false`) — the atomic swap. **Interrupted deploys need no special handling.** Re-run `pages deploy`. The blobs that made it are still in the CAS, so the new deployment's manifest negotiation returns a much smaller missing set. There is no resume protocol because none is needed. ### Rollback ```sh pages deployment list --project demo # newest first; ACTIVE marks what is live pages deployment activate dpl_… # one pointer store ``` The superseded deployment stays `ready` on disk until retention expires it, so rollback costs nothing but the swap. `pages deployment show dpl_… --files` gives the manifest of any of them if you need to see what a version contained before switching to it, and `pages deployment delete dpl_…` removes one — except the active one, which answers `409 deployment_active` until something else is activated. Every `pages deployment` command accepts a project key, so a CI job can roll its own project back without an admin credential. `pages system gc` and `pages system fsck` are server-wide and need an admin key. ### Base paths — the one thing that will bite you Sites are served under `/~PROJECT/`. A build that assumes it lives at `/` will request `/assets/app.js`, get a 404, and look broken. **Set the base path at build time:** | Tool | Setting | |---|---| | Vite | `vite build --base=/~demo/` | | Create React App | `PUBLIC_URL=/~demo` | | Next.js (static export) | `basePath: '/~demo'` | | Hugo | `--baseURL=/~demo/` | | plain HTML | `` | This is inherent to prefix hosting; the server cannot fix it. If you need sites at the root of their own domain, you need host routing (§8). ### Project settings ```sh pages project update demo --spa --not-found-file 404.html --retention 20 ``` - `--index-file` (default `index.html`) — what a directory URL serves. - `--not-found-file` — served *with* a 404 status. Without it, 404s are bare. - `--spa` — serve the index document for unknown paths, **but only when the request's `Accept` header includes `text/html`.** That condition matters: it is why a missing `/assets/app.js` still returns a real 404 instead of an HTML page with status 200 and an afternoon of `Unexpected token '<'`. - `--cache-control` (default `public, max-age=0, must-revalidate`) — correct by construction here, because every response carries a strong content-hash ETag. Repeat visits are cheap 304s and deploys are visible immediately. Only raise `max-age` for content served under hashed filenames. - `--retention` / `--retention-grace` — see §6. --- ## 6. Garbage collection and retention The sweep runs every `gc_interval`, and on demand via `pages system gc` (admin), which is `POST /api/v1/gc`. `pages system gc --dry-run` reports what would go without deleting anything — it cannot report the bytes, because nothing was deleted and the content is all still referenced. Per project, a deployment is deleted when **all** of these hold: - it is not active; - it was superseded more than `retention_grace` ago (default 1 h) — a deployment that was never activated is measured from when it was created, so an uploaded-but-unused one is kept for the grace period too; - it is not among the newest `retention_count` finished deployments (default 10). Failed deployments are dropped after 24 h. Blobs are removed once their refcount reaches zero and they have been unreferenced for an hour. **Why the grace period exists.** A request that has already taken its snapshot and is about to open a blob gets an hour of slack. Combined with POSIX's guarantee that an open file descriptor keeps an inode alive, an in-flight download survives even a forced GC. If the server ever logs a missing blob on the read path, that is a genuine bug — it is logged at ERROR precisely so it cannot be mistaken for normal behaviour. `pages system fsck` (`POST /api/v1/fsck`, admin) recomputes every blob refcount from the manifests and reports drift; `--repair` (`{"repair": true}`) corrects it. On a healthy server it always reports none — the counts are maintained by database triggers — so it is for the cases outside normal operation: a restored backup, or a database edited by hand. A count that reads *low* is the dangerous one, because it is content the collector will delete while a deployment still needs it. --- ## 7. Security model and its limits Three of these are accepted properties of the v1 design, not deferred work. They are stated here so the decision is yours and not a surprise. ### 7.1 ⚠️ Path routing is not a security boundary between projects `/~a/` and `/~b/` are the **same origin**. That means project A's JavaScript can: - `fetch('/~b/secret.json')` and read the response; - read and write cookies and `localStorage` for the whole host; - register a service worker scoped to `/`, intercepting **every request to every project on that host**, including future ones. There is no browser mechanism that prevents this. `X-Content-Type-Options: nosniff` is always sent, and it does not help here. **Only host projects that trust each other.** If you need to host sites from mutually distrusting tenants, give each project its own domain so each gets its own origin. That is host routing, which v1 does not implement. ### 7.2 ⚠️ Assembled files share inodes with CAS blobs When hardlink assembly is in use (the default when supported), a file under `$WEBROOT/~demo/` and its blob in `$DATA_DIR/cas/` are **the same inode**. Editing that file in place corrupts the blob — and therefore corrupts every other project and every other deployment referencing the same content. Mitigations in place: blobs and assembled files are `0444`, directories `0755`, and client-supplied file modes are ignored entirely (there is no `mode` column in the schema, so a setuid bit can never reach disk). **Treat `$WEBROOT` as read-only to everything except `pages-server`.** Do not point `rsync --delete` at it, do not let a deploy script `chmod -R` it, do not edit a file "just to check something". If you must modify content, deploy it. Setting `assemble_mode = "copy"` removes the shared inode at the cost of disk. ### 7.3 ⚠️ The manifest endpoint is a blob existence oracle A project-scoped key can put any digest in a manifest and learn from the `missing` response whether the server already holds that content — including content belonging to another project. The attacker must already know the exact SHA-256, so this only ever confirms "somebody here hosts this file I already have". It cannot be used to read content or to enumerate anything. This is the price of cross-project deduplication, which is the feature that makes repeated CI deploys fast. If it matters to you, the fixes are: partition the CAS per project (losing all dedup), or always report digests the calling project has never referenced as missing (losing cross-project dedup only). ### 7.4 What is structurally prevented - **Read-path traversal.** The only filesystem path built while serving is `cas/ab/cd/<64 hex>`, derived from a 32-byte digest that came out of an in-memory map lookup. No user-controlled byte reaches the filesystem on the read path, so traversal is not blocked — it is impossible. - **Write-path traversal.** Every manifest path must satisfy both `fs.ValidPath` and `filepath.Localize`, and NUL bytes, control characters, segments over 255 bytes, total lengths over 4096, duplicates, and case-insensitively colliding paths are all rejected. - **Poisoned blobs.** The server never trusts a client's digest: it recomputes SHA-256 over the received stream and rejects a mismatch. Without this, a client could claim another project's digest and overwrite shared content. - **Zip bombs and tar-slip.** There is no archive format and no server-side decompression. Files arrive individually. *Do not "improve" this into tarball upload without re-reviewing this section.* - **Symlink escape from uploaded content.** The server creates only directories and hardlinks inside a deployment tree; it never creates a symlink there. ### 7.5 Tokens Format `pgs__`: an 80-bit key id and a 256-bit random secret. Stored as SHA-256 (not bcrypt — the secret is uniformly random, so there is no dictionary to defend against, and a KDF per request would hand every unauthenticated client a CPU-exhaustion lever), compared in constant time. - Tokens are returned **once**, at creation. - They are **never logged, never accepted in a query string, never included in an error message.** A test asserts this by running a request through the middleware and grepping the log output. - Prefer `PAGES_TOKEN` or `--token-file` over `--token`. On a shared CI runner `argv` is world-readable through `/proc//cmdline`. - Revocation is immediate: the auth cache is invalidated by a generation bump. - The CLI config file is written `0600`. Failed authentications are rate limited per client IP (see `trusted_proxy_cidrs`). Successful ones are not. --- ## 8. Scaling limits **Run exactly one `pages-server` process per database.** The in-memory registry is updated synchronously by the process that performed the write; a second process on the same SQLite file would keep serving stale content indefinitely, because nothing tells it the active deployment changed. Vertically there is plenty of room — serving is a map lookup plus a file read, and reads never touch SQLite. The bound worth knowing is memory: resident manifests cost roughly 100 bytes per file, and only active deployments are resident. A thousand projects of fifty thousand files each would be about 6 GB. Horizontal scaling would need a change-notification mechanism (polling `deployments.updated_at`, or external pub/sub). It is not in v1. --- ## 9. Monitoring and troubleshooting `/healthz` and `/readyz` are served on **both** listeners, so either can be the one your load balancer targets. `/healthz` never touches the database; `/readyz` returns 503 until the database is reachable and the registry is loaded. `GET /api/v1/system/info` (admin) reports version, uptime, project and deployment counts, CAS size and the detected link mode. `pages system info` formats it. Access logs are one line per request with `method`, `path`, `status`, `bytes`, `dur_ms`, `project`, `deployment`, `key_id` and `req_id`. | Symptom | Cause | Fix | |---|---|---| | Site returns 503 | project exists, nothing activated yet | deploy, or activate an existing deployment | | Site returns 404 at `/~name/` | no such project (404 and 503 are deliberately different answers) | check the name | | Assets 404, page loads | build has the wrong base path | rebuild with `--base=/~name/` (§5) | | `$WEBROOT/~name` missing | external tool removed it, or `assemble_mode = "none"` | the reconciler restores it within `reconcile_interval` | | Disk grows steadily | retention too generous, or copy-mode assembly | lower `--retention`, run `pages system gc`, check the link mode in the startup log | | ERROR "blob missing from the content store" | genuine bug, or somebody deleted CAS files | run `pages system fsck`; redeploy affected projects | | Deploy hangs at upload | `max_concurrent_uploads` saturated by another job | wait, or raise the limit | **Recovering a database with an intact CAS** is normal: the server marks blobs whose files are gone as absent at startup and the next deploy re-uploads them. Losing both means redeploying, which for a CI-driven site is one pipeline run. ### Graceful shutdown `SIGINT`/`SIGTERM` stops accepting connections, gives in-flight requests `shutdown_grace` to finish, cancels background workers, checkpoints the WAL and closes cleanly. A second signal forces an immediate close. --- ## 10. Backups Stop the server, or use `sqlite3 pages.db ".backup"` — a plain `cp` of a live WAL database is not safe. Then copy `$DATA_DIR/cas/`. `$DATA_DIR/deployments/` does **not** need backing up: it is reconstructible from the CAS and the database. `$WEBROOT` does not need backing up either; it holds only symlinks, regenerated on the next activation and by the reconciler within `reconcile_interval`.