From 5782207744647de2ef5ff804607114cb3b25860a Mon Sep 17 00:00:00 2001 From: iceBear67 Date: Tue, 28 Jul 2026 05:29:10 +0000 Subject: [PATCH] Add CLAUDE.md Distils the things that took a debugging session to learn and cannot be read off a single file: the two-build-tree workflow, the one-way module dependency chain, the invariants around the single lwIP stack, and the traps (asio error categories, self-re-arming io work blocking shutdown, slice-driven tests being blind to that class of bug). Co-Authored-By: Claude Opus 5 --- CLAUDE.md | 135 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 135 insertions(+) create mode 100644 CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..a39a016 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,135 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## What this is + +A userspace VPN gateway: OpenVPN 3 core builds a tunnel to a VPNGate node, lwIP terminates it +in-process, and a SOCKS5 server (RFC 1928/1929) serves traffic over it. **No root, no tun device, +no routing table changes** — that constraint is why the netstack exists at all, and it is not +negotiable. + +`docs/FEASIBILITY.md` and `docs/ARCHITECTURE.md` (both Chinese) are the design record. Read +FEASIBILITY §1 before touching anything switch-related: carrying established TCP connections +across a node switch is physically impossible, and the code is shaped around that fact. + +## Build and test + +Two build trees by convention — the same source with the tunnel egress off and on: + +```sh +cmake -S . -B build -DCMAKE_BUILD_TYPE=Release -DOVG_WITH_TUNNEL=OFF +cmake -S . -B build-tunnel -DCMAKE_BUILD_TYPE=Release -DOVG_WITH_TUNNEL=ON +cmake --build build -j8 && cmake --build build-tunnel -j8 +``` + +`OFF` compiles in seconds and is the right loop for everything except netstack/ovpn work; `ON` +drags in openvpn3 and lwIP. **Run both suites before declaring anything done** — `netstack` tests +only exist in the tunnel build, and app wiring is `#if OVG_WITH_TUNNEL`'d in places. + +openvpn3 and lwIP are source dependencies. This working copy points at `/tmp/ovpn3` and `/tmp/lwip` +via `OVG_OPENVPN3_DIR` / `OVG_LWIP_DIR`; if `/tmp` was cleared, re-configure without those and +FetchContent re-downloads. + +```sh +./build/tests/ovg_tests # all +./build/tests/ovg_tests manager_walks # substring filter on the test name — this is the only filter +ctest --test-dir build # wraps the whole binary as one test +``` + +Tests are `OVG_TEST(name) { ... }` with `CHECK`/`CHECK_EQ`/`SKIP(reason)` from `tests/harness.h`. +No gtest. Expected: 155 passed / 2 skipped (no tunnel), 172 passed / 3 skipped (tunnel). + +There is no linter or formatter config. Match the surrounding style: comments explain *why*, and +several load-bearing ones exist specifically to stop a future reader "simplifying" a subtlety back +out. Don't delete them to make a diff tidy. + +## Architecture: the parts that span files + +Dependency direction is strictly one-way and enforced by review, not by tooling: +`app → {health, socks5, egress, selector} → {netstack, ovpn, vpngate} → common`. +`socks5` must never learn that `ovpn`/`netstack` exist — it only holds an `Egress`. `netstack` +must never learn `ovpn` exists — it only gets an fd. + +**`egress::Egress` is the seam.** `TunnelEgress` and `DirectEgress` implement it; everything above +is written against the interface, which is what makes `egress_mode = direct` (host sockets, no VPN) +a usable test path and what makes hot-swapping a tunnel possible at all. + +**Refcount is the drain.** `shared_ptr::use_count()` *is* the live session count. A replaced +egress is kept alive by the sessions still on it and disappears when the last one lets go. There is +no separate session registry to keep consistent — don't add one, and don't stash an `EgressPtr` +anywhere that outlives a session (a `promotions` vector in a test that holds egresses instead of +labels will hang the drain forever). + +**Switching is make-before-break** (`EgressManager`, `SwitchController`): the new tunnel is fully up +before the old one is replaced. New sessions land on the new egress; zero-progress TCP sessions are +re-dialled transparently; UDP associations are re-homed in place; everything else drains under +`switch.drain_grace` and is then closed. The promote hook fans out in `app/`, **socks5 first** (it +re-homes while the old egress is still whole), then the switch controller. + +**lwIP has one Stack per process** (`g_stack_live` throws otherwise — its state is in globals) and +every lwIP call happens on `Stack::strand_`. Public `TcpStream`/`UdpSocket` methods post themselves +there, so callers may use them from any thread; completions post back to the caller's executor. +Objects a raw lwIP callback can still reach are destroyed through `common/strand_deleter.h`, not by +the last `shared_ptr` dropping. + +**Selection is two-phase** (`selector/`): a cheap prior over the whole ~95-node list, then real TCP +handshake timing of only the top K. Probing needs a TCP remote — a UDP "connect" completes locally +and measures nothing — and VPNGate is mostly UDP-only, so a top-10 routinely yields 3–4 real probe +targets. That is expected, and the selector logs it so it doesn't read as a broken filter. + +**Health probing must stay a TCP handshake** through the egress (`health/`). A DNS lookup can be +answered from cache without a byte crossing the tunnel, which reports a dead tunnel as the healthiest +node in the fleet. Scores drop terms whose inputs are unavailable and renormalise the remaining +weights: *missing signal ≠ bad signal*. + +## Traps this codebase has already been bitten by + +- **`ec == std::errc::connection_refused` is false for asio errors.** They live in the `asio.system` + category, whose `default_error_condition` doesn't map to `std::errc`. Normalise through + `DirectEgress::map_ec` (and the errno fallback in `socks5_reply_for`) rather than comparing raw. +- **Anything that re-arms on the io_context prevents shutdown.** Dropping the work guard cannot + retire pending work. The lwIP timer (`Stack::stop()`) and the re-armed `signal_set` + (`signals_.cancel()`) both had to be shut down explicitly; without them the process logged a + flawless graceful shutdown and then hung forever. If you add a self-rescheduling timer, add its + teardown to `App::begin_shutdown` in the same commit. +- **The test suite is structurally blind to that class of bug**: tests drive the loop in + `io.run_for(2ms)` slices, while the service calls `run()` once and waits for it to return. See + `stack_stop_lets_the_io_context_drain` for the shape of a test that actually checks it. +- **A cold start with no directory yet is not a switch failure.** Charging it to the backoff ladder + cost 30s of dead service on every start and could exhaust the startup budget. `Selector::directory_loaded()` + distinguishes "nothing has arrived yet" from "everything was rejected"; the former uses + `retry_startup`, which touches neither backoff nor failure counters. +- **Phase transitions must be claimed atomically.** `claim_switch_phase()` exists because releasing + the phase during a retry let the tunnel-down watchdog and the startup poll both start selections + and build two tunnels for one slot. +- **Never `cat`/`head` the VPNGate CSV.** Lines run to ~13.5 KB because the whole .ovpn profile is + base64 in the last column. Decode field 15 in a script and print only what you need. + +## This sandbox + +Outbound TCP is transparently intercepted: connects to unroutable addresses (e.g. `192.0.2.1:1213`) +"succeed" in ~0.9 ms and then return nothing. Consequences, all environmental rather than bugs — +do not go chasing them in the code: + +- every latency probe reports ~1 ms, so selection here is effectively arbitrary; +- OpenVPN handshakes die with `NETWORK_EOF_ERROR`, so the tunnel **data plane cannot be verified + here**. The control path (selection, connect attempts, reconnect, graceful shutdown) can be, and is; +- three tests `SKIP` for exactly this reason; +- a local OpenVPN server is not an option either: no `/dev/net/tun`, `TUNSETIFF` gives EPERM, no + kernel modules. + +`tools/tunnel_smoke.cpp` (built as `ovg_tunnel_smoke`, meaningful only in the tunnel build) brings up +one tunnel, pings through it, and exits — the first thing to run on a machine with real network access. + +## Running it + +```sh +./build/src/openvpngate -c etc/openvpngate.conf --check # validate + print config, don't start +./build/src/openvpngate -c etc/openvpngate.conf --egress direct --listen 127.0.0.1:1080 +``` + +`etc/openvpngate.conf` documents every key with its default and the reasoning. Admin HTTP (default +`127.0.0.1:9080`, **no auth**) exposes `/status /nodes /sessions /health /metrics /healthz` and +`POST /switch`; `/nodes` explains each node's score, which is the fastest way to understand a +selection decision. `SIGHUP` reloads credentials and the node list without dropping sessions.