Add container build, compose example and deployment docs
Multi-stage Dockerfile on debian:trixie-slim. openvpn3 and lwIP are cloned at pinned refs and handed to CMake through OVG_OPENVPN3_DIR/OVG_LWIP_DIR rather than left to FetchContent, whose GIT_TAG master would make the same Dockerfile build a different VPN client each week. The unit suite runs in the builder stage. docker/openvpngate.conf overrides only the keys whose host default is wrong inside a container -- loopback listen addresses, which make a published port reach nothing, and relative state paths, which put the node failure history on a layer that gets thrown away. Everything else stays absent and takes the compiled-in default so the file cannot drift from the code. The compose example drops every capability, runs read-only as uid 10001 and publishes both ports to host loopback: the admin endpoint has no auth and includes POST /switch. That configuration is the design constraint of this project (no root, no tun device) turned into something testable. docs/DOCKER.md 8 records what was checked against the source and what was not: this sandbox has no docker daemon, so neither the image build nor the compose file has actually been run. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
iceBear67
co-authored by
Claude Opus 5
Claude Opus 5
parent
5782207744
commit
f37cd0a125
@@ -0,0 +1,32 @@
|
||||
# The build context is COPYied wholesale into the builder stage. Without this
|
||||
# the two local build trees (~1 GB with the openvpn3 objects) would be shipped
|
||||
# to the daemon on every `docker build`, only to be overwritten by the
|
||||
# container's own cmake run.
|
||||
build/
|
||||
build-tunnel/
|
||||
build*/
|
||||
|
||||
# The image builds from the working tree, not from git history.
|
||||
.git/
|
||||
.gitignore
|
||||
.dockerignore
|
||||
|
||||
# Runtime state. The node cache and the outcome history belong to whichever
|
||||
# machine produced them; the container gets its own in a volume.
|
||||
var/
|
||||
|
||||
# Local operator config and credentials. The container reads its config from a
|
||||
# mount (see docker-compose.yml), never from something baked into the layer.
|
||||
etc/
|
||||
# Patterns match the whole context-relative path, so a bare *.auth would only
|
||||
# cover the top level -- and the credential file lives in docker/.
|
||||
**/*.auth
|
||||
|
||||
# Editor and OS noise
|
||||
*.swp
|
||||
*~
|
||||
.vscode/
|
||||
.idea/
|
||||
.DS_Store
|
||||
.cache/
|
||||
compile_commands.json
|
||||
@@ -129,6 +129,13 @@ one tunnel, pings through it, and exits — the first thing to run on a machine
|
||||
./build/src/openvpngate -c etc/openvpngate.conf --egress direct --listen 127.0.0.1:1080
|
||||
```
|
||||
|
||||
There is a container build too (`Dockerfile`, `docker-compose.yml`, `docs/DOCKER.md`). Two things
|
||||
about it are load-bearing: it clones openvpn3/lwIP at pinned refs and passes them via
|
||||
`OVG_OPENVPN3_DIR`/`OVG_LWIP_DIR`, because FetchContent's `GIT_TAG master` would otherwise make the
|
||||
image a different VPN client every week; and `docker/openvpngate.conf` overrides the two host
|
||||
defaults that are silently wrong in a container — loopback listen addresses (a published port then
|
||||
reaches nothing) and relative state paths (the node history dies with the container).
|
||||
|
||||
`etc/openvpngate.conf` documents every key with its default and the reasoning. Admin HTTP (default
|
||||
`127.0.0.1:9080`, **no auth**) exposes `/status /nodes /sessions /health /metrics /healthz` and
|
||||
`POST /switch`; `/nodes` explains each node's score, which is the fastest way to understand a
|
||||
|
||||
+155
@@ -0,0 +1,155 @@
|
||||
# syntax=docker/dockerfile:1
|
||||
#
|
||||
# openvpngate as a container.
|
||||
#
|
||||
# The interesting property of this image is what it does *not* need: no
|
||||
# --privileged, no --cap-add NET_ADMIN, no --device /dev/net/tun, no
|
||||
# --sysctl. The tunnel is terminated in userspace by lwIP, so from the kernel's
|
||||
# point of view this is an ordinary unprivileged process that listens on a
|
||||
# socket. docker-compose.yml drops every capability to make that testable
|
||||
# rather than merely claimed.
|
||||
#
|
||||
# See docs/DOCKER.md for the container-specific configuration traps -- the
|
||||
# defaults in etc/openvpngate.conf are written for a host, and two of them
|
||||
# (loopback listen addresses, relative state paths) are actively wrong here.
|
||||
|
||||
# Pinned rather than :latest, and pinned to *trixie* specifically: the runtime
|
||||
# package names below are release-specific (bookworm ships libssl3 and libfmt9,
|
||||
# trixie ships libssl3t64 and libfmt10). Moving this tag means re-checking them.
|
||||
ARG DEBIAN_TAG=trixie-slim
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Stage 1 -- build
|
||||
# ---------------------------------------------------------------------------
|
||||
FROM debian:${DEBIAN_TAG} AS build
|
||||
|
||||
# openvpn3 and lwIP are source dependencies. Left to itself, CMake's
|
||||
# FetchContent pulls openvpn3 at GIT_TAG master, which makes this image
|
||||
# un-reproducible: the same Dockerfile and the same commit of this repo would
|
||||
# build a different VPN client next week. So clone them here at pinned refs and
|
||||
# hand the checkouts to CMake through OVG_OPENVPN3_DIR / OVG_LWIP_DIR, the knob
|
||||
# it already has for a pre-existing checkout.
|
||||
#
|
||||
# The openvpn3 default is the commit this tree was developed and tested against.
|
||||
# Bump it deliberately, and re-run the suite when you do.
|
||||
ARG OPENVPN3_REF=1512c16622288f3c01da09d3278ac61a86dca26d
|
||||
ARG LWIP_REF=STABLE-2_2_1_RELEASE
|
||||
|
||||
# OFF builds without openvpn3/lwIP: `direct` egress only, host sockets, no VPN.
|
||||
# Useful for testing the SOCKS5 layer in isolation, useless as a gateway -- and
|
||||
# the binary refuses to start in tunnel mode rather than proxying in the clear.
|
||||
ARG OVG_WITH_TUNNEL=ON
|
||||
|
||||
# The unit tests are the only thing between "it compiled" and "it works", and
|
||||
# they cost seconds against a build measured in minutes. Turn off with
|
||||
# --build-arg OVG_RUN_TESTS=0 on a builder without loopback networking: a dozen
|
||||
# tests bind and connect on 127.0.0.1.
|
||||
ARG OVG_RUN_TESTS=1
|
||||
|
||||
ARG BUILD_JOBS=
|
||||
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
build-essential \
|
||||
cmake \
|
||||
pkg-config \
|
||||
git \
|
||||
ca-certificates \
|
||||
libasio-dev \
|
||||
libssl-dev \
|
||||
liblz4-dev \
|
||||
libfmt-dev \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Fetched before the source is copied, so editing a .cpp does not re-clone the
|
||||
# dependencies. --filter=blob:none keeps the clone small while still allowing a
|
||||
# checkout of an arbitrary ref -- which a --depth 1 clone cannot do for a bare
|
||||
# commit id, and the openvpn3 pin above is one.
|
||||
WORKDIR /deps
|
||||
RUN git clone --filter=blob:none --no-checkout \
|
||||
https://github.com/OpenVPN/openvpn3.git openvpn3 \
|
||||
&& git -C openvpn3 checkout --detach "${OPENVPN3_REF}" \
|
||||
&& git clone --filter=blob:none --no-checkout \
|
||||
https://github.com/lwip-tcpip/lwip.git lwip \
|
||||
&& git -C lwip checkout --detach "${LWIP_REF}"
|
||||
|
||||
WORKDIR /src
|
||||
COPY . .
|
||||
|
||||
RUN cmake -S . -B build \
|
||||
-DCMAKE_BUILD_TYPE=Release \
|
||||
-DOVG_WITH_TUNNEL="${OVG_WITH_TUNNEL}" \
|
||||
-DOVG_BUILD_TESTS=ON \
|
||||
-DOVG_OPENVPN3_DIR=/deps/openvpn3 \
|
||||
-DOVG_LWIP_DIR=/deps/lwip \
|
||||
&& cmake --build build -j"${BUILD_JOBS:-$(nproc)}"
|
||||
|
||||
RUN if [ "${OVG_RUN_TESTS}" = "1" ]; then ./build/tests/ovg_tests; fi
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Stage 2 -- runtime
|
||||
# ---------------------------------------------------------------------------
|
||||
FROM debian:${DEBIAN_TAG} AS runtime
|
||||
|
||||
LABEL org.opencontainers.image.title="openvpngate" \
|
||||
org.opencontainers.image.description="OpenVPN client with an authenticated SOCKS5 front door, terminated in userspace" \
|
||||
org.opencontainers.image.version="0.1.0" \
|
||||
org.opencontainers.image.licenses="NOASSERTION"
|
||||
|
||||
# Release-specific names -- see the DEBIAN_TAG comment at the top.
|
||||
# curl is here only for the HEALTHCHECK below; drop both together if you run
|
||||
# with admin.enabled = false.
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
libssl3t64 \
|
||||
liblz4-1 \
|
||||
libfmt10 \
|
||||
ca-certificates \
|
||||
curl \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# A fixed uid, not a distro-assigned one: a bind-mounted state directory has to
|
||||
# be chown'd to a number the host knows in advance.
|
||||
RUN groupadd --system --gid 10001 ovg \
|
||||
&& useradd --system --uid 10001 --gid 10001 \
|
||||
--home-dir /var/lib/openvpngate --shell /usr/sbin/nologin ovg
|
||||
|
||||
COPY --from=build /src/build/src/openvpngate /usr/local/bin/openvpngate
|
||||
# Brings up one tunnel, pings through it, exits. The first thing worth running
|
||||
# on a host with real network access, and the fastest way to tell "the image is
|
||||
# broken" from "this network cannot reach VPNGate".
|
||||
COPY --from=build /src/build/src/ovg_tunnel_smoke /usr/local/bin/ovg_tunnel_smoke
|
||||
|
||||
# Baked in so `docker run` alone works; docker-compose.yml mounts over it so the
|
||||
# config can be edited without a rebuild.
|
||||
COPY docker/openvpngate.conf /etc/openvpngate/openvpngate.conf
|
||||
|
||||
# The one writable path the service needs: the node cache and the per-node
|
||||
# outcome history. An empty named volume mounted here inherits this ownership,
|
||||
# which is what makes `read_only: true` on the rest of the rootfs work.
|
||||
RUN install -d -o ovg -g ovg -m 0750 /var/lib/openvpngate
|
||||
|
||||
# So that a relative path in a user-supplied config resolves somewhere writable
|
||||
# instead of failing at the first cache write.
|
||||
WORKDIR /var/lib/openvpngate
|
||||
|
||||
# 1080 SOCKS5, 9080 admin. The admin endpoint has NO authentication; publish it
|
||||
# to host loopback or not at all.
|
||||
EXPOSE 1080 9080
|
||||
|
||||
USER ovg
|
||||
|
||||
# Liveness only -- /healthz answers 200 as soon as the admin server is up and
|
||||
# says nothing about whether a tunnel is established. Tunnel state is
|
||||
# GET /status, and the service already reacts to a degraded tunnel by switching
|
||||
# nodes on its own; see docs/DOCKER.md before wiring restart-on-unhealthy.
|
||||
# start-period covers a cold start: fetch the directory, probe candidates,
|
||||
# negotiate with a volunteer-run server on the other side of the world.
|
||||
HEALTHCHECK --interval=30s --timeout=5s --start-period=90s --retries=3 \
|
||||
CMD curl -fsS http://127.0.0.1:9080/healthz || exit 1
|
||||
|
||||
# Explicit because the second one means something different: SIGTERM starts a
|
||||
# graceful shutdown, a second SIGTERM/SIGINT during it stops immediately. The
|
||||
# binary runs as pid 1 (exec form, no shell wrapper), so it receives them.
|
||||
STOPSIGNAL SIGTERM
|
||||
|
||||
ENTRYPOINT ["/usr/local/bin/openvpngate"]
|
||||
CMD ["-c", "/etc/openvpngate/openvpngate.conf"]
|
||||
@@ -18,6 +18,7 @@ SOCKS5 客户端 ──► socks5::Server ──► egress::Egress ──► lwI
|
||||
| [docs/FEASIBILITY.md](docs/FEASIBILITY.md) | 动手前的可行性结论。**需求中唯一不可能的部分在 §1**；UDP ASSOCIATE 的明确表态在 §5；1000 并发的真实天花板在 §4 |
|
||||
| [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) | 模块边界、线程模型、切换状态机、选点与健康检查的具体算法 |
|
||||
| [etc/openvpngate.conf](etc/openvpngate.conf) | 全部配置项，每一项都带默认值和「为什么是这个默认值」 |
|
||||
| [docs/DOCKER.md](docs/DOCKER.md) | 容器部署。**容器里有三处宿主机默认值是错的**（§3）；`cap_drop: ALL` 为什么能成立（§5） |
|
||||
|
||||
---
|
||||
|
||||
@@ -125,6 +126,20 @@ curl -sS --socks5-hostname alice:changeme@127.0.0.1:1080 https://example.com -o
|
||||
`--socks5-hostname` 让 curl 把域名交给代理解析（DNS 不泄漏路径）；`--socks5` 则是本地解析。
|
||||
两条路径都支持。
|
||||
|
||||
### 用容器跑
|
||||
|
||||
```sh
|
||||
cp docker/socks5.auth.example docker/socks5.auth # 改口令
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
镜像以 uid 10001、`cap_drop: ALL`、只读 rootfs 运行——不需要 `NET_ADMIN`，不需要
|
||||
`/dev/net/tun`，这正是用户态终结隧道换来的东西，而容器让这句话第一次可以当场验证。
|
||||
|
||||
**但宿主机的默认配置在容器里有两处是静默错误的**（监听环回口、状态文件用相对路径），
|
||||
所以镜像自带一份 `docker/openvpngate.conf` 覆盖它们。细节、暴露面取舍、以及这套编排
|
||||
**哪些部分没有被真正跑过**，见 [docs/DOCKER.md](docs/DOCKER.md)。
|
||||
|
||||
---
|
||||
|
||||
## 4. 管理接口
|
||||
|
||||
@@ -0,0 +1,157 @@
|
||||
# Example deployment. See docs/DOCKER.md for what each block is protecting
|
||||
# against; the short version is that a userspace VPN client should be able to
|
||||
# run with every capability dropped, and this file is where that gets proven
|
||||
# rather than asserted.
|
||||
#
|
||||
# cp docker/socks5.auth.example docker/socks5.auth # then edit it
|
||||
# docker compose up -d --build
|
||||
# curl -x socks5h://alice:...@127.0.0.1:1080 https://ifconfig.me
|
||||
#
|
||||
name: openvpngate
|
||||
|
||||
services:
|
||||
openvpngate:
|
||||
build:
|
||||
context: .
|
||||
args:
|
||||
# OFF drops openvpn3 and lwIP: `direct` egress only, no VPN. Builds in
|
||||
# seconds, useful for exercising the SOCKS5 layer, useless as a gateway.
|
||||
OVG_WITH_TUNNEL: "ON"
|
||||
image: openvpngate:0.1.0
|
||||
restart: unless-stopped
|
||||
|
||||
# Both published to host loopback, and that is doing real work in each case:
|
||||
# 1080 the proxy is only as private as who can reach it;
|
||||
# 9080 the admin endpoint has NO authentication and includes POST /switch.
|
||||
# Change these to 0.0.0.0 only after reading docs/DOCKER.md 4.
|
||||
ports:
|
||||
- "127.0.0.1:1080:1080"
|
||||
- "127.0.0.1:9080:9080"
|
||||
|
||||
volumes:
|
||||
# Long syntax with create_host_path: false on purpose. Docker's default is
|
||||
# to silently create a *directory* when a bind mount's source is missing,
|
||||
# which for the auth file means the proxy starts and refuses every login,
|
||||
# and for the config means it starts on defaults that listen on loopback
|
||||
# inside the container and are reachable by nobody. Fail at `up` instead.
|
||||
- type: bind
|
||||
source: ./docker/openvpngate.conf
|
||||
target: /etc/openvpngate/openvpngate.conf
|
||||
read_only: true
|
||||
bind:
|
||||
create_host_path: false
|
||||
|
||||
- type: bind
|
||||
source: ./docker/socks5.auth
|
||||
target: /etc/openvpngate/socks5.auth
|
||||
read_only: true
|
||||
bind:
|
||||
create_host_path: false
|
||||
|
||||
# The node cache and the per-node failure history. Worth persisting for
|
||||
# more than startup speed: the history is the record of which volunteer
|
||||
# nodes have already failed on you, and throwing it away on every `up`
|
||||
# means walking back into the same one.
|
||||
- ovg-state:/var/lib/openvpngate
|
||||
|
||||
# ---- the point of the exercise ----------------------------------------
|
||||
# No NET_ADMIN, no /dev/net/tun, no --privileged, no sysctls, uid 10001.
|
||||
# The tunnel is terminated by lwIP inside the process, so nothing here
|
||||
# needs kernel networking privileges. If a change to this project ever
|
||||
# makes one of these lines necessary, the change is wrong.
|
||||
cap_drop:
|
||||
- ALL
|
||||
security_opt:
|
||||
- no-new-privileges:true
|
||||
read_only: true
|
||||
tmpfs:
|
||||
# Nothing is known to need it; it is here so that a library that decides
|
||||
# to spool to /tmp fails loudly at write time rather than at connect time.
|
||||
- /tmp:size=16m,mode=1777
|
||||
|
||||
# SIGTERM starts a graceful shutdown (stop accepting, drain, tear the tunnel
|
||||
# down); it normally completes in well under a second. The window is wide
|
||||
# because the alternative when it does not is SIGKILL in the middle of
|
||||
# writing the node history.
|
||||
stop_grace_period: 30s
|
||||
|
||||
# socks5.max_sessions defaults to 1200 and each session costs a client-side
|
||||
# fd (plus an egress-side one in direct mode). The Docker default is usually
|
||||
# far higher than this, but it is not guaranteed to be.
|
||||
ulimits:
|
||||
nofile:
|
||||
soft: 8192
|
||||
hard: 8192
|
||||
|
||||
# A gateway logs one line per session at info. Unrotated json-file logging
|
||||
# is how a container quietly fills a host disk.
|
||||
logging:
|
||||
driver: json-file
|
||||
options:
|
||||
max-size: "10m"
|
||||
max-file: "5"
|
||||
|
||||
# The image's HEALTHCHECK probes admin /healthz. It is liveness only -- it
|
||||
# says the process is answering, not that a tunnel is up. Do NOT wire an
|
||||
# unhealthy-triggers-restart supervisor on top: the service already responds
|
||||
# to a degrading tunnel by switching nodes, and a restart throws away the
|
||||
# drain, the session, and the freshly-learned reason the node was bad.
|
||||
|
||||
# -------------------------------------------------------------------------
|
||||
# Proxy without a VPN, for splitting "is the SOCKS5 implementation correct"
|
||||
# from "is the tunnel up". Egress is a plain host socket -- traffic through
|
||||
# this one is NOT tunnelled, which is why it is behind a profile and on its
|
||||
# own port.
|
||||
#
|
||||
# docker compose --profile test up openvpngate-direct
|
||||
# -------------------------------------------------------------------------
|
||||
openvpngate-direct:
|
||||
profiles: [test]
|
||||
# Same image and same build inputs as above, so this resolves to the one
|
||||
# already built rather than trying to pull it from a registry.
|
||||
build:
|
||||
context: .
|
||||
args:
|
||||
OVG_WITH_TUNNEL: "ON"
|
||||
image: openvpngate:0.1.0
|
||||
command:
|
||||
- "-c"
|
||||
- "/etc/openvpngate/openvpngate.conf"
|
||||
- "--egress"
|
||||
- "direct"
|
||||
- "--no-admin"
|
||||
ports:
|
||||
- "127.0.0.1:1081:1080"
|
||||
volumes:
|
||||
- type: bind
|
||||
source: ./docker/openvpngate.conf
|
||||
target: /etc/openvpngate/openvpngate.conf
|
||||
read_only: true
|
||||
bind:
|
||||
create_host_path: false
|
||||
- type: bind
|
||||
source: ./docker/socks5.auth
|
||||
target: /etc/openvpngate/socks5.auth
|
||||
read_only: true
|
||||
bind:
|
||||
create_host_path: false
|
||||
# Inherited from the image, but --no-admin means there is nothing listening
|
||||
# on 9080 to answer it; left armed it would report this container unhealthy
|
||||
# forever.
|
||||
healthcheck:
|
||||
disable: true
|
||||
cap_drop:
|
||||
- ALL
|
||||
security_opt:
|
||||
- no-new-privileges:true
|
||||
read_only: true
|
||||
tmpfs:
|
||||
- /tmp:size=16m,mode=1777
|
||||
# This one still fetches and caches the node directory even though it
|
||||
# never dials a node, so it needs its state directory writable. Throwaway
|
||||
# rather than a volume: nothing produced by a no-VPN test run is worth
|
||||
# keeping, least of all a failure history for nodes it never contacted.
|
||||
- /var/lib/openvpngate:size=64m,mode=1777
|
||||
|
||||
volumes:
|
||||
ovg-state:
|
||||
@@ -0,0 +1,71 @@
|
||||
# openvpngate -- configuration baked into the container image.
|
||||
#
|
||||
# This file sets ONLY the keys whose host default is wrong inside a container.
|
||||
# Everything else is left absent and takes the compiled-in default, so this
|
||||
# file cannot drift from the code the way a copied-and-edited full config does.
|
||||
#
|
||||
# etc/openvpngate.conf in the source tree is the annotated reference: every key,
|
||||
# its default, and why that is the default. Read that one to change behaviour,
|
||||
# then add the key here.
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
[socks5]
|
||||
# ---------------------------------------------------------------------------
|
||||
# The default is 127.0.0.1, which inside a container means "reachable from this
|
||||
# container only" -- a published port would connect to nothing. Bind everywhere
|
||||
# and let the container runtime decide who gets to reach it: docker-compose.yml
|
||||
# publishes this to host loopback, not to the LAN.
|
||||
listen_address = 0.0.0.0
|
||||
listen_port = 1080
|
||||
|
||||
require_auth = true
|
||||
|
||||
# Credentials stay out of the image. Mount a file here (docker-compose.yml
|
||||
# does) -- SIGHUP re-reads it without dropping a live session:
|
||||
#
|
||||
# docker compose kill -s HUP openvpngate
|
||||
#
|
||||
# If this path is missing the process exits at startup with
|
||||
# "config: cannot open auth file"; if it is a *directory* -- which is what
|
||||
# Docker silently creates for a bind mount whose source does not exist -- the
|
||||
# file parses as empty and startup warns "every login will be refused". The
|
||||
# compose file uses create_host_path: false so that case fails loudly instead.
|
||||
auth_file = /etc/openvpngate/socks5.auth
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
[vpngate]
|
||||
# ---------------------------------------------------------------------------
|
||||
# Absolute. The default is relative (var/vpngate_cache.csv) and would resolve
|
||||
# against the working directory; inside the image that is /var/lib/openvpngate,
|
||||
# which happens to be right, but only by accident. Say it explicitly, because
|
||||
# this is also the one path that survives a container replacement and the one
|
||||
# directory a read-only rootfs still permits writing to.
|
||||
cache_path = /var/lib/openvpngate/vpngate_cache.csv
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
[selector]
|
||||
# ---------------------------------------------------------------------------
|
||||
# Same reasoning. Worth persisting for a different reason though: this is the
|
||||
# record of which nodes have failed on you, and losing it on every `up` means
|
||||
# walking back into the same broken node with a flattering API score.
|
||||
history_path = /var/lib/openvpngate/node_history.tsv
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
[admin]
|
||||
# ---------------------------------------------------------------------------
|
||||
# Bound everywhere for the same reason as socks5 above -- but this endpoint has
|
||||
# NO authentication and includes POST /switch, so whoever can reach it can force
|
||||
# your gateway onto another node. It is protected here by *publishing* rules,
|
||||
# not by the bind address: keep it on 127.0.0.1 on the host side, or set
|
||||
# enabled = false and drop the HEALTHCHECK from the Dockerfile, which probes it.
|
||||
enabled = true
|
||||
listen_address = 0.0.0.0
|
||||
listen_port = 9080
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
[log]
|
||||
# ---------------------------------------------------------------------------
|
||||
# stderr, so that `docker logs` and the compose log driver see it. Logging to a
|
||||
# file inside a container puts the record on the layer that gets thrown away.
|
||||
level = info
|
||||
file = -
|
||||
@@ -0,0 +1,32 @@
|
||||
# SOCKS5 credentials, one per line: <user>:<secret>
|
||||
#
|
||||
# cp docker/socks5.auth.example docker/socks5.auth
|
||||
# $EDITOR docker/socks5.auth
|
||||
#
|
||||
# docker-compose.yml mounts docker/socks5.auth read-only into the container.
|
||||
# Reloaded on SIGHUP without dropping a live session:
|
||||
#
|
||||
# docker compose kill -s HUP openvpngate
|
||||
#
|
||||
# Two accepted forms.
|
||||
#
|
||||
# 1. Plaintext. Hashed with a random salt when the file is read, so it is never
|
||||
# held in memory in the clear -- but it is sitting in the clear right here,
|
||||
# which is the part that matters on a shared host.
|
||||
#
|
||||
# alice:changeme
|
||||
#
|
||||
# 2. Pre-hashed: sha256$<salt_hex>$<sha256_hex(salt_hex + password)>. Note the
|
||||
# salt is concatenated as its *hex text*, not as raw bytes. Generate one:
|
||||
#
|
||||
# salt=$(openssl rand -hex 16)
|
||||
# printf 'alice:sha256$%s$%s\n' "$salt" \
|
||||
# "$(printf '%s' "${salt}${PASSWORD}" | sha256sum | cut -d' ' -f1)"
|
||||
#
|
||||
# Lines starting with '#' and blank lines are ignored. A malformed line is a
|
||||
# hard startup error, not a skipped entry: half-loaded credentials are worse
|
||||
# than none.
|
||||
#
|
||||
# The entry below is an example and will be rejected by anyone paying
|
||||
# attention. Replace it.
|
||||
alice:changeme
|
||||
+237
@@ -0,0 +1,237 @@
|
||||
# 容器部署
|
||||
|
||||
这份文档存在的理由有两个，都不是「怎么打个包」：
|
||||
|
||||
1. **`cap_drop: ALL` 在这里是可以成立的。** 全项目最核心的约束是「不需要 root、不需要 tun
|
||||
设备、不需要改路由表」（README 开头那句），在宿主机上这句话只能靠读代码相信；在容器里
|
||||
它变成一条可以当场验证的断言——把所有 capability 丢掉、rootfs 只读、非 root uid，服务
|
||||
照常工作。`docker-compose.yml` 里那几行安全选项不是装饰，是这个设计的验收条件。
|
||||
2. **宿主机的默认配置在容器里有两处是错的**，而且都是「静默地错」：监听环回口会让发布的
|
||||
端口连不到任何东西，相对路径的状态文件会写到一个随容器一起消失的层里。
|
||||
|
||||
先读哪一份：镜像与编排的**为什么**在这里，全部配置项的含义在
|
||||
[etc/openvpngate.conf](../etc/openvpngate.conf)，设计本身在
|
||||
[ARCHITECTURE.md](ARCHITECTURE.md)。
|
||||
|
||||
---
|
||||
|
||||
## 1. 快速开始
|
||||
|
||||
```sh
|
||||
cp docker/socks5.auth.example docker/socks5.auth
|
||||
$EDITOR docker/socks5.auth # 换掉 alice:changeme
|
||||
|
||||
docker compose up -d --build
|
||||
docker compose logs -f
|
||||
```
|
||||
|
||||
冷启动要花几十秒，这是正常的：抓 1.3 MB 的节点目录 → 对候选节点做真实 TCP 握手计时 →
|
||||
和地球另一端一台志愿者跑的服务器完成 OpenVPN 握手。镜像里 `HEALTHCHECK` 的
|
||||
`--start-period=90s` 就是按这个量级给的。日志里出现 `egress ready on <节点>` 才算真正可用。
|
||||
|
||||
验证：
|
||||
|
||||
```sh
|
||||
curl -x socks5h://alice:口令@127.0.0.1:1080 https://ifconfig.me
|
||||
```
|
||||
|
||||
`socks5h` 让 curl 把域名交给代理去解析（DNS 走隧道，不泄漏）；`socks5` 是本地解析后只把
|
||||
IP 交过来。两条路径都支持，但只有前者是你部署 VPN 网关想要的那条。
|
||||
|
||||
---
|
||||
|
||||
## 2. 镜像里有什么
|
||||
|
||||
| 路径 | 内容 |
|
||||
|---|---|
|
||||
| `/usr/local/bin/openvpngate` | 主程序，以 uid/gid `10001` 运行，ENTRYPOINT |
|
||||
| `/usr/local/bin/ovg_tunnel_smoke` | 只建一条隧道、ping、退出的诊断工具。**接手一台新机器后第一个该跑的东西**：它能把「镜像坏了」和「这个网络到不了 VPNGate」分开 |
|
||||
| `/etc/openvpngate/openvpngate.conf` | 容器默认配置，compose 会挂载覆盖它 |
|
||||
| `/var/lib/openvpngate` | 唯一需要可写的目录：节点缓存 + 节点历史。工作目录也是这里 |
|
||||
|
||||
构建参数：
|
||||
|
||||
| `--build-arg` | 默认 | 说明 |
|
||||
|---|---|---|
|
||||
| `DEBIAN_TAG` | `trixie-slim` | **改它要同时改运行时包名**：bookworm 是 `libssl3`/`libfmt9`，trixie 是 `libssl3t64`/`libfmt10` |
|
||||
| `OPENVPN3_REF` | `1512c166…` | openvpn3 的固定 commit |
|
||||
| `LWIP_REF` | `STABLE-2_2_1_RELEASE` | lwIP 的 tag |
|
||||
| `OVG_WITH_TUNNEL` | `ON` | `OFF` 只编 `direct` 出口，不拉 openvpn3/lwIP，几秒编完；此时二进制拒绝以 tunnel 模式启动 |
|
||||
| `OVG_RUN_TESTS` | `1` | 在构建阶段跑完整单元测试。构建机没有环回网络时关掉（十几个测试要 bind 127.0.0.1） |
|
||||
| `BUILD_JOBS` | `$(nproc)` | 编译并发度 |
|
||||
|
||||
### 为什么要钉住 openvpn3 的 commit
|
||||
|
||||
`cmake/Dependencies.cmake` 里 openvpn3 的 `FetchContent` 用的是 `GIT_TAG master`。放着不管的
|
||||
话，同一个 Dockerfile、同一个本仓库 commit，下周构建出来的是一个不同的 VPN 客户端——这在
|
||||
本地开发里只是有点烦，在镜像里等于没有可复现构建。所以 Dockerfile 自己按固定 ref clone，
|
||||
再通过 `OVG_OPENVPN3_DIR` / `OVG_LWIP_DIR` 交给 CMake，用的是它本来就有的那个开关。
|
||||
|
||||
默认那个 commit 就是本仓库开发和测试时用的那个。升级它是个需要**主动做**的决定，做完请重跑
|
||||
测试套件。
|
||||
|
||||
---
|
||||
|
||||
## 3. 容器里必须改的三处配置
|
||||
|
||||
`docker/openvpngate.conf` 只写了这三类键，其余全部留空走编译内置默认值——这样它不会像
|
||||
「复制一份完整配置再改几行」那样慢慢和代码脱节。
|
||||
|
||||
| 键 | 宿主机默认 | 容器里 | 为什么 |
|
||||
|---|---|---|---|
|
||||
| `socks5.listen_address` | `127.0.0.1` | `0.0.0.0` | 环回口在容器里的意思是「只有本容器能连」，`-p` 发布出去的端口会连到一个没人监听的地址。安全性改由**发布规则**提供：compose 把它发布到宿主机环回口 |
|
||||
| `admin.listen_address` | `127.0.0.1` | `0.0.0.0` | 同上。但这个接口**没有认证**，见 §4 |
|
||||
| `vpngate.cache_path`<br>`selector.history_path` | `var/…`（相对） | `/var/lib/…`（绝对） | 相对路径按工作目录解析，写进容器可写层就随容器一起没了。节点历史尤其不该丢：它是「哪些节点已经坑过你」的记录，丢掉就会顶着一个好看的 API 分数再走进同一个坑 |
|
||||
| `socks5.auth_file` | 未设置 | `/etc/openvpngate/socks5.auth` | 凭据不进镜像层。挂载进来，`SIGHUP` 热重载 |
|
||||
|
||||
### 凭据文件那个 Docker 陷阱
|
||||
|
||||
bind mount 的源文件不存在时，Docker 的默认行为是**在宿主机上建一个同名目录**。之后：
|
||||
|
||||
- 配置文件被挂成目录 → 服务读到空配置，用内置默认值启动，监听容器内环回口，谁也连不上；
|
||||
- 认证文件被挂成目录 → 解析出零条凭据，启动时打印
|
||||
`warning: socks5.require_auth is on but no credentials are configured; every login will be refused`，
|
||||
然后拒绝所有人。
|
||||
|
||||
所以 `docker-compose.yml` 里这两个挂载用的是长语法加 `create_host_path: false`：宁可在
|
||||
`up` 的时候直接失败。而如果 `auth_file` 指向的路径**完全不存在**，进程会在启动时以
|
||||
`config: cannot open auth file: …` 退出——这是好事，是快速失败。
|
||||
|
||||
---
|
||||
|
||||
## 4. 端口与暴露面
|
||||
|
||||
| 端口 | 内容 | 建议 |
|
||||
|---|---|---|
|
||||
| 1080 | SOCKS5，需要用户名/口令 | 发布到宿主机环回口，或者放进一个只有客户端在的 docker 网络 |
|
||||
| 9080 | 管理 HTTP | **没有任何认证**，而且包含 `POST /switch` |
|
||||
|
||||
把 9080 发布到 `0.0.0.0` 意味着：能访问到它的人可以读到你在线会话的列表，也可以随时强制
|
||||
你的网关换节点（换节点会掐掉所有已经传过字节的连接）。这不是理论风险，是一个 HTTP POST。
|
||||
|
||||
不想要它的话，三处一起改：`docker/openvpngate.conf` 里 `admin.enabled = false`、命令行加
|
||||
`--no-admin`、并把 Dockerfile 里的 `HEALTHCHECK` 去掉（它探的就是 `/healthz`）。
|
||||
|
||||
---
|
||||
|
||||
## 5. 权限：验证那句「不需要 root」
|
||||
|
||||
compose 里这几行是断言，也是测试：
|
||||
|
||||
```yaml
|
||||
cap_drop: [ALL]
|
||||
security_opt: [no-new-privileges:true]
|
||||
read_only: true
|
||||
```
|
||||
|
||||
跑起来之后自己查：
|
||||
|
||||
```sh
|
||||
docker compose exec openvpngate cat /proc/self/status | grep -E 'Cap(Eff|Prm)|^Uid'
|
||||
# Uid: 10001 10001 10001 10001
|
||||
# CapPrm: 0000000000000000
|
||||
# CapEff: 0000000000000000
|
||||
|
||||
docker compose exec openvpngate ls /dev/net/tun # No such file or directory
|
||||
```
|
||||
|
||||
零 capability、非 root、没有 tun 设备，SOCKS5 照常出流量。对照绝大多数 VPN 客户端容器
|
||||
需要的 `--cap-add NET_ADMIN --device /dev/net/tun`——省掉它们的代价是进程内自带了一个
|
||||
用户态 TCP/IP 栈，这笔账 [FEASIBILITY.md](FEASIBILITY.md) §2 算过。
|
||||
|
||||
**如果哪天某个改动让上面任何一行不得不放开，那个改动是错的。**
|
||||
|
||||
---
|
||||
|
||||
## 6. 日常运维
|
||||
|
||||
```sh
|
||||
# 换了凭据 / 想立刻刷新节点列表；不断开任何在途连接
|
||||
docker compose kill -s HUP openvpngate
|
||||
|
||||
# 手动换节点（被拒绝时返回具体原因，不是一句 false）
|
||||
curl -s -XPOST 127.0.0.1:9080/switch
|
||||
|
||||
# 为什么选了这个节点
|
||||
curl -s 127.0.0.1:9080/nodes | jq
|
||||
curl -s 127.0.0.1:9080/status | jq
|
||||
```
|
||||
|
||||
**退出语义。** `docker stop` 发 SIGTERM：停止 accept → 排空 → 拆隧道 → 退出，正常在一秒内
|
||||
完成。宽限期给到 30s 不是因为它慢，而是因为超时之后 Docker 发的是 SIGKILL，而那有可能落在
|
||||
写节点历史的中间。容器里手动再发一次 SIGTERM 会立即退出（第二次信号的语义就是「别排空了」）。
|
||||
|
||||
**healthcheck 是存活探针，不是就绪探针。** `/healthz` 只要管理服务在监听就回 200，它不表示
|
||||
隧道是通的；隧道状态在 `/status`。特别地：**不要**在它上面接「unhealthy 就重启容器」的
|
||||
监工。节点变差时服务自己会换节点，重启会把排空、在线会话、以及刚刚学到的「这个节点不行」
|
||||
一起扔掉——恰好是在它已经在正确处理问题的时候。
|
||||
|
||||
**日志**走 stderr，由 compose 的 json-file driver 收，配了 10 MB × 5 的轮转。一个繁忙的网关
|
||||
每条会话至少一行 info，不轮转就是在慢慢填满宿主机磁盘。
|
||||
|
||||
---
|
||||
|
||||
## 7. 卷与文件属主
|
||||
|
||||
状态目录用的是命名卷 `ovg-state`。空的命名卷会继承镜像里该目录的属主（Dockerfile 里
|
||||
`install -d -o ovg -g ovg`），所以开箱即用。
|
||||
|
||||
换成 bind mount 就没这个待遇，宿主机目录的属主说了算：
|
||||
|
||||
```sh
|
||||
mkdir -p ./state && sudo chown 10001:10001 ./state
|
||||
```
|
||||
|
||||
uid 写死成 10001 而不是让发行版随便分配，就是为了这条命令里能有一个提前知道的数字。
|
||||
|
||||
`read_only: true` 之下，容器里唯一可写的是这个卷和 `/tmp`（16 MB tmpfs）。目前没有已知的
|
||||
东西需要 `/tmp`，它在那里是为了让某个决定往 `/tmp` 写东西的库在写的时候就报错，而不是在
|
||||
连接的时候才莫名其妙失败。
|
||||
|
||||
---
|
||||
|
||||
## 8. 这份编排验证到了什么、没验证到什么
|
||||
|
||||
延续 README §6 的规矩：**没跑过就是没跑过。**
|
||||
|
||||
开发这套文件的沙箱里**没有 docker daemon**，所以 `docker build` 和 `docker compose up`
|
||||
**一次都没有真正执行过**。下面把「照着源码核对过的」和「没跑过的」分开列。
|
||||
|
||||
**已核对（对着本仓库源码或在本机实测）**
|
||||
|
||||
- 构建依赖与运行时共享库：`ldd` 实测二进制只依赖 `liblz4.so.1 / libfmt.so.10 /
|
||||
libssl.so.3 / libcrypto.so.3` 加 libc 三件套，对应的 trixie 包名逐个 `apt-cache policy`
|
||||
查过；
|
||||
- 产物路径 `build/src/openvpngate`、`build/src/ovg_tunnel_smoke`、`build/tests/ovg_tests`；
|
||||
- `OVG_OPENVPN3_DIR` / `OVG_LWIP_DIR` 确实能跳过 FetchContent（`cmake/Dependencies.cmake`），
|
||||
钉住的两个 ref 就是本机构建通过的那两个；
|
||||
- 配置键名、相对路径默认值、`auth_file` 缺失时的失败方式（`config: cannot open auth file`）、
|
||||
空凭据时的警告文案，全部来自 `src/common/config.cpp` 与 `src/app/main.cpp`；
|
||||
- CLI 参数与信号语义来自 `src/app/main.cpp` 的 usage 与 `App::begin_shutdown`；
|
||||
- **预哈希凭据格式是实测的**：用文档里那条 `openssl rand -hex 16` + `sha256sum` 生成一条
|
||||
`sha256$salt$hash`，起 direct 模式代理，正确口令拿到 200，错误口令和不存在的用户都被
|
||||
`auth rejected` 挡下；
|
||||
- 管理路由清单来自 `src/app/admin_server.cpp`；
|
||||
- `docker-compose.yml` 的 YAML 结构解析通过（挂载目标、profiles、tmpfs、healthcheck 覆盖）。
|
||||
|
||||
**没有验证**
|
||||
|
||||
- 镜像构建本身：apt 装包、两个 clone、CMake 配置与编译、构建阶段跑测试；
|
||||
- `HEALTHCHECK` 是否真的能命中 `/healthz`；
|
||||
- `read_only: true` 下有没有哪个库偷偷要写别处（`/tmp` 的 tmpfs 是按这个可能性预留的）；
|
||||
- `create_host_path: false` 的报错行为；
|
||||
- 容器里 `CapEff` 是否真是全零（按 Docker 语义应当如此，但没实跑）。
|
||||
|
||||
接手后按顺序跑一遍就能补齐：
|
||||
|
||||
```sh
|
||||
docker compose build # 构建 + 构建阶段的单元测试
|
||||
docker compose run --rm --entrypoint ovg_tunnel_smoke openvpngate # 数据面
|
||||
docker compose up -d && docker compose ps # 看 healthy
|
||||
curl -x socks5h://用户:口令@127.0.0.1:1080 https://ifconfig.me # 出口 IP 应是节点 IP
|
||||
docker compose exec openvpngate cat /proc/self/status | grep CapEff # 应为全零
|
||||
docker compose stop # 应在一秒内退出，不是等满 30s
|
||||
```
|
||||
|
||||
最后一条尤其值得看：本项目已经被「优雅退出日志打得漂漂亮亮然后永远不退出」这类 bug 咬过
|
||||
一次（README §5 末尾那段），`docker stop` 卡满宽限期然后被 SIGKILL，就是它在容器里的样子。
|
||||
Reference in New Issue
Block a user