iceBear67
4df2560331
Worker frames are now FrameType + payload; there is no stream id. Each player gets its own worker conn. maxTunnels (default 256) caps concurrent tunnels. The old maxConn pool size is ignored so existing configs do not silently admit only a handful of players. Resume, per-direction windows, the control session, and the DATA-only shaper stay. A dropped worker still hangs that one player and reattaches over a fresh conn. Add a hub-side per-IP limiter for player intents only (default 8/s, burst 16, 64 concurrent). Unmatched hostnames consume a token; Intent 17 is never counted. 0 disables each knob.
276 lines
16 KiB
Markdown
276 lines
16 KiB
Markdown
# redapricot (红杏)
|
||
|
||
A **central-hub P2P tunnel that speaks the Minecraft Java Edition protocol.**
|
||
redapricot lets a Minecraft server that lives behind NAT/CGNAT (a "client")
|
||
publish itself through a public **hub**, so that players connecting to the hub
|
||
are transparently forwarded to the hidden server — no port forwarding required.
|
||
|
||
It works by *extending* the Minecraft handshake: the hub listens on a single
|
||
port and tells players apart from tunnel endpoints by the handshake `Intent`
|
||
field, so a vanilla Minecraft client needs no modification.
|
||
|
||
> 红杏出墙 — "the red apricot reaches over the wall": a server behind a wall,
|
||
> made reachable from the outside.
|
||
|
||
```
|
||
Player ──MC──▶ Hub (Java) ══ Worker Conn (1:1) ══▶ Client (Go) ──MC──▶ Real MC server
|
||
vanilla client public IP one encrypted TCP behind NAT (localhost)
|
||
│ per player ▲
|
||
└────────────── Control Session ─────────┘
|
||
(pattern registration + control requests)
|
||
```
|
||
|
||
* **Hub / server** — Java 21+, [Vert.x](https://vertx.io). One public TCP port.
|
||
* **Client** — Go. Registers hostnames with the hub and forwards to real servers.
|
||
* **Player** — any Minecraft client. Connects to the hub using a registered
|
||
hostname; the connection lands on the hidden server.
|
||
|
||
The full wire protocol is specified in **[PROTOCOL.md](PROTOCOL.md)**; the
|
||
design rationale is in **[docs/architecture.md](docs/architecture.md)**.
|
||
|
||
---
|
||
|
||
## How it works (in one paragraph)
|
||
|
||
A client opens a **control session** to the hub: it sends a Minecraft handshake
|
||
with `Intent = 17` and a `Server Address` equal to `hex(SHA3-224(PSK))`, then
|
||
the link switches to ChaCha20-encrypted frames keyed by the shared **PSK**,
|
||
re-keyed to a per-connection secret. Over that session the client **registers**
|
||
one or more hostname patterns — each a **regular expression**. When a player
|
||
connects to the hub with a hostname that matches a registered pattern (and any
|
||
normal `Intent`), the hub assigns a random **CID**, buffers
|
||
the player's bytes, and asks the client (via the control session) to take over.
|
||
The client dials a fresh **worker connection** — one encrypted TCP link per
|
||
player — announces the CID with `SYN`, dials the real destination (optionally
|
||
announcing the player's real IP with the **HAProxy v2** protocol), and bridges
|
||
the two ends. `maxTunnels` is the only player cap; the retired `maxConn` field
|
||
is ignored so an old config does not silently admit only four players.
|
||
|
||
## Repository layout
|
||
|
||
```
|
||
PROTOCOL.md normative wire spec (read this to build another impl)
|
||
docs/architecture.md design, sequence diagrams, threading, limitations
|
||
server/ Java hub (Gradle, Vert.x)
|
||
src/main/java/io/icybear/redapricot/
|
||
client/ Go client library
|
||
wire/ VarInt/MC codec, SHA3+ChaCha20, encrypted framing
|
||
cmd/redapricot-client/ Go client binary
|
||
e2e/ end-to-end integration tests (Java hub + Go client)
|
||
scripts/build.sh build hub + client
|
||
scripts/e2e.sh build, then run all tests
|
||
```
|
||
|
||
## Prerequisites
|
||
|
||
* **JDK 21+** (the hub compiles at Java 21; it runs fine on newer JDKs).
|
||
* **Gradle 8.5+ / 9.x** (Gradle 9.2.1 is used here).
|
||
* **Go 1.24+** (needs the standard-library `crypto/sha3`; developed with Go 1.26).
|
||
|
||
If you use [SDKMAN!](https://sdkman.io), `scripts/build.sh` auto-discovers a JDK
|
||
at `~/.sdkman/candidates/java/current` and Gradle at
|
||
`~/.sdkman/candidates/gradle/current`. Otherwise set `JAVA_HOME` and make
|
||
`gradle` / `go` available on `PATH`.
|
||
|
||
## Build
|
||
|
||
```bash
|
||
./scripts/build.sh
|
||
```
|
||
|
||
This produces:
|
||
|
||
* the hub at `server/build/install/redapricot-server/bin/redapricot-server`
|
||
* the client binary at `bin/redapricot-client`
|
||
|
||
Alternatively, build a single self-contained **fat jar** for the hub (via the
|
||
[Shadow](https://gradleup.com/shadow/) plugin):
|
||
|
||
```bash
|
||
gradle -p server shadowJar # (with JAVA_HOME set)
|
||
java -jar server/build/libs/redapricot-server-0.1.0-all.jar hub.json
|
||
```
|
||
|
||
## Run
|
||
|
||
**1. Start the hub** (public machine). Copy and edit the example config:
|
||
|
||
```bash
|
||
cp server/config.example.json hub.json # set a strong "psk"
|
||
server/build/install/redapricot-server/bin/redapricot-server hub.json
|
||
```
|
||
|
||
On startup the hub logs its PSK handshake address, e.g.
|
||
`PSK handshake address: 90188f2d...` — this confirms the PSK the hub expects.
|
||
|
||
**2. Start the client** (machine next to the real Minecraft server). Edit the
|
||
example config so `psk` matches the hub, `server` points at the hub, and each
|
||
mapping routes a hostname to a real server:
|
||
|
||
```bash
|
||
cp client/config.example.json client.json
|
||
# {
|
||
# "server": "hub.example.com:25565",
|
||
# "psk": "same-as-the-hub",
|
||
# "maxTunnels": 256,
|
||
# "mappings": [
|
||
# { "pattern": "mc\\.example\\.com", "destination": "127.0.0.1:25566", "proxyProtocol": true }
|
||
# ]
|
||
# }
|
||
bin/redapricot-client client.json
|
||
```
|
||
|
||
**3. Connect a player.** Point a DNS record for `mc.example.com` at the hub (or
|
||
just add the hub's IP with that hostname), then join `mc.example.com` in
|
||
Minecraft. The hub matches the hostname against the registered regex patterns
|
||
and tunnels you to `127.0.0.1:25566` behind the client. With `proxyProtocol: true`, the real server sees your true IP
|
||
(enable `proxy-protocol` / a compatible front-end on that server to consume it).
|
||
For a Paper backend, setting `velocitySecret` instead is usually nicer: the
|
||
client answers the backend's Velocity modern-forwarding login query, so the
|
||
server sees your real IP, username and UUID without any front-end — configure
|
||
the backend with `proxies.velocity.enabled: true` and the same secret.
|
||
|
||
## Container image (client)
|
||
|
||
The Go client ships two ways to build an image.
|
||
|
||
**Dockerfile** (multi-stage, distroless static, build from the repo root):
|
||
|
||
```bash
|
||
docker build -t redapricot-client .
|
||
docker run --rm -v "$PWD/client.json:/etc/redapricot/client.json" \
|
||
redapricot-client /etc/redapricot/client.json
|
||
```
|
||
|
||
**[ko](https://ko.build)** (Dockerfile-less; used by CI) builds and pushes
|
||
straight from the Go package:
|
||
|
||
```bash
|
||
export KO_DOCKER_REPO=your-registry/namespace/redapricot-client
|
||
ko build ./cmd/redapricot-client --bare
|
||
```
|
||
|
||
CI publishes the image on version tags via
|
||
`.github/workflows/publish-client-image.yml`. **Before using it, edit the
|
||
`KO_DOCKER_REPO` placeholder** at the top of that file to your registry, and (for
|
||
a non-ghcr.io registry) set the `REGISTRY_USERNAME` / `REGISTRY_PASSWORD` repo
|
||
secrets. The base image and build flags live in `.ko.yaml`.
|
||
|
||
## Configuration reference
|
||
|
||
### Hub (`server/config.example.json`)
|
||
|
||
| Key | Default | Meaning |
|
||
|---------------------|------------------|---------|
|
||
| `listen` | `0.0.0.0:25565` | Host:port the hub accepts all connections on. |
|
||
| `psk` | *(required)* | Shared secret; must match every client. |
|
||
| `timestampWindowMs` | `30000` | Allowed clock skew for a client's rekey timestamp. |
|
||
| `pendingTimeoutMs` | `10000` | How long a matched player waits for a worker to take over. |
|
||
| `sessionIdleTimeoutMs` | `90000` | Close an established control/worker session that receives no frame for this long. Must exceed the client's `pingIntervalMs`; `0` disables. Player connections are unaffected. |
|
||
| `streamWindowBytes` | `262144` | Advertised per-stream receive window, clamped to [32 KiB, 8 MiB]. |
|
||
| `streamResume` | `true` | Hang a player when its worker connection drops, so the client can reattach the stream instead of the player being disconnected. `false` restores the previous behaviour exactly and retains nothing. |
|
||
| `resumeGraceMs` | `20000` | How long a hung player is held. Advertised to clients, which clamp their own retry budget below it. Must exceed the client's grace by at least one dial. |
|
||
| `maxParkedStreams` | `256` | Cap on simultaneously hung players; `maxParkedBytes` (default `maxParkedStreams × 2 × streamWindowBytes`) caps what they retain. Past either, the oldest are dropped. |
|
||
| `statsIntervalMs` | `0` (off) | Log a periodic line with live/hung stream counts, retained bytes and pattern count. |
|
||
| `registrationGraceMs` | `15000` | Keep a closed control session's routes as *orphaned* for this long, holding players that arrive on them instead of refusing them, and replaying their requests once the client re-registers. `0` disables it. |
|
||
| `playerRatePerSec` | `8` | Per-IP token-bucket rate for **player** connections only (Intent ∉ {17, 18}). Unmatched hostnames still consume a token. `0` disables. |
|
||
| `playerBurst` | `16` | Token-bucket depth for `playerRatePerSec`. |
|
||
| `maxPlayersPerIp` | `64` | Concurrent player sockets (pending + live + parked) from one IP. `0` disables. Intent 17 is never counted. |
|
||
|
||
### Client (`client/config.example.json`)
|
||
|
||
| Key | Default | Meaning |
|
||
|------------------|--------------------|---------|
|
||
| `server` | *(required)* | Hub `host:port`. |
|
||
| `psk` | *(required)* | Shared secret; must match the hub. |
|
||
| `maxTunnels` | `256` (clamped 1–4096) | Max concurrent player tunnels. The retired `maxConn` field is ignored. |
|
||
| `pingIntervalMs` | `20000` (min 1000) | Heartbeat interval for the control session and every worker conn. A session with no reply for `3×` this is dropped and re-established. |
|
||
| `maxBandwidth` | *(unlimited)* | Caps what the client uploads to the hub, summed over every player — the direction carrying the game server's output, and the one a home uplink runs out of first. `"20mbps"`, `"512kbps"`, `"2MB/s"`, or a bare number of bytes/sec. **Bit units are decimal (`20mbps` = 20,000,000 bit/s); byte units are binary (`2MB/s` = 2 MiB/s).** Set it slightly below your real upload speed — framing and TCP/IP overhead are not counted. The budget is shared fairly across players, so one person loading chunks cannot time the others out. |
|
||
| `streamWindowBytes` | `262144` | Advertised per-stream receive window, clamped to [32 KiB, 8 MiB]. |
|
||
| `streamResume` | `true` | Reattach streams over a fresh connection when a worker connection drops, instead of disconnecting those players. `false` restores the previous behaviour exactly: nothing is retained and the send path is unchanged. |
|
||
| `resumeGraceMs` | `15000` (min 2000) | How long a stream keeps trying to reattach, clamped below the hub's advertised grace. Sized against the *backend*: a hung player stops answering the game server's KeepAlive, and vanilla disconnects a silent client at 30s, so a longer grace only resumes sessions the backend then kicks. |
|
||
| `statsIntervalMs` | `0` (off) | Log a periodic diagnostics line, plus a summary per stream at close: bytes each way, how long the stream was blocked on the flow-control window versus the bandwidth cap, receive-queue high-water mark, and heartbeat round-trip time per connection. Those distinguish a slow backend from a saturated uplink from a bad path, which throughput alone cannot. |
|
||
| `mappings[]` | *(≥1 required)* | Route table (below). |
|
||
| `mappings[].pattern` | — | Regex matched against the whole player hostname, case-insensitively. Escape dots (`mc\.example\.com`); `.` is a wildcard. |
|
||
| `mappings[].destination` | — | Real server `host:port` to forward to. |
|
||
| `mappings[].proxyProtocol` | `false` | Prepend a HAProxy v2 header carrying the player's IP. |
|
||
| `mappings[].velocitySecret` | *(off)* | Answer the destination's [Velocity modern forwarding](https://docs.papermc.io/velocity/player-information-forwarding/) login query with this secret, forwarding the player's real IP, username and UUID. Match it to the backend's `proxies.velocity.secret` (Paper). The forwarded profile carries no skin properties — the tunnel performs no Mojang authentication. |
|
||
|
||
## Testing
|
||
|
||
```bash
|
||
./scripts/e2e.sh # build, Go unit tests, then the full e2e suite
|
||
```
|
||
|
||
Or run pieces directly:
|
||
|
||
```bash
|
||
# Go unit tests (codec, crypto, encrypted framing)
|
||
go test ./client/... -v
|
||
|
||
# Java unit tests (VarInt/codec, SHA3-224 vector, key derivation, normalization)
|
||
JAVA_HOME=$HOME/.sdkman/candidates/java/current \
|
||
gradle -p server test
|
||
|
||
# End-to-end (spawns the real Java hub + in-process Go client + a mock destination)
|
||
go test ./e2e/... -v
|
||
```
|
||
|
||
The e2e suite covers: a full player round-trip with verbatim handshake
|
||
forwarding and case-insensitive matching, regex wildcard pattern routing,
|
||
multi-megabyte transfers, concurrent players each on their own worker
|
||
connection, HAProxy v2 source-address
|
||
propagation, Velocity modern-forwarding interception (signed player-info
|
||
handoff to a mock Paper backend), player- and destination-initiated disconnect
|
||
propagation, wrong-PSK rejection, dropping of unmatched hostnames, isolation
|
||
under a slow player and under a slow destination (no head-of-line blocking),
|
||
rejection of pre-flow-control peers, and stream resumption — a tunnel
|
||
hard-reset mid-transfer with the player connection held open, asserting the
|
||
byte stream neither gains nor loses a byte, across concurrent players, plus
|
||
grace expiry and the resume-disabled path, and control-outage handling — a
|
||
player arriving while the client's control session is down is held and then
|
||
served once it re-registers, with the grace-disabled and grace-expired paths
|
||
pinned too, and the per-IP player limiter — burst overflow and
|
||
`maxPlayersPerIp` drop extras before they become pending, Intent 17 is never
|
||
counted, and both knobs at `0` restore the unlimited path. The Go and Java
|
||
crypto layers are independently pinned to the same SHA3-224 test vector so they
|
||
cannot silently drift apart.
|
||
|
||
## Design notes & limitations
|
||
|
||
* **Security is deliberately light.** The PSK proves membership; traffic is
|
||
ChaCha20-encrypted (no AEAD tag) to minimize overhead. This protects against
|
||
casual sniffing, not a determined active attacker (see the note at the top of
|
||
[PROTOCOL.md](PROTOCOL.md) and [docs/architecture.md](docs/architecture.md) §8).
|
||
* **Per-tunnel flow control.** Each worker connection has credit-based windows
|
||
in both directions (windows exchanged at session setup, default 256 KiB), so
|
||
a slow player or slow destination jams only its own tunnel at a bounded
|
||
buffer. There is no application-level head-of-line blocking: each player
|
||
owns a TCP connection.
|
||
* **Liveness is explicit.** Every session heartbeats, every socket write is
|
||
bounded, and session establishment has a deadline. A path that dies silently —
|
||
no `FIN`, no `RST`, as when a NAT or firewall forgets an established flow — is
|
||
detected within `3 × pingIntervalMs`, the dead connection is dropped, and
|
||
service is restored without operator action. TCP keepalive is on as a
|
||
second line of defence.
|
||
* **A control-session reconnect no longer refuses new players.** While a client
|
||
is reconnecting the hub has no route for it, so arriving players used to be
|
||
told there is no such server. Those routes are now held briefly and the
|
||
players with them, then served once the client re-registers.
|
||
* **A dropped tunnel no longer drops the players.** A worker connection is only
|
||
the middle leg of the player it carries; when it dies both terminal sockets
|
||
are usually still healthy. The hub now hangs that player while the client
|
||
reattaches the tunnel over a fresh connection, replaying byte-exactly from
|
||
the offset the peer reports, so a conntrack expiry costs a stall rather than
|
||
a disconnect. Negotiated, and `streamResume: false` on either side restores
|
||
the old behaviour.
|
||
* **Single hub event loop.** The hub deploys one Vert.x verticle, so all state
|
||
is confined to one event loop (no locking). Throughput is bounded by one core;
|
||
ample for hundreds of players, not designed for tens of thousands.
|
||
|
||
## Attribution
|
||
|
||
The Minecraft protocol reference bundled as `CURRENT_MC_PROTO.txt` is derived
|
||
from the [Minecraft Wiki](https://minecraft.wiki/w/Java_Edition_protocol/Packets)
|
||
and is licensed under [CC BY-SA 3.0](https://creativecommons.org/licenses/by-sa/3.0/).
|