418 lines
18 KiB
Markdown
418 lines
18 KiB
Markdown
# redapricot (红杏) wire protocol
|
||
|
||
redapricot is a central-hub P2P tunnel that speaks (an extension of) the
|
||
Minecraft Java Edition protocol. This document is the normative wire spec that
|
||
the Java **server** (the *hub*) and the Go **client** both implement. It is
|
||
self-contained: everything needed to write an interoperable implementation is
|
||
here.
|
||
|
||
There are three roles:
|
||
|
||
| Role | Language | Description |
|
||
|------------|----------|-------------|
|
||
| **Server** | Java | The central hub. Accepts every inbound TCP connection (players *and* clients) on one port. |
|
||
| **Client** | Go | Registers routing patterns with the hub and forwards player traffic to real destinations. |
|
||
| **Player** | any | An ordinary Minecraft client connecting through the hub. |
|
||
|
||
```
|
||
Player ──MC──▶ Hub(server) ══WorkerConn(mux)══▶ Client ──MC──▶ Destination
|
||
▲ registers patterns / receives control requests │
|
||
└────────────── Control Session ────────────────────┘
|
||
```
|
||
|
||
Security is intentionally lightweight: the goal is connectivity, not
|
||
confidentiality against a determined attacker. The single shared secret is the
|
||
**PSK** (pre-shared key), a UTF-8 passphrase configured on the hub and every
|
||
client.
|
||
|
||
---
|
||
|
||
## 1. Primitive data types
|
||
|
||
These follow the Minecraft protocol exactly.
|
||
|
||
| Type | Encoding |
|
||
|-----------|----------|
|
||
| `VarInt` | LEB128, 7 data bits per byte, high bit = continuation, little-endian groups, two's-complement, max 5 bytes. |
|
||
| `String` | `VarInt` byte-length of the UTF-8 encoding, followed by the UTF-8 bytes. |
|
||
| `U16` | unsigned 16-bit, **big-endian**. |
|
||
| `I64` | signed 64-bit, **big-endian**. |
|
||
| `Bytes[N]`| exactly N raw bytes, no length prefix. |
|
||
| `u8` | single unsigned byte. |
|
||
|
||
## 2. Minecraft packet framing (plaintext)
|
||
|
||
Every connection begins as an ordinary Minecraft connection. An uncompressed
|
||
Minecraft packet is:
|
||
|
||
```
|
||
[Length: VarInt][PacketID: VarInt][Data...] Length = len(PacketID)+len(Data)
|
||
```
|
||
|
||
redapricot **never** enables Minecraft compression on the hub link. Player
|
||
traffic that is compressed end-to-end (negotiated between the player and the
|
||
real destination) is irrelevant — the hub forwards raw bytes and never inspects
|
||
anything past the Handshake.
|
||
|
||
### 2.1 Handshake
|
||
|
||
The first packet on every connection is the Handshake (packet id `0x00`,
|
||
Handshaking state):
|
||
|
||
```
|
||
ProtocolVersion : VarInt
|
||
ServerAddress : String (≤ 255)
|
||
ServerPort : U16
|
||
Intent : VarInt
|
||
```
|
||
|
||
The hub reads exactly one Handshake packet and dispatches on `Intent`:
|
||
|
||
| Intent | Meaning |
|
||
|---------------|---------|
|
||
| `17` | redapricot session establishment (control session *or* worker conn). |
|
||
| `18` | Reserved for redapricot management/status. Never matched against patterns. The reference hub replies with a status line and closes. |
|
||
| anything else | **Player** connection. `ServerAddress` is matched against the registered **regex** PATTERNs (§5.1). |
|
||
|
||
For `Intent == 17` the hub additionally requires
|
||
`ServerAddress == lowercase_hex(SHA3-224(PSK))` — a 56-character hex string.
|
||
This is the first (cheap) proof that the peer knows the PSK. A mismatch closes
|
||
the connection.
|
||
|
||
For player connections the hub normalizes `ServerAddress` before matching:
|
||
lower-cased, and any trailing `.` or Forge/FML `\0`-suffix (`host\0FML\0`)
|
||
stripped to the bare hostname. The resulting hostname is then tested against the
|
||
registered regex patterns (§5.1).
|
||
|
||
## 3. Encryption
|
||
|
||
Immediately **after** the `Intent == 17` Handshake, the connection switches to
|
||
an encrypted, self-delimiting frame stream. redapricot uses **ChaCha20**
|
||
(RFC 8439, 32-bit block counter, 96-bit nonce) as a raw stream cipher applied to
|
||
frame payloads (no Poly1305 tag — padding/space overhead is minimized, matching
|
||
the design goal).
|
||
|
||
Each direction is an independent ChaCha20 keystream. Keys are derived from a
|
||
"phase key" `PK` (raw bytes) as:
|
||
|
||
```
|
||
keyC2S = SHA3-256(PK ‖ 0x01) # client → server
|
||
keyS2C = SHA3-256(PK ‖ 0x02) # server → client
|
||
nonce = 0x00 × 12 # both directions
|
||
counter starts at 0 # both directions
|
||
```
|
||
|
||
Using distinct keys per direction avoids a two-time-pad while keeping the nonce
|
||
trivially fixed. Each side keeps two ChaCha20 instances (one encrypt, one
|
||
decrypt) and feeds bytes through them incrementally; the keystream position is
|
||
maintained across writes.
|
||
|
||
There are two phases:
|
||
|
||
* **Phase A** — `PK = PSK` (the configured passphrase, UTF-8 bytes).
|
||
* **Phase B** — `PK = REKEY` (see §4), used for the remainder of the connection.
|
||
|
||
### 3.1 Encrypted frames
|
||
|
||
Once encryption is on, the connection speaks length-prefixed **frames**:
|
||
|
||
```
|
||
[Length: VarInt] # PLAINTEXT (not encrypted)
|
||
[Payload: Bytes[Length]] # ciphertext (ChaCha20)
|
||
```
|
||
|
||
Only the payload is encrypted; the `Length` prefix is sent in the clear. The
|
||
cipher is a continuous per-direction keystream: each frame's payload advances
|
||
the keystream by exactly `Length` bytes (the length prefix consumes no
|
||
keystream). This keeps framing trivial — a reader reads a plaintext VarInt, then
|
||
decrypts exactly that many following bytes as one unit — and lets the cipher
|
||
phase switch (§4) happen cleanly on a frame boundary without ever decrypting a
|
||
later frame's bytes with the wrong key. Max payload length is `1 MiB`; larger
|
||
closes the connection.
|
||
|
||
## 4. Session establishment (Intent 17)
|
||
|
||
The first frame is sent by the peer that opened the connection (client → server)
|
||
and is encrypted with **Phase A**. Its payload is the **Rekey** message:
|
||
|
||
```
|
||
Magic : u8 # 0x01 = control session, 0x02 = worker conn
|
||
RandLen : VarInt # 8 ≤ RandLen ≤ 64
|
||
Rand : Bytes[RandLen] # cryptographically random
|
||
Timestamp : I64 # client's epoch milliseconds
|
||
Flags : VarInt # feature flags; bit 0x01 (STREAM_FC) MUST be set
|
||
RecvWindow: VarInt # client's per-stream receive window, bytes (§7.3)
|
||
```
|
||
|
||
`Flags` is a bitfield of features. Bit `0x01` (STREAM_FC) declares
|
||
**per-stream flow control** (§7.3) and is mandatory: `RecvWindow` advertises
|
||
the client's per-stream receive window in bytes and must be positive. The hub
|
||
closes the connection if the flag is missing, `RecvWindow` is absent or
|
||
non-positive, or the fields are malformed.
|
||
|
||
The hub:
|
||
|
||
1. Decrypts frame 1 with Phase A.
|
||
2. Rejects (closes) if `|now − Timestamp| > timestampWindowMs` (default 30000),
|
||
or if `RandLen` is out of range.
|
||
3. Computes `REKEY = Rand ‖ Timestamp` (the 8 timestamp bytes big-endian
|
||
appended to Rand — the `Magic` byte is **not** included).
|
||
4. Switches **both** its ciphers to Phase B keys derived from `REKEY`.
|
||
|
||
The client, after sending frame 1 with Phase A, likewise switches both its
|
||
ciphers to Phase B. In practice **only frame 1 uses Phase A**; every later
|
||
frame (both directions) is Phase B, counters reset to 0.
|
||
|
||
The hub then sends one Phase-B frame to confirm success:
|
||
|
||
```
|
||
SessionReady : payload = [ 0x00, Flags: VarInt, RecvWindow: VarInt ]
|
||
```
|
||
|
||
The hub echoes the accepted flags (STREAM_FC set) followed by its own
|
||
per-stream receive window. A client must reject a SessionReady without the
|
||
STREAM_FC flag or without a positive window (an unsupported hub).
|
||
|
||
A hub that rejects the session simply closes the TCP connection (optionally
|
||
after a Phase-B `Error` frame, §6). After `SessionReady`:
|
||
|
||
* `Magic == 0x01` → the connection is a **Control Session** (§5).
|
||
* `Magic == 0x02` → the connection is a **Worker Conn** (§7).
|
||
|
||
## 5. Control session messages
|
||
|
||
After `SessionReady`, a control session exchanges **control messages**, one per
|
||
encrypted frame. Frame payload:
|
||
|
||
```
|
||
Type : u8
|
||
... : type-specific fields
|
||
```
|
||
|
||
| Type | Name | Direction | Fields |
|
||
|--------|----------------|-----------|--------|
|
||
| `0x00` | SessionReady | S → C | *(none)* — the confirmation frame from §4 |
|
||
| `0x01` | Register | C → S | `Pattern: String` |
|
||
| `0x02` | Unregister | C → S | `Pattern: String` |
|
||
| `0x03` | RegisterAck | S → C | `Pattern: String`, `Status: u8` (0 = ok, 1 = invalid pattern) |
|
||
| `0x04` | ControlRequest | S → C | `CID: Bytes[16]`, `Pattern: String`, `PlayerIP: String`, `PlayerPort: U16` |
|
||
| `0x05` | Ping | C → S | `Nonce: I64` |
|
||
| `0x06` | Pong | S → C | `Nonce: I64` |
|
||
|
||
* **Register / Unregister**: the client may (un)register a PATTERN at any time.
|
||
A PATTERN is a **regular expression** (§5.1) and is stored **verbatim** — the
|
||
exact string is the registry key (never normalized, so regex metacharacters are
|
||
preserved). Re-registering the identical pattern string reassigns it to the
|
||
newest session (last writer wins); `Unregister` only removes it if the
|
||
requesting session still owns it.
|
||
* **RegisterAck**: acknowledges a `Register`. `Status` is `0` on success, or `1`
|
||
if the pattern is not a valid regular expression (in which case nothing is
|
||
registered). The `Pattern` echoes the string that was registered.
|
||
* **ControlRequest**: emitted by the hub when a player Handshake matches a
|
||
PATTERN this session registered. `Pattern` is the **registered pattern string
|
||
that matched** (echoed verbatim), *not* the player's hostname — so the client
|
||
can look the pattern up in its own route table. `CID` is 16
|
||
cryptographically-random bytes generated by the hub, unique to that pending
|
||
player. `PlayerIP`/`PlayerPort` are the player's source address (used for
|
||
HAProxy v2).
|
||
* **Ping/Pong**: optional keepalive so idle control sessions survive NAT
|
||
timeouts. The client pings periodically; the hub echoes the nonce.
|
||
|
||
### 5.1 Pattern matching
|
||
|
||
A registered PATTERN is a **regular expression** (the reference hub uses
|
||
`java.util.regex`). Matching is:
|
||
|
||
* **Case-insensitive** — patterns are compiled with a case-insensitive flag, and
|
||
the player hostname is lower-cased during normalization (§2.1).
|
||
* **Whole-string (anchored)** — the pattern must match the *entire* normalized
|
||
hostname, as if wrapped in `^…$`. `mc\.example\.com` matches `mc.example.com`
|
||
but not `mc.example.com.evil` or `sub.mc.example.com`.
|
||
* **First match wins** — the hostname is tested against every registered pattern;
|
||
the first that matches routes the player. If several patterns overlap, which
|
||
one wins is unspecified.
|
||
|
||
Because the pattern is a regex, a literal dot must be escaped (`mc\.example\.com`);
|
||
an unescaped `.` is the regex "any character" wildcard. A pattern that fails to
|
||
compile is rejected at `Register` time with `RegisterAck` status `1`.
|
||
|
||
## 6. Error frame (any redapricot connection)
|
||
|
||
At any time either side may send, then close:
|
||
|
||
```
|
||
Type : u8 = 0x7F
|
||
Msg : String
|
||
```
|
||
|
||
Purely informational; the receiver logs it.
|
||
|
||
## 7. Worker conn & multiplexing
|
||
|
||
A **Worker Conn** (`Magic == 0x02`) carries player↔destination traffic for many
|
||
players over one TCP connection using a minimal stream multiplexer. The unit of
|
||
work is a **stream**. Stream ids are assigned by the **client** (the only side
|
||
that opens streams), unique per worker conn, starting at 1 and increasing.
|
||
|
||
Each encrypted frame on a worker conn carries one **mux frame**:
|
||
|
||
```
|
||
FrameType : u8
|
||
StreamID : VarInt
|
||
Data : Bytes[...] # remainder of the frame payload
|
||
```
|
||
|
||
| FrameType | Name | Direction | Data |
|
||
|-----------|------|-----------|------|
|
||
| `0x00` | SYN | C → S | `CID: Bytes[16]` — open a stream to take over the pending player identified by CID. |
|
||
| `0x01` | DATA | both | raw tunneled bytes for the stream. |
|
||
| `0x02` | FIN | both | *(empty)* — graceful close of the stream (both directions). This is the "disconnect" the hub sends when the player leaves. |
|
||
| `0x03` | RST | both | *(optional 1 byte reason)* — abnormal close (e.g. CID unknown/expired, destination dial failed). |
|
||
| `0x04` | WND | both | `Delta: VarInt` — flow-control credit grant (§7.3). |
|
||
|
||
There is no explicit SYN-ACK: success is implied by the hub forwarding the
|
||
buffered Handshake as the stream's first `DATA`; failure is an `RST`.
|
||
|
||
### 7.1 Stream allocation (client side)
|
||
|
||
The client keeps a pool of `1 ≤ N ≤ max_conn` worker conns (`max_conn`
|
||
configurable, `1..8`). To place a new stream:
|
||
|
||
1. Pick the worker conn with the **fewest active streams**.
|
||
2. If that minimum conn is **saturated** (active streams `> 8`) **and**
|
||
`poolSize < max_conn`, dial a new worker conn and use it instead.
|
||
3. Otherwise use the least-loaded conn (even if it exceeds 8 at `max_conn`).
|
||
|
||
### 7.2 End-to-end player flow
|
||
|
||
1. Player connects to the hub and sends a Handshake with a matching
|
||
`ServerAddress` and `Intent ∉ {17,18}`.
|
||
2. Hub normalizes the address, finds the registering control session, generates
|
||
`CID`, **pauses** the player socket, buffers everything read so far (the raw
|
||
Handshake plus any pipelined bytes), and sends `ControlRequest` on the
|
||
control session. If no SYN arrives within `pendingTimeoutMs` (default 10000)
|
||
the pending entry is dropped and the player socket closed.
|
||
3. The client receives `ControlRequest`, looks up the destination for `Pattern`,
|
||
allocates a worker conn + `StreamID`, and sends `SYN(StreamID, CID)`. In
|
||
parallel it dials the destination and (if configured) writes a HAProxy v2
|
||
header (§8) carrying `PlayerIP:PlayerPort`.
|
||
4. The hub matches `CID` to the pending player, binds
|
||
`(workerConn, StreamID) ↔ playerSocket`, forwards the buffered bytes as
|
||
`DATA`, and resumes the player socket. Subsequent player bytes become `DATA`
|
||
frames; `DATA` frames from the client are written to the player socket. If
|
||
`CID` is unknown/expired the hub replies `RST`.
|
||
5. When the player disconnects the hub sends `FIN` on the stream; the client
|
||
closes the destination. When the destination closes, the client sends `FIN`;
|
||
the hub closes the player socket. `RST` is treated the same way (hard close).
|
||
|
||
Data on a worker conn is subject to that TCP connection's back-pressure for
|
||
its **aggregate** bandwidth; *per-stream* fairness is governed by the credit
|
||
windows of §7.3.
|
||
|
||
### 7.3 Per-stream flow control
|
||
|
||
Every stream carries an independent credit window per direction:
|
||
|
||
* Each side advertised its **receive window** W (bytes) at session setup. A
|
||
sender may have at most W un-credited DATA bytes outstanding per stream; the
|
||
initial budget is W, spent as DATA is sent (`Data` length only — SYN/FIN/RST
|
||
frames are free) starting with the very first DATA on the stream (including
|
||
the hub's forwarded handshake).
|
||
* The receiver returns credit with `WND(Delta)` once bytes are **delivered to
|
||
the terminal socket** (written to the player / destination connection), not
|
||
when they are merely buffered. Receivers should batch grants (the reference
|
||
implementations send one `WND` per W/2 bytes consumed).
|
||
* A sender whose window is exhausted pauses reading **that stream's source
|
||
socket only**; the shared worker conn is never paused because of a single
|
||
stream. A receiver that observes more than W un-credited bytes on a stream
|
||
may reset it (`RST`) as a protocol violation.
|
||
* Senders should also cap individual DATA payloads (the reference
|
||
implementations use 32 KiB) so one stream cannot monopolize the link for a
|
||
full 1-MiB frame.
|
||
|
||
Both windows may differ (each side enforces the one its peer advertised).
|
||
`Delta` must be positive; a `WND` for an unknown stream id is ignored.
|
||
|
||
## 8. HAProxy protocol v2 (optional)
|
||
|
||
When a mapping has `proxyProtocol: true`, the client prepends a PROXY v2 header
|
||
to the destination connection *before* any tunneled bytes, so the real server
|
||
sees the player's true source address.
|
||
|
||
```
|
||
Signature : 0D 0A 0D 0A 00 0D 0A 51 55 49 54 0A (12 bytes)
|
||
VerCmd : 0x21 (v2, PROXY command)
|
||
FamProto : 0x11 (TCP/IPv4) | 0x21 (TCP/IPv6)
|
||
Len : U16 (length of the address block)
|
||
Addresses : IPv4 → srcAddr[4] dstAddr[4] srcPort[2] dstPort[2] (12 bytes)
|
||
IPv6 → srcAddr[16] dstAddr[16] srcPort[2] dstPort[2] (36 bytes)
|
||
```
|
||
|
||
`src` is the player; `dst` is the destination the client dialed. Ports are
|
||
big-endian.
|
||
|
||
## 9. Configuration
|
||
|
||
### 9.1 Hub (server) — JSON
|
||
|
||
```json
|
||
{
|
||
"listen": "0.0.0.0:25565",
|
||
"psk": "change-me",
|
||
"timestampWindowMs": 30000,
|
||
"pendingTimeoutMs": 10000,
|
||
"streamWindowBytes": 262144
|
||
}
|
||
```
|
||
|
||
`streamWindowBytes` (optional, default 262144, clamped to [32768, 8388608]) is
|
||
the hub's advertised per-stream receive window (§7.3).
|
||
|
||
### 9.2 Client — JSON
|
||
|
||
```json
|
||
{
|
||
"server": "127.0.0.1:25565",
|
||
"psk": "change-me",
|
||
"maxConn": 4,
|
||
"pingIntervalMs": 20000,
|
||
"streamWindowBytes": 262144,
|
||
"mappings": [
|
||
{ "pattern": "mc\\.example\\.com", "destination": "127.0.0.1:25566", "proxyProtocol": true }
|
||
]
|
||
}
|
||
```
|
||
|
||
`streamWindowBytes` (optional, default 262144, clamped to [32768, 8388608]) is
|
||
the client's advertised per-stream receive window (§7.3).
|
||
|
||
Each `pattern` is a regular expression (§5.1) matched against the whole
|
||
normalized player hostname, case-insensitively. Escape literal dots (`mc\.example\.com`,
|
||
which is `mc\\.example\\.com` in JSON); an unescaped `.` matches any character.
|
||
Use ordinary regex to route wildcards, e.g. `.*\.example\.com` for every
|
||
subdomain or `(alpha|beta)\.mc\.net` for a fixed set.
|
||
|
||
## 10. Constants summary
|
||
|
||
| Name | Value |
|
||
|------|-------|
|
||
| redapricot Handshake intent | `17` |
|
||
| reserved management intent | `18` |
|
||
| Handshake address for Intent 17 | `hex(SHA3-224(PSK))` |
|
||
| cipher | ChaCha20 (RFC 8439), 12-byte zero nonce, per-direction key, payload-only |
|
||
| frame length prefix | plaintext VarInt |
|
||
| key derivation | `SHA3-256(PK ‖ 0x01)` c→s, `SHA3-256(PK ‖ 0x02)` s→c |
|
||
| rekey material | `Rand ‖ Timestamp(I64 BE)` |
|
||
| Magic: control / worker | `0x01` / `0x02` |
|
||
| RegisterAck status: ok / invalid pattern | `0x00` / `0x01` |
|
||
| pattern matching | case-insensitive, whole-string regex; first match wins |
|
||
| CID length | 16 bytes |
|
||
| max frame payload | 1 MiB |
|
||
| saturation threshold | active streams `> 8` |
|
||
| max worker conns | `max_conn ∈ [1,8]` |
|
||
| feature flag: per-stream flow control | `0x01` |
|
||
| stream window default / bounds | 256 KiB, clamped to [32 KiB, 8 MiB] |
|
||
| WND grant batching (reference) | one grant per window/2 consumed |
|
||
| DATA chunk cap (reference) | 32 KiB |
|
||
```
|