icybear/newgrok
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Set up CraftBukkit-style patch workflow for xai-org/grok-build

Browse Source 
Upstream is a periodic one-way export of xAI's monorepo (linear
"Synced from monorepo" commits, SOURCE_REV pinning the internal SHA)
and explicitly refuses external contributions. So local changes can
never be upstreamed, and upstream re-drops the whole tree on every
sync -- structurally the same problem CraftBukkit has with Mojang.

Adopt the Spigot/BuildTools model: patches/ is the source of truth,
work/ is a disposable build artifact regenerated from upstream.rev
plus patches/.

  scripts/apply-patches.sh    ~ applyPatches.sh
  scripts/rebuild-patches.sh  ~ rebuildPatches.sh
  scripts/update-upstream.sh  forward-ports patches onto a new sync
  scripts/setup.sh            toolchain: rust 1.94.0, dotslash/protoc
  upstream.rev                ~ BuildTools versions/*.json pin

format-patch uses --zero-commit so rebases do not rewrite the From
line of every patch, and apply's git clean preserves work/target so
replaying patches does not cost a cold Rust rebuild.

Ships two [EXAMPLE PATCH] commits demonstrating a source edit and a
new-file addition; both are safe to delete.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
iceBear67
2026-08-14 04:50:07 +00:00
co-authored by Claude Opus 5
commit 618c8e31ee
12 changed files with 841 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+7
View File
@@ -0,0 +1,7 @@
# Derived trees -- rebuilt any time from upstream.rev + patches/.
/upstream/
/work/
# Editor / OS noise
*.swp
.DS_Store
Makefile
+71
View File
@@ -0,0 +1,71 @@
SHELL := /usr/bin/env bash
.DEFAULT_GOAL := help
S := ./scripts
.PHONY: help setup apply rebuild update update-dry build check test clippy fmt run release status clean distclean
help: ## Show this help
@echo "grok-build fork -- patch-based workflow"
@echo
@grep -hE '^[a-zA-Z_-]+:.*?## .*$$' $(MAKEFILE_LIST) \
| awk 'BEGIN{FS=":.*?## "}{printf " \033[36m%-12s\033[0m %s\n", $$1, $$2}'
@echo
@echo "Typical loop:"
@echo " make apply # work/ = upstream + patches/"
@echo " \$$EDITOR work/crates/.../foo.rs # edit"
@echo " cd work && git commit -am '...' # one commit == one patch"
@echo " make rebuild # write it back to patches/"
setup: ## Install the toolchain (rustup, dotslash/protoc)
@$(S)/setup.sh
apply: ## Rebuild work/ from upstream.rev + patches/ (FORCE=1 to discard local changes)
@FORCE=$(FORCE) $(S)/apply-patches.sh
rebuild: ## Export work/ commits back into patches/
@$(S)/rebuild-patches.sh
update: ## Pin the newest upstream sync and forward-port patches
@$(S)/update-upstream.sh
update-dry: ## Show what an update would pull in, changing nothing
@$(S)/update-upstream.sh --dry-run
build: ## Build the xai-grok-pager debug binary
@$(S)/build.sh
release: ## Build the optimised binary
@$(S)/build.sh --release
check: ## Fast type-check without codegen
@CARGO_CMD=check $(S)/build.sh
test: ## Run tests (PKG=<crate> to narrow, default: the version crate)
@CARGO_CMD=test PKG=$(or $(PKG),xai-grok-version) $(S)/build.sh
clippy: ## Lint
@CARGO_CMD=clippy $(S)/build.sh
fmt: ## Format the patched tree
@cd work && cargo fmt --all
run: ## Build and launch the TUI
@CARGO_CMD=run $(S)/build.sh
status: ## Show pin, patch count and work/ state
@printf 'upstream.rev : %s\n' "$$(grep -vE '^\s*(#|$$)' upstream.rev | head -n1)"
@printf 'patches : %s file(s)\n' "$$(ls -1 patches/*.patch 2>/dev/null | wc -l | tr -d ' ')"
@if [ -d work/.git ]; then \
printf 'work/ : %s commit(s) above base\n' "$$(git -C work rev-list --count base..HEAD 2>/dev/null || echo '?')"; \
git -C work log --oneline --no-decorate base..HEAD 2>/dev/null | sed 's/^/ /'; \
if [ -n "$$(git -C work status --porcelain)" ]; then echo ' (uncommitted changes present)'; fi; \
else \
printf 'work/ : not created yet -- run make apply\n'; \
fi
clean: ## Remove build output, keep the checkout
@rm -rf work/target && echo "removed work/target"
distclean: ## Remove work/ and upstream/ entirely (patches/ is untouched)
@rm -rf work upstream && echo "removed work/ and upstream/"
README.md
+188
View File
@@ -0,0 +1,188 @@
# newgrok — `xai-org/grok-build` 的 patch 化开发工作流
以 CraftBukkit / Spigot 的模型，在**不接受外部 PR 的上游**之上维护本地改动。
## 为什么是 patch，而不是 fork 后直接改
上游 [`xai-org/grok-build`](https://github.com/xai-org/grok-build) 有两个决定性特征：
1. 它是 xAI 内部 monorepo 的**周期性单向导出**。历史是一串线性的 `Synced from monorepo`
压扁提交，根目录 `SOURCE_REV` 记录对应的内部 commit。上游随时会把整棵树重新投放一次。
2. `CONTRIBUTING.md` 明确写着**不接受任何外部 PR 或补丁**。
也就是说：改动推不回去，而上游会不断整体覆盖。直接在 fork 上改，每次同步都要打一场
merge 混战，而且没人说得清「我们到底改了什么」。
这正是 CraftBukkit 面对 Mojang 的处境。解法也一样——**把改动本身作为版本库里的唯一真相**：
> `patches/` 是源码，`work/` 是编译产物。
## 目录结构
```
newgrok/ ← 你提交的仓库
├── patches/ ★ 唯一真相：0001-*.patch，一个补丁一件事
├── scripts/ 工作流脚本
├── upstream.rev 钉住的上游 commit
├── Makefile
├── upstream/ 派生：上游的纯净 clone （.gitignore）
└── work/ 派生：upstream@rev + patches（.gitignore）
```
对照 Spigot：
| 这里 | Spigot / BuildTools |
|------|---------------------|
| `upstream/` | CraftBukkit 的 clone（纯净上游） |
| `work/` | 打完补丁的工作目录 |
| `patches/` | `CraftBukkit-Patches/` |
| `scripts/apply-patches.sh` | `applyPatches.sh` |
| `scripts/rebuild-patches.sh` | `rebuildPatches.sh` |
| `upstream.rev` | `versions/*.json` 里的版本钉 |
`upstream/` 和 `work/` 都**不进 git**——它们任何时候都能从 `upstream.rev` + `patches/`
一字不差地重建。
## 快速开始
```sh
make setup # 装 Rust 工具链（1.94.0，由 rust-toolchain.toml 钉住）+ dotslash/protoc
make apply # 用 upstream.rev + patches/ 生成 work/
make build # 编译 work/target/debug/xai-grok-pager
make run # 直接启动 TUI
```
`make setup` 是幂等的，可以随时重跑。
> Rust 装在 `~/.cargo`，脚本会自己把它加进 `PATH`。若要在交互 shell 里直接用 `cargo`：
> fish 执行 `source ~/.cargo/env.fish`，bash 执行 `source ~/.cargo/env`。
## 日常循环
改代码永远在 `work/` 里，**一个提交 = 一个补丁**：
```sh
make apply # 拿到干净的、打好补丁的树
$EDITOR work/crates/codegen/.../foo.rs # 改
cd work && git add -A && git commit -m '...' # 提交（提交信息就是补丁标题）
cd .. && make rebuild # 写回 patches/
git add patches/ && git commit -m '...' # 提交到 fork 仓库
```
**`make rebuild` 是唯一的出口。** 没跑过它的改动，下次 `make apply` 就没了。
修改已有的补丁，用普通的 git 手段就行——`work/` 是个正常的 git 仓库：
```sh
cd work
git rebase -i base # 改写、合并、重排、删除任意补丁
cd .. && make rebuild # patches/ 会整体重新生成
```
随时看当前状态：
```sh
make status
```
## 跟进上游同步
```sh
make update-dry # 只看会拉进来什么，不动任何东西
make update # 重新钉 upstream.rev，并把补丁前移到新基线
```
`make update` 会：
1. `git fetch` 上游，列出 `upstream.rev..origin/main` 之间的新提交和 `SOURCE_REV` 变化
2. 把 `upstream.rev` 更新为最新
3. 在新基线上重放 `patches/`
4. 干净通过则自动 `make rebuild`，让补丁上下文对齐新代码
冲突时脚本会停下并给出解决步骤——和处理 rebase 冲突完全一样：
```sh
cd work
git status # 看冲突文件
$EDITOR <冲突文件> # 消掉冲突标记
git add -A
git am --continue # 或 git am --skip / git am --abort
cd .. && make rebuild # 把前移后的结果写回去
```
补丁数量多的时候，冲突是这个模型的固有成本，也是它的价值所在：冲突精确地指出上游改动
撞上了你的哪一处改动，而不是让它悄悄消失。
## 全部命令
| 命令 | 作用 |
|------|------|
| `make setup` | 安装工具链（rustup / dotslash / protoc） |
| `make apply` | `upstream.rev + patches/ → work/`；加 `FORCE=1` 丢弃 `work/` 里的未提交改动 |
| `make rebuild` | `work/` 中 `base` 之上的提交 → `patches/` |
| `make update` | 跟进最新上游同步并前移补丁 |
| `make update-dry` | 预览上游差异，不做修改 |
| `make build` / `make release` | 编译 debug / release 二进制 |
| `make check` / `make clippy` / `make fmt` | 快速类型检查 / lint / 格式化 |
| `make test` | 跑测试（`PKG=<crate>` 指定 crate） |
| `make run` | 编译并启动 TUI |
| `make status` | 显示当前钉住的 rev、补丁数、`work/` 状态 |
| `make clean` | 删 `work/target` |
| `make distclean` | 删 `work/` 和 `upstream/`（不动 `patches/`） |
`make build` 之后的额外参数会透传给 cargo，例如 `./scripts/build.sh --release --locked`。
## 安全网
脚本刻意不会静默吞掉工作成果：
- `work/` 有**未提交改动**时，`make apply` 拒绝执行（提示先 commit + rebuild，或 `FORCE=1`）
- `work/` 的提交数和 `patches/` 的文件数**对不上**时同样拒绝——这基本意味着你忘了 `make rebuild`
- `git am` 进行到一半时，`make rebuild` / `make build` 会拒绝执行，而不是产出半成品
- `make apply` 的 `git clean` 显式保留 `work/target`，否则每次打补丁都要付一次冷编译的代价
## 补丁格式
`rebuild` 用的是：
```
git format-patch --no-stat -N --zero-commit --full-index --no-signature
```
- `--zero-commit` —— 否则每次 rebase 后每个补丁的 `From <sha>` 行都会变，`git diff` 里全是噪音
- `--full-index` —— 给 `git am --3way` 留下按 blob 哈希回退的余地，上游漂移时更容易自动合上
## 构建依赖
`make setup` 会处理，这里说明它在做什么：
- **Rust 1.94.0** —— 由上游 `rust-toolchain.toml` 钉住，rustup 自动装
- **protoc** —— 只有 `xai-grok-tools-api` 需要（编译 `proto/grok-tools.proto`）。上游的解析顺序是
`$PROTOC` → 向上查找 `bin/protoc` → `PATH`。`bin/protoc` 是个
[DotSlash](https://dotslash-cli.com) wrapper，会按需下载 protoc 29.3；所以 `setup.sh`
装 dotslash，失败则回退到系统 `protobuf-compiler`
两个 git 依赖（`our-forks/async-openai`、`helix-editor/nucleo`）都是公开可访问的。
## 示例补丁
`patches/` 里预置了两个补丁，提交信息都标了 `[EXAMPLE PATCH]`：
1. **`0001`** 改上游 Rust 源码——给 `xai-grok-version` 加 `FORK_NAME` / `fork_tag()`，
并接进 pager 的版本输出，于是 `--version` 显示 `grok [newgrok] 1.0.3 (…)`。
标记插在 `grok ` 之后而不是追加到末尾，因为 `main.rs` 的
`version_output_writer_preserves_channel_aware_contract` 断言该行仍以 channel label
（或 `)`）结尾——**改上游代码时顺手确认它的测试仍然成立**，这个补丁本身就是个例子。
2. **`0002`** 新增文件——根目录 `FORK.md`。
不需要就直接删：
```sh
rm patches/0001-*.patch patches/0002-*.patch
make apply
```
## 许可
上游代码为 Apache-2.0（见 `work/LICENSE`）。`patches/` 里的改动同样按 Apache-2.0 分发。
本仓库是非官方 fork，与 xAI 无关。
patches/0001-Brand-fork-builds-in-version-output.patch
+66
View File
@@ -0,0 +1,66 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: iceBear67 <icebear67@sfclub.cc>
Date: Fri, 14 Aug 2026 04:47:50 +0000
Subject: [PATCH] Brand fork builds in --version output
Adds FORK_NAME/fork_tag() to xai-grok-version and splices the tag into
the pager's version line, so a locally built binary reads
grok [newgrok] 1.0.3 (abc1234)
and is never mistaken for an official xAI release.
The tag is placed immediately after "grok " rather than appended,
because main.rs's version_output_writer_preserves_channel_aware_contract
test asserts the line still ends with the channel label (or ")").
[EXAMPLE PATCH] Demonstrates patching upstream Rust source. Safe to drop:
delete the patch file and run 'make apply'.
diff --git a/crates/codegen/xai-grok-pager-bin/src/main.rs b/crates/codegen/xai-grok-pager-bin/src/main.rs
index ac1adc4fc7800e507742d4ae4bb7067edf2959f5..19238e835bb37d8d9670c70b0402d43caf248827 100644
--- a/crates/codegen/xai-grok-pager-bin/src/main.rs
+++ b/crates/codegen/xai-grok-pager-bin/src/main.rs
@@ -1788,7 +1788,8 @@ fn install_heap_profile_hooks() {
}
fn version_text(channel_label: &str) -> String {
format!(
- "grok {}\n",
+ "grok {} {}\n",
+ xai_grok_version::fork_tag(),
xai_grok_version::display_version_with_commit(env!("VERSION_WITH_COMMIT"), channel_label,)
)
}
diff --git a/crates/codegen/xai-grok-version/src/lib.rs b/crates/codegen/xai-grok-version/src/lib.rs
index 29fdfc1e64ea1df96f0ca3e1afe78c9ac37843bd..f56d96d34b828a0c10267fca407bc0b14e0c2a55 100644
--- a/crates/codegen/xai-grok-version/src/lib.rs
+++ b/crates/codegen/xai-grok-version/src/lib.rs
@@ -9,6 +9,15 @@ pub const VERSION: &str = match option_env!("GROK_VERSION") {
None => env!("CARGO_PKG_VERSION"),
};
+/// Name of this downstream fork. Builds from this tree are not official xAI
+/// releases, so every user-facing version line carries the tag below.
+pub const FORK_NAME: &str = "newgrok";
+
+/// Bracketed fork marker for version output, e.g. `"[newgrok]"`.
+pub fn fork_tag() -> String {
+ format!("[{FORK_NAME}]")
+}
+
/// [`TEST_VERSION_ENV`] override first, then [`VERSION`]. Trimmed so
/// non-semver-aware callers can pass the result straight into parsing.
pub fn installed() -> String {
@@ -72,4 +81,12 @@ mod tests {
assert_eq!(display_version(""), VERSION);
assert!(display_version(" [stable]").ends_with("[stable]"));
}
+
+ /// The fork marker must stay non-empty and bracketed -- the pager splices it
+ /// into `--version` output, whose format test asserts the surrounding shape.
+ #[test]
+ fn test_fork_tag_is_bracketed_fork_name() {
+ assert!(!FORK_NAME.is_empty());
+ assert_eq!(fork_tag(), format!("[{FORK_NAME}]"));
+ }
}
patches/0002-Add-FORK.md-identifying-this-tree-as-a-downstream-fo.patch
+37
View File
@@ -0,0 +1,37 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: iceBear67 <icebear67@sfclub.cc>
Date: Fri, 14 Aug 2026 04:48:16 +0000
Subject: [PATCH] Add FORK.md identifying this tree as a downstream fork
Upstream refuses external contributions, so anyone who lands in this
checkout needs to know it is patched and that work/ is disposable.
[EXAMPLE PATCH] Demonstrates a new-file patch. Safe to drop: delete the
patch file and run 'make apply'.
diff --git a/FORK.md b/FORK.md
new file mode 100644
index 0000000000000000000000000000000000000000..1afd0f0fbe91bc70c2e463373d361ae25b60f705
--- /dev/null
+++ b/FORK.md
@@ -0,0 +1,20 @@
+# newgrok — a downstream fork of xai-org/grok-build
+
+This tree is **not** an official xAI build. It is upstream `grok-build` with a
+set of local patches replayed on top of it.
+
+- Upstream: <https://github.com/xai-org/grok-build> (Apache-2.0)
+- The exact upstream commit this was built from is pinned in the fork repo's
+ `upstream.rev`; upstream's own `SOURCE_REV` records the monorepo commit.
+- Binaries built here identify themselves as `grok [newgrok] <version>`.
+
+## Do not edit this tree directly and expect it to survive
+
+`work/` is a derived artifact. It is deleted and regenerated from
+`upstream.rev` + `patches/` whenever patches are applied or upstream is
+updated. Changes only persist if you **commit** them here and then run
+`make rebuild`, which writes them back out to `patches/`.
+
+Upstream does not accept external contributions (see `CONTRIBUTING.md`), which
+is the whole reason this fork carries its changes as patches instead of as
+merged commits.
scripts/apply-patches.sh
Executable
+104
View File
@@ -0,0 +1,104 @@
#!/usr/bin/env bash
# Rebuild work/ from scratch: pristine upstream@upstream.rev + every patch in
# patches/, each replayed as its own commit.
#
# This is the CraftBukkit/Spigot `applyPatches.sh` step. work/ is a DERIVED
# artifact -- anything in it that is not a commit is discarded.
#
# usage: apply-patches.sh [-f|--force]
# -f discard uncommitted changes in work/ without asking
source "$(dirname "${BASH_SOURCE[0]}")/lib.sh"
FORCE="${FORCE:-0}"
case "${1:-}" in
-f|--force) FORCE=1 ;;
"") ;;
*) die "unknown argument: $1" ;;
esac
ensure_upstream
REV="$(read_rev)"
if [[ ! -d "$WORK_DIR/.git" ]]; then
info "Creating work/ from upstream/"
git clone --origin upstream --no-checkout "$UPSTREAM_DIR" "$WORK_DIR"
else
# Refuse to nuke work the developer has not saved anywhere. patches/ is the
# source of truth, so anything only present in work/ is about to be lost.
if [[ "$FORCE" != "1" ]]; then
assert_no_am_in_progress
if [[ -n "$(git -C "$WORK_DIR" status --porcelain)" ]]; then
die "work/ has uncommitted changes; they would be destroyed.
Commit them (cd work && git add -A && git commit) then 'make rebuild',
or re-run with: make apply FORCE=1"
fi
if git -C "$WORK_DIR" rev-parse -q --verify "refs/tags/$BASE_TAG" >/dev/null; then
have="$(git -C "$WORK_DIR" rev-list --count "$BASE_TAG..HEAD" 2>/dev/null || echo 0)"
want="$(count_patches)"
if [[ "$have" != "$want" ]]; then
die "work/ has $have commit(s) above $BASE_TAG but patches/ has $want patch file(s).
You probably forgot to run 'make rebuild' after committing.
To discard the work/ commits and replay patches/ as-is: make apply FORCE=1"
fi
fi
fi
step "Fetching into work/"
git -C "$WORK_DIR" remote set-url upstream "$UPSTREAM_DIR"
git -C "$WORK_DIR" fetch --tags --prune upstream
fi
cd "$WORK_DIR"
git cat-file -e "${REV}^{commit}" 2>/dev/null \
|| die "pinned revision $REV not found in upstream. Run 'make update' to re-pin."
# Abandon any half-finished am from a previous run before resetting.
gitdir="$(git rev-parse --git-dir)"
[[ -d "$gitdir/rebase-apply" ]] && git am --abort >/dev/null 2>&1 || true
info "Resetting work/ to upstream $(git rev-parse --short "$REV")"
git checkout -q -B "$WORK_BRANCH" "$REV"
git reset -q --hard "$REV"
# -x removes ignored files too, which is what we want for a pristine tree --
# EXCEPT target/, which holds the Rust incremental cache. Wiping it would turn
# every apply into a 30-minute cold rebuild.
git clean -qfdx -e /target
# The boundary between "upstream" and "ours". rebuild-patches.sh formats
# everything after this tag.
git tag -f "$BASE_TAG" HEAD >/dev/null
n="$(count_patches)"
if [[ "$n" -eq 0 ]]; then
info "No patches to apply -- work/ is pristine upstream."
exit 0
fi
info "Applying $n patch(es)"
if ! git am --3way --keep-cr --whitespace=nowarn "$PATCHES_DIR"/*.patch; then
failed="$(basename "$(cat "$(git rev-parse --git-dir)/rebase-apply/original-commit" 2>/dev/null || true)" 2>/dev/null || true)"
cat >&2 <<EOF
${C_RED}${C_BOLD}==> patch application failed${C_RESET}
This is normal after an upstream sync that touched the same code.
Resolve it the same way you would a rebase:
cd work
git status # see the conflicting files
\$EDITOR <conflicted files> # fix the conflict markers
git add -A
git am --continue # or: git am --skip / git am --abort
When 'git am' finishes cleanly, write the result back to patches/:
make rebuild
EOF
exit 1
fi
applied="$(git rev-list --count "$BASE_TAG..HEAD")"
info "work/ is ready: upstream $(git rev-parse --short "$BASE_TAG") + $applied patch(es)"
scripts/build.sh
Executable
+31
View File
@@ -0,0 +1,31 @@
#!/usr/bin/env bash
# Compile the patched tree. Any extra arguments are passed straight to cargo,
# e.g. ./scripts/build.sh --release
#
# Set CARGO_CMD to run something other than `build` (check, test, clippy, run).
source "$(dirname "${BASH_SOURCE[0]}")/lib.sh"
setup_path
assert_work_ready
require_cmd cargo "Run 'make setup' first."
CARGO_CMD="${CARGO_CMD:-build}"
PKG="${PKG:-$BIN_PKG}"
cd "$WORK_DIR"
# bin/protoc is a DotSlash wrapper; it needs dotslash on PATH to self-resolve.
# If that is unavailable, hand the build script a system protoc instead so it
# does not silently skip codegen.
if ! ./bin/protoc --version >/dev/null 2>&1; then
if command -v protoc >/dev/null 2>&1 && [[ -z "${PROTOC:-}" ]]; then
export PROTOC="$(command -v protoc)"
warn "bin/protoc unusable (dotslash missing?) -- using PROTOC=$PROTOC"
else
die "no working protoc. Run 'make setup'."
fi
fi
info "cargo $CARGO_CMD -p $PKG ${*:-}"
exec cargo "$CARGO_CMD" -p "$PKG" "$@"
scripts/lib.sh
Executable
+117
View File
@@ -0,0 +1,117 @@
#!/usr/bin/env bash
# Shared helpers for the patch workflow. Sourced by every script in this dir.
#
# Layout (see README.md):
# upstream/ pristine clone of the vendor repo -- never edited by hand
# work/ upstream@REV + patches/ applied -- where you actually edit
# patches/ the source of truth -- tracked in git
set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd -P)"
UPSTREAM_URL="${UPSTREAM_URL:-https://github.com/xai-org/grok-build.git}"
UPSTREAM_BRANCH="${UPSTREAM_BRANCH:-main}"
UPSTREAM_DIR="$ROOT/upstream"
WORK_DIR="$ROOT/work"
PATCHES_DIR="$ROOT/patches"
REV_FILE="$ROOT/upstream.rev"
# Branch created inside work/, and the tag marking the pristine upstream commit
# that patches are generated against. `base` is the boundary: everything after
# it is ours.
WORK_BRANCH="fork"
BASE_TAG="base"
# The package that produces the shipping binary.
BIN_PKG="xai-grok-pager-bin"
BIN_NAME="xai-grok-pager"
if [[ -t 1 ]]; then
C_RESET=$'\033[0m'; C_BOLD=$'\033[1m'; C_RED=$'\033[31m'
C_GREEN=$'\033[32m'; C_YELLOW=$'\033[33m'; C_BLUE=$'\033[34m'
else
C_RESET=""; C_BOLD=""; C_RED=""; C_GREEN=""; C_YELLOW=""; C_BLUE=""
fi
info() { printf '%s==>%s %s\n' "$C_BLUE$C_BOLD" "$C_RESET" "$*"; }
step() { printf '%s ->%s %s\n' "$C_GREEN" "$C_RESET" "$*"; }
warn() { printf '%s==> warning:%s %s\n' "$C_YELLOW$C_BOLD" "$C_RESET" "$*" >&2; }
die() { printf '%s==> error:%s %s\n' "$C_RED$C_BOLD" "$C_RESET" "$*" >&2; exit 1; }
# Rust lives in CARGO_HOME/bin, which is not on PATH by default in
# non-interactive shells. Every script that shells out to cargo needs this.
setup_path() {
export CARGO_HOME="${CARGO_HOME:-$HOME/.cargo}"
export RUSTUP_HOME="${RUSTUP_HOME:-$HOME/.rustup}"
case ":$PATH:" in
*":$CARGO_HOME/bin:"*) ;;
*) export PATH="$CARGO_HOME/bin:$PATH" ;;
esac
}
require_cmd() {
command -v "$1" >/dev/null 2>&1 \
|| die "'$1' not found on PATH.${2:+ $2}"
}
# The pinned upstream commit. Comments and blank lines are allowed in the file
# so the pin can carry a note about which sync it corresponds to.
read_rev() {
[[ -f "$REV_FILE" ]] || die "missing $REV_FILE -- run 'make update' to pin an upstream revision"
local rev
rev="$(grep -vE '^\s*(#|$)' "$REV_FILE" | head -n1 | tr -d '[:space:]')"
[[ -n "$rev" ]] || die "$REV_FILE contains no revision"
printf '%s' "$rev"
}
write_rev() {
local rev="$1"
{
echo "# Pinned upstream revision of $UPSTREAM_URL"
echo "# Bump with: make update (never edit by hand unless you know why)"
echo "$rev"
} > "$REV_FILE"
}
# Clone on first use, otherwise fetch. Deliberately NOT a shallow clone: forward
# -porting patches across upstream syncs needs real history for merge bases.
ensure_upstream() {
if [[ ! -d "$UPSTREAM_DIR/.git" ]]; then
info "Cloning upstream $UPSTREAM_URL -> upstream/"
git clone --branch "$UPSTREAM_BRANCH" "$UPSTREAM_URL" "$UPSTREAM_DIR"
else
step "Fetching upstream"
git -C "$UPSTREAM_DIR" fetch --tags --prune origin \
"+refs/heads/$UPSTREAM_BRANCH:refs/remotes/origin/$UPSTREAM_BRANCH"
fi
}
# Number of .patch files currently in patches/ (0 when the dir is empty).
count_patches() {
local n=0
shopt -s nullglob
local f
for f in "$PATCHES_DIR"/*.patch; do n=$((n + 1)); done
shopt -u nullglob
printf '%s' "$n"
}
# Guard against running rebuild/build while a patch application is half-done.
assert_no_am_in_progress() {
local gitdir
gitdir="$(git -C "$WORK_DIR" rev-parse --git-dir)"
if [[ -d "$WORK_DIR/$gitdir/rebase-apply" ]]; then
die "a 'git am' is still in progress in work/.
Resolve it first:
cd work && git status
# fix conflicts, then: git add -A && git am --continue
# or abandon it with: git am --abort"
fi
}
assert_work_ready() {
[[ -d "$WORK_DIR/.git" ]] || die "work/ does not exist yet -- run 'make apply' first"
assert_no_am_in_progress
git -C "$WORK_DIR" rev-parse -q --verify "refs/tags/$BASE_TAG" >/dev/null \
|| die "work/ has no '$BASE_TAG' tag -- it was not created by apply-patches.sh. Run 'make apply'."
}
scripts/rebuild-patches.sh
Executable
+49
View File
@@ -0,0 +1,49 @@
#!/usr/bin/env bash
# Export every commit in work/ above the `base` tag back into patches/.
#
# This is the CraftBukkit/Spigot `rebuildPatches.sh` step and the only way
# changes leave work/. Run it after every commit you make in work/.
source "$(dirname "${BASH_SOURCE[0]}")/lib.sh"
assert_work_ready
cd "$WORK_DIR"
if [[ -n "$(git status --porcelain)" ]]; then
warn "work/ has uncommitted changes -- they will NOT be exported."
warn "Commit them first: cd work && git add -A && git commit"
fi
n="$(git rev-list --count "$BASE_TAG..HEAD")"
info "Exporting $n commit(s) above $BASE_TAG -> patches/"
mkdir -p "$PATCHES_DIR"
# Clear first: dropping or reordering commits in work/ must not leave orphaned
# patch files behind, and format-patch only ever writes, never deletes.
rm -f "$PATCHES_DIR"/*.patch
if [[ "$n" -eq 0 ]]; then
info "No commits above $BASE_TAG -- patches/ is now empty."
exit 0
fi
# --zero-commit the "From <sha>" line would otherwise change on every single
# rebase, producing a diff in every patch file for no reason
# --full-index full blob hashes give `git am --3way` something to fall back
# on when context has drifted
# --no-signature drops the trailing "-- \n2.47.3" git version banner
# --no-stat / -N match Spigot's output shape (no diffstat, no "[PATCH n/m]")
git format-patch \
--quiet \
--no-stat \
-N \
--zero-commit \
--full-index \
--no-signature \
--output-directory "$PATCHES_DIR" \
"$BASE_TAG"
info "patches/ now holds $(count_patches) patch file(s):"
for f in "$PATCHES_DIR"/*.patch; do
printf ' %s\n' "$(basename "$f")"
done
scripts/setup.sh
Executable
+83
View File
@@ -0,0 +1,83 @@
#!/usr/bin/env bash
# One-time host setup: Rust toolchain + the protoc that proto codegen needs.
# Idempotent -- safe to re-run.
source "$(dirname "${BASH_SOURCE[0]}")/lib.sh"
setup_path
info "Checking Rust toolchain"
if ! command -v rustup >/dev/null 2>&1; then
step "rustup not found -- installing"
curl --proto '=https' --tlsv1.2 -fsSL https://sh.rustup.rs \
| sh -s -- -y --no-modify-path --profile default
setup_path
else
step "rustup $(rustup --version 2>/dev/null | head -n1)"
fi
# rust-toolchain.toml pins the exact version (and extra targets). Running rustup
# inside the tree makes it materialise that toolchain rather than the default.
TOOLCHAIN_SRC=""
for d in "$WORK_DIR" "$UPSTREAM_DIR"; do
[[ -f "$d/rust-toolchain.toml" ]] && { TOOLCHAIN_SRC="$d"; break; }
done
if [[ -n "$TOOLCHAIN_SRC" ]]; then
pinned="$(grep -E '^\s*channel' "$TOOLCHAIN_SRC/rust-toolchain.toml" | head -n1 | cut -d'"' -f2)"
step "Materialising pinned toolchain ${pinned:-from rust-toolchain.toml}"
(cd "$TOOLCHAIN_SRC" && rustup show >/dev/null)
step "$(cd "$TOOLCHAIN_SRC" && rustc --version)"
else
warn "no checkout yet -- pinned toolchain will be installed on first 'make apply'"
fi
# protoc: xai-grok-tools-api's build script compiles proto/grok-tools.proto.
# Upstream resolves it as $PROTOC -> ./bin/protoc (a DotSlash wrapper that
# downloads protoc 29.3) -> protoc on PATH. Satisfy path 2, fall back to 3.
info "Checking protoc"
if command -v dotslash >/dev/null 2>&1; then
step "dotslash $(dotslash --version 2>/dev/null | head -n1)"
else
step "dotslash not found -- installing (enables the hermetic bin/protoc)"
cargo install dotslash || warn "cargo install dotslash failed; will fall back to a system protoc"
setup_path
fi
probe_protoc() {
local d="$1"
[[ -x "$d/bin/protoc" ]] || return 1
(cd "$d" && ./bin/protoc --version) 2>/dev/null
}
resolved=""
for d in "$WORK_DIR" "$UPSTREAM_DIR"; do
if out="$(probe_protoc "$d")"; then
step "hermetic bin/protoc works: $out"
resolved=1
break
fi
done
if [[ -z "$resolved" ]]; then
if command -v protoc >/dev/null 2>&1; then
step "falling back to system protoc: $(protoc --version)"
elif [[ -d "$WORK_DIR" || -d "$UPSTREAM_DIR" ]]; then
step "no working protoc -- installing protobuf-compiler"
sudo apt-get update -qq && sudo apt-get install -y -qq protobuf-compiler \
|| die "could not provide a protoc. Install DotSlash (https://dotslash-cli.com) or protobuf-compiler manually."
step "system protoc: $(protoc --version)"
else
warn "no checkout yet -- protoc will be verified on first 'make apply'"
fi
fi
info "Setup complete."
cat <<EOF
Add Rust to your interactive shell if you have not already:
fish: source $CARGO_HOME/env.fish
bash: source $CARGO_HOME/env
Next:
make apply # build work/ from upstream + patches
make build # compile the $BIN_NAME binary
EOF
scripts/update-upstream.sh
Executable
+85
View File
@@ -0,0 +1,85 @@
#!/usr/bin/env bash
# Follow a new upstream sync: re-pin upstream.rev, replay patches/ on top of it,
# and write the forward-ported result back out.
#
# usage: update-upstream.sh [--dry-run] [--to <rev>]
source "$(dirname "${BASH_SOURCE[0]}")/lib.sh"
DRY_RUN=0
TARGET=""
while [[ $# -gt 0 ]]; do
case "$1" in
-n|--dry-run) DRY_RUN=1; shift ;;
--to) TARGET="${2:?--to needs a revision}"; shift 2 ;;
*) die "unknown argument: $1" ;;
esac
done
ensure_upstream
OLD="$(read_rev)"
if [[ -n "$TARGET" ]]; then
NEW="$(git -C "$UPSTREAM_DIR" rev-parse --verify "${TARGET}^{commit}")" \
|| die "revision '$TARGET' not found in upstream"
else
NEW="$(git -C "$UPSTREAM_DIR" rev-parse --verify "origin/$UPSTREAM_BRANCH")"
fi
if [[ "$OLD" == "$NEW" ]]; then
info "Already pinned to the newest upstream commit ($(git -C "$UPSTREAM_DIR" rev-parse --short "$NEW")). Nothing to do."
exit 0
fi
count="$(git -C "$UPSTREAM_DIR" rev-list --count "$OLD..$NEW" 2>/dev/null || echo '?')"
info "Upstream moved: $(git -C "$UPSTREAM_DIR" rev-parse --short "$OLD") -> $(git -C "$UPSTREAM_DIR" rev-parse --short "$NEW") ($count new commit(s))"
git -C "$UPSTREAM_DIR" log --oneline --no-decorate "$OLD..$NEW" | sed 's/^/ /'
# Each upstream commit is a squashed monorepo drop; SOURCE_REV names the
# internal commit it came from, which is the only human-meaningful version here.
old_src="$(git -C "$UPSTREAM_DIR" show "$OLD:SOURCE_REV" 2>/dev/null | tr -d '[:space:]' || true)"
new_src="$(git -C "$UPSTREAM_DIR" show "$NEW:SOURCE_REV" 2>/dev/null | tr -d '[:space:]' || true)"
[[ -n "$old_src$new_src" ]] && info "SOURCE_REV: ${old_src:-?} -> ${new_src:-?}"
changed="$(git -C "$UPSTREAM_DIR" diff --name-only "$OLD" "$NEW" | wc -l | tr -d ' ')"
info "$changed file(s) changed upstream"
if [[ "$DRY_RUN" == "1" ]]; then
info "--dry-run: upstream.rev left at $OLD"
exit 0
fi
info "Pinning upstream.rev -> $NEW"
write_rev "$NEW"
info "Replaying patches on the new base"
if ! "$ROOT/scripts/apply-patches.sh" --force; then
cat >&2 <<EOF
${C_YELLOW}${C_BOLD}==> upstream.rev has been bumped, but patches did not apply cleanly.${C_RESET}
Finish the 'git am' in work/ (instructions above), then run:
make rebuild
To abandon this update entirely and go back:
cd work && git am --abort
printf '%s\\n' "$OLD" > upstream.rev # or: git checkout upstream.rev
make apply
EOF
exit 1
fi
info "Patches replayed cleanly -- regenerating patches/ against the new base"
"$ROOT/scripts/rebuild-patches.sh"
cat <<EOF
${C_GREEN}${C_BOLD}==> Update complete.${C_RESET}
Review and commit the result:
git diff --stat upstream.rev patches/
make build
git add upstream.rev patches/ && git commit -m 'Update upstream to $(git -C "$UPSTREAM_DIR" rev-parse --short "$NEW")'
EOF
upstream.rev
+3
View File
@@ -0,0 +1,3 @@
# Pinned upstream revision of https://github.com/xai-org/grok-build.git
# Bump with: make update (never edit by hand unless you know why)
eb267feff13129e568df38fb6fdf0ceb65f735d6