diff --git a/.agents/docs/2026-08-05-add-grpc-plan.md b/.agents/docs/2026-08-05-add-grpc-plan.md new file mode 100644 index 0000000..8c2865d --- /dev/null +++ b/.agents/docs/2026-08-05-add-grpc-plan.md @@ -0,0 +1,99 @@ +# 新增 mcpplibs.grpc 1.83.0(2026-08-05) + +[gRPC 收录可行性分析](2026-08-04-grpc-feasibility-analysis.md) 的终点。索引侧此前已备齐依赖 +([abseil+protobuf](2026-08-04-add-abseil-and-protobuf-plan.md)、 +[re2+upb](2026-08-04-add-re2-and-protobuf-upb-plan.md)、 +[c-ares](2026-08-04-add-c-ares-plan.md)),本次加入 gRPC 本体。 + +产出:`pkgs/g/grpc.lua`(Form A,指向 [mcpplibs/grpc-m](https://github.com/mcpplibs/grpc-m) +的 release)+ workspace 成员 `tests/examples/grpc-module`。 + +## 1. 为什么是独立仓库 + +这是本次唯一一个不能用 compat 描述符表达的库,理由只有一条但很硬:**gRPC 不发布任何自包含的 +源码产物**。v1.83.0 完全没有 release asset,而其 tag 归档里 abseil / protobuf / re2 / +boringssl / zlib 全是**空的 submodule 占位**(每个只有一个目录条目),`url` + `sha256` 无处可指。 + +`grpc-m` 的 release tarball 就是那个缺失的产物:上游 `src/` 与 `include/` **零补丁** vendor, +外加 gRPC 自己真正带内容的两块 third_party(`address_sorting`、`xxhash`)。 + +**而它不 vendor 的东西才是它属于本索引的理由**:abseil、protobuf(+upb)、re2、c-ares、 +OpenSSL、zlib 全部取自本索引的包。于是一个同时直接使用 protobuf 或 abseil 的消费者链进去的是 +**同一份**,而不是和第二份 vendored 副本撞车。 + +## 2. 构建形态 + +无 CMake、无 Bazel、无 configure —— 这不是取巧,而是核实过的事实:gRPC 源码树里**没有任何 +`.h.in` 或 `config.h.cmake`**,且其 upb 生成码上游已 check-in,所以 mcpp 只需要 include 路径。 +1001 个 TU 全部由解析出的工具链编译,**没有任何外部构建系统参与**,因此不存在 +`compat.openssl` 那种"外部 `c++` 编出的产物与 mcpp 链接侧 C++ ABI 不一致"的风险 +(那正是可行性分析里否掉 `install()` + CMake 路线的原因)。 + +源码清单是**上游自己的**:`add_library(gpr)` + `add_library(grpc)` + `add_library(grpc++)` + +`add_library(address_sorting)` 之并(995 条,c-ares 的 7 条另计)。`grpc-m` 的 +`tools/gen_sources.py` 能从上游 checkout 重新生成它,`--check` 在该仓 CI 里运行,证明 manifest +与 vendored 源码树没有漂移。 + +刻意排除一个文件:`src/core/ext/upb-gen/google/protobuf/descriptor.upb_minitable.c` —— +与 `compat.protobuf` 的 `upb` feature 已编译的 bootstrap 版本**逐字节完全相同**,两份同编会重复符号。 + +## 3. `import grpc;` 与顺序约束 + +`grpc-m` 提供真实的 C++23 模块层(`src/grpc.cppm`),它同时也充当 mcpp `kind = "lib"` 约定要求的 +lib root。导出面只取**公开 API**:`namespace grpc` 里还有约 280 个实现细节名字 +(`grpc::internal`、`CallOp*` 机器),整表扫描会把它们一起导出。每个名字都由编译器验证 —— +`export using ::grpc::X;` 在 X 不存在时直接编译失败 —— 这已经当场抓出三个并不存在的名字 +(`GrpcLibraryCodegen`、`DisableDefaultHealthCheckService`,以及补 include 之前的 +`ServerReaderWriter`)。 + +**`import grpc;` 必须放在该 TU 所有 textual `#include` 之后**,标准库头也算。全局模块片段把大半个 +标准库带进了 BMI,import 之后再 textual include 会让同一批声明到达两次。两种顺序都实测过 +(gcc 16.1.0): + +- import 在前 → `redefinition of std::__is_constant_evaluated` 加一大片 `` 冲突 +- 仅有 std 头跟在 import 之后 → `std::string` 上的 `ambiguous overload for operator==` +- **include 全部在前、import 在最后 → 正常**,因为导出实体属于**全局模块**,textual 视图与 + import 视图是同一批实体 + +这不是本包发明的约束:gRPC 的代码生成器产出的是**头文件**,任何真实程序都会同时用到两个世界。 + +## 4. `ares` feature 的方向是被迫的 + +c-ares **默认开启**,与上游 gRPC 及各发行版一致;`default-features = false` 关闭。 + +方向不能反过来:mcpp 的 feature 是**只增不减**的,若把 `-DGRPC_ARES=0` 写进基础 flag,再让 +feature 翻成 1,两个 `-DGRPC_ARES` 会同时出现在命令行。因此 manifest 对 `GRPC_ARES` **只字不提** +(`port_platform.h` 用 `#ifndef` 兜底成 1,所以"沉默"就是开启态),关闭态由 `build.mcpp` 在 +feature 缺席时发 `GRPC_ARES=0`;而那 7 个解析器 TU 走相反方向 —— 声明式地放在 +`[features.ares].sources` 里。 + +## 5. 验证结论 + +`grpc-m` 侧(与本索引 CI 同款配置:mcpp 2026.8.3.3、gcc@16.1.0、`MCPP_BUILD_CACHE=local`, +且**从已发布索引解析、无任何本地重定向**),1412 个目标文件入链: + +``` +mcpp test -> grpc::Version() = 1.83.0 ; module: OK +examples/helloworld -> server listening on 127.0.0.1:34733 + Greeter replied: Hello mcpp + Greeter rejected the empty name: name must not be empty +``` + +helloworld 刻意做成**单进程**:在真实 loopback 端口上起真实服务端,经真实 HTTP/2 通道发起真实 +unary RPC,所以 `mcpp run` 用退出码回答"gRPC 到底能不能用"。**错误路径也走线** —— 空 name 必须 +从服务端回 `INVALID_ARGUMENT` —— 因此一个"什么都答 OK"的桩通不过。 + +索引侧成员 `tests/examples/grpc-module` 全程只用 `import grpc;`,不含任何 protoc 产物: +gRPC 的 codegen 需要 mcpp 无法交付给消费者的宿主工具,所以生成物那条路径由 grpc-m 自己的 +example 覆盖,索引成员验证模块面。 + +## 6. 遗留:codegen 工具的分发 + +`protoc` 与 `grpc_cpp_plugin` 目前仍需用户自备(`grpc-m` 的模板与 example 把生成产物**签入**, +所以开箱即用)。根本原因是 mcpp 没有把依赖的构建产物交给消费者的机制:`mcpp::dep_dir()` 给的是 +依赖的**源码目录**,而 `kind = "bin"` 的 target 不对消费者暴露。 + +有利条件已核实:protobuf 上游发布**全平台预编译 protoc**,所以缺口只有 `grpc_cpp_plugin` +(本次为生成 helloworld 的 stub,已用 vendored 源码在本地编出过它,157 TU 的 libprotoc + 3 个 +grpc compiler TU)。后续可做成 `xim:grpc-tools` 预编译包,消费者以 `[xlings] deps` 声明, +再由 `build.mcpp` 调用 —— 这条链路 mcpp 已经具备,只差把工具打包。 diff --git a/README.md b/README.md index f6f1b6e..0e65ff2 100644 --- a/README.md +++ b/README.md @@ -37,7 +37,7 @@ Two kinds of packages live here: | Shape | Examples | |------|------| -| Native module library (Form A) | [`mcpplibs.xpkg`](pkgs/x/xpkg.lua) · [`mcpplibs.tinyhttps`](pkgs/t/tinyhttps.lua) · [`tensorvia-cpu`](pkgs/t/tensorvia-cpu.lua) · [`ffmpeg`](pkgs/f/ffmpeg.lua) (module layer; sources compiled directly through `compat.ffmpeg`) · [`opencv`](pkgs/o/opencv.lua) (single repository: the module layer and the full OpenCV 5 source build both live in the package, and only this descriptor stays on the index side) | +| Native module library (Form A) | [`mcpplibs.xpkg`](pkgs/x/xpkg.lua) · [`mcpplibs.tinyhttps`](pkgs/t/tinyhttps.lua) · [`tensorvia-cpu`](pkgs/t/tensorvia-cpu.lua) · [`ffmpeg`](pkgs/f/ffmpeg.lua) (module layer; sources compiled directly through `compat.ffmpeg`) · [`opencv`](pkgs/o/opencv.lua) (single repository: the module layer and the full OpenCV 5 source build both live in the package, and only this descriptor stays on the index side) · [`mcpplibs.grpc`](pkgs/g/grpc.lua) (gRPC 1.83.0 — the one library here that CANNOT be a compat descriptor: upstream publishes no self-contained source artifact, its tag archive carrying abseil/protobuf/re2/boringssl/zlib as empty submodule placeholders, so [grpc-m](https://github.com/mcpplibs/grpc-m)'s release tarball IS that artifact. It vendors only gRPC's own source and takes the five dependencies from this index, so a consumer that also uses protobuf links one copy rather than two) | | C-source compat (with `features`) | [`compat.cjson`](pkgs/c/compat.cjson.lua) · [`compat.zlib`](pkgs/c/compat.zlib.lua) | | C++-source compat, one depending on the other | [`compat.abseil`](pkgs/c/compat.abseil.lua) (151 TUs; a wildcard over `absl/**` trimmed by upstream's test/benchmark naming conventions) · [`compat.protobuf`](pkgs/c/compat.protobuf.lua) (the libprotobuf runtime, 79 TUs transcribed from upstream's own `src/file_lists.cmake`; declares `compat.abseil` as a dependency because protobuf's public headers include `absl/…`, and its `gzip` feature defines `HAVE_ZLIB` and pulls `compat.zlib`, while `upb` adds protobuf's 64-TU C runtime out of the same tarball) · [`compat.re2`](pkgs/c/compat.re2.lua) (22 TUs, upstream's own `RE2_SOURCES`) | | header-only (with `features`) | [`compat.eigen`](pkgs/c/compat.eigen.lua) | diff --git a/README.zh-CN.md b/README.zh-CN.md index bd6b5ce..d80c213 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -34,7 +34,7 @@ mcpp self config --mirror CN # 切换至国内镜像,默认使用 GLOBAL 上 | 形态 | 示例 | |------|------| -| 原生模块库(Form A) | [`mcpplibs.xpkg`](pkgs/x/xpkg.lua) · [`mcpplibs.tinyhttps`](pkgs/t/tinyhttps.lua) · [`tensorvia-cpu`](pkgs/t/tensorvia-cpu.lua) · [`ffmpeg`](pkgs/f/ffmpeg.lua)(模块层,源码经 `compat.ffmpeg` 直编) · [`opencv`](pkgs/o/opencv.lua)(单仓库:模块层与 OpenCV 5 全源码构建同在包内,索引侧只留本描述符) | +| 原生模块库(Form A) | [`mcpplibs.xpkg`](pkgs/x/xpkg.lua) · [`mcpplibs.tinyhttps`](pkgs/t/tinyhttps.lua) · [`tensorvia-cpu`](pkgs/t/tensorvia-cpu.lua) · [`ffmpeg`](pkgs/f/ffmpeg.lua)(模块层,源码经 `compat.ffmpeg` 直编) · [`opencv`](pkgs/o/opencv.lua)(单仓库:模块层与 OpenCV 5 全源码构建同在包内,索引侧只留本描述符) · [`mcpplibs.grpc`](pkgs/g/grpc.lua)(gRPC 1.83.0 —— 本索引里唯一**无法**做成 compat 描述符的库:上游不发布任何自包含源码产物,其 tag 归档里 abseil/protobuf/re2/boringssl/zlib 全是空 submodule 占位,因此 [grpc-m](https://github.com/mcpplibs/grpc-m) 的 release tarball 才是那个产物。它只 vendor gRPC 自己的源码,五个依赖全取自本索引,故同时直接使用 protobuf 的消费者链进去的是同一份而非两份)| | C 源码 compat(含 `features`) | [`compat.cjson`](pkgs/c/compat.cjson.lua) · [`compat.zlib`](pkgs/c/compat.zlib.lua) | | C++ 源码 compat(彼此依赖) | [`compat.abseil`](pkgs/c/compat.abseil.lua)(151 TU;对 `absl/**` 取通配后,按上游自身的 test/benchmark 命名约定裁剪) · [`compat.protobuf`](pkgs/c/compat.protobuf.lua)(libprotobuf 运行时,79 TU 逐条转录自上游 `src/file_lists.cmake`;因 protobuf 公开头文件 include 了 `absl/…`,故显式依赖 `compat.abseil`;`gzip` feature 定义 `HAVE_ZLIB` 并拉入 `compat.zlib`,`upb` feature 则从同一个 tarball 里再编出 protobuf 的 64 TU C 运行时) · [`compat.re2`](pkgs/c/compat.re2.lua)(22 TU,取自上游自身的 `RE2_SOURCES`) | | header-only(含 `features`) | [`compat.eigen`](pkgs/c/compat.eigen.lua) | diff --git a/mcpp.toml b/mcpp.toml index 73f85f9..7683cf2 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -35,6 +35,7 @@ members = [ "tests/examples/godot-cpp-module", "tests/examples/godot-cpp-module-v10", "tests/examples/godot-cpp-v10", + "tests/examples/grpc-module", "tests/examples/gui-stack", "tests/examples/imgui", "tests/examples/imgui-module", diff --git a/pkgs/g/grpc.lua b/pkgs/g/grpc.lua new file mode 100644 index 0000000..0c414fb --- /dev/null +++ b/pkgs/g/grpc.lua @@ -0,0 +1,76 @@ +-- Form A descriptor: the public gRPC package ships its own mcpp.toml, so this +-- file carries metadata and a download address and nothing else. mcpp's default +-- lookup finds /*/mcpp.toml inside the GitHub source tarball wrap. +-- +-- WHY THIS IS A SEPARATE REPOSITORY. Every other heavy library in this index is +-- a `compat` descriptor pointing at an upstream tarball. gRPC cannot be: it +-- publishes NO self-contained source artifact. v1.83.0 has no release assets at +-- all, and its tag archive carries abseil, protobuf, re2, boringssl and zlib as +-- EMPTY submodule placeholders (one directory entry each), so there is nothing +-- for `url` + `sha256` to point at. grpc-m's release tarball IS that artifact: +-- upstream's src/ and include/ vendored with zero patches, plus the two +-- third_party pieces gRPC really does ship (address_sorting, xxhash). +-- +-- WHAT IT DOES NOT VENDOR is the reason it belongs in this index rather than +-- standing alone. abseil, protobuf(+upb), re2, c-ares, OpenSSL and zlib are all +-- taken from the packages here, so a consumer that also uses protobuf or abseil +-- directly links ONE copy instead of colliding with a second vendored set. The +-- descriptors it depends on landed in #147 (abseil, protobuf), #148 (re2, the +-- protobuf `upb` feature) and #149 (c-ares). +-- +-- The package builds without CMake, Bazel or a configure step: gRPC's tree +-- contains no .h.in or config.h.cmake, and its generated upb code is checked in +-- upstream, so mcpp needs only include paths. 1001 TUs, all compiled by the +-- resolved toolchain — no external build system runs, so nothing inherits a +-- foreign C++ ABI. +-- +-- Exposes `import grpc;` over gRPC's public C++ API. The import must come AFTER +-- every textual #include in a TU (gRPC's codegen emits headers, so real +-- programs always mix the two); the package's README explains why and the +-- module's own header comment carries the measured failure modes. +-- +-- linux + macOS only; see the note in the xpm table. +-- +-- Optional feature `ares` is ON by default, matching upstream gRPC: +-- `grpc = { version = "1.83.0", default-features = false }` drops the c-ares +-- resolver and its dependency, which is upstream's own `grpc_no_ares=true`. +package = { + spec = "1", + name = "grpc", + namespace = "mcpplibs", + description = "gRPC 1.83.0 — vendored upstream source build with import grpc; (abseil/protobuf/re2/c-ares/OpenSSL come from this index)", + licenses = {"Apache-2.0"}, + repo = "https://github.com/mcpplibs/grpc-m", + type = "package", + + xpm = { + linux = { + ["1.83.0"] = { + url = { + GLOBAL = "https://github.com/mcpplibs/grpc-m/archive/refs/tags/v1.83.0.tar.gz", + CN = "https://gitcode.com/mcpp-res/grpc/releases/download/1.83.0/grpc-m-1.83.0.tar.gz", + }, + sha256 = "99cf3bda0a4e025f674f08dd68183538c2f655c3f72a55e1e2c1f4e05a282f43", + }, + }, + macosx = { + ["1.83.0"] = { + url = { + GLOBAL = "https://github.com/mcpplibs/grpc-m/archive/refs/tags/v1.83.0.tar.gz", + CN = "https://gitcode.com/mcpp-res/grpc/releases/download/1.83.0/grpc-m-1.83.0.tar.gz", + }, + sha256 = "99cf3bda0a4e025f674f08dd68183538c2f655c3f72a55e1e2c1f4e05a282f43", + }, + }, + -- No windows block, and the reason is a DEPENDENCY rather than gRPC: + -- compat.openssl has no windows xpm entry ("windows deferred — + -- requires prebuilt MSVC libs"), so resolution there fails with + -- E_NOT_FOUND: package 'compat:openssl@3.5.1' not found + -- before anything is compiled. gRPC's secure build cannot drop TLS, so + -- this package's platform coverage is exactly compat.openssl's, and it + -- widens the day that entry lands — the package already carries its + -- windows compile/link flags. + }, + + -- (no `mcpp` field -- default lookup will find /*/mcpp.toml) +} diff --git a/tests/examples/grpc-module/mcpp.toml b/tests/examples/grpc-module/mcpp.toml new file mode 100644 index 0000000..8ff7a29 --- /dev/null +++ b/tests/examples/grpc-module/mcpp.toml @@ -0,0 +1,34 @@ +# grpc test project: consumes the mcpplibs.grpc Form-A package (grpc-m) and +# asserts behavior under `mcpp test`. +# +# The `default` namespace redirect points the built-in mcpplibs namespace at +# this checkout, so the descriptor under test is pkgs/g/grpc.lua — not whatever +# the published index happens to hold. grpc's own compat.* dependencies still +# resolve from the published index, which is the point: this member proves the +# descriptor works against the real dependency set. +# +# NO protoc output anywhere: gRPC's codegen needs host tools mcpp cannot hand a +# consumer, so this exercises the module surface instead. grpc-m's own +# examples/helloworld covers the generated-stub path end to end. +[package] +name = "grpc-module-tests" +version = "0.1.0" + +# linux + macOS only: the package has no windows xpm entry (its compat.openssl +# dependency has none yet), so on windows this member carries no dependency at +# all and the test compiles to a no-op main() — the same shape +# tests/examples/openssl uses for the same underlying reason. +[target.'cfg(linux)'.dependencies.mcpplibs] +grpc = "1.83.0" + +[target.'cfg(linux)'.build] +cxxflags = ["-DHAVE_GRPC=1"] + +[target.'cfg(macos)'.dependencies.mcpplibs] +grpc = "1.83.0" + +[target.'cfg(macos)'.build] +cxxflags = ["-DHAVE_GRPC=1"] + +[indices] +default = { path = "../../.." } diff --git a/tests/examples/grpc-module/tests/module.cpp b/tests/examples/grpc-module/tests/module.cpp new file mode 100644 index 0000000..1c0c2e0 --- /dev/null +++ b/tests/examples/grpc-module/tests/module.cpp @@ -0,0 +1,45 @@ +// Behavioral test for the mcpplibs.grpc package, driven entirely through +// `import grpc;` — there is no include in this file. Every call +// reaches a real symbol in the linked gRPC, so a package that resolved and +// "built" but linked nothing could not pass. +// +// Note the ORDER: textual #includes first, `import grpc;` last. The module +// carries the standard library in its BMI, so a std header included after the +// import arrives twice and GCC fails on std::string's operator==. +// HAVE_GRPC comes from this project's own cfg-gated cxxflags — the package is +// linux/macOS-only (see mcpp.toml), so elsewhere this file is an empty main(). +#ifdef HAVE_GRPC + +#include +#include + +import grpc; + +int main() { + // A version string read out of the library, not a header constant. + const std::string version = grpc::Version(); + std::printf("grpc::Version() = %s\n", version.c_str()); + if (version.rfind("1.83", 0) != 0) return 1; + + // grpc::Status, error path included — an always-OK stub cannot pass. + const grpc::Status bad(grpc::StatusCode::UNAVAILABLE, "down"); + if (bad.ok() || bad.error_code() != grpc::StatusCode::UNAVAILABLE || + bad.error_message() != "down" || !grpc::Status::OK.ok()) { + return 1; + } + + // A real channel object. Channels connect lazily, so no server is needed; + // IDLE before the first RPC proves this is a live object from the library. + auto channel = grpc::CreateChannel("127.0.0.1:1", grpc::InsecureChannelCredentials()); + if (!channel) return 1; + if (channel->GetState(false) != GRPC_CHANNEL_IDLE) return 1; + + std::puts("grpc module: OK"); + return 0; +} + +#else // !HAVE_GRPC + +int main() { return 0; } + +#endif