build: reorganize repo layout (docs_src, dist/meson, test/bench, LSP compdb) - #128
Merged
Conversation
Rename the Markdown documentation source tree to docs_src (underscore, matching Python-friendly naming). Updates every reference: - docs_src/build.py, _migrate/*.py path constants (DOCS/OUT/NEW) - docs_src/_migrate/codespell-baseline.txt keys (path prefix) - .github/workflows/docs.yml (all docs-src paths) - .gitignore (docs_src/**/__pycache__, benchmark bins repointed) - flake.nix devShell comments - docs_src/PLAN.md, index.md, _templates/*, _migrate/README.md build.py resolves paths via Path(__file__).parent so the rename is transparent to its logic; verified: --selfcheck ok, 1465 HTML + 789 man pages build from docs_src/.
The scraped Oracle DocBook under docs/ is superseded by docs_src/ (its content was migrated at ~100% retention, verified in PRs #119/#120). The ENTIRE old docs/ tree is also preserved on the gh-pages branch under docs/db/5.3.28/ (confirmed: gsg C/CXX/JAVA variants and the 528-file java/ Javadoc are all present there), so removing it from master loses nothing permanently. Disposition of each docs/ subtree: - api_reference, articles, bdb-sql, collections, gsg (C), gsg_db_rep (C), gsg_txn (C), installation, porting, programmer_reference, upgrading, index.html: MIGRATED into docs_src/ -> removed. - java (528 Javadoc HTML), and the CXX/JAVA variants of the gsg guides: NOT yet in docs_src (Java is a supported binding). DEFERRED to a follow-up docs PR; the content is safe on gh-pages (docs/db/5.3.28/) meanwhile. Removed from master here only because the trees are interleaved with migrated ones. - ssi/ (M2/M4 SIREAD design notes): internal design intent, not user docs -> RELOCATED to rfc/0003/ as the working notes of a new RFC 0003 (SSI, Status Implemented), with an INDEX.md row. - legal + license: PRESERVED. docs/license/license_db.html and the per-component license texts (ASM/BSD/CDDL/HARVARD/SPL) relocated to a top-level LICENSES/ (with a README) -- these are the fork's legal basis and are not deleted. No build/source referenced docs/legal or docs/license. Build targets (dist/Makefile.in): - install_docs now installs docs-build/html if present (build is a separate `make docs` concern; a plain install never needs the docs toolchain). - `make docs` -> python3 docs_src/build.py (HTML+man, +PDF if weasyprint) - `make docs-check` -> the docs.yml gates locally (build+self-check, API completeness, spelling, man-lint) - `make bench` -> build the test/bench microbenchmark drivers (CC=cc, to bypass the parent build's libtool CC wrapper) - `make compdb` -> compile_commands.json via bear All guard on their toolchain (python3/pandoc/bear) and skip-with-note if absent, so they never hard-fail a build that isn't about them. docs.yml: the no-loss gate (verify_all.py, which diffed the old docs/ against docs_src/) is retired -- its migration source no longer exists on master; the surviving hard gates (build, completeness, spelling, links, man-lint) stay.
lab/bench is a real microbenchmark regression suite (scale_bench, lock_bench, ssi_abort_bench, tproc_b/c/h) -- move it under test/ where the rest of the regression tooling lives. Its Makefile's `BDB ?= ../../build_unix` default still resolves from the new location (same directory depth), so it is unchanged; `make bench` (Autoconf) and `ninja bench` (Meson) now build the drivers via the targets added in the previous commit. With bench moved and the LSM prototype already graduated to rfc/0001/ (PR #127), lab/ held only README.md, whose content was entirely bench docs -- its useful parts move to test/bench/README.md and lab/ is removed. Fixes the two lab/bench/scale_bench references in rfc/0002.
Move the Meson build logic (meson.build) and its helpers (meson/gen_header.py,
meson/db_subs.json) under dist/, alongside the Autoconf build files.
The root-meson.build problem: Meson requires meson.build AND meson_options.txt
at the setup directory, and its subdir() cannot climb out with '..'. So:
- dist/meson.build holds the real build logic, anchored at the project root via
`root = '..'` (sources under src/, templates, dist/clib_port.in all resolve).
- A thin root meson.build owns project()/cc/options and does subdir('dist');
because subdir() shares the interpreter scope, dist/meson.build's vars (inc,
db_h, thread_dep, libdb) stay visible to the shim, which then enters
test/pbt (the PBT subdir must run from the root, not dist/, since subdir
cannot use '..').
- meson_options.txt stays at the repo root: Meson binds the options file to the
project() directory. Moving it to dist/ would force `meson setup build dist`,
breaking the zero-arg `meson setup build` and the nix flake's `src = ./.`
meson hook. This is the one file the relocation can't move cleanly.
The library now builds under <builddir>/dist/libdb.so; consumers updated:
- ci.yml smoke test: -I/-L build-meson/dist, LD_LIBRARY_PATH=build-meson/dist
- ci-extended.yml config-parity: build-meson/dist/db_config.h
- dist/android/build_android.sh: copy <builddir>/dist/libdb.so up to the stable
<builddir>/libdb.so artifact path
- android.yml / pbt.yml path filters: dist/meson.build, dist/meson/**
- ocr/rule.json: dist/meson.build / dist/meson/
Also adds Meson run-targets (dist/meson/run_docs.py, run_bench.py):
ninja docs / docs-check render docs_src/ + run the gates
ninja bench build the test/bench drivers vs the fresh libdb
README: split Building into Autoconf + Meson sections with the new targets.
Verified: meson setup build && ninja (libdb.so), ninja docs/bench, nix build
.#libdb-meson -> result/lib/libdb.so, meson setup -Dhegel=enabled + ninja (PBT).
The Autoconf build emits no compilation database, so `make compdb` (added earlier) wraps the build with bear to produce a repo-root compile_commands.json for clangd. Add bear to the nix dev shell so the target works out of the box. (The Meson build emits build/compile_commands.json automatically via Ninja; both paths are documented in the README and git-ignored.)
Coccinelle convention checksNo new violations. ✅ Resolved since baseline (2) -- update dist/cocci/baseline.txt to lock these in. |
ABI diff vs
|
bear only records commands for files actually (re)compiled; running 'make compile-commands' against an already-built tree produced an empty compile_commands.json. Clean first so the LSP database is always complete regardless of build state (566 entries from a full library_build).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Build-layout reorganization
Reorganizes the repo layout across both build systems (Autoconf + Meson) +
nix, in five logical commits. Build-layout only — no
src/engine changes.Core smoke (
test001btree/hash) still passes.1.
docs-src→docs_srcRename the Markdown docs source (underscore, Python-friendly). Updated every
reference:
build.py/_migrate/*.pypath constants, the codespell baselinekeys,
.github/workflows/docs.yml,.gitignore,flake.nix,PLAN.md,index.md, templates.build.pyresolves paths viaPath(__file__).parent, soits logic is transparent to the rename. Verified:
--selfcheckok, 1465 HTML +789 man pages build.
2. Remove the old scraped
docs/archive; add doc/bench/compdb targetsThe Oracle DocBook under
docs/is superseded bydocs_src/and fullypreserved on the
gh-pagesbranch underdocs/db/5.3.28/(confirmed: thegsg C/CXX/JAVA variants and the 528-file
java/Javadoc are all there).Disposition of each
docs/subtree:docs_src/→ removedjava/(528 Javadoc HTML), the CXX/JAVAgsg*variantsdocs_src(Java is supported); safe on gh-pages; follow-up docs PR migrates themssi/(M2/M4 SIREAD design notes)rfc/0003/as the working notes of new RFC 0003 (SSI, Implemented) + INDEX rowlicense/license_db.html+ per-component texts (ASM/BSD/CDDL/HARVARD/SPL)LICENSES/(with README) — the fork's legal basis, not deletedAutoconf targets (
dist/Makefile.in, guarded on toolchain, skip-with-note):make docs→python3 docs_src/build.py(HTML+man, +PDF if weasyprint)make docs-check→ the docs.yml gates locally (build+self-check, APIcompleteness, spelling, man-lint)
install_docsinstallsdocs-build/htmlif present (build is a separatemake docsconcern)docs.yml: the no-loss gate (verify_all.py, which diffed the now-removeddocs/againstdocs_src/) is retired; the other hard gates remain.3.
lab/bench→test/bench; drop emptylab/Moved the microbenchmark regression suite under
test/. Its Makefile'sBDB ?= ../../build_unixstill resolves (same depth).lab/held only a README(bench docs →
test/bench/README.md; LSM prototype already inrfc/0001/), solab/is removed. Fixed the twolab/bench/scale_benchrefs inrfc/0002.4.
meson→dist(via a thin root shim)git mv meson dist/meson,meson.build → dist/meson.build. Meson requiresmeson.build+meson_options.txtat the setup root and itssubdir()cannotuse
.., so:dist/meson.buildholds the real logic, anchored at the project root viaroot = '..'.meson.buildownsproject()/cc/options and doessubdir('dist')(scope is shared, soinc/db_h/libdbstay visible),then enters
test/pbt(which must run from the root, notdist/).meson_options.txtstays at the repo root — Meson binds it to theproject()directory. This is the one file the relocation can't move cleanly:moving it would force
meson setup build distand break the zero-argmeson setup build+ the nix flake'ssrc = ./.hook.The lib now builds under
<builddir>/dist/; updated ci.yml smoke test,ci-extended config-parity, android build script + workflow filters, pbt filters,
and the OCR rule. Added Meson
run_targets:ninja docs/docs-check/bench(viadist/meson/run_docs.py,run_bench.py).5. compile_commands.json for LSPs
build/compile_commands.jsonautomatically (Ninja); alreadygit-ignored; documented in the README.
make compdbwraps the build withbear→repo-root
compile_commands.json. Addedbearto the nix dev shell.Validation (all green)
configure && make -j4clean (0 errors);make docsbuildsdocs-build/html;make docs-checkPASS;make benchbuilds all 6 drivers;make compdb→ valid repo-rootcompile_commands.json(512src/*.centries).meson setup build && ninjaclean →build/dist/libdb.so;compile_commands.jsonpresent (269 entries);ninja docs+ninja benchbuild; PBT builds with
-Dhegel=enabled.nix develop --command trueevaluates;nix build .#libdb-meson→result/lib/libdb.so.configureregen needed.docs/tree.test001 btree+test001 hashpass.Dormant Oracle-era dist tooling (
dist/s_javadoc,s_sql_drivers,buildpkg,validate/s_chk_stats) still names olddocs/paths but is out of themake/ninja/nixbuild flow and left untouched.