Fix dead docs links and fail the build on dangling internal links

.github/workflows/deploy.yml

@@ -3,7 +3,8 @@ name: Build and deploy site
 on:
   push:
     branches: [main]
-  # Build (but don't deploy) on PRs, so a broken build is caught before merge.
+  # Build (but don't deploy) on PRs, so a broken build or dangling link is
+  # caught before merge.
   pull_request:
   workflow_dispatch:
 
@@ -24,6 +25,7 @@ env:
   HUGO_VERSION: 0.115.0
   MDBOOK_VERSION: 0.4.31
   MDBOOK_KATEX_VERSION: 0.5.4
+  LYCHEE_VERSION: 0.11.1
 
 jobs:
   build:
@@ -46,6 +48,8 @@ jobs:
             | tar -xz -C "$RUNNER_TEMP/bin" mdbook
           curl -sSfL "https://github.com/lzanini/mdbook-katex/releases/download/v${MDBOOK_KATEX_VERSION}/mdbook-katex-v${MDBOOK_KATEX_VERSION}-x86_64-unknown-linux-gnu.tar.gz" \
             | tar -xz -C "$RUNNER_TEMP/bin" mdbook-katex
+          curl -sSfL "https://github.com/lycheeverse/lychee/releases/download/v${LYCHEE_VERSION}/lychee-v${LYCHEE_VERSION}-x86_64-unknown-linux-gnu.tar.gz" \
+            | tar -xz -C "$RUNNER_TEMP/bin" lychee
           echo "$RUNNER_TEMP/bin" >> "$GITHUB_PATH"
 
       - name: Build

.lycheeignore

@@ -0,0 +1,26 @@
+# Known-dangling internal links that predate the link check in build.sh.
+# One regex per line, matched against the resolved link target (a file:// URL
+# under public/). Remove an entry once its page renders -- for spec pages that
+# means listing the file in dither-spec/SUMMARY.md, since mdbook only renders
+# chapters listed there -- or once the link itself is fixed or removed.
+
+# Spec pages that exist in dither-spec but are not listed in SUMMARY.md.
+# Linked from dither/02-routing.md, comparisons.md, stack/roadmap.md,
+# applications.md and dither/interoperability.md.
+/docs/dither/discovery\.html$
+/docs/dither/measuring-anonymity\.html$
+/docs/dither/decentralized-data-ideas\.html$
+/docs/applications/(dithca|dithix|dither-git|nomia)\.html$
+
+# dither/reverse-hash-lookup.md links ../../applications/... (one ../ too
+# many), which escapes /docs/ entirely.
+/public/applications/(web-of-beliefs|dithca)\.html$
+
+# Targets that do not exist at the linked path: distance-based-routing.md was
+# superseded by dither/02-routing.md; zero-knowledge-proofs.md and
+# tree-of-math-old.md only exist under old_ideas/ and old-ideas/;
+# applications/dither-dex.md was never written.
+/docs/dither/distance-based-routing\.html$
+/docs/dither/zero-knowledge-proofs\.html$
+/docs/tree-of-math-old\.html$
+/docs/applications/dither-dex\.html$

README.md

@@ -23,15 +23,22 @@ git push                                        # triggers the deploy
 
 ## Local development
 
-Requires [nix](https://nixos.org) with flakes (or hugo-extended, mdbook and
-mdbook-katex on `PATH`). `direnv allow` picks up the devshell automatically.
+Requires [nix](https://nixos.org) with flakes (or hugo-extended, mdbook,
+mdbook-katex and lychee on `PATH`). `direnv allow` picks up the devshell automatically.
 
 ```sh
-nix develop        # hugo-extended, mdbook, mdbook-katex
+nix develop        # hugo-extended, mdbook, mdbook-katex, lychee
 hugo server        # live preview of the site shell
 ./build.sh         # full build (site + /docs/) into public/
 ```
 
 CI pins the same tool versions the devshell provides, so a local `./build.sh`
 produces byte-identical output to a deploy. When bumping `flake.lock`, update
 the pinned versions in `.github/workflows/deploy.yml` to match.
+
+`build.sh` ends by running [lychee](https://github.com/lycheeverse/lychee) over
+`public/` and fails on any internal link whose target was not generated (a
+typo in a path, or a spec page that exists but is not listed in
+`dither-spec/SUMMARY.md` -- mdbook only renders chapters listed there). Known
+dangling links that predate the check are allowlisted in `.lycheeignore`;
+remove an entry once its page is listed or the link is fixed.

build.sh

@@ -16,3 +16,12 @@ mdbook build dither-spec -d ../public/docs
 # stale build dir left behind by running `mdbook build` there without -d. A
 # fresh CI checkout has neither; drop them so local builds match.
 rm -rf public/docs/.git public/docs/public
+
+# Fail on dangling internal links. Offline: only file links are checked, never
+# http(s). --base resolves root-relative hrefs (/docs/...) from the Hugo pages
+# against public/. print.html is mdbook's concatenation of every chapter, and
+# its rewritten relative links are artifacts of that concatenation, not of the
+# sources. Known-dangling links that predate this check are allowlisted in
+# .lycheeignore, which lychee reads from the working directory.
+lychee --offline --no-progress --base public \
+  --exclude-path public/docs/print.html public

content/inspiration.md

@@ -18,12 +18,12 @@ As fundamental freedoms are continually encroached upon, having a decentralized
  - Monero is the only cryptocurrency that actually has most of what a crypto *currency* hould have. Transactions are completely anonymous, transaction fees are very low and it's not managed by a single company. Dither will improve on this with its own suite of cryptocurrencies that are all completely anonymous and take advantage of better consensus protocols for functionally instant payment.
 
 [IPFS](https://ipfs.io/)
- - IPFS is a core project in the world of web3, and a really good start to the decentralized internet. However, it is simply too [slow and hard to interface with](https://www.publish0x.com/ecosystem-overviews-and-analysis/the-precarious-state-of-ipfs-in-the-year-2020-xmvxeg). Dither's [Directional Trail Search](docs/dither/directional-trail-search.html) will greatly improve on the speed and efficiency of fetching data, as well as supporting more ways of representing data and storing data more efficiently.
+ - IPFS is a core project in the world of web3, and a really good start to the decentralized internet. However, it is simply too [slow and hard to interface with](https://www.publish0x.com/ecosystem-overviews-and-analysis/the-precarious-state-of-ipfs-in-the-year-2020-xmvxeg). Dither's [Directional Trail Search](/docs/dither/directional-trail-search.html) will greatly improve on the speed and efficiency of fetching data, as well as supporting more ways of representing data and storing data more efficiently.
 
-DNS - The Domain Name System is structurally decentralized, but it is centrally managed by ICANN, an organization with the authority to control domain name registration. Dither will have its own naming system that will replace DNS and allow for much better systems of naming things. This naming system will be built on the [Reverse Hash Lookup](docs/dither/reverse-hash-lookup.html) system.
+DNS - The Domain Name System is structurally decentralized, but it is centrally managed by ICANN, an organization with the authority to control domain name registration. Dither will have its own naming system that will replace DNS and allow for much better systems of naming things. This naming system will be built on the [Reverse Hash Lookup](/docs/dither/reverse-hash-lookup.html) system.
 
 [TOR](https://www.torproject.org/) & [I2P](https://geti2p.net/en/)
- - These "Hidden Networks" are very important for obfuscating connections between people on the internet. However, they are *very slow* due to their nature of randomly routing connections through different nodes. Dither will improve on this with [Distance-Aware Anonymous Routing](docs/dither/02-routing.html), a routing protocol that takes latency and throughput into account when routing packets, allowing for the user to choose between better privacy or faster connection. Dither may also implement a anonymous routing technology called [HORNET](https://arxiv.org/pdf/1507.05724v3.pdf) to allow for additional efficiency when establishing obfuscated connections.
+ - These "Hidden Networks" are very important for obfuscating connections between people on the internet. However, they are *very slow* due to their nature of randomly routing connections through different nodes. Dither will improve on this with [Distance-Aware Anonymous Routing](/docs/dither/02-routing.html), a routing protocol that takes latency and throughput into account when routing packets, allowing for the user to choose between better privacy or faster connection. Dither may also implement a anonymous routing technology called [HORNET](https://arxiv.org/pdf/1507.05724v3.pdf) to allow for additional efficiency when establishing obfuscated connections.
 
 Other projects that are inspiring dither are:
 [Nomia](https://github.com/scarf-sh/nomia),

content/papers.md

@@ -1,2 +1,2 @@
 ### Papers
-[Distance Based Routing Whitepaper](./papers/distance-based-routing-whitepaper.pdf)
+[Distance Based Routing Whitepaper](/papers/distance-based-routing-whitepaper.pdf)

flake.nix

@@ -14,6 +14,7 @@
 			hugo
 			mdbook
 			mdbook-katex
+			lychee
 		];
 	in utils.lib.mkFlake {
 		inherit self inputs;