Skip to content

✨ Replace the Google Books API with Open Library - #102

Merged
Logan2234 merged 5 commits into
mainfrom
claude/google-books-open-library-1evp6r
Aug 18, 2026
Merged

✨ Replace the Google Books API with Open Library#102
Logan2234 merged 5 commits into
mainfrom
claude/google-books-open-library-1evp6r

Conversation

@Logan2234

Copy link
Copy Markdown
Owner

Open Library becomes the sole book catalogue source, replacing the Google Books API. It is keyless and unmetered (Google Books required an API key and capped the free tier at 1,000 requests/day) and imposes no branding requirement, so the "Powered by Google" marks give way to plain courtesy attribution.

Ships as 1.6.0 (version bumped in lockstep, CHANGELOG.md entry added, Quackback changelog left in draft — publishing it sends the release newsletter, so that's Logan's call).

The provider

OpenLibraryProvider replaces GoogleBooksProvider, covering the same four capabilities:

Capability How
search(query) /search.json (Solr), explicit fields so the response stays small
searchByIsbn(isbn) q=isbn:…
searchByIsbns(isbns[]) q=isbn:(A OR B …), batches of 20, mapped back through each doc's edition ISBN list
getDetails(id) /works/{id}.json (description) + search.json?q=key:/works/{id} (page count, publishers, subjects, rating)

Notable behaviours:

  • Merged works redirect. Open Library serves a type: /type/redirect stub for a work merged into another; details follow it (bounded hops) and key everything — summary, permalink, external id — off the canonical id, so the cache never stores a stale alias.
  • Same-author suggestions resolve through the stable author_key and sort=rating, instead of Google's display-name matching.
  • Subjects are filtered before becoming genre chips: Open Library's crowd-sourced list mixes real genres with machine tags (nyt:mass-market-monthly=2021-11-07), sentence-long entries and case variants — all dropped, capped at 10.
  • Covers use the -L variant (~500px) rather than Google's ~128px thumbnails.
  • Verified against the live API (search, ISBN, bulk ISBN, details, redirect) before the smoke test was removed; 15 unit tests cover the mapping.

Data model

BookSource.GOOGLE_BOOKSOPEN_LIBRARY, addressed by work id (OL893414W).

BookItem is an on-demand cache keyed by the source's own id, so the migration drops the cached books rather than converting them — along with everything referencing one through a polymorphic, FK-less target that nothing cascades from: book reviews, comments, list items, activity events, book-linked notifications, and the reports/moderation decisions resolved through those reviews and comments. ReadingGoal references no book and is untouched.

⚠️ Breaking for self-hosters: book library entries don't survive the upgrade (BookEntry hangs off BookItem). No production data is affected. Other domains are untouched.

Trade-off accepted

Books lose their adult signal. Open Library exposes no equivalent of Google's MATURE maturity rating, so BookItem.isAdult is always false. The column, the filterAdultContent calls and the per-account gate all stay in place for the day a source provides one — documented at each site rather than removed.

Config

  • GOOGLE_BOOKS_API_KEY removed from .env.example and docker/docker-compose.yml.
  • MUSICBRAINZ_CONTACTAPI_CONTACT, now shared by both keyless providers for the identifying User-Agent their usage policies ask for (Open Library and MusicBrainz).
  • /admin/services: the Livres entry becomes keyless with no quota ceiling (no published limit), probed via search.json.

Also in here

  • Fixes the seed writing its book call counter under google_books while QuotaTrackerService wrote googleBooks — the row was rendering under its raw key instead of a label on /admin/stats.
  • Legal pages (mentions notice, privacy policy ×3, terms), settings → data sources, landing copy, Goodreads/StoryGraph import copy, README, apps/api/README, docs/data-model.md and the regenerated docs/erd.md.
  • messages/{fr,en}.json: book_google_noticebook_openlibrary_notice, settings_datasources_googlebooks_notice…_openlibrary_notice; the Google brand mark is dropped from provider-brands.ts (simple-icons has no Open Library glyph, so it's listed logo-less like OMDb).

Checks

pnpm test (665 passed), pnpm lint, pnpm --filter @loomkeep/web check (0 errors), pnpm knip (only pre-existing findings).

🤖 Generated with Claude Code

https://claude.ai/code/session_01WqHpRYcQuZ5safw87XgVPd


Generated by Claude Code

claude added 2 commits August 18, 2026 14:16
Open Library becomes the sole book catalogue source. It is keyless and
unmetered (Google Books required an API key and capped the free tier at
1,000 requests/day), and imposes no branding requirement, so the "Powered
by Google" marks give way to plain courtesy attribution.

- `OpenLibraryProvider` replaces `GoogleBooksProvider`: `/search.json`
  (Solr) serves both search and a work's aggregate metadata, `/works/{id}
  .json` the description Solr does not carry. Merged works redirect, and
  details resolve to the canonical id so the cache never stores an alias.
  Same-author books now key off `author_key` instead of matching a display
  name, and crowd-sourced subjects are filtered down to genre-shaped ones.
- `BookSource.GOOGLE_BOOKS` → `OPEN_LIBRARY`, addressed by work id
  ("OL893414W"). The migration drops the cached books rather than
  converting them — BookItem is an on-demand cache — along with everything
  referencing one through a polymorphic (FK-less) target.
- Books lose their adult signal: Open Library exposes no maturity rating,
  so `isAdult` is always false. The column and its per-account gate stay
  in place for the day a source provides one.
- `MUSICBRAINZ_CONTACT` becomes `API_CONTACT`, now shared by both keyless
  providers' User-Agent, and `GOOGLE_BOOKS_API_KEY` is gone.
- Fixes the seed writing its book counter under `google_books` while the
  tracker wrote `googleBooks`, which left the row unlabelled on /admin/stats.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WqHpRYcQuZ5safw87XgVPd
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WqHpRYcQuZ5safw87XgVPd
@github-actions github-actions Bot added documentation Improvements or additions to documentation docker Dockerfiles, docker-compose, self-hosting stack api apps/api (NestJS) web apps/web (SvelteKit) shared packages/shared labels Aug 18, 2026

Copy link
Copy Markdown
Owner Author

The knip check is red here, but it is not caused by this PR — it fails identically on main (latest main run, same knip job, conclusion: failure), and the job is continue-on-error: true so it doesn't gate the merge.

Cause: the knip job runs pnpm install + generate:paraglide but not pnpm build:package, unlike lint-build-test. Without packages/shared/dist, knip can't resolve how api/web consume the shared package, so it reports every exported DTO in packages/shared/src/dto/* as unused — ~200 findings spanning files this PR never touches (list.ts, music.ts, stats.ts, social.ts…).

Locally with a built dist, pnpm knip reports only 5 pre-existing findings, none introduced here.

The fix is one line in .github/workflows/ci.yml (a - run: pnpm build:package step before pnpm knip), but that's CI hygiene unrelated to the Open Library swap — happy to do it in a separate PR if you want it.


Generated by Claude Code

The knip job ran `pnpm install` + `generate:paraglide` but not
`pnpm build:package`, unlike lint-build-test. api/web import the shared
package through its `dist/`, so without it knip couldn't resolve those
imports and reported all ~215 exported DTOs in packages/shared as unused —
drowning the 5 real findings and leaving the job red on every run,
including on main (hidden by `continue-on-error`).

Building the package first drops it to those 5, all of them genuinely dead:

- `SimklWatchlistStatus`, `SimklEpisode`, `SimklSeason` are used only
  inside `simkl-api.types.ts` — unexported rather than removed.
- `AdminVersionDto` is superseded by `RuntimeConfigDto.version`, which is
  what the settings footer actually reads.
- `AddBookReplayDto` (shared) is superseded by the API's own validated
  class of the same name; the web posts an empty body.

knip now exits 0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WqHpRYcQuZ5safw87XgVPd
@github-actions github-actions Bot added the ci CI/CD, GitHub Actions workflows label Aug 18, 2026
@Logan2234
Logan2234 enabled auto-merge (squash) August 18, 2026 14:55
claude added 2 commits August 18, 2026 15:14
Branch protection can't require `docker-build`: a matrix job reports one
check per combination ("docker-build (api, apps/api/Dockerfile)"), and the
bare job name is only ever reported when the whole job is skipped. So a
rule on `docker-build` was satisfied on a push to main (where the job is
skipped in favour of docker-push) but sat at "Expected — waiting for status
to be reported" on every PR touching apps/api or apps/web, blocking
auto-merge indefinitely.

`docker-build-ok` needs docker-build, runs on always(), and passes on
success or skipped — so it always reports, under a name the matrix can't
change. Require it instead of `docker-build`.

Requiring the expanded matrix names instead would deadlock the mirror case:
a PR touching neither app skips the job, which then reports only the bare
name and never those two contexts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WqHpRYcQuZ5safw87XgVPd
Follow-up on a60bc27: adding `docker-build-ok` alongside the matrix still
required flipping the branch protection rule over to the new name, and left
`docker-build` sitting at "Expected — waiting for status to be reported"
until that happened.

Swapping the names instead means no settings change: the matrix job becomes
`docker-build-images` (its legs report as "docker-build-images (api,
apps/api/Dockerfile)", where a real build break stays individually
visible), and the aggregating gate takes the `docker-build` name the rule
already requires. It always reports, and goes red when a leg does.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WqHpRYcQuZ5safw87XgVPd
@Logan2234
Logan2234 merged commit 3fa6702 into main Aug 18, 2026
15 checks passed
@Logan2234
Logan2234 deleted the claude/google-books-open-library-1evp6r branch August 18, 2026 15:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api apps/api (NestJS) ci CI/CD, GitHub Actions workflows docker Dockerfiles, docker-compose, self-hosting stack documentation Improvements or additions to documentation shared packages/shared web apps/web (SvelteKit)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants