Save series information into ebook files so readers group them correctly - #59
Open
JordanFromIT wants to merge 1 commit into
Open
Conversation
If you read your Chaptarr library in something like Kavita, books that belong to a series often show up as unrelated standalone titles instead of being grouped together. A reader works out which series a book belongs to by looking inside the ebook file. Chaptarr knows what series a book is in and where it falls in the order, but never writes any of that into the file, so the reader never sees it. Chaptarr already has a setting for this, Write Book Tags, and an ebook tag-writing subsystem behind it. That writer only ever runs for books registered with a Calibre content server, because it is gated on an id only Calibre hands out. Without Calibre the setting silently does nothing. This makes it work by writing the series name and position straight into the epub; Calibre users keep the existing path unchanged. In code terms, ebook tag writing was reachable only for files carrying a BookFiles.CalibreId, populated exclusively by cdb/add-book/. MetadataTagService.WriteTags gated the ebook branch on CalibreId > 0, and SyncTags, RetagFiles and RetagAuthor each filtered on CalibreId != 0. The fields that matter are calibre:series and calibre:series_index in the OPF, which ReadAllEpubTags already parses on the way in, so only the write side was missing. Behaviour: - Writes to a temporary file and moves it into place, so an interrupted write cannot truncate a book, and keeps mimetype first and stored. - Idempotent: an epub whose tags already match is left byte for byte alone. - Series positions that are not numeric, such as "2 - Heavy Metal", record the series without a position rather than a value readers cannot parse. Fractional positions such as 11.5 are preserved, and the index is always written with an invariant decimal separator. - Series data comes from the denormalised Book.SeriesName and SeriesPosition columns, falling back to the relational series links only when those are empty. - Bulk re-tagging no longer stops at the first unwritable file, matching how the import path already treats tag writing as optional post-processing. - Honours the existing writeBookTags setting rather than adding one. The default, NewFiles, still means an existing library is never touched until the user either downloads a new book or asks for a re-tag. - EBookTagService.WriteTagsInternal logged "No calibre id, skipping writing tags" without returning, so the message was misleading. Now it returns. The package document is edited with XmlDocument and PreserveWhitespace rather than XDocument. XDocument re-derives namespace prefixes on save, and an OPF package commonly binds a prefix to the namespace it already uses by default, so a round trip rewrote <metadata> as <opf:metadata> and every <meta> as <opf:meta>. That is namespace-equivalent, but it changes far more of the reader's book than asked and risks tripping prefix-sensitive OPF parsers. XmlDocument writes each node back with the prefix it was read with, preserves the existing formatting, and emits no byte order mark. Retag previews are unchanged and remain Calibre-only, so for a non-Calibre library the preview lists no changes while the re-tag itself now does work. Closing that gap needs a non-Calibre "read current tags" implementation and is left for a follow-up. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Q4m3BKXM9ENfugXvVCbt5G
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
The problem. If you read your Chaptarr library in something like Kavita, books that belong to a series often show up as unrelated standalone titles instead of being grouped together. A reader works out which series a book belongs to by looking inside the ebook file. Chaptarr knows perfectly well what series a book is in and where it falls in the order — it just never writes any of that into the file, so the reader never sees it. Today the only ebooks that group correctly are the ones whose original packager happened to fill the field in already; Chaptarr itself contributes nothing either way.
Why it happens. Chaptarr already has a setting for this, Write Book Tags, and a whole ebook tag-writing subsystem behind it. But that writer only ever runs for books that have been registered with a Calibre content server, because it's gated on an id that only Calibre hands out. If you don't run Calibre — which is most people — the setting silently does nothing at all. It isn't broken so much as unreachable.
What this does. Makes that existing setting actually work without Calibre, by writing the series name and position directly into the epub. Anyone already using Calibre keeps the current behaviour, unchanged.
So this is less a new feature than making an existing setting, and an existing subsystem, do something for the majority of users.
Technical detail
Ebook tag writing is reachable only for files that carry a
BookFiles.CalibreId, and that column is populated exclusively bycdb/add-book/.MetadataTagService.WriteTagsgates the ebook branch onbookFile.CalibreId > 0, andSyncTags/RetagFiles/RetagAuthoreach filter onCalibreId != 0. With no Calibre server there is no id, so nothing downstream ever runs. (EBookTagService.WriteTagsInternalalso logs "No calibre id … skipping writing tags" without actually returning, so the message is misleading; that's fixed here too.)The specific fields that matter are
calibre:seriesandcalibre:series_indexin the epub's OPF package document — Kavita decides series membership solely from those.ReadAllEpubTagsalready parses both on the way in, so only the write side was missing.This adds an OPF writer for
.epub/.kepub, plus routing to it when a file's root folder is not a Calibre library (RootFolder.IsCalibreLibrary). Calibre libraries keep their existingCalibreProxy.SetFieldspath untouched.Deliberate choices worth flagging for review:
writeBookTagsenum. The default isNewFiles, so an existing library is never touched until the user either downloads a new book or explicitly asks for a re-tag. There is no scheduled task that triggers re-tagging.mimetypestays first and stored.SeriesPositionvalues include things like2 - Heavy Metal. Those record the series without a position rather than writing a value readers can't parse — the book still groups correctly, it just loses its ordering. Fractional positions like11.5are kept, and the index always uses an invariant decimal separator.Book.SeriesName/Book.SeriesPosition, falling back to the relational series links only when those are empty. Happy to switch to the links if you'd rather be consistent withSetFields.XmlDocument, notXDocument.XDocumentre-derives namespace prefixes on save. An OPF package commonly binds a prefix to the same namespace URI it already uses by default, and in that case a round trip rewrote<metadata>as<opf:metadata>and every<meta>as<opf:meta>. Namespace-equivalent, but it rewrites much more of the user's book than intended and risks tripping prefix-sensitive OPF parsers.XmlDocumentwithPreserveWhitespacewrites each node back with the prefix it was read with and emits no BOM.ImportApprovedBooks.TryWriteTagsalready treats tag writing as optional post-processing.Known gap, deliberately not addressed: retag previews still go through Calibre only, so for a non-Calibre library the preview lists no changes while the re-tag itself now does work. Closing that needs a non-Calibre "read the current tags back and diff them" implementation, which felt like it belonged in its own PR rather than doubling the size of this one. Happy to fold it in if you'd prefer.
Database Migration
NO. No schema changes and no new columns — this reads
Books.SeriesName/Books.SeriesPositionandRootFolders.IsCalibreLibrary, all of which already exist.How was this tested?
Unit tests — 23 new tests, all written before the code and watched failing first.
develop(537eb64): 2834 → this branch: 2857 (+23), 0 failing.Run against this repo's own CI commands rather than just locally convenient ones, all clean:
package.jsonparseversion_guard.py syncversion_guard.py monotonic --compare-ref origin/developversion_guard.py commit-hygiene --compare-ref origin/developorigin/develop..HEADdotnet build src/Chaptarr.NoTests.sln --configuration Releasedotnet test src/Chaptarr.Core.Test/Chaptarr.Core.Test.csproj --configuration Releasedotnet publish src/NzbDrone.Console/Chaptarr.Console.csproj -c ReleaseNo frontend files are touched (8 files changed, all
.cs), so the yarn steps that live outsidebuild.ymlaren't applicable here.EpubSeriesTagWriterFixture— writes when absent, updates when stale, reports unchanged and leaves the file byte-identical when tags already match, omits and clears the index when the position is unknown, keepsmimetypefirst and stored, invariant decimal separator under a comma-decimal culture, doesn't re-prefix elements it didn't need to touch, leaves unrelateddc:metadata alone, and a round trip proving the bundledVersOne.Epubreader can read back what was written.EbookTagServiceDirectWriteFixture— routing byIsCalibreLibrary(with a Calibre proxy that throws on any call, so reaching Calibre fails the test), non-numeric and fractional position handling, no series and unsupported-format cases, config gating, backfill viaRetagAuthor, sync, and continuing past a file that throws.MetadataTagServiceEbookDispatchFixture— an epub with no Calibre id now dispatches; a Calibre-managed file of any format still dispatches; an unsupported format with no Calibre id still doesn't.Test epubs are generated by the fixture from text written for the purpose; nothing is committed as a binary.
End to end, Docker on Linux: this branch's
Chaptarr.Core.dllbind-mounted over the publishedchaptarr/chaptarr:0.9.929image (the assembly version matches, so no full image rebuild), against an empty config and a scratch library, with three generated epubs and a non-Calibre root folder. RanRetagAuthorfrom the API and checksummed before and after:An Example Series/3An Example Series/2 - Heavy MetalThe resulting OPF keeps its original prefixes, indentation and unrelated metadata, with
mimetypestill first and stored. This also confirms the new interface resolves through DryIoc's auto-registration at runtime, which the unit tests can't prove.Screenshots (UI changes only)
None — no UI changes.