Skip to content

Extend whelk to RDF dump feature with n-quads support - #1810

Draft
niklasl wants to merge 6 commits into
developfrom
feature/rdf-dump-writers
Draft

Extend whelk to RDF dump feature with n-quads support#1810
niklasl wants to merge 6 commits into
developfrom
feature/rdf-dump-writers

Conversation

@niklasl

@niklasl niklasl commented Jul 10, 2026

Copy link
Copy Markdown
Member

Example usage:

cd importers
../gradlew jar
mkdir -p /tmp/libris-dumps
java -Dxl.secret.properties=../secret.properties.dev -jar build/libs/xlimporter.jar \
        lddbToNquads /tmp/libris-dumps/dev-bib-nq 4 --gzip bib

This is still WIP for various reasons:

  • Compliation of importers failed on develop (possibly due to gradle bump). This is adressed in 5502f9e which we may want to cherry-pick if that's the way to go.
  • We must to use a unique bnodes minter (e.g. doc-and-batch-count as prefix, or uuid:s) per conversion (a.k.a. "standardize apart"); the current nquads writer here will result in horrendous bnode smushing!
  • I prefer to isolate trld-java API dependency a bit more (any may improve its public API further to do so).
  • TRLD 0.3.0 includes proper n-quads support (and c14n, though we don't need it here); so we should either just bump trld-java, or backport that part (crucially fixes newlines in strings).
  • We do some intricate cleanup in JsonLdToTrigSerializer which we may want to go about differently to reuse here (ideally of course fix the data in XL to prevent e.g. broken IRIs from ever entering).

@niklasl
niklasl requested review from andersju and olovy July 10, 2026 15:42
@niklasl
niklasl force-pushed the feature/rdf-dump-writers branch from bc52236 to 897247f Compare August 17, 2026 14:34
@niklasl

niklasl commented Aug 17, 2026

Copy link
Copy Markdown
Member Author

(Rebased on develop to account for switch to slf4j.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant