Synonym

A human-oriented pdf explorer

I made synonym because I had a shit ton of pdfs, and wanted to categorize them based on their content. Synonym is a multi label classifier built to do exactly that.

This is a quick and dirty claude frontend thrown over the (in hindsight quite limited) model, but we all start somewhere

Usage

If you're on linux/mac and have a posix compliant shell (Bash):

./run.sh

This'll set up a venv, download the requirements, and then

• Start the Analysis server running the model
• Launch a frontend to then visualize that output.

Manual Run (Windows, or if you'd rather run each half yourself)

1. Run the local server (for PDF analysis)

I highly reccomend setting up a venv for this. If you already have one, activate it and follow from the next command over.

source ~/.venv/bin/activate

cd server
pip install -r requirements.txt

python/pipeline.py automatically downloads the NLTK and other modules / embedder it needs.

Now: navigate back to the repo root and launch the server.

cd ..
#back to the root of repo

python -m uvicorn server.main:app --reload --port 8000

(GENRE_MODEL_PATH defaults to python/genre_classifier.pkl if you omit it.)

Check it's up: curl http://localhost:8000/api/health

2. Run the frontend

npm install
npm run dev

Open the printed local URL. The sidebar shows a connection dot for the server (default http://localhost:8000 click the URL to change it if you're running the server elsewhere or on a different port).

Final out:

npm run build   # outputs to dist/

NOTE: SOMETIMES SOME OBJECTS DON'T GET REAPED PROPERLY AND YOU HAVE A RESIDUAL PROCESS LISTENING ON THE PORT

FIX: Jank but works

ss -ltnp | grep 8000
# will show some process pid 

kill <pid of process>

Showcase :

Project structure

├─ run.sh
├── frontend.log
├── index.html
├── python
│   ├── firstrun.py
│   ├── genre_classifier.pkl
│   ├── pipeline.py
├── README.md
├── run.sh
├── server
│   ├── __init__.py
│   ├── main.py
│   └── requirements.txt
├── server.log
├── src
│   ├── App.jsx
│   ├── components
│   │   ├── AxisControls.jsx
│   │   ├── DetailPanel.jsx
│   │   ├── FileViewer.jsx
│   │   ├── Ledger.jsx
│   │   ├── ScatterPlot.jsx
│   │   └── ScoreTable.jsx
│   ├── lib
│   │   ├── backend.js
│   │   └── results.js
│   ├── main.jsx
│   └── styles.css
└── vite.config.js

Thoughts and takeaways:

I really only built this because of being less into my work and more into pdf hoarding. The model is a very simple multi label classifier, trained on a 10k annotated book dataset.

Due to which, has a lot of very weird genres. And weirder associations. As an example, model thinks that a lot of math is fiction. Probably because of the way of writing(?)

However, it is quite performant. Much of the processing time for larger pdfs is spent in ingestion, and not inference.

Will definitely revisit this, to , like we did last time, build this into a specific app. I prefer the terminal more and hence stripped it down to what i would keep, but it definitely isn't as user friendly.

Will also try and retrain the model, not into a different architecture, but definitely on a better dataset.

If you have any features to suggest/ bugs to report please feel free to raise a PR or submit issues!