From eac85a4a83faeb6b75da594124ff3883f6baaa48 Mon Sep 17 00:00:00 2001 From: duckduckgrayduck <102841251+duckduckgrayduck@users.noreply.github.com> Date: Mon, 13 Jul 2026 13:00:40 -0500 Subject: [PATCH 1/2] Add documentation about setting up local OCR data --- README.md | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 7b8f77de..e22d6ac1 100644 --- a/README.md +++ b/README.md @@ -57,13 +57,23 @@ You must first have these set up and ready to go: 12. Go to [Django admin for DocumentCloud](https://api.dev.documentcloud.org/admin) and add the required static [flat page](https://api.dev.documentcloud.org/admin/flatpages/flatpage/) called `/tipofday/`. It can be blank. Do not prefix the URL with `/pages/`. Specifying the `Site` as `example.com` is alright. 13. Create an initial Minio bucket to simulate AWS S3 locally: - Run `inv initialize-minio` -14. Upload a document: +14. Download and setup Tesseract language data locally so that OCR works. + From within documentcloud/documents/processing/ocr/tesseract/tessdata/ run the following to download the trained data for the English language: + ```curl -L -o eng.traineddata https://github.com/tesseract-ocr/tessdata_fast/raw/main/eng.traineddata``` + You'll also need to install the orientation and script detection data: + ```curl -L -o osd.traineddata https://github.com/tesseract-ocr/tessdata_fast/raw/main/osd.traineddata``` + Lastly, you need to use the invoke command `inv manage upload_languages` to take the trained data from that directory and + place it in the correct location in the Minio bucket to test OCR in a local development environment. You may add additional languages as needed by modifying the + curl command for the language data above. +15. Upload a document: - **Check your memory allocation on Docker is at least 7gb.** A sign that you do not have enough memory allocated is if containers are randomly failing or if your system is swapping heavily, especially when uploading documents. - The "upload" button should not be grayed out (if it is, check your user organization Verified Journalist status above) - If you get an error on your console about signatures, fix minio as above. - If you get an error on your console about tipofday not found, add the static page as above. -15. Develop DocumentCloud and its frontend! -16. You can run the tests with `inv test`. +16. Develop DocumentCloud and its frontend! +17. Tests can be run using `inv test`. If a test doesn't exist for new functionality, please create tests for the new functionality to ensure test coverage. +Before submitting a pull request, ensure all code passes quality and formatting checks by running `inv format` and `inv pylint` and resolving any issues. + - If you want to run a subset of the tests, you can specify the directory containing the test you want with the `path` switch like so: `inv test --path documentcloud/documents`. - You can specify a single file in `--path` if you only want to run the tests in that file. From 616975c584a61ba562f7614ac451c5c3a6ef0eaf Mon Sep 17 00:00:00 2001 From: duckduckgrayduck <102841251+duckduckgrayduck@users.noreply.github.com> Date: Mon, 13 Jul 2026 13:03:16 -0500 Subject: [PATCH 2/2] Formatting attempt --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index e22d6ac1..75383701 100644 --- a/README.md +++ b/README.md @@ -58,11 +58,11 @@ You must first have these set up and ready to go: 13. Create an initial Minio bucket to simulate AWS S3 locally: - Run `inv initialize-minio` 14. Download and setup Tesseract language data locally so that OCR works. - From within documentcloud/documents/processing/ocr/tesseract/tessdata/ run the following to download the trained data for the English language: + - From within documentcloud/documents/processing/ocr/tesseract/tessdata/ run the following to download the trained data for the English language: ```curl -L -o eng.traineddata https://github.com/tesseract-ocr/tessdata_fast/raw/main/eng.traineddata``` - You'll also need to install the orientation and script detection data: + - You'll also need to install the orientation and script detection data: ```curl -L -o osd.traineddata https://github.com/tesseract-ocr/tessdata_fast/raw/main/osd.traineddata``` - Lastly, you need to use the invoke command `inv manage upload_languages` to take the trained data from that directory and + - Lastly, you need to use the invoke command `inv manage upload_languages` to take the trained data from that directory and place it in the correct location in the Minio bucket to test OCR in a local development environment. You may add additional languages as needed by modifying the curl command for the language data above. 15. Upload a document: