Skip to content

[sub-issue #53] Implement Tooling #54

Description

@emapuljak

Sub-issue - Tooling foundation, community files, bug fixes & gitignore

Description

Establish the packaging/tooling configuration and repository hygiene that everything else builds
on, plus the low-risk pre-existing fixes. This is the first of three PRs (A → B → C) and must be
green on arrival.

Scope = sub-issues 1 (packaging & tooling config), 4 (community health files),
6 (pre-existing bug fixes) and 7 (.gitignore cleanup).

Out of scope (later PRs): wiring Ruff/mypy/bandit into pre-commit and CI (PR B), and the
autodoc / docstring-backfill / MyST-Markdown docs overhaul (PR C).

⚠️ Do not enable the Ruff D (docstring) rule in this PR. The codebase has ~230 docstrings
across ~900 functions, so turning D on now would flood ruff check with errors on untouched
code. It gets enabled in PR C together with the docstring backfill.


Tasks

1. Packaging & tooling config (pyproject.toml)

  • Add [tool.ruff]line-length = 125, target-version = "py310", src = ["src","tests"].
  • Add [tool.ruff.lint] with the agreed rule select excluding the D family, the ML ignores (E501, N803, N806, D100, D104), per-file ignores (tests/** = ["D","ARG"], __init__.py = ["F401"]), and [tool.ruff.lint.pydocstyle] convention = "google".
  • Add [tool.mypy] (files = ["src","tests"], ignore_missing_imports = true, no_strict_optional = true, check_untyped_defs = true).
  • Add [tool.bandit] (exclude_dirs = ["tests","build","docs"]).
  • Add [tool.pytest.ini_options] (testpaths = ["tests"], addopts = "-ra --strict-markers").
  • Add [tool.coverage.run] (source = ["src"], branch = true) and [tool.coverage.report] (show_missing = true, fail_under = custom) where custom means number that you get when running coverage check now - 40% for now.
  • Add pytest-cov to the test extra.
  • Add a docs extra mirroring docs/requirements.txt (sphinx, furo/sphinx_rtd_theme, myst-parser, sphinx-autodoc-typehints, …).
  • Add a dev extra: ruff, mypy, bandit[toml], pre-commit, nbmake, ipykernel + [test,docs].
  • Replace packages = ["pquant"] with [tool.setuptools.packages.find] where = ["src"].
  • Add [tool.setuptools.package-data] to ship configs/*.yaml and finetuning.yaml.
  • Run the one-time ruff format . + ruff check --fix . reformat as its own commit.

2. Community health files + changelog

  • Add CONTRIBUTING.md (contribution workflow: issue → branch → PR → review).
  • Add CHANGELOG.md at repo root — Keep a Changelog + SemVer, seeded with ## [Unreleased] and the current release version. (Created here so the CI changelog gate added in PR B has something to check.)
  • Add docs/source/changelog.md that pulls in the root file via MyST include (```{include} ../../CHANGELOG.md ```) and add changelog to the toctree in docs/source/index.rst.

3. Pre-existing bug fixes

  • tox.ini: [testenv] extras = testingtest.
  • .readthedocs.yaml: sphinx.configuration: docs/conf.pydocs/source/conf.py; uncomment the python.install docs-requirements block.
  • docs/Makefile: sphinx-apidoc ... ../src/HGQ../src/pquant; align the sphinx-build.yml workflow to build against docs/source/.
  • README.md: fix links to non-existent docs/pruning_methods.md / docs/quantization_parameters.md.
  • Remove stray root __init__.py (accidental; real package is src/pquant).
  • docs/source/conf.py: read version from package metadata instead of hardcoded release = "1.0.0".
  • Delete [all] optional dependencies because it's not being used - in future version there will be a matrix for running tests on different backends
  • src/pquant/core/torch/layers.py: load_torch_hgq_model() calls torch.load(path_to_checkpoint) without weights_only=True, so loading a checkpoint unpickles arbitrary objects and can execute arbitrary code (CWE-502; bandit B614:pytorch_load, Medium severity). The function only needs a state_dict (plain tensors), so pass weights_only=True. Must be fixed before the bandit gate to CI implementation.

4. .gitignore cleanup

  • Add tool caches/artifacts: .ruff_cache/, .mypy_cache/, .pytest_cache/, .coverage, coverage.xml, htmlcov/.
  • Narrow the blanket *.txt (e.g. to logs/*.txt) or remove it and ignore intended files explicitly; confirm docs/requirements.txt stays tracked.

5. Expand the README with installation, backend selection, quick start and standard sections.

  • Add: Installation (Python 3.10+, backend required — plain pip install pquant-ml is not usable), Selecting the backend (export KERAS_BACKEND=torch, must be set before import), Quick start (pdp_config()add_compression_layers()train_model()), plus Contributing / License sections linking to the new CONTRIBUTING.md, CHANGELOG.md and LICENSE.

Acceptance Criteria

  • ruff check . and ruff format --check . pass at line-length = 125 (with the D rule excluded).
  • [tool.ruff], [tool.mypy], [tool.bandit], [tool.pytest.ini_options], [tool.coverage.*] are all present in pyproject.toml.
  • packages = ["pquant"] is replaced with [tool.setuptools.packages.find] where = ["src"].
  • python -m build produces a wheel containing all subpackages (pquant.core, pquant.pruning_methods, pquant.data_models, pquant.configs) and the configs/*.yaml + finetuning.yaml files (verify with unzip -l dist/*.whl).
  • test extra includes pytest-cov; new docs and dev extras exist and resolve (pip install -e ".[dev]" succeeds).
  • CONTRIBUTING.md, and a seeded CHANGELOG.md ([Unreleased] section) exist at repo root.
  • docs/source/changelog.md includes the root changelog via MyST {include} and appears in the toctree.
  • tox.ini uses extras = test.
  • .readthedocs.yaml points to docs/source/conf.py and installs docs/requirements.txt.
  • sphinx-build.yml builds against docs/source/.
  • README.md broken links are fixed.
  • Stray root __init__.py is removed.
  • docs/source/conf.py reads its version from package metadata (no hardcoded 1.0.0).
  • .gitignore ignores .ruff_cache/, .mypy_cache/, .pytest_cache/, .coverage, coverage.xml, htmlcov/.
  • The blanket *.txt ignore is narrowed/removed and docs/requirements.txt remains tracked.
  • The one-time ruff format reformat is a separate commit from the config changes.
  • Update README
  • - run mypy and fix all errors before implementing CI pipeline

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions