From 336ca66fb7663d14bdeed15d548525172ec9ee4e Mon Sep 17 00:00:00 2001 From: laughingman7743 Date: Tue, 11 Aug 2026 12:32:44 +0900 Subject: [PATCH] Add Codex project instructions --- AGENTS.md | 108 +++++++++++++++++++++++++++++++++++++++++++++++++++++ CLAUDE.md | 109 +----------------------------------------------------- 2 files changed, 109 insertions(+), 108 deletions(-) create mode 100644 AGENTS.md mode change 100644 => 120000 CLAUDE.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..67a5fc53 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,108 @@ +# PyAthena Development Guide for AI Assistants + +## Project Overview + +PyAthena is a Python DB API 2.0 (PEP 249) compliant client for Amazon Athena. See `pyproject.toml` for Python version support and dependencies. + +## Rules and Constraints + +### Git Workflow + +- **NEVER** commit directly to `master` — always create a feature branch and PR +- Create PRs as drafts: `gh pr create --draft` + +### Import Rules + +- **NEVER** use runtime imports (inside functions, methods, or conditional blocks) +- All imports must be at the top of the file, after the license header +- Exception: the existing codebase uses runtime imports for optional dependencies (`pyarrow`, `pandas`, etc.) in source code. For new code, use `TYPE_CHECKING` instead when possible + +### Code Quality — Always Run Before Committing + +```bash +just format # Auto-fix formatting and imports +just lint # Lint + format check + mypy +``` + +### Testing + +```bash +# ALWAYS run `just lint` first — tests will fail if lint doesn't pass +just test pyathena # Unit tests (runs lint first) +just test sqla # SQLAlchemy dialect tests +just test sqla-async # SQLAlchemy async dialect tests +``` + +Tests require AWS environment variables. Use a `.env` file (gitignored): + +```bash +AWS_DEFAULT_REGION= +AWS_ATHENA_S3_STAGING_DIR=s3://// +AWS_ATHENA_WORKGROUP= +AWS_ATHENA_SPARK_WORKGROUP= +``` + +```bash +export $(cat .env | xargs) && uv run pytest tests/pyathena/test_file.py -v +``` + +- Tests mirror source structure under `tests/pyathena/` +- Use pytest fixtures from `conftest.py` +- New features require tests; changes to SQLAlchemy dialects must pass `just test sqla` + +#### Test Conventions + +- **Class-based tests** for integration tests that use fixtures (cursors, engines): `class TestCursor:` with methods like `def test_fetchone(self, cursor):` +- **Standalone functions** for unit tests of pure logic (converters, parsers, utils): `def test_to_struct_json_formats(input_value, expected):` +- Test file naming mirrors source: `pyathena/parser.py` → `tests/pyathena/test_parser.py` +- **Fixtures**: Cursor/engine fixtures are defined in `conftest.py` and injected by name (e.g., `cursor`, `engine`, `async_cursor`). Use `indirect=True` parametrization to pass connection options: + + ```python + @pytest.mark.parametrize("engine", [{"driver": "rest"}], indirect=True) + def test_query(self, engine): + engine, conn = engine + ``` + +- **Parametrize** with `@pytest.mark.parametrize(("input", "expected"), [...])` for data-driven tests +- **Integration tests** (need AWS) use cursor/engine fixtures with real Athena queries; **unit tests** (no AWS) call functions directly with test data + +### Markdown Lint + +`docs/**/*.md` and project-root `*.md` files are linted with [markdownlint-cli2](https://github.com/DavidAnson/markdownlint-cli2). The config lives at `.markdownlint-cli2.jsonc`. CI runs lint + Sphinx build on PRs that touch docs (`.github/workflows/docs-lint.yaml`). + +`markdownlint-cli2` is pinned in `.mise.toml`, so [`mise`](https://mise.jdx.dev/) installs the exact version used in CI. Run locally: + +```bash +mise install # one-time: installs markdownlint-cli2 +just docs lint # check +just docs format # auto-fix what's possible +just docs build # build the Sphinx site under docs/_build/html +``` + +## Architecture — Key Design Decisions + +These are non-obvious conventions that can't be discovered by reading code alone. + +### PEP 249 Compliance + +All cursor types must implement: `execute()`, `fetchone()`, `fetchmany()`, `fetchall()`, `close()`. New cursor features must follow the DB API 2.0 specification. + +### Cursor Module Pattern + +Each cursor type lives in its own subpackage (`pandas/`, `arrow/`, `polars/`, `s3fs/`, `spark/`) with a consistent structure: `cursor.py`, `async_cursor.py`, `converter.py`, `result_set.py`. When adding features, consider impact on all cursor types. + +### Filesystem (fsspec) Compatibility + +`pyathena/filesystem/s3.py` implements fsspec's `AbstractFileSystem`. When modifying: + +- Match `s3fs` library behavior where possible (users migrate from it) +- Use `delimiter="/"` in S3 API calls to minimize requests +- Handle edge cases: empty paths, trailing slashes, bucket-only paths + +### Version Management + +Versions are derived from git tags via `hatch-vcs` — never edit `pyathena/_version.py` manually. + +### Google-style Docstrings + +Use Google-style docstrings for public methods. See existing code for examples. diff --git a/CLAUDE.md b/CLAUDE.md deleted file mode 100644 index 67a5fc53..00000000 --- a/CLAUDE.md +++ /dev/null @@ -1,108 +0,0 @@ -# PyAthena Development Guide for AI Assistants - -## Project Overview - -PyAthena is a Python DB API 2.0 (PEP 249) compliant client for Amazon Athena. See `pyproject.toml` for Python version support and dependencies. - -## Rules and Constraints - -### Git Workflow - -- **NEVER** commit directly to `master` — always create a feature branch and PR -- Create PRs as drafts: `gh pr create --draft` - -### Import Rules - -- **NEVER** use runtime imports (inside functions, methods, or conditional blocks) -- All imports must be at the top of the file, after the license header -- Exception: the existing codebase uses runtime imports for optional dependencies (`pyarrow`, `pandas`, etc.) in source code. For new code, use `TYPE_CHECKING` instead when possible - -### Code Quality — Always Run Before Committing - -```bash -just format # Auto-fix formatting and imports -just lint # Lint + format check + mypy -``` - -### Testing - -```bash -# ALWAYS run `just lint` first — tests will fail if lint doesn't pass -just test pyathena # Unit tests (runs lint first) -just test sqla # SQLAlchemy dialect tests -just test sqla-async # SQLAlchemy async dialect tests -``` - -Tests require AWS environment variables. Use a `.env` file (gitignored): - -```bash -AWS_DEFAULT_REGION= -AWS_ATHENA_S3_STAGING_DIR=s3://// -AWS_ATHENA_WORKGROUP= -AWS_ATHENA_SPARK_WORKGROUP= -``` - -```bash -export $(cat .env | xargs) && uv run pytest tests/pyathena/test_file.py -v -``` - -- Tests mirror source structure under `tests/pyathena/` -- Use pytest fixtures from `conftest.py` -- New features require tests; changes to SQLAlchemy dialects must pass `just test sqla` - -#### Test Conventions - -- **Class-based tests** for integration tests that use fixtures (cursors, engines): `class TestCursor:` with methods like `def test_fetchone(self, cursor):` -- **Standalone functions** for unit tests of pure logic (converters, parsers, utils): `def test_to_struct_json_formats(input_value, expected):` -- Test file naming mirrors source: `pyathena/parser.py` → `tests/pyathena/test_parser.py` -- **Fixtures**: Cursor/engine fixtures are defined in `conftest.py` and injected by name (e.g., `cursor`, `engine`, `async_cursor`). Use `indirect=True` parametrization to pass connection options: - - ```python - @pytest.mark.parametrize("engine", [{"driver": "rest"}], indirect=True) - def test_query(self, engine): - engine, conn = engine - ``` - -- **Parametrize** with `@pytest.mark.parametrize(("input", "expected"), [...])` for data-driven tests -- **Integration tests** (need AWS) use cursor/engine fixtures with real Athena queries; **unit tests** (no AWS) call functions directly with test data - -### Markdown Lint - -`docs/**/*.md` and project-root `*.md` files are linted with [markdownlint-cli2](https://github.com/DavidAnson/markdownlint-cli2). The config lives at `.markdownlint-cli2.jsonc`. CI runs lint + Sphinx build on PRs that touch docs (`.github/workflows/docs-lint.yaml`). - -`markdownlint-cli2` is pinned in `.mise.toml`, so [`mise`](https://mise.jdx.dev/) installs the exact version used in CI. Run locally: - -```bash -mise install # one-time: installs markdownlint-cli2 -just docs lint # check -just docs format # auto-fix what's possible -just docs build # build the Sphinx site under docs/_build/html -``` - -## Architecture — Key Design Decisions - -These are non-obvious conventions that can't be discovered by reading code alone. - -### PEP 249 Compliance - -All cursor types must implement: `execute()`, `fetchone()`, `fetchmany()`, `fetchall()`, `close()`. New cursor features must follow the DB API 2.0 specification. - -### Cursor Module Pattern - -Each cursor type lives in its own subpackage (`pandas/`, `arrow/`, `polars/`, `s3fs/`, `spark/`) with a consistent structure: `cursor.py`, `async_cursor.py`, `converter.py`, `result_set.py`. When adding features, consider impact on all cursor types. - -### Filesystem (fsspec) Compatibility - -`pyathena/filesystem/s3.py` implements fsspec's `AbstractFileSystem`. When modifying: - -- Match `s3fs` library behavior where possible (users migrate from it) -- Use `delimiter="/"` in S3 API calls to minimize requests -- Handle edge cases: empty paths, trailing slashes, bucket-only paths - -### Version Management - -Versions are derived from git tags via `hatch-vcs` — never edit `pyathena/_version.py` manually. - -### Google-style Docstrings - -Use Google-style docstrings for public methods. See existing code for examples. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 120000 index 00000000..47dc3e3d --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +AGENTS.md \ No newline at end of file