Skip to content

Adds a Ukrainian argument structure gallery example and supporting core fixes#10

Closed
pkuchmiichuk wants to merge 21 commits into
FACTSlab:mainfrom
pkuchmiichuk:ukrainian-pr
Closed

Adds a Ukrainian argument structure gallery example and supporting core fixes#10
pkuchmiichuk wants to merge 21 commits into
FACTSlab:mainfrom
pkuchmiichuk:ukrainian-pr

Conversation

@pkuchmiichuk

Copy link
Copy Markdown
Collaborator

Description

Adds a Ukrainian argument structure example to the gallery, plus the five bead/ core changes that building it required.

Gallery (gallery/ukr/argument_structure/)

An end-to-end 2AFC acceptability pipeline for Ukrainian verb argument structure, following the structure of gallery/eng/ and gallery/kor/ (korean branch): a Makefile, a local config.yaml, and numbered pipeline stages covering lexicon generation, template generation, template filling, 2AFC pair construction, list partitioning, and jsPsych/JATOS deployment.

Ukrainian specifics worth noting:

  • Lexicons are built from VESUM Unimorph partition rather than standard UniMorph, via a UniMorphAdapter subclass in utils/vesum_adapter.py.
  • Pairs contrast morphological case, since Ukrainian argument structure is case-marked. Case forms are filtered to unambiguous ones.
  • Verb selection is frequency-based, driven by a committed wordfreq-derived table (resources/verb_frequencies.csv). build_frequencies.py regenerates it, but is meant to be run rarely so that the pipeline does not depend on wordfreq at run time.
  • Scoring uses a causal LM for final scoring.

Core changes to bead/

Change Files
ListConstraint subclasses now default constraint_type to their own discriminator value, so constraints no longer have to restate it at construction. bead/lists/constraints.py
ListPartitioner warns once per expression when a constraint property expression cannot be evaluated, and once per type when a constraint type is not enforced during assignment. Previously a misconfigured constraint scored zero every iteration and was silently ignored. bead/lists/partitioner.py
Causal LMs load in the checkpoint's own dtype by default (dtype="auto"), and the requested dtype is passed from LanguageModelScorer through to the adapter. Loading a half-precision checkpoint as float32 doubled its memory. bead/items/adapters/huggingface.py, bead/items/scoring.py
UniMorphAdapter gained an overridable _load_dataset hook so a subclass can read a non-standard file layout. bead/resources/adapters/unimorph.py

All four core changes are backward compatible. The new parameters are keyword arguments with defaults preserving existing behavior, and the constraint defaults only add a value where one previously had to be supplied.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Refactoring (no functional changes)
  • Tests (adding or updating tests)

Checklist

  • I have read the CONTRIBUTING guidelines
  • My code follows the project's style guidelines
  • I have run uv run ruff check . and uv run ruff format .
  • I have run uv run pyright with no errors
  • I have added tests that prove my fix/feature works
  • All tests pass (uv run pytest tests/)
  • I have updated documentation as needed

Testing

Six test files cover the core changes:

Test file Covers
tests/lists/test_constraints.py Each constraint type constructs without an explicit constraint_type, and the discriminator still round-trips.
tests/lists/test_partitioner.py The unresolved-expression warning fires once per expression.
tests/lists/test_partitioner_batch.py The unenforced-type warning fires once per constraint type.
tests/items/adapters/test_huggingface.py The adapter forwards dtype to from_pretrained and defaults to "auto".
tests/items/test_scoring.py LanguageModelScorer passes its dtype through to the adapter.
tests/resources/adapters/test_unimorph.py A subclass overriding _load_dataset is used in place of the module-level loader.

Note on gallery test coverage: gallery/ is excluded from ruff, pyright, and testpaths, so the Ukrainian pipeline scripts are not covered by CI, matching how gallery/eng/ and gallery/kor/ (korean branch) are currently treated.

@aaronstevenwhite

Copy link
Copy Markdown
Collaborator

Can you pull the core changes into a separate PR from the gallery addition and target the core changes at main but the gallery addition at ukrainian?

@pkuchmiichuk

Copy link
Copy Markdown
Collaborator Author

Split into two as requested: core changes target main in #11 , Ukrainian gallery targets ukrainian in #12 .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants