load_h5 accepts a dataset only when it is 1-D, or 2-D with a leading singleton axis, which is squeezed (data/loaders/h5.py:143-145). Anything else with ndim > 1 raises a ValueError from inside the per-dataset loop (h5.py:146-152), so the first per-residue array aborts the whole run rather than skipping that protein — confirmed for (L, d) with L varying, (L, d) with L constant, and a 3-D (1, 3, 4).
Two things follow that seem worth pinning down.
Document the shape rules. They are not stated in the input-format docs, and no test covers them (grepping the error string across apps/protspace/tests returns nothing), so the behaviour is unpinned. A short paragraph plus a few parametrized tests over (L,d), (1,d), (d,) and 3-D would fix that. The (1, d) squeeze deserves a decision of its own: a per-residue array for a single-residue protein, or any pooled vector still carrying a batch axis, is currently accepted silently as a per-protein embedding.
Consider pooling instead of erroring. Per-residue HDF5 is the default output of several embedding tools, so users have to pre-pool by hand. The repo already has pool_residues (data/embedding/local.py:155), but it lives inside local inference and is unreachable from a user-supplied H5. If pooling on load is too implicit, an explicit opt-in (e.g. --pool-residues mean) with the current error as the default would remove the manual step while keeping the failure loud.
Context: this came up while verifying a manuscript sentence that reads "per-residue arrays are rejected". That is accurate today, but "rejected" understates it — the run aborts — and the (1, d) case is an exception to it.
load_h5accepts a dataset only when it is 1-D, or 2-D with a leading singleton axis, which is squeezed (data/loaders/h5.py:143-145). Anything else withndim > 1raises aValueErrorfrom inside the per-dataset loop (h5.py:146-152), so the first per-residue array aborts the whole run rather than skipping that protein — confirmed for(L, d)with L varying,(L, d)with L constant, and a 3-D(1, 3, 4).Two things follow that seem worth pinning down.
Document the shape rules. They are not stated in the input-format docs, and no test covers them (grepping the error string across
apps/protspace/testsreturns nothing), so the behaviour is unpinned. A short paragraph plus a few parametrized tests over(L,d),(1,d),(d,)and 3-D would fix that. The(1, d)squeeze deserves a decision of its own: a per-residue array for a single-residue protein, or any pooled vector still carrying a batch axis, is currently accepted silently as a per-protein embedding.Consider pooling instead of erroring. Per-residue HDF5 is the default output of several embedding tools, so users have to pre-pool by hand. The repo already has
pool_residues(data/embedding/local.py:155), but it lives inside local inference and is unreachable from a user-supplied H5. If pooling on load is too implicit, an explicit opt-in (e.g.--pool-residues mean) with the current error as the default would remove the manual step while keeping the failure loud.Context: this came up while verifying a manuscript sentence that reads "per-residue arrays are rejected". That is accurate today, but "rejected" understates it — the run aborts — and the
(1, d)case is an exception to it.