"Latest" riskcore data (only) - #12
Draft
aclark02-arcus wants to merge 23 commits into
Draft
Conversation
… speed up load times
Some scored/assessed bundles were missing the `label` attribute on `pkg_score`-classed columns (e.g. `has_examples` in cran bundle 03), which caused dplyr::bind_rows() to fail via vctrs with "Some attributes are incompatible". Add a harmonize_bundle_attrs() helper that unions attributes across the per-bundle list before binding, filling in any missing attributes from the first bundle that supplied them. Applied to both the assessed and scored bundle lists in repo_united(). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fix repo_united() bind_rows attribute mismatch for pkg_score columns
Previously only `.../<ver>/bioc` (the software subrepo, ~2.3k pkgs) was listed, so available.packages() capped Bioconductor at ~2,300 pkgs instead of the expected ~3,699 for BioC 3.22. Add the annotation, experiment, and workflows subrepos so all Bioconductor pkgs are picked up. Also broaden the post-filter from "bioc" to "bioconductor" so the annotation/experiment/workflows URLs (which don't contain the substring "bioc" in their path) are included alongside BioCsoft. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Include all four Bioconductor subrepos so BioC pkg count isn't capped at ~2.3k
riskmetric::pkg_ref() returns pkg_missing entries for packages not found in the configured repos (e.g. Bioconductor sub-repos not enabled). The downstream as_tibble.list_of_pkg_ref calls vapply(x, function(xi) as.character(xi), character(1L)) which errors with 'values must be length 1, but FUN(X[[1]]) result is length 0' because pkg_missing refs have no resolvable version. Detect and drop pkg_missing refs (with a message listing them) before calling dplyr::as_tibble() and riskmetric::pkg_assess(). If a batch contains no resolvable refs, skip it cleanly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Instead of dropping pkg_missing refs entirely, split them out before riskmetric assess/score (which would crash on their length-0 version), run the pipeline on the resolvable refs, and then append flagged rows for the missing packages to both the assessed and scored bundles: - New logical column 'pkg_missing' on both bundles (FALSE for normally assessed/scored rows, TRUE for rows added back for missing packages). - Missing rows have version = NA and pkg_score = NA (scored bundle). - If an entire batch is pkg_missing, emit a flagged-only bundle rather than failing. This preserves the original goal of representing every requested package in the output while clearly annotating those that couldn't be resolved in the enabled repos. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…sing
New argument `keep_missing` (default TRUE) on incrmt_repo():
- TRUE = retain pkg_missing packages in the final assessed/scored bundles
as flagged rows (pkg_missing = TRUE, version/pkg_score = NA).
- FALSE = drop them entirely from the output; the pkg_missing column is
still present (all FALSE) so bundle schemas remain consistent.
Log message now reflects the chosen action ('Flagging' vs 'Dropping').
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When called with a single-element pkg_names, riskmetric::pkg_ref() returns a bare pkg_ref (an environment) rather than a list_of_pkg_ref. The subsequent vapply(ass_repo00, function(xi) inherits(xi, 'pkg_missing'), logical(1L)) then iterated over the environment's fields instead of packages, so pkg_missing detection silently found nothing and the pipeline re-encountered the original vapply length-0 crash inside as_tibble.list_of_pkg_ref. Wrap the single ref in vctrs::new_list_of(..., class = 'list_of_pkg_ref') so the rest of the function handles the single- and multi-package cases identically. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Packages that live only in a Bioc *data* sub-repo (annotation / experiment / workflows) pass riskmetric's bioc availability check — their Repository URL matches a Bioc mirror URL — so the ref is created with class 'pkg_bioc_remote' rather than 'pkg_missing'. However pkg_bioc() looks up versions only against the release *software* PACKAGES file, so 'xi$version' resolves to character(0) and the downstream vapply in as_tibble.list_of_pkg_ref crashes with the original 'values must be length 1, but FUN(X[[1]]) result is length 0' error. Broaden the detection to any ref with a zero-length or NA version (still catching pkg_missing too). Update log message accordingly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
strip_recording() converts EVERY column of the assessed bundle into a list-column (each row wrapped in list(), including package and version). Previously-run batches were saved that way, so repo_united()'s dplyr::bind_rows() rejected the newly-added missing rows because their package column was <character> rather than <list>: Error in dplyr::bind_rows(): ! Can't combine ..1$package <list> and ..9$package <character>. Build missing_assessed with list-columns for package and version (and give the empty all-missing skeleton the same shape) so bundles combine cleanly across batches. Scored bundle continues to use character package (pkg_score never runs strip_recording). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
riskmetric::pkg_score() attaches a 'pkg_score' S3 class (with a label attribute) to the summary pkg_score column. My earlier fix appended missing-package rows with pkg_score = NA_real_ (plain numeric), which silently downgraded the batch's pkg_score column to plain numeric. Downstream repo_united() then tried to bind_rows() this batch with previously-run batches whose pkg_score column is still <pkg_score>, producing: Error in dplyr::bind_rows(): ! Can't combine ..1 <pkg_score> and ..9 <vctrs:::common_class_fallback>. Use vctrs::vec_c() with a zero-length prototype from the existing scored_repo$pkg_score column so the NA values inherit the same class and attributes. Falls back to copying attributes manually if vec_c() can't reconcile. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Appending flagged missing rows inside each batch is fragile: riskmetric
decorates many columns (all metric cols plus pkg_score) with the
'pkg_score' S3 class and attributes, and strip_recording() turns every
assessed-side column into a list-column. Naively binding NA-filled rows
downgrades those column types, so a later cross-batch dplyr::bind_rows()
in repo_united() fails with vctrs common_class_fallback errors like:
Can't combine ..1 <pkg_score> and ..9 <vctrs:::common_class_fallback>.
Refactor so:
incrmt_repo()
- Saves only resolvable packages to <repo>_(assessed|scored)_bundle_.rds
(schema untouched from riskmetric).
- When all packages in a batch are pkg_missing, skips assess/score
entirely and writes no bundle file (only the missing-names file).
- When keep_missing = TRUE and there are unresolved packages, writes
the character vector to <repo>_missing_<label>.rds.
repo_united()
- Discovers batch labels from filenames so gaps (all-missing batches)
are handled.
- After combining resolvable bundles, calls append_missing_bundle_rows()
which loads all _missing_*.rds files and appends one flagged row per
package using vctrs::vec_init() with the fully-combined bundle's per-
column prototypes. This preserves S3 classes and attributes so no
downgrade occurs.
- Adds pkg_missing = FALSE on real rows, TRUE on flagged rows.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Preserve pkg_missing packages in incrmt_repo() output with a flag column
…2026-08-15 snapshot date.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Creating version of 'riskscore' that only contains the latest data to speed up load times.
Install with: