Skip to content

New runs between running git-write.js and one of the scoring scripts will fail #157

Description

@foolip

The problem is that first we do this to populate the most recent runs:

node git-write.js --max-time=300 --max-age-days=5

The in various scoring scripts, we use the wpt.fyi API to enumerate runs:

const alignedRuns = await lib.runs.fetchAlignedRunsFromServer(
products, from, to, experimental);

And then fail if any of those aren't in the "cache":

if (!localRunIds.has(run.id)) {
// If you see this, you probably need to run git-write.js or just update
// your results-analysis-cache.git repo; see the README.md.
console.error(`Run ${run.id} missing from local git repo (${date})`);
hadErrors = true;
}

This happened in https://github.com/web-platform-tests/results-analysis/actions/runs/4240081460/jobs/7368750202

This doesn't happen often, but it will happen occasionally.

A better setup would be one of:

  • Assume the cache is sufficiently up-to-date, don't try to update it, and ignore any run not in it
  • Don't run git-write.js at all, and instead handle cache misses by creating the tree from the full wptreport.json. This would work without the results-analysis-cache repo, just very slowly.
  • Get the runs from the wpt.fyi API once, ensure those are in the cache, and then run scoring scripts (storing the runs in some intermediate JSON file)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions