Skip to content

Pull requests: mlcommons/endpoints

New pull request New
31 Open 277 Closed
31 Open 277 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

feat(swe-bench): bind each unit to an endpoint by shard index size/normal PR Review Policy: <=500 non-test lines & <=20 files
#477 opened Aug 26, 2026 by arekay-nv Collaborator • 9/9 Loading…
10 tasks
1
fix(swe-bench): validate model_params before reading generation settings size/normal PR Review Policy: <=500 non-test lines & <=20 files
#476 opened Aug 26, 2026 by arekay-nv Collaborator • 8/9 Loading…
10 tasks
1
feat(swe-bench): SWEBenchFleetScorer - fan out units across a service fleet size/large PR Review Policy: 501-1500 lines or 21-50 files
#475 opened Aug 26, 2026 by arekay-nv Collaborator • 7/9 Loading…
10 tasks
2
fix(swe-bench): fingerprint endpoint identity, not the time of asking size/normal PR Review Policy: <=500 non-test lines & <=20 files
#474 opened Aug 26, 2026 by arekay-nv Collaborator • 6/9 Loading…
10 tasks
1
feat(swe-bench): pre-dispatch gates that must prove their own scale size/normal PR Review Policy: <=500 non-test lines & <=20 files
#473 opened Aug 26, 2026 by arekay-nv Collaborator • 5/9 Loading…
10 tasks
3
feat(swe-bench): eval-phase infra-vs-genuine error classifier size/normal PR Review Policy: <=500 non-test lines & <=20 files
#472 opened Aug 26, 2026 by arekay-nv Collaborator • 4/9 Loading…
10 tasks
1
feat(swe-bench): conservative claim reaper + PID-only memory guard size/large PR Review Policy: 501-1500 lines or 21-50 files
#471 opened Aug 26, 2026 by arekay-nv Collaborator • 3/9 Loading…
10 tasks
2
feat(swe-bench): all-or-nothing merge gate scoped to exactly one run id size/normal PR Review Policy: <=500 non-test lines & <=20 files
#470 opened Aug 26, 2026 by arekay-nv Collaborator • 2/9 Loading…
10 tasks
1
feat(swe-bench): content-addressed unit plan + durable mkdir-atomic work queue size/large PR Review Policy: 501-1500 lines or 21-50 files
#469 opened Aug 26, 2026 by arekay-nv Collaborator • 1/9 Loading…
10 tasks
4
[docs] Benchmark suite and ruleset proposal size/normal PR Review Policy: <=500 non-test lines & <=20 files
#468 opened Aug 25, 2026 by arekay-nv Collaborator Loading…
10 tasks
1
fix: fail fast on endpoint response stalls size/normal PR Review Policy: <=500 non-test lines & <=20 files
#462 opened Aug 22, 2026 by roborluo Collaborator Loading…
9
fix(lcb-service): pin base image to Python 3.11 to match LiveCodeBench size/normal PR Review Policy: <=500 non-test lines & <=20 files
#457 opened Aug 19, 2026 by liayan Member Loading…
10 tasks
3
fix(swebench-service): give Pyxis container creation its own deadline size/normal PR Review Policy: <=500 non-test lines & <=20 files
#456 opened Aug 19, 2026 by leopck Collaborator Loading…
10 tasks
2
fix(benchmark): an accuracy run that produces no number must fail size/normal PR Review Policy: <=500 non-test lines & <=20 files
#454 opened Aug 19, 2026 by leopck Collaborator Loading…
1 of 4 tasks
2
fix(benchmark): don't reject an accuracy-only run for listing many endpoints size/normal PR Review Policy: <=500 non-test lines & <=20 files
#453 opened Aug 19, 2026 by leopck Collaborator Loading…
1 of 4 tasks
2
fix(swebench-service): let srun find its config and honour proxy policy size/normal PR Review Policy: <=500 non-test lines & <=20 files
#452 opened Aug 19, 2026 by leopck Collaborator Loading…
1 of 4 tasks
6
docs: steady-state metrics reporting design proposal size/very-large PR Review Policy: >1500 lines or >50 files
#447 opened Aug 13, 2026 by nv-alicheng Collaborator Loading…
2 of 10 tasks
32
Added Conversational Aware Routing for Dynamo support
#434 opened Jul 30, 2026 by leopck Collaborator • Draft
3
fix(lcb-service): distinguish infra crashes from submission failures in grading children size/normal PR Review Policy: <=500 non-test lines & <=20 files
#433 opened Jul 29, 2026 by liayan Member Loading…
4 of 7 tasks
57
Add License Scanning and Reporting for Endpoints Changes on Main.
#412 opened Jul 14, 2026 by arav-agarwal2 Collaborator Loading…
4 of 10 tasks
18
feat :Testing workflow to push images size/large PR Review Policy: 501-1500 lines or 21-50 files
#405 opened Jul 9, 2026 by arekay-nv Collaborator Loading…
10 tasks
11
fix(endpoint-client): retry pre-response connection resets (opt-in)
#404 opened Jul 8, 2026 by Palanivelg Contributor Loading…
3 tasks done
18
DRAFT: feat(power): vendor-agnostic power/energy monitoring
#383 opened Jun 30, 2026 by viraatc Collaborator Loading…
15
Feat/gptoss accuracy agentic
#376 opened Jun 26, 2026 by tianmu-li Collaborator • Draft
10 tasks
5
Feat: Support creation of run_metadata.json at the end of benchmark run
#374 opened Jun 23, 2026 by anandhu-eng Contributor Loading…
5 of 10 tasks
10
ProTip! Exclude everything labeled bug with -label:bug.