Skip to content

Upload BC-Bench to repo and GH Page #813

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Sun Haoran (haoranpb) merged 6 commits into from
Aug 24, 2026
Merged

Upload BC-Bench to repo and GH Page #813

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
60d4287
prepare GitHub Page for paper link
haoranpb May 11, 2026
1264340
update the arxiv with real link
haoranpb Aug 24, 2026
963392e
Add citation section to RAEDME
haoranpb Aug 24, 2026
065c6ad
include latex source for future collaboration on version updates
haoranpb Aug 24, 2026
a0b8fe1
remove unintentional change to settings file
haoranpb Aug 24, 2026
83aaf05
fix local github page serve task
haoranpb Aug 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .devcontainer/devcontainer.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
"version": "3.3"
}
},
"postCreateCommand": "uv sync --all-groups && pre-commit install && cd docs && printf 'source \"https://rubygems.org\"\\ngem \"github-pages\", group: :jekyll_plugins\\ngem \"webrick\"\\n' > Gemfile && bundle install",
"postCreateCommand": "uv sync --all-groups && pre-commit install && cd docs && printf 'source \"https://rubygems.org\"\\ngem \"github-pages\", \"~> 232\", group: :jekyll_plugins\\ngem \"webrick\"\\n' > Gemfile && bundle install",
"customizations": {
"vscode": {
"extensions": [
Expand Down
17 changes: 17 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,20 @@ BC-Bench is open source, and you're welcome to fork and adapt it for your own us
- **[CONTRIBUTING.md](CONTRIBUTING.md)** — fork setup, repo layout, versioning, day-to-day maintainer ops
- **[EXPERIMENT.md](EXPERIMENT.md)** — run an experiment (toggle instructions / skills / agents / MCP / model) against an existing category
- **[CATEGORIES.md](CATEGORIES.md)** — add a new evaluation category alongside the existing `bug-fix` / `test-generation` / `code-review` / `nl2al`

## Citation

The [paper](https://arxiv.org/abs/2608.20851) and its [LaTeX source](paper/) describe BC-Bench's design and evaluation. If you use BC-Bench in your research, please cite:

```bibtex
@misc{sun2026bcbench,
title={{BC-Bench}: Evaluating Agentic Engineering in a Domain-Specific Language for ERP},
author={Sun, Haoran and Hansen, Klaus Marius},
year={2026},
eprint={2608.20851},
archivePrefix={arXiv},
primaryClass={cs.SE},
doi={10.48550/arXiv.2608.20851},
url={https://arxiv.org/abs/2608.20851}
}
```
47 changes: 47 additions & 0 deletions docs/_layouts/default.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
<!--
# Override of jekyll-theme-cayman's default layout to add an arXiv button.
# Copied verbatim from the cayman gem with one extra <a class="btn"> added
-->

<!DOCTYPE html>
<html lang="{{ site.lang | default: "en-US" }}">
<head>
<meta charset="UTF-8">

{% seo %}
<link rel="preconnect" href="https://fonts.gstatic.com">
<link rel="preload" href="https://fonts.googleapis.com/css?family=Open+Sans:400,700&display=swap" as="style" type="text/css" crossorigin>
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="theme-color" content="#157878">
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">
<link rel="stylesheet" href="{{ '/assets/css/style.css?v=' | append: site.github.build_revision | relative_url }}">
{% include head-custom.html %}
</head>
<body>
<a id="skip-to-content" href="#content">Skip to the content.</a>

<header class="page-header" role="banner">
<h1 class="project-name">{{ page.title | default: site.title | default: site.github.repository_name }}</h1>
<h2 class="project-tagline">{{ page.description | default: site.description | default: site.github.project_tagline }}</h2>
{% if site.github.is_project_page %}
<a href="{{ site.github.repository_url }}" class="btn">View on GitHub</a>
{% endif %}
<a href="https://arxiv.org/abs/2608.20851" class="btn" target="_blank" rel="noopener">Read the Paper (arXiv)</a>
{% if site.show_downloads %}
<a href="{{ site.github.zip_url }}" class="btn">Download .zip</a>
<a href="{{ site.github.tar_url }}" class="btn">Download .tar.gz</a>
{% endif %}
</header>

<main id="content" class="main-content" role="main">
{{ content }}

<footer class="site-footer">
{% if site.github.is_project_page %}
<span class="site-footer-owner"><a href="{{ site.github.repository_url }}">{{ site.github.repository_name }}</a> is maintained by <a href="{{ site.github.owner_url }}">{{ site.github.owner_name }}</a>.</span>
{% endif %}
<span class="site-footer-credits">This page was generated by <a href="https://pages.github.com">GitHub Pages</a>.</span>
</footer>
</main>
</body>
</html>
Loading
Loading