[GSoC'26] CodeGen Test Harness

[Shubh-Raj]

Problem Statement

The concerto-codegen package generates code for TypeScript, Java, C#, and Go, but there is no automated verification that the generated code compiles. The current test suite only checks text output against snapshots — it doesn't validate that the output is syntactically valid code.

Description

This project builds a compile-verification infrastructure for concerto-codegen — a CI-integrated quality assurance system that ensures generated code is always valid.

Core deliverables:

• Multi-language compilation pipeline in GitHub Actions (TypeScript, Java, C#)
• Containerized compiler environments for reproducible builds
• Comprehensive test model suite covering edge cases (enums, scalars, inheritance, reserved keywords)
• Extensible architecture for adding new target languages

This infrastructure transforms code generation from "hope it works" to "verified to compile" — catching regressions automatically and preventing broken releases from reaching users.

Expected Outcomes

1. GitHub Actions workflow with compile verification for TypeScript, Java, and C#
2. Containerized build environments for each target language
3. Edge-case test model fixtures covering language-specific quirks
4. Extensible harness design with documentation for adding new languages
5. Validation of harness by fixing existing known issues

Skills Required

• Required: JavaScript, GitHub Actions/CI pipelines, multi-language compilation basics
• Preferred: Docker, TypeScript, Java, C#

Possible Mentors

TBD

Expected Size

Medium (~175 hours)

Expected Difficulty

Medium

[ekarademir]

Would this require a separate project?

[Shubh-Raj]

Would this require a separate project?

Thanks for the question!

The test harness would not require a separate project — it would be integrated directly into concerto-codegen as:

1. New GitHub Actions workflow(s) that run compilation checks on generated code (TypeScript, Java, C#)
2. Test model fixtures (.cto files) in the existing test/ directory covering edge cases (enums, scalars, reserved keywords, inheritance)
3. Containerized compiler environments (using GitHub-hosted runners or Docker) for reproducible builds

This follows the pattern of the 2024 GSoC Rust/Alias project, which extended the existing repo rather than creating a new one.

That said, if maintainers prefer a separate verification repo (e.g., concerto-codegen-tests), I can adjust the proposal accordingly. What's your preference?

[sanketshevkar]

I'm not sure, if we could run containers with different runtimes on every PR or every merge, @mttrbrts this would add cost to our CI/CD pipeline right? But this could surely be script and a set of container images for each environment that someone else could trigger in their pipeline or workflow.

To sponsor a project we have problem of prioritisation due limited resources in terms mentors and their time. We'll I'll surely bring this up during our ideas list finalisation.

Thanks!