Skip to content

[Reproducibility] Request for the complete official CIA release and experiment assets #2

Description

@zjh529

Hi authors,

Thank you for releasing CIA. I am trying to reproduce the official implementation first, rather than reimplementing the method from the paper.

I audited a clean clone of commit 9655cc732de85c7678a76b4e4ee9b39b48b38faf. Several author-specific components required for faithful reproduction cannot be uniquely recovered from the paper or public upstream repositories.

Could you please provide or clarify the following?

1. Complete runnable official source

Please publish a corrected commit, branch, or archive in which the CIA entrypoints, model, trainer, and data processor are mutually consistent. The current snapshot has missing imports, incompatible model/trainer signatures, and undefined variables in the semantic-model entrypoint. Local guesses would no longer be an unambiguous official reproduction.

2. Exact Reasoning Output Induction implementation

Please provide the exact prompts and injection points for the Cumulative-Propagation, Task-Focused, and Predecessor-Review constraints, the expected JSON schema, and the CIA-specific preprocessing functions.

The code imports gsm_data_process_adversial, svamp_data_process_adversial, and record_to_input_adversial, but these are absent from both the CIA snapshot and public upstream G-Designer. This is related to #1. One official example/cached reasoning-output file would be enough to validate the parser.

3. Official topology/checkpoint assets

For a minimal smoke test, please provide one G-Designer + GSM8K checkpoint or generated topology. For exact table reproduction, please provide the actual G-Designer/AGP/ARG-Designer assets, or the exact upstream commits, configurations, task selections, and seeds. LLM-based regeneration from approximate settings may not recreate the paper's topologies.

4. Official ablation definitions

Please provide the exact definitions or runnable flags for CIA full, CIA w/o GBD, and CIA w/o LWS. The w/o GBD representation and remaining losses cannot be uniquely determined from the released materials.

5. Training and evaluation protocol

Please clarify:

  • whether CIA is trained separately for each graph/task;
  • epoch count and model-selection rule;
  • whether ground-truth AUC selects the best epoch;
  • whether the decision node is included;
  • how symmetric scores become directed edges;
  • which candidate edges are used for AUC/ACC/F1/FPR;
  • how metrics are aggregated.

For Tables 1/3/4/5, the exact 100-task subsets, AGP/ARG-Designer assets, Table 3 ground-truth intermediate outputs, baseline configurations/caches, and FPR protocol would also be helpful.

Public raw datasets, public model weights, API credentials, and generic metric formulas are not being requested. A pinned environment file would be helpful but is not the main blocker.

A minimal official smoke test would already be valuable: G-Designer + GSM8K, 5–20 samples, one official topology/checkpoint, predicted edges, and AUC/ACC/F1.

If a fuller branch, archive, or newer release exists, could you please link it?

Thank you!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions