MASFly

Official implementation of MAS-on-the-Fly: In-Context Structural Adaptation of LLM-Based Multi-Agent Systems, accepted by EMNLP 2026.

Authors: Guangyi Liu, Haojun Lin, Huan Zeng, Heng Wang, and Quanming Yao

Paper: https://arxiv.org/pdf/2602.13671

Overview

MASFly enables LLM-based multi-agent systems to adapt their collaboration structure in context at test time. This repository contains the core framework, experiment implementations, SOP repositories, and evaluation utilities used in the paper.

Requirements

• Go 1.24.1 or later
• Python dependencies required by the benchmark-specific evaluation scripts
• Access to an OpenAI-compatible API endpoint

Quick Start

1. Install dependencies

go mod tidy

2. Configure the model API

Set the following environment variables. Never commit real credentials to the repository.

export OPENAI_API_KEY="your-api-key"
export OPENAI_MODEL="your-model-name"
export OPENAI_BASE_URL="https://your-openai-compatible-endpoint/v1"

OPENAI_BASE_URL can be omitted when using the default OpenAI endpoint.

3. Run TravelPlanner evaluation

Download the TravelPlanner database and extract it to dataset/travelplanner.

Then run:

cd examples/multi-agent-example/travelplanner
mkdir -p log output
go run . --eval 2  # Use the built-in SOP repository

4. Evaluate generated results

cd check
# Update the target result path in the evaluation script before running it.
bash eval.sh

5. Build an SOP repository from training data

To build the SOP Repository and Personalized Experience Pool from training data:

# Run on the training set.
go run . --eval 0

# Evaluate generated results.
cd check
bash eval.sh

# Distill reflection experience.
cd ../SOP
go run .

Other benchmark implementations are available under examples/multi-agent-example.

Citation

If you find this work useful, please cite:

@inproceedings{liu2026masonthefly,
  title     = {{MAS-on-the-Fly}: In-Context Structural Adaptation of {LLM}-Based Multi-Agent Systems},
  author    = {Liu, Guangyi and Lin, Haojun and Zeng, Huan and Wang, Heng and Yao, Quanming},
  booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing},
  year      = {2026},
  url       = {https://arxiv.org/abs/2602.13671}
}

The citation metadata may be updated when the final proceedings version becomes available.

Acknowledgments

This project is built on Ant Group's AIEvo framework. We thank the AIEVO contributors for providing the underlying agent framework.

We also thank the creators and maintainers of the TravelPlanner, GAIA, HumanEval, and MBPP benchmarks and their associated resources. Their work makes reproducible evaluation of LLM-based agent systems possible.

Security

API credentials are read from environment variables and must not be committed to source control. If a credential is accidentally committed, revoke and rotate it immediately; removing it only from the latest revision is not sufficient because it may remain in Git history.

License

This project is released under the Apache License 2.0.