---

Abstract

Synth4bench is an open-source framework for benchmarking tumor-only somatic variant calling algorithms using synthetic next-generation sequencing data with known ground truth. Built on the NEAT simulator, it generates controlled synthetic datasets and provides a reproducible end-to-end pipeline for data generation, preprocessing, variant calling and performance evaluation. The framework supports Mutect2, FreeBayes, LoFreq, VarDict and VarScan2, enabling systematic comparison across different sequencing conditions. By combining controlled low-allele-frequency variant generation with comprehensive benchmarking metrics, synth4bench offers a flexible and extensible platform for studying variant caller performance and improving benchmarking practices in genomics.

---

Table of Contents

• Abstract
• Description
• Installation
• Data Download
• Execution
• Documentation
• Contribute
• Citation

---

Description

Synth4bench addresses the challenge of benchmarking tumor-only somatic variant callers, particularly for low-allele-frequency (≤10%) variants, by providing synthetic datasets with known ground truth. The framework includes:

1. Synthetic data generation using the NEAT simulator to create controlled, reproducible NGS datasets.
2. Three independent benchmarking analyses for SNV true variants, SNV noise and small indels.
3. Comprehensive benchmarking of five widely used variant callers: Mutect2, FreeBayes, LoFreq, VarDict and VarScan2.
4. Systematic performance evaluation using metrics such as Precision, Recall, allele frequency estimation (ΔAF), and detailed indel classification.

Synth4bench provides a reproducible and extensible platform for studying variant caller behavior and benchmarking new methods under controlled sequencing conditions.

---

Data Download

All data are openly available on Zenodo. For specific instructions, refer to our User Guide.

---

Installation

1. Create the Conda environment:

   conda env create -f environment.yml
   conda activate synth4bench

2. Install NEAT v3.3:

   Download version v3.3.
   To call the main script:

   python gen_reads.py --help

   For further details, see the NEAT README included in the download.

3. Install bam-readcount:

   Follow their installation instructions.
   After building, verify installation:

   build/bin/bam-readcount --help

   If you encounter issues during the make process, you can alternatively use the executable available here and place it in the bam-readcount/build/bin folder.

4. Download VarScan Extra Script:

   The extra script vscan_pileup2cns2vcf.py for VarScan is available here.

---

Execution

Simply configure your parameters in the parameters.yaml file, then execute:

bash s4b_run.sh

This single command generates synthetic data, runs variant calling for all selected tools, and performs downstream analysis and plotting.

For full execution instructions, see our User Guide.

---

Documentation

For further documentation, visit the documentation page.

---

Contribute

We welcome and greatly appreciate any feedback or contributions!

If you have questions, please open an issue here or email sfragkoul@certh.gr.

---

Citation

Our work has been publised in Frontiers in Bioinformatics. If you use synth4bench, or any of our scripts/code, please cite:

Fragkouli S-C, Pechlivanis N, Anastasiadou A, Karakatsoulis G, Orfanou A, Kollia P, Agathangelidis A and Psomopoulos F (2026), Synth4bench: generating synthetic data for benchmarking tumor-only somatic variant calling algorithms. Front. Bioinform. 6:1858375. doi: 10.3389/fbinf.2026.1858375

---