Vscreen

A parallel pipeline for mining antibody / nanobody V-domain repertoires out of raw NGS reads. It takes a FASTQ file of amplicon reads, annotates the V(D)J architecture with IgBLAST, groups reads into clonal families, and emits a consensus amino-acid sequence per family — ranked so the most abundant clones come first.

The whole thing is one self-contained class, AntibodyPipeline, so you can either run the end-to-end pipeline or call any single stage from your own code.

Pipeline stages

#	Method	What it does
1	quality_filter_fastq()	Drops reads below a mean Phred threshold (default 30) or shorter than a minimum length (default 100 nt)
2	split_fastq()	Round-robin splits the filtered reads into num_chunks files, so chunks stay balanced regardless of where the good reads sit in the file
3	run_igblast_parallel()	Fans the chunks out across a ProcessPoolExecutor, one single-threaded igblastn call per chunk, then merges the outputs
4	parse_igblast_output()	Scrapes query ID, V gene, J gene, CDR3, and the full V-domain nucleotide sequence into a pandas DataFrame
5	cluster_sequences()	Buckets records by (V gene, J gene) and then splits each bucket by exact CDR3 identity to define clonal families
6	reconstruct_consensus_parallel()	Per cluster, takes the most frequent V-domain nucleotide sequence as the consensus and translates it; results sorted by cluster size
7	output_results()	Writes top_root_sequences.fasta (all consensus sequences) plus one FASTA per cluster containing the consensus and every member read

Parallelism shows up twice: IgBLAST annotation (step 3) and consensus reconstruction (step 6). Both are process-based, so they get around the GIL.

Install

pip install -r requirements.txt

You also need the NCBI IgBLAST binary (igblastn) on your system, plus germline databases built from IMGT reference sequences:

# macOS
brew install igblast
# or download from https://ftp.ncbi.nih.gov/blast/executables/igblast/release/

Germline reference data

IMGT reference FASTA files are not redistributed in this repo — IMGT's terms require you to obtain them yourself. Download the V-QUEST reference directory from https://www.imgt.org/vquest/refseqh.html, then build BLAST databases:

makeblastdb -parse_seqids -dbtype nucl -in IGHV.fasta -out immunoglobulin_db_V
makeblastdb -parse_seqids -dbtype nucl -in IGHD.fasta -out immunoglobulin_db_D
makeblastdb -parse_seqids -dbtype nucl -in IGHJ.fasta -out immunoglobulin_db_J

The pipeline appends _V / _D / _J to whatever you pass as db, so the databases above correspond to db="immunoglobulin_db".

Usage

Command line:

python app/pipeline_oop.py reads.fastq

Or configure it from Python (see app/work.py):

from pipeline_oop import AntibodyPipeline

pipeline = AntibodyPipeline(
    fastq_file="reads.fastq",
    working_dir="pipeline_work",
    igblast_path="/usr/local/bin/igblastn",
    db="immunoglobulin_db",
    num_chunks=50,
)
pipeline.run()

Everything intermediate and final lands in working_dir.

Knobs worth knowing

• num_chunks (default 50) sets both the chunk count and the process-pool size for IgBLAST. Set it near your core count rather than leaving it at 50 on a laptop.
• quality_filter_fastq(quality_threshold=..., min_length=...) — call it directly instead of via run() if you want non-default filtering.
• output_results(..., top_n=2000) caps how many clusters get their own FASTA.
• The IgBLAST call hardcodes -organism human and -outfmt 7; edit run_igblast_chunk() for other species.

Tests

Nine unit tests cover every stage, using tmp_path fixtures and synthetic records — no IgBLAST install or reference data needed. test_run_igblast_chunk replaces the method with a stub that writes a canned report, so nothing shells out to igblastn.

cd app && python -m pytest -q

.........                                    [100%]
9 passed

Layout

app/
  pipeline_oop.py   the AntibodyPipeline class + CLI entry point
  work.py           minimal example driver
  tests/            one test module per pipeline stage

Caveats

• Clustering uses exact CDR3 matching within a V/J bucket. That's strict: clones separated by a single somatic mutation or sequencing error in CDR3 land in different families. Swap in an edit-distance threshold if you need lineage-level grouping.
• parse_igblast_output() reads the human-readable IgBLAST report line by line. If you change -outfmt, the parser needs to change with it.
• Consensus is the modal full-length nucleotide sequence, not a position-by-position vote — deliberate, since it always returns a real observed sequence, but it means a cluster with no repeated read returns an arbitrary member.

License

MIT — see LICENSE.