LIPFusion

Official PyTorch implementation of

LIPFusion: A Lightweight Information-Preserving Feature Fusion Module for Hybrid CNN–Transformer Medical Image Segmentation

LIPFusion is a lightweight feature fusion module designed for hybrid CNN–Transformer medical image segmentation. It replaces the original BiFusion module of TransFuse with a computationally efficient information-preserving fusion strategy, aiming to reduce computational complexity while preserving segmentation performance.

---

Highlights

• Lightweight feature fusion module
• Plug-and-play replacement for BiFusion
• Reduced computational complexity
• Lower GPU memory consumption
• Faster inference
• Fully compatible with the original TransFuse framework
• PyTorch implementation

---

Model Overview

Figure 1. Overall architecture of the proposed LIPFusion framework.

The proposed LIPFusion architecture preserves the original parallel CNN–Transformer design of TransFuse while replacing the computationally expensive BiFusion module with a lightweight information-preserving feature fusion block.

---

Repository Structure

LIPFusion
│
├── configs/
├── data/
├── datasets/
├── imgs/
│   └── model.png
├── models/
├── pretrained/
├── snapshots/
├── utils/
│
├── process.py
├── train_isic.py
├── test_isic.py
├── requirements.txt
└── README.md

---

Requirements

• Python >= 3.8
• PyTorch >= 1.6.0
• timm == 0.3.2
• torchvision
• numpy
• scipy
• opencv-python

Install all required packages using

pip install -r requirements.txt

---

Dataset Preparation

ISIC2017 Skin Lesion Segmentation Challenge

Download the ISIC2017 dataset from

https://challenge.isic-archive.com/data

Place the downloaded dataset inside

./data

Then preprocess the dataset by running

python process.py

This generates

data_train.npy
mask_train.npy

data_val.npy
mask_val.npy

data_test.npy
mask_test.npy

---

Pretrained Models

Before training, download the pretrained backbone models.

DeiT-Small

Official repository:

https://github.com/facebookresearch/deit

Place the downloaded checkpoint inside

./pretrained

---

ResNet-34

Download from

https://download.pytorch.org/models/resnet34-333f7ec4.pth

Place the checkpoint inside

./pretrained

---

Training

Train the model using

python train_isic.py

Training hyperparameters can be modified directly inside the training script.

---

Evaluation

Evaluate the trained model using

python test_isic.py --ckpt_path snapshots/LIPFusion_best.pth

---

Results

LIPFusion is designed to preserve complementary local and global representations while significantly reducing the computational overhead introduced by bilinear feature interaction.

Compared with the original BiFusion module, the proposed LIPFusion module aims to

• Reduce computational complexity
• Reduce GPU memory consumption
• Reduce the number of trainable parameters
• Improve inference efficiency
• Maintain competitive segmentation performance

Detailed quantitative and qualitative experimental results are reported in the accompanying paper.

---

Built Upon TransFuse

This project is developed based on the excellent TransFuse framework.

If you use this repository, please also consider citing the original TransFuse paper.

Original repository:

https://github.com/Rayicer/TransFuse

---

Acknowledgements

We sincerely thank Yundong Zhang, Huiye Liu, and Qiang Hu for making the TransFuse implementation publicly available.

This work would not have been possible without their valuable contribution to the medical image segmentation community.

---

Citation

If you find this repository useful for your research, please cite the original TransFuse paper.

@article{zhang2022transfuse,
  title={TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation},
  author={Zhang, Yundong and Liu, Huiye and Hu, Qiang},
  journal={Medical Image Analysis},
  volume={81},
  pages={102615},
  year={2022},
  publisher={Elsevier}
}

If you use LIPFusion, please cite our paper as well.

@article{amiridoumari2026lipfusion,
  title={LIPFusion: A Lightweight Information-Preserving Feature Fusion Module for Hybrid CNN--Transformer Medical Image Segmentation},
  author={Mohammadmahdi Amiri Doumari and Fatemeh Afsari},
  journal={Under Review},
  year={2026}
}

---

License

This project is released under the MIT License.

---

Contact

Mohammadmahdi Amiri Doumari

Department of Computer Engineering

Shahid Bahonar University of Kerman

Iran

For questions, suggestions, or collaborations, please open an Issue or submit a Pull Request.