CVRIE

CVRIE is a two-part applied machine-learning project built around healthcare-related data.

It combines:

• a supervised pipeline for binary chest X-ray classification (NORMAL vs PNEUMONIA)
• an unsupervised pipeline for clustering short patient testimonies into coherent symptom-language groups

The repository is organized as a reproducible analytical codebase rather than a single notebook dump. Each project section contains:

• a research notebook
• reusable Python modules under src/
• generated artifacts such as metrics, figures, and models

Project Overview

1. Supervised learning

The supervised track evaluates whether classical machine learning can classify chest X-rays while avoiding common dataset traps such as:

• patient leakage across splits
• duplicate images across subsets
• augmentation bias
• class-correlated image geometry shortcuts

Main outcome:

• best pipeline: normalized preprocessing + HOG + RBF SVM
• final image-level holdout performance:
  • Accuracy: 0.9607
  • Balanced accuracy: 0.9506
  • F1-score: 0.9727
  • ROC-AUC: 0.9906
• final patient-level ROC-AUC: 0.9949

This part is not just a classifier benchmark. It is also a leakage audit and a stricter re-evaluation of a widely reused medical-imaging dataset.

Detailed documentation: supervised/README.md

2. Unsupervised learning

The unsupervised track clusters short, unlabelled patient testimonies using only text content.

It includes:

• dataset structure repair
• linguistic preprocessing
• TF-IDF vectorization
• latent semantic projection with TruncatedSVD
• clustering comparison across multiple algorithms
• stability analysis with Adjusted Rand Index
• cluster interpretation

Main outcome:

• best pipeline: extended preprocessing + TF-IDF + TruncatedSVD(50) + KMeans(k=12)
• final metrics:
  • Silhouette: 0.0982
  • Calinski-Harabasz: 30.2734
  • Davies-Bouldin: 2.9877
• stability across 10 runs:
  • Mean ARI: 0.7000
  • ARI std: 0.0548

The final clusters capture recurring themes such as respiratory symptoms, headaches, rashes, burns, limb pain, and blood-sugar-related complaints.

Detailed documentation: unsupervised/README.md

Repository Structure

cvrie/
|- data/
|  |- chest_xray/
|  `- Student_Dataset.csv
|- supervised/
|  |- artifacts/
|  |- notebooks/
|  |- src/
|  `- README.md
|- unsupervised/
|  |- artifacts/
|  |- notebooks/
|  |- src/
|  `- README.md
|- requirements.txt
`- README.md

Datasets

Supervised dataset

• Source: Kaggle Chest X-Ray Images (Pneumonia)
• Original file count: 8,530
• Leakage-aware rebuilt split:
  • Train: 6,016 images
  • Validation: 1,266 images
  • Test: 1,248 images

Unsupervised dataset

• Source file: data/Student_Dataset.csv
• Rows: 1,011
• Structural repairs applied: 397 shifted rows corrected

Technical Highlights

• modular Python implementation instead of notebook-only logic
• reproducible preprocessing and inference helpers
• saved metrics and analysis artifacts
• explicit attention to data quality, leakage, and evaluation design
• classical ML and NLP methods used with strong experimental discipline

Installation

Install dependencies from the repository root:

pip install -r requirements.txt

How To Explore The Project

Start with the section README files:

1. supervised/README.md
2. unsupervised/README.md

Then open the notebooks:

jupyter notebook supervised/notebooks/supervised.ipynb
jupyter notebook unsupervised/notebooks/unsupervised.ipynb

Key Takeaways

• On medical-image tasks, careful split design and leakage control are as important as model choice.
• On short-text clustering tasks, internal metrics alone are not enough; stability and interpretability matter.
• In both tracks, the most valuable output is not only the final score, but the rigor of the methodology used to obtain it.

Scope And Caution

This repository is an academic and engineering project.

• The supervised model is not a clinical diagnostic tool.
• The unsupervised clusters are not diagnoses.
• Neither pipeline should be used for medical decision-making.