title	MONOLITH

MONOLITH Hybrid Search Ranking

This repository contains the complete codebase for the Intelligent Candidate Discovery & Ranking Challenge.

The system ranks a pool of 100,000 candidates for a Senior AI Engineer (Founding Team) role at Redrob AI, balancing deep semantic matching, technical competence matching, and behavioral availability signals, all while respecting the constraints of running offline in under 5 minutes on a CPU.

---

Project Structure

AI_Recruiter_Project/
│
├── data/                               # Source dataset directory 
│   ├── candidates.jsonl                # 100k candidates database (not uploaded to GitHub)
│   └── job_description.docx            # Target job description
│
├── models/                             # Pre-downloaded model cache 
│   └── all-MiniLM-L6-v2/               # Local SentenceTransformer weights
│
├── artifacts/                          # Pre-computed offline features & cache
│   ├── candidate_text.parquet          # Normalized profile textual representations
│   ├── features.parquet                # Extracted tabular profiles & behavioral signals
│   └── candidate_embeddings.npy        # pre-computed dense embeddings (Float32)
│
├── src/                                # Pipeline source code
|   ├── init.py                         # Package initialization
│   ├── parser.py                       # Parses job_description.docx
│   ├── preprocess.py                   # Preprocesses candidates.jsonl into plain text profiles
│   ├── features.py                     # Feature engineering, stuffer & honeypot detection
│   ├── embeddings.py                   # Generates BGE/MiniLM embeddings locally
│   ├── ranker.py                       # Core hybrid score ranker & dynamic weight engine
│   ├── explainability.py               # Generates customized candidate reasonings
│   └── utils.py                        # Common dates and consulting firm helpers
│
├── outputs/                            # Target output directory
│   └── MONOLITH.csv                    # final top-100 ranked candidates
│
├── run.py                              # Unified CLI entrypoint
├── validate_submission.py              # Verification tool for format constraints
├── requirements.txt                    # Project package dependencies
└── submission_metadata.yaml            # Team metadata file

Architecture Design

1. Precomputation Stage (Run Once)

The offline preprocessing pipeline runs to extract features and generate dense embeddings, which are cached as local artifacts. This guarantees the online matching runs within the strict CPU/time constraints:

• Preprocessing: Clean candidate summary, headline, and career history to build a unified profile text.
• Feature Extraction: Computes total YOE, education scores, and maps out the 23 behavioral signals.
• Honeypot/Trap Detection: Filters impossible profiles (e.g. expert proficiency with 0 months used, start dates preceding company founding dates). Flagged profiles receive a score of -999.0 so they never appear in the top-100 shortlist.
• Embedding Generation: Encodes candidate profiles using the lightweight, local all-MiniLM-L6-v2 model.

2. Ranking Stage

Runs end-to-end in under 2 minutes on CPU:

• Job Description Parsing: Parses the target requirements (locations, experience, skills) from job_description.docx.
• Semantic Retrieval: Encodes the JD and performs a vector dot product against all 100k pre-cached candidate embeddings.
• Dynamic Tabular Scorer: Scores candidates based on Noida/Pune location compatibility, target YOE fit (5-9 years), notice period, and platform active dates.
• IT Services / Stuffer Filters: Penalizes resumes from service-based consulting firms (TCS, Wipro, Infosys, etc.) and keyword stuffers with non-tech titles (Marketing, HR, Sales).
• lexicographical tie-breaker: Sorts equal-scoring candidates by candidate_id in ascending order.
• Explainability Generator: Populates custom, candidate-specific 1-2 sentence reasons detailing experience, matching skills, and notice period constraints.

---

Getting Started

1. Setup Data

Place your dataset files inside the data/ folder : The full candidates.jsonl dataset (~475 MB) is not included in this repository due to size constraints. Place the dataset in: data/candidates.jsonl before running the pipeline. You can also download it from here : "Link "

• candidates.jsonl
• job_description.docx

---

2. Installation

Option A: Using a Virtual Environment (Recommended)

# Create a virtual environment
python -m venv AIenv

# Activate it (Windows)
.\AIenv\Scripts\activate

# Activate it (macOS/Linux)
source AIenv/bin/activate

# Install the required packages
pip install -r requirements.txt

Option B: Installing Globally (No Virtual Environment)

If you prefer to run the project without a virtual environment, you can install the dependencies directly:

pip install -r requirements.txt

(Note: Use pip install --user -r requirements.txt if you encounter permission errors).

How to Run the Pipeline

Step 1: Run Pipeline (Precomputation Part Run Once)

Extract features and build dense embedding cache:

python run.py --mode offline

Step 2: Run Online Ranking (Sandbox Mode)

Run the matching engine and generate the final ranked shortlist:

python run.py --candidates ./data/candidates.jsonl --out ./outputs/MONOLITH.csv

Step 3: Validate the Submission

Verify that the output CSV conforms exactly to the hackathon validation rules:

python validate_submission.py ./outputs/MONOLITH.csv

---

Running the Web UI (Streamlit)

To launch the interactive dashboard in your web browser:

streamlit run app.py

(This starts a local development server at http://localhost:8501 where you can upload candidate profiles and job descriptions in real-time).

---