A face similarity application that estimates whether two people are likely to be related based on their facial features.
Live demo: AreWeRelated
Upload two photos, and AreWeRelated estimates the likelihood that the two people are related. The model is trained on the Recognizing Faces in the Wild (RFIW/FIW) kinship dataset and uses facial embeddings extracted with InsightFace.
The application consists of three main steps:
The model was trained using the Recognizing Faces in the Wild (RFIW/FIW) dataset. The dataset contains labeled kinship pairs from hundreds of families, including relationships such as:
- Parent–child
- Siblings
The original dataset provides related pairs. To create the negative class, I wrote a script that generates unrelated pairs by combining images from different families. This results in a binary classification problem:
1→ Related0→ Unrelated
For each uploaded image, I use a pre-trained InsightFace model to extract:
- A 512-dimensional face embedding
- 68 3D facial landmarks
The face embedding captures high-level facial characteristics, while the landmarks describe the geometry of different facial regions.
I experimented with six similarity features:
- Overall face embedding similarity
- Jawline similarity
- Eyebrow similarity
- Eye similarity
- Nose similarity
- Mouth similarity
For each pair of images, the similarity between corresponding features is calculated using cosine similarity.
I trained a Logistic Regression classifier using these similarity features. To evaluate the model while preventing identity and family leakage, I used 5-fold family-grouped cross-validation. In each split, all images belonging to a family remain entirely within either the training or validation set. This prevents the model from seeing the same person's face, or another member of the same family, during training and evaluation.
I initially expected the landmark-based facial region features to improve the prediction compared with using the face embedding alone. However, the experiment showed otherwise:
| Features | Mean ROC-AUC |
|---|---|
| Embedding similarity only | 0.800 |
| Embedding + 5 landmark similarities | 0.799 |
The five landmark-based features provided no measurable improvement over embedding similarity alone. The ROC curves were also almost completely overlapping.
Based on this experiment, the deployed model is: InsightFace → 512D face embeddings → Cosine Similarity → Logistic Regression
The landmark-based similarity calculations remain in the codebase. They are still used for visualizing facial regions on the uploaded images and remain available for anyone who wants to continue experimenting with them, but they are not used by the deployed classifier.
cd backend
python3 -m venv .env
.env/bin/pip install -r requirements.txt
.env/bin/uvicorn app.main:app --reload --port 8000In another terminal:
cd frontend
npm install
npm run devThe frontend expects the API to be available at:
http://localhost:8000
MIT — see LICENSE.