A model that flags fraudulent card transactions, wrapped in the machinery that keeps it reproducible — a DVC pipeline from raw data to a scored model, experiment tracking in MLflow, an HTTP API around the result, and CI that tests and releases it.
Fraud is 0.17 per cent of the data, so accuracy is meaningless here and the choice of metric is the substance of the problem. Recall on the fraud class comes first, because a missed fraud is the expensive error; precision on the same class is watched next, since every false alarm becomes manual work; F1 keeps the two honest against each other; and PR-AUC is used to compare models, being far more informative than ROC-AUC at this level of imbalance.
Four stages defined in dvc.yaml, each declaring its inputs, parameters and outputs so that
dvc repro reruns only what has actually changed.
| Stage | What it does |
|---|---|
load_data |
Splits the raw dataset into train and test |
preprocess |
Builds the feature and target files and pins the column order |
train |
Fits a random forest, logs the run to MLflow, writes the model and the metrics |
Everything the run depends on lives in params.yaml, the split, the forest of 400 trees at
depth 12, and the MLflow experiment, so a change of hyperparameters is a change of one file.
mlops/api/main.py loads the trained model and exposes it over FastAPI, with /health and
a /predict that accepts a transaction either as named features or as a plain vector of
thirty values. The image is built by the release workflow on a tag and pushed to Docker Hub;
CI on every push runs flake8, the tests with coverage, and pip-audit over the dependencies.
make up
make predict-fraudThe notebooks under notebooks/ hold the exploratory work and the comparison that led to
the forest, and every figure it produced is drawn per class, so that the shape of the
fraudulent minority can be read against the legitimate majority.
Correlations within each class. The per-feature distributions, box plots and scatter plots
are in reports/figures/Class, left as files because each is a
single column of panels thousands of pixels tall.
make requirements
dvc repro