Climate Visibility Prediction

An end-to-end MLOps project that predicts maximum visibility distance from weather parameters (temperature, humidity, wind speed/direction, sea-level pressure)

Project Overview

The objective is to develop a machine learning model that accurately predicts maximum visibility distance for a given location and weather condition, using parameters such as humidity, temperature, wind speed, and atmospheric pressure. The model is trained on historical weather/visibility data and served through a web interface, aiming to improve safety and efficiency in aviation, transportation, and outdoor activities.

Tech Stack

Layer	Technology
Language	Python 3.10
ML	scikit-learn, XGBoost
API	FastAPI (+ Flask-compatible structure)
Database	MongoDB (data source)
Containerization	Docker
Local deployment	FastAPI + local model store

Project Architecture

The pipeline follows the flow from the project's architecture diagram:

Data Validation -> Data Transformation -> Preprocessing -> Model Building -> Model Pusher
        -> Local model file -> Prediction API -> Client

Data ingestion pulls from MongoDB (Ds-1/2/3 in the data-collection diagram conceptually correspond to different source batches); in this repo it's simplified to a single MongoDB collection, with a synthetic data generator as an offline fallback so the whole pipeline can run without any live infrastructure.

Project Structure

climate-visibility/
├── app.py                        # FastAPI web app (prediction UI)
├── main.py                       # Training pipeline entrypoint
├── setup.py
├── requirements.txt
├── Dockerfile
├── .dockerignore
├── .env.example
├── config/
│   ├── schema.yaml
│   └── model.yaml
├── src/
│   ├── components/                # data_ingestion, data_validation,
│   │                               # data_transformation, model_trainer,
│   │                               # model_pusher
│   ├── configuration/              # mongo_db_connection
│   ├── constant/training_pipeline/  # all pipeline constants
│   ├── entity/                       # config_entity, artifact_entity
│   ├── exception/                     # custom exception class
│   ├── logger/                         # logging setup
│   ├── ml/
│   │   ├── model/estimator.py          # ClimateVisibilityModel wrapper
│   │   └── metric/regression_metric.py # RMSE / MAE / R2
│   ├── pipeline/
│   │   ├── training_pipeline.py         # orchestrates all components
│   │   └── prediction_pipeline.py        # loads model, serves /predict
│   └── utils/main_utils.py                # yaml/pickle/numpy IO helpers
├── templates/                              # index.html, result.html
├── static/style.css
├── notebooks/EDA.ipynb                      # dataset exploration
├── tests/                                    # pytest smoke tests
├── artifacts/                                 # per-run pipeline outputs (gitignored)
└── saved_models/                               # local model store the app loads from

Flow of the Project (matches the demo flow)

1. Introduction
2. Environment Setup
3. Dataset Exploration — notebooks/EDA.ipynb
4. Data Preprocessing — src/components/data_validation.py, data_transformation.py
5. Feature Engineering — numerical pipeline in data_transformation.py; derived-feature examples in the notebook
6. Model Training & Evaluation — src/components/model_trainer.py (Linear Regression, Random Forest, XGBoost; RMSE/MAE/R²)
7. End-to-End Pipeline Run — main.py / src/pipeline/training_pipeline.py
8. FastAPI app — app.py
9. Docker — Dockerfile
10. Local Model Store — saved_models/
11. Version Control — Git/GitHub, .gitignore

Setup & Installation

1. Clone and create environment

git clone https://github.com/PWskills-DataScienceTeam/Climate-Visibility.git
cd Climate-Visibility
conda create -n climate-visibility python=3.10 -y
conda activate climate-visibility
pip install -r requirements.txt

2. Configure environment variables

cp .env.example .env
# then fill in MONGODB_URL if needed

If MONGODB_URL isn't set, the pipeline automatically falls back to a synthetically generated (but physically plausible) climate dataset, so you can run everything end-to-end without any live database.

3. Run the training pipeline

python main.py

This runs Data Ingestion → Data Validation → Data Transformation → Model Trainer → Model Pusher, and saves the final model to saved_models/model.pkl.

4. Run the web app

uvicorn app:app --host 0.0.0.0 --port 8062 --reload

Navigate to http://localhost:8062, enter weather parameters, and click Predict the Climate Visibility — matching the demo screenshots in the project deck.

You can also trigger training from the browser/API via GET /train.

5. Run with Docker

docker build -t climate-visibility .
docker run -p 8062:8062 --env-file .env climate-visibility

Running Tests

pytest tests/ -q

tests/test_pipeline.py runs a full smoke test of the pipeline on synthetic data end-to-end (ingestion → prediction).

Libraries Used

See requirements.txt for the full pinned list, matching the libraries table from the project deck: dill, dnspython, evidently, fastapi, from-root, httptools, imbalanced-learn, pip-chill, pymongo, jinja2, python-dotenv, uvicorn, watchfiles, websockets, wincertstore (Windows only), xgboost, python-multipart, neuro_mf, kneed, plus core data-science and Flask/FastAPI dependencies.

Real-World Applications

• Transportation Safety — real-time prediction can help reduce accidents in foggy, rainy, or dusty conditions.
• Aviation & Marine Navigation — supports flight takeoff/landing protocols and ship routing decisions.
• Smart Cities — can be integrated into intelligent traffic systems to manage flow and alerts.
• Environmental Monitoring — useful for assessing visibility trends with climate change.

Conclusion

This project predicts visibility distance from climatic parameters (temperature, humidity, wind speed, pressure, and more) — a safety-critical forecasting task for aviation, maritime, and road transport. It combines data science, meteorology, and MLOps practices (versioned pipeline stages and a local workflow) into a single, fully functional predictive system.

License

Provided for educational purposes. See LICENSE.