An end-to-end automated data collection, transformation, and visualization system that tracks regional software and data job postings. The pipeline automatically scrapes job boards using Playwright, triggers automated workflows via n8n hosted on AWS, stores structured job data, and streams live metrics into an interactive Power BI dashboard.
- Web Scraping: Python
- Workflow Automation: n8n
- Cloud Infrastructure: AWS EC2
- Data Analytics & BI: Power BI, DAX, SQL
- Messaging/Alerts: Telegram Bot API
- Automated Headless Scraping: Configured Playwright scripts to handle dynamic JavaScript rendering and pagination on multi-page job portals.
- Cloud Workflow Engine: Orchestrated ETL tasks using n8n hosted on AWS to run on a scheduled cron trigger.
- Instant Notification Pipeline: Connected scraped payloads directly to Telegram Bot API for real-time job posting alerts.
- Dynamic Business Intelligence: Designed Power BI report modeling job demand, required tool combinations, and hiring trends over time.