DynamoDB Streams to capture data change in Amazon DynamoDB by CRUD with REST API in Amazon API Gateway
DFS is an efficient mysql data synchronisation framework that consists of a millisecond incremental data capture module (CDC), a sqlite-based persistent queue, an automatic relational processing module (rel_mgr), and a complex data transformation module (DPU), encapsulated using docker containers.
This project showcases the use of Kafka connectors for real-time data streaming from Postgres databases using Spring Boot. Two branches are included, one for JSON converter and another for Avro converter."
Lightweight MSSQL Change Data Capture in Go
Experiment with Kafka, Debezium. Source Connectors: Monitor MySQL & MongoDb changes, push messages to Kafka. springboot service: Listen to Kafka.
DevOps & Data Engineer | Kafka, PostgreSQL, Kubernetes, Docker, Odoo, BigQuery, and real-time data pipelines.
Personal knowledge hub built with data-engineering rigor, "idempotent ETL pipelines", "Bronze/Silver/Gold layers", and "lineage tracking", to gather the high scattered documents in data center. It feed a dashboard & vector-search RAG technique, turning into a single source of truth.
To associate your repository with the capture-data-change topic, visit your repo's landing page and select "manage topics."