⚡ Streaming lakehouse pipelines
Kafka → Flink → Iceberg ingestion for high-volume sensor and process data, sustaining roughly 3M records/sec into a petabyte-scale lake. Benchmarked Iceberg against Hudi across insert, upsert, query and maintenance workloads before committing to the table format; built out the Flink Kubernetes Operator deployment, and Spark (Scala) maintenance jobs — compaction, sorting, snapshot expiry — orchestrated on Airflow.

🔎 Query engine performance
A 100+ node Trino cluster fields 300+ queries per minute, individual queries scanning terabytes. Most of the wins came from the storage side — partition strategy, sort order, compaction cadence — and the rest from the query side: pushdown improvements and approximate aggregations where exactness wasn't required. This architecture was the subject of my Trino Summit 2023 talk.

☸️ Multi-tenant runtime on Kubernetes
A self-service platform that provisions per-user Spark and StarRocks clusters on demand, operating thousands of them concurrently: custom operators and CRDs, a Java/Spring orchestration service, Helm packaging, ArgoCD GitOps, and Prometheus/OpenSearch observability. Running an OLAP engine as a multi-tenant service — rather than as one shared cluster — is where most of my engine-internals work comes from.

🤖 Agents for platform operations
An agent that consumes query history and cluster state snapshots off Kafka, diagnoses resource, configuration and query-level problems, and proposes concrete fixes — applied only after human approval. Deterministic rules and cost models do the heavy lifting; the model handles classifying unseen symptoms, correlating signals, and rewriting queries.

🧠 Domain-expert agents
A knowledge-graph-driven agent for operational troubleshooting, structured as graph selection → graph traversal → path evaluation: LLM-based graph selection to cover the gaps in vector similarity search, hierarchical filtering over a partitioned graph, parallel sub-agents that traverse branches and prune infeasible paths early, self-correcting tool-call loops, and a final helpfulness/groundedness evaluation that picks or aggregates the answer path. Includes Text-to-SQL over Oracle and Postgres, and on-prem inference tuning with vLLM (speculative decoding, prefix caching).

🎯 Domain-specialized LLM
Fine-tuned SK Telecom's A.X foundation model (32B) with DeepSpeed and TRL into an HR-domain assistant, reaching ~90% of the then-SOTA general model's quality on evaluations scored by domain experts — at a fraction of the serving cost.

Efficient Kappa Architecture with Trino Trino Summit 2023 One pipeline instead of two — serving terabyte-scale queries over data that is still arriving, on Kafka, Flink, Iceberg and Trino.

Building Domain-Expert Agents AI Community Conference (AICO) Seoul 2025 Moving an agent from general capability to domain expertise — knowledge graphs over flat retrieval, traversal with early pruning, and grading the path rather than only the answer.

Fixes to the query engines, catalogs and operators we run in production.

_{B.S. in Information Systems, Hanyang University}

Streaming & batch
Lakehouse & query
AI & LLM
Platform
Languages