Skip to content

Add a MongoDB connector #797

Description

@Leomrlin

What & why

GeaFlow already supports JDBC / Hive / ES / Neo4j, but there's no connector for
the most popular document database, MongoDB. It's a frequent community
request — Mongo can serve as both a vertex/edge data source and a result sink.

GeaFlow 已支持 JDBC/Hive/ES/Neo4j,但缺主流文档数据库 MongoDB。这是社区高频需求:
Mongo 既能做点/边数据源,也能做结果输出。

The task

Create a new module geaflow-dsl-connector-mongodb with a source (partitioned
reads, optional predicate pushdown) and a sink (batched writes).

新建模块 geaflow-dsl-connector-mongodb,实现 source(分区读、谓词下推可选)+ sink(批量写)。

Where to look / 怎么做

  1. Copy the most complete references for scaffolding: streaming
    geaflow-dsl-connector-kafka, partition/format geaflow-dsl-connector-file,
    remote-store class geaflow-dsl-connector-elasticsearch.
  2. Implement MongoTableConnector (TableReadableConnector +
    TableWritableConnector), MongoTableSource
    (init/open/listPartitions/getDeserializer/fetch), and MongoTableSink
    (init/open/write/flush/close).
  3. SPI registration:
    src/main/resources/META-INF/services/org.apache.geaflow.dsl.connector.api.TableConnector.
  4. Add the module to the parent pom.xml and to the connector aggregation pom.
  5. Docs (CN + EN) + an examples GQL + an integration test.

Done when

  • Both source and sink work, with basic connection config (uri / db / collection / batch size)
  • listPartitions splits by shard/range to enable parallel reads
  • Integration test included (Testcontainers can spin up MongoDB)
  • CN + EN docs and one examples GQL
  • Passes checkstyle and RAT (ASF header)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions