Skip to content

Shared handler pattern + source-gen JSON + Airflow Job integration for Fabric MCP - #102

Merged
Ebram Tawfik (EbramTawfik) merged 28 commits into
mainfrom
fabric-mcp-tools-integration
May 18, 2026
Merged

Shared handler pattern + source-gen JSON + Airflow Job integration for Fabric MCP#102
Ebram Tawfik (EbramTawfik) merged 28 commits into
mainfrom
fabric-mcp-tools-integration

Conversation

@Ebram-Tawfik

@Ebram-Tawfik Ebram-Tawfik commented May 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

Introduces a shared handler architecture and source-generated JSON serialization to make DataFactory.MCP.Core a production-ready NuGet package for integration into Microsoft's Fabric MCP Server (see microsoft/mcp#2584).

What Changed

Shared Handler Pattern

  • PipelineHandler, DataflowHandler, DataflowQueryHandler — domain handlers that own ALL business logic (validation, service calls, result shaping, error handling)
  • ToolResult<T> — framework-agnostic result type with IsSuccess/Error pattern
  • SDK tools ([McpServerTool]) become thin 3-8 line delegators to handlers
  • Handlers are reusable across both the standalone MCP server and Fabric MCP Server's GlobalCommand<T> pattern

Source-Generated JSON (ILLinker/Trim Safety)

  • DataFactoryJsonContext — AOT-compatible JsonSerializerContext for all request/response types (including Airflow Job types)
  • All serialization in FabricServiceBase uses source-gen context instead of reflection-based JsonSerializerOptions
  • Enables safe IL trimming in the NuGet package consumers

TokenCredential Auth Bridge

  • Added TokenCredential auth bridge for host integration scenarios

Apache Airflow Job Integration

  • Merged Airflow Job MCP tools from main (7 new tools)
  • Registered all Airflow Job types in DataFactoryJsonContext for source-gen compatibility
  • Fixed anonymous type serialization in GetAirflowJobDefinitionAsync

NuGet Package Pipeline

  • Added DataFactory.MCP.Core to the pack-and-sign pipeline
  • Package published as Microsoft.DataFactory.MCP.Core on NuGet

Project Cleanup

  • Removed DataFactory.MCP.Fabric adapter project (moved to microsoft/mcp repo)
  • Organized commands and options into domain folders
  • Version bumped to 0.22.0-beta

Architecture

Handler (Core)     -> ALL business logic (validation, API call, error handling)
SDK Tool (Core)    -> [McpServerTool] shim, delegates to handler
Fabric Command     -> GlobalCommand<T> shim in mcp repo, delegates to handler

Key Files

Area Files
Handlers Core/Handlers/PipelineHandler.cs, DataflowHandler.cs, DataflowQueryHandler.cs
Result type Core/Handlers/ToolResult.cs, ToolResultExtensions.cs
Source-gen JSON Core/Configuration/DataFactoryJsonContext.cs
Serialization Core/Abstractions/FabricServiceBase.cs (uses DataFactoryJsonContext.Default)
Auth bridge TokenCredential integration for host scenarios

Ebram Tawfik and others added 9 commits May 6, 2026 13:01
Ebram Tawfik
Make DataFactory.MCP.Core AOT-compatible
51f7747 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik
Add DataFactory.MCP.Fabric adapter project for Fabric MCP Server inte…
0843119 
…gration

Create a new adapter project that bridges DataFactory.MCP.Core services
to Microsoft.Mcp.Core's IAreaSetup/GlobalCommand pattern used by
Fabric.Mcp.Server.

- DataFactoryAreaSetup implements IAreaSetup (single-line integration)
- 5 commands: ListWorkspaces, ListPipelines, CreatePipeline, GetPipeline, RunPipeline
- AOT-compatible with JsonSerializerContext source generators
- Delegates all business logic to Core services (no duplication)
- Follows exact Fabric.Mcp.Tools.Core pattern

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik
Fix Microsoft.Mcp.Core reference for submodule and standalone builds
87c04aa 
Use conditional ItemGroup to resolve Microsoft.Mcp.Core:
- When used as submodule in mcp repo: references ../../../core/...
- When built standalone: references ../../mcp/core/...

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik
Prototype shared handler pattern for ListPipelines
9b0d8d6 
Extract business logic into ListPipelinesHandler in Core, making both
the SDK tool and Fabric command thin delegators. This proves the
handler pattern reduces duplication before applying to all tools.

New files:
- Core/Handlers/ToolResult.cs - Framework-agnostic result type
- Core/Handlers/Pipeline/ListPipelinesHandler.cs - All business logic
- Core/Handlers/ToolResultExtensions.cs - Error type mapping

Modified:
- PipelineTool.ListPipelinesAsync: 45 lines -> 8 lines
- ListPipelinesCommand.ExecuteAsync: delegates to handler
- ServiceCollectionExtensions: register handler in DI

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik
Extract Options classes into separate files under Options/
d598185 
Move each GlobalOptions subclass from its command file into a
dedicated file in DataFactory.MCP.Fabric/Options/ for cleaner
separation of concerns.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik
Organize commands and options into domain folders
7c51eb3 
Mirror the Core/Tools structure (Pipeline/, Workspace/) in the Fabric
adapter project for commands and options.

Structure:
  Commands/Pipeline/ - ListPipelines, CreatePipeline, GetPipeline, RunPipeline
  Commands/Workspace/ - ListWorkspaces
  Options/Pipeline/ - Pipeline-specific options
  Options/Workspace/ - Workspace-specific options
  Options/ - Shared option definitions

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik
Remove ListWorkspacesCommand and related files
f021aee 
Workspace listing doesn't need a Fabric command wrapper — the SDK
tool handles it fine. Removes command, options, and result type.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik
Add ListDataflows and CreateDataflow Fabric commands
df831c6 
- ListDataflowsCommand: lists dataflows in a workspace
- CreateDataflowCommand: creates a new dataflow with name/description
- Options classes in Options/Dataflow/ folder
- Result records in DataFactoryJsonContext
- Registered in DataFactoryAreaSetup

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik
Remove DataFactory.MCP.Fabric - moved to mcp repo as Fabric.Mcp.Tools…
0b25987 
….DataFactory
@github-actions

github-actions Bot commented May 6, 2026 •
edited
Loading

Copy link
Copy Markdown

✅ Integration Tests PASSED

Workflow: PR Validation
Commit: bcd1e230b836855842e55f069ce2dc6a2b65b4bd
Result: PASSED

🎉 All integration tests passed! This PR is ready for review.

View full results

Ebram Tawfik and others added 8 commits May 6, 2026 16:11
Ebram Tawfik
Unified domain handlers: PipelineHandler + DataflowHandler
d3e4348 
- Replace ListPipelinesHandler with unified PipelineHandler (List/Create/Get/Run)
- Add DataflowHandler (List/Create)
- Update PipelineTool and DataflowTool to delegate to handlers
- Update DI registrations
- All business logic now lives in handlers, tools are thin shims

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik
Add DataFactory.MCP.Core to NuGet pack-and-sign pipeline
241abef 
- Add Core project to path triggers
- Add restore, build, sign, and pack steps for Core
- Core .nupkg outputs to same directory for OneBranch auto-publish

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik
Remove AOT/trim compatibility flags from Core package
b19ab07 
The Core library uses reflection-based System.Text.Json and DataAnnotations
validation which are not trim-safe. Removing IsAotCompatible/EnableTrimAnalyzer
flags so consuming projects with PublishTrimmed=true don't fail.

Bump version to 0.19.1-beta.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik
Suppress trim analysis warnings for IL linker compatibility
25cbebb 
Add #pragma warning disable IL2026/IL3050 around JsonSerializer and
DataAnnotations usage to suppress trim analyzer warnings at build time.
Bump version to 0.19.1-beta.

Note: These pragmas suppress Roslyn analyzer warnings. For full ILLinker
compatibility when consumed as a NuGet package, these will need to be
replaced with [UnconditionalSuppressMessage] attributes in a follow-up.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik
Move IL trim suppressions to project-level NoWarn
fbf2a89 
Replace per-file #pragma warning disable/restore IL2026/IL3050 with a
single <NoWarn> entry in the csproj. Cleaner files, same effect.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik
Add DataflowQueryHandler and wire into DataflowQueryTool
cd836bc 
Extract query execution logic into DataflowQueryHandler following the
shared handler pattern. DataflowQueryTool is now a thin MCP wrapper
that delegates to the handler.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik
Revert comment-only change in DataflowRefreshJob.cs
fb13228 
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik
Move IL2091 suppression to project-level NoWarn, revert whitespace-on…
bb6369d 
…ly changes

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Ebram-Tawfik
Ebram-Tawfik force-pushed the fabric-mcp-tools-integration branch from 6e0b344 to bb6369d Compare May 7, 2026 17:27
Ebram Tawfik and others added 6 commits May 7, 2026 11:21
Ebram Tawfik
feat: migrate to source-generated JSON for ILLinker trim safety
eef43f7 
- Create DataFactoryJsonContext with 40+ registered types
- Replace reflection-based JsonSerializer calls with source-gen context
- Fix ConnectionJsonConverter and GatewayJsonConverter polymorphic dispatch
- Fix FabricServiceBase.PostAsync/PatchAsync serialization
- Fix HttpResponseMessageExtensions.ReadAsJsonAsync deserialization
- Suppress IL2026 on ValidationService (DataAnnotations requires reflection)
- Bump version to 0.20.0-beta

Resolves all 16 IL2026 ILLinker trim warnings that would fail CI
in Fabric.Mcp.Server (TreatWarningsAsErrors=true).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik
fix: update npm dependencies to resolve security vulnerabilities
0dfcb66 
- Update @modelcontextprotocol/ext-apps from ^1.0.1 to ^1.7.1
- Run npm audit fix to patch vite, rollup, postcss, picomatch
- Resolves all 17 component governance security alerts

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik
fix: pin vite to 7.3.2 for CI feed compatibility
03b24cf 
vite 7.3.3 is not yet cached in the DevOps Gateway-AdminPortal npm feed.
Pin to 7.3.2 which fixes all security vulnerabilities and is available
on the CI feed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik
docs: add CI feed configuration skill for mcp integration
eb6b75e 
Document DevOps NuGet feed caching behavior, how to trigger upstream
cache after publishing, and npm feed troubleshooting for CI.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik
Revert "docs: add CI feed configuration skill for mcp integration"
1f06d4a 
This reverts commit eb6b75e.
Ebram Tawfik
docs: add DevOps agent and Fabric.Mcp.Server integration skill
b538ed1 
Create devops.agent.md for CI/publishing workflows.
Create devops.fabric-mcp-integration skill for DevOps feed caching,
version management, and trim safety requirements.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Ebram-Tawfik
Ebram-Tawfik force-pushed the fabric-mcp-tools-integration branch from b7fdefa to b538ed1 Compare May 7, 2026 19:15
Ebram Tawfik
Merge origin/main into fabric-mcp-tools-integration
6101dbe 
Resolve conflict in FabricServiceBase.cs: keep source-gen
serialization (DataFactoryJsonContext.Default) from this branch,
adopt LogDebug from main (security hardening PR #99).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Ebram-Tawfik Ebram-Tawfik changed the title Shared handler pattern + Fabric adapter for DataFactory.MCP Shared handler pattern + source-gen JSON for Fabric MCP integration May 12, 2026
Ebram Tawfik
docs: add Handlers layer to architecture documentation
c2ab715 
- Add Handlers Layer to TOC, ASCII diagram, and component details
- Explain why handlers exist (dual-framework: SDK tools + Fabric commands)
- Document ToolResult<T>, handler table, before/after examples
- Add note to Data Flow section about Tool → Handler → Service path
- Add handler-based tool pattern to Extension Points

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik and others added 3 commits May 12, 2026 11:20
Ebram Tawfik
fix: source-gen JSON fallback for anonymous types and JsonElement
9363d66 
SerializeRequest() tries DataFactoryJsonContext first, falls back to
reflection-based JsonOptions for types not registered in the source-gen
context (anonymous types in RunCopyJobAsync, JsonElement in
CreateScheduleRequest.Configuration).

Fixes CI test failures in CopyJobToolIntegrationTests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik
fix: make source-gen JSON fully AOT-safe (no reflection fallback)
05924fe 
Replace anonymous types with concrete DTOs (RunOnDemandRequest,
EmptyRequest) and change object parameters to JsonElement for AOT
source-gen compatibility. Register all types in DataFactoryJsonContext.
Remove reflection-based SerializeRequest fallback from FabricServiceBase.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ebram Tawfik
feat: add TokenCredential auth bridge for host integration
8abe983 
When DataFactory.MCP.Core is hosted inside a system that already provides
authentication (e.g., Fabric MCP Server with DefaultAzureCredential),
the new TokenCredentialAuthenticationService automatically delegates
token acquisition to the host's TokenCredential.

- Add TokenCredentialAuthenticationService implementing IAuthenticationService
- Use TryAddSingleton with conditional: if TokenCredential is in DI, use bridge;
  otherwise fall back to existing standalone auth (device code/interactive/SP)
- Add Azure.Core package reference
- Bump version to 0.21.0-preview

No breaking changes — standalone mode works exactly as before.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@Ebram-Tawfik Ebram-Tawfik mentioned this pull request May 12, 2026
Merged
@EbramTawfik
Ebram Tawfik (EbramTawfik) merged commit 7c5ef1b into main May 18, 2026
14 checks passed
@Ebram-Tawfik Ebram-Tawfik changed the title Shared handler pattern + source-gen JSON for Fabric MCP integration Shared handler pattern + source-gen JSON + Airflow Job integration for Fabric MCP May 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@EbramTawfik Ebram Tawfik (EbramTawfik) EbramTawfik approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Ebram-Tawfik @EbramTawfik