Feat/add http ingestion route - #103
Draft
foukou19 wants to merge 7 commits into
Draft
Conversation
…object and array payloads Signed-off-by: foukou19 <ferial.oukoukas@decathlon.com>
Signed-off-by: foukou19 <ferial.oukoukas@decathlon.com>
Signed-off-by: foukou19 <ferial.oukoukas@decathlon.com>
Signed-off-by: foukou19 <ferial.oukoukas@decathlon.com>
Signed-off-by: foukou19 <ferial.oukoukas@decathlon.com>
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
| @Service | ||
| @Slf4j | ||
| public class SecurityService { | ||
| public boolean validate(String rawPayload, Map<String, Object> headers, |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a new HTTP ingestion entrypoint (Apache Camel) and extends the “entity dynamic mapping” feature set by adding a dry-run validation endpoint, while migrating mapping relations from a JSON object/map representation to an ordered array/list model across persistence and API layers.
Changes:
- Add
POST /api/v1/entity_dynamic_mappings/dry-runto validate a single mapping against a sample payload (no persistence). - Migrate
entity_dynamic_mapping.relationsfrom JSONB object-map to JSONB array/list and update DTOs/mappers/tests accordingly. - Add a Camel-based
/webhooks/{webhookIdentifier}ingestion route and supporting services/config/dependencies.
Reviewed changes
Copilot reviewed 60 out of 63 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| src/test/resources/db/test/R__4_insert_webhook_test_data.sql | Update webhook test data to use new relations array format |
| src/test/resources/db/test/R__1_Insert_test_data.sql | Add new relation definition link in test dataset |
| src/test/java/com/decathlon/idp_core/infrastructure/adapters/persistence/mapper/common/EntityDynamicMappingJsonbHelperTest.java | New unit tests for JSONB relation list deserialization |
| src/test/java/com/decathlon/idp_core/infrastructure/adapters/persistence/EntityDynamicMappingAdaptorTest.java | Update test mapping construction for relations list |
| src/test/java/com/decathlon/idp_core/infrastructure/adapters/entity_mapping/jslt/JsltMappingEngineAdapterTest.java | New tests for relation extraction from payloads |
| src/test/java/com/decathlon/idp_core/infrastructure/adapters/entity_mapping/jslt/JsltEntityMappingValidatorTest.java | Update validator tests for new exception type + relation list |
| src/test/java/com/decathlon/idp_core/infrastructure/adapters/api/mapper/webhook/InboundWebhookMapperTest.java | Update mapping fixture to relations list |
| src/test/java/com/decathlon/idp_core/infrastructure/adapters/api/mapper/entity_dynamic_mapping/EntityDynamicMappingMapperTest.java | New mapper tests for null-safety and relation DTO conversion |
| src/test/java/com/decathlon/idp_core/infrastructure/adapters/api/mapper/entity_dynamic_mapping/EntityDynamicMappingDryRunDtoInMapperTest.java | New tests for dry-run payload normalization/serialization |
| src/test/java/com/decathlon/idp_core/infrastructure/adapters/api/handler/ApiExceptionHandlerTest.java | Update expected HTTP status codes to 422 for mapping/template violations |
| src/test/java/com/decathlon/idp_core/infrastructure/adapters/api/dto/in/EntityMappingDtoInDeserializationTest.java | New tests for relations deserialization from map or array |
| src/test/java/com/decathlon/idp_core/infrastructure/adapters/api/controller/InboundWebhookManagementControllerTest.java | Update controller test payloads for relations array |
| src/test/java/com/decathlon/idp_core/infrastructure/adapters/api/controller/EntityDynamicMappingControllerTest.java | Extend controller tests for relations array + dry-run endpoint |
| src/test/java/com/decathlon/idp_core/domain/service/webhook/WebhookConnectorValidationServiceTest.java | Update domain fixtures for relations list |
| src/test/java/com/decathlon/idp_core/domain/service/webhook/WebhookConnectorServiceTest.java | Update fixtures for relations list |
| src/test/java/com/decathlon/idp_core/domain/service/webhook/EntityDynamicMappingValidationServiceTest.java | Adapt relation validation inputs to RelationMapping list |
| src/test/java/com/decathlon/idp_core/domain/service/webhook/DynamicMappingServiceTest.java | Update mapping fixtures for relations list |
| src/test/java/com/decathlon/idp_core/domain/service/relation/RelationValidationServiceTest.java | Add coverage for dry-run relation validation skipping DB checks |
| src/test/java/com/decathlon/idp_core/domain/service/entity_dynamic_mapping/EntityDynamicMappingDryRunServiceTest.java | New unit tests for dry-run domain service behavior |
| src/main/resources/db/migration/V6_4__migrate_entity_dynamic_mapping_relations_to_array.sql | Flyway migration converting relations JSONB object → array |
| src/main/resources/application.yml | Configure Camel platform-http engine setting |
| src/main/resources/application-local.yml | Mirror Camel platform-http engine setting; comment out mock security config |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/persistence/mapper/EntityDynamicMappingPersistenceMapper.java | MapStruct mapping updated to relation-list JSONB converters |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/persistence/mapper/common/EntityDynamicMappingJsonbHelper.java | Add JSONB (de)serialization for relation list |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/ingestion/service/SecurityService.java | New ingestion security validation stub/service |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/ingestion/service/IngestionService.java | New ingestion service applying mappings and persisting entities |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/ingestion/route/WebhookExceptionRouteBuilder.java | New Camel exception-to-HTTP mapping route |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/ingestion/route/GenericWebhookRouteBuilder.java | New Camel ingestion pipeline wiring |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/ingestion/configuration/CamelRestConfiguration.java | New Camel REST DSL endpoint definition for ingestion |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/entity_mapping/jslt/JsltMappingEngineAdapter.java | New MappingEnginePort implementation (JSLT mapping + relations extraction) |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/entity_mapping/jslt/JsltExpressionEvaluator.java | New helper for expression evaluation with fallback behavior |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/entity_mapping/jslt/JsltEntityMappingValidator.java | Validator refactor to use ExpressionEngine abstraction + relation list |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/entity_mapping/jslt/JsltEngine.java | New ExpressionEngine implementation for JSLT |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/entity_mapping/jslt/JsltConfiguration.java | Provide shared ObjectMapper bean for mapping adapters |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/entity_mapping/engine/ExpressionEngine.java | New engine abstraction interface |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/api/mapper/entity_dynamic_mapping/EntityDynamicMappingMapper.java | Update mapping DTO↔domain conversions for relation list |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/api/mapper/entity_dynamic_mapping/EntityDynamicMappingDryRunDtoOutMapper.java | New domain→DTO mapping for dry-run responses |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/api/mapper/entity_dynamic_mapping/EntityDynamicMappingDryRunDtoInMapper.java | New transport payload normalizer (Object/String → JSON string) |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/api/handler/ApiExceptionHandler.java | Add 422 mappings for expression/mapping/template validation errors |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/api/dto/out/entity_dynamic_mapping/RelationMappingDtoOut.java | New output DTO for relation mappings |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/api/dto/out/entity_dynamic_mapping/EntityDynamicMappingDtoOut.java | Change relations shape in API output to list and normalize nulls |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/api/dto/out/entity_dynamic_mapping/EntityDynamicMappingDryRunDtoOut.java | New dry-run response DTO schema |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/api/dto/in/RelationMappingsDeserializer.java | New custom deserializer accepting relations as map or array |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/api/dto/in/RelationMappingDtoIn.java | New input DTO for relation mapping entries |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/api/dto/in/EntityMappingDtoIn.java | Update inbound mapping DTO to relation list with custom deserializer |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/api/dto/in/EntityDynamicMappingDryRunDtoIn.java | New request DTO for dry-run endpoint (payload object/string) |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/api/controller/EntityDynamicMappingController.java | Add /dry-run endpoint and required mappers/services |
| src/main/java/com/decathlon/idp_core/infrastructure/adapters/api/configuration/SwaggerDescription.java | Add Swagger strings for dry-run endpoint |
| src/main/java/com/decathlon/idp_core/domain/service/relation/RelationValidationService.java | Add dry-run relation validation path (skip DB existence checks) |
| src/main/java/com/decathlon/idp_core/domain/service/entity/EntityValidationService.java | Make validateForCreation public; add validateForDryRun |
| src/main/java/com/decathlon/idp_core/domain/service/entity_dynamic_mapping/EntityDynamicMappingValidationService.java | Update relation-name extraction for list-based relations |
| src/main/java/com/decathlon/idp_core/domain/service/entity_dynamic_mapping/EntityDynamicMappingService.java | Rename injected validation service field for clarity |
| src/main/java/com/decathlon/idp_core/domain/service/entity_dynamic_mapping/EntityDynamicMappingDryRunService.java | New domain service to execute dry-run validation flow |
| src/main/java/com/decathlon/idp_core/domain/port/MappingEnginePort.java | New domain port for mapping engine |
| src/main/java/com/decathlon/idp_core/domain/model/enums/ErrorType.java | New enum for dry-run result error categorization |
| src/main/java/com/decathlon/idp_core/domain/model/entity_mapping/RelationMapping.java | New domain record for relation mapping entries |
| src/main/java/com/decathlon/idp_core/domain/model/entity_mapping/EntityDynamicMapping.java | Change relations from Map→List and normalize immutably |
| src/main/java/com/decathlon/idp_core/domain/model/entity_mapping/DryRunResult.java | New domain model for dry-run outcomes |
| src/main/java/com/decathlon/idp_core/domain/exception/entity_dynamic_mapping/ExpressionEvaluationFailedException.java | New exception for expression evaluation failures |
| src/main/java/com/decathlon/idp_core/domain/exception/entity_dynamic_mapping/EntityDynamicMappingJsltErrorException.java | New exception type for JSLT/mapping errors |
| src/main/java/com/decathlon/idp_core/domain/constant/ValidationMessages.java | Add dry-run validation/result messages |
| pom.xml | Add Apache Camel dependencies/BOM and related starters |
| docs/src/static/swagger.yaml | Update OpenAPI docs for relation list + add dry-run endpoint schema |
Comment on lines
+40
to
+42
| if (node == null || node.isNull()) { | ||
| return List.of(); | ||
| } |
Comment on lines
+16
to
+21
| @Component | ||
| @RequiredArgsConstructor | ||
| public class EntityDynamicMappingDryRunDtoInMapper { | ||
|
|
||
| private final ObjectMapper objectMapper = new ObjectMapper(); | ||
|
|
Comment on lines
+19
to
22
| /// - Properties are serialized as a flat JSON object: {"key": "expression"} | ||
| /// - Relations are serialized as a JSON array: [{"name": "owner", "expression": ".sender.login"}] | ||
| /// | ||
| /// This is a pure utility class with no Spring dependencies, facilitating testability and reusability. |
Comment on lines
+14
to
+20
| public boolean validate(String rawPayload, Map<String, Object> headers, | ||
| WebhookConnector webhookConnector) { | ||
| // Implement security validation logic here | ||
| log.info("WebhookSecurity rawpalyload: {} strategy: {}", rawPayload, | ||
| webhookConnector.security()); | ||
| return true; | ||
| } |
Comment on lines
+54
to
+60
| from("direct:fetch-configuration").routeId("fetch-webhook-config") | ||
| .log(LoggingLevel.DEBUG, | ||
| "Fetching configuration for webhook ID: ${exchangeProperty.webhookIdentifier}") | ||
| .bean(webhookConnectorService, "getWebhookConnector(${exchangeProperty.webhookIdentifier})") | ||
| // Evaluates bean without overwriting exchange body | ||
| .setProperty("webhookConfig", method(webhookConnectorService, | ||
| "getWebhookConnector(${exchangeProperty.webhookIdentifier})")); |
Comment on lines
+46
to
+48
| // Step E: Return HTTP 202 Accepted Response | ||
| .setHeader(Exchange.HTTP_RESPONSE_CODE, constant(200)) | ||
| .setHeader(Exchange.CONTENT_TYPE, constant("application/json")).setBody(constant( |
Comment on lines
+74
to
+76
| .bean(ingestionService, | ||
| "ingest(${exchangeProperty.rawJsonPayload}, ${exchangeProperty.webhookConfig})") | ||
| .setProperty("mappedEntity", body()); // Store domain entity in Exchange property |
Comment on lines
+618
to
+620
| @Nested | ||
| @DisplayName("POST /api/v1/entity-dynamic-mappings/dry-run - Dry-run mapping") | ||
| @Order(6) |
Comment on lines
+18
to
+20
| @Test | ||
| @DisplayName("Should deserialize legacy V6.4 'expression' field as expressions list") | ||
| void shouldDeserializeLegacyExpressionField() { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR Description
What this PR Provides
Fixes
Review
The reviewer must double-check these points:
!after the type/scope to identify the breakingchange in the release note and ensure we will release a major version.
How to test
Please refer (copy/paste) the test section from the User Story. This should include
(for example, ensure the data xxx exists in idp-back to be able to test the feature)
(for example, go to page xxx, fill the xxx field and click the 'send' button)
(for example, there is a link in the database between component X and component Y.
You can retrieve the information with a
GETrequest to the API)Breaking changes (if any)
Context of the Breaking Change
For example: we redefined the component types list in the DPAC referential
Result of the Breaking Change
For example: your component of type xxx will migrate to the type yyy