Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
9ccced6
refactor: delete model fields left over from the sqlc/Wire-proto era
MariusVolkhart Sep 5, 2026
6bd5f9a
refactor: resolve a query block once, not three times
MariusVolkhart Sep 5, 2026
f9a63d1
refactor: define "children" once on PgNodeExpression
MariusVolkhart Sep 5, 2026
399b3e5
refactor: route every SQL text scanner through SqlLexer
MariusVolkhart Sep 5, 2026
c62eed7
refactor: express a void transaction as a discarded-result transaction
MariusVolkhart Sep 6, 2026
f509e43
refactor: parse the range table once, derive the specialised views
MariusVolkhart Sep 6, 2026
2ef3356
refactor: build every :many signature from mapperFunction
MariusVolkhart Sep 6, 2026
4b0a655
refactor: split TypeRepository.kt into per-concern files
MariusVolkhart Sep 6, 2026
02497d0
refactor: merge the base-type and JDBC-type-info tables into one
MariusVolkhart Sep 6, 2026
6af048f
refactor: describe a wire type by how JDBC reaches it, not by four flags
MariusVolkhart Sep 6, 2026
6b77c50
refactor: give the nullability analyzer its own catalog
MariusVolkhart Sep 6, 2026
75d0527
docs: retarget stale PgCatalogLoader references to ColumnNullabilityA…
MariusVolkhart Sep 6, 2026
da08ec3
docs: describe the node-tree route, not the deleted probe/stub machinery
MariusVolkhart Sep 6, 2026
440bbe0
test: describe the prosqlbody route in QueryAnalysisTest
MariusVolkhart Sep 6, 2026
b5aa6f6
test: de-flake the catalog test, and say what the MERGE analysis does
MariusVolkhart Sep 6, 2026
591589f
refactor: name the adapter-parameter ordering constraint
MariusVolkhart Sep 6, 2026
671ee20
docs: keep the counterexample, cut the argument around it
MariusVolkhart Sep 7, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ norm/
├── e2e-tests/ # End-to-end tests (standalone, no framework)
├── e2e-tests-micronaut/ # Micronaut integration tests
├── e2e-tests-spring/ # Spring integration tests
├── proto/ # Protocol buffer definitions (internal Wire types)
├── test-scenarios/ # Test scenarios with golden files
├── test-scenarios-frameworks/ # Framework-specific test scenarios
└── buildSrc/ # Shared Gradle build logic
Expand All @@ -52,7 +51,6 @@ norm/

- **Kotlin** - Primary language
- **KotlinPoet** - Code generation library (in `generator`)
- **Wire** - Protocol buffer types used as internal model (from `proto/codegen.proto`)
- **Testcontainers** - Starts PostgreSQL for JDBC-based schema/query analysis
- **Gradle** - Build system with convention plugins in `buildSrc`

Expand Down Expand Up @@ -84,7 +82,7 @@ Example Gradle tasks:

1. **gradle-plugin** starts a PostgreSQL Testcontainer and applies schema SQL files
2. **JdbcAnalyzer** uses JDBC metadata APIs to build a `Catalog` (tables, columns, enums, domains) and analyze queries (parameter types, result column types)
3. **generator** takes the `Catalog` + analyzed `Query` objects (Wire proto types from `proto/codegen.proto`) and produces Kotlin via KotlinPoet
3. **generator** takes the `Catalog` + analyzed `Query` objects (model data classes in `generator/src/main/kotlin/norm/generator/Model.kt`) and produces Kotlin via KotlinPoet
4. **gradle-plugin** writes the generated `.kt` files

### Runtime Library
Expand Down Expand Up @@ -136,7 +134,7 @@ Commands: `:one` (single result), `:many` (multiple results), `:execrows` (retur
- `gradle-plugin/src/main/kotlin/norm/gradle/NormGenerateTask.kt` - Gradle task orchestrating the pipeline
- `runtime/src/main/kotlin/norm/NormDriver.kt` - Core runtime driver
- `runtime/src/main/kotlin/norm/Query.kt` - Dynamic query API
- `proto/codegen.proto` - Wire proto definitions for internal model types
- `generator/src/main/kotlin/norm/generator/Model.kt` - Model data classes for internal model types

## Testing

Expand Down
1,064 changes: 420 additions & 644 deletions generator/src/main/kotlin/norm/generator/ColumnNullabilityAnalyzer.kt

Large diffs are not rendered by default.

14 changes: 7 additions & 7 deletions generator/src/main/kotlin/norm/generator/DomainBuilder.kt
Original file line number Diff line number Diff line change
Expand Up @@ -122,20 +122,20 @@ internal fun domainAdapterPropertyName(domain: Domain): String = "${domain.name.
/**
* Maps a Postgres base type name to the corresponding Kotlin [TypeName].
*
* Delegates to [resolveJdbcTypeInfo] as the single source of truth for type mappings. Every type
* Delegates to [resolveWireCodec] as the single source of truth for type mappings. Every type
* with an entry there is usable as a domain base, including `json` and `jsonb`: the value class
* wraps the same Kotlin type the plain column would produce, and the binding difference
* (`setObject(..., Types.OTHER)` rather than `setString`) is carried by the [JdbcTypeInfo] that
* (`setObject(..., Types.OTHER)` rather than `setString`) is carried by the [WireCodec] that
* [TypeRepository] hands to [AdaptedTypeSqlMappable], not by the wrapped Kotlin type.
*
* [resolveJdbcTypeInfo] has an entry for every type [TypeRepository]'s `BASE_TYPE_RESOLVERS`
* accepts as a plain column type (enforced by [ColumnTypeMappingTest]'s domain-base-type-parity
* sweep), so [error] here is unreachable for a domain built on any of those — `CREATE DOMAIN d AS
* timestamptz`/`uuid`/`date`/etc. all resolve. It stays reachable for a Postgres type Norm has
* [resolveWireCodec] reads [POSTGRES_BASE_TYPES], the same map [TypeRepository.resolveBaseType]
* reads for a plain column's type, so [error] here is unreachable for a domain built on any type
* that map supports — `CREATE DOMAIN d AS timestamptz`/`uuid`/`date`/etc. all resolve, by
* construction. It stays reachable for a Postgres type Norm has
* never mapped to Kotlin at all, as a plain column or otherwise (`xml`, `interval`, `money`, ...).
* Postgres allows a domain over any of these, so hitting this is expected, not a bug — failing
* fast with the unsupported type's name beats silently guessing a mapping Norm has no tested
* behavior for.
*/
internal fun domainKotlinBaseType(baseTypeName: String): TypeName =
resolveJdbcTypeInfo(baseTypeName)?.kotlinType ?: error("Unsupported domain base type: $baseTypeName")
resolveWireCodec(baseTypeName)?.kotlinType ?: error("Unsupported domain base type: $baseTypeName")
92 changes: 13 additions & 79 deletions generator/src/main/kotlin/norm/generator/GroupRteSubstitution.kt
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import norm.generator.NodeTreeNullabilityAnalyzer.Companion.MAX_EXPRESSION_DEPTH

/**
* Substitutes every [PgNodeExpression.Var] in [expression] that references a PostgreSQL 18+ GROUP
* RTE (see [PgNodeTreeParser.parseGroupRteExpressions]) with the resolved `:groupexprs` expression
* RTE (see [groupExpressions]) with the resolved `:groupexprs` expression
* it stands in for, restoring the same tree shape PostgreSQL 16 and 17 produce directly (where the
* original grouping-key expression is left in the target list, never masked behind a `Var`). This
* lets [NodeTreeNullabilityAnalyzer] apply one set of nullability rules to every supported
Expand All @@ -20,7 +20,7 @@ import norm.generator.NodeTreeNullabilityAnalyzer.Companion.MAX_EXPRESSION_DEPTH
* shape this parser does not (yet) model correctly, and substituting against a nonexistent entry
* would silently invent an expression PostgreSQL never produced.
* - The resolved expression is not [PgNodeExpression.Unknown] — an unmodelled node type or a parse
* failure must not replace a `Var` that [PgNodeTreeParser.parseGroupRteMap]'s coarser, `Var`-only
* failure must not replace a `Var` that [groupRteMap]'s coarser, `Var`-only
* resolution could still succeed at (see that method's continued use as a fallback wherever this
* substitution declines to apply).
*
Expand All @@ -40,28 +40,10 @@ import norm.generator.NodeTreeNullabilityAnalyzer.Companion.MAX_EXPRESSION_DEPTH
* GROUP-RTE-shaped) structurally unable to loop, without this function ever having to prove
* PostgreSQL cannot emit such a cycle — a proof this function does not attempt.
*
* Child coverage mirrors [NodeTreeNullabilityAnalyzer.containsVarOutsideRelation], not
* [NodeTreeNullabilityAnalyzer.safetyWalkChildren]: the latter drops several children (e.g.
* [PgNodeExpression.Aggref]'s own arguments, [PgNodeExpression.JsonExpr]'s `PASSING`-adjacent
* fields, everything past [PgNodeExpression.SubLink.outerOperand]) for reasons specific to the
* grouping-set safety analysis it backs, which do not apply here. A GROUP RTE `Var` can appear
* buried inside a non-`Aggref` node this substitution must walk into — e.g. `count(*) +
* 0::bigint` when `0::bigint` is also the grouping key rewrites that operand of the `+` to a GROUP
* RTE `Var`, sitting alongside the `Aggref` as an [PgNodeExpression.OpExpr] argument.
* [PgNodeExpression.Aggref]'s own arguments are a different case: PostgreSQL never rewrites them at
* all, on PG18 or otherwise — `count(lower(a))`/`string_agg(lower(a), ',')` under
* `GROUP BY ROLLUP(lower(a))` keep the real base-relation `Var` inside the `AGGREF`'s `:args`,
* because aggregate arguments are evaluated pre-grouping, before any GROUP RTE substitution could
* apply. This substitution still walks into `Aggref`'s arguments anyway — not because any known PG18
* shape needs it, but defensively: the exhaustive `when` below requires some branch for `Aggref`
* regardless, and doing the real rewrite there (rather than treating it as a childless leaf) means a
* future PostgreSQL shape, or an unrelated caller that hands this function an `Aggref` subtree
* directly, cannot silently escape substitution. Every child this substitution's `when` retains must
* be rewritten, or a buried GROUP RTE `Var` would silently survive un-substituted.
*
* The `when` below is exhaustive over the sealed [PgNodeExpression] hierarchy with no `else ->`
* branch: adding a new [PgNodeExpression] subtype without updating this function fails the build,
* rather than silently leaving that subtype's children unwalked.
* Every non-`Var` node's children are walked via [PgNodeExpression.mapChildren], which is exhaustive
* over the sealed [PgNodeExpression] hierarchy, so a buried GROUP RTE `Var` — e.g. `count(*) +
* 0::bigint` when `0::bigint` is also the grouping key, sitting alongside an `Aggref` as an
* [PgNodeExpression.OpExpr] argument — cannot silently escape substitution.
*
* @param depth remaining recursion budget, mirroring [MAX_EXPRESSION_DEPTH]; once exhausted,
* [expression] is returned unchanged — failing toward the current, un-substituted (and therefore
Expand All @@ -75,61 +57,13 @@ internal fun substituteGroupRteVars(
): PgNodeExpression {
if (depth <= 0) return expression
val recurse = { child: PgNodeExpression -> substituteGroupRteVars(child, groupExpressionsByVarno, depth - 1) }
return when (expression) {
is PgNodeExpression.Var -> {
val resolved = if (expression.levelsUp == 0) {
groupExpressionsByVarno[expression.varno]?.let { groupExpressions ->
groupExpressions.getOrNull(expression.varattno - 1)
}
} else {
null
}
if (resolved != null && resolved !is PgNodeExpression.Unknown) resolved else expression
if (expression !is PgNodeExpression.Var) return expression.mapChildren(recurse)
val resolved = if (expression.levelsUp == 0) {
groupExpressionsByVarno[expression.varno]?.let { groupExpressions ->
groupExpressions.getOrNull(expression.varattno - 1)
}

is PgNodeExpression.Const,
is PgNodeExpression.SqlValueFunction,
is PgNodeExpression.NextValExpr,
is PgNodeExpression.Unknown,
-> expression

is PgNodeExpression.FuncExpr -> expression.copy(arguments = expression.arguments.map(recurse))
is PgNodeExpression.OpExpr -> expression.copy(arguments = expression.arguments.map(recurse))
is PgNodeExpression.ScalarArrayOpExpr -> expression.copy(arguments = expression.arguments.map(recurse))
is PgNodeExpression.CoalesceExpr -> expression.copy(arguments = expression.arguments.map(recurse))
is PgNodeExpression.NullIfExpr -> expression.copy(arguments = expression.arguments.map(recurse))
is PgNodeExpression.MinMaxExpr -> expression.copy(arguments = expression.arguments.map(recurse))
is PgNodeExpression.Aggref -> expression.copy(arguments = expression.arguments.map(recurse))
is PgNodeExpression.WindowFunc -> expression.copy(arguments = expression.arguments.map(recurse))
is PgNodeExpression.SubLink -> expression.copy(outerOperand = expression.outerOperand?.let(recurse))
is PgNodeExpression.CaseExpr -> expression.copy(
resultExpressions = expression.resultExpressions.map(recurse),
defaultResult = expression.defaultResult?.let(recurse),
testExpression = expression.testExpression?.let(recurse),
whenConditions = expression.whenConditions.map(recurse),
)

is PgNodeExpression.BoolExpr -> expression.copy(arguments = expression.arguments.map(recurse))
is PgNodeExpression.RelabelType -> expression.copy(argument = recurse(expression.argument))
is PgNodeExpression.CoerceViaIo -> expression.copy(argument = recurse(expression.argument))
is PgNodeExpression.ArrayCoerceExpr -> expression.copy(argument = recurse(expression.argument))
is PgNodeExpression.CollateExpr -> expression.copy(argument = recurse(expression.argument))
is PgNodeExpression.CoerceToDomain -> expression.copy(argument = recurse(expression.argument))
is PgNodeExpression.NullTest -> expression.copy(argument = recurse(expression.argument))
is PgNodeExpression.BooleanTest -> expression.copy(argument = recurse(expression.argument))
is PgNodeExpression.DistinctExpr -> expression.copy(arguments = expression.arguments.map(recurse))
is PgNodeExpression.ArrayExpr -> expression.copy(elements = expression.elements.map(recurse))
is PgNodeExpression.RowExpr -> expression.copy(arguments = expression.arguments.map(recurse))
is PgNodeExpression.GroupingFunc -> expression.copy(arguments = expression.arguments.map(recurse))
is PgNodeExpression.FieldSelect -> expression.copy(argument = recurse(expression.argument))
is PgNodeExpression.JsonIsPredicate -> expression.copy(argument = recurse(expression.argument))
is PgNodeExpression.JsonConstructorExpr -> expression.copy(arguments = expression.arguments.map(recurse))
is PgNodeExpression.JsonExpr -> expression.copy(
argument = recurse(expression.argument),
onEmptyDefault = expression.onEmptyDefault?.let(recurse),
onErrorDefault = expression.onErrorDefault?.let(recurse),
)

is PgNodeExpression.XmlExpr -> expression.copy(arguments = expression.arguments.map(recurse))
} else {
null
}
return if (resolved != null && resolved !is PgNodeExpression.Unknown) resolved else expression
}
Original file line number Diff line number Diff line change
Expand Up @@ -115,28 +115,12 @@ private fun TypeSpec.Builder.addManyImplementation(statement: SqlStatement) {
val mapperReturnType = resultRowShape.mapperReturnType
val returnTypeVariable = TypeVariableName("Return")
// 1. Private helper function
val helperFunction = FunSpec.builder(statement.name)
val helperFunction = mapperFunction(statement)
.addModifiers(KModifier.PRIVATE)
.addTypeVariable(mapperReturnType)
.addTypeVariable(returnTypeVariable)
.apply {
for ((index, parameter) in statement.parameters.withIndex()) {
addParameter(ParameterSpec(statement.getParameterName(index), statement.resolveColumnType(parameter.column!!)))
}
}
.addParameter(
ParameterSpec(
MAPPER_PARAMETER_NAME,
LambdaTypeName.get(
parameters = resultRowShape.creationParameters.toTypedArray<ParameterSpec>(),
returnType = mapperReturnType,
),
),
)
.addParameter(
"processor",
ClassName("norm", "ManyProcessor")
.parameterizedBy(mapperReturnType, returnTypeVariable),
MANY_PROCESSOR.parameterizedBy(mapperReturnType, returnTypeVariable),
)
.returns(returnTypeVariable)
.addStatement("val sql = %S", statement.sql)
Expand All @@ -160,24 +144,8 @@ private fun TypeSpec.Builder.addManyImplementation(statement: SqlStatement) {
.build()
addFunction(helperFunction)
// 2. Public Many variant: override fun <T : Any> queryName(mapper: ...) -> Many<T>
val manyFunction = FunSpec.builder(statement.name)
val manyFunction = mapperFunction(statement)
.addModifiers(KModifier.OVERRIDE)
.addTypeVariable(mapperReturnType)
.apply {
for ((index, parameter) in statement.parameters.withIndex()) {
addParameter(ParameterSpec(statement.getParameterName(index), statement.resolveColumnType(parameter.column!!)))
}
}
.addParameter(
ParameterSpec(
MAPPER_PARAMETER_NAME,
LambdaTypeName.get(
parameters = resultRowShape.creationParameters.toTypedArray<ParameterSpec>(),
returnType = mapperReturnType,
),
),
)
.returns(statement.command.applyTo(mapperReturnType))
.apply {
val args = (
statement.parameters.indices.map { CodeBlock.of("%N", statement.getParameterName(it)) } + listOf(
Expand All @@ -191,19 +159,9 @@ private fun TypeSpec.Builder.addManyImplementation(statement: SqlStatement) {
addFunction(manyFunction)
// 3. If eligible, public Query variant: override fun <T : Any> queryNameDynamically(mapper: ...) -> Query<T>
if (statement.canBeDynamic) {
val dynamicName = "${statement.name}Dynamically"
val dynamicFunction = FunSpec.builder(dynamicName)
val dynamicFunction = mapperFunction(statement).build()
.toBuilder("${statement.name}Dynamically")
.addModifiers(KModifier.OVERRIDE)
.addTypeVariable(mapperReturnType)
.addParameter(
ParameterSpec(
MAPPER_PARAMETER_NAME,
LambdaTypeName.get(
parameters = resultRowShape.creationParameters.toTypedArray<ParameterSpec>(),
returnType = mapperReturnType,
),
),
)
.returns(Command.NORM_QUERY.parameterizedBy(mapperReturnType))
.addStatement(
"return %N(%N) { sql, rowReader, _ -> driver.dynamic(sql, rowReader) }",
Expand Down Expand Up @@ -484,6 +442,8 @@ private fun buildBatchWithReturn(statement: SqlStatement): FunSpec = batchWithRe
endControlFlow()
}.build()

private val MANY_PROCESSOR = ClassName(RUNTIME_PACKAGE, "ManyProcessor")

private val PROCESS_EXEC_RESULTS = MemberName(RUNTIME_PACKAGE, "combineExecBatchResults")

private val READ_GENERATED_KEYS = MemberName(RUNTIME_PACKAGE, "readGeneratedKeys")
Expand Down
Loading