Skip to content

Run the SQL dialect over the client/server protocol and in the TTY#42

Draft
jnidzwetzki wants to merge 1 commit into
mainfrom
jan/integrate_optimizer
Draft

Run the SQL dialect over the client/server protocol and in the TTY#42
jnidzwetzki wants to merge 1 commit into
mainfrom
jan/integrate_optimizer

Conversation

@jnidzwetzki

Copy link
Copy Markdown
Member

The kernel could only execute plans; the SQL dialect had to be optimized either in the standalone Prolog-embedded binaries (SecondoPLTTY) or via the separate Java OptServer. This teaches the server and the TTY to run SQL directly through the embedded optimizer.

Protocol: add command level 2 (SQL dialect) to the existing framing. SecondoServer::CallSecondo routes level 2 to the embedded optimizer (before any si->Secondo call, so the optimizer's secondo/2 catalog callbacks are not reentrant), turns the SQL into an executable plan, executes it, and returns the two-element list (plan result) -- a new, backward-compatible shape only level-2 clients see. SecondoInterfaceCS learns to send level 2 and to ask an capability query.

Runtime toggle: the server reads Environment/EnableOptimizer (default on; override with SECONDO_PARAM_EnableOptimizer). SecondoMonitor gains a --no-optimizer flag (optimizer builds only) that sets that for the servers it forks. When disabled, level-2 returns ERR_OPTIMIZER_NOT_AVAILABLE and the capability query reports unavailable.

Optimizer embedding: initEmbeddedOptimizer/embeddedSqlToPlan/ embeddedOptimizerUseDatabase (UserInterfaces/SecondoPL.cpp) start SWI-Prolog and load the optimizer in-process, sharing the host interface (no second SMI environment) and loading the open database's schema on demand. Preload library(error) so restricted autoloading no longer spams must_be/2 errors while the optimizer loads.

TTY: a leading "sql" or "select" token (case-insensitive, prefix optional) is optimized -- over the network in SecondoCS, in-process in SecondoTTYBDB -- and the generated plan is shown. Both report optimizer availability at startup.

Consolidation/cleanup: the regular TTY now runs SQL, so retire the hybrids -- remove SecondoPLCS/SecondoPLTTYCS and the -pl/-pltty logic from SecondoCS, and remove SecondoPLTTY/SecondoPLTTYNT and SecondoPLTTYMode (SecondoTTYBDB replaces -pltty). Keep the -pl raw Prolog shell.

Build/CI: the default optimizer build now also generates the Optimizer/optimizer program the embedded optimizer consults at runtime (gitignored). A new CI step optimizes and executes an SQL query over the network deterministically.

@jnidzwetzki
jnidzwetzki force-pushed the jan/integrate_optimizer branch from ec047a9 to d0b3d08 Compare July 21, 2026 16:42
The kernel could only execute plans; the SQL dialect had to be optimized
either in the standalone Prolog-embedded binaries (SecondoPL*TTY*) or via the
separate Java OptServer. This teaches the server and the TTY to run SQL
directly through the embedded optimizer.

Protocol: add command level 2 (SQL dialect) to the existing <Secondo> framing.
SecondoServer::CallSecondo routes level 2 to the embedded optimizer (before any
si->Secondo call, so the optimizer's secondo/2 catalog callbacks are not
reentrant), turns the SQL into an executable plan, executes it, and returns the
two-element list (plan result) -- a new, backward-compatible shape only level-2
clients see. SecondoInterfaceCS learns to send level 2 and to ask an
<OptimizerAvailable/> capability query.

Runtime toggle: the server reads Environment/EnableOptimizer (default on;
override with SECONDO_PARAM_EnableOptimizer). SecondoMonitor gains a
--no-optimizer flag (optimizer builds only) that sets that for the servers it
forks. When disabled, level-2 returns ERR_OPTIMIZER_NOT_AVAILABLE and the
capability query reports unavailable.

Optimizer embedding: initEmbeddedOptimizer/embeddedSqlToPlan/
embeddedOptimizerUseDatabase (UserInterfaces/SecondoPL.cpp) start SWI-Prolog and
load the optimizer in-process, sharing the host interface (no second SMI
environment) and loading the open database's schema on demand. Preload
library(error) so restricted autoloading no longer spams must_be/2 errors while
the optimizer loads.

TTY: a leading "sql" or "select" token (case-insensitive, prefix optional) is
optimized -- over the network in SecondoCS, in-process in SecondoTTYBDB -- and
the generated plan is shown. Both report optimizer availability at startup.

Consolidation/cleanup: the regular TTY now runs SQL, so retire the hybrids --
remove SecondoPLCS/SecondoPLTTYCS and the -pl/-pltty logic from SecondoCS, and
remove SecondoPLTTY/SecondoPLTTYNT and SecondoPLTTYMode (SecondoTTYBDB replaces
-pltty). Keep the -pl raw Prolog shell.

Build/CI: the default optimizer build now also generates the Optimizer/optimizer
program the embedded optimizer consults at runtime (gitignored). A new CI step
optimizes and executes an SQL query over the network deterministically.
@jnidzwetzki
jnidzwetzki force-pushed the jan/integrate_optimizer branch from d0b3d08 to ed0dd04 Compare July 21, 2026 17:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant