Compile templates at build time - #40
Conversation
Qiq compiled templates during the request: Catalog::getCompiled() calls QiqCompiler::compile(), which mirrors the absolute source path under sys_get_temp_dir()/qiq and rewrites it whenever the mtime moves. A prod deployment on a read-only filesystem cannot serve at all, and the cache key stops matching as soon as the tree is deployed to another path. QiqCompileStep implements BEAR\Sunday\Compile\CompileStepInterface and does the whole catalog in the build phase; Compiler is bound to QiqServeCompiler, which only ever reads. Keys come from TemplateKey: the source path minus the qiq_paths root it lives under, longest match first, no realpath(). A source under no root raises TemplateOutsideRootException instead of quietly picking another key. Catalog::compileAll() needed two things guarded. Its enumeration overwrites a duplicate key with the last root, while Catalog::source() serves the first readable one, so QiqBuildCompiler skips a key that already exists and the step empties its own directory first. And RecursiveDirectoryIterator throws on a root that does not exist, which is the state of every skeleton before its first template, so missing roots are dropped from the spec list. QiqErrorPageRenderer went through Template::new(), a static factory that builds its own QiqCompiler(null): the error page kept writing during serve and would have died on a read-only filesystem, hiding the original error. It now renders with the injected Template. Templates inside a phar are still out of reach, and not because of the key rule: Catalog::split() cuts a template root at its first colon, so the "phar" scheme becomes the collection name and Catalog::source() never resolves one. That is a Qiq-level limit. This breaks existing prod modules. QiqProdModule takes over the Compiler binding, so an application that installs it and does not run the compile step raises TemplateNotCompiledException on its first render. Hpplus.Eclat and Hpplus.Spur both install it with an explicit cache path, which keeps resolving but is no longer written to by anything. bear/sunday is on ^1.8@dev until CompileStepInterface and #[BuildDir] are released; a plain 1.x-dev fails `composer validate --strict`, which the coding-standards workflow runs. The argument-less QiqProdModule is unusable until bear/package binds #[BuildDir]; the conflict entry keeps a bear/package without that binding from resolving, where Qiq would fail with Unbound at serve time.
The build directory drops the context from its path, so #[BuildDir] and the
module that bound it go away with it: bear/sunday deletes the qualifier, and
the read side derives the directory from AbstractAppMeta instead. A compiled
artifact has no context to carry - a phar does not know which context built it
and has no use for the answer.
QiqProdModule loses its cache path argument. The writer is bear/package, which
computes {appDir}/var/build/{step name}; a path the application passed could
only ever disagree with it. Applications installing the module drop the
argument.
The cost of the fixed path: one tree holds one compile. A second compile of
another context overwrites the first, so two read-only deployments need two
artifacts.
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The runner writes {appDir}/var/build/{context}/{binding key} and the read side
looked in {appDir}/var/build/{binding key}, so no artifact was ever found. It
did not fail at serve time either: the preload worker boots the application, so
bin/compile.php exited 255 and the build never finished.
AbstractAppMeta::$buildDir carries the whole path, so QiqServeCompiler stops
computing one. The BUILD_DIR constant and the concatenation onto appDir go away
with it. A module that does not build the path cannot drift from the runner
again, which is how the context went missing in the first place.
The unit tests were green throughout that mismatch, because they built the
expected directory the same wrong way the module did. Correcting one made the
other fail, so there was nothing to notice. They now run the caller side of
CompileStepInterface - mkdir {$meta->buildDir}/{binding key} for every bound
step, then invoke - and let rendering be the assertion. The directory is named
once, from the Meta the container holds and the key the Map yields. Deriving it
from appDir again fails three of the four tests.
FakeAppMeta puts a context segment in its build directory for the same reason:
without one, appDir arithmetic and $buildDir agree and the regression hides.
Rendering goes through ResourceObject::toString() rather than the cast, since
__toString() catches Throwable and trades it for a warning and an empty string.
^1.13 resolves to nothing installable: 1.13.0 is not tagged, and the 1.x branch that satisfies the range does not carry the property, so every job died in composer install. The inline alias installs the branch under test, which is what makes the suite mean anything here. Both dependency pins - this one and bear/sunday ^1.8@dev - come off when the releases exist. bear/app-meta goes back to ^1.13.
A Meta baked into DI scripts from before bear/app-meta 1.13 has no $buildDir, and QiqServeCompiler reads it in its constructor, so the failure lands while the container is built. Resolving AppInterface reaches Compiler through RenderInterface, QiqRenderer, Template and Catalog, which means every request stops, not the ones that render a template. Measured on a Meta serialized without the property: unserialize succeeds, the constructor throws Error.
1.13.0 is tagged, so the dev-build-dir pin can go.
Integration test against merged BEAR.Package
|
| step | result |
|---|---|
| compile | pass — the real CompileSteps runner wrote var/build/prod-html-app/qiq/Page/Index.php, {{h $greeting}} compiled to <?= $this->h($greeting) ?> |
| serve | pass — 200 / Qiq says: Hello Qapp, and the step artifact's mtime is unchanged, so serving only reads |
| serve with the artifact removed | pass — TemplateNotCompiledException(.../src/templates/Page/Index.php) |
| pack | pass — 7.3MB, 2238 files; src/templates/Page/Index.php and var/build/prod-html-app/qiq/Page/Index.php both packed |
| boot the archive | fail — Qiq\Exception\FileNotFound |
Two blockers, both on the source lookup
BEAR.Package hands the right values in. Measured inside the archive:
appDir phar:///private/tmp/qship2.phar
buildDir phar:///private/tmp/qship2.phar/var/build/prod-html-app
tmpDir /private/tmp/qwrite2/MyVendor/Qapp/prod-html-app/tmp
qiq_paths /private/tmp/qapp2/src/templates <- the build machine
AbstractAppMeta::__wakeup() re-points appDir and buildDir into the archive, and tmpDir correctly stays at the write directory. QiqServeCompiler reads $meta->buildDir, so the compiled side is portable. Only qiq_paths is not.
1. QiqModule::configure() freezes the path. toInstance([$this->templateDir]) is evaluated when the container is compiled, so the absolute path of the build machine is written into the DI script. __DIR__ in the installing module does not help: it is evaluated at compile time too. A provider reading AbstractAppMeta::$appDir at injection time would follow the archive, as Madapaja.TwigModule's AppPathProvider does.
2. A provider is not enough: phar:// is unparseable by Qiq's collection syntax. With qiq_paths bound to a provider returning {appDir}/src/templates, the archive still fails, and the reported paths are now empty:
Paths: Array
(
)
Qiq\Catalog::split() reads a collection:path spec, so a stream URI splits on its scheme:
spec phar:///private/tmp/qship3.phar/src/templates
strpos(':') 4
collection phar
path ///private/tmp/qship3.phar/src/templates
__DEFAULT__ ends up empty. BEAR\QiqModule\TemplateKey::split() carries the same logic — its own docblock says it mirrors Catalog::split() — so the module would need the same fix even if upstream took one. The existing PHP_OS_FAMILY === 'Windows' ? 2 : 0 offset is the same class of problem for C:; a stream URI is the third case.
The shape underneath
Serving resolves the source before it reaches the compiler:
// Qiq\Catalog::getCompiled()
$source = $this->source($name); // must be readable
if ($source === null) { throw new Exception\FileNotFound(...); }
$this->compiled[$name] = $this->compiler->compile($source);So prod needs the source tree present and enumerable even though QiqServeCompiler only ever reads {buildDir}/qiq. A read-only prod that has its compiled templates should not need the sources at all — and that is what would make an archive work without touching Catalog::split().
Related: a fix for the stream-URI split was written during earlier work on this (Keep stream URIs out of collection names, on a local Qiq checkout) and was never pushed anywhere. It is gone; it would have to be written again.
Reproduction is a fixture of eight files; say the word and I will put it somewhere you can run it.
Measured: the archive boots with no upstream change and no unfreezingFollowing the two blockers above — I tried the third option instead: let prod resolve a name, not a source.
// vendor/bear/qiq-module/src/QiqProdCatalog.php
final class QiqProdCatalog extends Catalog
{
private string $compiledDir;
public function __construct(
AbstractAppMeta $meta,
Compiler $compiler,
#[Named('qiq_extension')] string $extension = '.php',
) {
parent::__construct([], $extension, $compiler);
$this->compiledDir = $meta->buildDir . '/' . QiqCompileStep::NAME;
}
public function has(string $name): bool
{
return is_file($this->file($name));
}
/** @throws TemplateNotCompiledException */
public function getCompiled(string $name): string
{
$file = $this->file($name);
if (! is_file($file)) {
throw new TemplateNotCompiledException($name);
}
return $file;
}
/** A name is a Qiq spec, never a path: a colon in it separates the collection */
private function file(string $name): string
{
return $this->compiledDir . '/' . str_replace(':', '/', $name) . $this->extension;
}
}and in $this->bind(Catalog::class)->to(QiqProdCatalog::class);What that measuredSame fixture,
And the read-only contract holds, tested on the extracted tree:
So both blockers become moot rather than fixed: What this replaces
$key = new TemplateKey($this->paths);
$compiler = new QiqBuildCompiler($stepDir, $key);
$catalog = new Catalog($this->specs($key), $this->extension, $compiler);so paths and Open for your judgment
The upstream |
|
Addendum: I wrote above that A trace written into The render succeeds, so the absence is not a failure upstream of the call. |
Qiq\Catalog::getCompiled() resolves the source file with is_readable() before it calls the
compiler, so prod needed the template tree even though only {buildDir}/qiq is ever read.
QiqProdCatalog maps a template name straight onto its artifact, so an application that
ships no templates serves, and a phar:// appDir resolves like any other.
has() and getCompiled() are the only public Catalog methods that reach source(); both are
overridden. compileAll() iterates the paths instead, and runs at build time from a catalog
that has real ones.
QiqServeCompiler goes. With the catalog resolving by name its compile() is unreachable, it
duplicated TemplateNotCompiledException, and it was the last serve-time reader of the
qiq_paths value frozen into the DI script. Prod binds Qiq\Compiler\NonCompiler so that
getCompiler() reports a compiler that cannot write.
The name-to-artifact mapping lives in TemplateKey::ofName(), beside the source-to-key
direction the compile step uses, so the collection-to-subdirectory rule is stated once.
qiq_paths keeps its toInstance binding. Nothing reads it at serve time any more, and
QiqModule has no AbstractAppMeta, so a provider could only guess which prefix of a frozen
absolute path used to be the appDir.
|
Prod now resolves a template by name under the build directory, so an application that ships only
Three decisions:
New tests: |
1 participant
Qiq compiles a template on first render and writes the result, so a read-only tree or a packed
archive dies on the write. This compiles every template at build time instead, and the serve side
only reads.
QiqCompileStepimplementsBEAR\Sunday\Compile\CompileStepInterface, bound by engine nameQiqServeCompilerreturns{buildDir}/qiq/{key}or throwsTemplateNotCompiledException. Ittakes the directory from
AbstractAppMeta::$buildDirinstead of composing one, so it cannotdrift from the runner that writes there
QiqErrorPageRenderertakes an injectedTemplate;Template::new()used to write tosys_get_temp_dir()while serving, so a read-only filesystem killed the error page itselfQiqProdModuleno longer takes a cache path: bear/package writes the artifacts and fixes theirlocation, so a path from the application could only disagree with the writer.
BREAKING: an application that installs
QiqProdModulemust run the compile step before itserves, must drop the constructor argument, and must compile into an empty
var/buildrather thanon top of the old one. A
Metabaked into DI scripts from beforebear/app-meta1.13 has no$buildDir, and it is read while the container is built, so a leftover build stops the applicationat boot instead of losing a single template. PHP 8.2+, from 8.1.
Verified on a renamed,
chmod -R a-wtree: page and error page both render andsys_get_temp_dir()/qiqstays absent. The tests run the caller side ofCompileStepInterfaceandlet rendering carry the assertion, so
/var/build/appears in neither the module nor the tests.Three items to settle before tagging:
bear/app-metais pinned todev-build-dir as 1.13.0; back to^1.13once 1.13.0 is taggedbear/sundayis^1.8@dev;^1.9once 1.9.0 shipsbear/packageconflict floor is a guess until a release runs the compile stepsTemplates inside a phar remain unsupported: Qiq's
Catalog::split()readsphar:as a collectionname.