compiler: implement go:cgo_import_dynamic and lower Darwin cgo import trampolines (#5604) - #5612
compiler: implement go:cgo_import_dynamic and lower Darwin cgo import trampolines (#5604)#5612neomantra wants to merge 3 commits into
Conversation
Signed-off-by: Evan Wies <evan@neomantra.net>
Route the remaining variadic imports (open, openat, fcntl) through fixed-signature C wrappers the way ioctl already is: the syscall engine calls imported addresses through fixed-signature function pointers, and a variadic callee reads its variadic arguments from the stack on darwin/arm64, so direct calls silently receive garbage arguments (an x/sys unix.Open created files with mode 0 instead of the requested mode). Of the symbols darwin's generated syscall wrappers import, exactly open, openat, fcntl, and ioctl are variadic. Also accept the one- and two-operand forms of //go:cgo_import_dynamic like the gc compiler does, ignore the unused library operand, and only replace loads of uintptr-typed trampoline globals. Signed-off-by: Evan Wies <evan@neomantra.net>
The standard library's function-based trampoline pattern has the same variadic problem as the address-global pattern: of the libc functions darwin's zsyscall wrappers import, open, openat, fcntl, and ioctl are variadic, but createDarwinFuncPCABI0Call only routed open through a fixed-signature wrapper. As a result syscall.SetNonblock (fcntl F_SETFL with the new flags in the variadic slot) observably wrote garbage file flags on darwin/arm64, and the stdlib ioctl and openat paths were equally unsound. Use the shared darwinVariadicImports table for this path too, replacing the open-only special case. Signed-off-by: Evan Wies <evan@neomantra.net>
|
This also covers the stdlib fcntl fix in #5632. I will close that narrower duplicate and point the Darwin process work in #5634 here. The regression tests from #5632 remain available in src/os/fcntl_test.go at yohimik/tinygo@9161a8c. They cover integer F_SETFL and pointer F_GETLK arguments. Please reuse them if useful. I have read this implementation and its test record, but have not yet run those tests against this branch. |
Use posix_spawn on hosted Linux and Darwin. Map process files, apply the working directory and process group, and clear the child signal mask. Use wait4 for process status and support Kill and Signal. Mark Darwin pipes close-on-exec under ForkLock. Darwin also needs the fcntl wrapper in PR tinygo-org#5612 and the libSystem symbols in PR tinygo-org#5636. Concurrent spawn and pipe creation need the RWMutex fix in PR tinygo-org#5630. Keep the process stubs on other targets and add process regression tests.
|
Tested your head 08df152 in the combined hosted candidate e7d34c8, built by CI. The downloaded Darwin arm64 compiler builds unchanged Crier 7edaff9, and the earlier failure before main in x/sys/cpu.darwinSysctlEnabled is absent. Crier E2E passes 142 cases with no failures and one platform trust-store skip. The retained F_SETFL integer and F_GETLK pointer tests both pass, as do all process remapping tests. Full Linux and macOS candidate CI passed. This is combined-candidate evidence, not a claim that this PR alone supplies all Crier dependencies. Exact artifact IDs, hashes, logs, and limits are in https://github.com/yohimik/crier/blob/7c4ea43/docs/operations/tinygo-comparison-2026-09-05.md . Thank you for the broader stdlib and x/sys support. |
As noted in #5604, this was worked through with LLM. It adds parsing of the
go:cgo_import_dynamicdirective and implementation for Darwin.The other Tinygo-supported platform that could use this is OpenBSD, but is more important for Darwin. It targets
loadASTCommentsgenerally because these directives are not necessarily attached to a function/module/etc; the comment there explains it.The following description of the PR is LLM-generated from our journey; it is verbose but comprehensive. I have read it and I have reviewed all the submitted code, both humanually and with multiple LLMs. Thanks for the review.
Summary
Extend TinyGo's existing Darwin stdlib trampoline lowering to the variable-based pattern used by
golang.org/x/sys/unixandinternal/syscall/unix, and route all four variadic libc imports (open,openat,fcntl,ioctl) through fixed-signature wrappers on both trampoline paths.Fixes #5604.
Fixes #5365.
Builds on #5401.
The companion
_ioctl/___sincos_stretstubs (macos-minimal-sdk#5) are already indevvia the macos-minimal-sdk v0.1.0 submodule update.Problem
Darwin syscall wrappers generated by x/sys declare globals such as
libc_ioctl_trampoline_addrand associate them with dylib symbols using file-level//go:cgo_import_dynamicpragmas. TinyGo does not compile the assembly that initializes those globals, so the existing libc syscall engine receives address zero and crashes beforemainin Bubble Tea programs.TinyGo already avoids the equivalent assembly trampoline for the standard library:
createDarwinFuncPCABI0Callrecognizesabi.FuncPCABI0(libc_*_trampoline)and substitutes the address of an external libc declaration. This change applies that same lowering to the x/sys global-load pattern.That existing stdlib lowering also carries a silent bug of its own: among its variadic imports it routes only
openthrough a fixed-signature wrapper, so stdlibfcntl,ioctl, andopenatcalls corrupt their variadic arguments on darwin/arm64. This PR fixes that too, since the new lowering needs the same wrapper set anyway.Scope
This is a targeted implementation of the
//go:cgo_import_dynamicpattern used by Darwin's generated syscall wrappers, not general cross-platform support for the directive. Although the file-level metadata is parsed during compilation, thelibc_*_trampoline_addrsymbol-address substitution is restricted toGOOS=darwin; non-Darwin targets retain their existing behavior.Upstream Go also uses
//go:cgo_import_dynamicon platforms such as OpenBSD, AIX, Solaris, and illumos. Supporting those platforms, the directive's library operand, other use patterns, and platform-specific linking or calling conventions remains out of scope.Implementation
//go:cgo_import_dynamicpragmas into a package-local map for Darwin trampoline lowering, accepting the same one-, two-, and three-operand forms as the gc compiler (local [remote ["library"]]); the remote symbol defaults to the local one and the library operand is ignored (the linker already resolves against libSystem).libc_*_trampoline_addrglobals withptrtointof an external declaration for the pragma's remote symbol. Globals of any other type keep their normal load.$INODE64variants.darwinVariadicImportstable. Of the symbols Darwin's generated syscall wrappers import (zsyscall_darwin_*.goin x/sys and the standard library), exactlyopen,openat,fcntl, andioctlare variadic. The syscall engine calls imported addresses through fixed-signature function pointers (tinygo_syscallXand friends) that pass every argument in a register, while a variadic callee reads its variadic arguments from the stack on darwin/arm64, so direct calls silently receive garbage in the variadic slot (observed:EFAULTfromioctl;unix.Opencreating files with mode 0 instead of the requested mode).FuncPCABI0trampoline lowering (createDarwinFuncPCABI0Call) at the same table, replacing itsopen-only special case. This fixes the pre-existing silent bug on the stdlib path:syscall.SetNonblock(fcntlF_SETFLwith the new flags in the variadic slot) observably wrote garbage file flags on darwin/arm64, and the stdlibioctlandopenattrampolines were equally unsound.The companion
_ioctland___sincos_stretstub additions landed upstream in macos-minimal-sdk#5 and reacheddevwith the macos-minimal-sdk v0.1.0 submodule update, so this branch (rebased onto that dev) builds and links darwin x/sys programs out of the box with no SDK changes of its own. No new stubs are needed foropen,openat, orfcntl— those symbols are already in the stock stub list, and their prototypes come from the SDK'ssys/fcntl.h.Verification
go test -tags llvm22 ./compiler -count=1golang.org/x/sys/unix.IoctlGetTermiosrepro under a PTY: returned a populated termios value and<nil>.github.com/charmbracelet/x/term.IsTerminalunder a PTY:true.term.MakeRawandterm.Restoreunder a PTY: both<nil>.examples/spinner: initialized, rendered animated frames, and exited onq.examples/list-default: initialized, rendered the list UI, and exited onq.unix.Openwith mode0o640created the file with mode640exactly,unix.FcntlInt(F_DUPFD, 50)returned fd50, andunix.Openatwith mode0o600created mode600exactly. The pre-fix compiler produced mode0, fd49, and mode41on the same program — the silent-corruption failure mode.FuncPCABI0path):syscall.SetNonblock(fd, true)followed by anF_GETFLreadback yields flags exactly0o4(O_NONBLOCK), and a read from the empty pipe returnsEAGAINimmediately. Without theFuncPCABI0change, the same program showed garbage flags (0o20000110),O_NONBLOCKunset, and the read blocked forever; the blocking read was also reproduced on stockdevat f71b630 with a stdlib-only variant.No startup panic or
os/signal/SIGWINCH failure surfaced in this ladder.Context
This is the actual current Darwin-native root cause behind #5365. It is the "class 2" failure described by @mparrett in #4794 and is independent of the public
syscall.Syscall*API discussed there.An earlier attempt in #5403 made public
syscall.Syscall*calls work by lowering syscall numbers directly to raw Darwin kernel instructions. That was a separate route which bypassed Darwin's libc-based dispatch and was never used by the x/sys/Bubble Tea call chain.After #5401 made x/sys's private linknames resolve, the failing program reached TinyGo's existing
syscall.syscalln/syscall.rawsyscallnlibc dispatcher with the correctioctlarguments but a zero function pointer. The dispatcher was not broken; the assembly-generated bridge that supplied its libc function pointer was missing. This change supplies that address and lets x/sys use TinyGo's existing libc path as intended.This does not implement public
syscall.Syscall*-by-number on Darwin, nor does it guarantee that every libc symbol is already present in the macOS minimal SDK.