Skip to content

Support R 4.x, modernise the build, and bring the console up to current libraries - #220

Open
SamLovick wants to merge 73 commits into
sdllc:masterfrom
SamLovick:console-upgrade
Open

Support R 4.x, modernise the build, and bring the console up to current libraries#220
SamLovick wants to merge 73 commits into
sdllc:masterfrom
SamLovick:console-upgrade

Conversation

@SamLovick

@SamLovick SamLovick commented Sep 3, 2026

Copy link
Copy Markdown

Summary

Makes BERT work with current R (tested with 4.5.2) without dropping older R, moves the native build onto a current toolchain, and brings the console up to current versions of Electron, TypeScript, Monaco, xterm, rxjs and protobuf. Addresses #194, #207 and #124.

R controller

  • The controller refused any R other than 3.5. It now checks the version at startup: a floor of 3.5, a warning above the tested major series, no ceiling. One ControlR.exe, built against R 4.5.2, starts under R 3.5.0, 4.2.2 and 4.5.2.
  • String encoding is explicit on every R version: strings from Excel and the console are marked UTF-8 when handed to R, strings taken from R are translated to UTF-8, and console text is converted to and from the Windows code page only when R's native encoding is not UTF-8.
  • ControlR.exe carries the UTF-8 code-page manifest that R's own executables use. Without it, R 4.2+ embedded in another process runs on the system code page, and strings outside it cannot exist in the session.
  • Console messages are built from the length R supplies rather than a terminating NUL; the old code could send stale bytes after the text.
  • R_LEGACY_RCOMPLEX is defined because MSVC cannot compile the _Complex member R 4.3 added to Rcomplex.

Build

  • v145 toolset, Windows SDK from the installed VS, ARM64EC configurations alongside x64.
  • protobuf 5.29 via vcpkg in manifest mode; generated code regenerated, .proto unchanged so the wire format is unaffected.
  • The R installation is an overridable MSBuild property (BertRHome); import libraries regenerated from R 4.5.2 with their .def files checked in, and R's exported variables marked DATA so the ARM64X import libraries link.
  • controlr.cc is stored as UTF-8 (it was UTF-16, which git shows as binary).

Console

  • Electron 1.8 → 44, TypeScript 2.7 → 7, Monaco 0.10 → 0.56, xterm 3 → 6, rxjs 5 → 7, google-protobuf 3 → 4, and the smaller libraries. The architecture is unchanged (Node in the renderer as an explicit opt-in, @electron/remote for the few main-process calls; navigator.clipboard replaces the removed renderer clipboard module).
  • The xterm port no longer patches xterm's prototype; the two remaining reaches into its internals (cell size, scrollback trim event) are isolated in one file.
  • The release update check is skipped for tagged development builds, and the add-in's version string carries a branch tag and build time so a local build is distinguishable in the console.
  • npm with a committed lockfile instead of yarn.

Docs

docs/BUILDING.md describes the build, the R version policy and what has been exercised; docs/MODERNISATION.md records what changed in R and the toolchain since 2018 and the remaining work (installer still bundles R 3.5, Julia 0.x controllers, strict TypeScript, Monaco as ES modules, file paths from the add-in in the code page).

Verification

  • Full solution builds Release x64 (ControlR.exe, BERT64.xll, BERTRibbon2x64.dll); ControlR also builds Release ARM64EC.
  • Used with Excel and R 4.5.2: functions from cells, the console shell and editor, copy/paste, and non-ASCII strings ("café ñ 日本" round-trips through both a cell and the shell; nchar("café") is 4).
  • R 4.2.2 and 3.5.0 have had the controller's startup check only.

Generated with the assistance of Claude Code

Sam Lovick and others added 18 commits September 3, 2026 11:17
The R 4.x work introduced several large, fully regenerable trees that
should never enter version control:

  - vcpkg_installed/ under BERT, ControlR, ControlR-45 and Ribbon
    (~750 MB each, rebuilt from vcpkg.json)
  - ARM64EC/ and ARM64/ build output alongside the existing x64/Win32
  - R-3.5.0/ and R-4.5.2/, which are local R installations used only as
    build inputs (headers and import libraries)
  - .vs/ per-machine state and .vscode/ editor-local settings

Also ignores package-lock.json under Console, which uses yarn; carrying
both lockfiles produces divergent dependency installs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The tree previously built against a source checkout of protobuf 3.5.0
sitting next to the repository, with each project hard-coding a relative
path into ../../protobuf-3.5.0. That version predates R 4.x and no longer
compiles cleanly with a current toolset.

Protobuf is now consumed through vcpkg in manifest mode. Each native
project gains a vcpkg.json declaring the dependency, and a shared
vcpkg-configuration.json pins the registry baseline so the resolved
version is reproducible.

Regenerated the protobuf output to match:

  - PB/variable.pb.cc and PB/variable.pb.h with protoc 5.29.5
  - Console/generated/variable_pb.js with the 3.21 JavaScript generator,
    and bumped the google-protobuf dependency to ^3.21.4

The .proto schema itself is unchanged, so the wire format between the
add-in, the console and the language controllers is unaffected.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The file was UTF-16, which git treats as binary, so no change to it
could be reviewed as a diff. The content is plain ASCII; only the
encoding changes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Move the projects from v141 to v145 and from a pinned Windows SDK to
the latest installed one, enable vcpkg manifest mode so protobuf comes
from the manifests added in the previous commit, and add Debug and
Release ARM64EC configurations alongside x64. The solution header moves
to Visual Studio 18 accordingly.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The controller refused to start unless the hosted R reported exactly
3.5, which is why BERT could not be pointed at a newer R even though
the embedding interface it uses had not changed. Reject only versions
older than the floor this code is built against, and warn rather than
refuse on a major series newer than the one the build was tested with.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
R for Windows switched its native encoding to UTF-8 in 4.2.0. Before
that, console output arrived in the Windows code page and was converted
to UTF-8 on the way to the console, after a validity check that let
already-UTF-8 output through. With a UTF-8 R that check is at best
redundant and at worst harmful, because a multi-byte character split
across two writes fails it and gets mangled by the conversion. Decide
once at startup, from the version R reports, whether output is passed
straight through, so one binary behaves correctly on both sides of the
4.2.0 change.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Take the R headers and import libraries from a single, overridable
location: BertRHome, defaulting to R-4.5.2 beside the projects and
settable with /p:BertRHome=<path>. Replace the checked-in R 3.5 import
libraries with ones generated from R 4.5.2, and check in the .def files
they were built from so the export lists are reviewable. Add ARM64X
import libraries for the ARM64EC configurations.

Define R_LEGACY_RCOMPLEX in every configuration: from 4.3.0,
R_ext/Complex.h defines Rcomplex as a union with a C99 _Complex member,
which MSVC does not support, and the legacy definition is the
documented escape hatch. The layout is identical, and this code only
reads and writes the r and i fields.

RebuildLibs.ps1 learns to build the ARM64X libraries, gains a usage
note, and loses a stray debugging line.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Two fixes in the console, both surfaced by running against a current CRAN
and a current R.

The package chooser scrapes available_packages_by_name.html and took the
anchor text as the package name. CRAN now wraps some names in
<span class="CRAN">, so those entries appeared in the list with raw
markup around them. Names are stripped of any tags rather than that one
span, and empty results are skipped, so a further change to the page
layout degrades instead of producing garbage entries.

MessageUtilities.VariableToObject logged every protobuf value case it
could not translate. R 4.x sends several of these, and the result was a
devtools console too noisy to read. The log is now behind a BERT_DEBUG
environment variable, following the BERT_VERSION convention already used
in update_check.ts, so the diagnostic survives but is off by default.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… roadmap

docs/BUILDING.md covers the toolchain, how the R installation is chosen
(BertRHome), the import libraries and how to regenerate them, running
against an R other than the bundled one, and which R versions the
controller accepts and which have actually been exercised.
docs/MODERNISATION.md records what changed in R and the toolchain since
2018, what this branch does about each change, and the remaining work in
a suggested order. The README points at both.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ries link

lib treats every export in a .def as a function unless told otherwise.
x64 gets away with that because R's headers import variables through
__declspec(dllimport), but the ARM64X libraries need to know which
exports have no code behind them, or the ARM64EC link fails with
LNK1373 looking for an exit thunk for R_DimNamesSymbol. RebuildLibs.ps1
now marks every name R's headers declare with LibExtern, plus the
GA_-prefixed spellings graphapp.h maps them onto, and the .def and .lib
files are regenerated from R 4.5.2. The x64 build is unaffected; the
ARM64EC configuration now links.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The console's Help menu shows the version string the add-in exports, and
until now a locally built add-in was indistinguishable from the 2.4.3
release. Split the define into the release number, which the installer
script reads, and the string the console shows, which now carries a
branch tag and the compile time of bert.cc.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The console asks bert-toolkit.com for the latest release and offers it
whenever the add-in's version is lower. A build from a development
branch carries a tag after its release number and is not a release, so
the comparison is meaningless; on the r4-support branch it offered the
2018 release as an upgrade. Return before fetching when the version is
not a plain number.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The console was pinned to Electron 1.8, TypeScript 2.7 and Monaco 0.10.
This brings the platform up to date while keeping the console's
architecture: the renderer still talks to the add-in over named pipes
and reads and watches files itself, which current Electron allows as an
explicit choice (nodeIntegration on, contextIsolation off), and the
removed remote module is replaced by @electron/remote.

Changes forced by the new versions:

  - dialogs return promises now; the editor and shell use the *Sync
    variants, which keep the existing control flow
  - clipboard.readText is asynchronous; Paste awaits it
  - Menu.popup takes an options object
  - modules exported as a function (markdown-it, less) are imported as
    defaults, since the namespace shim only copies own properties
  - Promise sites whose resolve carries no value are typed Promise<void>
  - Monaco's JSON defaults moved to a top-level namespace, and the
    model's language comes from getLanguageId() instead of a private
    field
  - the AMD loader is loaded after the console's own code, so that
    UMD-packaged libraries register with node rather than with the
    loader's global define, and Monaco loading waits for the loader
    script if it gets there first

Also: the "first run" path no longer rejects when an example file is
missing, the version parser ignores the tag a development build
carries, strict mode is explicitly off (the code predates it; making it
strict-clean is separate work), and the package manager is npm with a
committed lockfile instead of yarn.

Verified by launching the console from source: the welcome page
renders, a file opens through the File menu and Monaco highlights it.
The terminal needs Excel and is not covered yet.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rxjs 5 to 7: operators move into pipe(), Observable.create becomes the
constructor, and merge is a top-level function. Every site changes the
same way.

xterm 3 to 6: the terminal no longer patches xterm's prototype or
imports its internals. The fit, cursor-position and annotation addons
become plain functions and a class that use the public buffer API
(cursorX, cursorY, viewportY, baseY, getLine), the fit addon for the
proposed geometry, and the web-links addon. Cell sizes and the
scrollback trim event have no public equivalent, so xterm_internals.ts
holds those two reaches into xterm's core, with a fallback, and nothing
else touches it. Horizontal scrolling of wide output, which used to
shift xterm's screen element under its own viewport, now scrolls the
terminal's container instead; the fit still keeps the column count at
least as wide as the widest line. xterm stopped exporting its character
width table, so wcwidth comes from the package of that name.

google-protobuf 3.21 to 4.0 (the generated code is unchanged), chokidar
2 to 4 (5 is ESM-only), markdown-it 8 to 15, less 3 to 4, js-base64 2
to 3, all without source changes beyond the imports already made.

Verified by launching from source: startup is clean and the editor
works. The terminal needs Excel and is covered next.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…he version

Electron no longer provides its clipboard module to renderer processes,
so Copy in the shell, Paste, and the editor's copy-path command threw
on an undefined module. The standard navigator.clipboard API is what
Electron recommends instead; a small wrapper keeps the call sites as
they were. Checked in a bare renderer: the round trip works, and xterm
keeps its selection across the right-click that opens the context menu.

startup.R split BERT_VERSION on dots and coerced every part to a
number, so the tag a development build now carries produced an NA
warning that R reported at the next prompt. Parse the leading numeric
part only; the full string is still kept as version.string.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Strings from the add-in and the console are UTF-8 whatever R is hosted:
the add-in converts Excel's UTF-16 strings and COM names to UTF-8, and
the console is a web page. The controller used to hand them to R as
native strings, which is right from 4.2.0 and wrong before, and took
strings back from R with CHAR(), guessing at the encoding with a
validity check and a code-page conversion.

Now strings into R are marked UTF-8 (Rf_mkCharLenCE with CE_UTF8) and
strings out of R go through Rf_translateCharUTF8, which honours the
encoding mark each string carries. Console keyboard input is converted
to the Windows code page under an R older than 4.2.0, the mirror of the
existing output conversion, keyed off the same startup check. The one
string left native is the path given to source(): the add-in produces
it with the ANSI file APIs, and the fix for that belongs on its side.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Two things stood between the encoding work and a working shell.

R 4.2.0 and later use UTF-8 as their native encoding only in a process
that has the UTF-8 code page, which R's own executables get from a
manifest. Embedded in ControlR.exe, R was running on code page 1252, so
the version check alone was the wrong test for UTF-8 pass-through:
nchar("café") gave 5. ControlR.exe now carries the same manifest
setting, and the pass-through is decided by the version and GetACP()
together, with the code-page conversion as the fallback.

ConsoleMessage built the protobuf string from R's buffer as if it were
NUL-terminated, ignoring the length R passes. Stale bytes after the
text went out with it; the old JavaScript protobuf runtime substituted
characters silently, the current one rejects the message, and the
prompt that followed was lost, leaving the shell busy for ever. The
console now also logs and skips a frame it cannot decode instead of
abandoning the rest of the batch.

The console gains a BERT_CONSOLE_DEBUG_PORT environment variable that
exposes the DevTools protocol, which is how this was diagnosed and
verified: l10n_info() reports UTF-8, nchar("café") is 4, and
"café ñ 日本" round-trips through the shell.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sam Lovick and others added 11 commits September 3, 2026 15:21
…se script

The NSIS script now installs over an existing BERT 2 without touching
bert-config.json, user-stylesheet.less or Documents\BERT2 (created from
templates only when absent, as before), replaces the application-owned
console, module and startup directories outright, and ships only what
the x64 build produces: no 32-bit add-in, no Julia controllers, no
bundled R. It finds the installed 64-bit R in the registry and writes it
into bert-languages.json as the last candidate home, after the R-3.5.0
an older installer may have left, so an upgraded installation moves to
the installed R without editing a file. BERT.R.home still overrides.

build-release.ps1 replaces the 2018 build-installer.ps1 (Visual Studio
2017 paths, code signing, 32-bit builds): it builds the native projects,
installs the BERTModule R package into Build\module with the registered
R, packages the console, runs makensis, and zips the same files together
with INSTALL-FROM-ZIP.md for people who copy by hand.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…butable

The add-in, ribbon and controller already use the static C runtime, but
protobuf and abseil came from vcpkg's x64-windows triplet as DLLs built
against the dynamic runtime. On a machine without the Visual C++
redistributable the add-in could not load its dependency and Excel
dropped it silently: the ribbon appeared, R functions gave #NAME?, and
the console would not open. The x64 configurations now use the
x64-windows-static triplet, so protobuf and abseil are linked in and
the shipped files import only Windows system DLLs.

The installer stops shipping the two DLLs and removes them from an
earlier installation; the release script refuses to package a binary
that still imports the runtime or protobuf DLLs, and finds makensis
where the NSIS installer puts it. Build tag bumped to -r5 so the
corrected release is distinguishable from -r4.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Three separate faults, all in the editor pane:

The right-click menu was built by asking monaco's contextmenu
contribution for its entries. That is private API, and in monaco 0.56 the
method needs arguments it was not given, so the handler threw and no menu
appeared. The entries are listed explicitly now, with the execute
commands this class registers per language appended, so the menu no
longer depends on monaco's internals.

Monaco no longer registers cut, copy and paste actions at all. It only
ever registered them where the document.execCommand editing commands
were available, and chromium has withdrawn those from web content. Cut,
copy, paste and select all are therefore implemented here, against the
model and the clipboard.

Since monaco moved to the EditContext input surface, the browser's own
paste no longer reaches the editor: a plain textarea in the same window
receives Ctrl+V, the editor does not. The clipboard shortcuts are bound
in monaco, which also stops the browser acting on the same keystroke.

The Edit menu's cut, copy, paste, find and replace items were never
wired to anything; they fell through to a debug log. They now act on
whichever pane last had the focus, remembered as the menu itself takes
the focus away. The clipboard helper falls back to the main process
clipboard, because the web clipboard API refuses to run while the
document is not focused, which can happen around native menus.

Verified over the devtools protocol: menu and keyboard cut, copy and
paste, save writing to disk and clearing the dirty flag, and the context
menu handler running without error.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…or IntelliSense

A function described with a single string was silently ignored. The
controller only read the description attribute when it arrived as an
array, and a character vector of length one crosses the wire as a scalar,
so attr(f, "description") <- "text" did nothing while c("text", "") worked.

docs/FUNCTION-HELP.md now covers the whole facility: how to write the
descriptions, where Excel shows them (the Insert Function and Function
Arguments dialogs), and why the inline tooltip is not among them. Excel
draws that tooltip only for its own functions, and gives an add-in no way
to produce it. The example functions and the console welcome page point
at it, since none of this was written down anywhere.

For the tooltip, the add-in now writes its registered functions and their
descriptions to BERT-IntelliSense.intellisense.xml, in the format the
Excel-DNA IntelliSense add-in reads. That add-in draws the tooltip Excel
withholds. It looks for the file beside a loaded workbook or workbook
add-in and ignores XLL add-ins, so BERT-IntelliSense.xlam is shipped as
the thing to find it beside: an empty add-in with no code. Both are
optional, and BERT does not depend on either.

Verified with Excel and R 4.5.2: a function described with one string and
one described argument by argument both reach the generated file, with
the reserved characters escaped.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Loaded the carrier add-in and Excel-DNA IntelliSense 1.9.0 into 64-bit
Excel with BERT: typing =R.TestAdd( shows the signature with the current
argument in bold, the function description and the argument description,
taken from the description attribute on the R function. Notes the
version tested and that the add-in is not code-signed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…for R

The components page now offers "Function help in the formula bar", off by
default. Selecting it installs the Excel-DNA IntelliSense add-in and the
carrier add-in, and adds both to Excel's add-in list; running the installer
again with the box cleared takes them out again, and uninstalling BERT
removes them too. /HELP-FEATURE and /NO-HELP-FEATURE set it from the
command line, for scripted installs.

Excel keeps its add-in list as OPEN, OPEN1, OPEN2 and so on, with no gaps.
The installer adds to the first free slot, skips entries already there, and
on removal moves the last entry into the gap, so add-ins the user already
had are left alone. The IntelliSense add-in is MIT licensed and ships with
its licence; the build script downloads it, pinned by version and checked
against its hash, so the binary is not in the repository.

R detection was one registry value: the InstallPath the current R records.
An R that did not write it, or wrote it and was then removed, looked like
no R at all, and the installer said so even where R was installed. It now
looks at that value for both R64 and R, per user and per machine, then at
the per-version subkeys below them, then in Program Files, and keeps the
newest it finds. It says nothing unless there is no R at all, or the one it
found is older than 4.2, and it says which one that is.

Message boxes gained silent defaults, so an unattended install cannot sit
waiting behind a dialog.

Verified on this machine: installing with the feature registers both
add-ins and the tooltip works in the session straight afterwards; installing
without it removes them and leaves the three add-ins already registered
untouched; three installs in a row produce no duplicates; and a per-user R
registered only as a version subkey is found and preferred when it is newer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Closing the console does not close it: the window cancels the close and
asks the add-in to hide it, so the ribbon command normally just shows the
window again. That path hid the window, rewrote its window styles by hand
(moving it between an application window and a tool window, and setting
the visible bit itself), and showed it again. Rewriting the styles leaves
the window out of step with what chromium believes about it, and under
current chromium the console came back with a blank client area that took
no input: a title bar and menus over nothing. It now shows and hides the
window and leaves the styles alone, which is all it needed to do; a
hidden window has no taskbar button anyway, which is what the style
changes were for.

The process id was also never cleared, so if the console had gone away
the ribbon command would look for a window that no longer existed and do
nothing at all, for the rest of the Excel session. It now checks whether
the process is still running and starts a new console if it is not.

Reproduced and fixed against Excel: open the console, close it, open it
again, and it comes back with the editor and shell as it was.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Install Packages sat on "loading package list" for ever. The list URL was
built with path.join, which is for file paths: on windows it turned the
mirror into ".\https:\cloud.r-project.org\web\packages\..." and the fetch
failed. It is a URL, so it is joined as one now, and a fetch that fails
says so in the dialog instead of leaving it waiting.

getOption("repos") also does not always arrive in the same shape. R holds
it in a named character vector, which reaches the console as an object,
but a vector of one loses its names on the way and arrives as a plain
string, which is what happens when CRAN is the only repo set. The code
only handled the object, so it either offered the mirror list again or
threw. It now accepts either. Choosing a mirror no longer writes the
string "undefined" into the option as CRANextra when there was none.

The list is now two columns, a fixed one for the package name and the
rest for the description, so the descriptions line up down the list;
either can be clipped, and the full text is in the tooltip.

One notch of the mouse wheel used to scroll the list all the way to the
end. The list recycles its rows as it scrolls, so the content above the
viewport keeps changing size; chromium's scroll anchoring corrects the
scroll position to compensate, which fires another scroll event, which
recycles more rows. Scroll anchoring is switched off for these lists.
Measured: one notch now moves 300px and stops, where it used to climb
about 6000px every 400ms until it ran out of list.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The terminal already puts itself on the window object, and the editor was
added alongside it. The menu utilities go there too, so the menu commands
can be driven and inspected while testing the interface. All three are
behind the dev flags the add-in passes, so a normal install exposes
nothing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Closing a modified file discarded the changes without a word; the code has
carried a "FIXME: warn if dirty" since 2018. Closing one now asks whether to
save, discard or keep it open, from the File menu and from the tab's own
close button alike. An unmodified file still closes straight away.

Saving from that question has to be able to fail -- an empty document opens
the save-as dialog, which the user can cancel -- so SaveTab now returns a
promise that resolves true only once the file is on disk, and the tab stays
open if it does not.

Close All and Close Others ask one file at a time and stop at the first file
the user decides to keep, rather than firing every question at once.

Alerts had a related fault: the outgoing alert hides itself when its fade-out
transition ends, which would hide an alert opened in the meantime, leaving a
modal nobody could answer. The next alert now calls off the pending fade.

Verified against the packaged console over the devtools protocol: cancel,
discard and save from the menu and from the tab button, escape, an
unmodified file, and Close All over five modified files.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
With three answers -- save, don't save, cancel -- the labels ran together
into what read as a single line of words on one green block.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sam Lovick and others added 30 commits September 4, 2026 13:34
vcpkg-configuration.json, which pins the registry baseline, sat only in
BERT/. vcpkg looks for it beside the manifest it is resolving, so building
ControlR or Ribbon on their own took whatever baseline the local vcpkg
happened to default to.

That is how the CI build failed: it resolved a protobuf newer than the 5.29
that generated the checked-in PB/*.pb.*, and protobuf's generated headers
refuse a runtime they were not generated against --

  PB/variable.pb.h(16,1): error C1189: "Protobuf C++ gencode is built with an
  incompatible version of"

It does not show up on a machine whose vcpkg already resolves 5.29, which is
why the local build has been fine. Copying the same configuration beside each
manifest makes all three resolve the same versions anywhere. Verified by
rebuilding ControlR locally from scratch: same protobuf, same binary.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The job used r-version: release, which is now R 4.6.1, and the controller
does not compile against it: Rf_isFrame is gone from the API, and the
ReadConsole callback signature changed. Supporting 4.6 is work in its own
right; until then the job should build what the tree claims to support, per
docs/BUILDING.md, rather than sit permanently red.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
R 4.6 removed Rf_isFrame and changed the ReadConsole callback signature, so
the controller does not compile against its headers. That is work for another
day; what matters now is that nobody is left to discover it by accident.

The runtime gate only warned when the major version was newer than 4, so R
4.6 passed it in silence and the first sign of trouble would have come later
and less clearly. It now compares the minor version too, and says the version
is not supported yet rather than merely "newer than tested".

The README, the build notes and the changelog all state the supported range
in the same terms.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The controller declared

  int R_ReadConsole(const char *prompt, char *buf, int len, int addtohistory)

which is R 3.5's signature. R 4.2 changed the buffer to unsigned char *, so
against a stock R 4.2 or later that assignment does not convert:

  rinterface_win.cc(174,21): error C2440: '=': cannot convert from
  'int (__cdecl *)(const char *,char *,int,int)'

It built here only because the R-4.5.2 tree this repository builds against
had RStartup.h edited to say char * -- so the shipped binaries were compiled
against a modified R header, and a clean clone with a stock R could not build
the controller at all. CI found it: the same source failed on the runner
against a stock 4.5.2 from CRAN.

The callback now follows the headers it is compiled against, and the buffer
is cast where it is used. Verified by building against stock R 4.5.2 and
stock R 4.2.2, and against the repository's own tree with the header put
back the way CRAN ships it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The controller build never got green on a hosted runner. It is dropped rather
than left failing or muted, with the whole attempt recorded in the workflow:
what it needed (R 4.5.2, BertRHome, vcpkg integrate install, toolset v143),
where it stops (an unresolved __std_rotate from libprotobuf and abseil, a C
runtime mismatch -- vcpkg resolves the dynamic-CRT triplet on the runner while
the project links the static one), and what to try next.

It earned its keep before being parked: it found the vcpkg baseline pinned for
the add-in only, and R_ReadConsole carrying R 3.5's signature while compiling
locally against a hand-edited R header. Both are fixed, and both would have
gone unnoticed otherwise.

What remains runs on every push and passes: the console build and startup.R
against four R versions. The native build is exercised locally by
Install/build-release.ps1 on every release.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two separate things, found together.

R 4.6 removed Rf_isFrame. The one place that used it now calls
Rf_inherits(sexp, "data.frame"), which does the same job and has been stable
API since long before 3.5 -- Rf_isDataFrame would be the documented
replacement, but it only arrived in 4.5 and this has to build against 3.5.
That was the only source change 4.6 needed: the controller compiles against
4.6's headers, links against the existing import libraries, and runs.

The larger finding is about graphics. BERTModule carries the graphics devices
and R checks its graphics engine version when a device is created; that
version changes between R series (4.2 R_GE_group, 4.5 R_GE_glyphs, 4.6
R_GE_fontVar). One module was built and shipped, so graphics only ever worked
on whichever series the build machine had: r9 shipped a 4.5 module, and
plotting from a cell failed on R 4.2 exactly as it failed on 4.6, with
"Graphics API version mismatch". Nobody noticed because the build machine
runs 4.5.

The install now ships module/<major>.<minor> and startup.R loads the one
matching the hosted R. When there is no match it falls back to any module it
has -- a module from another series still provides the xlReference class and
the helpers, so everything except drawing keeps working -- and says at
startup that graphics will not be available. build-release.ps1 takes
-ModuleRHomes and builds one module per series, cleaning Module/src between
them: R CMD INSTALL will otherwise relink from objects compiled against
another R, which is what made a module built "for 4.6" still fail the version
check while it was being diagnosed.

Verified in Excel against R 4.5.2, 4.6.1 and 4.2.2: drawing works on 4.5 and
4.6, and on 4.2 the cell functions, callbacks and Excel references all work
while drawing reports the mismatch.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The per-series module loop used $home as its variable, which powershell
refuses to assign. Renamed to $r_home.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
BERTModule has to be built once per R series, and each series needs its own R
and its matching Rtools -- well over a gigabyte of downloads to do by hand,
which is why r10 shipped modules for 4.5 and 4.6 only.

Runners can do it instead. This builds the module for 4.2 through 4.6, with
3.5 allowed to fail rather than fail the run, and uploads each as an artifact.
Each job cleans Module/src first: R CMD INSTALL will relink from objects
compiled against another R, and the resulting module claims one series while
behaving as another. Each job also checks the DESCRIPTION says what it should
before uploading, so that cannot pass unnoticed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Github runs pwsh steps with $ErrorActionPreference = 'Stop', so
Get-ChildItem matching nothing is fatal -- and Module/src holds no object
files on a fresh checkout, so all six module builds failed before R was
reached. Guard the cleanup.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The support position was scattered across a sentence here and a release note
there, and it was easy to read the automatic module selection as meaning any
R would work. It does not: the selection is automatic, but it can only choose
from the modules that ship, and r10 ships 4.5 and 4.6.

Adds a section at the top of the changelog: a table of what works on which R
-- functions, console, Excel references, graphics, function help -- what was
tested against and what is inferred, why it varies (the graphics engine
version changes between R series, and a module from a nearby series still
loads while one from 3.5 does not), and what would widen it.

It also records why separate releases per R version are not the answer: the
only thing that differs is a 175 KB module, selection is automatic, and
splitting would make a user who upgrades R need a different download.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every module build died in about a second with no output: the first command
in the step was an Rscript call to ask R its own version, and when that fails
under ErrorActionPreference=Stop the step ends before anything is logged. The
matrix already carries the version, so derive the series from it and let the
DESCRIPTION check catch any mismatch afterwards.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Powershell aliases R to Invoke-History, so '& R CMD INSTALL' ran the history
cmdlet and the step failed with an Invoke-History error while the log looked
as though R had been reached. Rscript has no such alias, which is why the
version check on the line above printed correctly and made this look like an
R problem rather than a shell one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
r10 shipped modules for 4.5 and 4.6, so drawing failed on 4.2 through 4.4 and
3.5 lost Excel references as well. Modules now ship for 3.5, 4.2, 4.3, 4.4,
4.5 and 4.6 -- the whole range the controller runs on -- and they come to
about 3 MB together.

What made this practical was building them on runners rather than here: each
series needs its own R and its matching Rtools, which is a gigabyte-plus of
downloads per series on a build machine. build-release.ps1 -FetchModules
takes them from the last successful run of the "graphics modules" workflow,
checking each artifact really was built for the R series it claims before it
goes into the package -- the workflow checks the same thing, because a module
relinked from stale objects will otherwise claim one series and behave as
another.

The gh calls name the repository explicitly: this clone has an upstream
remote too, and gh resolves there by default, where the workflow does not
exist.

Verified from the installer in Excel on 3.5.0, 4.2.2, 4.5.2 and 4.6.1:
drawing and Excel references work on all four. 4.3 and 4.4 ship modules built
and checked on CI but are not tested here for want of those R versions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Excel has no complex type, so BERT sends one as text in the a+bi form Excel's
own engineering functions read. The conversion used a default stringstream,
which formats to six significant figures, so

  314159.2653589793 + 2.718281828459045i

reached the cell as "314159+2.71828i" -- and IMREAL of it returned 314159.
No error, no warning; the digits were simply gone. Doubles were never
affected, since they cross as binary doubles.

Set 15 significant digits, which is what Excel keeps internally and what R
prints. Checked in Excel: the value now shows all 15 digits, Excel's IM*
functions still parse it, and IMREAL of a complex 1/3 returned from R minus
1/3 computed in the sheet is exactly zero.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
In rendered markdown -- the release notes and the welcome page -- paragraphs
are padded 10px on the left while h2 and h3 were padded 16px, so every
subheading sat further in than the text it introduced. All three heading
levels now share the paragraph indent.

Measured after the change: h1, h2, h3, p, table and pre all start at the same
left edge.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two things in rendered markdown, following on from the heading indent.

A table carries no left padding of its own, so while its box lined up with
the paragraphs, its first column started ten pixels further left than the
text above it and read as an outdent. It now has the same left offset, with
cell padding and a rule under the header row so the columns are legible.

List items ran together. They now get 1em beneath them, which measures as the
same 14px gap that separates paragraphs.

Measured after the change: heading, paragraph and table text all start at the
same left edge, and the gap between list items matches the gap between
paragraphs exactly.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
So they are not lost between now and whenever r13 is cut.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Text started hard against the edge of the pane. Inset the scrolling
container by 6px rather than the terminal element inside it: the fit
addon sizes columns from the scrolling node's width, so a narrower
terminal would overflow it and add a spurious horizontal scrollbar.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…Build

Build\bert2.ico was an untracked duplicate of the checked-in Install copy,
and the release build fails without it. The build now copies it, and Build
ignores the copy, so a clean tree stays clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Excel gives a function one line in its dialogs and no way to show more.
It does have a help topic -- the "Help on this function" button -- but it
takes a URL or a .chm and nothing else, and BERT was filling that slot
with the registration index, which is not a topic and did nothing.

So: startup.R reads the `#'` block above each definition (a comment is
not part of the closure, so the file has to be read after it is sourced),
renders it into a self-contained page under <home>\help, and the add-in
serves those pages from a loopback listener and registers the address as
the topic. The IntelliSense XML gets the same address as HelpTopic, which
that add-in draws as a link in the tooltip.

The block also fills in the one-line description and the argument
descriptions when the function has no description attribute, so a
function is documented once and both routes agree. attr(f, "help") takes
the same text, and attr(f, "help.url") points somewhere else entirely.

Two fields on the function descriptor carry it: help_url and help_file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Open files are remembered across sessions, but the text came back from
localStorage. If the file changed while the console was not running -- an
edit elsewhere, or a different file saved over it -- the tab showed the
old contents with no sign of it, and a deleted file came back as though
it still existed.

Restored tabs now read from disk, and a tab whose file has gone is not
restored at all (its cache entry goes too). A modified, never-saved tab
keeps its text: that only exists in the cache, so throwing it away would
lose work. Since a dropped tab counts as removed, the unclose list is
cleared after the restore, so a deleted file cannot be unclosed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Open and Save As started wherever the shell last left them, which is
rarely where the R files BERT loads live. Both now start at
functionsDirectory from bert-config.json, expanding environment variables
the way the add-in does when it reads the same setting. Save As only
suggests it for a document that has never been saved; one with a path
still opens where it lives.

Falls back to the previous behaviour when the setting is absent or names
a directory that no longer exists.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Excel fixes arity at registration and calls a distinct entry point per
function, so the pool of dispatchers is compiled with a fixed parameter
count -- 16 since the beginning, with the seventeenth argument silently
not passed. The count is now BERT_MAX_ARGUMENTS, one constant, with the
type string and the xlfRegister parameter block derived from it, and the
parameter list written once as a macro list so the four places that
repeat it cannot drift. BERT.Call gets the same width.

64 rather than Excel's ceiling of 254 because every one of the 2048
dispatchers carries the full width: 16 -> 64 costs 1.4 MB of code.

Two things the wider functions exposed, both fixed here: an argument-name
list longer than 255 characters makes excel reject the registration
outright, so it is now cut at the last name that fits; and building that
list for a function with no arguments read past the end of an empty
string.

Verified in Excel with functions of 16, 17, 20, 40, 64 and 70 arguments.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant