Machine code monitor debugger

[chrisgleissner]

Overview

This PR adds debugging support to the Machine Code Monitor's Assembly view.

It provides:

• Step Over, Step Into, Step Out, Continue, and Run to Cursor.
• Up to 10 breakpoints.
• Live CPU state and prediction of the next execution target.
• Debugging of RAM, RAM under ROM, and visible BASIC/KERNAL ROM on the U64.
• Support for Telnet, UI Overlay, and UI Freeze modes.

Current status

Ultimate 64 Elite I: all three monitor suites pass on firmware built from this branch and deployed over JTAG.

Suite	Result
machine-code-monitor (Telnet, Overlay, Freeze)	pass
machine-code-monitor-debug	89 of 89 checks
machine-code-monitor-matrix	45 of 45 cells, 0 failed, 0 blocked

U2+L in a C64 Ultimate host: the lanes the cartridge supports pass.

Lane	Result
Harness unit tests	77 of 77
Matrix --focus alerts	pass
Matrix --focus banking	pass
Matrix --focus entry-footer	24 of 24 cells
monitor_test.py --target u2	25 of 25 checks

Remaining validation:

• Run the complete debugger E2E suite repeatedly with no failures on an Ultimate 64 Elite I.
• Show CPU banking on the monitor's footer from the moment it opens, on both targets.
• Bring monitor_debug_test.py green against the U2+L over Telnet. First measurement is 10 passed, 21 failed, 6 skipped; see "Outstanding work" below.

Demo

The demo shows a small program cycling the background colour, followed by debugging through KERNAL and BASIC code:

https://youtu.be/ECsqq5HKPlE

---

Features

This chapter shows the main features of the debugger. For full details, please read the Debug Mode chapter of the Machine Code Monitor documentation.

Execution control

Action	Shortcut	Behaviour
Step Over	D	Executes the current instruction without entering a JSR.
Step Into	T	Executes the current instruction and enters a JSR.
Step Out	O	Continues to the return address of the current subroutine.
Continue	G	Runs until an enabled breakpoint is reached.
Run to Cursor	K	Runs until the selected instruction is reached.

Debugging supports:

• RAM.
• RAM under ROM.
• Visible BASIC and KERNAL ROM on the U64.
• Telnet, UI Overlay, and UI Freeze modes.

Live CPU state

The debug footer shows:

• Program counter.
• Accumulator, X register, Y register, and stack pointer.
• Processor flags.
• IRQ and NMI vectors at $0314/$0315 and $0318/$0319.
• Predicted jump, branch, call, and return targets.

Active flags and important values are highlighted. Branch targets are highlighted only when the branch will be taken.

The next instruction is marked in the Assembly view with >...<, allowing the cursor to move elsewhere while the current execution position remains visible.

Breakpoints

The debugger supports up to 10 non-persistent breakpoints.

Action	Shortcut
Toggle a breakpoint on the cursor line	R
Open the breakpoint list	C=+R
Jump to a breakpoint slot	0 to 9
Change its label	L
Set it to the cursor address	S
Enable or disable it	E
Delete it	DEL

Breakpoints appear as [BRKx], where x is the slot number. A custom label replaces this with [LABL].

RAM breakpoints work on both U64 and U2. On the U64, visible-ROM breakpoints temporarily modify the FPGA's writable copies of the BASIC and KERNAL ROM images. Persistent ROM storage is never changed.

Monitor integration

Debug mode extends the existing Assembly view without replacing the monitor's other functionality.

• Memory, ASCII, Screen Code, Binary, and Assembly views remain available.
• Memory can be inspected or edited without ending the debug session.
• Edit mode and Debug mode can be active at the same time.
• C=+X resets the C64 and returns the monitor to a clean state.
• Debug mode can be left with C=+D or RUN/STOP.

---

Screenshots

Debugger

The debugger is paused in the KERNAL SCNKEY routine:

• Dbg indicates that Debug mode is active.
• [KEY] marks a labelled breakpoint at $EA87.
• The CPU is stopped at $EA98.
• The highlighted target $EAFB shows that the current branch will be taken.
• The footer shows the current CPU state.

Breakpoint list

The breakpoint popup follows the existing bookmark-list controls.

Debug help

Shortcuts whose meaning changes while Debug mode is active are shown at the top of the help screen.

---

Design

BRK-based debugging

The FPGA core does not provide hardware breakpoints or direct access to the 6510 registers for the application-hosted monitor. The debugger therefore stops execution by temporarily replacing instructions with BRK.

For each temporary breakpoint, it:

1. Saves the original byte.
2. Writes $00, the 6510 BRK opcode.
3. Resumes the CPU.
4. Captures the register state when the BRK is reached.
5. Restores the original byte.

Each modification records the address, original byte, and CPU-port state needed to restore it correctly. Debugger working memory and interrupt-vector locations cannot be used as breakpoint addresses.

The debugger temporarily uses the cassette buffer for its handler, resume code, NMI code, and working state. It also temporarily changes the RAM BRK vector at $0316/$0317. All changes are restored when Debug mode ends.

Platform interface

MemoryBackend::create_debug_session() separates the monitor UI from the U64- and U2-specific implementations.

• Host tests use test implementations.
• Firmware builds use the U64 or U2 implementation.
• The monitor UI interacts only with the shared DebugSession interface.

Stepping

The debugger has no hardware single-step support. Instead, it decodes the current instruction and calculates the addresses that may execute next.

Temporary BRK instructions are then placed at those addresses, the CPU resumes, and the debugger captures whichever breakpoint is reached.

Step Out uses return addresses recorded when Step Into enters a JSR, rather than relying solely on the current stack contents.

When Continue starts on an existing breakpoint, the debugger first executes past it to avoid immediately stopping at the same address again.

ROM support

On the U64, BASIC and KERNAL breakpoints temporarily modify writable copies of the ROM images held by the FPGA.

The U2 reads the C64 ROM directly and has no equivalent writable copy, so visible-ROM breakpoints are unavailable on the U2. RAM breakpoints and register capture use the same shared debugger implementation on both devices.

Cleanup and mode handling

Temporary instructions, vectors, and working memory are restored on every exit path, including:

• Normal Debug-mode exit.
• Timeout or cancellation.
• Reset.
• Monitor close.
• RUN/STOP.
• C=+O.
• C=+X.

In Overlay mode, the debugger prepares the resume code before restoring modified program bytes. This prevents the running CPU from encountering partially restored code.

Freeze mode temporarily resumes the C64 while an instruction is executed, then freezes it again. Telnet and Overlay modes do not require this cycle.

---

Implementation

The main files are:

File	Responsibility
machine_monitor.cc	UI integration, keyboard handling, and Debug/Edit interaction.
monitor_debug.{h,cc}	Debug state, footer formatting, and help text.
monitor_breakpoints.{h,cc}	Ten-slot non-persistent breakpoint table.
monitor_debug_session.h	Shared interface for U64, U2, and host tests.
monitor_debug_brk_session.cc	BRK handling, stepping, byte restoration, return addresses, and cleanup.
monitor_debug_u64.cc	U64 hardware access and visible-ROM support.
monitor_debug_u2.cc	U2 hardware access.

---

Testing

The monitor E2E tests live under tests/e2e/monitor/ and are registered with the repository-root run-tests runner.

Selector	Script	Mode
monitor-harness	monitor_harness_test.py	Automatic, host only
machine-code-monitor	monitor_test.py	Manual
machine-code-monitor-debug	monitor_debug_test.py	Manual
machine-code-monitor-matrix	monitor_debug_matrix_test.py	Manual

Unit and host tests

All three host-side monitor suites under target/pc/linux/machinemonitortest pass, covering the core monitor, bookmarks, and debugger.

software/test/monitor/machine_monitor_debug_test.cc contains 174 cases covering instruction prediction, breakpoint handling, execution controls, Debug/Edit interaction, cleanup, timeout recovery, Freeze/Overlay behaviour, Step Out tracking, and U64 BASIC/KERNAL stepping.

Debugger matrix

monitor_debug_matrix_test.py is the main debugger release test. It exercises:

{Telnet, UI Overlay, UI Freeze}
x
{RAM, RAM under ROM, visible ROM,
 RAM->ROM->RAM, RAM->RAM-under-ROM->ROM->RAM-under-ROM->RAM}

The two traversal modes are important because normal debugging crosses memory-region boundaries rather than entering each region from a fresh bootstrap.

• ram-rom-ram starts in RAM, enters BASIC ROM at $BC0F, then returns to RAM.
• ram-rur-rom-ram traverses RAM, RAM under ROM, visible ROM, and back again while switching $01 between legs.

A direct RAM-under-ROM to visible-ROM traversal is not practical because changing $01 while executing from the banked region immediately replaces the instruction stream being executed. The fixture therefore returns to ordinary RAM, changes the memory mapping there, and then enters ROM, matching how real 6510 code normally performs such transitions.

The traversal fixtures are also executed on the host through mcm6502.py, which verifies the expected region sequence and catches broken fixtures before a hardware run begins.

These traversal modes increase the matrix from 9 to 15 combinations per repetition. They are host-verified and now also pass on hardware.

Each matrix combination covers Step Over, Step Into, Step Out, Run to Cursor, breakpoint Continue, normal Continue, and Reset. Validation includes CPU state, memory effects, a 100-instruction comparison against both an independent 6510 interpreter and VICE, and a separate 1000-instruction live run.

Matrix results

Latest full run on an Ultimate 64 Elite I on 10 Aug 2026, firmware built from this branch after the test-merge merge and deployed over JTAG:

Metric	Result
Cells	45 of 45 passed, 15 combinations x 3 repetitions
Execution controls	All seven pass in every cell
Step Into nesting	32 levels
Straight-line Step Over run	32 consecutive calls at one stack level
Instructions stepped	4311, each compared against both the 6510 interpreter and VICE
1000-instruction live run	Passed, 1440 steps plus 1152 in nested calls, no errors
Recovery resets, command retries, breakpoint replants	Zero

The straight-line run complements the nesting chain: it repeats the same call from the same stack state, so a leaked breakpoint slot or a park and resume that drifts the stack shows up there and nowhere else.

Each run appends to a local ledger, by default under doc/research/machine-code-monitor/matrix-runs/, which is not tracked by git. It records the commit, start and end times, per-cell status, and failure details in both Markdown and JSONL, so runs can be compared over time.

Additional E2E coverage

The existing monitor regression suite remains available:

./run-tests -H <u64-ip> -s machine-code-monitor

The broader debugger suite is run with:

./run-tests -H <u64-ip> -s machine-code-monitor-debug

It contains 89 checks covering stepping, breakpoints, memory-region entry, Continue behaviour, cleanup, and leaving Debug mode without affecting the monitor or C64.

The complete automatic E2E run is:

./run-tests -H u64

---

Known limitations

• Conditional breakpoints, watchpoints, and CPU execution history are not supported.
• Breakpoints stop only between instructions, and each debugging operation has a fixed maximum wait time.
• Visible-ROM breakpoints are supported only on the U64 because the U2 has no writable copy of the C64 ROM that the debugger can temporarily modify.
• On the U2+L the CPU port is read by running a short stub on the 6510 through the NMI vector. While the machine is frozen that reading cannot go stale, because the CPU is halted. On a machine left running it is the port as sampled when the monitor opened.

---

Outstanding work

Three items are open. None of them affects the U64 results above, and each is written up with its evidence in doc/research/machine-monitor/debug/u2/handover-prompt-2026-08-10.md.

1. monitor_debug_test.py against the U2+L over Telnet. First full measurement is 10 passed, 21 failed, 6 skipped. This lane has not been green before and is not part of a registered gate: run-tests has no --c64-host, so the split-host U2 setup is driven by scripts rather than the runner, and the matrix runs only a two-check subset of this suite as a preflight. That subset failing is what currently stops --focus matrix on the U2.

The 21 failures are not one defect. They fall into three groups, and each needs to be resolved as a firmware defect, a stale test expectation, or a genuinely unsupported operation, rather than converted into a skip:

• Checks 19 and 21 require [BRKx][CPU] and [READ][CPU] on breakpoint rows. The CPU tag beside each assembly row was deliberately removed, and with the banking now resolved on the U2 that tag reads RAM, BAS or KRN. Checks 13 and 14 assert the earlier U64-shaped footer.
• Six checks covering Step Out fail with CPU label row not found. The message is misleading: the code scans for the row holding PC, SP and NV-BDIZC, so the actual symptom is that the debug register row never appeared.
• The remainder are interaction failures over Telnet: ASM Edit not entering Edit mode, the C=+R breakpoint popup, leaving Debug mode, and the combined Debug and Edit header.

2. A stepping disagreement found by the stress oracle on the U2+L. monitor_debug_stress.py --banking ram --focus steps reports scratch mismatch at $C8C7: oracle FE dev FD, on the same iteration at the same address on every run. It is reproducible rather than intermittent. Whether the fault is in the stepping engine or in mcm6502.py is not yet established; running the same seed against the U64 distinguishes the two.

3. The U64 volatile KERNAL image. The banked-breakpoints group in the debugger suite leaves $E000 reading $EE instead of $85 while reporting its own checks as passing, so the matrix has to start from a fresh firmware deployment. This is pre-existing rather than introduced here, proven by bisecting this branch's changes back out and observing it unchanged. All four firmware writers into the ROM window were instrumented and traced zero writes into $A000+ during a reproducing run, which leaves the FPGA-level question of whether U64_KERNAL_BASE and the address the 6510 fetches at $E000 are the same copy.

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR introduces a debugger-capable Machine Code Monitor with breakpoint support across U64 and U2 targets, plus new automation tooling (repro scripts + soak test) and documentation updates to validate and explain the new debug behaviors.

Changes:

• Adds a Debug mode execution backend (BRK-based stepping, breakpoints, reset/re-entry orchestration) with target-specific implementations (U64/U2).
• Extends monitor UI/input handling for debug actions, global reset behavior, and updated status/banking display.
• Adds new deterministic repro scripts, soak testing, and updates docs/snapshots/build files to cover the new functionality.

Reviewed changes

Copilot reviewed 57 out of 61 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tools/developer/machine-code-monitor/snapshots/expected_snapshots.json	Updates expected CPU/view status line fragments to the new CxOy format.
tools/developer/machine-code-monitor/regression_repro.py	Adds deterministic REST-driven repro cases for monitor regressions.
tools/developer/machine-code-monitor/monitor_debug_soak.py	Adds a telnet-based debug soak test with a lightweight 6510 model comparison.
tools/developer/machine-code-monitor/issue_repro.py	Adds autonomous REST repro cases for current monitor blockers.
tools/developer/machine-code-monitor/README.md	Documents debug tests/soak usage and new environment variables.
target/u64ii/riscv/ultimate/Makefile	Builds new monitor debug/breakpoint sources for U64II RISC-V.
target/u64/riscv/ultimate/Makefile	Builds new monitor debug/breakpoint sources for U64 RISC-V.
target/u64/nios2/ultimate/Makefile	Builds new monitor debug/breakpoint sources for U64 Nios2.
target/u2plus_L/riscv/ultimate/Makefile	Builds new monitor debug/breakpoint sources for U2+L RISC-V.
target/u2plus/nios/ultimate/Makefile	Builds new monitor debug/breakpoint sources for U2+ Nios.
target/u2/riscv/ultimate/Makefile	Builds new monitor debug/breakpoint sources for U2 RISC-V.
target/pc/linux/machinemonitortest/Makefile	Adds PC-side machinemonitordebugtest suite and required sources.
software/userinterface/userinterface.h	Adds active monitor tracking and reset re-entry hook into HostClient.
software/userinterface/userinterface.cc	Implements global reset shortcut handling and wires it into keymapper.
software/userinterface/ui_elements.cc	Treats keymapper -2 as “global accelerator consumed” to exit popups.
software/u64/u64_machine.h	Adds raw/visible poke/peek variants and “preserving freeze restore” write.
software/u64/u64_machine.cc	Implements raw/visible memory access helpers and improves serve-control handling.
software/test/monitor/machine_monitor_test_support.h	Extends FakeKeyboard to allow pushing a key ahead of scripted input.
software/test/monitor/machine_monitor_test_support.cc	Implements FakeKeyboard push-head and updates UI string_edit stub signature.
software/test/monitor/machine_monitor_bookmarks_test.cc	Updates expected bookmark popup strings and key sequences for new flows.
software/monitor/u64_memory_backend.h	Adds reset/debug-session support and observed live CPU port tracking.
software/monitor/u64_memory_backend.cc	Updates U64 backend mapping semantics and creates U64 debug sessions.
software/monitor/u2_memory_backend.h	Adds reset/debug-session support for U2 backend.
software/monitor/u2_memory_backend.cc	Implements U2 reset and debug-session creation.
software/monitor/run_machine_monitor.cc	Reworks monitor lifecycle for reset re-entry and interface swap teardown.
software/monitor/monitor_init.h	Adds weak global-reset-cancel hook for monitor/debug cancellation.
software/monitor/monitor_file_io.h	Adds debug-context resume/staging APIs to safely hand off to execution.
software/monitor/monitor_file_io.cc	Implements U64 NMI trampoline helpers and staged NMI handoff paths.
software/monitor/monitor_debug_u64.h	Declares U64 debug session factory and helper for step CPU port.
software/monitor/monitor_debug_u64.cc	Implements U64-specific BRK debug session with volatile ROM patching support.
software/monitor/monitor_debug_u2.h	Declares U2 debug session factory.
software/monitor/monitor_debug_u2.cc	Implements U2-specific BRK debug session (no visible ROM patching).
software/monitor/monitor_debug_session.h	Introduces the DebugSession interface and result codes for debugger ops.
software/monitor/monitor_debug_predictor.h	Adds instruction classification for stepping prediction.
software/monitor/monitor_debug_predictor.cc	Implements predictor using fast opcode cases + disassembler length fallback.
software/monitor/monitor_debug_brk_session.h	Declares shared BRK-based debug session implementation and patch tracking.
software/monitor/monitor_debug.h	Defines DebugContext and MonitorDebug footer/help formatting API.
software/monitor/monitor_debug.cc	Implements debug footer layout + help text formatting.
software/monitor/monitor_breakpoints.h	Adds in-memory breakpoint table, labels, and popup formatting.
software/monitor/monitor_breakpoints.cc	Implements slot allocation, normalization, and popup row formatting.
software/monitor/memory_backend.h	Adds backing-store classification helpers and debug-session/reset hooks.
software/monitor/machine_monitor.h	Extends monitor state, disasm lane, debug/breakpoint UI plumbing and APIs.
software/monitor/disassembler_6502.h	Exposes operand_spec() for shared operand classification.
software/monitor/disassembler_6502.cc	Renames illegal mnemonics and refactors operand parsing to use operand_spec().
software/monitor/assembler_6502.cc	Canonicalizes additional illegal mnemonic aliases during assembly lookup.
software/io/usb/tests/usb_keyboard_queue_test.cpp	Adds regression for Ctrl+R mapping distinct from cursor-down behavior.
software/io/usb/keyboard_usb.cc	Maps Ctrl+R to KEY_CTRL_R in control keymap.
software/io/stream/keyboard_vt100.cc	Adds Ctrl+R decoding from stream input (0x12 / ESC+r).
software/io/c64/keyboard_c64.cc	Maps matrix Ctrl+R to KEY_CTRL_R instead of PETSCII 0x12 collision.
software/io/c64/keyboard.h	Introduces KEY_CTRL_R and documents why 0x12 cannot be used.
software/io/c64/c64_subsys.cc	Cancels debug waits on reset and normalizes formatting/whitespace.
software/io/c64/c64.h	Adds begin/end stopped-session helpers and a refreeze() convenience.
software/io/c64/c64.cc	Adds pristine ROM snapshot/restore on reset + stopped-session helpers + refreeze().
software/infra/host.h	Adds host callback to request reset re-entry after C64 reset.
doc/machine_code_monitor.md	Updates public documentation for modes, status line, edit/debug/breakpoints.

Comments suppressed due to low confidence (4)

software/monitor/disassembler_6502.cc:1

• Branch opcode templates were changed to use an operand spec of rel (e.g. \"BCC rel\", \"BNE rel\"), but operand_length()/format_operand() no longer have the branch-special-case and also don’t recognize rel. This will cause branch instructions to disassemble with the wrong operand length and likely render an empty/incorrect operand/target, breaking both UI and any predictor logic that relies on disassembly output. Fix by handling rel explicitly (length=1 and formatting $%04X target), or by reinstating a branch-specific path keyed off spec == \"rel\".

#include "disassembler_6502.h"

software/monitor/disassembler_6502.cc:147

• Branch opcode templates were changed to use an operand spec of rel (e.g. \"BCC rel\", \"BNE rel\"), but operand_length()/format_operand() no longer have the branch-special-case and also don’t recognize rel. This will cause branch instructions to disassemble with the wrong operand length and likely render an empty/incorrect operand/target, breaking both UI and any predictor logic that relies on disassembly output. Fix by handling rel explicitly (length=1 and formatting $%04X target), or by reinstating a branch-specific path keyed off spec == \"rel\".

        !strncmp(spec, "$nn", 3) || !strncmp(spec, "#", 1)) {
        return 1;
    }
    return 0;
}

tools/developer/machine-code-monitor/issue_repro.py:1

• This line assigns session.dump_ui_screen(...) into mdt.wait_stable_dump, overwriting the imported function/attribute on the monitor_direct_test module. That is almost certainly unintended and can break subsequent calls that rely on mdt.wait_stable_dump. Change this to only assign the frame (e.g., frame = session.dump_ui_screen(...)) or call the real wait helper if you intended to use it.
  tools/developer/machine-code-monitor/README.md:1
• monitor_debug_soak.py (as added in this PR) does not define --copy-roms-to-ram or --yes-copy-roms arguments, so this example command is not runnable as documented. Either update the README to match the actual CLI flags, or add the missing argparse options and implement the described behavior.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Kugelblitz360]

Just: WOW! Thank you!