fix(tokenizer): keep lone markup-opener chars as plain text

[SayreBlades]

Summary

Fixes a tokenizer bug where a lone markup-opener character (* ` ~ ! [) that does not begin a recognized markup span was emitted as its own 1-character token. markdown-table-wrap-cell then rejoined tokens with single spaces and re-wrapped, corrupting cells that happened to contain one of these characters.

The classic victims: bare bracketed labels like [Zig] / [raylib], approximations like ~68 / ~36 hot / ~29–32, and any cell with a stray * or !.

The bug, concretely

With a column width equal to the cell's natural width, markdown-table-wrap-cell produced spurious spaces and/or unwanted line breaks:

cell	before (buggy)	after (fixed)
~68	("~ 68") — spurious space	("~68")
~36 hot	("~ 36" "hot") — split across 2 lines	("~36 hot")
[raylib]	("[" "raylib]") — torn across 2 lines	("[raylib]")
[Zig]	("[ Zig]") — spurious space	("[Zig]")

A bracketed label like [raylib] (8 cols wide) assigned to an 8-col column would render as:

│ [
│ raylib]

instead of the obvious:

│ [raylib]

Root cause

markdown-table-wrap--tokenize-cell-text excluded the opener chars (* ` ~ ! [) from plain-text runs and fell through to a "stray markup char" branch that consumed exactly one character:

;; Plain text — up to whitespace or markup delimiter.
((and (string-match "[^ \t*`~![]+" text pos) ...)
 ...)
;; Stray markup char — consume one to make progress.
(t
 (push (substring text pos (1+ pos)) tokens)   ; <- 1-char token!
 (setq pos (1+ pos)))

So [raylib] → tokens ("[" "raylib]"), and ~68 → ("~" "68"). The opener char became a 1-char token even though it began no real span ([label] has no (url), ~ is not ~~).

The fix

In the plain-text branch, consume until whitespace or a real span start. Since every recognized span begins with one of those opener chars, we only re-check span-re when we encounter one — so a real span embedded later in the run (e.g. foo[bar](url)) still tokenizes as a span. A lone opener that begins no span is absorbed into the surrounding plain-text token.

(t
 (let ((start pos) (ch nil))
   (while (and (< pos len)
               (progn (setq ch (aref text pos))
                      (and (not (memq ch '(?\s ?\t)))
                           (or (not (memq ch '(?* ?\` ?~ ?! ?\[)))
                               (not (and (string-match span-re text pos)
                                         (= (match-beginning 0) pos))))))))
     (setq pos (1+ pos)))
   ...)))

No regression for real markup spans

Recognized markup spans are unaffected — they still tokenize as single span tokens (verified by markdown-table-wrap-test-tokenize-real-spans-unaffected):

cell	tokens (unchanged)
**bold**	("**bold**")
*it*	("*it*")
~~strike~~	("~~strike~~")
`code`	("code")
[t](u)	("[t](u)")
	("")
***bi***	("***bi***")
*a **b** c*	("*a **b** c*")
foo[bar](url)	("foo" "[bar](url)")

Tests

Adds 12 tests under two new sections:

• Tokenizer: Lone markup-opener characters (8 tests) — verify ~, [, *, ! that begin no span stay in the plain run; real spans after plain runs still split correctly; real spans are not collapsed.
• End-to-end: lone-opener cells (4 tests) — verify markdown-table-wrap-cell renders ~68, [raylib], ~36 hot, [Zig] without spurious spaces or splits.

Each lone-opener test documents the exact before/after behavior in its docstring.

$ make check
=== Byte-compile ===
=== Checkdoc ===
OK
=== Package-lint ===
=== Tests ===
Ran 165 tests, 165 results as expected, 0 unexpected
1 expected failures

The single expected failure (markdown-table-wrap-test-e2e-no-orphan-markup) is pre-existing and documents known CJK/italic limitations unrelated to this change.