fix: harden lexer and parser error handling

[shayyz-code]

Summary

• make legacy tokenization UTF-8-safe for strings, characters, comments, and whitespace
• propagate malformed literals, comments, unsupported identifiers, and unexpected EOF as typed parse errors
• add ten focused regression specifications and complete the roadmap checkpoint

Issue

Closes #17

Verification

• cargo fmt --all -- --check
• cargo check --all-targets
• cargo clippy --all-targets --all-features -- -D warnings
• cargo test (58 integration specifications and 1 doctest)
• cargo run --quiet -- examples/app.poo
• manual valid UTF-8 and unterminated-comment CLI checks
• mdbook build docs
• mdbook test docs
• oranda build
• go run github.com/rhysd/actionlint/cmd/actionlint@v1.7.7
• git diff --check

Documentation and Visual Impact

Roadmap status only; no visual impact.

[cursor]

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.