Skip to content

Add a test suite under Tests#103

Open
TetzkatLipHoka wants to merge 1 commit into
pleriche:masterfrom
TetzkatLipHoka:test-suite
Open

Add a test suite under Tests#103
TetzkatLipHoka wants to merge 1 commit into
pleriche:masterfrom
TetzkatLipHoka:test-suite

Conversation

@TetzkatLipHoka

Copy link
Copy Markdown

Nine console test programs plus a script that builds and runs them with every Delphi installation it finds, for Win32 and Win64. Nothing in FastMM5.pas changes - this is purely additive, and if you never run it, it costs nothing.

The motivation is #102: the corruption scan silently stopped seeing small debug blocks, and it stayed unnoticed because there was nothing that would have run the same check across all three size classes after a change. FastMM5Test_ScanCoverage is exactly that check, and it fails on the affected commits and passes on either side of them.

How it works

No test framework. Each program is a plain console application that prints one line per check and exits with 0 when everything passed, otherwise with the number of failed checks. RunTests.ps1 builds and runs them all and exits with the number of failed runs, so it drops straight into a CI step:

pwsh -File Tests\RunTests.ps1                  # every configured compiler, Win32 and Win64
pwsh -File Tests\RunTests.ps1 -Only D13.1      # a single compiler
pwsh -File Tests\RunTests.ps1 -Quick           # shorter stress runs

The $Compilers table at the top of the script is the only thing that needs adjusting for a different machine.

What is covered

Program What it covers
FastMM5Test_DebugMode Entering and leaving debug mode, allocate/write/realloc/free in both modes, allocated bytes returning to the starting value.
FastMM5Test_SizeClasses Every size class from 1 byte to 2 MB plus reallocations across the boundaries, each verified with a fill pattern.
FastMM5Test_UsagePerSizeClass Churn per size class: nothing may stay allocated, and the committed address space must not keep growing from phase to phase.
FastMM5Test_ModeTransition The Begin/End contract from #85, including the case that a failed Begin still has to be balanced.
FastMM5Test_DoubleFree #73: the second free is rejected and leaves the pending free list intact (no self cycle). A walker thread sleeping on the block under test forces the pending free path deterministically.
FastMM5Test_ScanCoverage #102: corrupted header checksum, overrun into the footer, and write into a freed block - across small, medium and large.
FastMM5Test_ScanRace The other direction: threads churning small debug blocks while a scanner runs must not produce a false positive or a crash.
FastMM5Test_ScanHeaderBounds Corrupts UserSize / StackTraceEntryCount, i.e. the fields that decide where the scan reads; each case must give a clean report rather than an A/V.
FastMM5Test_MultiThreadStress Multithreaded stress with optional cross thread frees through a lock free mailbox; content integrity plus closing balance.

FastMM_TestUtils.pas holds the assertions and the exit code convention. It also clears FastMM_MessageBoxEvents and FastMM_LogToFileEvents for the duration of a run, since several tests corrupt blocks on purpose and a modal dialog would hang an unattended run.

Testing

36 of 36 runs pass: Delphi 10 Seattle and Delphi 13.1, each Win32 and Win64.

Two notes

The Tests/README.md records two things that cost me time and would otherwise be rediscovered by whoever writes the next test: corrupting a freed small block is not observable (raising the report allocates the exception object, which is handed exactly that block, so the process dies before any handler runs - use medium or large blocks there), and a corruption test that happens to use a large block proves nothing about the small block path, which is how #102 stayed hidden.

The sources carry {$if CompilerVersion >= ...} guards in a few places so the same files also build on older compilers in a fork. They are inert on XE3 and later; happy to strip them if you would rather not carry them.

Nine console test programs plus a script that builds and runs them with every
Delphi installation it finds, for Win32 and Win64.  Nothing in FastMM5.pas
changes;  this is purely additive.

Each test exits with 0 when all of its checks passed and with the number of
failures otherwise, so no test framework is needed and the runner (or a CI step)
only has to look at the exit code.  RunTests.ps1 exits with the number of failed
runs.

Covered:  the block size classes including reallocations across their
boundaries, debug mode basics, usage accounting per size class (both leaks and
unbounded address space growth), the mode transition contract from pleriche#85, double
free handling from pleriche#73, the corruption scan from pleriche#102 in both directions
(detection must work, false positives must not happen), corrupted size fields in
the debug header, and multithreaded stress with cross thread frees.

Verified with Delphi 10 Seattle and Delphi 13.1, Win32 and Win64:  36 of 36 runs
pass.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant