Add two optional profiling add-ons: sampling profiler and heap snapshot diff#100
Open
TetzkatLipHoka wants to merge 2 commits into
Open
Add two optional profiling add-ons: sampling profiler and heap snapshot diff#100TetzkatLipHoka wants to merge 2 commits into
TetzkatLipHoka wants to merge 2 commits into
Conversation
…ot diff Both units build purely on the public FastMM5 API - they do not touch the memory manager itself and have no effect unless the unit is added to a project. FastMM_SamplingProfiler samples the memory manager state from a low priority background thread and appends each sample to a CSV file, giving time series for the process footprint, allocated/reserved/overhead bytes, efficiency and the small/medium/large breakdown, plus an optional detail CSV with one row per small block size class per sample. A callback may be registered to receive samples live. This surfaces growth and fragmentation trends that a single snapshot cannot show. FastMM_SnapshotDiff captures snapshots of all live allocations aggregated by block content (class instances by class name, probable string data, unclassified blocks) and diffs two snapshots, answering "which classes grew between point A and point B?" without debug mode, allocation groups or a recompile. Each unit comes with a demo under Demos\Profiling that doubles as a self-test (exit code 0 = all checks passed). Both demos were built and run with Delphi 13.1, Win32 and Win64.
Free Pascal does not predefine Delphi's CompilerVersion constant, so the unit scope name and inline switches in these files failed to evaluate. Supply it as a macro under FPC, the same way FastMM5.pas itself does. This is inert for Delphi. Built and run with FPC 3.2.2 on Win32 and Win64, and re-checked with Delphi 7 and 13.1 (Win32 and Win64).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two optional add-on units that build purely on the public FastMM5 API, each with a demo that doubles as a self-test. Neither unit touches the memory manager itself and neither has any effect unless the unit is explicitly added to a project, so the risk to existing users is nil.
Profiling/FastMM_SamplingProfiler.pas
Runs a low priority background thread that periodically samples the memory manager state and appends the result as a row to a CSV file. Over the run of an application this gives time series for the process footprint, the allocated / reserved / overhead byte counts, the memory manager efficiency and the small / medium / large block breakdown, plus the arena lock contention counters. Optionally a second CSV is written with one row per small block size class per sample, so per-bin fragmentation (reserved address space versus bytes actually in use) can be tracked over time. A callback may be registered to receive every sample live, e.g. to feed a dashboard.
The point is to surface growth and fragmentation trends that a single point-in-time snapshot cannot show. Numbers are written with a '.' decimal separator regardless of locale, so the CSV is portable, and the allocated / reserved / overhead / efficiency values are derived from a single state walk using the same UsableSize based accounting as FastMM_GetUsageSummary, so they match that call exactly.
Profiling/FastMM_SnapshotDiff.pas
Captures point-in-time snapshots of all live allocations, aggregated by block content (class instances by class name, probable string data, unclassified blocks), and compares two snapshots. This answers the most common profiling question - "which classes grew between point A and point B?" - without requiring debug mode, allocation groups or a recompile. FastMM_LogStateToFile can only diff allocation group ranges (which requires debug mode), whereas this diffs two arbitrary points in time in any mode.
The capture walks the pool via FastMM_WalkBlocks and allocates nothing during the walk. It keys its aggregation hash table on the first native word of the block content, so the expensive content detection (which involves VirtualQuery) runs only once per distinct class pointer - the same optimization FastMM_LogStateToFile uses.
Demos
Demos\Profiling\Sampling Profiler\SamplingProfilerDemo.dpr- runs a grow-then-shrink workload while the sampler collects a series, then verifies that the collected samples show the expected rise and fall and that the CSV files have the expected shape.Demos\Profiling\Snapshot Diff\SnapshotDiffDemo.dpr- allocates known objects, strings and raw blocks between snapshots and verifies the reported deltas, including a run inside debug mode.Both are console programs that exit with code 0 when all checks pass, so they can be used as regression tests.
Testing
Built and run against this branch (i.e. against unmodified FastMM5.pas) with Delphi 13.1, Win32 and Win64: 13/13 and 22/22 checks pass on each target. Also verified with Delphi 7 and FPC 3.2.2 (Win32 and Win64) in my fork, which is where these units are used.
Note on the FPC blocks
Each file carries a small
{$ifdef FPC}block that supplies CompilerVersion as a macro, mirroring what FastMM5.pas itself does. It is inert for Delphi. Happy to strip those out if you would rather not carry them here.