Skip to content

docs: Document async-profiler usage in the runner image - #379

Open
velo wants to merge 1 commit into
mainfrom
docs/async-profiler-usage
Open

docs: Document async-profiler usage in the runner image#379
velo wants to merge 1 commit into
mainfrom
docs/async-profiler-usage

Conversation

@velo

@velo velo commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

async-profiler is already installed in the image (/opt/async-profiler, asprof on PATH), but nothing documents it. This adds a README section covering how to actually use it.

Verified, not assumed

Built the image locally, ran a CPU-heavy job on a real cluster container (datagen -> MD5(SHA256(UPPER(txt))) -> blackhole) and attached to the TaskManager. The resulting flame graph contains genuine stacks all the way down to codegen and JIT intrinsics:

SourceStreamTask$LegacySourceFunctionThread.run
  StreamSource.run
    DataGeneratorSource.run
      ...CopyingChainingOutput.pushToOperator
        StreamExecCalc$12.processElement
          BinaryStringDataUtil.hash
            MessageDigest.digest -> SHA2.implDigest -> md5_implCompress

wall, alloc, lock and collapsed output were exercised too.

Findings worth documenting

  • ctimer fallback works, no extra capabilities needed. Hardware perf events are blocked in an unprivileged container (asprof -e cycles -> perf_event_open ... Operation not permitted), but -e cpu silently falls back to ctimer and still produces complete stacks. No SYS_ADMIN, no perf_event_paranoid tuning.
  • Write output inside the container. The profiled JVM writes the file itself as the flink user, so a mounted host volume fails with Could not open output file.
  • One session per JVM. [ERROR] Profiler already started needs asprof stop -f <file> <pid> to collect.
  • DebugNonSafepoints is optional. Documented rather than enabled by default — stacks are already usable without it, and hardcoding JVM opts into the image would be silently overridden by any user-supplied env.java.opts.all.

Docs only, no image or code changes.

🤖 Generated with Claude Code

Signed-off-by: Marvin Froeder <marvin@datasqrl.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant