Our continuous benchmarking setup (https://hermit-os.github.io/hermit-bench/) is lacking. It is run on our local VM cluster, which leads to a high variance in measured run times. While reworking our stack with Bencher might be interesting, even running bare-metal is not perfectly consistent.
Instead of running with KVM, we could try emulation instead. QEMU's TCG supports instruction counting. There is an example TCG plugin that counts the number of instructions (docs, tests/tcg/plugins/insn.c). While this will run slower and will not provide wall-time measurements, these should be extremely consistent and show even small changes in instruction count.
A slower, but more accurate way to gauge performance would be to run Hermit on a CPU simulator such as gem5.
An alternative would be to explore Valgrind support to instructions, L1 hits, LL hits, RAM hits, total read+write, and estimated cycles.
Hardware performance counters would also be interesting, but they are often vendor- and CPU-specific.
Our continuous benchmarking setup (https://hermit-os.github.io/hermit-bench/) is lacking. It is run on our local VM cluster, which leads to a high variance in measured run times. While reworking our stack with Bencher might be interesting, even running bare-metal is not perfectly consistent.
Instead of running with KVM, we could try emulation instead. QEMU's TCG supports instruction counting. There is an example TCG plugin that counts the number of instructions (docs, tests/tcg/plugins/insn.c). While this will run slower and will not provide wall-time measurements, these should be extremely consistent and show even small changes in instruction count.
A slower, but more accurate way to gauge performance would be to run Hermit on a CPU simulator such as gem5.
An alternative would be to explore Valgrind support to instructions, L1 hits, LL hits, RAM hits, total read+write, and estimated cycles.
Hardware performance counters would also be interesting, but they are often vendor- and CPU-specific.