Commits
- Commit:
04d982393b89e9663c5254fb3c0b3bd33d26f8c4
- From:
- Sergey Bronnikov <sergeyb@tarantool.org>
- Date:
perf/lua: add context section to test output
Google Benchmark output format contains a section "context" that
describes useful information about test environment.
Google Benchmark output format has been supported in Lua
microbenchmarks in commit 3110ef9a9498
("perf: introduce benchmark.lua helper module"). However, produced
output contains test results only and section "context" is missed.
The patch add a section "context" with the following fields:
date, load average, hostname, tarantool's version, build flags
and a name of build target.
```
$ tarantool uri_escape_unescape.lua --output=res.json --output_format=json
$ jq ".context" res.json
{
"build_target": "Linux-x86_64-RelWithDebInfo",
"host_name": "pony",
"date": "2024-07-04 19:09:11",
"tarantool_version": "3.2.0-entrypoint-114-g9e5dca29ad",
"build_flags": " -fexceptions -funwind-tables -fasynchronous-unwind-tables -fno-common -msse2 -Wformat -Wformat-security -Werror=format-security -fstack-protector-strong -fPIC -fmacro-prefix-map=/home/sergeyb/sources/MRG/tarantool=. -std=c11 -Wall -Wextra -Wno-gnu-alignof-expression -Wno-cast-function-type -O2 -g -DNDEBUG -ggdb -O2 ",
"load_avg": [
"0.76",
"0.74",
"0.63"
]
}
```
NO_CHANGELOG=perf
NO_DOC=perf
NO_TEST=perf
- Commit:
25f860db29201fdf2da287eaf46bc19d7039330b
- From:
- Sergey Bronnikov <sergeyb@tarantool.org>
- Date:
perf/lua: protect require of column module
NO_CHANGELOG=perf
NO_DOC=perf
NO_TEST=perf
- Commit:
d5cc2cc2fafc2e7cd2d2d755f310ad59aae60bf8
- From:
- Sergey Bronnikov <sergeyb@tarantool.org>
- Date:
ci: enable BENCH_CMD
The patch enable environment variable `BENCH_CMD` introduced in
a previous commit. The `taskset` alone will pin all the process
threads into a single (random) isolated CPU, there's a ticket [1]
about this in the Linux kernel bugtracker. The workaround is using
realtime scheduler for the isolated task using `chrt` [2], e. g.:
`taskset 0xef chrt 50`.
1. https://bugzilla.kernel.org/show_bug.cgi?id=116701
2. https://www.man7.org/linux/man-pages/man1/chrt.1.html
NO_CHANGELOG=performance testing
NO_DOC=performance testing
NO_TEST=performance testing
- Commit:
861046e20e8499b58e3e7a6e4ba21921226827da
- From:
- Sergey Bronnikov <sergeyb@tarantool.org>
- Date:
perf: introduce BENCH_CMD environment variable
The patch introduces a BENCH_CMD, environment variable that
could be set to a string with command and its arguments on CMake
configuration stage and this string will be used as a pre-command
for executing performance tests. Examples of these commands are
`taskset` [1] and `numactl` [2], or any other utilities, see [3].
1. https://man7.org/linux/man-pages/man1/taskset.1.html
2. https://man7.org/linux/man-pages/man8/numactl.8.html
3. https://github.com/tarantool/tarantool/wiki/Benchmarking#run-benchmarks
NO_CHANGELOG=performance infra
NO_DOC=performance infra
NO_TEST=performance infra
- Commit:
f81c6f64b248684864424c81972511cef98cf695
- From:
- Sergey Bronnikov <sergeyb@tarantool.org>
- Date:
perf: add a script for setting environment
"Benchmarking" article [0] in Tarantool's wiki contains a lot of
recommendations that help to setup the Linux operating system and
avoid potential reproducibility pitfalls when executing
performance tests in a Linux-based environment. These
recommendations written in plain text with examples of commands
that could be executed manually. We desire to execute benchmarks
automatically and in continuous mode, therefore we need a way to
setup the test environment automatically before running
benchmarks.
There are many guides with benchmarking tips, but unfortunately
there is no script that will do these steps automatically.
I found only temci [1] and pyperf (`pyperf system` [2]) projects.
The patch adds a script for setting the environment before running
performance tests. All settings used in the proposed script are
described in the article [3]. Note, that uncertain settings were
not implemented.
0. https://github.com/tarantool/tarantool/wiki/Benchmarking
1. https://github.com/parttimenerd/temci
2. https://github.com/travisdowns/uarch-bench/blob/master/uarch-bench.sh
3. https://pyperf.readthedocs.io/en/latest/cli.html#system-cmd
NO_CHANGELOG=performance
NO_DOC=performance
NO_TEST=performance