Commit Briefs

04d982393b Sergey Bronnikov

perf/lua: add context section to test output (ligurio/setup-perf-env, origin/ligurio/setup-perf-env)

Google Benchmark output format contains a section "context" that describes useful information about test environment. Google Benchmark output format has been supported in Lua microbenchmarks in commit 3110ef9a9498 ("perf: introduce benchmark.lua helper module"). However, produced output contains test results only and section "context" is missed. The patch add a section "context" with the following fields: date, load average, hostname, tarantool's version, build flags and a name of build target. ``` $ tarantool uri_escape_unescape.lua --output=res.json --output_format=json $ jq ".context" res.json { "build_target": "Linux-x86_64-RelWithDebInfo", "host_name": "pony", "date": "2024-07-04 19:09:11", "tarantool_version": "3.2.0-entrypoint-114-g9e5dca29ad", "build_flags": " -fexceptions -funwind-tables -fasynchronous-unwind-tables -fno-common -msse2 -Wformat -Wformat-security -Werror=format-security -fstack-protector-strong -fPIC -fmacro-prefix-map=/home/sergeyb/sources/MRG/tarantool=. -std=c11 -Wall -Wextra -Wno-gnu-alignof-expression -Wno-cast-function-type -O2 -g -DNDEBUG -ggdb -O2 ", "load_avg": [ "0.76", "0.74", "0.63" ] } ``` NO_CHANGELOG=perf NO_DOC=perf NO_TEST=perf


25f860db29 Sergey Bronnikov

perf/lua: protect require of column module

NO_CHANGELOG=perf NO_DOC=perf NO_TEST=perf


d5cc2cc2fa Sergey Bronnikov

ci: enable BENCH_CMD

The patch enable environment variable `BENCH_CMD` introduced in a previous commit. The `taskset` alone will pin all the process threads into a single (random) isolated CPU, there's a ticket [1] about this in the Linux kernel bugtracker. The workaround is using realtime scheduler for the isolated task using `chrt` [2], e. g.: `taskset 0xef chrt 50`. 1. https://bugzilla.kernel.org/show_bug.cgi?id=116701 2. https://www.man7.org/linux/man-pages/man1/chrt.1.html NO_CHANGELOG=performance testing NO_DOC=performance testing NO_TEST=performance testing


861046e20e Sergey Bronnikov

perf: introduce BENCH_CMD environment variable

The patch introduces a BENCH_CMD, environment variable that could be set to a string with command and its arguments on CMake configuration stage and this string will be used as a pre-command for executing performance tests. Examples of these commands are `taskset` [1] and `numactl` [2], or any other utilities, see [3]. 1. https://man7.org/linux/man-pages/man1/taskset.1.html 2. https://man7.org/linux/man-pages/man8/numactl.8.html 3. https://github.com/tarantool/tarantool/wiki/Benchmarking#run-benchmarks NO_CHANGELOG=performance infra NO_DOC=performance infra NO_TEST=performance infra


f81c6f64b2 Sergey Bronnikov

perf: add a script for setting environment

"Benchmarking" article [0] in Tarantool's wiki contains a lot of recommendations that help to setup the Linux operating system and avoid potential reproducibility pitfalls when executing performance tests in a Linux-based environment. These recommendations written in plain text with examples of commands that could be executed manually. We desire to execute benchmarks automatically and in continuous mode, therefore we need a way to setup the test environment automatically before running benchmarks. There are many guides with benchmarking tips, but unfortunately there is no script that will do these steps automatically. I found only temci [1] and pyperf (`pyperf system` [2]) projects. The patch adds a script for setting the environment before running performance tests. All settings used in the proposed script are described in the article [3]. Note, that uncertain settings were not implemented. 0. https://github.com/tarantool/tarantool/wiki/Benchmarking 1. https://github.com/parttimenerd/temci 2. https://github.com/travisdowns/uarch-bench/blob/master/uarch-bench.sh 3. https://pyperf.readthedocs.io/en/latest/cli.html#system-cmd NO_CHANGELOG=performance NO_DOC=performance NO_TEST=performance


b0b32bff02 Sergey Kaplun

uuid: relax UUID value validation

This patch completely relaxes UUID checks and accepts an arbitrary 128-bit sequence as an UUID for binary data. String representations still should match the grammars in RFC 4122, Section 3 [1] and RFC 9562, Section 4 [2]. [1]: https://datatracker.ietf.org/doc/html/rfc4122#section-3 [2]: https://datatracker.ietf.org/doc/html/rfc9562#name-uuid-format Closes #5444 @TarantoolBot document Title: uuid: relaxed UUID validation [The UUID module documentation][1] mentions that Tarantool generates UUIDs following the rules for RFC 4122,[version 4, variant 1][2]. It is worth mentioning that the user can store an arbitrary 128-bit sequence as an UUID for binary data. String representations still should match the grammars in RFC 4122, [Section 3][3], and RFC 9562, [Section 4][4]. [1]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/uuid/ [2]: https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_(random) [3]: https://datatracker.ietf.org/doc/html/rfc4122#section-3 [4]: https://datatracker.ietf.org/doc/html/rfc9562#name-uuid-format


462b20dc10 Sergey Kaplun

test: rewrite engine/uuid.test.lua with luatest

NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring


5e83fb1ba5 Sergey Kaplun

test: rewrite app/uuid.test.lua with luatest

NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring


bf01fb2007 Sergey Kaplun

luajit: bump new version

* Correct fix for stack check when recording BC_VARG. * test: remove inline suppressions of _TARANTOOL * FFI: Fix ffi.alignof() for reference types. * FFI: Fix sizeof expression in C parser for reference types. * FFI: Allow ffi.metatype() for typedefs with attributes. * FFI: Fix ffi.metatype() for non-raw types. * Maintain chain invariant in DCE. * build: introduce option LUAJIT_ENABLE_TABLE_BUMP * ci: add tablebump flavor for exotic builds * test: allow `jit.parse` to return aborted traces * Handle all types of errors during trace stitching. * Use generic trace error for OOM during trace stitching. * Check for IR_HREF vs. IR_HREFK aliasing in non-nil store check. * cmake: set cmake_minimum_required only once * cmake: fix warning about minimum required version * ci: add a workflow for testing with AVX512 enabled * test: introduce a helper read_file * OSX/iOS/ARM64: Fix generation of Mach-O object files. * OSX/iOS/ARM64: Fix bytecode embedding in Mach-O object file. * build: introduce LUAJIT_USE_UBSAN option * ci: enable UBSan for sanitizers testing workflow * cmake: add the build directory to the .gitignore * Prevent sanitizer warning in snap_restoredata(). * Avoid negation of signed integers in C that may hold INT*_MIN. * Show name of NYI bytecode in -jv and -jdump. Closes #9924 Closes #8473 NO_DOC=LuaJIT submodule bump NO_TEST=LuaJIT submodule bump


1e1bf36dbc Vladimir Davydov

fiber: phohibit fiber self join

In this case join will just hang. Instead let's raise an error in case of Lua API and panic in case of C API. Closes #10196 NO_DOC=minor


Branches



























































































Tags

Tree

.editorconfigcommits | blame
.gdbinitcommits | blame
.gitattributescommits | blame
.github/
.gitignorecommits | blame
.gitmodulescommits | blame
.luacheckrccommits | blame
.pack.mkcommits | blame
.test.mkcommits | blame
AUTHORScommits | blame
CMakeLists.txtcommits | blame
CONTRIBUTING.mdcommits | blame
Doxyfilecommits | blame
Doxyfile.API.incommits | blame
FreeBSD/
LICENSEcommits | blame
README.FreeBSDcommits | blame
README.MacOSXcommits | blame
README.OpenBSDcommits | blame
README.mdcommits | blame
TODOcommits | blame
apk/
asan/
changelogs/
cmake/
debian/
doc/
docker/
extra/
patches/
perf/
rpm/
rump/
src/
static-build/
test/
test-run$commits | blame
third_party/
tools/

README.md

# Tarantool

[![Actions Status][actions-badge]][actions-url]
[![Code Coverage][coverage-badge]][coverage-url]
[![OSS Fuzz][oss-fuzz-badge]][oss-fuzz-url]
[![Telegram][telegram-badge]][telegram-url]
[![GitHub Discussions][discussions-badge]][discussions-url]
[![Stack Overflow][stackoverflow-badge]][stackoverflow-url]

[Tarantool][tarantool-url] is an in-memory computing platform consisting of a
database and an application server.

It is distributed under [BSD 2-Clause][license] terms.

Key features of the application server:

* Heavily optimized Lua interpreter with incredibly fast tracing JIT compiler,
  based on LuaJIT 2.1.
* Cooperative multitasking, non-blocking IO.
* [Persistent queues][queue].
* [Sharding][vshard].
* [Cluster and application management framework][cartridge].
* Access to external databases such as [MySQL][mysql] and [PostgreSQL][pg].
* A rich set of built-in and standalone [modules][modules].

Key features of the database:

* MessagePack data format and MessagePack based client-server protocol.
* Two data engines: 100% in-memory with complete WAL-based persistence and an
  own implementation of LSM-tree, to use with large data sets.
* Multiple index types: HASH, TREE, RTREE, BITSET.
* Document oriented JSON path indexes.
* Asynchronous master-master replication.
* Synchronous quorum-based replication.
* RAFT-based automatic leader election for the single-leader configuration.
* Authentication and access control.
* ANSI SQL, including views, joins, referential and check constraints.
* [Connectors][connectors] for many programming languages.
* The database is a C extension of the application server and can be turned
  off.

Supported platforms are Linux (x86_64, aarch64), Mac OS X (x86_64, M1), FreeBSD
(x86_64).

Tarantool is ideal for data-enriched components of scalable Web architecture:
queue servers, caches, stateful Web applications.

To download and install Tarantool as a binary package for your OS or using
Docker, please see the [download instructions][download].

To build Tarantool from source, see detailed [instructions][building] in the
Tarantool documentation.

To find modules, connectors and tools for Tarantool, check out our [Awesome
Tarantool][awesome-list] list.

Please report bugs to our [issue tracker][issue-tracker]. We also warmly
welcome your feedback on the [discussions][discussions-url] page and questions
on [Stack Overflow][stackoverflow-url].

We accept contributions via pull requests. Check out our [contributing
guide][contributing].

Thank you for your interest in Tarantool!

[actions-badge]: https://github.com/tarantool/tarantool/workflows/release/badge.svg
[actions-url]: https://github.com/tarantool/tarantool/actions
[coverage-badge]: https://coveralls.io/repos/github/tarantool/tarantool/badge.svg?branch=master
[coverage-url]: https://coveralls.io/github/tarantool/tarantool?branch=master
[telegram-badge]: https://img.shields.io/badge/Telegram-join%20chat-blue.svg
[telegram-url]: http://telegram.me/tarantool
[discussions-badge]: https://img.shields.io/github/discussions/tarantool/tarantool
[discussions-url]: https://github.com/tarantool/tarantool/discussions
[stackoverflow-badge]: https://img.shields.io/badge/stackoverflow-tarantool-orange.svg
[stackoverflow-url]: https://stackoverflow.com/questions/tagged/tarantool
[oss-fuzz-badge]: https://oss-fuzz-build-logs.storage.googleapis.com/badges/tarantool.svg
[oss-fuzz-url]: https://oss-fuzz.com/coverage-report/job/libfuzzer_asan_tarantool/latest
[tarantool-url]: https://www.tarantool.io/en/
[license]: LICENSE
[modules]: https://www.tarantool.io/en/download/rocks
[queue]: https://github.com/tarantool/queue
[vshard]: https://github.com/tarantool/vshard
[cartridge]: https://github.com/tarantool/cartridge
[mysql]: https://github.com/tarantool/mysql
[pg]: https://github.com/tarantool/pg
[connectors]: https://www.tarantool.io/en/download/connectors
[download]: https://www.tarantool.io/en/download/
[building]: https://www.tarantool.io/en/doc/latest/dev_guide/building_from_source/
[issue-tracker]: https://github.com/tarantool/tarantool/issues
[contributing]: CONTRIBUTING.md
[awesome-list]: https://github.com/tarantool/awesome-tarantool/