- Description:
- [Mirror] Middleware for data
- Last Change:
- Clone URL:
Commit Briefs
perf/lua: add context section to test output (ligurio/setup-perf-env, origin/ligurio/setup-perf-env)
Google Benchmark output format contains a section "context" that describes useful information about test environment. Google Benchmark output format has been supported in Lua microbenchmarks in commit 3110ef9a9498 ("perf: introduce benchmark.lua helper module"). However, produced output contains test results only and section "context" is missed. The patch add a section "context" with the following fields: date, load average, hostname, tarantool's version, build flags and a name of build target. ``` $ tarantool uri_escape_unescape.lua --output=res.json --output_format=json $ jq ".context" res.json { "build_target": "Linux-x86_64-RelWithDebInfo", "host_name": "pony", "date": "2024-07-04 19:09:11", "tarantool_version": "3.2.0-entrypoint-114-g9e5dca29ad", "build_flags": " -fexceptions -funwind-tables -fasynchronous-unwind-tables -fno-common -msse2 -Wformat -Wformat-security -Werror=format-security -fstack-protector-strong -fPIC -fmacro-prefix-map=/home/sergeyb/sources/MRG/tarantool=. -std=c11 -Wall -Wextra -Wno-gnu-alignof-expression -Wno-cast-function-type -O2 -g -DNDEBUG -ggdb -O2 ", "load_avg": [ "0.76", "0.74", "0.63" ] } ``` NO_CHANGELOG=perf NO_DOC=perf NO_TEST=perf
perf/lua: protect require of column module
NO_CHANGELOG=perf NO_DOC=perf NO_TEST=perf
ci: enable BENCH_CMD
The patch enable environment variable `BENCH_CMD` introduced in a previous commit. The `taskset` alone will pin all the process threads into a single (random) isolated CPU, there's a ticket [1] about this in the Linux kernel bugtracker. The workaround is using realtime scheduler for the isolated task using `chrt` [2], e. g.: `taskset 0xef chrt 50`. 1. https://bugzilla.kernel.org/show_bug.cgi?id=116701 2. https://www.man7.org/linux/man-pages/man1/chrt.1.html NO_CHANGELOG=performance testing NO_DOC=performance testing NO_TEST=performance testing
perf: introduce BENCH_CMD environment variable
The patch introduces a BENCH_CMD, environment variable that could be set to a string with command and its arguments on CMake configuration stage and this string will be used as a pre-command for executing performance tests. Examples of these commands are `taskset` [1] and `numactl` [2], or any other utilities, see [3]. 1. https://man7.org/linux/man-pages/man1/taskset.1.html 2. https://man7.org/linux/man-pages/man8/numactl.8.html 3. https://github.com/tarantool/tarantool/wiki/Benchmarking#run-benchmarks NO_CHANGELOG=performance infra NO_DOC=performance infra NO_TEST=performance infra
perf: add a script for setting environment
"Benchmarking" article [0] in Tarantool's wiki contains a lot of recommendations that help to setup the Linux operating system and avoid potential reproducibility pitfalls when executing performance tests in a Linux-based environment. These recommendations written in plain text with examples of commands that could be executed manually. We desire to execute benchmarks automatically and in continuous mode, therefore we need a way to setup the test environment automatically before running benchmarks. There are many guides with benchmarking tips, but unfortunately there is no script that will do these steps automatically. I found only temci [1] and pyperf (`pyperf system` [2]) projects. The patch adds a script for setting the environment before running performance tests. All settings used in the proposed script are described in the article [3]. Note, that uncertain settings were not implemented. 0. https://github.com/tarantool/tarantool/wiki/Benchmarking 1. https://github.com/parttimenerd/temci 2. https://github.com/travisdowns/uarch-bench/blob/master/uarch-bench.sh 3. https://pyperf.readthedocs.io/en/latest/cli.html#system-cmd NO_CHANGELOG=performance NO_DOC=performance NO_TEST=performance
uuid: relax UUID value validation
This patch completely relaxes UUID checks and accepts an arbitrary 128-bit sequence as an UUID for binary data. String representations still should match the grammars in RFC 4122, Section 3 [1] and RFC 9562, Section 4 [2]. [1]: https://datatracker.ietf.org/doc/html/rfc4122#section-3 [2]: https://datatracker.ietf.org/doc/html/rfc9562#name-uuid-format Closes #5444 @TarantoolBot document Title: uuid: relaxed UUID validation [The UUID module documentation][1] mentions that Tarantool generates UUIDs following the rules for RFC 4122,[version 4, variant 1][2]. It is worth mentioning that the user can store an arbitrary 128-bit sequence as an UUID for binary data. String representations still should match the grammars in RFC 4122, [Section 3][3], and RFC 9562, [Section 4][4]. [1]: https://www.tarantool.io/en/doc/latest/reference/reference_lua/uuid/ [2]: https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_(random) [3]: https://datatracker.ietf.org/doc/html/rfc4122#section-3 [4]: https://datatracker.ietf.org/doc/html/rfc9562#name-uuid-format
test: rewrite engine/uuid.test.lua with luatest
NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring
test: rewrite app/uuid.test.lua with luatest
NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring
luajit: bump new version
* Correct fix for stack check when recording BC_VARG. * test: remove inline suppressions of _TARANTOOL * FFI: Fix ffi.alignof() for reference types. * FFI: Fix sizeof expression in C parser for reference types. * FFI: Allow ffi.metatype() for typedefs with attributes. * FFI: Fix ffi.metatype() for non-raw types. * Maintain chain invariant in DCE. * build: introduce option LUAJIT_ENABLE_TABLE_BUMP * ci: add tablebump flavor for exotic builds * test: allow `jit.parse` to return aborted traces * Handle all types of errors during trace stitching. * Use generic trace error for OOM during trace stitching. * Check for IR_HREF vs. IR_HREFK aliasing in non-nil store check. * cmake: set cmake_minimum_required only once * cmake: fix warning about minimum required version * ci: add a workflow for testing with AVX512 enabled * test: introduce a helper read_file * OSX/iOS/ARM64: Fix generation of Mach-O object files. * OSX/iOS/ARM64: Fix bytecode embedding in Mach-O object file. * build: introduce LUAJIT_USE_UBSAN option * ci: enable UBSan for sanitizers testing workflow * cmake: add the build directory to the .gitignore * Prevent sanitizer warning in snap_restoredata(). * Avoid negation of signed integers in C that may hold INT*_MIN. * Show name of NYI bytecode in -jv and -jdump. Closes #9924 Closes #8473 NO_DOC=LuaJIT submodule bump NO_TEST=LuaJIT submodule bump
fiber: phohibit fiber self join
In this case join will just hang. Instead let's raise an error in case of Lua API and panic in case of C API. Closes #10196 NO_DOC=minor
Branches
Tree
README.md
# Tarantool [![Actions Status][actions-badge]][actions-url] [![Code Coverage][coverage-badge]][coverage-url] [![OSS Fuzz][oss-fuzz-badge]][oss-fuzz-url] [![Telegram][telegram-badge]][telegram-url] [![GitHub Discussions][discussions-badge]][discussions-url] [![Stack Overflow][stackoverflow-badge]][stackoverflow-url] [Tarantool][tarantool-url] is an in-memory computing platform consisting of a database and an application server. It is distributed under [BSD 2-Clause][license] terms. Key features of the application server: * Heavily optimized Lua interpreter with incredibly fast tracing JIT compiler, based on LuaJIT 2.1. * Cooperative multitasking, non-blocking IO. * [Persistent queues][queue]. * [Sharding][vshard]. * [Cluster and application management framework][cartridge]. * Access to external databases such as [MySQL][mysql] and [PostgreSQL][pg]. * A rich set of built-in and standalone [modules][modules]. Key features of the database: * MessagePack data format and MessagePack based client-server protocol. * Two data engines: 100% in-memory with complete WAL-based persistence and an own implementation of LSM-tree, to use with large data sets. * Multiple index types: HASH, TREE, RTREE, BITSET. * Document oriented JSON path indexes. * Asynchronous master-master replication. * Synchronous quorum-based replication. * RAFT-based automatic leader election for the single-leader configuration. * Authentication and access control. * ANSI SQL, including views, joins, referential and check constraints. * [Connectors][connectors] for many programming languages. * The database is a C extension of the application server and can be turned off. Supported platforms are Linux (x86_64, aarch64), Mac OS X (x86_64, M1), FreeBSD (x86_64). Tarantool is ideal for data-enriched components of scalable Web architecture: queue servers, caches, stateful Web applications. To download and install Tarantool as a binary package for your OS or using Docker, please see the [download instructions][download]. To build Tarantool from source, see detailed [instructions][building] in the Tarantool documentation. To find modules, connectors and tools for Tarantool, check out our [Awesome Tarantool][awesome-list] list. Please report bugs to our [issue tracker][issue-tracker]. We also warmly welcome your feedback on the [discussions][discussions-url] page and questions on [Stack Overflow][stackoverflow-url]. We accept contributions via pull requests. Check out our [contributing guide][contributing]. Thank you for your interest in Tarantool! [actions-badge]: https://github.com/tarantool/tarantool/workflows/release/badge.svg [actions-url]: https://github.com/tarantool/tarantool/actions [coverage-badge]: https://coveralls.io/repos/github/tarantool/tarantool/badge.svg?branch=master [coverage-url]: https://coveralls.io/github/tarantool/tarantool?branch=master [telegram-badge]: https://img.shields.io/badge/Telegram-join%20chat-blue.svg [telegram-url]: http://telegram.me/tarantool [discussions-badge]: https://img.shields.io/github/discussions/tarantool/tarantool [discussions-url]: https://github.com/tarantool/tarantool/discussions [stackoverflow-badge]: https://img.shields.io/badge/stackoverflow-tarantool-orange.svg [stackoverflow-url]: https://stackoverflow.com/questions/tagged/tarantool [oss-fuzz-badge]: https://oss-fuzz-build-logs.storage.googleapis.com/badges/tarantool.svg [oss-fuzz-url]: https://oss-fuzz.com/coverage-report/job/libfuzzer_asan_tarantool/latest [tarantool-url]: https://www.tarantool.io/en/ [license]: LICENSE [modules]: https://www.tarantool.io/en/download/rocks [queue]: https://github.com/tarantool/queue [vshard]: https://github.com/tarantool/vshard [cartridge]: https://github.com/tarantool/cartridge [mysql]: https://github.com/tarantool/mysql [pg]: https://github.com/tarantool/pg [connectors]: https://www.tarantool.io/en/download/connectors [download]: https://www.tarantool.io/en/download/ [building]: https://www.tarantool.io/en/doc/latest/dev_guide/building_from_source/ [issue-tracker]: https://github.com/tarantool/tarantool/issues [contributing]: CONTRIBUTING.md [awesome-list]: https://github.com/tarantool/awesome-tarantool/