Commit Briefs

ace933e65e Sergey Bronnikov

test: fix tarantool process teardown (ligurio/gh-6128-fix-teardown)

Test uses a popen module that starts tarantool process in background mode. Tarantool process started in background mode forks a new process and closes a parent, after that popen loses a PID of the started process and `ph:kill()` and `ph:terminate()` doesn't work anymore. It leads to non-terminated tarantool processes after running the test. Patch fixes that by running `kill` using os.execute with a PID of tarantool process written to a pid file. Follows up #6128 NO_CHANGELOG=fix test NO_DOC=fix test


9041d7eda8 Yaroslav Lobankov

ci: add workflow for static build packaging

The created packages can be found in the job artifacts. NO_DOC=ci NO_TEST=ci NO_CHANGELOG=ci Co-authored-by: Yaroslav Lobankov <y.lobankov@tarantool.org>


ed4060039e Yaroslav Lobankov

make: add new target `package-static` to .pack.mk

This target is a wrapper on the command to build DEB and RPM packages with a statically compiled Tarantool binary inside. It just runs the `./static-build/make_packages.sh` script with the properly defined VERSION env variable. Example of usage: $ make -f .pack.mk package-static NO_DOC=make NO_TEST=make NO_CHANGELOG=make


390cddbbbd Yaroslav Lobankov

build: make packages with static binary inside

This patch adds facility to make DEB and RPM packages with a statically compiled Tarantool binary inside. The build is performed in a Docker container, using PackPack docker image (centos-7) and CPack. The packpack/packpack:centos-7 image has all the necessary dependencies for building Tarantool and quite old glibc 2.17 which theoretically provides compatibility of the created packages with any distro where glibc >= 2.17. The build can be run with the command below: $ VERSION=3.0.0 ./static-build/make_packages.sh In the `static_build` directory, there will be the following packages: tarantool_3.0.0-1_amd64.deb tarantool-dev_3.0.0-1_amd64.deb tarantool-3.0.0-1.x86_64.rpm tarantool-devel-3.0.0-1.x86_64.rpm `tarantool_3.0.0-1_amd64.deb`, `tarantool-3.0.0-1.x86_64.rpm` are packages with the Tarantool server binary inside. `tarantool-dev_3.0.0-1_amd64.deb`, `tarantool-devel-3.0.0-1.x86_64.rpm` are packages with the Tarantool server development files inside. NO_DOC=build NO_TEST=build Co-authored-by: Yaroslav Lobankov <y.lobankov@tarantool.org>


dc26e47e1b Vladimir Davydov

box: introduce formats for standalone tuples

Introduce `box.tuple.format` object, a Lua wrapper around tuple format: these objects own a tuple format, which is almost equivalent to `space:format`, except for check constraints and foreign key constraints being disabled (though they appear to be present for compatibility with `space:format`). Add an option table argument to `box.tuple.new` with 'format' option, allowing to create formatted spaceless tuples. Closes #4693 @TarantoolBot document Title: Formats for standalone tuples and `box_tuple_new_vararg` compat opt A new box.tuple.format library was added, with a tuple format constructor (`new`) and a tuple format validator (`is`). New tuple format objects (userdata) were added, which can be used with the same format clause as for the `space:format` method (except that check constraints and foreign keys are disabled for them): NO_WRAP ```lua f = box.tuple.format.new(box.space._space:format()) f = box.tuple.format.new{{name = 'field1', type = 'string', is_nullable = true, nullable_action = 'none', collation = 'unicode_uk_s2', default = 'UPPER("string")', constraint = {ck = 'box.schema.user.info'}, foreign_key = {fk = {space = '_space', field = 'name'}}}, {name = 'field2', nullable_action = 'ignore', foreign_key = {fk = {space = '_space', field = 1}}}} ``` NO_WRAP Format objects have several introspection methods: `:pairs`, `:ipairs`, `totable`, and also have a `__serialize` metamethod — these methods return the original (i.e., user-provided) format clause. `:pairs` is an alias to `ipairs` (since the format clause is an array by nature), and the `totable` method is an alias to the `__serialize` metamethod, which returns an array of field definitions. Format objects also have a `:tostring` method, which simply returns a "box.tuple.format" literal. The standalone tuple constructor, `box.tuple.new` was extended with an options parameter which currently has one available option, `format` (default value is `nil`, i.e., no format). The format option is either a tuple format object previously created using `box.tuple.format.new` or a format clause. Examples of standalone tuple creation with formats: NO_WRAP ```lua box.tuple.new({1}, {format = {{name = 'field', type = 'number'}}}) box.tuple.new({1}, {format = {{'field', type = 'number'}}}) box.tuple.new({1}, {format = {{'field', 'number'}}}) f = box.tuple.format.new({{name = 'field', type = 'number'}}) box.tuple.new({}, {format = f}) box.tuple.new({1}, {format = f}) box.tuple.new({'str'}, {format = f}) -- error: Tuple field 1 (field) type does not match one required by operation: expected number, got string box.tuple.new({'str'}, {format = f}) ``` NO_WRAP See also the design document https://www.notion.so/tarantool/Schemafull-IPROTO-cc315ad6bdd641dea66ad854992d8cbf?pvs=4#a33e2d7418d249679969e5f21ef2832c A new `box_tuple_new_vararg` compatibility option was introduced: a new page needs to be created for it (https://tarantool.io/compat/box_tuple_new_vararg) This option controls whether `box.tuple.new` should interpret an argument list as an array of tuple fields (i.e., vararg, old behaviour), or as a value plus a tuple format (new default behaviour). The value can be either a scalar, an array or a box tuple. The old behaviour does not allow creating formatted standalone tuples. Old behaviour examples: ```lua box.tuple.new(1) box.tuple.new{1} box.tuple.new(1, 2, 3) box.tuple.new{1, 2, 3} -- This won't create a formatted tuple: the format option will become the -- second tuple field. box.tuple.new({1, 2, 3}, {format = box.tuple.format.new{{'field'}}}) ``` New behaviour examples: ```lua box.tuple.new(1) box.tuple.new(1, {format = box.tuple.format.new{{'field'}}}) box.tuple.new{1} box.tuple.new({1}, {format = box.tuple.format.new{{'field'}}}) box.tuple.new(1, 2, 3) -- error box.tuple.new(1, 2, 3, {format = box.tuple.format.new{{'field'}}}) -- error box.tuple.new{1, 2, 3} box.tuple.new({1, 2, 3}, {format = box.tuple.format.new{{'field'}}}) ``` See also the design document https://www.notion.so/tarantool/Schemafull-IPROTO-cc315ad6bdd641dea66ad854992d8cbf?pvs=4#6f74f0c70005463b8438830edd1a0117.


05e8b1de21 Vladimir Davydov

box: account for all tuple format field definitions in `cmp` and `hash`

Previously, reusable tuple formats were only used for ephemeral spaces and by `net.box` (which only used 'name' definitions), so non-space tuple format definitions were ignored in tuple format comparison and hash functions, but in scope of #4693 reusable tuple formats will be interchangeable with space formats, so now we need to account for these too. Needed for #4693 NO_CHANGELOG=internal NO_DOC=internal


602060ae15 Vladimir Davydov

box: save MsgPack encoding of original (user-provided) format clause

In scope of #4693 we need to save the MsgPack encoding of the original (i.e, user-provided) format clause in the tuple format structure for serialization to Lua and IPROTO (#8147, #8633): since tuple formats for spaces are created from space definitions, we need to also save the MsgPack encoding in the latter. Since we need to pass the format clause MsgPack encoding to the runtime tuple format constructor, refactor it to perform field definition decoding in-place. We cannot use default field definition array decoding for external formats (67578d1), so introduce a new field name decoder for this case. Needed for #4693 NO_CHANGELOG=refactoring NO_DOC=refactoring NO_TEST=refactoring


3a73bad33c Vladimir Davydov

box: make Lua tuple formats objects userdata instead of cdata

userdata is the preferred type for Lua objects wrapping C structures, but tuple formats were made cdata for no good reason: change them to userdata. Needed for #4693 NO_CHANGELOG=refactoring NO_DOC=refactoring NO_TEST=refactoring


f6db046422 Vladimir Davydov

box: move Lua tuple format wrapper to new `tuple_format` submodule

Since we are going to make public Lua tuple format objects, introduce a new `tuple_format` submodule and move the code related to these objects there. Needed for #4693 NO_DOC=refactoring NO_CHANGELOG=refactoring NO_TEST=refactoring


47380bb762 Igor Munkin

test: fix flakiness in gh_6128_background_mode_test

Test runs an external process with tarantool that writes to a log file. Then test reads that log file and searches a string with required message in it (see function check_err_msg). Test was flaky on macOS and I suspect it was happening due to a high log level - timeout was not enough to wait message in the log file. Patch decreases a log level to a default value and replaces io functions with the similar alternatives in a fio module. Using fio functions allows to not block fibers. NO_CHANGELOG=test fix NO_DOC=test fix


Branches



























































































Tags

Tree

.editorconfigcommits | blame
.gdbinitcommits | blame
.gitattributescommits | blame
.github/
.gitignorecommits | blame
.gitmodulescommits | blame
.luacheckrccommits | blame
.pack.mkcommits | blame
.test.mkcommits | blame
AUTHORScommits | blame
CMakeLists.txtcommits | blame
CONTRIBUTING.mdcommits | blame
Doxyfilecommits | blame
Doxyfile.API.incommits | blame
FreeBSD/
LICENSEcommits | blame
README.FreeBSDcommits | blame
README.MacOSXcommits | blame
README.OpenBSDcommits | blame
README.mdcommits | blame
TODOcommits | blame
apk/
asan/
changelogs/
cmake/
debian/
doc/
extra/
perf/
rpm/
rump/
src/
static-build/
test/
test-run$commits | blame
third_party/
tools/

README.md

# Tarantool

[![Actions Status][actions-badge]][actions-url]
[![Code Coverage][coverage-badge]][coverage-url]
[![OSS Fuzz][oss-fuzz-badge]][oss-fuzz-url]
[![Telegram][telegram-badge]][telegram-url]
[![GitHub Discussions][discussions-badge]][discussions-url]
[![Stack Overflow][stackoverflow-badge]][stackoverflow-url]

[Tarantool][tarantool-url] is an in-memory computing platform consisting of a
database and an application server.

It is distributed under [BSD 2-Clause][license] terms.

Key features of the application server:

* Heavily optimized Lua interpreter with incredibly fast tracing JIT compiler,
  based on LuaJIT 2.1.
* Cooperative multitasking, non-blocking IO.
* [Persistent queues][queue].
* [Sharding][vshard].
* [Cluster and application management framework][cartridge].
* Access to external databases such as [MySQL][mysql] and [PostgreSQL][pg].
* A rich set of built-in and standalone [modules][modules].

Key features of the database:

* MessagePack data format and MessagePack based client-server protocol.
* Two data engines: 100% in-memory with complete WAL-based persistence and an
  own implementation of LSM-tree, to use with large data sets.
* Multiple index types: HASH, TREE, RTREE, BITSET.
* Document oriented JSON path indexes.
* Asynchronous master-master replication.
* Synchronous quorum-based replication.
* RAFT-based automatic leader election for the single-leader configuration.
* Authentication and access control.
* ANSI SQL, including views, joins, referential and check constraints.
* [Connectors][connectors] for many programming languages.
* The database is a C extension of the application server and can be turned
  off.

Supported platforms are Linux (x86_64, aarch64), Mac OS X (x86_64, M1), FreeBSD
(x86_64).

Tarantool is ideal for data-enriched components of scalable Web architecture:
queue servers, caches, stateful Web applications.

To download and install Tarantool as a binary package for your OS or using
Docker, please see the [download instructions][download].

To build Tarantool from source, see detailed [instructions][building] in the
Tarantool documentation.

To find modules, connectors and tools for Tarantool, check out our [Awesome
Tarantool][awesome-list] list.

Please report bugs to our [issue tracker][issue-tracker]. We also warmly
welcome your feedback on the [discussions][discussions-url] page and questions
on [Stack Overflow][stackoverflow-url].

We accept contributions via pull requests. Check out our [contributing
guide][contributing].

Thank you for your interest in Tarantool!

[actions-badge]: https://github.com/tarantool/tarantool/workflows/release/badge.svg
[actions-url]: https://github.com/tarantool/tarantool/actions
[coverage-badge]: https://coveralls.io/repos/github/tarantool/tarantool/badge.svg?branch=master
[coverage-url]: https://coveralls.io/github/tarantool/tarantool?branch=master
[telegram-badge]: https://img.shields.io/badge/Telegram-join%20chat-blue.svg
[telegram-url]: http://telegram.me/tarantool
[discussions-badge]: https://img.shields.io/github/discussions/tarantool/tarantool
[discussions-url]: https://github.com/tarantool/tarantool/discussions
[stackoverflow-badge]: https://img.shields.io/badge/stackoverflow-tarantool-orange.svg
[stackoverflow-url]: https://stackoverflow.com/questions/tagged/tarantool
[oss-fuzz-badge]: https://oss-fuzz-build-logs.storage.googleapis.com/badges/tarantool.svg
[oss-fuzz-url]: https://oss-fuzz.com/coverage-report/job/libfuzzer_asan_tarantool/latest
[tarantool-url]: https://www.tarantool.io/en/
[license]: LICENSE
[modules]: https://www.tarantool.io/en/download/rocks
[queue]: https://github.com/tarantool/queue
[vshard]: https://github.com/tarantool/vshard
[cartridge]: https://github.com/tarantool/cartridge
[mysql]: https://github.com/tarantool/mysql
[pg]: https://github.com/tarantool/pg
[connectors]: https://www.tarantool.io/en/download/connectors
[download]: https://www.tarantool.io/en/download/
[building]: https://www.tarantool.io/en/doc/latest/dev_guide/building_from_source/
[issue-tracker]: https://github.com/tarantool/tarantool/issues
[contributing]: CONTRIBUTING.md
[awesome-list]: https://github.com/tarantool/awesome-tarantool/