Commit Briefs

24694eb5d5 Sergey Bronnikov

test: fix flakiness in gh_6128_background_mode_test (ligurio/gh-6128-fix-flaky-test-2)

Previous attempt to fix flakiness in commit 6a2c73f81b30 ("test: fix flakiness in gh_6128_background_mode_test") used a constant buffer size in check_err_msg function. Tarantool 2.10 has a bit bigger log before a desired message that other versions of Tarantool and it leads to a this resulted in a truncated message ("entering the even" instead of "entering the event loop"). Patch replaces check_err_msg() implementation to grep_log used in luatest, it reads the whole log. Also patch renames check_err_msg to check_msg, because "entering the event loop" is not an error message. Follows up #6128 NO_CHANGELOG=fix test NO_DOC=fix test


6a2c73f81b Igor Munkin

test: fix flakiness in gh_6128_background_mode_test

Test runs an external process with tarantool that writes to a log file. Then test reads that log file and searches a string with required message in it (see function check_err_msg). Test was flaky on macOS and I suspect it was happening due to a high log level - timeout was not enough to wait message in the log file. Patch decreases a log level to a default value and replaces io functions with the similar alternatives in a fio module. Using fio functions allows to not block fibers. NO_CHANGELOG=test fix NO_DOC=test fix (cherry picked from commit 47380bb7626544800239ca179a6b2c490ebb6d1c)


803cfffc3f Vladimir Davydov

lua/xlog: don't ignore unknown header fields

The xlog reader Lua module uses the xlog_cursor_next_row, which decodes the row header with xrow_header_decode. The latter silently ignores any unknown fields, which complicates catching bugs when garbage is written to a row header by mistake, for example, see #8783. Let's parse a row header without using xrow_header_decode in the xlog reader module, like we parse a row body, and output all unknown/invalid keys as is. To do that, we have to extend the xlog cursor API with the new method xlog_cursor_next_row_raw that returns a pointer to the position in the tx buffer where the next xrow is stored without advancing it. To avoid a memory leak in case the caller fails to parse an xrow returned by this function, we also have to move the call to xlog_tx_cursor_destroy from xlog_tx_cursor_next_row to xlog_cursor_next_tx. While we are at it, - Don't raise an error if a key type encountered in a row body is invalid (not an integer). Instead, silently ignore such keys. - Remove the useless body MsgPack validness check because we already check it after decoding the header. - Add error injection based tests to check all the corner cases. NO_DOC=bug fix (cherry picked from commit 8a25d170ea897d5aa2f1a81e1cbe8292596a6a6d)


f886d90696 Vladimir Davydov

txn: reset stream_id row header field

To avoid garbage written to xlog. Closes #8783 NO_DOC=bug fix NO_TEST=next commit (cherry picked from commit f058cee74eca494e8fca8e3aae6b94ce8105165b)


fd67e4dee6 Igor Munkin

box: reduce luaT_pushtuple Lua GC memory usage

<luaT_pushtuple> function created a new GCfunc object for a tuple __gc handler (i.e. <lbox_tuple_gc>), having no upvalues, on each call when tuple object for Lua world is created. The change introduces the reference to the single GCfunc object created on Tarantool startup and stored to Lua registry. Using this approach in scope of <luaT_pushtuple> is aimed to reduce Lua GC memory usage. Part of #5201 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring Reviewed-by: Vladimir Davydov <vdavydov@tarantool.org> Reviewed-by: Sergey Kaplun <skaplun@tarantool.org> Signed-off-by: Igor Munkin <imun@tarantool.org> (cherry picked from commit 8d7d14609853ac7020f15d5239ee05c65817ad29)


0daf852a1b Aleksandr Lyapunov

memtx: abort readers of rollbacked prepared

There's case when a transaction is rolled back from prepared state. This happens when WAL fails, synchronized replication confirmation failure or perhaps in other similar cases. By design other RW transactions and transactions with READ_COMMITTED isolation level are allowed to read prepared state. All these transactions must be aborted in case of rollback of prepared transaction since they definitely have read no more possible database state. This patch implements this abortion. Closed #8654 NO_DOC=bugfix (cherry picked from commit 54986902663db1e5d85525ab29499c4714bb3b73)


6ace23079d Aleksandr Lyapunov

memtx: fix and refactor memtx_tx_history_rollback_stmt

Rollback is rather complicated part if MVCC implementation that is meant to handle two kinds of rollback: * rollback from in-progress state, if box.rollback() is called. * rollback from prepared state, when WAL fails. Unfortunately the last one was not properly tested and surely has at least one flaw. When an inserting transaction becomes prepared its stories could be linked as deleted (via del_stmt pointer) by other in-progress transactions in order to maintain correct visibility. The problem is that in case of rollback of such prepared transaction those links remained. Broken links breaks the chain structure, and some older changes (that were linked as deleted before that hapless preparation) can become visible to other transaction. Refactor, simplify a bit that part of code and fix the issue described above; cover with tests. Closes #8648 NO_DOC=bugfix (cherry picked from commit 85569d9c2d75218fb402d74de0d159f1276123e3)


980b414d2c Aleksandr Lyapunov

memtx: simplify and refactor memtx_tx_history_prepare_stmt

Almost completely rewrite, simplify and comment this part of code. Part of #8648 Part of #8654 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring (cherry picked from commit e90150746d68299d24d96c437776ee71483dcb88)


513005da34 Aleksandr Lyapunov

memtx: simplify and refactor memtx_tx_history_add_stmt

Almost completely rewrite, simplify and comment this part of code. Part of #8648 Part of #8654 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring (cherry picked from commit 63da3bed43e9dbdc765913f7419bf58e82d25e31)


ff1e2614cf Aleksandr Lyapunov

memtx: replace is_pure_insert flag with is_own_change flag

The latter flag is a bit wider: it reveals not only inserting statements after deleting by the same transaction, but also replacing and deleting statements after all kinds of previois changes but the same transaction. This extended behavior will be used in further commits. Part of #8648 Part of #8654 NO_DOC=refactoring NO_TEST=refactoring NO_CHANGELOG=refactoring (cherry picked from commit 3cfa6756d31e3d215db7ad2c2714ac3bb473d66b)


Branches



























































































Tags

Tree

.editorconfigcommits | blame
.gdbinitcommits | blame
.gitattributescommits | blame
.github/
.gitignorecommits | blame
.gitmodulescommits | blame
.luacheckrccommits | blame
.pack.mkcommits | blame
.test.mkcommits | blame
AUTHORScommits | blame
CMakeLists.txtcommits | blame
Doxyfilecommits | blame
Doxyfile.API.incommits | blame
FreeBSD/
LICENSEcommits | blame
README.FreeBSDcommits | blame
README.MacOSXcommits | blame
README.OpenBSDcommits | blame
README.mdcommits | blame
TODOcommits | blame
apk/
asan/
changelogs/
cmake/
debian/
doc/
extra/
perf/
rpm/
rump/
src/
static-build/
test/
test-run$commits | blame
third_party/
tools/

README.md

# Tarantool

[![Actions Status][actions-badge]][actions-url]
[![Code Coverage][coverage-badge]][coverage-url]
[![Telegram][telegram-badge]][telegram-url]
[![GitHub Discussions][discussions-badge]][discussions-url]
[![Stack Overflow][stackoverflow-badge]][stackoverflow-url]

[Tarantool][tarantool-url] is an in-memory computing platform consisting of a
database and an application server.

It is distributed under [BSD 2-Clause][license] terms.

Key features of the application server:

* Heavily optimized Lua interpreter with incredibly fast tracing JIT compiler,
  based on LuaJIT 2.1.
* Cooperative multitasking, non-blocking IO.
* [Persistent queues][queue].
* [Sharding][vshard].
* [Cluster and application management framework][cartridge].
* Access to external databases such as [MySQL][mysql] and [PostgreSQL][pg].
* A rich set of built-in and standalone [modules][modules].

Key features of the database:

* MessagePack data format and MessagePack based client-server protocol.
* Two data engines: 100% in-memory with complete WAL-based persistence and an
  own implementation of LSM-tree, to use with large data sets.
* Multiple index types: HASH, TREE, RTREE, BITSET.
* Document oriented JSON path indexes.
* Asynchronous master-master replication.
* Synchronous quorum-based replication.
* RAFT-based automatic leader election for the single-leader configuration.
* Authentication and access control.
* ANSI SQL, including views, joins, referential and check constraints.
* [Connectors][connectors] for many programming languages.
* The database is a C extension of the application server and can be turned
  off.

Supported platforms are Linux (x86_64, aarch64), Mac OS X (x86_64, M1), FreeBSD
(x86_64).

Tarantool is ideal for data-enriched components of scalable Web architecture:
queue servers, caches, stateful Web applications.

To download and install Tarantool as a binary package for your OS or using
Docker, please see the [download instructions][download].

To build Tarantool from source, see detailed [instructions][building] in the
Tarantool documentation.

To find modules, connectors and tools for Tarantool, check out our [Awesome
Tarantool][awesome-list] list.

Please report bugs to our [issue tracker][issue-tracker]. We also warmly
welcome your feedback on the [discussions][discussions-url] page and questions
on [Stack Overflow][stackoverflow-url].

We accept contributions via pull requests. Check out our [How to get
involved][get-involved] guide.

Thank you for your interest in Tarantool!

[actions-badge]: https://github.com/tarantool/tarantool/workflows/release/badge.svg
[actions-url]: https://github.com/tarantool/tarantool/actions
[coverage-badge]: https://coveralls.io/repos/github/tarantool/tarantool/badge.svg?branch=master
[coverage-url]: https://coveralls.io/github/tarantool/tarantool?branch=master
[telegram-badge]: https://img.shields.io/badge/Telegram-join%20chat-blue.svg
[telegram-url]: http://telegram.me/tarantool
[discussions-badge]: https://img.shields.io/github/discussions/tarantool/tarantool
[discussions-url]: https://github.com/tarantool/tarantool/discussions
[stackoverflow-badge]: https://img.shields.io/badge/stackoverflow-tarantool-orange.svg
[stackoverflow-url]: https://stackoverflow.com/questions/tagged/tarantool
[tarantool-url]: https://www.tarantool.io/en/
[license]: LICENSE
[modules]: https://www.tarantool.io/en/download/rocks
[queue]: https://github.com/tarantool/queue
[vshard]: https://github.com/tarantool/vshard
[cartridge]: https://github.com/tarantool/cartridge
[mysql]: https://github.com/tarantool/mysql
[pg]: https://github.com/tarantool/pg
[connectors]: https://www.tarantool.io/en/download/connectors
[download]: https://www.tarantool.io/en/download/
[building]: https://www.tarantool.io/en/doc/latest/dev_guide/building_from_source/
[issue-tracker]: https://github.com/tarantool/tarantool/issues
[get-involved]: https://www.tarantool.io/en/doc/latest/contributing/contributing/
[awesome-list]: https://github.com/tarantool/awesome-tarantool/