Commit Briefs

f611002bfe Sergey Bronnikov

test/cmake: add a function for generating unit test targets (ligurio/unit-testing-depends-on-changes)

Commit 9adedc1f00d2 ("test: add new `make` test targets") introduced new targets for running test-run.py with unit tests. However, this target doesn't depend on changes in tested libraries and changes in unit tests. Proposed patch introduces a function that creates a build targets for unit tests and fixes the described problem with dependencies. Additionally, patch added missed dependence for popen and popen-child. NO_CHANGELOG=internal NO_DOC=internal


f8a27936db Sergey Bronnikov

test/cmake: refactoring for unit tests

NO_CHANGELOG=internal NO_DOC=internal NO_TEST=refactoring Needed for the next commit


c9155ac863 Vladislav Shpilevoy

raft: persist new term and vote separately

If a node persisted a foreign term + vote request at the same time, it increased split-brain probability. A node could vote for a candidate having smaller vclock than the local one. For example, via the following scenario: - Node1, node2, node3 are started; - Node1 becomes a leader; - The topology becomes node1 <-> node2 <-> node3 due to network issues; - Node1 sends a synchro txn to node2. The txn starts a WAL write; - Node3 bumps term and votes for self. Sends it all to node2; - Node2 votes for node3, because their vclocks are equal; - Node2 finishes all pending WAL writes, including the txn from node1. Now its vclock is > node3's one and the vote was wrong. - Node3 wins, writes PROMOTE, and it conflicts with node1 writing CONFIRM. This patch makes so a node can't persist a vote in a new term in the same WAL write as the term bump. Term bump is written first and alone. It serves as a WAL sync after which the node's vclock is not supposed to change except for the 0 (local) component. The vote requests are re-checked after term bump is persisted to see if they still can be applied. Part of #7253 NO_DOC=bugfix


ec6281002f Vladislav Shpilevoy

qsync: fix txn fiber hang on fencing at CONFIRM

If the limbo was fenced during CONFIRM WAL write, then the confirmed txn was committed just fine, but its author-fiber kept hanging. This is because when it was woken up, it checked if the limbo is frozen and went to infinite waiting before actually checking if the txn is completed. The fiber would unfreeze if would be woken up explicitly as a workaround. The fix is simple - change the checks order. Part of #7253 NO_DOC=bugfix


ab08dad950 Vladislav Shpilevoy

promote: abort it when become non-candidate

box.ctl.promote() bumps the term, makes the node a candidate, and waits for the term outcome. The waiting used to be until there is a leader elected or the node lost connection quorum or the term was bumped again. There was a bug that a node could hang in box.ctl.promote() even when became a voter. It could happen if the quorum was still there and a leader couldn't be elected in the current term at all. For instance, others could have `election_mode='off'`. The fix is to stop waiting for the term outcome if the node can't win anyway. NO_DOC=bugfix


dd89c57e78 Vladislav Shpilevoy

promote: fix infinite elections with multi-promote

If box.ctl.promote() was called on more than one instance, then it could lead to infinite or extremely long elections bumping thousands of terms in just a few seconds. This was because box.ctl.promote() used to be a loop. The loop retried term bump + voted for self until the node won. Retry happened immediately as the node saw the term was bumped again and there was no leader elected or the connection quorum was lost. If 2 nodes would start box.ctl.promote() almost at the same time, they could bump each other's terms, not see any winner, bump them again, and so on. For example: - Node1 term=1, node2 term=2; - Promote is called on both; - Node1 term=2, node2 term=3. They receive the messages. Node2 ignores node1's old term. Node1 term is bumped and it votes for node2, but it didn't win, so box.ctl.promote() bumps its term to 4. - Node2 receives term 4 from node1. Its own box.ctl.promote() sees the term was bumped and no winner, so it bumps it to 5 and the process continues for a long time. It worked good enough in tests - the problem happened sometimes, terms could roll like 80k times in a few seconds, but the tests ended fine anyway. One of the next commits will make term bump + vote written in separate WAL records. That aggravates the problem drastically. Basically, this mutual term bump loop could end only if one node would receive vote for self from another node and send back the message 'I am a leader' before the other node's box.ctl.promote() notices the term was bumped externally. This will get much harder to achieve. The patch simply drops the loop. Let box.ctl.promote() fail if the term was bumped outside. There was an alternative to keep running it in a loop with a randomized election timeout like it works inside of raft. But the current solution is just simpler. NO_DOC=bugfix NO_TEST=election_split_vote_test.lua catches it already


fe594e3ccc Vladimir Davydov

read_view: add space upgrade function to space_read_view

There may be a space upgrade in progress at the time when a read view is created. In this case, we should apply the upgrade function to tuples retrieved from the space read view. We can't just use the space upgrade function as is, because it may be dropped while the read view is still in use. So we need to create a special upgrade function for the read view. This commit adds stubs for this, which will be implemented in the EE repository. Needed for https://github.com/tarantool/tarantool-ee/issues/163 NO_DOC=internal NO_TEST=ee NO_CHANGELOG=internal


4156440aa1 Vladimir Davydov

read_view: assert that read view is used only in one thread

Although currenlty feasible, using a read view from multiple threads simultaneously is dangerous on its own because of inevitably arising object life time issues. With introduction of space upgrade handling, it'll become hardly possible, because we'll have to attach Lua function to a read view, which can only be used from one thread. Let's add some assertions to guarantee that a read view is used only from one thread. To do that, we need to introduce the concept of an active read view: from now on, a read view must be activated before it can be used and deactivated before it can be closed; both operations must be done from the same thread - the thread that accesses the read view. https://github.com/tarantool/tarantool-ee/issues/163 NO_DOC=debug NO_TEST=debug NO_CHANGELOG=debug


20b0665664 Vladimir Davydov

fiber: make fiber_set_cancellable a no-op

Currently this function is not used inside Tarantool, however it is available via C module API. Deprecate it, because it is very confusing and has nothing to do with the fiber cancellation. Closes #7166 @TarantoolBot document Title: fiber: get rid of fiber_set_cancellable Product: Tarantool Since: 2.11 Audience/target: dev Root document: https://www.tarantool.io/en/doc/latest/dev_guide/reference_capi/fiber/#c.fiber_set_cancellable SME: @Gumix This function is a no-op since 2.11 and should be dropped from the documentation.


dd65d5c673 Vladimir Davydov

main: allow spurious wakeups in on_shutdown_f

It will yield until the is_shutting_down flag is set by tarantool_exit(). This allows to get rid of the FIBER_IS_CANCELLABLE flag, which is no longer used anywhere in Tarantool. Part of #7166 NO_DOC=internal NO_CHANGELOG=internal


Branches



























































































Tags

Tree

.editorconfigcommits | blame
.gdbinitcommits | blame
.github/
.gitignorecommits | blame
.gitmodulescommits | blame
.luacheckrccommits | blame
.pack.mkcommits | blame
.test.mkcommits | blame
AUTHORScommits | blame
CMakeLists.txtcommits | blame
CONTRIBUTING.mdcommits | blame
Doxyfilecommits | blame
Doxyfile.API.incommits | blame
FreeBSD/
LICENSEcommits | blame
README.FreeBSDcommits | blame
README.MacOSXcommits | blame
README.OpenBSDcommits | blame
README.mdcommits | blame
TODOcommits | blame
apk/
asan/
changelogs/
cmake/
debian/
doc/
extra/
perf/
rpm/
rump/
src/
static-build/
test/
test-run$commits | blame
third_party/
tools/

README.md

# Tarantool

[![Actions Status][actions-badge]][actions-url]
[![Code Coverage][coverage-badge]][coverage-url]
[![Telegram][telegram-badge]][telegram-url]
[![GitHub Discussions][discussions-badge]][discussions-url]
[![Stack Overflow][stackoverflow-badge]][stackoverflow-url]

[Tarantool][tarantool-url] is an in-memory computing platform consisting of a
database and an application server.

It is distributed under [BSD 2-Clause][license] terms.

Key features of the application server:

* Heavily optimized Lua interpreter with incredibly fast tracing JIT compiler,
  based on LuaJIT 2.1.
* Cooperative multitasking, non-blocking IO.
* [Persistent queues][queue].
* [Sharding][vshard].
* [Cluster and application management framework][cartridge].
* Access to external databases such as [MySQL][mysql] and [PostgreSQL][pg].
* A rich set of built-in and standalone [modules][modules].

Key features of the database:

* MessagePack data format and MessagePack based client-server protocol.
* Two data engines: 100% in-memory with complete WAL-based persistence and an
  own implementation of LSM-tree, to use with large data sets.
* Multiple index types: HASH, TREE, RTREE, BITSET.
* Document oriented JSON path indexes.
* Asynchronous master-master replication.
* Synchronous quorum-based replication.
* RAFT-based automatic leader election for the single-leader configuration.
* Authentication and access control.
* ANSI SQL, including views, joins, referential and check constraints.
* [Connectors][connectors] for many programming languages.
* The database is a C extension of the application server and can be turned
  off.

Supported platforms are Linux (x86_64, aarch64), Mac OS X (x86_64, M1), FreeBSD
(x86_64).

Tarantool is ideal for data-enriched components of scalable Web architecture:
queue servers, caches, stateful Web applications.

To download and install Tarantool as a binary package for your OS or using
Docker, please see the [download instructions][download].

To build Tarantool from source, see detailed [instructions][building] in the
Tarantool documentation.

To find modules, connectors and tools for Tarantool, check out our [Awesome
Tarantool][awesome-list] list.

Please report bugs to our [issue tracker][issue-tracker]. We also warmly
welcome your feedback on the [discussions][discussions-url] page and questions
on [Stack Overflow][stackoverflow-url].

We accept contributions via pull requests. Check out our [contributing
guide][contributing].

Thank you for your interest in Tarantool!

[actions-badge]: https://github.com/tarantool/tarantool/workflows/release/badge.svg
[actions-url]: https://github.com/tarantool/tarantool/actions
[coverage-badge]: https://coveralls.io/repos/github/tarantool/tarantool/badge.svg?branch=master
[coverage-url]: https://coveralls.io/github/tarantool/tarantool?branch=master
[telegram-badge]: https://img.shields.io/badge/Telegram-join%20chat-blue.svg
[telegram-url]: http://telegram.me/tarantool
[discussions-badge]: https://img.shields.io/github/discussions/tarantool/tarantool
[discussions-url]: https://github.com/tarantool/tarantool/discussions
[stackoverflow-badge]: https://img.shields.io/badge/stackoverflow-tarantool-orange.svg
[stackoverflow-url]: https://stackoverflow.com/questions/tagged/tarantool
[tarantool-url]: https://www.tarantool.io/en/
[license]: LICENSE
[modules]: https://www.tarantool.io/en/download/rocks
[queue]: https://github.com/tarantool/queue
[vshard]: https://github.com/tarantool/vshard
[cartridge]: https://github.com/tarantool/cartridge
[mysql]: https://github.com/tarantool/mysql
[pg]: https://github.com/tarantool/pg
[connectors]: https://www.tarantool.io/en/download/connectors
[download]: https://www.tarantool.io/en/download/
[building]: https://www.tarantool.io/en/doc/latest/dev_guide/building_from_source/
[issue-tracker]: https://github.com/tarantool/tarantool/issues
[contributing]: CONTRIBUTING.md
[awesome-list]: https://github.com/tarantool/awesome-tarantool/