Commit Briefs

1e440c53f1 Sergey Bronnikov

vinyl test (ligurio/gh-5076-make-vinyl-stable-again, origin/ligurio/gh-5076-make-vinyl-stable-again)


25382617b9 Kirill Yukhin

replication: append NOP as the last tx row

Since we stopped sending local space operations in replication, the last tx row has to be global in order to preserve tx boundaries on replica. If the last row happens to be a local one, replica will never receive the tx end marker, yielding the following errors: `ER_UNSUPPORTED: replication does not support interleaving transactions`. In order to fix the problem append a global NOP row at the tx end if it happens to end on a local row. Follow-up #4114 Closes #4928 Reviewed-by: Cyrill Gorcunov <gorcunov@gmail.com>


f41d1ddd5f Kirill Yukhin

wal: fix tx boundaries

In order to preserve transaction boundaries in replication protocol, wal assigns each tx row a transaction sequence number (tsn). Tsn is equal to the lsn of the first transaction row. Starting with commit 7eb4650eecf1ac382119d0038076c19b2708f4a1, local space requests are assigned a special replica id, 0, and have their own lsns. These operations are not replicated. If a transaction starting with a local space operation ends up in the WAL, it gets a tsn equal to the lsn of the local space request. Then, during replication, when such a transaction is replicated, the local space request is omitted, and replica receives a global part of the transaction with a seemingly random tsn, yielding an ER_PROTOCOL error: "Transaction id must be equal to LSN of the first row in the transaction". Assign tsn as equal to the lsn of the first global row in the transaction to fix the problem, and assign tsn as before for fully local transactions. Follow-up #4114 Part-of #4928 Reviewed-by: Cyrill Gorcunov <gorcunov@gmail.com>


9fcbbb3e7d Kirill Yukhin

applier: fix tx boundary check for half-applied txns

In case there are 2 "new" instances, running tarantool 2.2+, master and replica, and one "old" instance, running an earlier tarantool version, in a full-mesh cluster, it may happen that the "new" replica receives part of a tx from an "old" instance, and the remaining part from a "new" instance. Since "new" instances preserve tx boundaries, "new" replica would skip the tx remains assuming it has already applied the full tx if it has applied the first tx row. This leads to gaps in "new" replica's WAL and to skipping the remaining part of the tx forever. Fix this behaviour to apply the full tx even if it's beginning is already applied in mixed clusters. Closes #5125


4c7d828150 Kirill Yukhin

test: fix flaky box/net.box_readahead_gh-3958 test

Issue: [014] --- box/net.box_readahead_gh-3958.result Mon Jun 15 15:33:23 2020 [014] +++ box/net.box_readahead_gh-3958.reject Tue Jun 16 02:24:04 2020 [014] @@ -46,6 +46,7 @@ [014] ... [014] test_run:wait_log('default', 'readahead limit is reached', 1024, 0.1) [014] --- [014] +- readahead limit is reached [014] ... [014] s:drop() [014] --- [014] [014] Last 15 lines of Tarantool Log file [Instance "box"][/tarantool/test/var/014_box/box.log]: [014] 2020-06-16 02:24:03.792 [5585] main/121/console/unix/: I> set 'read_only' configuration option to false [014] 2020-06-16 02:24:03.834 [5585] iproto iproto.cc:606 W> stopping input on connection fd 26, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached [014] 2020-06-16 02:24:03.835 [5585] iproto iproto.cc:606 W> stopping input on connection fd 26, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached [014] 2020-06-16 02:24:03.835 [5585] iproto iproto.cc:606 W> stopping input on connection fd 26, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached [014] 2020-06-16 02:24:03.836 [5585] iproto iproto.cc:606 W> stopping input on connection fd 26, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached [014] 2020-06-16 02:24:03.836 [5585] iproto iproto.cc:606 W> stopping input on connection fd 26, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached [014] 2020-06-16 02:24:03.836 [5585] iproto iproto.cc:606 W> stopping input on connection fd 26, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached [014] 2020-06-16 02:24:03.836 [5585] iproto iproto.cc:606 W> stopping input on connection fd 26, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached [014] 2020-06-16 02:24:03.837 [5585] iproto iproto.cc:606 W> stopping input on connection fd 26, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached [014] 2020-06-16 02:24:03.837 [5585] iproto iproto.cc:606 W> stopping input on connection fd 26, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached [014] 2020-06-16 02:24:03.837 [5585] iproto iproto.cc:606 W> stopping input on connection fd 26, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached [014] 2020-06-16 02:24:03.951 [5585] main/121/console/unix/: space.h:336 E> ER_NO_SUCH_INDEX_ID: No index #1 is defined in space '_space' [014] 2020-06-16 02:24:04.180 [5585] main/121/console/unix/: I> set 'readahead' configuration option to 128 [014] 2020-06-16 02:24:04.183 [5585] main/121/console/unix/: I> set 'readahead' configuration option to 102400 [014] 2020-06-16 02:24:04.189 [5585] main/453/console/unix/: I> set 'readahead' configuration option to 16320 Found that the root cause of the issue, was the previously run test 'box/net.box_call_blocks_gh-946.test.lua' on the same worker, in this case the log output mistakenly checked by wait_log/grep_log test_run function, which finds the grepping string in the log of the previous test. To avoid of it the tests can be swapped in worker running queue and in this case both tests pass, check swapped log output: 2020-06-17 10:57:39.881 [69372] main C> entering the event loop 2020-06-17 10:57:39.896 [69372] main/119/console/unix/: I> set 'readahead' configuration option to 128 2020-06-17 10:57:39.898 [69372] main/119/console/unix/: I> set 'readahead' configuration option to 102400 2020-06-17 10:57:40.003 [69372] main/156/console/unix/: I> set 'readahead' configuration option to 16320 2020-06-17 10:57:40.053 [69372] iproto iproto.cc:606 W> stopping input on connection fd 33, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached 2020-06-17 10:57:40.056 [69372] iproto iproto.cc:606 W> stopping input on connection fd 33, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached 2020-06-17 10:57:40.056 [69372] iproto iproto.cc:606 W> stopping input on connection fd 33, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached 2020-06-17 10:57:40.058 [69372] iproto iproto.cc:606 W> stopping input on connection fd 33, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached 2020-06-17 10:57:40.058 [69372] iproto iproto.cc:606 W> stopping input on connection fd 33, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached 2020-06-17 10:57:40.061 [69372] iproto iproto.cc:606 W> stopping input on connection fd 33, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached 2020-06-17 10:57:40.061 [69372] iproto iproto.cc:606 W> stopping input on connection fd 33, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached 2020-06-17 10:57:40.062 [69372] iproto iproto.cc:606 W> stopping input on connection fd 33, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached 2020-06-17 10:57:40.062 [69372] iproto iproto.cc:606 W> stopping input on connection fd 33, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached 2020-06-17 10:57:40.063 [69372] iproto iproto.cc:606 W> stopping input on connection fd 33, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached 2020-06-17 10:57:40.067 [69372] main C> got signal 15 - Terminated Also found that 'readahead' issue from the first test blocks its printing to log file due to suppressed. To fix this issue the default server must be restarted at the very start of the test. Closes #5082


27ee93827c Kirill Yukhin

Correct cleanup gitlab-ci for perf jobs

Found that some perf jobs were forgot to be updated with local cleanup routine as was done for the other jobs at commit: 892a188bc56dc7052bcbe082607a83aa73f28c5b "Correct cleanup gitlab-ci" Follows up #5036


b74a46231b Kirill Yukhin

build: static build needs more cleanup in sources

Building Tarantool sources on make command run may fail with: [ 10%] make[2]: *** [test/small] Error 1 [ 10%] make[1]: *** [test/CMakeFiles/symlink_small_tests.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs.... The root cause of the issue that Dockerfile.staticbuild uses local copy of sources: COPY . /tarantool Which may have broken links in tests, like: $ ls -al test ... luajit-tap -> /<wrong path>/third_party/luajit/test small -> /<wrong path>/src/lib/small/test/ ... To fix the issue this links should be removed from the docker local copy of sources before build, like: rm -rf test/small test/luajit-tap Closes #5025


275b4fb0d8 Vladislav Shpilevoy

decimal: introduce decimal_is_int

This function will be used to determine, whether we can safely convert to integer without an information loss. Needed for #4415


2709573ed4 Vladislav Shpilevoy

decimal: introduce strtodec function

The behavior is similar with other strto* functions: parse the valid beginning of the string and optionally store the pointer to the first invalid character. Needed for tarantool/tarantool#4415


4e65cfe675 Vladislav Shpilevoy

decNumber: bump new version


Branches



























































































Tags

Tree

.appveyor.ymlcommits | blame
.editorconfigcommits | blame
.gdbinitcommits | blame
.github/
.gitignorecommits | blame
.gitlab-ci.ymlcommits | blame
.gitlab.mkcommits | blame
.gitmodulescommits | blame
.travis.mkcommits | blame
.travis.ymlcommits | blame
AUTHORScommits | blame
CMakeLists.txtcommits | blame
Dockerfile.staticbuildcommits | blame
Doxyfilecommits | blame
Doxyfile.API.incommits | blame
FreeBSD/
Jenkinsfilecommits | blame
LICENSEcommits | blame
README.FreeBSDcommits | blame
README.MacOSXcommits | blame
README.OpenBSDcommits | blame
README.mdcommits | blame
TODOcommits | blame
apk/
asan/
cmake/
debian/
doc/
extra/
rpm/
rump/
snapcraft.yamlcommits | blame
src/
test/
test-run$commits | blame
third_party/
tools/
vinyl.luacommits | blame

README.md

# Tarantool

[![Build Status][travis-badge]][travis-url]
[![Build Status][gitlab-ci-badge]][gitlab-ci-url]
[![Code Coverage][coverage-badge]][coverage-url]
[![Telegram][telegram-badge]][telegram-url]
[![Slack][slack-badge]][slack-url]
[![Gitter][gitter-badge]][gitter-url]
[![Google Groups][groups-badge]][groups-url]

https://tarantool.io/en/

Patch submissions and discussion of particular patches https://lists.tarantool.org/mailman/listinfo/tarantool-patches/

General development discussions https://lists.tarantool.org/mailman/listinfo/tarantool-discussions/

Tarantool is an in-memory database and application server.

Key features of the application server:
 * 100% compatible drop-in replacement for Lua 5.1,
   based on LuaJIT 2.1.
   Simply use `#!/usr/bin/tarantool` instead of
   `#!/usr/bin/lua` in your script.
 * full support for Lua modules and a rich set of
   own modules, including cooperative multitasking,
   non-blocking I/O, access to external databases, etc

Key features of the database:
 * ANSI SQL, including views, joins, referential
   and check constraints
 * MsgPack data format and MsgPack based
   client-server protocol
 * two data engines: 100% in-memory with
   optional persistence and an own implementation of LSM-tree, 
   to use with large data sets
 * multiple index types: HASH, TREE, RTREE, BITSET
 * asynchronous master-master replication
 * authentication and access control
 * the database is just a C extension to the
   application server and can be turned off

Supported platforms are Linux/x86, FreeBSD/x86 and OpenBSD/x86, Mac OS X.

Tarantool is ideal for data-enriched components of
scalable Web architecture: queue servers, caches,
stateful Web applications.

To download and install Tarantool as a binary package for your OS, please visit
https://tarantool.io/en/download/.

To build Tarantool from source, see detailed instructions in the Tarantool
documentation at https://tarantool.io/en/doc/2.1/dev_guide/building_from_source/.

Please report bugs at https://github.com/tarantool/tarantool/issues
We also warmly welcome your feedback in the discussion mailing
list, tarantool@googlegroups.com.

Thank you for your interest in Tarantool!

[travis-badge]: https://api.travis-ci.org/tarantool/tarantool.svg?branch=master
[travis-url]: https://travis-ci.org/tarantool/tarantool
[gitlab-ci-badge]: https://gitlab.com/tarantool/tarantool/badges/master/pipeline.svg
[gitlab-ci-url]: https://gitlab.com/tarantool/tarantool/commits/master
[coverage-badge]: https://coveralls.io/repos/github/tarantool/tarantool/badge.svg?branch=master
[coverage-url]: https://coveralls.io/github/tarantool/tarantool?branch=master
[groups-badge]: https://img.shields.io/badge/Google-Groups-orange.svg
[groups-url]: https://groups.google.com/forum/#!forum/tarantool
[telegram-badge]: https://img.shields.io/badge/Telegram-join%20chat-blue.svg
[telegram-url]: http://telegram.me/tarantool
[slack-badge]: https://img.shields.io/badge/Slack-join%20chat-lightgrey.svg
[slack-url]: http://slack.tarantool.org/
[gitter-badge]: https://badges.gitter.im/Join%20Chat.svg
[gitter-url]: https://gitter.im/tarantool/tarantool