- Description:
- [Mirror] Middleware for data
- Last Change:
- Clone URL:
Commit Briefs
vinyl test (ligurio/gh-5076-make-vinyl-stable-again, origin/ligurio/gh-5076-make-vinyl-stable-again)
replication: append NOP as the last tx row
Since we stopped sending local space operations in replication, the last tx row has to be global in order to preserve tx boundaries on replica. If the last row happens to be a local one, replica will never receive the tx end marker, yielding the following errors: `ER_UNSUPPORTED: replication does not support interleaving transactions`. In order to fix the problem append a global NOP row at the tx end if it happens to end on a local row. Follow-up #4114 Closes #4928 Reviewed-by: Cyrill Gorcunov <gorcunov@gmail.com>
wal: fix tx boundaries
In order to preserve transaction boundaries in replication protocol, wal assigns each tx row a transaction sequence number (tsn). Tsn is equal to the lsn of the first transaction row. Starting with commit 7eb4650eecf1ac382119d0038076c19b2708f4a1, local space requests are assigned a special replica id, 0, and have their own lsns. These operations are not replicated. If a transaction starting with a local space operation ends up in the WAL, it gets a tsn equal to the lsn of the local space request. Then, during replication, when such a transaction is replicated, the local space request is omitted, and replica receives a global part of the transaction with a seemingly random tsn, yielding an ER_PROTOCOL error: "Transaction id must be equal to LSN of the first row in the transaction". Assign tsn as equal to the lsn of the first global row in the transaction to fix the problem, and assign tsn as before for fully local transactions. Follow-up #4114 Part-of #4928 Reviewed-by: Cyrill Gorcunov <gorcunov@gmail.com>
applier: fix tx boundary check for half-applied txns
In case there are 2 "new" instances, running tarantool 2.2+, master and replica, and one "old" instance, running an earlier tarantool version, in a full-mesh cluster, it may happen that the "new" replica receives part of a tx from an "old" instance, and the remaining part from a "new" instance. Since "new" instances preserve tx boundaries, "new" replica would skip the tx remains assuming it has already applied the full tx if it has applied the first tx row. This leads to gaps in "new" replica's WAL and to skipping the remaining part of the tx forever. Fix this behaviour to apply the full tx even if it's beginning is already applied in mixed clusters. Closes #5125
test: fix flaky box/net.box_readahead_gh-3958 test
Issue: [014] --- box/net.box_readahead_gh-3958.result Mon Jun 15 15:33:23 2020 [014] +++ box/net.box_readahead_gh-3958.reject Tue Jun 16 02:24:04 2020 [014] @@ -46,6 +46,7 @@ [014] ... [014] test_run:wait_log('default', 'readahead limit is reached', 1024, 0.1) [014] --- [014] +- readahead limit is reached [014] ... [014] s:drop() [014] --- [014] [014] Last 15 lines of Tarantool Log file [Instance "box"][/tarantool/test/var/014_box/box.log]: [014] 2020-06-16 02:24:03.792 [5585] main/121/console/unix/: I> set 'read_only' configuration option to false [014] 2020-06-16 02:24:03.834 [5585] iproto iproto.cc:606 W> stopping input on connection fd 26, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached [014] 2020-06-16 02:24:03.835 [5585] iproto iproto.cc:606 W> stopping input on connection fd 26, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached [014] 2020-06-16 02:24:03.835 [5585] iproto iproto.cc:606 W> stopping input on connection fd 26, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached [014] 2020-06-16 02:24:03.836 [5585] iproto iproto.cc:606 W> stopping input on connection fd 26, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached [014] 2020-06-16 02:24:03.836 [5585] iproto iproto.cc:606 W> stopping input on connection fd 26, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached [014] 2020-06-16 02:24:03.836 [5585] iproto iproto.cc:606 W> stopping input on connection fd 26, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached [014] 2020-06-16 02:24:03.836 [5585] iproto iproto.cc:606 W> stopping input on connection fd 26, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached [014] 2020-06-16 02:24:03.837 [5585] iproto iproto.cc:606 W> stopping input on connection fd 26, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached [014] 2020-06-16 02:24:03.837 [5585] iproto iproto.cc:606 W> stopping input on connection fd 26, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached [014] 2020-06-16 02:24:03.837 [5585] iproto iproto.cc:606 W> stopping input on connection fd 26, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached [014] 2020-06-16 02:24:03.951 [5585] main/121/console/unix/: space.h:336 E> ER_NO_SUCH_INDEX_ID: No index #1 is defined in space '_space' [014] 2020-06-16 02:24:04.180 [5585] main/121/console/unix/: I> set 'readahead' configuration option to 128 [014] 2020-06-16 02:24:04.183 [5585] main/121/console/unix/: I> set 'readahead' configuration option to 102400 [014] 2020-06-16 02:24:04.189 [5585] main/453/console/unix/: I> set 'readahead' configuration option to 16320 Found that the root cause of the issue, was the previously run test 'box/net.box_call_blocks_gh-946.test.lua' on the same worker, in this case the log output mistakenly checked by wait_log/grep_log test_run function, which finds the grepping string in the log of the previous test. To avoid of it the tests can be swapped in worker running queue and in this case both tests pass, check swapped log output: 2020-06-17 10:57:39.881 [69372] main C> entering the event loop 2020-06-17 10:57:39.896 [69372] main/119/console/unix/: I> set 'readahead' configuration option to 128 2020-06-17 10:57:39.898 [69372] main/119/console/unix/: I> set 'readahead' configuration option to 102400 2020-06-17 10:57:40.003 [69372] main/156/console/unix/: I> set 'readahead' configuration option to 16320 2020-06-17 10:57:40.053 [69372] iproto iproto.cc:606 W> stopping input on connection fd 33, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached 2020-06-17 10:57:40.056 [69372] iproto iproto.cc:606 W> stopping input on connection fd 33, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached 2020-06-17 10:57:40.056 [69372] iproto iproto.cc:606 W> stopping input on connection fd 33, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached 2020-06-17 10:57:40.058 [69372] iproto iproto.cc:606 W> stopping input on connection fd 33, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached 2020-06-17 10:57:40.058 [69372] iproto iproto.cc:606 W> stopping input on connection fd 33, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached 2020-06-17 10:57:40.061 [69372] iproto iproto.cc:606 W> stopping input on connection fd 33, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached 2020-06-17 10:57:40.061 [69372] iproto iproto.cc:606 W> stopping input on connection fd 33, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached 2020-06-17 10:57:40.062 [69372] iproto iproto.cc:606 W> stopping input on connection fd 33, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached 2020-06-17 10:57:40.062 [69372] iproto iproto.cc:606 W> stopping input on connection fd 33, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached 2020-06-17 10:57:40.063 [69372] iproto iproto.cc:606 W> stopping input on connection fd 33, aka unix/:(socket), peer of unix/:(socket), readahead limit is reached 2020-06-17 10:57:40.067 [69372] main C> got signal 15 - Terminated Also found that 'readahead' issue from the first test blocks its printing to log file due to suppressed. To fix this issue the default server must be restarted at the very start of the test. Closes #5082
Correct cleanup gitlab-ci for perf jobs
Found that some perf jobs were forgot to be updated with local cleanup routine as was done for the other jobs at commit: 892a188bc56dc7052bcbe082607a83aa73f28c5b "Correct cleanup gitlab-ci" Follows up #5036
build: static build needs more cleanup in sources
Building Tarantool sources on make command run may fail with: [ 10%] make[2]: *** [test/small] Error 1 [ 10%] make[1]: *** [test/CMakeFiles/symlink_small_tests.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs.... The root cause of the issue that Dockerfile.staticbuild uses local copy of sources: COPY . /tarantool Which may have broken links in tests, like: $ ls -al test ... luajit-tap -> /<wrong path>/third_party/luajit/test small -> /<wrong path>/src/lib/small/test/ ... To fix the issue this links should be removed from the docker local copy of sources before build, like: rm -rf test/small test/luajit-tap Closes #5025
decimal: introduce decimal_is_int
This function will be used to determine, whether we can safely convert to integer without an information loss. Needed for #4415
decimal: introduce strtodec function
The behavior is similar with other strto* functions: parse the valid beginning of the string and optionally store the pointer to the first invalid character. Needed for tarantool/tarantool#4415
Branches
Tree
README.md
# Tarantool [![Build Status][travis-badge]][travis-url] [![Build Status][gitlab-ci-badge]][gitlab-ci-url] [![Code Coverage][coverage-badge]][coverage-url] [![Telegram][telegram-badge]][telegram-url] [![Slack][slack-badge]][slack-url] [![Gitter][gitter-badge]][gitter-url] [![Google Groups][groups-badge]][groups-url] https://tarantool.io/en/ Patch submissions and discussion of particular patches https://lists.tarantool.org/mailman/listinfo/tarantool-patches/ General development discussions https://lists.tarantool.org/mailman/listinfo/tarantool-discussions/ Tarantool is an in-memory database and application server. Key features of the application server: * 100% compatible drop-in replacement for Lua 5.1, based on LuaJIT 2.1. Simply use `#!/usr/bin/tarantool` instead of `#!/usr/bin/lua` in your script. * full support for Lua modules and a rich set of own modules, including cooperative multitasking, non-blocking I/O, access to external databases, etc Key features of the database: * ANSI SQL, including views, joins, referential and check constraints * MsgPack data format and MsgPack based client-server protocol * two data engines: 100% in-memory with optional persistence and an own implementation of LSM-tree, to use with large data sets * multiple index types: HASH, TREE, RTREE, BITSET * asynchronous master-master replication * authentication and access control * the database is just a C extension to the application server and can be turned off Supported platforms are Linux/x86, FreeBSD/x86 and OpenBSD/x86, Mac OS X. Tarantool is ideal for data-enriched components of scalable Web architecture: queue servers, caches, stateful Web applications. To download and install Tarantool as a binary package for your OS, please visit https://tarantool.io/en/download/. To build Tarantool from source, see detailed instructions in the Tarantool documentation at https://tarantool.io/en/doc/2.1/dev_guide/building_from_source/. Please report bugs at https://github.com/tarantool/tarantool/issues We also warmly welcome your feedback in the discussion mailing list, tarantool@googlegroups.com. Thank you for your interest in Tarantool! [travis-badge]: https://api.travis-ci.org/tarantool/tarantool.svg?branch=master [travis-url]: https://travis-ci.org/tarantool/tarantool [gitlab-ci-badge]: https://gitlab.com/tarantool/tarantool/badges/master/pipeline.svg [gitlab-ci-url]: https://gitlab.com/tarantool/tarantool/commits/master [coverage-badge]: https://coveralls.io/repos/github/tarantool/tarantool/badge.svg?branch=master [coverage-url]: https://coveralls.io/github/tarantool/tarantool?branch=master [groups-badge]: https://img.shields.io/badge/Google-Groups-orange.svg [groups-url]: https://groups.google.com/forum/#!forum/tarantool [telegram-badge]: https://img.shields.io/badge/Telegram-join%20chat-blue.svg [telegram-url]: http://telegram.me/tarantool [slack-badge]: https://img.shields.io/badge/Slack-join%20chat-lightgrey.svg [slack-url]: http://slack.tarantool.org/ [gitter-badge]: https://badges.gitter.im/Join%20Chat.svg [gitter-url]: https://gitter.im/tarantool/tarantool