Commit Briefs

2af6721e50 Sergey Bronnikov

test/static: add a seed corpus for decimal_to_int64_fuzzer (ligurio/gh-xxxx-iproto-fuzzing-2.10)

NO_DOC=seed corpus NO_CHANGELOG=seed corpus NO_TEST=seed corpus (cherry picked from commit 4894863ec313f62df8a2eac5dd06deaa1beaf305)


1b96f873ed Sergey Bronnikov

test/fuzz: add fuzzing test for decoding decimals

NO_DOC=testing NO_CHANGELOG=testing (cherry picked from commit 4deadeb84e2b3011018eadbbfe2d1e798d95fe5f)


4435f091dd Sergey Bronnikov

test/static: add a seed corpus for IPROTO decoders

NO_DOC=seed corpus NO_CHANGELOG=seed corpus NO_TEST=seed corpus (cherry picked from commit 4b5fb95396efb3c5f84a3566ae4ba510ac449690)


f269bb52f6 Sergey Bronnikov

test/fuzz: add fuzzing tests for IPROTO decoders

Examples of IPROTO decoding issues: #3900, #1928, #6781. Patch adds a number of fuzzing tests that covers IPROTO decoding: - xrow_decode_auth - xrow_decode_begin - xrow_decode_call - xrow_decode_dml - xrow_decode_id - xrow_decode_raft - xrow_decode_sql - xrow_decode_watch - xrow_greeting_decode NO_DOC=testing NO_CHANGELOG=testing (cherry picked from commit 46cacf35bd6454102a6f8e3e47281f66cc3fdd8c)


14a9269e7d Sergey Bronnikov

test/fuzz: collect and print Lua metrics

Fuzzing test for LuaJIT generates random Lua programs and executes them. We want to build a fuzzing test that will produce Lua programs that will not contain semantic errors and will trigger as much as possible components in LuaJIT. This proposed patch introduces metrics that gathered after running the test. LuaJIT metrics gathered using LuaJIT getmetrics module [1]. All gathered metrics test will output after running with a finite number of runs or finite duration of time (options `-runs` and `-max_total_time`) or after sending SIGUSR1 to a test process. ``` $ ./build/test/fuzz/luaL_loadbuffer/luaL_loadbuffer_fuzzer -runs=1000 <snipped> Done 1000 runs in 1 second(s) Total number of samples: 1000 Total number of samples with errors: 438 (43%) Total number of samples with recorded traces: 87 (8%) Total number of samples with snap restores: 30 (3%) Total number of samples with abort traces: 55 (5%) ``` 1. https://www.tarantool.io/en/doc/latest/reference/tooling/luajit_getmetrics/#getmetrics-c-api NO_CHANGELOG=testing NO_DOC=testing (cherry picked from commit 430fa6a2ae173845c4ae78fd07c5801cb261433f)


877a42c7ca Sergey Bronnikov

test/fuzz: refactor LuaJIT fuzzer

This refactoring will: 1. Move macros from a header to the source file. Macros should be used in header only with undef to avoid redefinitions. Undef directive is not useful since we want to use these macros in the source file. 2. Remove `using namespace lua_grammar` from header. https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rs-using-directive 3. Moving serializer entry point and constant parameters into luajit_fuzzer namespace. It's a common practice in C++ to avoid name collisions. 4. Move serializer functions into anonymous namespace. These functions are not a part of the interface so should have static linkage. https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rs-unnamed2 5. Fix ConvertToStringDefault function. It was logically wrong so it would generate an identifier `123` from `*123`. NO_CHANGELOG=internal NO_DOC=fuzzer fix (cherry picked from commit 56488e15cc697164bbad2541f8e9f26f38964200)


14eac8a511 Sergey Bronnikov

test/fuzz: fix luaJIT fuzzer timeout

LuaJIT fuzzer used to stop due to timeout caused by infinite cycles and recursions. Counters were introduced for every cycle and function to address LuaJIT fuzzer timeouts. The idea is to add unique counters for every cycle and function to ensure finite code execution, if it wasn't already. For while, repeat, for cycles, local and global named, anonymous functions, counters will be initialized before the code generated from protobuf, and checked in the first body statement. An entry point for the serializer was created to count cycles and functions for counter initialization. The idea was taken from a paper "Program Reconditioning: Avoiding Undefined Behaviour When Finding and Reducing Compiler Bugs" [1]. Here is an example of a change in serialized code made by this commit. Before: ```lua while (true) do foo = 'bar' end function bar() bar() end ``` After: ```lua counter_0 = 0 counter_1 = 0 while (true) do if counter_0 > 5 then break end counter_0 = counter_0 + 1 foo = 'bar' end function bar() if counter_1 > 5 then return end counter_1 = counter_1 + 1 bar() end ``` Protobuf structures that reproduce the timeout problem were added to the LuaJIT fuzzer corpus. [1] https://www.doc.ic.ac.uk/~afd/homepages/papers/pdfs/2023/PLDI.pdf NO_CHANGELOG=internal NO_DOC=fuzzer fix (cherry picked from commit 4d004bbe24768c6b1afe634d87d9bea8e60b1995)


dddd9eb492 Sergey Bronnikov

test/fuzz: add breaks to switch-case

Cases in two switches had no breaks, so they were falling through. Breaks were added to solve the problem. Code generated by the LuaJIT fuzzer became more various. NO_CHANGELOG=internal NO_DOC=fuzzer fix (cherry picked from commit 4430cac9d3b9b612e8b34bf41302d737e0e057af)


2d81934784 Sergey Bronnikov

test/fuzz: fix datetime_strptime fuzzing test

Function `datetime_strptime` decodes string with datetime according to specified format, it accepts a datetime struct, buffer with datetime and string with format in arguments. Fuzzing test used static string "iso8601" as a format and it blocked fuzzing test to cover functions used by datetime_strptime under the hood. Fuzz introspector shows that code coveraged by a test is quite low. Patch updates the test to make it more effective: buffer with datetime and format string are generated using FDP (Fuzzing Data Provider). Test file extension was changed to .cc, because FuzzingDataProvider is used and we need building it by C++ compiler. Function `tnt_strptime` uses assert, that triggered by fuzzing tests. Therefore it was replaced with to if..then. 1. https://storage.googleapis.com/oss-fuzz-introspector/tarantool/ Fixes #8490 NO_CHANGELOG=fuzzing test NO_DOC=fuzzing test NO_TEST=fuzzing test (cherry picked from commit a1bd6e0b799ab4b2cbe070917f6556f6239681f6)


5ec9aab8db Sergey Bronnikov

datetime: fix buffer overflow in tnt_strptime

Fixes #8502 Needed for #8490 NO_DOC=bugfix NO_TEST=covered by fuzzing test (cherry picked from commit 783a70406f79b7cc59bfe65a21a11fa35206fb66)


Branches



























































































Tags

Tree

.editorconfigcommits | blame
.gdbinitcommits | blame
.gitattributescommits | blame
.github/
.gitignorecommits | blame
.gitmodulescommits | blame
.luacheckrccommits | blame
.pack.mkcommits | blame
.test.mkcommits | blame
AUTHORScommits | blame
CMakeLists.txtcommits | blame
Doxyfilecommits | blame
Doxyfile.API.incommits | blame
FreeBSD/
LICENSEcommits | blame
README.FreeBSDcommits | blame
README.MacOSXcommits | blame
README.OpenBSDcommits | blame
README.mdcommits | blame
TODOcommits | blame
apk/
asan/
changelogs/
cmake/
debian/
doc/
extra/
perf/
rpm/
rump/
src/
static-build/
test/
test-run$commits | blame
third_party/
tools/

README.md

# Tarantool

[![Actions Status][actions-badge]][actions-url]
[![Code Coverage][coverage-badge]][coverage-url]
[![Telegram][telegram-badge]][telegram-url]
[![GitHub Discussions][discussions-badge]][discussions-url]
[![Stack Overflow][stackoverflow-badge]][stackoverflow-url]

[Tarantool][tarantool-url] is an in-memory computing platform consisting of a
database and an application server.

It is distributed under [BSD 2-Clause][license] terms.

Key features of the application server:

* Heavily optimized Lua interpreter with incredibly fast tracing JIT compiler,
  based on LuaJIT 2.1.
* Cooperative multitasking, non-blocking IO.
* [Persistent queues][queue].
* [Sharding][vshard].
* [Cluster and application management framework][cartridge].
* Access to external databases such as [MySQL][mysql] and [PostgreSQL][pg].
* A rich set of built-in and standalone [modules][modules].

Key features of the database:

* MessagePack data format and MessagePack based client-server protocol.
* Two data engines: 100% in-memory with complete WAL-based persistence and an
  own implementation of LSM-tree, to use with large data sets.
* Multiple index types: HASH, TREE, RTREE, BITSET.
* Document oriented JSON path indexes.
* Asynchronous master-master replication.
* Synchronous quorum-based replication.
* RAFT-based automatic leader election for the single-leader configuration.
* Authentication and access control.
* ANSI SQL, including views, joins, referential and check constraints.
* [Connectors][connectors] for many programming languages.
* The database is a C extension of the application server and can be turned
  off.

Supported platforms are Linux (x86_64, aarch64), Mac OS X (x86_64, M1), FreeBSD
(x86_64).

Tarantool is ideal for data-enriched components of scalable Web architecture:
queue servers, caches, stateful Web applications.

To download and install Tarantool as a binary package for your OS or using
Docker, please see the [download instructions][download].

To build Tarantool from source, see detailed [instructions][building] in the
Tarantool documentation.

To find modules, connectors and tools for Tarantool, check out our [Awesome
Tarantool][awesome-list] list.

Please report bugs to our [issue tracker][issue-tracker]. We also warmly
welcome your feedback on the [discussions][discussions-url] page and questions
on [Stack Overflow][stackoverflow-url].

We accept contributions via pull requests. Check out our [How to get
involved][get-involved] guide.

Thank you for your interest in Tarantool!

[actions-badge]: https://github.com/tarantool/tarantool/workflows/release/badge.svg
[actions-url]: https://github.com/tarantool/tarantool/actions
[coverage-badge]: https://coveralls.io/repos/github/tarantool/tarantool/badge.svg?branch=master
[coverage-url]: https://coveralls.io/github/tarantool/tarantool?branch=master
[telegram-badge]: https://img.shields.io/badge/Telegram-join%20chat-blue.svg
[telegram-url]: http://telegram.me/tarantool
[discussions-badge]: https://img.shields.io/github/discussions/tarantool/tarantool
[discussions-url]: https://github.com/tarantool/tarantool/discussions
[stackoverflow-badge]: https://img.shields.io/badge/stackoverflow-tarantool-orange.svg
[stackoverflow-url]: https://stackoverflow.com/questions/tagged/tarantool
[tarantool-url]: https://www.tarantool.io/en/
[license]: LICENSE
[modules]: https://www.tarantool.io/en/download/rocks
[queue]: https://github.com/tarantool/queue
[vshard]: https://github.com/tarantool/vshard
[cartridge]: https://github.com/tarantool/cartridge
[mysql]: https://github.com/tarantool/mysql
[pg]: https://github.com/tarantool/pg
[connectors]: https://www.tarantool.io/en/download/connectors
[download]: https://www.tarantool.io/en/download/
[building]: https://www.tarantool.io/en/doc/latest/dev_guide/building_from_source/
[issue-tracker]: https://github.com/tarantool/tarantool/issues
[get-involved]: https://www.tarantool.io/en/doc/latest/contributing/contributing/
[awesome-list]: https://github.com/tarantool/awesome-tarantool/