- Description:
- [Mirror] Middleware for data
- Last Change:
- Clone URL:
Commit Briefs
test/static: add a seed corpus for decimal_to_int64_fuzzer (ligurio/gh-xxxx-iproto-fuzzing-2.10)
NO_DOC=seed corpus NO_CHANGELOG=seed corpus NO_TEST=seed corpus (cherry picked from commit 4894863ec313f62df8a2eac5dd06deaa1beaf305)
test/fuzz: add fuzzing test for decoding decimals
NO_DOC=testing NO_CHANGELOG=testing (cherry picked from commit 4deadeb84e2b3011018eadbbfe2d1e798d95fe5f)
test/static: add a seed corpus for IPROTO decoders
NO_DOC=seed corpus NO_CHANGELOG=seed corpus NO_TEST=seed corpus (cherry picked from commit 4b5fb95396efb3c5f84a3566ae4ba510ac449690)
test/fuzz: add fuzzing tests for IPROTO decoders
Examples of IPROTO decoding issues: #3900, #1928, #6781. Patch adds a number of fuzzing tests that covers IPROTO decoding: - xrow_decode_auth - xrow_decode_begin - xrow_decode_call - xrow_decode_dml - xrow_decode_id - xrow_decode_raft - xrow_decode_sql - xrow_decode_watch - xrow_greeting_decode NO_DOC=testing NO_CHANGELOG=testing (cherry picked from commit 46cacf35bd6454102a6f8e3e47281f66cc3fdd8c)
test/fuzz: collect and print Lua metrics
Fuzzing test for LuaJIT generates random Lua programs and executes them. We want to build a fuzzing test that will produce Lua programs that will not contain semantic errors and will trigger as much as possible components in LuaJIT. This proposed patch introduces metrics that gathered after running the test. LuaJIT metrics gathered using LuaJIT getmetrics module [1]. All gathered metrics test will output after running with a finite number of runs or finite duration of time (options `-runs` and `-max_total_time`) or after sending SIGUSR1 to a test process. ``` $ ./build/test/fuzz/luaL_loadbuffer/luaL_loadbuffer_fuzzer -runs=1000 <snipped> Done 1000 runs in 1 second(s) Total number of samples: 1000 Total number of samples with errors: 438 (43%) Total number of samples with recorded traces: 87 (8%) Total number of samples with snap restores: 30 (3%) Total number of samples with abort traces: 55 (5%) ``` 1. https://www.tarantool.io/en/doc/latest/reference/tooling/luajit_getmetrics/#getmetrics-c-api NO_CHANGELOG=testing NO_DOC=testing (cherry picked from commit 430fa6a2ae173845c4ae78fd07c5801cb261433f)
test/fuzz: refactor LuaJIT fuzzer
This refactoring will: 1. Move macros from a header to the source file. Macros should be used in header only with undef to avoid redefinitions. Undef directive is not useful since we want to use these macros in the source file. 2. Remove `using namespace lua_grammar` from header. https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rs-using-directive 3. Moving serializer entry point and constant parameters into luajit_fuzzer namespace. It's a common practice in C++ to avoid name collisions. 4. Move serializer functions into anonymous namespace. These functions are not a part of the interface so should have static linkage. https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rs-unnamed2 5. Fix ConvertToStringDefault function. It was logically wrong so it would generate an identifier `123` from `*123`. NO_CHANGELOG=internal NO_DOC=fuzzer fix (cherry picked from commit 56488e15cc697164bbad2541f8e9f26f38964200)
test/fuzz: fix luaJIT fuzzer timeout
LuaJIT fuzzer used to stop due to timeout caused by infinite cycles and recursions. Counters were introduced for every cycle and function to address LuaJIT fuzzer timeouts. The idea is to add unique counters for every cycle and function to ensure finite code execution, if it wasn't already. For while, repeat, for cycles, local and global named, anonymous functions, counters will be initialized before the code generated from protobuf, and checked in the first body statement. An entry point for the serializer was created to count cycles and functions for counter initialization. The idea was taken from a paper "Program Reconditioning: Avoiding Undefined Behaviour When Finding and Reducing Compiler Bugs" [1]. Here is an example of a change in serialized code made by this commit. Before: ```lua while (true) do foo = 'bar' end function bar() bar() end ``` After: ```lua counter_0 = 0 counter_1 = 0 while (true) do if counter_0 > 5 then break end counter_0 = counter_0 + 1 foo = 'bar' end function bar() if counter_1 > 5 then return end counter_1 = counter_1 + 1 bar() end ``` Protobuf structures that reproduce the timeout problem were added to the LuaJIT fuzzer corpus. [1] https://www.doc.ic.ac.uk/~afd/homepages/papers/pdfs/2023/PLDI.pdf NO_CHANGELOG=internal NO_DOC=fuzzer fix (cherry picked from commit 4d004bbe24768c6b1afe634d87d9bea8e60b1995)
test/fuzz: add breaks to switch-case
Cases in two switches had no breaks, so they were falling through. Breaks were added to solve the problem. Code generated by the LuaJIT fuzzer became more various. NO_CHANGELOG=internal NO_DOC=fuzzer fix (cherry picked from commit 4430cac9d3b9b612e8b34bf41302d737e0e057af)
test/fuzz: fix datetime_strptime fuzzing test
Function `datetime_strptime` decodes string with datetime according to specified format, it accepts a datetime struct, buffer with datetime and string with format in arguments. Fuzzing test used static string "iso8601" as a format and it blocked fuzzing test to cover functions used by datetime_strptime under the hood. Fuzz introspector shows that code coveraged by a test is quite low. Patch updates the test to make it more effective: buffer with datetime and format string are generated using FDP (Fuzzing Data Provider). Test file extension was changed to .cc, because FuzzingDataProvider is used and we need building it by C++ compiler. Function `tnt_strptime` uses assert, that triggered by fuzzing tests. Therefore it was replaced with to if..then. 1. https://storage.googleapis.com/oss-fuzz-introspector/tarantool/ Fixes #8490 NO_CHANGELOG=fuzzing test NO_DOC=fuzzing test NO_TEST=fuzzing test (cherry picked from commit a1bd6e0b799ab4b2cbe070917f6556f6239681f6)
datetime: fix buffer overflow in tnt_strptime
Fixes #8502 Needed for #8490 NO_DOC=bugfix NO_TEST=covered by fuzzing test (cherry picked from commit 783a70406f79b7cc59bfe65a21a11fa35206fb66)
Branches
Tree
README.md
# Tarantool [![Actions Status][actions-badge]][actions-url] [![Code Coverage][coverage-badge]][coverage-url] [![Telegram][telegram-badge]][telegram-url] [![GitHub Discussions][discussions-badge]][discussions-url] [![Stack Overflow][stackoverflow-badge]][stackoverflow-url] [Tarantool][tarantool-url] is an in-memory computing platform consisting of a database and an application server. It is distributed under [BSD 2-Clause][license] terms. Key features of the application server: * Heavily optimized Lua interpreter with incredibly fast tracing JIT compiler, based on LuaJIT 2.1. * Cooperative multitasking, non-blocking IO. * [Persistent queues][queue]. * [Sharding][vshard]. * [Cluster and application management framework][cartridge]. * Access to external databases such as [MySQL][mysql] and [PostgreSQL][pg]. * A rich set of built-in and standalone [modules][modules]. Key features of the database: * MessagePack data format and MessagePack based client-server protocol. * Two data engines: 100% in-memory with complete WAL-based persistence and an own implementation of LSM-tree, to use with large data sets. * Multiple index types: HASH, TREE, RTREE, BITSET. * Document oriented JSON path indexes. * Asynchronous master-master replication. * Synchronous quorum-based replication. * RAFT-based automatic leader election for the single-leader configuration. * Authentication and access control. * ANSI SQL, including views, joins, referential and check constraints. * [Connectors][connectors] for many programming languages. * The database is a C extension of the application server and can be turned off. Supported platforms are Linux (x86_64, aarch64), Mac OS X (x86_64, M1), FreeBSD (x86_64). Tarantool is ideal for data-enriched components of scalable Web architecture: queue servers, caches, stateful Web applications. To download and install Tarantool as a binary package for your OS or using Docker, please see the [download instructions][download]. To build Tarantool from source, see detailed [instructions][building] in the Tarantool documentation. To find modules, connectors and tools for Tarantool, check out our [Awesome Tarantool][awesome-list] list. Please report bugs to our [issue tracker][issue-tracker]. We also warmly welcome your feedback on the [discussions][discussions-url] page and questions on [Stack Overflow][stackoverflow-url]. We accept contributions via pull requests. Check out our [How to get involved][get-involved] guide. Thank you for your interest in Tarantool! [actions-badge]: https://github.com/tarantool/tarantool/workflows/release/badge.svg [actions-url]: https://github.com/tarantool/tarantool/actions [coverage-badge]: https://coveralls.io/repos/github/tarantool/tarantool/badge.svg?branch=master [coverage-url]: https://coveralls.io/github/tarantool/tarantool?branch=master [telegram-badge]: https://img.shields.io/badge/Telegram-join%20chat-blue.svg [telegram-url]: http://telegram.me/tarantool [discussions-badge]: https://img.shields.io/github/discussions/tarantool/tarantool [discussions-url]: https://github.com/tarantool/tarantool/discussions [stackoverflow-badge]: https://img.shields.io/badge/stackoverflow-tarantool-orange.svg [stackoverflow-url]: https://stackoverflow.com/questions/tagged/tarantool [tarantool-url]: https://www.tarantool.io/en/ [license]: LICENSE [modules]: https://www.tarantool.io/en/download/rocks [queue]: https://github.com/tarantool/queue [vshard]: https://github.com/tarantool/vshard [cartridge]: https://github.com/tarantool/cartridge [mysql]: https://github.com/tarantool/mysql [pg]: https://github.com/tarantool/pg [connectors]: https://www.tarantool.io/en/download/connectors [download]: https://www.tarantool.io/en/download/ [building]: https://www.tarantool.io/en/doc/latest/dev_guide/building_from_source/ [issue-tracker]: https://github.com/tarantool/tarantool/issues [get-involved]: https://www.tarantool.io/en/doc/latest/contributing/contributing/ [awesome-list]: https://github.com/tarantool/awesome-tarantool/