- 30 Sep, 2021 1 commit
-
-
Yunier Pérez authored
While the original issue was on Linux, this should work for other platforms as well.
-
- 09 Sep, 2021 1 commit
-
-
yancz2000 authored
Add make test-cluster option
-
- 08 Aug, 2021 1 commit
-
-
Yossi Gottlieb authored
-
- 04 Aug, 2021 1 commit
-
-
Meir Shpilraien (Spielrein) authored
## Current state 1. Lua has its own parser that handles parsing `reds.call` replies and translates them to Lua objects that can be used by the user Lua code. The parser partially handles resp3 (missing big number, verbatim, attribute, ...) 2. Modules have their own parser that handles parsing `RM_Call` replies and translates them to RedisModuleCallReply objects. The parser does not support resp3. In addition, in the future, we want to add Redis Function (#8693) that will probably support more languages. At some point maintaining so many parsers will stop scaling (bug fixes and protocol changes will need to be applied on all of them). We will probably end up with different parsers that support different parts of the resp protocol (like we already have today with Lua and modules) ## PR Changes This PR attempt to unified the reply parsing of Lua and modules (and in the future Redis Function) by introducing a new parser unit (`resp_parser.c`). The new parser handles parsing the reply and calls different callbacks to allow the users (another unit that uses the parser, i.e, Lua, modules, or Redis Function) to analyze the reply. ### Lua API Additions The code that handles reply parsing on `scripting.c` was removed. Instead, it uses the resp_parser to parse and create a Lua object out of the reply. As mentioned above the Lua parser did not handle parsing big numbers, verbatim, and attribute. The new parser can handle those and so Lua also gets it for free. Those are translated to Lua objects in the following way: 1. Big Number - Lua table `{'big_number':'<str representation for big number>'}` 2. Verbatim - Lua table `{'verbatim_string':{'format':'<verbatim format>', 'string':'<verbatim string value>'}}` 3. Attribute - currently ignored and not expose to the Lua parser, another issue will be open to decide how to expose it. Tests were added to check resp3 reply parsing on Lua ### Modules API Additions The reply parsing code on `module.c` was also removed and the new resp_parser is used instead. In addition, the RedisModuleCallReply was also extracted to a separate unit located on `call_reply.c` (in the future, this unit will also be used by Redis Function). A nice side effect of unified parsing is that modules now also support resp3. Resp3 can be enabled by giving `3` as a parameter to the fmt argument of `RM_Call`. It is also possible to give `0`, which will indicate an auto mode. i.e, Redis will automatically chose the reply protocol base on the current client set on the RedisModuleCtx (this mode will mostly be used when the module want to pass the reply to the client as is). In addition, the following RedisModuleAPI were added to allow analyzing resp3 replies: * New RedisModuleCallReply types: * `REDISMODULE_REPLY_MAP` * `REDISMODULE_REPLY_SET` * `REDISMODULE_REPLY_BOOL` * `REDISMODULE_REPLY_DOUBLE` * `REDISMODULE_REPLY_BIG_NUMBER` * `REDISMODULE_REPLY_VERBATIM_STRING` * `REDISMODULE_REPLY_ATTRIBUTE` * New RedisModuleAPI: * `RedisModule_CallReplyDouble` - getting double value from resp3 double reply * `RedisModule_CallReplyBool` - getting boolean value from resp3 boolean reply * `RedisModule_CallReplyBigNumber` - getting big number value from resp3 big number reply * `RedisModule_CallReplyVerbatim` - getting format and value from resp3 verbatim reply * `RedisModule_CallReplySetElement` - getting element from resp3 set reply * `RedisModule_CallReplyMapElement` - getting key and value from resp3 map reply * `RedisModule_CallReplyAttribute` - getting a reply attribute * `RedisModule_CallReplyAttributeElement` - getting key and value from resp3 attribute reply * New context flags: * `REDISMODULE_CTX_FLAGS_RESP3` - indicate that the client is using resp3 Tests were added to check the new RedisModuleAPI ### Modules API Changes * RM_ReplyWithCallReply might return REDISMODULE_ERR if the given CallReply is in resp3 but the client expects resp2. This is not a breaking change because in order to get a resp3 CallReply one needs to specifically specify `3` as a parameter to the fmt argument of `RM_Call` (as mentioned above). Tests were added to check this change ### More small Additions * Added `debug set-disable-deny-scripts` that allows to turn on and off the commands no-script flag protection. This is used by the Lua resp3 tests so it will be possible to run `debug protocol` and check the resp3 parsing code. Co-authored-by:
Oran Agra <oran@redislabs.com> Co-authored-by:
Yossi Gottlieb <yossigo@gmail.com>
-
- 22 Jun, 2021 1 commit
-
-
Evan authored
Previously, passing 0 for newlen would not truncate the string at all. This adds handling of this case, freeing the old string and creating a new empty string. Other changes: - Move `src/modules/testmodule.c` to `tests/modules/basics.c` - Introduce that basic test into the test suite - Add tests to cover StringTruncate - Add `test-modules` build target for the main makefile - Extend `distclean` build target to clean modules too
-
- 16 Jun, 2021 1 commit
-
-
yoav-steinberg authored
Gopher support was added mainly because it was simple (trivial to add). But apparently even something that was trivial at the time, does cause complications down the line when adding more features. We recently ran into a few issues with io-threads conflicting with the gopher support. We had to either complicate the code further in order to solve them, or drop gopher. AFAIK it's completely unused, so we wanna chuck it, rather than keep supporting it.
-
- 26 Apr, 2021 2 commits
-
-
Yossi Gottlieb authored
-
Andy Pan authored
Most of the ae.c backends didn't explicitly handle errors, and instead ignored all errors and did an implicit retry. This is desired for EAGAIN and EINTER, but in case of other systematic errors, we prefer to fail and log the error we got rather than get into a busy loop.
-
- 10 Apr, 2021 1 commit
-
-
yjph authored
-
- 10 Mar, 2021 1 commit
-
-
sundb authored
1. Add `redis-server test all` support to run all tests. 2. Add redis test to daily ci. 3. Add `--accurate` option to run slow tests for more iterations (so that by default we run less cycles (shorter time, and less prints). 4. Move dict benchmark to REDIS_TEST. 5. fix some leaks in tests 6. make quicklist tests run on a specific fill set of options rather than huge ranges 7. move some prints in quicklist test outside their loops to reduce prints 8. removing sds.h from dict.c since it is now used in both redis-server and redis-cli (uses hiredis sds)
-
- 24 Feb, 2021 1 commit
-
-
Yossi Gottlieb authored
-
- 23 Dec, 2020 2 commits
-
-
Greg Femec authored
When a database on a 64 bit build grows past 2^31 keys, the underlying hash table expands to 2^32 buckets. After this point, the algorithms for selecting random elements only return elements from half of the available buckets because they use random() which has a range of 0 to 2^31 - 1. This causes problems for eviction policies which use dictGetSomeKeys or dictGetRandomKey. Over time they cause the hash table to become unbalanced because, while new keys are spread out evenly across all buckets, evictions come from only half of the available buckets. Eventually this half of the table starts to run out of keys and it takes longer and longer to find candidates for eviction. This continues until no more evictions can happen. This solution addresses this by using a 64 bit PRNG instead of libc random(). Co-authored-by:
Greg Femec <gfemec@google.com>
-
Felix Bünemann authored
Homebrew for darwin-arm64 uses /opt/homebrew instead of /usr/local as the prefix, so that it can coexist with darwin-x86_64 using Rosetta 2.
-
- 17 Dec, 2020 1 commit
-
-
Nick Revin authored
-
- 13 Dec, 2020 1 commit
-
-
Yossi Gottlieb authored
* Allow runtest-moduleapi use a different 'make', for systems where GNU Make is 'gmake'. * Fix issue with builds on Solaris re-building everything from scratch due to CFLAGS/LDFLAGS not stored. * Fix compile failure on Solaris due to atomicvar and a bunch of warnings. * Fix garbled log timestamps on Solaris.
-
- 22 Nov, 2020 1 commit
-
-
Yossi Gottlieb authored
When USE_SYSTEMD=yes is specified, try to use pkg-config to determine libsystemd linker flags. If not found, silently fall back to simply using "-lsystemd". We now use a LIBSYSTEMD_LIBS variable so users can explicitly override it and specify their own library. If USE_SYSTEMD is unspecified the old behavior of auto-enabling it if both pkg-config and libsystemd are available is retained.
-
- 28 Oct, 2020 1 commit
-
-
filipe oliveira authored
-
- 26 Oct, 2020 1 commit
-
-
filipe oliveira authored
- add test suite coverage for redis-benchmark - add --version (similar to what redis-cli has) - fix bug sending more requests than intended when pipeline > 1. - when done sending requests, avoid freeing client in the write handler, in theory before responses are received (probably dead code since the read handler will call clientDone first) Co-authored-by:
Oran Agra <oran@redislabs.com>
-
- 01 Oct, 2020 1 commit
-
-
Rafi Einstein authored
-
- 29 Sep, 2020 1 commit
-
-
David CARLIER authored
-
- 28 Sep, 2020 1 commit
-
-
WuYunlong authored
-
- 23 Sep, 2020 1 commit
-
-
David CARLIER authored
mainly backtrace and register dump support.
-
- 21 Sep, 2020 1 commit
-
- 17 Sep, 2020 1 commit
-
-
Wang Yuan authored
Redis 6.0 introduces I/O threads, it is so cool and efficient, we use C11 _Atomic to establish inter-thread synchronization without mutex. But the compiler that must supports C11 _Atomic can compile redis code, that brings a lot of inconvenience since some common platforms can't support by default such as CentOS7, so we want to implement redis atomic type to make it more portable. We have implemented our atomic variable for redis that only has 'relaxed' operations in src/atomicvar.h, so we implement some operations with 'sequentially-consistent', just like the default behavior of C11 _Atomic that can establish inter-thread synchronization. And we replace all uses of C11 _Atomic with redis atomic variable. Our implementation of redis atomic variable uses C11 _Atomic, __atomic or __sync macros if available, it supports most common platforms, and we will detect automatically which feature we use. In Makefile we use a dummy file to detect if the compiler supports C11 _Atomic. Now for gcc, we can compile redis code theoretically if your gcc version is not less than 4.1.2(starts to support __sync_xxx operations). Otherwise, we remove use mutex fallback to implement redis atomic variable for performance and test. You will get compiling errors if your compiler doesn't support all features of above. For cover redis atomic variable tests, we add other CI jobs that build redis on CentOS6 and CentOS7 and workflow daily jobs that run the tests on them. For them, we just install gcc by default in order to cover different compiler versions, gcc is 4.4.7 by default installation on CentOS6 and 4.8.5 on CentOS7. We restore the feature that we can test redis with Helgrind to find data race errors. But you need install Valgrind in the default path configuration firstly before running your tests, since we use macros in helgrind.h to tell Helgrind inter-thread happens-before relationship explicitly for avoiding false positives. Please open an issue on github if you find data race errors relate to this commit. Unrelated: - Fix redefinition of typedef 'RedisModuleUserChangedFunc' For some old version compilers, they will report errors or warnings, if we re-define function type.
-
- 01 Sep, 2020 1 commit
-
-
Yossi Gottlieb authored
All user-supplied variables that affect the build should be explicitly persisted. Fixes #7254
-
- 28 Aug, 2020 1 commit
-
-
Jim Brunner authored
Update adds a general source for retrieving a monotonic time. In addition, AE has been updated to utilize the new monotonic clock for timer processing. This performance improvement is **not** enabled in a default build due to various H/W compatibility concerns, see README.md for details. It does however change the default use of gettimeofday with clock_gettime and somewhat improves performance. This update provides the following 1. An interface for retrieving a monotonic clock. getMonotonicUs returns a uint64_t (aka monotime) with the number of micro-seconds from an arbitrary point. No more messing with tv_sec/tv_usec. Simple routines are provided for measuring elapsed milli-seconds or elapsed micro-seconds (the most common use case for a monotonic timer). No worries about time moving backwards. 2. High-speed assembler implementation for x86 and ARM. The standard method for retrieving the monotonic clock is POSIX.1b (1993): clock_gettime(CLOCK_MONOTONIC, timespec*). However, most modern processors provide a constant speed instruction clock which can be retrieved in a fraction of the time that it takes to call clock_gettime. For x86, this is provided by the RDTSC instruction. For ARM, this is provided by the CNTVCT_EL0 instruction. As a compile-time option, these high-speed timers can be chosen. (Default is POSIX clock_gettime.) 3. Refactor of event loop timers. The timer processing in ae.c has been refactored to use the new monotonic clock interface. This results in simpler/cleaner logic and improved performance.
-
- 25 Aug, 2020 1 commit
-
-
filipe oliveira authored
A first step to enable a consistent full percentile analysis on query latency so that we can fully understand the performance and stability characteristics of the redis-server system we are measuring. It also improves the instantaneous reported metrics, and the csv output format.
-
- 11 Aug, 2020 1 commit
-
-
Mota authored
Obsoletes the need to run `make` before `make test`.
-
- 10 Jul, 2020 1 commit
-
-
James Hilliard authored
-
- 12 May, 2020 1 commit
-
-
David Carlier authored
This platform supports CPU affinity (but not OpenBSD).
-
- 02 May, 2020 1 commit
-
-
zhenwei pi authored
Currently, there are several types of threads/child processes of a redis server. Sometimes we need deeply optimise the performance of redis, so we would like to isolate threads/processes. There were some discussion about cpu affinity cases in the issue: https://github.com/antirez/redis/issues/2863 So implement cpu affinity setting by redis.conf in this patch, then we can config server_cpulist/bio_cpulist/aof_rewrite_cpulist/ bgsave_cpulist by cpu list. Examples of cpulist in redis.conf: server_cpulist 0-7:2 means cpu affinity 0,2,4,6 bio_cpulist 1,3 means cpu affinity 1,3 aof_rewrite_cpulist 8-11 means cpu affinity 8,9,10,11 bgsave_cpulist 1,10-11 means cpu affinity 1,10,11 Test on linux/freebsd, both work fine. Signed-off-by:
zhenwei pi <pizhenwei@bytedance.com>
-
- 25 Apr, 2020 1 commit
-
-
Madelyn Olson authored
-
- 27 Mar, 2020 1 commit
-
-
antirez authored
-
- 12 Mar, 2020 1 commit
-
-
bodong.ybd authored
-
- 01 Jan, 2020 1 commit
-
-
John Sully authored
-
- 29 Nov, 2019 1 commit
-
-
antirez authored
-
- 19 Nov, 2019 1 commit
-
-
Johannes Truschnigg authored
This adds Makefile/build-system support for USE_SYSTEMD=(yes|no|*). This variable's value determines whether or not libsystemd will be linked at build-time. If USE_SYSTEMD is set to "yes", make will use PKG_CONFIG to check for libsystemd's presence, and fail the build early if it isn't installed/detected properly. If USE_SYSTEM is set to "no", libsystemd will *not* be linked, even if support for it is available on the system redis is being built on. For any other value that USE_SYSTEM might assume (e.g. "auto"), PKG_CONFIG will try to determine libsystemd's presence, and set up the build process to link against it, if it was indicated as being installed/available. This approach has a number of repercussions of its own, most importantly the following: If you build redis on a system that actually has systemd support, but no libsystemd-dev package(s) installed, you'll end up *without* support for systemd notification/status reporting support in redis-server. This changes established runtime behaviour. I'm not sure if the build system and/or the server binary should indicate this. I'm also wondering if not actually having systemd-notify-support, but requesting it via the server's config, should result in a fatal error now.
-
- 16 Oct, 2019 1 commit
-
-
Yossi Gottlieb authored
-
- 07 Oct, 2019 2 commits
-
-
Yossi Gottlieb authored
Add configuration options for TLS protocol versions, ciphers/cipher suites selection, etc.
-
Yossi Gottlieb authored
* Introduce a connection abstraction layer for all socket operations and integrate it across the code base. * Provide an optional TLS connections implementation based on OpenSSL. * Pull a newer version of hiredis with TLS support. * Tests, redis-cli updates for TLS support.
-