- 28 May, 2023 2 commits
-
-
Oran Agra authored
So far clients being blocked and unblocked by a module command would update the c->woff variable and so WAIT was ineffective and got released without waiting for the command actions to propagate. This seems to have existed since forever, but not for RM_BlockClientOnKeys. It is problematic though to know if the module did or didn't propagate anything in that command, so for now, instead of adding an API, we'll just update the woff to the latest offset when unblocking, this will cause the client to possibly wait excessively, but that's not that bad.
-
Wen Hui authored
XREAD only supports a special ID of $ and XREADGROUP only supports ^. make sure not to suggest the wrong one when rerunning an error about unbalanced ID arguments Co-authored-by:
Oran Agra <oran@redislabs.com>
-
- 24 May, 2023 4 commits
-
-
judeng authored
postpone the initialization of oject's lru&lfu until it is added to the db as a value object (#11626) This pr can get two performance benefits: 1. Stop redundant initialization when most robj objects are created 2. LRU_CLOCK will no longer be called in io threads, so we can avoid the `atomicGet` Another code optimization: deleted the redundant judgment in dbSetValue, no matter in LFU or LRU, the lru field inold robj is always the freshest (it is always updated in lookupkey), so we don't need to judge if in LFU
-
Wen Hui authored
HVALS, HKEYS and HEXISTS commands wrong type test cases were not covered so added the test cases.
-
Binbin authored
We add a new loglevel 'nothing' to disable logging in #12133. This PR syncs that config change to sentinel. Because in #11214 we support modifying loglevel in runtime. Although I think sentinel doesn't need this nothing config, it's better to be consistent.
-
Binbin authored
In #12166, we removed a call to CLUSTER SLAVES, which then caused reply-schemas ci to fail: ``` WARNING! The following commands were not hit at all: cluster|slaves ERROR! at least one command was not hit by the tests ``` Because we already have command output that cover CLUSTER REPLICAS elsewhere, here we simply add some dummy tests to fix the ci.
-
- 23 May, 2023 1 commit
-
-
Ping Xie authored
This commit excludes aux fields from the output of the `cluster nodes` and `cluster replicas` command. We may decide to re-introduce them in some form or another in the future, but not in v7.2.
-
- 22 May, 2023 1 commit
-
-
binfeng-xin authored
A single SPOP with command with count argument resulted in many SPOP commands being propagated to the replica. This is inefficient because the key name is repeated many times, and is also being looked-up many times. also it results in high QPS metrics on the replica. To solve that, we flush batches of 1024 fields per SPOP command. Co-authored-by:
zhaozhao.zz <zhaozhao.zz@alibaba-inc.com>
-
- 18 May, 2023 2 commits
-
-
Binbin authored
Performance improvement to ZADD and ZRANGESTORE, convert to skiplist and expand dict in advance (#12185) For zsets that will eventually be stored as the skiplist encoding (has a dict), we can convert it to skiplist ahead of time. This change checks the number of arguments in the ZADD command, and converts the data-structure if the number of new entries exceeds the listpack-max-entries configuration. This can cause us to over-allocate memory if there are duplicate entries in the input, which is unexpected. For ZRANGESTORE, we know the size of the zset, so we can expand the dict in advance, to avoid the temporary dict from being rehashed while it grows. Simple benchmarks shows it provides some 4% improvement in ZADD and 20% in ZRANGESTORE
-
Hanna Fadida authored
Current tests for BITFIELD_RO command are skipped in the external mode, and therefore reply-schemas-validator reports a coverage error. This PR adds basic tests to increase coverage.
-
- 17 May, 2023 1 commit
-
-
Wen Hui authored
Extend SENTINEL CONFIG SET and SENTINEL CONFIG GET to be compatible with variadic CONFIG SET and CONFIG GET and allow multiple parameters to be modified in a single call atomically. Co-authored-by:
Oran Agra <oran@redislabs.com>
-
- 16 May, 2023 2 commits
-
-
Binbin authored
In the judgment in setTypeCreate, we should judge size_hint <= max_entries. This results in the following inconsistencies: ``` 127.0.0.1:6379> config set set-max-intset-entries 5 set-max-listpack-entries 5 OK 127.0.0.1:6379> sadd intset_set1 1 2 3 4 5 (integer) 5 127.0.0.1:6379> object encoding intset_set1 "hashtable" 127.0.0.1:6379> sadd intset_set2 1 2 3 4 (integer) 4 127.0.0.1:6379> sadd intset_set2 5 (integer) 1 127.0.0.1:6379> object encoding intset_set2 "intset" 127.0.0.1:6379> sadd listpack_set1 a 1 2 3 4 (integer) 5 127.0.0.1:6379> object encoding listpack_set1 "hashtable" 127.0.0.1:6379> sadd listpack_set2 a 1 2 3 (integer) 4 127.0.0.1:6379> sadd listpack_set2 4 (integer) 1 127.0.0.1:6379> object encoding listpack_set2 "listpack" ``` This was introduced in #12019, added corresponding tests.
-
Wen Hui authored
Minor missing test case addition. SMEMBERS SCARD against non set SMEMBERS SCARD against non existing key
-
- 14 May, 2023 2 commits
- 12 May, 2023 1 commit
-
-
Chen Tianjie authored
The measured latency(duration) includes the list below, which can be shown by `INFO STATS`. ``` eventloop_cycles // ever increasing counter eventloop_duration_sum // cumulative duration of eventloop in microseconds eventloop_duration_cmd_sum // cumulative duration of executing commands in microseconds instantaneous_eventloop_cycles_per_sec // average eventloop count per second in recent 1.6s instantaneous_eventloop_duration_usec // average single eventloop duration in recent 1.6s ``` Also added some experimental metrics, which are shown only when `INFO DEBUG` is called. This section isn't included in the default INFO, or even in `INFO ALL` and the fields in this section can change in the future without considering backwards compatibility. ``` eventloop_duration_aof_sum // cumulative duration of writing AOF eventloop_duration_cron_sum // cumulative duration cron jobs (serverCron, beforeSleep excluding IO and AOF) eventloop_cmd_per_cycle_max // max number of commands executed in one eventloop eventloop_duration_max // max duration of one eventloop ``` All of these are being reset by CONFIG RESETSTAT
-
- 09 May, 2023 2 commits
-
-
Leibale Eidelman authored
in GEO commands, `STORE` and `STOREDIST` are mutually exclusive. use `oneof` block to contain them
-
Binbin authored
The test failed on MacOS: ``` *** [err]: EXPIRE precision is now the millisecond in tests/unit/expire.tcl Expected 'somevalue {}' to equal or match '{} {}' ``` `set a [r get x]`, even though we tried 10 times, sometimes we still get {}, this is a time-sensitive test. In this PR, we add the following changes: 1. More attempts, change it from 10 to 30. 2. More tolerant, change the `after 900` to `after 800`. In addition, we judging $a in advance and changing `after 1100` to `after 300`, this will save us some times.
-
- 08 May, 2023 1 commit
-
-
cui fliter authored
Remove several instances of duplicate "the" in comments
-
- 07 May, 2023 2 commits
-
-
Wen Hui authored
adding test case of expired key or not exist for GET and GETEX. for better test coverage.
-
sundb authored
When `RM_ZsetAdd()`/`RM_ZsetIncrby()`/`RM_StreamAdd()` fails, if a new key happens to be created using `moduleCreateEmptyKey()`, we should clean up the empty key. ## Test 1) Add new module commands(`zset.add` and `zset.incrby`) to cover `RM_ZsetAdd()`/`RM_ZsetIncrby()`. 2) Add a large-memory test to cover `RM_StreamAdd()`.
-
- 06 May, 2023 1 commit
-
-
zhaozhao.zz authored
When master reboot from RDB, if rsi in RDB is valid we should not free replication backlog, even if master_repl_offset or repl-offset is 0. Since if master doesn't send any data to replicas master_repl_offset is 0, it's a valid number. A clear example: 1. start a master and apply some write commands, the master's master_repl_offset is 0 since it has no replicas. 2. stop write commands on master, and start another instance and replicaof the master, trigger an FULLRESYNC 3. the master's master_repl_offset is still 0 (set a large number for repl-ping-replica-period), do BGSAVE and restart the master 4. master load master_repl_offset from RDB's rsi and it's still 0, and we should make sure replica can partially resync with master.
-
- 04 May, 2023 2 commits
-
-
Binbin authored
Tests occasionally fail since #12000: ``` *** [err]: query buffer resized correctly when not idle in tests/unit/querybuf.tcl Expected 0 > 32768 (context: type eval line 11 cmd {assert {$orig_test_client_qbuf > 32768}} proc ::test) *** [err]: query buffer resized correctly with fat argv in tests/unit/querybuf.tcl query buffer should not be resized when client idle time smaller than 2s ``` The reason may be because we set hz to 100, querybuf shrinks before we count client_query_buffer. We avoid this problem by setting pause-cron to 1.
-
guybe7 authored
1. reset the readraw mode after a test that uses it. undetected since the only test after that on the same server didn't read any replies. 2. fix a cross slot issue that was undetected in cluster mode because readraw doesn't throw exceptions on errors.
-
- 03 May, 2023 1 commit
-
-
Madelyn Olson authored
Technically declaring a prototype with an empty declaration has been deprecated since the early days of C, but we never got a warning for it. C2x will apparently be introducing a breaking change if you are using this type of declarator, so Clang 15 has started issuing a warning with -pedantic. Although not apparently a problem for any of the compiler we build on, if feels like the right thing is to properly adhere to the C standard and use (void).
-
- 01 May, 2023 1 commit
-
-
Wen Hui authored
Adding missing test case against wrong type for HRANDFIELD HGET HGETALL HDEL HINCRBY HINCRBYFLOAT HSTRLEN.
-
- 27 Apr, 2023 1 commit
-
-
Binbin authored
The change in #12018 break the CI (fixed by #12083). There are quite a few sentinel commands that are missing both test coverage and also schema. PR added reply-schema to the following commands: - sentinel debug - sentinel info-cache - sentinel pendding-scripts - sentinel reset - sentinel simulate-failure Added some very basic tests for other sentinel commands, just so that they have some coverage. - sentinel help - sentinel masters - sentinel myid - sentinel sentinels - sentinel slaves These tests should be improved / replaced in a followup PR.
-
- 25 Apr, 2023 1 commit
-
-
judeng authored
We can always know the array length of the response, so there is no need to use addReplyDeferredLen which may introduce some additional overheads.
-
- 19 Apr, 2023 1 commit
-
-
Wen Hui authored
Minor test case addition for DECR and DECRBY. Currently DECR and DECRBY do not have test case coverage for the scenarios where they run on a non-existing key.
-
- 18 Apr, 2023 3 commits
-
-
Binbin authored
In order to speed up tests, avoid saving an RDB (mostly notable on shutdown), except for tests that explicitly test the RDB mechanism In addition, use `shutdown-on-sigterm force` to prevetn shutdown from failing in case the server is in the middle of the initial AOFRW Also a a test that checks that the `shutdown-on-sigterm default` is to refuse shutdown if there's an initial AOFRW Co-authored-by:
Guy Benoish <guy.benoish@redislabs.com>
-
sundb authored
This PR is to fix the compilation warnings and errors generated by the latest complier toolchain, and to add a new runner of the latest toolchain for daily CI. ## Fix various compilation warnings and errors 1) jemalloc.c COMPILER: clang-14 with FORTIFY_SOURCE WARNING: ``` src/jemalloc.c:1028:7: warning: suspicious concatenation of string literals in an array initialization; did you mean to separate the elements with a comma? [-Wstring-concatenation] "/etc/malloc.conf", ^ src/jemalloc.c:1027:3: note: place parentheses around the string literal to silence warning "\"name\" of the file referenced by the symbolic link named " ^ ``` REASON: the compiler to alert developers to potential issues with string concatenation that may miss a comma, just like #9534 which misses a comma. SOLUTION: use `()` to tell the compiler that these two line strings are continuous. 2) config.h COMPILER: clang-14 with FORTIFY_SOURCE WARNING: ``` In file included from quicklist.c:36: ./config.h:319:76: warning: attribute declaration must precede definition [-Wignored-attributes] char *strcat(char *restrict dest, const char *restrict src) __attribute__((deprecated("please avoid use of unsafe C functions. prefer use of redis_strlcat instead"))); ``` REASON: Enabling _FORTIFY_SOURCE will cause the compiler to use `strcpy()` with check, it results in a deprecated attribute declaration after including <features.h>. SOLUTION: move the deprecated attribute declaration from config.h to fmacro.h before "#include <features.h>". 3) networking.c COMPILER: GCC-12 WARNING: ``` networking.c: In function ‘addReplyDouble.part.0’: networking.c:876:21: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=] 876 | dbuf[start] = '$'; | ^ networking.c:868:14: note: at offset -5 into destination object ‘dbuf’ of size 5152 868 | char dbuf[MAX_LONG_DOUBLE_CHARS+32]; | ^ networking.c:876:21: warning: writing 1 byte into a region of size 0 [-Wstringop-overflow=] 876 | dbuf[start] = '$'; | ^ networking.c:868:14: note: at offset -6 into destination object ‘dbuf’ of size 5152 868 | char dbuf[MAX_LONG_DOUBLE_CHARS+32]; ``` REASON: GCC-12 predicts that digits10() may return 9 or 10 through `return 9 + (v >= 1000000000UL)`. SOLUTION: add an assert to let the compiler know the possible length; 4) redis-cli.c & redis-benchmark.c COMPILER: clang-14 with FORTIFY_SOURCE WARNING: ``` redis-benchmark.c:1621:2: warning: embedding a directive within macro arguments has undefined behavior [-Wembedded-directive] #ifdef USE_OPENSSL redis-cli.c:3015:2: warning: embedding a directive within macro arguments has undefined behavior [-Wembedded-directive] #ifdef USE_OPENSSL ``` REASON: when _FORTIFY_SOURCE is enabled, the compiler will use the print() with check, which is a macro. this may result in the use of directives within the macro, which is undefined behavior. SOLUTION: move the directives-related code out of `print()`. 5) server.c COMPILER: gcc-13 with FORTIFY_SOURCE WARNING: ``` In function 'lookupCommandLogic', inlined from 'lookupCommandBySdsLogic' at server.c:3139:32: server.c:3102:66: error: '*(robj **)argv' may be used uninitialized [-Werror=maybe-uninitialized] 3102 | struct redisCommand *base_cmd = dictFetchValue(commands, argv[0]->ptr); | ~~~~^~~ ``` REASON: The compiler thinks that the `argc` returned by `sdssplitlen()` could be 0, resulting in an empty array of size 0 being passed to lookupCommandLogic. this should be a false positive, `argc` can't be 0 when strings are not NULL. SOLUTION: add an assert to let the compiler know that `argc` is positive. 6) sha1.c COMPILER: gcc-12 WARNING: ``` In function ‘SHA1Update’, inlined from ‘SHA1Final’ at sha1.c:195:5: sha1.c:152:13: warning: ‘SHA1Transform’ reading 64 bytes from a region of size 0 [-Wstringop-overread] 152 | SHA1Transform(context->state, &data[i]); | ^ sha1.c:152:13: note: referencing argument 2 of type ‘const unsigned char[64]’ sha1.c: In function ‘SHA1Final’: sha1.c:56:6: note: in a call to function ‘SHA1Transform’ 56 | void SHA1Transform(uint32_t state[5], const unsigned char buffer[64]) | ^ In function ‘SHA1Update’, inlined from ‘SHA1Final’ at sha1.c:198:9: sha1.c:152:13: warning: ‘SHA1Transform’ reading 64 bytes from a region of size 0 [-Wstringop-overread] 152 | SHA1Transform(context->state, &data[i]); | ^ sha1.c:152:13: note: referencing argument 2 of type ‘const unsigned char[64]’ sha1.c: In function ‘SHA1Final’: sha1.c:56:6: note: in a call to function ‘SHA1Transform’ 56 | void SHA1Transform(uint32_t state[5], const unsigned char buffer[64]) ``` REASON: due to the bug[https://gcc.gnu.org/bugzilla/show_bug.cgi?id=80922], when enable LTO, gcc-12 will not see `diagnostic ignored "-Wstringop-overread"`, resulting in a warning. SOLUTION: temporarily set SHA1Update to noinline to avoid compiler warnings due to LTO being enabled until the above gcc bug is fixed. 7) zmalloc.h COMPILER: GCC-12 WARNING: ``` In function ‘memset’, inlined from ‘moduleCreateContext’ at module.c:877:5, inlined from ‘RM_GetDetachedThreadSafeContext’ at module.c:8410:5: /usr/include/x86_64-linux-gnu/bits/string_fortified.h:59:10: warning: ‘__builtin_memset’ writing 104 bytes into a region of size 0 overflows the destination [-Wstringop-overflow=] 59 | return __builtin___memset_chk (__dest, __ch, __len, ``` REASON: due to the GCC-12 bug [https://gcc.gnu.org/bugzilla/show_bug.cgi?id=96503], GCC-12 cannot see alloc_size, which causes GCC to think that the actual size of memory is 0 when checking with __glibc_objsize0(). SOLUTION: temporarily set malloc-related interfaces to `noinline` to avoid compiler warnings due to LTO being enabled until the above gcc bug is fixed. ## Other changes 1) Fixed `ps -p [pid]` doesn't output `<defunct>` when using procps 4.x causing `replication child dies when parent is killed - diskless` test to fail. 2) Add a new fortify CI with GCC-13 and ubuntu-lunar docker image.
-
Wen Hui authored
Minor test case addition. Currently GETRANGE command does not have the test case coverage for the scenarios: An error is returned when key exists but of different type Added missing test cases for getrange command.
-
- 16 Apr, 2023 1 commit
-
-
judeng authored
this pr fix two wrongs: 1. When client’s querybuf is pre-allocated for a fat argv, we need to update the querybuf_peak of the client immediately to completely avoid the unexpected shrinking of querybuf in the next clientCron (before data arrives to set the peak). 2. the protocol's bulklen does not include `\r\n`, but the allocation and the data we read does. so in `clientsCronResizeQueryBuffer`, the `resize` or `querybuf_peak` should add these 2 bytes. the first bug is likely to hit us on large payloads over slow connections, in which case transferring the payload can take longer and a cron event will be triggered (specifically if there are not a lot of clients)
-
- 13 Apr, 2023 2 commits
-
-
Wen Hui authored
There is are some missing test cases for SUBSTR command. These might already be covered by GETRANGE, but no harm in adding them since they're simple. Added 3 test case. * start > stop * start and stop both greater than string length * when no key is present.
-
Wen Hui authored
Currently LINSERT command does not have the test case coverage for following scenarios. 1. When key does not exist, it is considered an empty list and no operation is performed. 2. An error is returned when key exists but does not hold a list value. Added above two missing test cases for linsert command.
-
- 12 Apr, 2023 2 commits
-
-
Binbin authored
* Add RM_ReplyWithErrorFormat that can support format Reply with the error create from a printf format and arguments. If the error code is already passed in the string 'fmt', the error code provided is used, otherwise the string "-ERR " for the generic error code is automatically added. The usage is, for example: RedisModule_ReplyWithErrorFormat(ctx, "An error: %s", "foo"); RedisModule_ReplyWithErrorFormat(ctx, "-WRONGTYPE Wrong Type: %s", "foo"); The function always returns REDISMODULE_OK.
-
Oran Agra authored
The MacOS CI in github actions often hangs without any logs. GH argues that it's due to resource utilization, either running out of disk space, memory, or CPU starvation, and thus the runner is terminated. This PR contains multiple attempts to resolve this: 1. introducing pause_process instead of SIGSTOP, which waits for the process to stop before resuming the test, possibly resolving race conditions in some tests, this was a suspect since there was one test that could result in an infinite loop in that case, in practice this didn't help, but still a good idea to keep. 2. disable the `save` config in many tests that don't need it, specifically ones that use heavy writes and could create large files. 3. change the `populate` proc to use short pipeline rather than an infinite one. 4. use `--clients 1` in the macos CI so that we don't risk running multiple resource demanding tests in parallel. 5. enable `--verbose` to be repeated to elevate verbosity and print more info to stdout when a test or a server starts.
-
- 11 Apr, 2023 1 commit
-
-
Binbin authored
We do have ZREMRANGEBYLEX tests, but it is a stress test marked with slow tag and then skipped in reply-schemas daily. In the past, we were able to succeed on a daily, i guess it was because there were some random command executions, such as corrupt-dump-fuzzy, which might call it. These test examples are taken from ZRANGEBYLEX basics test.
-
- 09 Apr, 2023 1 commit
-
-
Ozan Tezcan authored
Add `RM_RdbLoad()` and `RM_RdbSave()` to load/save RDB files from the module API. In our use case, we have our clustering implementation as a module. As part of this implementation, the module needs to trigger RDB save operation at specific points. Also, this module delivers RDB files to other nodes (not using Redis' replication). When a node receives an RDB file, it should be able to load the RDB. Currently, there is no module API to save/load RDB files. This PR adds four new APIs: ```c RedisModuleRdbStream *RM_RdbStreamCreateFromFile(const char *filename); void RM_RdbStreamFree(RedisModuleRdbStream *stream); int RM_RdbLoad(RedisModuleCtx *ctx, RedisModuleRdbStream *stream, int flags); int RM_RdbSave(RedisModuleCtx *ctx, RedisModuleRdbStream *stream, int flags); ``` The first step is to create a `RedisModuleRdbStream` object. This PR provides a function to create RedisModuleRdbStream from the filename. (You can load/save RDB with the filename). In the future, this API can be extended if needed: e.g., `RM_RdbStreamCreateFromFd()`, `RM_RdbStreamCreateFromSocket()` to save/load RDB from an `fd` or a `socket`. Usage: ```c /* Save RDB */ RedisModuleRdbStream *stream = RedisModule_RdbStreamCreateFromFile("example.rdb"); RedisModule_RdbSave(ctx, stream, 0); RedisModule_RdbStreamFree(stream); /* Load RDB */ RedisModuleRdbStream *stream = RedisModule_RdbStreamCreateFromFile("example.rdb"); RedisModule_RdbLoad(ctx, stream, 0); RedisModule_RdbStreamFree(stream); ```
-
- 02 Apr, 2023 1 commit
-
-
Slava Koyfman authored
The existing logic for killing pub-sub clients did not handle the `allchannels` permission correctly. For example, if you: ACL SETUSER foo allchannels Have a client authenticate as the user `foo` and subscribe to a channel, and then: ACL SETUSER foo resetchannels The subscribed client would not be disconnected, though new clients under that user would be blocked from subscribing to any channels. This was caused by an incomplete optimization in `ACLKillPubsubClientsIfNeeded` checking whether the new channel permissions were a strict superset of the old ones.
-