- 21 Sep, 2022 8 commits
-
-
Oran Agra authored
Redis 7.0 has #9890 which added an assertion when the propagation queue was not flushed and we got to beforeSleep. But it turns out that when processCommands calls getNodeByQuery and decides to reject the command, it can lead to a key that was lazy expired and is deleted without later flushing the propagation queue. This change prevents lazy expiry from deleting the key at this stage (not as part of a command being processed in `call`) (cherry picked from commit c789fb0a)
-
DarrenJiang13 authored
Fix bug with scripts ignoring client tracking NOLOOP and send an invalidation message anyway. (cherry picked from commit 44859a41)
-
Huang Zhw authored
`bitfield` with `get` may not be readonly. ``` 127.0.0.1:6384> acl setuser hello on nopass %R~* +@all OK 127.0.0.1:6384> auth hello 1 OK 127.0.0.1:6384> bitfield hello set i8 0 1 (error) NOPERM this user has no permissions to access one of the keys used as arguments 127.0.0.1:6384> bitfield hello set i8 0 1 get i8 0 1) (integer) 0 2) (integer) 1 ``` Co-authored-by:
Oran Agra <oran@redislabs.com> (cherry picked from commit ec5034a2)
-
Meir Shpilraien (Spielrein) authored
Fix #11030, use lua_rawget to avoid triggering metatables. #11030 shows how return `_G` from the Lua script (either function or eval), cause the Lua interpreter to Panic and the Redis processes to exit with error code 1. Though return `_G` only panic on Redis 7 and 6.2.7, the underline issue exists on older versions as well (6.0 and 6.2). The underline issue is returning a table with a metatable such that the metatable raises an error. The following example demonstrate the issue: ``` 127.0.0.1:6379> eval "local a = {}; setmetatable(a,{__index=function() foo() end}) return a" 0 Error: Server closed the connection ``` ``` PANIC: unprotected error in call to Lua API (user_script:1: Script attempted to access nonexistent global variable 'foo') ``` The Lua panic happened because when returning the result to the client, Redis needs to introspect the returning table and transform the table into a resp. In order to scan the table, Redis uses `lua_gettable` api which might trigger the metatable (if exists) and might raise an error. This code is not running inside `pcall` (Lua protected call), so raising an error causes the Lua to panic and exit. Notice that this is not a crash, its a Lua panic that exit with error code 1. Returning `_G` panics on Redis 7 and 6.2.7 because on those versions `_G` has a metatable that raises error when trying to fetch a none existing key. ### Solution Instead of using `lua_gettable` that might raise error and cause the issue, use `lua_rawget` that simply return the value from the table without triggering any metatable logic. This is promised not to raise and error. The downside of this solution is that it might be considered as breaking change, if someone rely on metatable in the returned value. An alternative solution is to wrap this entire logic with `pcall` (Lua protected call), this alternative require a much bigger refactoring. ### Back Porting The same fix will work on older versions as well (6.2, 6.0). Notice that on those version, the issue can cause Redis to crash if inside the metatable logic there is an attempt to accesses Redis (`redis.call`). On 7.0, there is not crash and the `redis.call` is executed as if it was done from inside the script itself. ### Tests Tests was added the verify the fix (cherry picked from commit 020e046b)
-
- 18 Jul, 2022 1 commit
-
-
Oran Agra authored
The temporary array for deleted entries reply of XAUTOCLAIM was insufficient, but also in fact the COUNT argument should be used to control the size of the reply, so instead of terminating the loop by only counting the claimed entries, we'll count deleted entries as well. Fix #10968 Addresses CVE-2022-31144 (cherry picked from commit 2825b605)
-
- 11 Jul, 2022 3 commits
-
-
Binbin authored
#10942 break the new test added in #10449 ``` Testing unit: 29-slot-migration-response.tcl Cluster Join and auto-discovery test: FAILED: Cluster failed to join into a full mesh. ``` It looks like we need to wait for the cluster in 28 to become stable.
-
Binbin authored
In #9389, we add a new `cluster-port` config and make cluster bus port configurable, and currently redis-cli --cluster create/add-node doesn't support with a configurable `cluster-port` instance. Because redis-cli uses the old way (port + 10000) to send the `CLUSTER MEET` command. Now we add this support on redis-cli `--cluster`, note we don't need to explicitly pass in the `cluster-port` parameter, we can get the real `cluster-port` of the node in `clusterManagerNodeLoadInfo`, so the `--cluster create` and `--cluster add-node` interfaces have not changed. We will use the `cluster-port` when we are doing `CLUSTER MEET`, also note that `CLUSTER MEET` bus-port parameter was added in 4.0, so if the bus_port (the one in redis-cli) is 0, or equal (port + 10000), we just call `CLUSTER MEET` with 2 arguments, using the old form. Co-authored-by:
Madelyn Olson <34459052+madolson@users.noreply.github.com>
-
Madelyn Olson authored
* Fix an engine crash when there are nodes in handshaking and a user calls cluster shards
-
- 04 Jul, 2022 2 commits
-
-
Wen Hui authored
* Add tests for error messages during slot migrations Co-authored-by:
Ubuntu <lucas.guang.yang1@huawei.com> Co-authored-by:
Madelyn Olson <madelyneolson@gmail.com>
-
Harkrishn Patro authored
Account sharded pubsub channels memory consumption in client memory usage computation to accurately evict client based on the set threshold for `maxmemory-clients`.
-
- 03 Jul, 2022 1 commit
-
-
Yossi Gottlieb authored
Before this commit, TLS tests on Ubuntu 22.04 would fail as dropped connections result with an ECONNABORTED error thrown instead of an empty read.
-
- 30 Jun, 2022 1 commit
-
-
Binbin authored
This was harmless because we marked the parent command with SENTINEL flag. So the populateCommandTable was ok. And we also don't show the flag (SENTINEL and ONLY-SENTNEL) in COMMAND INFO. In this PR, we also add the same CMD_SENTINEL and CMD_ONLY_SENTINEL flags check when populating the sub-commands. so that in the future it'll be possible to add some sub-commands to sentinel or sentinel-only but not others.
-
- 29 Jun, 2022 1 commit
-
-
Wen Hui authored
Fix regression of CLUSTER RESET command in redis 7.0. cluster reset command format is: CLUSTER RESET [ HARD | SOFT] According to the cluster reset command doc and codes, the third argument is optional, so the arity in json file should be -2 instead of 3. Add test to verify future regressions with RESET and RESET SOFT that were not covered. Co-authored-by:
Ubuntu <lucas.guang.yang1@huawei.com> Co-authored-by:
Oran Agra <oran@redislabs.com> Co-authored-by:
Binbin <binloveplay1314@qq.com>
-
- 28 Jun, 2022 1 commit
-
-
jonnyomerredis authored
When calling CLIENT INFO/LIST, and in various debug prints, Redis is printing the number of pubsub channels / patterns the client is subscribed to. With the addition of sharded pubsub, it would be useful to print the number of keychannels the client is subscribed to as well.
-
- 27 Jun, 2022 1 commit
-
-
Viktor Söderqvist authored
The module API docs mentions this macro, but it was not defined (so no one could have used it). Instead of adding it as is, we decided to add a _V1 macro, so that if / when we some day extend this struct, modules that use this API and don't need the extra fields, will still use the old version and still be compatible with older redis version (despite being compiled with newer redismodule.h)
-
- 26 Jun, 2022 3 commits
-
-
RinChanNOW! authored
Since the ranges of `unsigned long long` and `long long` are different, we cannot read an `unsigned long long` integer from a `RedisModuleString` by `RedisModule_StringToLongLong` . So I added two new Redis Module APIs to support the conversion between these two types: * `RedisModule_StringToULongLong` * `RedisModule_CreateStringFromULongLong` Signed-off-by:
RinChanNOWWW <hzy427@gmail.com>
-
Binbin authored
This commit has two topics. ## Passing config name and value in the same arg In #10660 (Redis 7.0.1), when we supported the config values that can start with `--` prefix (one of the two topics of that PR), we broke another pattern: `redis-server redis.config "name value"`, passing both config name and it's value in the same arg, see #10865 This wasn't a intended change (i.e we didn't realize this pattern used to work). Although this is a wrong usage, we still like to fix it. Now we support something like: ``` src/redis-server redis.conf "--maxmemory '700mb'" "--maxmemory-policy volatile-lru" --proc-title-template --my--title--template --loglevel verbose ``` ## Changes around --save Also in this PR, we undo the breaking change we made in #10660 on purpose. 1. `redis-server redis.conf --save --loglevel verbose` (missing `save` argument before anotehr argument). In 7.0.1, it was throwing an wrong arg error. Now it will work and reset the save, similar to how it used to be in 7.0.0 and 6.2.x. 3. `redis-server redis.conf --loglevel verbose --save` (missing `save` argument as last argument). In 6.2, it did not reset the save, which was a bug (inconsistent with the previous bullet). Now we will make it work and reset the save as well (a bug fix).
-
Viktor Söderqvist authored
Adding Module APIs to let the module read and set the client name of an arbitrary connection.
-
- 23 Jun, 2022 1 commit
-
-
judeng authored
The new test added in #10891 can fail with a different error. see comment in networking.c saying ```c /* That's a best effort error message, don't check write errors. * Note that for TLS connections, no handshake was done yet so nothing * is written and the connection will just drop. */ ```
-
- 22 Jun, 2022 1 commit
-
-
judeng authored
my maxclients config: ``` redis-cli config get maxclients 1) "maxclients" 2) "4064" ``` Before this bug was fixed, creating 4065 clients appeared to be successful, but only 4064 were actually created``` ``` ./redis-benchmark -c 4065 -I Creating 4065 idle connections and waiting forever (Ctrl+C when done) cients: 4065 ``` now : ``` ./redis-benchmark -c 4065 -I Creating 4065 idle connections and waiting forever (Ctrl+C when done) Error from server: ERR max number of clients reached ./redis-benchmark -c 4064 -I Creating 4064 idle connections and waiting forever (Ctrl+C when done) clients: 4064 ```
-
- 21 Jun, 2022 1 commit
-
-
Meir Shpilraien (Spielrein) authored
The PR fixes 2 issues: ### RM_Call crash on script mode `RM_Call` can potentially be called from a background thread where `server.current_client` are not set. In such case we get a crash on `NULL` dereference. The fix is to check first if `server.current_client` is `NULL`, if it does we should verify disc errors and readonly replica as we do to any normal clients (no masters nor AOF). ### RM_Call block OOM commands when not needed Again `RM_Call` can be executed on a background thread using a `ThreadSafeCtx`. In such case `server.pre_command_oom_state` can be irrelevant and should not be considered when check OOM state. This cause OOM commands to be blocked when not necessarily needed. In such case, check the actual used memory (and not the cached value). Notice that in order to know if the cached value can be used, we check that the ctx that was used on the `RM_Call` is a ThreadSafeCtx. Module writer can potentially abuse the API and use ThreadSafeCtx on the main thread. We consider this as a API miss used.
-
- 19 Jun, 2022 1 commit
-
-
Moti Cohen authored
Co-authored-by:
moticless <moticless@github.com>
-
- 14 Jun, 2022 3 commits
-
-
Oran Agra authored
when we know the size of the zset we're gonna store in advance, we can check if it's greater than the listpack encoding threshold, in which case we can create a skiplist from the get go, and avoid converting the listpack to skiplist later after it was already populated.
-
Oran Agra authored
If a script made a modification and then was interrupted for taking too long. there's a chance redis will detect that a replica dropped and would like to reject write commands with NOREPLICAS due to insufficient good replicas. returning an error on a command in this case breaks the script atomicity. The same could in theory happen with READONLY, MISCONF, but i don't think these state changes can happen during script execution.
-
Oran Agra authored
I noticed that scripting.tcl uses INFO from within a script and thought it's an overkill and concluded it's nicer to use another CMD_STALE command, decided to use ECHO, and then noticed it's not at all allowed in stale mode. probably overlooked at #6843
-
- 12 Jun, 2022 1 commit
-
-
Binbin authored
The SET and BITFIELD command were added `get_keys_function` in #10148, causing them to be wrongly marked movablekeys in `populateCommandMovableKeys`. This was an unintended side effect introduced in #10148 (7.0 RC1) which could cause some clients an extra round trip for these commands in cluster mode. Since we define movablekeys as a way to determine if the legacy range [first, last, step] doesn't find all keys, then we need a completely different approach. The right approach should be to check if the legacy range covers all key-specs, and if none of the key-specs have the INCOMPLETE flag. This way, we don't need to look at getkeys_proc of VARIABLE_FLAG at all. Probably with the exception of modules, who may still not be using key-specs. In this PR, we removed `populateCommandMovableKeys` and put its logic in `populateCommandLegacyRangeSpec`. In order to properly serve both old and new modules, we must probably keep relying CMD_MODULE_GETKEYS, but do that only for modules that don't declare key-specs. For ones that do, we need to take the same approach we take with native redis commands. This approach was proposed by Oran. Fixes #10833 Co-authored-by:
Oran Agra <oran@redislabs.com>
-
- 09 Jun, 2022 1 commit
-
-
Christian Krieg authored
The test calls `ldd` on `redis-server` in order to find out whether the binary was linked against `libmusl`; However, `ldd` returns a value different from `0` when statically linking the binaries agains libc-musl, because `redis-server` is not a dynamic executable (as given by the exception thrown by the failing test), and `make test` terminates with an error:: $ ldd src/redis-server not a dynamic executable $ echo $? 1 This commit fixes the test by ignoring such failures. Co-authored-by:
Yossi Gottlieb <yossigo@gmail.com>
-
- 07 Jun, 2022 1 commit
-
-
Petr Vaněk authored
This change fixes failing `integration/logging.tcl` test in Gentoo with musl libc, where `ldd` returns ``` libc.so => /lib/ld-musl-x86_64.so.1 (0x7f9d5f171000) ``` unlike Alpine's ``` libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x7f82cfa16000) ``` The solution is to extend matching pattern introduced in #8532.
-
- 02 Jun, 2022 2 commits
-
-
zhaozhao.zz authored
Redis 7 adds some new alias config like `hash-max-listpack-entries` alias `hash-max-ziplist-entries`. If a config file contains both real name and alias like this: ``` hash-max-listpack-entries 20 hash-max-ziplist-entries 20 ``` after set `hash-max-listpack-entries` to 100 and `config rewrite`, the config file becomes to: ``` hash-max-listpack-entries 100 hash-max-ziplist-entries 20 ``` we can see that the alias config is not modified, and users will get wrong config after restart. 6.0 and 6.2 doesn't have this bug, since they only have the `slave` word alias. Co-authored-by:
Oran Agra <oran@redislabs.com>
-
zhugezy authored
Fix bugs in CONFIG REWRITE, omitting rename-command and include lines, and inserting comments around module and acl configs (#10761) A regression from #10285 (redis 7.0). CONFIG REWRITE would put lines with: `include`, `rename-command`, `user`, `loadmodule`, and any module specific config in a comment. For ACL `user`, `loadmodule` and module specific configs would be re-inserted at the end (instead of updating existing lines), so the only implication is a messy config file full of comments. But for `rename-command` and `include`, the implication would be that they're now missing, so a server restart would lose them. Co-authored-by:
Oran Agra <oran@redislabs.com>
-
- 01 Jun, 2022 2 commits
-
-
Oran Agra authored
The important part is that read-only scripts (not just EVAL_RO and FCALL_RO, but also ones with `no-writes` executed by normal EVAL or FCALL), will now be permitted to run during CLIENT PAUSE WRITE (unlike before where only the _RO commands would be processed). Other than that, some errors like OOM, READONLY, MASTERDOWN are now handled by processCommand, rather than the command itself affects the error string (and even error code in some cases), and command stats. Besides that, now the `may-replicate` commands, PFCOUNT and PUBLISH, will be considered `write` commands in scripts and will be blocked in all read-only scripts just like other write commands. They'll also be blocked in EVAL_RO (i.e. even for scripts without the `no-writes` shebang flag. This commit also hides the `may_replicate` flag from the COMMAND command output. this is a **breaking change**. background about may_replicate: We don't want to expose a no-may-replicate flag or alike to scripts, since we consider the may-replicate thing an internal concern of redis, that we may some day get rid of. In fact, the may-replicate flag was initially introduced to flag EVAL: since we didn't know what it's gonna do ahead of execution, before function-flags existed). PUBLISH and PFCOUNT, both of which because they have side effects which may some day be fixed differently. code changes: The changes in eval.c are mostly code re-ordering: - evalCalcFunctionName is extracted out of evalGenericCommand - evalExtractShebangFlags is extracted luaCreateFunction - evalGetCommandFlags is new code
-
Oran Agra authored
Fix broken protocol in MISCONF error, RM_Yield bugs, RM_Call(EVAL) OOM check bug, and new RM_Call checks. (#10786) * Fix broken protocol when redis can't persist to RDB (general commands, not modules), excessive newline. regression of #10372 (7.0 RC3) * Fix broken protocol when Redis can't persist to AOF (modules and scripts), missing newline. * Fix bug in OOM check of EVAL scripts called from RM_Call. set the cached OOM state for scripts before executing module commands too, so that it can serve scripts that are executed by modules. i.e. in the past EVAL executed by RM_Call could have either falsely fail or falsely succeeded because of a wrong cached OOM state flag. * Fix bugs with RM_Yield: 1. SHUTDOWN should only accept the NOSAVE mode 2. Avoid eviction during yield command processing. 3. Avoid processing master client commands while yielding from another client * Add new two more checks to RM_Call script mode. 1. READONLY You can't write against a read only replica 2. MASTERDOWN Link with MASTER is down and `replica-serve-stale-data` is set to `no` * Add new RM_Call flag to let redis automatically refuse `deny-oom` commands while over the memory limit. * Add tests to cover various errors from Scripts, Modules, Modules calling scripts, and Modules calling commands in script mode. Add tests: * Looks like the MISCONF error was completely uncovered by the tests, add tests for it, including from scripts, and modules * Add tests for NOREPLICAS from scripts * Add tests for the various errors in module RM_Call, including RM_Call that calls EVAL, and RM_call in "eval mode". that includes: NOREPLICAS, READONLY, MASTERDOWN, MISCONF
-
- 31 May, 2022 1 commit
-
-
Harkrishn Patro authored
To easily distinguish between sharded channel message and a global channel message, introducing `smessage` (instead of `message`) as message bulk for sharded channel publish message. This is gonna be a breaking change in 7.0.1! Background: Sharded pubsub introduced in redis 7.0, but after the release we quickly realized that the fact that it's problematic that the client can't distinguish between normal (global) pubsub messages and sharded ones. This is important because the same connection can subscribe to both, but messages sent to one pubsub system are not propagated to the other (they're completely separate), so if one connection is used to subscribe to both, we need to assist the client library to know which message it got so it can forward it to the correct callback.
-
- 30 May, 2022 1 commit
-
-
Madelyn Olson authored
* Add readonly flag to EVAL_RO, EVALSHA_RO and FCALL_RO * Require users to explicitly declare @scripting to get access to lua scripting.
-
- 29 May, 2022 1 commit
-
-
Binbin authored
There is a timing issue reported in test-sanitizer-address (gcc): ``` Sentinels (re)connection following SENTINEL SET mymaster auth-pass: FAILED: Expected to be disconnected from master due to wrong password ``` The reason we reach it, is because the test is fast enough to modify auth-pass and test sentinel connection status with the server, before its scheduled operation got the chance to update connection status with the server. We need to wait for `sentinelTimer` to kick in, and then update the connection status. Replace condition with wait_for_condition on the check. Fix just like #10480 did
-
- 27 May, 2022 1 commit
-
-
Vitaly authored
When `zrangestore` is called container destination object is created. Before this PR we used to create a listpack based object even if `zset-max-ziplist-entries` or equivalent`zset-max-listpack-entries` was set to 0. This triggered immediate conversion of the listpack into a skiplist in `zrangestore`, which hits an assertion resulting in an engine crash. Added a TCL test that reproduces this issue.
-