- 24 Oct, 2022 2 commits
-
-
guybe7 authored
We do not need to return the length of argv because it is equal to argc, which we return anyway. This change makes the code cleaner and adds a comment to explain something that might not be immediately clear.
-
guybe7 authored
This is a rare failure mode of a new feature of redis 7 introduced in #9217 (when the incremental part of the ID overflows). Till now, the outcome of that error was undetermined (could easily result in `Elements are too large to be stored` wrongly, due to unset `errno`).
-
- 23 Oct, 2022 2 commits
-
-
Moti Cohen authored
Funcion sentinelAddrEqualsHostname() of sentinel makes DNS resolve and based on it determines if two IP addresses are equal. Now, If the DNS resolve command fails, the function simply returns 0, even if the hostnames are identical. This might become an issue in case of failover such that sentinel might receives from Redis instance, response to regular INFO query it sent, and wrongly decide that the instance is pointing to is different leader than the one recorded because of this function, yet hostnames are identical. In turn sentinel disconnects the connection between sentinel and valid slave which leads to -failover-abort-no-good-slave. See issue #11241. I managed to reproduce only part of the flow in which the function return wrong result and trigger +fix-slave-config. The fix is even if the function failed to resolve then compare based on hostnames. That is our best effort as long as the server is unavailable for some reason. It is fine since Redis instance cannot have multiple hostnames for a given setup
-
qetu3790 authored
redis.set_repl() needs one arg, but the tips says two.
-
- 22 Oct, 2022 2 commits
-
-
Binbin authored
Make PFMERGE source key optional in docs, add tests with one input key, add tests on missing source keys (#11205) The following two cases will create an empty destkey HLL: 1. called with no source keys, like `pfmerge destkey` 2. called with non-existing source keys, like `pfmerge destkey non-existing-source-key` In the first case, in `PFMERGE`, the dest key is actually one of the source keys too. So `PFMERGE k1 k2` is equivalent to `SUNIONSTORE k1 k1 k2`, and `PFMERGE k1` is equivalent to `SUNIONSTORE k1 k1`. So the first case is reasonable, the source key is actually optional. And the second case, `PFMERGE` on missing keys should succeed and create an empty dest. This is consistent with `PFCOUNT`, and also with `SUNIONSTORE`, no need to change.
-
sundb authored
In the module, we will reuse the list iterator entry for RM_ListDelete, but `listTypeDelete` will only update `quicklistEntry->zi` but not `quicklistEntry->node`, which will result in `quicklistEntry->node` pointing to a freed memory address if the quicklist node is deleted. This PR sync `key->u.list.index` and `key->u.list.entry` to list iterator after `RM_ListDelete`. This PR also optimizes the release code of the original list iterator. Co-authored-by:
Viktor Söderqvist <viktor@zuiderkwast.se>
-
- 19 Oct, 2022 1 commit
-
-
FutabaRio authored
11 was the size of header/trailer in the old structure Ziplist, but now the size of header/trailer in the new structure Listpack should be 7.
-
- 18 Oct, 2022 5 commits
-
-
guybe7 authored
The use case is a module that wants to implement a blocking command on a key that necessarily exists and wants to unblock the client in case the key is deleted (much like what we implemented for XREADGROUP in #10306) New module API: * RedisModule_BlockClientOnKeysWithFlags Flags: * REDISMODULE_BLOCK_UNBLOCK_NONE * REDISMODULE_BLOCK_UNBLOCK_DELETED ### Detailed description of code changes blocked.c: 1. Both module and stream functions are called whether the key exists or not, regardless of its type. We do that in order to allow modules/stream to unblock the client in case the key is no longer present or has changed type (the behavior for streams didn't change, just code that moved into serveClientsBlockedOnStreamKey) 2. Make sure afterCommand is called in serveClientsBlockedOnKeyByModule, in order to propagate actions from moduleTryServeClientBlockedOnKey. 3. handleClientsBlockedOnKeys: call propagatePendingCommands directly after lookupKeyReadWithFlags to prevent a possible lazy-expire DEL from being mixed with any command propagated by the preceding functions. 4. blockForKeys: Caller can specifiy that it wants to be awakened if key is deleted. Minor optimizations (use dictAddRaw). 5. signalKeyAsReady became signalKeyAsReadyLogic which can take a boolean in case the key is deleted. It will only signal if there's at least one client that awaits key deletion (to save calls to handleClientsBlockedOnKeys). Minor optimizations (use dictAddRaw) db.c: 1. scanDatabaseForDeletedStreams is now scanDatabaseForDeletedKeys and will signalKeyAsReady for any key that was removed from the database or changed type. It is the responsibility of the code in blocked.c to ignore or act on deleted/type-changed keys. 2. Use the new signalDeletedKeyAsReady where needed blockedonkey.c + tcl: 1. Added test of new capabilities (FSL.BPOPGT now requires the key to exist in order to work)
-
Meir Shpilraien (Spielrein) authored
### Background The issue is that when saving an RDB with module AUX data, the module AUX metadata (moduleid, when, ...) is saved to the RDB even though the module did not saved any actual data. This prevent loading the RDB in the absence of the module (although there is no actual data in the RDB that requires the module to be loaded). ### Solution The solution suggested in this PR is that module AUX will be saved on the RDB only if the module actually saved something during `aux_save` function. To support backward compatibility, we introduce `aux_save2` callback that acts the same as `aux_save` with the tiny change of avoid saving the aux field if no data was actually saved by the module. Modules can use the new API to make sure that if they have no data to save, then it will be possible to load the created RDB even without the module. ### Concerns A module may register for the aux load and save hooks just in order to be notified when saving or loading starts or completed (there are better ways to do that, but it still possible that someone used it). However, if a module didn't save a single field in the save callback, it means it's not allowed to read in the read callback, since it has no way to distinguish between empty and non-empty payloads. furthermore, it means that if the module did that, it must never change it, since it'll break compatibility with it's old RDB files, so this is really not a valid use case. Since some modules (ones who currently save one field indicating an empty payload), need to know if saving an empty payload is valid, and if Redis is gonna ignore an empty payload or store it, we opted to add a new API (rather than change behavior of an existing API and expect modules to check the redis version) ### Technical Details To avoid saving AUX data on RDB, we change the code to first save the AUX metadata (moduleid, when, ...) into a temporary buffer. The buffer is then flushed to the rio at the first time the module makes a write operation inside the `aux_save` function. If the module saves nothing (and `aux_save2` was used), the entire temporary buffer is simply dropped and no data about this AUX field is saved to the RDB. This make it possible to load the RDB even in the absence of the module. Test was added to verify the fix.
-
Shuning authored
Seems excessive to call getExpire if we don't need it. This can maybe have some speedup on AOF file loading (saving a dictFind call) Co-authored-by:
lvshuning <lvshuning@meituan.com>
-
DarrenJiang13 authored
fix some malloc macros in `listpack.c`. listpack has it's own malloc aliases, but in some places normal redis malloc calls have slipped in.
-
Binbin authored
And fix a few newly detected typo. Closes #11394
-
- 16 Oct, 2022 3 commits
-
-
Shaya Potter authored
Motivation: for applications that use RM ACL verification functions, they would want to return errors back to the user, in ways that are consistent with Redis. While investigating how we should return ACL errors to the user, we realized that Redis isn't consistent, and currently returns ACL error strings in 3 primary ways. [For the actual implications of this change, see the "Impact" section at the bottom] 1. how it returns an error when calling a command normally ACL_DENIED_CMD -> "this user has no permissions to run the '%s' command" ACL_DENIED_KEY -> "this user has no permissions to access one of the keys used as arguments" ACL_DENIED_CHANNEL -> "this user has no permissions to access one of the channels used as arguments" 2. how it returns an error when calling via 'acl dryrun' command ACL_DENIED_CMD -> "This user has no permissions to run the '%s' command" ACL_DENIED_KEY -> "This user has no permissions to access the '%s' key" ACL_DENIED_CHANNEL -> "This user has no permissions to access the '%s' channel" 3. how it returns an error via RM_Call (and scripting is similar). ACL_DENIED_CMD -> "can't run this command or subcommand"; ACL_DENIED_KEY -> "can't access at least one of the keys mentioned in the command arguments"; ACL_DENIED_CHANNEL -> "can't publish to the channel mentioned in the command"; In addition, if one wants to use RM_Call's "dry run" capability instead of the RM ACL functions directly, one also sees a different problem than it returns ACL errors with a -ERR, not a -PERM, so it can't be returned directly to the caller. This PR modifies the code to generate a base message in a common manner with the ability to set verbose flag for acl dry run errors, and keep it unset for normal/rm_call/script cases ```c sds getAclErrorMessage(int acl_res, user *user, struct redisCommand *cmd, sds errored_val, int verbose) { switch (acl_res) { case ACL_DENIED_CMD: return sdscatfmt(sdsempty(), "User %S has no permissions to run " "the '%S' command", user->name, cmd->fullname); case ACL_DENIED_KEY: if (verbose) { return sdscatfmt(sdsempty(), "User %S has no permissions to access " "the '%S' key", user->name, errored_val); } else { return sdsnew("No permissions to access a key"); } case ACL_DENIED_CHANNEL: if (verbose) { return sdscatfmt(sdsempty(), "User %S has no permissions to access " "the '%S' channel", user->name, errored_val); } else { return sdsnew("No permissions to access a channel"); } } ``` The caller can append/prepend the message (adding NOPERM for normal/RM_Call or indicating it's within a script). Impact: - Plain commands, as well as scripts and RM_Call now include the user name. - ACL DRYRUN remains the only one that's verbose (mentions the offending channel or key name) - Changes RM_Call ACL errors from being a `-ERR` to being `-NOPERM` (besides for textual changes) **This somewhat a breaking change, but it only affects the RM_Call with both `C` and `E`, or `D`** - Changes ACL errors in scripts textually from being `The user executing the script <old non unified text>` to `ACL failure in script: <new unified text>`
-
Meir Shpilraien (Spielrein) authored
As discussed on #11084, `propagatePendingCommands` should happened after the del notification is fired so that the notification effect and the `del` will be replicated inside MULTI EXEC. Test was added to verify the fix.
-
David CARLIER authored
* Fixes build warning when CACHE_LINE_SIZE is already defined * Fixes wrong CACHE_LINE_SIZE on some FreeBSD systems where it could be set to 128 (e.g. on MIPS) * Fixes wrong CACHE_LINE_SIZE on Apple M1 (use 128 instead of 64) Wrong cache line size in that case can some false sharing of array elements between threads, see #10892
-
- 15 Oct, 2022 1 commit
-
-
filipe oliveira authored
optimizing d2string() and addReplyDouble() with grisu2: double to string conversion based on Florian Loitsch's Grisu-algorithm (#10587) All commands / use cases that heavily rely on double to a string representation conversion, (e.g. meaning take a double-precision floating-point number like 1.5 and return a string like "1.5" ), could benefit from a performance boost by swapping snprintf(buf,len,"%.17g",value) by the equivalent [fpconv_dtoa](https://github.com/night-shift/fpconv) or any other algorithm that ensures 100% coverage of conversion. This is a well-studied topic and Projects like MongoDB. RedPanda, PyTorch leverage libraries ( fmtlib ) that use the optimized double to string conversion underneath. The positive impact can be substantial. This PR uses the grisu2 approach ( grisu explained on https://www.cs.tufts.edu/~nr/cs257/archive/florian-loitsch/printf.pdf section 5 ). test suite changes: Despite being compatible, in some cases it produces a different result from printf, and some tests had to be adjusted. one case is that `%.17g` (which means %e or %f which ever is shorter), chose to use `5000000000` instead of 5e+9, which sounds like a bug? In other cases, we changed TCL to compare numbers instead of strings to ignore minor rounding issues (`expr 0.8 == 0.79999999999999999`)
-
- 13 Oct, 2022 2 commits
-
-
C Charles authored
MIGTATE with AUTH that contains "keys" is getting wrong key names in migrateGetKeys, leads to ACL errors (#11253) When using the MIGRATE, with a destination Redis that has the user name or password set to the string "keys", Redis would have determine the wrong set of key names the command is gonna access. This lead to ACL returning wrong authentication result. Destination instance: ``` 127.0.0.1:6380> acl setuser default >keys OK 127.0.0.1:6380> acl setuser keys on nopass ~* &* +@all OK ``` Source instance: ``` 127.0.0.1:6379> set a 123 OK 127.0.0.1:6379> acl setuser cc on nopass ~a* +@all OK 127.0.0.1:6379> auth cc 1 OK 127.0.0.1:6379> migrate 127.0.0.1 6380 "" 0 1000 auth keys keys a (error) NOPERM this user has no permissions to access one of the keys used as arguments 127.0.0.1:6379> migrate 127.0.0.1 6380 "" 0 1000 auth2 keys pswd keys a (error) NOPERM this user has no permissions to access one of the keys used as arguments ``` Using `acl dryrun` we know that the parameters of `auth` and `auth2` are mistaken for the `keys` option. ``` 127.0.0.1:6379> acl dryrun cc migrate whatever whatever "" 0 1000 auth keys keys a "This user has no permissions to access the 'keys' key" 127.0.0.1:6379> acl dryrun cc migrate whatever whatever "" 0 1000 auth2 keys pswd keys a "This user has no permissions to access the 'pswd' key" ``` Fix the bug by editing db.c/migrateGetKeys function, which finds the `keys` option and all the keys following.
-
Oran Agra authored
1. show the overcommit warning when overcommit is disabled (2), not just when it is set to heuristic (0). 2. improve warning text to mention the issue with jemalloc causing VM mapping fragmentation when set to 2.
-
- 12 Oct, 2022 1 commit
-
-
Meir Shpilraien (Spielrein) authored
PR #9320 introduces initialization order changes. Now cluster is initialized after modules. This changes causes a crash if the module uses RM_Call inside the load function on cluster mode (the code will try to access `server.cluster` which at this point is NULL). To solve it, separate cluster initialization into 2 phases: 1. Structure initialization that happened before the modules initialization 2. Listener initialization that happened after. Test was added to verify the fix.
-
- 11 Oct, 2022 1 commit
-
-
Nikita Tolkachev authored
remove double period at the end of sentence.
-
- 09 Oct, 2022 4 commits
-
-
Binbin authored
The old `server.unixtime*1000000` will overflow in 32-bits. This was introduced in #10300 (not released).
-
yancz2000 authored
Forgot to start redis-server when testing performance. When opening the benchmark for testing, it will always be stuck, and the process cpu will reach 100%.
-
Binbin authored
Freeze time during execution of scripts and all other commands. This means that a key is either expired or not, and doesn't change state during a script execution. resolves #10182 This PR try to add a new `commandTimeSnapshot` function. The function logic is extracted from `keyIsExpired`, but the related calls to `fixed_time_expire` and `mstime()` are removed, see below. In commands, we will avoid calling `mstime()` multiple times and just use the one that sampled in call. The background is, e.g. using `PEXPIRE 1` with valgrind sometimes result in the key being deleted rather than expired. The reason is that both `PEXPIRE` command and `checkAlreadyExpired` call `mstime()` separately. There are other more important changes in this PR: 1. Eliminate `fixed_time_expire`, it is no longer needed. When we want to sample time we should always use a time snapshot. We will use `in_nested_call` instead to update the cached time in `call`. 2. Move the call for `updateCachedTime` from `serverCron` to `afterSleep`. Now `commandTimeSnapshot` will always return the sample time, the `lookupKeyReadWithFlags` call in `getNodeByQuery` will get a outdated cached time (because `processCommand` is out of the `call` context). We put the call to `updateCachedTime` in `aftersleep`. 3. Cache the time each time the module lock Redis. Call `updateCachedTime` in `moduleGILAfterLock`, affecting `RM_ThreadSafeContextLock` and `RM_ThreadSafeContextTryLock` Currently the commandTimeSnapshot change affects the following TTL commands: - SET EX / SET PX - EXPIRE / PEXPIRE - SETEX / PSETEX - GETEX EX / GETEX PX - TTL / PTTL - EXPIRETIME / PEXPIRETIME - RESTORE key TTL And other commands just use the cached mstime (including TIME). This is considered to be a breaking change since it can break a script that uses a loop to wait for a key to expire.
-
Meir Shpilraien (Spielrein) authored
As mentioned on docs, `RM_ResetDataset` Performs similar operation to FLUSHALL. As FLUSHALL do not clean the function, `RM_ResetDataset` should not clean the functions as well.
-
- 07 Oct, 2022 2 commits
-
-
Oran Agra authored
Build fails with warnings in ARM CI after adding more aggressive optimizations (#11350) probably a result of more aggressive inlining ``` ziplist.c: In function ‘pop.constprop’: ziplist.c:1770:13: error: ‘vlong’ may be used uninitialized in this function [-Werror=maybe-uninitialized] printf("%lld", vlong); ^~~~~~~~~~~~~~~~~~~~~ ``` ``` listpack.c: In function ‘lpInsert.constprop’: listpack.c:406:9: error: argument 2 null where non-null expected [-Werror=nonnull] memcpy(buf+1,s,len); ^~~~~~~~~~~~~~~~~~~ ```
-
aradz44 authored
Added authentication failure and access denied metrics
-
- 06 Oct, 2022 2 commits
-
-
Moti Cohen authored
Refine getTimeoutFromObjectOrReply() out-of-range check. Timeout is parsed (and verifies out of range) as double and multiplied by 1000, added mstime() and stored in long-long which might lead to out-of-range value of long-long. Co-authored-by:
moticless <moticless@github.com> Co-authored-by:
Oran Agra <oran@redislabs.com> Co-authored-by:
Ozan Tezcan <ozantezcan@gmail.com>
-
Ozan Tezcan authored
Currently, we add -flto to the compile flags only. We are supposed to add it to the linker flags as well. Clang build fails because of this. Added a change to add -flto to REDIS_CFLAGS and REDIS_LDFLAGS if the build optimization flag is -O3. (noopt build will not use -flto)
-
- 03 Oct, 2022 1 commit
-
-
Madelyn Olson authored
This PR introduces a couple of changes to improve cluster test stability: 1. Increase the cluster node timeout to 3 seconds, which is similar to the normal cluster tests, but introduce a new mechanism to increase the ping period so that the tests are still fast. This new config is a debug config. 2. Set `cluster-replica-no-failover yes` on a wider array of tests which are sensitive to failovers. This was occurring on the ARM CI.
-
- 02 Oct, 2022 4 commits
-
-
David CARLIER authored
-
Maria Markova authored
Optimization update from -O2 to -O3 -flto gives up to 5% performance gain in 'redis-benchmarks-spec-client-runner' tests geomean where GCC 9.4.0 is used for build * Fix for false-positive warning in bitops.c Warning appeared with O3, on CentOS during inlininig procedure * Fixed unitialized streamID within streamTrim() (#1) Co-authored-by:
filipe oliveira <filipecosta.90@gmail.com>
-
Binbin authored
- fix `the the` typo - `LPOPRPUSH` does not exist, should be `RPOPLPUSH` - `CLUSTER GETKEYINSLOT` 's time complexity should be O(N) - `there bytes` should be `three bytes`, this closes #11266 - `slave` word to `replica` in log, modified the front and missed the back - remove useless aofReadDiffFromParent in server.h - `trackingHandlePendingKeyInvalidations` method adds a void parameter
-
Huang Zhw authored
- BITOP: turn argument `operation` from string to oneof - CLIENT KILL: turn argument `skipme` from string to oneof - COMMAND GETKEYS / GETKEYSANDFLAGS: change arguments to optional, and change arity to -3 (fixes regression in redis 7.0) - CLIENT PAUSE: this command was added in v3.0.0
-
- 30 Sep, 2022 1 commit
-
-
Binbin authored
We will always show the bus-port, and if the node hostname exists, it will also show it.
-
- 29 Sep, 2022 3 commits
-
-
Huang Zhw authored
Better redis-cli hints for commands that take other commands as arguments. ``` command getkeysandflags hello [protover [AUTH username password]] acl dryrun user hello [protover [AUTH username password]] ```
-
Wen Hui authored
The sentinel CONFIG GET command doesn't support multiple arguments, but the json file did. remove that.
-
Meir Shpilraien (Spielrein) authored
If Redis crashes due to calling an invalid function pointer, the `backtrace` function will try to dereference this invalid pointer which will cause a crash inside the crash report and will kill the processes without having all the crash report information. Example: ``` === REDIS BUG REPORT START: Cut & paste starting from here === 198672:M 19 Sep 2022 18:06:12.936 # Redis 255.255.255 crashed by signal: 11, si_code: 1 198672:M 19 Sep 2022 18:06:12.936 # Accessing address: 0x1 198672:M 19 Sep 2022 18:06:12.936 # Crashed running the instruction at: 0x1 // here the processes is crashing ``` This PR tries to fix this crash be: 1. Identify the issue when it happened. 2. Replace the invalid pointer with a pointer to some dummy function so that `backtrace` will not crash. I identification is done by comparing `eip` to `info->si_addr`, if they are the same we know that the crash happened on the same address it tries to accesses and we can conclude that it tries to call and invalid function pointer. To replace the invalid pointer we introduce a new function, `setMcontextEip`, which is very similar to `getMcontextEip` and it knows to set the Eip for the different supported OS's. After printing the trace we retrieve the old `Eip` value.
-
- 28 Sep, 2022 3 commits
-
-
sundb authored
Fix the missing server.dirty increment and redundant signalModifiedKey in serveClientBlockedOnList (#11326) Mainly fix two minor bug 1. When handle BL*POP/BLMOVE commands with blocked clients, we should increment server.dirty. 2. `listPopRangeAndReplyWithKey()` in `serveClientBlockedOnList()` should not repeat calling `signalModifiedKey()` which has been called when an element was pushed on the list. (was skipped in all bpop commands, other than blmpop) Other optimization add `signal` param for `listElementsRemoved` to skip `signalModifiedKey()` to unify all pop operation. Unifying all pop operations also prepares us for #11303, so that we can avoid having to deal with the conversion from quicklist to listpack() in various places when the list shrinks.
-
guybe7 authored
The original idea behind auto-setting the default (first,last,step) spec was to use the most "open" flags when the user didn't provide any key-spec flags information. While the above idea is a good approach, it really makes no sense to set CMD_KEY_VARIABLE_FLAGS if the user didn't provide the getkeys-api flag: in this case there's not way to retrieve these variable flags, so what's the point? Internally in redis there was code to ignore this already, so this fix doesn't change redis's behavior, it only affects the output of COMMAND command.
-
Phoeniwx authored
doing redis-cli --memkeys-samples without any additional arguments would have lead to a crash of the cli.
-