- 07 Feb, 2022 2 commits
-
-
yoav-steinberg authored
Changes: 1. Adds the `redis.acl_check_cmd()` api to lua scripts. It can be used to check if the current user has permissions to execute a given command. The new function receives the command to check as an argument exactly like `redis.call()` receives the command to execute as an argument. 2. In the PR I unified the code used to convert lua arguments to redis argv arguments from both the new `redis.acl_check_cmd()` API and the `redis.[p]call()` API. This cleans up potential duplicate code. 3. While doing the refactoring in 2 I noticed there's an optimization to reduce allocation calls when parsing lua arguments into an `argv` array in the `redis.[p]call()` implementation. These optimizations were introduced years ago in 48c49c48 and 4f686555. It is unclear why this was added. The original commit message claims a 4% performance increase which I couldn't recreate and might not be worth it even if it did recreate. This PR removes that optimization. Following are details of the benchmark I did that couldn't reveal any performance improvements due to this optimization: ``` benchmark 1: src/redis-benchmark -P 500 -n 10000000 eval 'return redis.call("ping")' 0 benchmark 2: src/redis-benchmark -P 500 -r 1000 -n 1000000 eval 'return redis.call("mset","k1__rand_int__","v1__rand_int__","k2__rand_int__","v2__rand_int__","k3__rand_int__","v3__rand_int__","k4__rand_int__","v4__rand_int__")' 0 benchmark 3: src/redis-benchmark -P 500 -r 1000 -n 100000 eval "for i=1,100,1 do redis.call('set','kk'..i,'vv'..__rand_int__) end return redis.call('get','kk5')" 0 benchmark 4: src/redis-benchmark -P 500 -r 1000 -n 1000000 eval 'return redis.call("mset","k1__rand_int__","v1__rand_int__","k2__rand_int__","v2__rand_int__","k3__rand_int__","v3__rand_int__","k4__rand_int__","v4__rand_int__xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx")' ``` I ran the benchmark on this branch with and without commit 68b71680a4d3bb8f0509e06578a9f15d05b92a47 Results in requests per second: cmd | without optimization | without optimization 2nd run | with original optimization | with original optimization 2nd run -- | -- | -- | -- | -- 1 | 461233.34 | 477395.31 | 471098.16 | 469946.91 2 | 34774.14 | 35469.8 | 35149.38 | 34464.93 3 | 6390.59 | 6281.41 | 6146.28 | 6464.12 4 | 28005.71 | | 27965.77 | As you can see, different use cases showed identical or negligible performance differences. So finally I decided to chuck the original optimization and simplify the code.
-
weiguo authored
-
- 06 Feb, 2022 2 commits
-
-
footpatch authored
when we fail opening `/proc`, we need to close the log file fd.
-
Binbin authored
`PSYNC replicationid str_offset` will crash the server. The reason is in `masterTryPartialResynchronization`, we will call `getLongLongFromObjectOrReply` check the offset. With a wrong offset, it will add a reply and then trigger a full SYNC and the client become a replica. So crash in `c->bufpos == 0 && listLength(c->reply) == 0`. In this commit, we check the psync_offset before entering function `masterTryPartialResynchronization`, and return. Regardless of that crash, accepting the sync, but also replying with an error would have corrupt the replication stream.
-
- 05 Feb, 2022 2 commits
-
-
Jason Elbaum authored
This is a followup to #9656 and implements the following step mentioned in that PR: * When possible, extract all the help and completion tips from COMMAND DOCS (Redis 7.0 and up) * If COMMAND DOCS fails, use the static help.h compiled into redis-cli. * Supplement additional command names from COMMAND (pre-Redis 7.0) The last step is needed to add module command and other non-standard commands. This PR does not change the interactive hinting mechanism, which still uses only the param strings to provide somewhat unreliable and inconsistent command hints (see #8084). That task is left for a future PR. Co-authored-by:
Oran Agra <oran@redislabs.com>
-
Binbin authored
1. Update fcall.json and fcall_ro.json 2. Update command.c 3. Update help.h
-
- 04 Feb, 2022 2 commits
-
-
Viktor Söderqvist authored
Adds RM_SetCommandInfo, allowing modules to provide the following command info: * summary * complexity * since * history * hints * arity * key specs * args This information affects the output of `COMMAND`, `COMMAND INFO` and `COMMAND DOCS`, Cluster, ACL and is used to filter commands with the wrong number of arguments before the call reaches the module code. The recently added API functions for key specs (never released) are removed. A minimalist example would look like so: ```c RedisModuleCommand *mycmd = RedisModule_GetCommand(ctx,"mymodule.mycommand"); RedisModuleCommandInfo mycmd_info = { .version = REDISMODULE_COMMAND_INFO_VERSION, .arity = -5, .summary = "some description", }; if (RedisModule_SetCommandInfo(mycmd, &mycmd_info) == REDISMODULE_ERR) return REDISMODULE_ERR; ```` Notes: * All the provided information (including strings) is copied, not keeping references to the API input data. * The version field is actually a static struct that contains the sizes of the the structs used in arrays, so we can extend these in the future and old version will still be able to take the part they can support.
-
Binbin authored
Change the sentinel config file to a directory in SENTINEL SET test. So it will now fail on the `rename` in `rewriteConfigOverwriteFile`. The test used to set the sentinel config file permissions to `000` to simulate failure. But it fails on centos7 / freebsd / alpine. (introduced in #10151) Other changes: 1. More error messages after the config rewrite failure. 2. Modify arg name `force_all` in `rewriteConfig` to `force_write`. (was rename in #9304) 3. Fix a typo in debug quicklist-packed-threshold, then -> than. (#9357)
-
- 03 Feb, 2022 3 commits
-
-
Wen Hui authored
When performing `SENTINEL SET`, Sentinel updates the local configuration file. Before this commit, failure to update the file would still result with an `+OK` reply. Now, a `-ERR Failed to save config file` error will be returned. Co-authored-by:
Yossi Gottlieb <yossigo@gmail.com>
-
Vo Trong Phuc authored
Add check enough good slaves for write command when evaluating scripts. This check is made before the script is executed, if we have function flags, and per redis command if we don't. Co-authored-by:
Phuc. Vo Trong <phucvt@vng.com.vn> Co-authored-by:
Oran Agra <oran@redislabs.com> Co-authored-by:
Meir Shpilraien (Spielrein) <meir@redis.com>
-
Viktor Söderqvist authored
The script which generates the markdown docs from module.c is updated to include the version in which each module API function was introduced. The script uses git tags to find this information. If git is not available or if we're not in a git repo, the 'since' is silently skipped. The line `**Available since:** (version)` is added after the function prototype Rename to utils/generate-module-api-doc.rb
-
- 02 Feb, 2022 1 commit
-
-
Meir Shpilraien (Spielrein) authored
update function help message, changed DESC->DESCRIPTION (doc was outdated)
-
- 01 Feb, 2022 2 commits
-
-
sundb authored
-
郭伟光 authored
This is done to avoid a crash when the timer fires after the module was unloaded. Or memory leaks in case we wanted to just ignore the timer. It'll cause the MODULE UNLOAD command to return with an error Co-authored-by:
sundb <sundbcn@gmail.com>
-
- 31 Jan, 2022 1 commit
-
-
Madelyn Olson authored
* The first field was previously only an IP address, it can now be a hostname or NULL. * There is now a fourth field, which has more information.
-
- 30 Jan, 2022 8 commits
-
-
Oran Agra authored
-
Ping Xie authored
Before this commit, notused1 was incorrectly resized resulting with a clusterMsg that is not backwards compatible as expected.
-
Moti Cohen authored
As Sentinel relies upon consensus algorithm, all sentinel instances, randomize a time to initiate their next attempt to become the leader of the group. But time after time, all raffled the same value. The problem is in the line `srand(time(NULL)^getpid())` such that all spinned up containers get same time (in seconds) and same pid which is always 1. Added material `tv_usec` and verify that even consecutive calls brings different values and makes the difference.
-
Tobias Nießen authored
Refs: https://github.com/redis/redis/pull/10141
-
Harkrishn Patro authored
For backwards compatibility in 6.x, channels default permission was set to `allchannels` however with 7.0, we should modify it and the default value should be `resetchannels` for better security posture. Also, with selectors in ACL, a client doesn't have to set channel rules everytime and by default the value will be `resetchannels`. Before this change ``` 127.0.0.1:6379> acl list 1) "user default on nopass ~* &* +@all" 127.0.0.1:6379> acl setuser hp on nopass +@all ~* OK 127.0.0.1:6379> acl list 1) "user default on nopass ~* &* +@all" 2) "user hp on nopass ~* &* +@all" 127.0.0.1:6379> acl setuser hp1 on nopass -@all (%R~sales*) OK 127.0.0.1:6379> acl list 1) "user default on nopass ~* &* +@all" 2) "user hp on nopass ~* &* +@all" 3) "user hp1 on nopass &* -@all (%R~sales* &* -@all)" ``` After this change ``` 127.0.0.1:6379> acl list 1) "user default on nopass ~* &* +@all" 127.0.0.1:6379> acl setuser hp on nopass +@all ~* OK 127.0.0.1:6379> acl list 1) "user default on nopass ~* &* +@all" 2) "user hp on nopass ~* resetchannels +@all" 127.0.0.1:6379> acl setuser hp1 on nopass -@all (%R~sales*) OK 127.0.0.1:6379> acl list 1) "user default on nopass ~* &* +@all" 2) "user hp on nopass ~* resetchannels +@all" 3) "user hp1 on nopass resetchannels -@all (%R~sales* resetchannels -@all)" ```
-
guybe7 authored
Add optional `notes` to keyspecs. Other changes: 1. Remove the "incomplete" flag from SORT and SORT_RO: it is misleading since "incomplete" means "this spec may not return all the keys it describes" but SORT and SORT_RO's specs (except the input key) do not return any keys at all. So basically: If a spec's begin_search is "unknown" you should not use it at all, you must use COMMAND KEYS; if a spec itself is "incomplete", you can use it to get a partial list of keys, but if you want all of them you must use COMMAND GETKEYS; otherwise, the spec will return all the keys 2. `getKeysUsingKeySpecs` handles incomplete specs internally
-
Oran Agra authored
Try to fix the rebalance cluster test that's failing with ASAN daily: Looks like `redis-cli --cluster rebalance` gets `ERR Please use SETSLOT only with masters` in `clusterManagerMoveSlot()`. it happens when `12-replica-migration-2.tcl` is run with ASAN in GH Actions. in `Resharding all the master #0 slots away from it` So the fix (assuming i got it right) is to call `redis-cli --cluster check` before `--cluster rebalance`. p.s. it looks like a few other checks in these tests needed that wait, added them too. Other changes: * in instances.tcl, make sure to catch tcl test crashes and let the rest of the code proceed, so that if there was a redis crash, we'll find it and print it too. * redis-cli, try to make sure it prints an error instead of silently exiting. specifically about redis-cli: 1. clusterManagerMoveSlot used to print an error, only if the caller also asked for it (should be the other way around). 2. clusterManagerCommandReshard asked for an error, but didn't use it (probably tried to avoid the double print). 3. clusterManagerCommandRebalance didn't ask for the error, now it does. 4. making sure that other places in clusterManagerCommandRebalance print something before exiting with an error.
-
Binbin authored
ZADD NX and XX was introduced in 3.0.2, not 6.2.0 ZADD GT and LT was introduced in 6.2.0, not 3.0.2 Add missing `COUNT ANY` history in georadius_ro Add missing `SHUTDOWN [NOW] [FORCE] [ABORT]` since in shutdown.json
-
- 29 Jan, 2022 3 commits
-
-
Moti Cohen authored
Sentinel tries to resolve instances hostname to IP only during registration. It might be that the instance is unavailable during that time, such as leader crashed and failover took place. Yet, promoted replica must support: - Register leader, even if it fails to resolve its hostname during failover - Try later to resolve it, if instance is disconnected. Note that this condition also support ip-change of an instance.
-
Binbin authored
Change the name to unix-time-seconds or unix-time-milliseconds to be consistent. Change the type from INTEGER to UNIX_TIME. SET (EXAT and PXAT) was already ok. and naming aside, both PXAT and EXAT everywhere used unit-time (for both milliseconds and seconds). the only ones that where wrong are GETEX and XCLAIM (using "integer" for both seconds and milliseconds)
-
weiguo authored
By a happy coincidence, sizeof(sds *) is equal to sizeof(sds) here, while it's logically consistent to use sizeof(sds) instead.
-
- 28 Jan, 2022 1 commit
-
-
Madelyn Olson authored
-
- 27 Jan, 2022 1 commit
-
-
Yossi Gottlieb authored
There are some inevitable changes between the old and new output, as a result of the different `commands.json` semantics.
-
- 26 Jan, 2022 2 commits
-
-
Binbin authored
SET is a R+W command, because it can also do `GET` on the data. SET without GET is a write-only command. SET with GET is a read+write command. In #9974, we added ACL to let users define write-only access. So when the user uses SET with GET option, and the user doesn't have the READ permission on the key, we need to reject it, but we rather not reject users with write-only permissions from using the SET command when they don't use GET. In this commit, we add a `getkeys_proc` function to control key flags in SET command. We also add a new key spec flag (VARIABLE_FLAGS) means that some keys might have different flags depending on arguments. We also handle BITFIELD command, add a `bitfieldGetKeys` function. BITFIELD GET is a READ ONLY command. BITFIELD SET or BITFIELD INCR are READ WRITE commands. Other changes: 1. SET GET was added in 6.2, add the missing since in set.json 2. Added tests to cover the changes in acl-v2.tcl 3. Fix some typos in server.h and cleanups in acl-v2.tcl Co-authored-by:
Madelyn Olson <madelyneolson@gmail.com> Co-authored-by:
Oran Agra <oran@redislabs.com>
-
guybe7 authored
-
- 25 Jan, 2022 5 commits
-
-
Viktor Söderqvist authored
Before: ``` 127.0.0.1:6379> command info get 1) 1) "get" 2) (integer) 2 3) 1~ readonly 2~ fast 4) (integer) 1 5) (integer) 1 6) (integer) 1 7) 1~ @read 2~ @string 3~ @fast 8) (empty set) 9) 1~ 1# "flags" => 1~ RO 2~ access 2# "begin_search" => 1# "type" => "index" 2# "spec" => 1# "index" => (integer) 1 3# "find_keys" => 1# "type" => "range" 2# "spec" => 1# "lastkey" => (integer) 0 2# "keystep" => (integer) 1 3# "limit" => (integer) 0 10) (empty set) ``` After: ``` 127.0.0.1:6379> command info get 1) 1) "get" 2) (integer) 2 3) 1~ readonly 2~ fast 4) (integer) 1 5) (integer) 1 6) (integer) 1 7) 1~ @read 2~ @string 3~ @fast 8) (empty set) 9) 1~ 1# "flags" => 1~ RO 2~ access 2# "begin_search" => 1# "type" => "index" 2# "spec" => 1# "index" => (integer) 1 3# "find_keys" => 1# "type" => "range" 2# "spec" => 1# "lastkey" => (integer) 0 2# "keystep" => (integer) 1 3# "limit" => (integer) 0 10) (empty set) ```
-
Meir Shpilraien (Spielrein) authored
Added the following statistics (per engine) to FUNCTION STATS command: * number of functions * number of libraries Output example: ``` > FUNCTION stats 1) "running_script" 2) (nil) 3) "engines" 4) 1) "LUA" 2) 1) "libraries_count" 2) (integer) 1 3) "functions_count" 4) (integer) 1 ``` To collect the stats, added a new dictionary to libraries_ctx that contains for each engine, the engine statistics representing the current libraries_ctx. Update the stats on: 1. Link library to libraries_ctx 2. Unlink library from libraries_ctx 3. Flushing libraries_ctx
-
chenyang8094 authored
-
Madelyn Olson authored
This PR aims to improve the flags associated with some commands and adds various tests around these cases. Specifically, it's concerned with commands which declare keys but have no ACL flags (think `EXISTS`), the user needs either read or write permission to access this type of key. This change is primarily concerned around commands in three categories: # General keyspace commands These commands are agnostic to the underlying data outside of side channel attacks, so they are not marked as ACCESS. * TOUCH * EXISTS * TYPE * OBJECT 'all subcommands' Note that TOUCH is not a write command, it could be a side effect of either a read or a write command. # Length and cardinality commands These commands are marked as NOT marked as ACCESS since they don't return actual user strings, just metadata. * LLEN * STRLEN * SCARD * HSTRLEN # Container has member commands These commands return information about the existence or metadata about the key. These commands are NOT marked as ACCESS since the check of membership is used widely in write commands e.g. the response of HSET. * SISMEMBER * HEXISTS # Intersection cardinality commands These commands are marked as ACCESS since they process data to compute the result. * PFCOUNT * ZCOUNT * ZINTERCARD * SINTERCARD
-
Madelyn Olson authored
Correctly handle variable arity checks in scripts
-
- 24 Jan, 2022 2 commits
-
-
yoav-steinberg authored
In #10025 we added a mechanism for flagging certain properties for Redis Functions. This lead us to think we'd like to "port" this mechanism to Redis Scripts (`EVAL`) as well. One good reason for this, other than the added functionality is because it addresses the poor behavior we currently have in `EVAL` in case the script performs a (non DENY_OOM) write operation during OOM state. See #8478 (And a previous attempt to handle it via #10093) for details. Note that in Redis Functions **all** write operations (including DEL) will return an error during OOM state unless the function is flagged as `allow-oom` in which case no OOM checking is performed at all. This PR: - Enables setting `EVAL` (and `SCRIPT LOAD`) script flags as defined in #10025. - Provides a syntactical framework via [shebang](https://en.wikipedia.org/wiki/Shebang_(Unix)) for additional script annotations and even engine selection (instead of just lua) for scripts. - Provides backwards compatibility so scripts without the new annotations will behave as they did before. - Appropriate tests. - Changes `EVAL[SHA]/_RO` to be flagged as `STALE` commands. This makes it possible to flag individual scripts as `allow-stale` or not flag them as such. In backwards compatibility mode these commands will return the `MASTERDOWN` error as before. - Changes `SCRIPT LOAD` to be flagged as a `STALE` command. This is mainly to make it logically compatible with the change to `EVAL` in the previous point. It enables loading a script on a stale server which is technically okay it doesn't relate directly to the server's dataset. Running the script does, but that won't work unless the script is explicitly marked as `allow-stale`. Note that even though the LUA syntax doesn't support hash tag comments `.lua` files do support a shebang tag on the top so they can be executed on Unix systems like any shell script. LUA's `luaL_loadfile` handles this as part of the LUA library. In the case of `luaL_loadbuffer`, which is what Redis uses, I needed to fix the input script in case of a shebang manually. I did this the same way `luaL_loadfile` does, by replacing the first line with a single line feed character.
-
Viktor Söderqvist authored
The keyspec API is not yet released and there is a plan to change it in #10108, which is going to be included in RC2. Therefore, we hide it in RC1 to avoid introducing a breaking change in RC2. Co-authored-by:
Oran Agra <oran@redislabs.com>
-
- 23 Jan, 2022 1 commit
-
-
Binbin authored
Summary of changes: 1. Rename `redisCommand->name` to `redisCommand->declared_name`, it is a const char * for native commands and SDS for module commands. 2. Store the [sub]command fullname in `redisCommand->fullname` (sds). 3. List subcommands in `ACL CAT` 4. List subcommands in `COMMAND LIST` 5. `moduleUnregisterCommands` now will also free the module subcommands. 6. RM_GetCurrentCommandName returns full command name Other changes: 1. Add `addReplyErrorArity` and `addReplyErrorExpireTime` 2. Remove `getFullCommandName` function that now is useless. 3. Some cleanups about `fullname` since now it is SDS. 4. Delete `populateSingleCommand` function from server.h that is useless. 5. Added tests to cover this change. 6. Add some module unload tests and fix the leaks 7. Make error messages uniform, make sure they always contain the full command name and that it's quoted. 7. Fixes some typos see the history in #9504, fixes #10124 Co-authored-by:
Oran Agra <oran@redislabs.com> Co-authored-by:
guybe7 <guy.benoish@redislabs.com>
-
- 22 Jan, 2022 1 commit
-
-
guybe7 authored
Recently we added extensive support for sub-commands in for redis 7.0, this meant that the old ACL mechanism for sub-commands wasn't needed, or actually was improved (to handle both include and exclude control, like for commands), but only for real sub-commands. The old mechanism in ACL was renamed to first-arg, and was able to match the first argument of any command (including sub-commands). We now realized that we might wanna completely delete that first-arg feature some day, so the first step was not to give it new capabilities in 7.0 and it didn't have before. Changes: 1. ACL: Block the first-arg mechanism on subcommands (we keep if in non-subcommands for backward compatibility) 2. COMMAND: When looking up a command, insist the command name doesn't contain extra words. Example: When a user issues `GET key` we want `lookupCommand` to return `getCommand` but when if COMMAND calls `lookupCommand` with `get|key` we want it to fail. Other changes: 1. ACLSetUser: prevent a redundant command lookup
-
- 20 Jan, 2022 1 commit
-
-
Madelyn Olson authored
* Implemented selectors which provide multiple different sets of permissions to users * Implemented key based permissions * Added a new ACL dry-run command to test permissions before execution * Updated module APIs to support checking key based permissions Co-authored-by:
Oran Agra <oran@redislabs.com>
-