- 22 Dec, 2021 3 commits
-
-
Oran Agra authored
## background Till now CONFIG SET was blocked during loading. (In the not so distant past, GET was disallowed too) We recently (not released yet) added an async-loading mode, see #9323, and during that time it'll serve CONFIG SET and any other command. And now we realized (#9770) that some configs, and commands are dangerous during async-loading. ## changes * Allow most CONFIG SET during loading (both on async-loading and normal loading) * Allow CONFIG REWRITE and CONFIG RESETSTAT during loading * Block a few config during loading (`appendonly`, `repl-diskless-load`, and `dir`) * Block a few commands during loading (list below) ## the blocked commands: * SAVE - obviously we don't wanna start a foregreound save during loading 8-) * BGSAVE - we don't mind to schedule one, but we don't wanna fork now * BGREWRITEAOF - we don't mind to schedule one, but we don't wanna fork now * MODULE - we obviously don't wanna unload a module during replication / rdb loading (MODULE HELP and MODULE LIST are not blocked) * SYNC / PSYNC - we're in the middle of RDB loading from master, must not allow sync requests now. * REPLICAOF / SLAVEOF - we're in the middle of replicating, maybe it makes sense to let the user abort it, but he couldn't do that so far, i don't wanna take any risk of bugs due to odd state. * CLUSTER - only allow [HELP, SLOTS, NODES, INFO, MYID, LINKS, KEYSLOT, COUNTKEYSINSLOT, GETKEYSINSLOT, RESET, REPLICAS, COUNT_FAILURE_REPORTS], for others, preserve the status quo ## other fixes * processEventsWhileBlocked had an issue when being nested, this could happen with a busy script during async loading (new), but also in a busy script during AOF loading (old). this lead to a crash in the scenario described in #6988
-
zhugezy authored
If a test fails at `wait_for_blocked_clients_count` after the `PAUSE` command, It won't send `UNPAUSE` to server, leading to the server hanging until timeout, which is bad and hard to debug sometimes when developing. This PR tries to fix this. Timeout in `CLIENT PAUSE` shortened from 1e5 seconds(extremely long) to 50~100 seconds.
-
Hongcai Ren authored
when count is 0, return NULL
-
- 21 Dec, 2021 3 commits
-
-
Meir Shpilraien (Spielrein) authored
The issue with MAY_REPLICATE is that all automatic mechanisms to handle write commands will not work. This require have a special treatment for: * Not allow those commands to be executed on RO replica. * Allow those commands to be executed on RO replica from primary connection. * Allow those commands to be executed on the RO replica from AOF. By setting those commands as WRITE commands we are getting all those properties from Redis. Test was added to verify that those properties work as expected. In addition, rearrange when and where functions are flushed. Before this PR functions were flushed manually on `rdbLoadRio` and cleaned manually on failure. This contradicts the assumptions that functions are data and need to be created/deleted alongside with the data. A side effect of this, for example, `debug reload noflush` did not flush the data but did flush the functions, `debug loadaof` flush the data but not the functions. This PR move functions deletion into `emptyDb`. `emptyDb` (renamed to `emptyData`) will now accept an additional flag, `NOFUNCTIONS` which specifically indicate that we do not want to flush the functions (on all other cases, functions will be flushed). Used the new flag on FLUSHALL and FLUSHDB only! Tests were added to `debug reload` and `debug loadaof` to verify that functions behave the same as the data. Notice that because now functions will be deleted along side with the data we can not allow `CLUSTER RESET` to be called from within a function (it will cause the function to be released while running), this PR adds `NO_SCRIPT` flag to `CLUSTER RESET` so it will not be possible to be called from within a function. The other cluster commands are allowed from within a function (there are use-cases that uses `GETKEYSINSLOT` to iterate over all the keys on a given slot). Tests was added to verify `CLUSTER RESET` is denied from within a script. Another small change on this PR is that `RDBFLAGS_ALLOW_DUP` is also applicable on functions. When loading functions, if this flag is set, we will replace old functions with new ones on collisions.
-
yancz2000 authored
Let's you see which version of redis this tool is part of (similarly to redis-cli and redis-benchmark)
-
zhugezy authored
# Background The main goal of this PR is to remove relevant logics on Lua script verbatim replication, only keeping effects replication logic, which has been set as default since Redis 5.0. As a result, Lua in Redis 7.0 would be acting the same as Redis 6.0 with default configuration from users' point of view. There are lots of reasons to remove verbatim replication. Antirez has listed some of the benefits in Issue #5292: >1. No longer need to explain to users side effects into scripts. They can do whatever they want. >2. No need for a cache about scripts that we sent or not to the slaves. >3. No need to sort the output of certain commands inside scripts (SMEMBERS and others): this both simplifies and gains speed. >4. No need to store scripts inside the RDB file in order to startup correctly. >5. No problems about evicting keys during the script execution. When looking back at Redis 5.0, antirez and core team decided to set the config `lua-replicate-commands yes` by default instead of removing verbatim replication directly, in case some bad situations happened. 3 years later now before Redis 7.0, it's time to remove it formally. # Changes - configuration for lua-replicate-commands removed - created config file stub for backward compatibility - Replication script cache removed - this is useless under script effects replication - relevant statistics also removed - script persistence in RDB files is also removed - Propagation of SCRIPT LOAD and SCRIPT FLUSH to replica / AOF removed - Deterministic execution logic in scripts removed (i.e. don't run write commands after random ones, and sorting output of commands with random order) - the flags indicating which commands have non-deterministic results are kept as hints to clients. - `redis.replicate_commands()` & `redis.set_repl()` changed - now `redis.replicate_commands()` does nothing and return an 1 - ...and then `redis.set_repl()` can be issued before `redis.replicate_commands()` now - Relevant TCL cases adjusted - DEBUG lua-always-replicate-commands removed # Other changes - Fix a recent bug comparing CLIENT_ID_AOF to original_client->flags instead of id. (introduced in #9780) Co-authored-by:
Oran Agra <oran@redislabs.com>
-
- 20 Dec, 2021 1 commit
-
-
Binbin authored
Recent PRs have introduced some failures, this commit try to fix these CI failures. Here are the changes: 1. Enable debug-command in sentinel test. ``` Master reboot in very short time: ERR DEBUG command not allowed. If the enable-debug-command option is set to "local", you can run it from a local connection, otherwise you need to set this option in the configuration file, and then restart the server. ``` 2. Enable protected-config in sentinel test. ``` SDOWN is triggered by misconfigured instance replying with errors: ERR CONFIG SET failed (possibly related to argument 'dir') - can't set protected config ``` 3. Enable debug-command in cluster test. ``` Verify slaves consistency: ERR DEBUG command not allowed. If the enable-debug-command option is set to "local", you can run it from a local connection, otherwise you need to set this option in the configuration file, and then restart the server. ``` 4. quicklist fill should be signed int. The reason for the modification is to eliminate the warning. Modify `int fill: QL_FILL_BITS` to `signed int fill: QL_FILL_BITS` The first three were introduced at #9920 (same issue). And the last one was introduced at #9962.
-
- 19 Dec, 2021 3 commits
-
-
Binbin authored
1. Local variable 's' hides a parameter of the same name. ``` int anetTcpAccept(char *err, int s, char *ip, size_t ip_len, int *port) { if ((fd = anetGenericAccept(err,s,(struct sockaddr*)&sa,&salen)) == ANET_ERR){} } ``` Change the parameter name from `s` to `serversock`, also unified with the header file definition. 2. Comparison is always false because i <= 48. ``` for (i = 0; i < DICT_STATS_VECTLEN-1; i++) { // i < 49 (i == DICT_STATS_VECTLEN-1)?">= ":"", // i == 49 } ``` `i == DICT_STATS_VECTLEN-1` always result false, it is a dead code. 3. Empty block without comment. `else if (!strcasecmp(opt,"ch")) {}`, add a comment to avoid warnings. 4. Bit field fill of type int should have explicitly unsigned integral, explicitly signed integral, or enumeration type. Modify `int fill: QL_FILL_BITS;` to `unsigned int fill: QL_FILL_BITS;` 5. The result of this call to reconnectingRedisCommand is not checked for null, but 80% of calls to reconnectingRedisCommand check for null. Just a cleanup job like others.
-
Oran Agra authored
- add needs:debug flag for some tests - disable "save" in external tests (speedup?) - use debug_digest proc instead of debug command directly so it can be skipped - use OBJECT ENCODING instead of DEBUG OBJECT to get encoding - add a proc for OBJECT REFCOUNT so it can be skipped - move a bunch of tests in latency_monitor tests to happen later so that latency monitor has some values in it - add missing close_replication_stream calls - make sure to close the temp client if DEBUG LOG fails
-
YaacovHazan authored
Block sensitive configs and commands by default. * `enable-protected-configs` - block modification of configs with the new `PROTECTED_CONFIG` flag. Currently we add this flag to `dbfilename`, and `dir` configs, all of which are non-mutable configs that can set a file redis will write to. * `enable-debug-command` - block the `DEBUG` command * `enable-module-command` - block the `MODULE` command These have a default value set to `no`, so that these features are not exposed by default to client connections, and can only be set by modifying the config file. Users can change each of these to either `yes` (allow all access), or `local` (allow access from local TCP connections and unix domain connections) Note that this is a **breaking change** (specifically the part about MODULE command being disabled by default). I.e. we don't consider DEBUG command being blocked as an issue (people shouldn't have been using it), and the few configs we protected are unlikely to have been set at runtime anyway. On the other hand, it's likely to assume some users who use modules, load them from the config file anyway. Note that's the whole point of this PR, for redis to be more secure by default and reduce the attack surface on innocent users, so secure defaults will necessarily mean a breaking change.
-
- 18 Dec, 2021 1 commit
-
-
guybe7 authored
some languages can build a json-like object by parsing a textual json, but it works poorly when attributes contain hyphens example in JS: ``` let j = JSON.parse(json) j['key-name'] <- works j.key-name <= illegal syntax ```
-
- 17 Dec, 2021 1 commit
-
-
ny0312 authored
Introduce memory management on cluster link buffers: * Introduce a new `cluster-link-sendbuf-limit` config that caps memory usage of cluster bus link send buffers. * Introduce a new `CLUSTER LINKS` command that displays current TCP links to/from peers. * Introduce a new `mem_cluster_links` field under `INFO` command output, which displays the overall memory usage by all current cluster links. * Introduce a new `total_cluster_links_buffer_limit_exceeded` field under `CLUSTER INFO` command output, which displays the accumulated count of cluster links freed due to `cluster-link-sendbuf-limit`.
-
- 16 Dec, 2021 4 commits
-
-
Meir Shpilraien (Spielrein) authored
Added `FUNCTION FLUSH` command. The new sub-command allows delete all the functions. An optional `[SYNC|ASYNC]` argument can be given to control whether or not to flush the functions synchronously or asynchronously. if not given the default flush mode is chosen by `lazyfree-lazy-user-flush` configuration values. Add the missing `functions.tcl` test to the list of tests that are executed in test_helper.tcl, and call FUNCTION FLUSH in between servers in external mode
-
guybe7 authored
Sort the sub-commands so that every time we execute the script it generates the exact same results. This will case less merge conflicts if two PRs edit different json files. also: * make the script agnostic to where it is executed (more flexible). * add documentation about commands.c and the json files in the readme. Co-authored-by:
Oran Agra <oran@redislabs.com>
-
yoav-steinberg authored
Support doing `CONFIG GET <x> <y> <z>`, each of them can also be a pattern with wildcards. This avoids duplicates in the result by looping over the configs and for each once checking all the patterns, once a match is found for a pattern we move on to the next config.
-
ranshid authored
* Fix too long unix domain socket file path Co-authored-by:
Madelyn Olson <madelyneolson@gmail.com>
-
- 15 Dec, 2021 4 commits
-
-
guybe7 authored
Delete the hardcoded command table and replace it with an auto-generated table, based on a JSON file that describes the commands (each command must have a JSON file). These JSON files are the SSOT of everything there is to know about Redis commands, and it is reflected fully in COMMAND INFO. These JSON files are used to generate commands.c (using a python script), which is then committed to the repo and compiled. The purpose is: * Clients and proxies will be able to get much more info from redis, instead of relying on hard coded logic. * drop the dependency between Redis-user and the commands.json in redis-doc. * delete help.h and have redis-cli learn everything it needs to know just by issuing COMMAND (will be done in a separate PR) * redis.io should stop using commands.json and learn everything from Redis (ultimately one of the release artifacts should be a large JSON, containing all the information about all of the commands, which will be generated from COMMAND's reply) * the byproduct of this is: * module commands will be able to provide that info and possibly be more of a first-class citizens * in theory, one may be able to generate a redis client library for a strictly typed language, by using this info. ### Interface changes #### COMMAND INFO's reply change (and arg-less COMMAND) Before this commit the reply at index 7 contained the key-specs list and reply at index 8 contained the sub-commands list (Both unreleased). Now, reply at index 7 is a map of: - summary - short command description - since - debut version - group - command group - complexity - complexity string - doc-flags - flags used for documentation (e.g. "deprecated") - deprecated-since - if deprecated, from which version? - replaced-by - if deprecated, which command replaced it? - history - a list of (version, what-changed) tuples - hints - a list of strings, meant to provide hints for clients/proxies. see https://github.com/redis/redis/issues/9876 - arguments - an array of arguments. each element is a map, with the possibility of nesting (sub-arguments) - key-specs - an array of keys specs (already in unstable, just changed location) - subcommands - a list of sub-commands (already in unstable, just changed location) - reply-schema - will be added in the future (see https://github.com/redis/redis/issues/9845) more details on these can be found in https://github.com/redis/redis-doc/pull/1697 only the first three fields are mandatory #### API changes (unreleased API obviously) now they take RedisModuleCommand opaque pointer instead of looking up the command by name - RM_CreateSubcommand - RM_AddCommandKeySpec - RM_SetCommandKeySpecBeginSearchIndex - RM_SetCommandKeySpecBeginSearchKeyword - RM_SetCommandKeySpecFindKeysRange - RM_SetCommandKeySpecFindKeysKeynum Currently, we did not add module API to provide additional information about their commands because we couldn't agree on how the API should look like, see https://github.com/redis/redis/issues/9944 . ### Somehow related changes 1. Literals should be in uppercase while placeholder in lowercase. Now all the GEO* command will be documented with M|KM|FT|MI and can take both lowercase and uppercase ### Unrelated changes 1. Bugfix: no_madaory_keys was absent in COMMAND's reply 2. expose CMD_MODULE as "module" via COMMAND 3. have a dedicated uint64 for ACL categories (instead of having them in the same uint64 as command flags) Co-authored-by:
Itamar Haber <itamar@garantiadata.com>
-
Meir Shpilraien (Spielrein) authored
Functions are considered data, so changing a function should be counted as a data change and should affect the persistence policy. For example if we want to persist an RDB each minute if there was a single change, functions should be counted as such change. Using `forceCommandPropagation` will not give us the desired effect and so we must switch to `server.dirty++`
-
丽媛自己动 authored
to do here
-
Wen Hui authored
When CONFIG SET fails, print the name of the config that failed. This is helpful since config set is now variadic. however, there are cases where several configs have the same apply function, and we can't be sure which one of them caused the failure.
-
- 13 Dec, 2021 2 commits
-
-
yoav-steinberg authored
This caused a crash when adding elements larger than 2GB to a set (same goes for hash keys). See #8455. Details: * The fix makes the dict hash functions receive a `size_t` instead of an `int`. In practice the dict hash functions call siphash which receives a `size_t` and the callers to the hash function pass a `size_t` to it so the fix is trivial. * The issue was recreated by attempting to add a >2gb value to a set. Appropriate tests were added where I create a set with large elements and check basic functionality on it (SADD, SCARD, SPOP, etc...). * When I added the tests I also refactored a bit all the tests code which is run under the `--large-memory` flag. This removed code duplication for the test framework's `write_big_bulk` and `write_big_bulk` code and also takes care of not allocating the test frameworks helper huge string used by these tests when not run under `--large-memory`. * I also added the _violoations.tcl_ unit tests to be part of the entire test suite and leaned up non relevant list related tests that were in there. This was done in this PR because most of the _violations_ tests are "large memory" tests.
-
Madelyn Olson authored
-
- 10 Dec, 2021 1 commit
-
-
Binbin authored
A test failure was reported in Daily CI (FreeBSD). `XREAD: XADD + DEL should not awake client` ``` *** [err]: XREAD: XADD + DEL should not awake client in tests/unit/type/stream.tcl Expected [lindex 0 0] eq {s1} (context: type eval line 11 cmd {assert {[lindex $res 0 0] eq {s1}}} proc ::test) ``` It seems that `r` is executed before `rd` enters the blocking state. And ended up getting a empty reply by timeout. We use `wait_for_blocked_clients_count` to wait for the blocking client to be ready and avoid this situation. Also fixed other test cases that may have the same issue.
-
- 09 Dec, 2021 1 commit
-
-
Wen Hui authored
When clients enter multiply parameters in the CONFIG SET command, the server side log displays the wrong config parameters
-
- 08 Dec, 2021 5 commits
-
-
sundb authored
When rdb creates a consumer without determining whether it exists in advance, it may return NULL and crash if it encounters corrupt data with duplicate consumers.
-
yoav-steinberg authored
Added `HIDDEN_CONFIG` to hide debug / dev / testing configs from CONFIG GET when it is used with a wildcard. These are not documented in redis.conf so now CONFIG GET only works when they are explicitly specified. The current configs are: ``` key-load-delay loading-process-events-interval-bytes rdb-key-save-delay use-exit-on-panic watchdog-period ```
-
Binbin authored
For `SENTINEL SET`, we can use in these ways: 1. SENTINEL SET mymaster quorum 3 2. SENTINEL SET mymaster quorum 5 parallel-syncs 1 For `SENTINEL SIMULATE-FAILURE`, although it is only used for testing: 1. SENTINEL SIMULATE-FAILURE CRASH-AFTER-ELECTION 2. SENTINEL SIMULATE-FAILURE CRASH-AFTER-ELECTION CRASH-AFTER-PROMOTION
-
leishiao authored
Co-authored-by:
xiaolei <xiaolei@91jkys.com>
-
- 07 Dec, 2021 3 commits
-
-
丽媛自己动 authored
-
yoav-steinberg authored
When disabling redis oom-score-adj managment we restore the base value read before enabling oom-score-adj management. This fixes an issue introduced in #9748 where updating `oom-score-adj-values` while `oom-score-adj` was set to `no` would write the base oom score adj value read on startup to `/proc`. This is a bug since while `oom-score-adj` is disabled we should never write to proc and let external processes manage it. Added appropriate tests.
-
Binbin authored
A test failure was reported in Daily CI. `Crash report generated on SIGABRT` with FreeBSD. ``` *** [err]: Crash report generated on SIGABRT in tests/integration/logging.tcl Expected [string match *crashed by signal* ### Starting...(logs) in tests/integration/logging.tcl] ``` It look like `tail -1000` was executed too early, before it printed out all the crash logs. We can give it a few more chances by using `wait_for_log_messages`. Other changes: 1. In `Server is able to generate a stack trace on selected systems`, use `wait_for_log_messages`to reduce the lines of code. And if it fails, there are more detailed logs that can be printed. 2. In `Crash report generated on DEBUG SEGFAULT`, we also use `wait_for_log_messages` to avoid possible timing issues.
-
- 06 Dec, 2021 1 commit
-
-
Madelyn Olson authored
-
- 04 Dec, 2021 1 commit
-
-
sundb authored
When an invalid listpack entry starts with EOF, we will skip it when we verify it in the loop.
-
- 02 Dec, 2021 4 commits
-
-
Oran Agra authored
# Redis Function This PR added the Redis Functions capabilities that were suggested on #8693. The PR also introduce a big refactoring to the current Lua implementation (i.e `scripting.c`). The main purpose of the refactoring is to have better code sharing between the Lua implementation that exists today on Redis (`scripting.c`) and the new Lua engine that is introduced on this PR. The refactoring includes code movements and file name changes as well as some logic changes that need to be carefully reviewed. To make the review easier, the PR was split into multiple commits. Each commit is deeply described later on but the main concept is that some commits are just moving code around without making any logical changes, those commits are less likely to cause any issues or regressions and can be reviewed fast. Other commits, which perform code and logic changes, need to be reviewed carefully, but those commits were created after the code movements so it's pretty easy to see what was changed. To sum up, it is highly recommended to review this PR commit by commit as it will be easier to see the changes, it is also recommended to read each commit description (written below) to understand what was changed on the commit and whether or not it's just a huge code movement or a logic changes. ## Terminology Currently, the terminology in Redis is not clearly defined. Scripts refer to Lua scripts and eval also refers only to Lua. Introducing Redis Function requires redefining those terms to be able to clearly understand what is been discussed on each context. * eval - legacy Lua script implementation. * Function - new scripting implementation (currently implemented in Lua but in the future, it might be other languages like javascript). * Engine - the component that is responsible for executing functions. * Script - Function or legacy Lua (executed with `eval` or `evalsha`) ## Refactoring New Structure Today, the entire scripting logic is located on `scripting.c`. This logic can be split into 3 main groups: 1. Script management - responsible for storing the scripts that were sent to Redis and retrieving them when they need to be run (base on the script sha on the current implementation). 2. Script invocation - invoke the script given on `eval` or `evalsha` command (this part includes finding the relevant script, preparing the arguments, ..) 3. Interact back with Redis (command invocation) Those 3 groups are tightly coupled on `scripting.c`. Redis Functions also need to use those groups logics, for example, to interact back with Redis or to execute Lua code. The refactoring attempts to split those 3 groups and define APIs so that we can reuse the code both on legacy Lua scripts and Redis Functions. In order to do so we define the following units: 1. script.c: responsible for interaction with Redis from within a script. 2. script_lua.c: responsible to execute Lua code, uses `script.c` to interact with Redis from within the Lua code. 3. function_lua.c: contains the Lua engine implementation, uses `script_lua.c` to execute the Lua code. 4. functions.c: Contains Redis Functions implementation (`FUNCTION` command,), uses `functions_lua.c` if the function it wants to invoke needs the Lua engine. 4. eval.c: the original `scripting.c` contains the Lua legacy implementation and was refactored to use `script_lua.c` to invoke the Lua code. ## Commits breakdown Notice: Some small commits are omitted from this list as they are small and insignificant (for example build fixes) ### First commit - code movements This commit rename `scripting.c` -> `eval.c` and introduce the new `script_lua.c` unit. The commit moves relevant code from `eval.c` (`scripting.c`) to `script_lua.c`, the purpose of moving the code is so that later we will be able to re-use the code on the Lua engine (`function_lua.c`). The commit only moves the code without modifying even a single line, so there is a very low risk of breaking anything and it also makes it much easier to see the changes on the following commits. Because the commit does not change the code (only moves it), it does not compile. But we do not care about it as the only purpose here is to make the review processes simpler. ### Second commit - move legacy Lua variables into `eval.c` Today, all Lua-related variables are located on the server struct. The commit attempt to identify those variable and take them out from the server struct, leaving only script related variables (variables that later need to be used also by engines) The following variable where renamed and left on the server struct: * lua_caller -> script_caller * lua_time_limit -> script_time_limit * lua_timedout -> script_timedout * lua_oom -> script_oom * lua_disable_deny_script -> script_disable_deny_script * in_eval -> in_script The following variables where moved to lctx under eval.c * lua * lua_client * lua_cur_script * lua_scripts * lua_scripts_mem * lua_replicate_commands * lua_write_dirty * lua_random_dirty * lua_multi_emitted * lua_repl * lua_kill * lua_time_start * lua_time_snapshot This commit is in a low risk of introducing any issues and it is just moving variables around and not changing any logic. ### Third commit - introducing script unit This commit introduces the `script.c` unit. Its purpose (as described above) is to provide an API for scripts to interact with Redis. Interaction includes mostly executing commands, but also other functionalities. The interaction is done using a `ScriptRunCtx` object that needs to be created by the user and initialized using `scriptPrepareForRun`. A detailed list of functionalities expose by the unit: 1. Calling commands (including all the validation checks such as acl, cluster, read only run, ...) 2. Set Resp 3. Set Replication method (AOF/REPLICATION/NONE) 4. Call Redis back on long-running scripts to allow Redis to reply to clients and perform script kill The commit introduces the new unit and uses it on eval commands to interact with Redis. ### Fourth commit - Moved functionality of invoke Lua code to `script_lua.c` This commit moves the logic of invoking the Lua code into `script_lua.c` so later it can be used also by Lua engine (`function_lua.c`). The code is located on `callFunction` function and assumes the Lua function already located on the top of the Lua stack. This commit also change `eval.c` to use the new functionality to invoke Lua code. ### Fith commit - Added Redis Functions unit (`functions.c`) and Lua engine (`function_lua.c`) Added Redis Functions unit under `functions.c`, included: 1. FUNCTION command: * FUNCTION CREATE * FUNCTION CALL * FUNCTION DELETE * FUNCTION KILL * FUNCTION INFO * FUNCTION STATS 2. Register engines In addition, this commit introduces the first engine that uses the Redis Functions capabilities, the Lua engine (`function_lua.c`) ## API Changes ### `lua-time-limit` configuration was renamed to `script-time-limit` (keep `lua-time-limit` as alias for backward compatibility). ### Error log changes When integrating with Redis from within a Lua script, the `Lua` term was removed from all the error messages and instead we write only `script`. For example: `Wrong number of args calling Redis command From Lua script` -> `Wrong number of args calling Redis command From script` ### `info memory` changes: Before stating all the changes made to memory stats we will try to explain the reason behind them and what we want to see on those metrics: * memory metrics should show both totals (for all scripting frameworks), as well as a breakdown per framework / vm. * The totals metrics should have "human" metrics while the breakdown shouldn't. * We did try to maintain backward compatibility in some way, that said we did make some repurpose to existing metrics where it looks reasonable. * We separate between memory used by the script framework (part of redis's used_memory), and memory used by the VM (not part of redis's used_memory) A full breakdown of `info memory` changes: * `used_memory_lua` and `used_memory_lua_human` was deprecated, `used_memory_vm_eval` has the same meaning as `used_memory_lua` * `used_memory_scripts` was renamed to `used_memory_scripts_eval` * `used_memory_scripts` and `used_memory_scripts_human` were repurposed and now return the total memory used by functions and eval (not including vm memory, only code cache, and structs). * `used_memory_vm_function` was added and represents the total memory used by functions vm's * `used_memory_functions` was added and represents the total memory by functions (not including vm memory, only code cache, and structs) * `used_memory_vm_total` and `used_memory_vm_total_human` was added and represents the total memory used by vm's (functions and eval combined) ### `functions.caches` `functions.caches` field was added to `memory stats`, representing the memory used by engines that are not functions (this memory includes data structures like dictionaries, arrays, ...) ## New API ### FUNCTION CREATE Usage: FUNCTION CREATE `ENGINE` `NAME` `[REPLACE]` `[DESC <DESCRIPTION>]` `<CODE>` * `ENGINE` - The name of the engine to use to create the script. * `NAME` - the name of the function that can be used later to call the function using `FUNCTION CALL` command. * `REPLACE` - if given, replace the given function with the existing function (if exists). * `DESCRIPTION` - optional argument describing the function and what it does * `CODE` - function code. The command will return `OK` if created successfully or error in the following cases: * The given engine name does not exist * The function name is already taken and `REPLACE` was not used. * The given function failed on the compilation. ### FCALL and FCALL_RO Usage: FCALL/FCALL_RO `NAME` `NUM_KEYS key1 key2` … ` arg1 arg2` Call and execute the function specified by `NAME`. The function will receive all arguments given after `NUM_KEYS`. The return value from the function will be returned to the user as a result. * `NAME` - Name of the function to run. * The rest is as today with EVALSHA command. The command will return an error in the following cases: * `NAME` does not exist * The function itself returned an error. The `FCALL_RO` is equivalent to `EVAL_RO` and allows only read-only commands to be invoked from the script. ### FUNCTION DELETE Usage: FUNCTION DELETE `NAME` Delete a function identified by `NAME`. Return `OK` on success or error on one of the following: * The given function does not exist ### FUNCTION INFO Usage: FUNCTION INFO `NAME` [WITHCODE] Return information about a function by function name: * Function name * Engine name * Description * Raw code (only if WITHCODE argument is given) ### FUNCTION LIST Usage: FUNCTION LIST Return general information about all the functions: * Function name * Engine name * Description ### FUNCTION STATS Usage: FUNCTION STATS Return information about the current running function: * Function name * Command that was used to invoke the function * Duration in MS that the function is already running If no function is currently running, this section is just a RESP nil. Additionally, return a list of all the available engines. ### FUNCTION KILL Usage: `FUNCTION KILL` Kill the currently executing function. The command will fail if the function already initiated a write command. ## Notes Note: Function creation/deletion is replicated to AOF but AOFRW is not implemented sense its going to be removed: #9794
-
meir@redislabs.com authored
Redis function unit is located inside functions.c and contains Redis Function implementation: 1. FUNCTION commands: * FUNCTION CREATE * FCALL * FCALL_RO * FUNCTION DELETE * FUNCTION KILL * FUNCTION INFO 2. Register engine In addition, this commit introduce the first engine that uses the Redis Function capabilities, the Lua engine.
-
Binbin authored
After the introduction of `Multiparam config set` in #9748, there are two tests cases failed. ``` [exception]: Executing test client: ERR Config set failed - Failed to set current oom_score_adj. Check server logs.. ERR Config set failed - Failed to set current oom_score_adj. Check server logs. ``` `CONFIG sanity` test failed on the `config set oom-score-adj-values` which is a "special" config that does not catch no-op changes. And then it will update `oom-score-adj` which not supported in MacOs. We solve it by adding `oom-score*` to the `skip_configs` list. ``` *** [err]: CONFIG SET rollback on apply error in tests/unit/introspection.tcl Expected an error but nothing was caught ``` `CONFIG SET rollback on apply error` test failed on the `config set port $used_port`. In theory, it should throw the error `Unable to listen on this port*`. But it failed on MacOs. We solve it by adding `-myaddr 127.0.0.1` to the socket call.
-
Binbin authored
When we use monitor in redis-cli but encounter an error reply, we will get stuck until we press Ctrl-C to quit. This is a harmless bug. It might be useful if we add parameters to monitor in the future, suck as monitoring only selected db. before: ``` 127.0.0.1:6379> monitor wrong (error) ERR wrong number of arguments for 'monitor' command or subcommand ^C(9.98s) 127.0.0.1:6379> ``` after: ``` 127.0.0.1:6379> monitor wrong (error) ERR wrong number of arguments for 'monitor' command or subcommand 127.0.0.1:6379> ```
-
- 01 Dec, 2021 2 commits
-
-
meir@redislabs.com authored
The functionality was moved to script_lua.c under callFunction function. Its purpose is to call the Lua function already located on the top of the Lua stack. Used the new function on eval.c to invoke Lua code. The function will also be used to invoke Lua code on the Lua engine.
-
meir@redislabs.com authored
Script unit is a new unit located on script.c. Its purpose is to provides an API for functions (and eval) to interact with Redis. Interaction includes mostly executing commands, but also functionalities like calling Redis back on long scripts or check if the script was killed. The interaction is done using a scriptRunCtx object that need to be created by the user and initialized using scriptPrepareForRun. Detailed list of functionalities expose by the unit: 1. Calling commands (including all the validation checks such as acl, cluster, read only run, ...) 2. Set Resp 3. Set Replication method (AOF/REPLICATION/NONE) 4. Call Redis back to on long running scripts to allow Redis reply to clients and perform script kill The commit introduce the new unit and uses it on eval commands to interact with Redis.
-