1. 16 Dec, 2021 3 commits
  2. 15 Dec, 2021 4 commits
    • guybe7's avatar
      Auto-generate the command table from JSON files (#9656) · 86781600
      guybe7 authored
      Delete the hardcoded command table and replace it with an auto-generated table, based
      on a JSON file that describes the commands (each command must have a JSON file).
      
      These JSON files are the SSOT of everything there is to know about Redis commands,
      and it is reflected fully in COMMAND INFO.
      
      These JSON files are used to generate commands.c (using a python script), which is then
      committed to the repo and compiled.
      
      The purpose is:
      * Clients and proxies will be able to get much more info from redis, instead of relying on hard coded logic.
      * drop the dependency between Redis-user and the commands.json in redis-doc.
      * delete help.h and have redis-cli learn everything it needs to know just by issuing COMMAND (will be
        done in a separate PR)
      * redis.io should stop using commands.json and learn everything from Redis (ultimately one of the release
        artifacts should be a large JSON, containing all the information about all of the commands, which will be
        generated from COMMAND's reply)
      * the byproduct of this is:
        * module commands will be able to provide that info and possibly be more of a first-class citizens
        * in theory, one may be able to generate a redis client library for a strictly typed language, by using this info.
      
      ### Interface changes
      
      #### COMMAND INFO's reply change (and arg-less COMMAND)
      
      Before this commit the reply at index 7 contained the key-specs list
      and reply at index 8 contained the sub-commands list (Both unreleased).
      Now, reply at index 7 is a map of:
      - summary - short command description
      - since - debut version
      - group - command group
      - complexity - complexity string
      - doc-flags - flags used for documentation (e.g. "deprecated")
      - deprecated-since - if deprecated, from which version?
      - replaced-by - if deprecated, which command replaced it?
      - history - a list of (version, what-changed) tuples
      - hints - a list of strings, meant to provide hints for clients/proxies. see https://github.com/redis/redis/issues/9876
      - arguments - an array of arguments. each element is a map, with the possibility of nesting (sub-arguments)
      - key-specs - an array of keys specs (already in unstable, just changed location)
      - subcommands - a list of sub-commands (already in unstable, just changed location)
      - reply-schema - will be added in the future (see https://github.com/redis/redis/issues/9845)
      
      more details on these can be found in https://github.com/redis/redis-doc/pull/1697
      
      only the first three fields are mandatory 
      
      #### API changes (unreleased API obviously)
      
      now they take RedisModuleCommand opaque pointer instead of looking up the command by name
      
      - RM_CreateSubcommand
      - RM_AddCommandKeySpec
      - RM_SetCommandKeySpecBeginSearchIndex
      - RM_SetCommandKeySpecBeginSearchKeyword
      - RM_SetCommandKeySpecFindKeysRange
      - RM_SetCommandKeySpecFindKeysKeynum
      
      Currently, we did not add module API to provide additional information about their commands because
      we couldn't agree on how the API should look like, see https://github.com/redis/redis/issues/9944
      
      .
      
      ### Somehow related changes
      1. Literals should be in uppercase while placeholder in lowercase. Now all the GEO* command
         will be documented with M|KM|FT|MI and can take both lowercase and uppercase
      
      ### Unrelated changes
      1. Bugfix: no_madaory_keys was absent in COMMAND's reply
      2. expose CMD_MODULE as "module" via COMMAND
      3. have a dedicated uint64 for ACL categories (instead of having them in the same uint64 as command flags)
      Co-authored-by: default avatarItamar Haber <itamar@garantiadata.com>
      86781600
    • Meir Shpilraien (Spielrein)'s avatar
      Use `server.dirty++` instead if `forceCommandPropagation` on FUNCTION commands (#9945) · fbfdf513
      Meir Shpilraien (Spielrein) authored
      Functions are considered data, so changing a function should be counted
      as a data change and should affect the persistence policy. For example
      if we want to persist an RDB each minute if there was a single change,
      functions should be counted as such change. Using `forceCommandPropagation`
      will not give us the desired effect and so we must switch to `server.dirty++`
      fbfdf513
    • 丽媛自己动's avatar
    • Wen Hui's avatar
      Error message improvement for CONFIG SET command (#9924) · a09bc504
      Wen Hui authored
      When CONFIG SET fails, print the name of the config that failed.
      This is helpful since config set is now variadic.
      
      however, there are cases where several configs have the same apply
      function, and we can't be sure which one of them caused the failure.
      a09bc504
  3. 13 Dec, 2021 2 commits
    • yoav-steinberg's avatar
      Fix possible int overflow when hashing an sds. (#9916) · c7dc17fc
      yoav-steinberg authored
      This caused a crash when adding elements larger than 2GB to a set (same goes for hash keys). See #8455.
      
      Details:
      * The fix makes the dict hash functions receive a `size_t` instead of an `int`. In practice the dict hash functions
        call siphash which receives a `size_t` and the callers to the hash function pass a `size_t` to it so the fix is trivial.
      * The issue was recreated by attempting to add a >2gb value to a set. Appropriate tests were added where I create
        a set with large elements and check basic functionality on it (SADD, SCARD, SPOP, etc...).
      * When I added the tests I also refactored a bit all the tests code which is run under the `--large-memory` flag.
        This removed code duplication for the test framework's `write_big_bulk` and `write_big_bulk` code and also takes
        care of not allocating the test frameworks helper huge string used by these tests when not run under `--large-memory`.
      * I also added the _violoations.tcl_ unit tests to be part of the entire test suite and leaned up non relevant list related
        tests that were in there. This was done in this PR because most of the _violations_ tests are "large memory" tests.
      c7dc17fc
    • Madelyn Olson's avatar
  4. 10 Dec, 2021 1 commit
    • Binbin's avatar
      Fix timing issue in strem blocking tests (#9927) · b93ccee4
      Binbin authored
      A test failure was reported in Daily CI (FreeBSD).
      `XREAD: XADD + DEL should not awake client`
      
      ```
      *** [err]: XREAD: XADD + DEL should not awake client in tests/unit/type/stream.tcl
      Expected [lindex  0 0] eq {s1} (context: type eval line 11 cmd {assert {[lindex $res 0 0] eq {s1}}} proc ::test)
      ```
      
      It seems that `r` is executed before `rd` enters the blocking
      state. And ended up getting a empty reply by timeout.
      
      We use `wait_for_blocked_clients_count` to wait for the
      blocking client to be ready and avoid this situation.
      Also fixed other test cases that may have the same issue.
      b93ccee4
  5. 09 Dec, 2021 1 commit
  6. 08 Dec, 2021 5 commits
  7. 07 Dec, 2021 3 commits
    • 丽媛自己动's avatar
      0b26e389
    • yoav-steinberg's avatar
      Don't write oom score adj to proc unless we're managing it. (#9904) · 1736fa4d
      yoav-steinberg authored
      When disabling redis oom-score-adj managment we restore the
      base value read before enabling oom-score-adj management.
      
      This fixes an issue introduced in #9748 where updating
      `oom-score-adj-values` while `oom-score-adj` was set to `no`
      would write the base oom score adj value read on startup to `/proc`.
      This is a bug since while `oom-score-adj` is disabled we should
      never write to proc and let external processes manage it.
      
      Added appropriate tests.
      1736fa4d
    • Binbin's avatar
      Fix timing issue in logging.tcl with FreeBSD (#9910) · b947049f
      Binbin authored
      A test failure was reported in Daily CI.
      `Crash report generated on SIGABRT` with FreeBSD.
      
      ```
      *** [err]: Crash report generated on SIGABRT in tests/integration/logging.tcl
      Expected [string match *crashed by signal* ### Starting...(logs) in tests/integration/logging.tcl]
      ```
      
      It look like `tail -1000` was executed too early, before it
      printed out all the crash logs. We can give it a few more
      chances by using `wait_for_log_messages`.
      
      Other changes:
      1. In `Server is able to generate a stack trace on selected systems`,
      use `wait_for_log_messages`to reduce the lines of code. And if it
      fails, there are more detailed logs that can be printed.
      
      2. In `Crash report generated on DEBUG SEGFAULT`, we also use
      `wait_for_log_messages` to avoid possible timing issues.
      b947049f
  8. 06 Dec, 2021 1 commit
  9. 04 Dec, 2021 1 commit
  10. 02 Dec, 2021 4 commits
    • Oran Agra's avatar
      Merge Redis Functions PR (#9780) · 64f61596
      Oran Agra authored
      # Redis Function
      This PR added the Redis Functions capabilities that were suggested on #8693.
      The PR also introduce a big refactoring to the current Lua implementation
      (i.e `scripting.c`). The main purpose of the refactoring is to have better
      code sharing between the Lua implementation that exists today on Redis
      (`scripting.c`) and the new Lua engine that is introduced on this PR.
      The refactoring includes code movements and file name changes as well as some
      logic changes that need to be carefully reviewed. To make the review easier,
      the PR was split into multiple commits. Each commit is deeply described later on
      but the main concept is that some commits are just moving code around without
      making any logical changes, those commits are less likely to cause any issues
      or regressions and can be reviewed fast. Other commits, which perform code and
      logic changes, need to be reviewed carefully, but those commits were created
      after the code movements so it's pretty easy to see what was changed. To sum up,
      it is highly recommended to review this PR commit by commit as it will be easier
      to see the changes, it is also recommended to read each commit description
      (written below) to understand what was changed on the commit and whether or not
      it's just a huge code movement or a logic changes.
      
      ## Terminology
      Currently, the terminology in Redis is not clearly defined. Scripts refer to Lua
      scripts and eval also refers only to Lua. Introducing Redis Function requires
      redefining those terms to be able to clearly understand what is been discussed
      on each context.
      * eval - legacy Lua script implementation.
      * Function - new scripting implementation (currently implemented in Lua but in
        the future, it might be other languages like javascript).
      * Engine - the component that is responsible for executing functions.
      * Script - Function or legacy Lua (executed with `eval` or `evalsha`)
      
      ## Refactoring New Structure
      Today, the entire scripting logic is located on `scripting.c`. This logic can
      be split into 3 main groups:
      1. Script management - responsible for storing the scripts that were sent to
         Redis and retrieving them when they need to be run (base on the script sha
         on the current implementation).
      2. Script invocation - invoke the script given on `eval` or `evalsha` command
         (this part includes finding the relevant script, preparing the arguments, ..)
      3. Interact back with Redis (command invocation)
      
      Those 3 groups are tightly coupled on `scripting.c`. Redis Functions also need
      to use those groups logics, for example,  to interact back with Redis or to
      execute Lua code. The refactoring attempts to split those 3 groups and define
      APIs so that we can reuse the code both on legacy Lua scripts and Redis Functions.
      
      In order to do so we define the following units:
      1. script.c: responsible for interaction with Redis from within a script.
      2. script_lua.c: responsible to execute Lua code, uses `script.c` to interact
         with Redis from within the Lua code.
      3. function_lua.c: contains the Lua engine implementation, uses `script_lua.c`
         to execute the Lua code.
      4. functions.c: Contains Redis Functions implementation (`FUNCTION` command,),
         uses `functions_lua.c` if the function it wants to invoke needs the Lua
         engine.
      4. eval.c: the original `scripting.c` contains the Lua legacy implementation and
         was refactored to use `script_lua.c` to invoke the Lua code.
      
      ## Commits breakdown
      Notice: Some small commits are omitted from this list as they are small and
      insignificant (for example build fixes)
      
      ### First commit - code movements
      This commit rename `scripting.c` -> `eval.c` and introduce the new `script_lua.c`
      unit. The commit moves relevant code from `eval.c` (`scripting.c`) to
      `script_lua.c`, the purpose of moving the code is so that later we will be able
      to re-use the code on the Lua engine (`function_lua.c`). The commit only moves
      the code without modifying even a single line, so there is a very low risk of
      breaking anything and it also makes it much easier to see the changes on the
      following commits.
      Because the commit does not change the code (only moves it), it does not compile.
      But we do not care about it as the only purpose here is to make the review
      processes simpler.
      
      ### Second commit - move legacy Lua variables into `eval.c`
      Today, all Lua-related variables are located on the server struct. The commit
      attempt to identify those variable and take them out from the server struct,
      leaving only script related variables (variables that later need to be used
      also by engines)
      The following variable where renamed and left on the server struct:
         * lua_caller 			-> script_caller
         * lua_time_limit 		-> script_time_limit
         * lua_timedout 		-> script_timedout
         * lua_oom 			-> script_oom
         * lua_disable_deny_script 	-> script_disable_deny_script
         * in_eval			-> in_script
      
      The following variables where moved to lctx under eval.c
         * lua
         * lua_client
         * lua_cur_script
         * lua_scripts
         * lua_scripts_mem
         * lua_replicate_commands
         * lua_write_dirty
         * lua_random_dirty
         * lua_multi_emitted
         * lua_repl
         * lua_kill
         * lua_time_start
         * lua_time_snapshot
      
      This commit is in a low risk of introducing any issues and it is just moving
      variables around and not changing any logic.
      
      ### Third commit - introducing script unit
      This commit introduces the `script.c` unit. Its purpose (as described above) is
      to provide an API for scripts to interact with Redis. Interaction includes
      mostly executing commands, but also other functionalities. The interaction is
      done using a `ScriptRunCtx` object that needs to be created by the user and
      initialized using `scriptPrepareForRun`. A detailed list of functionalities
      expose by the unit:
      1. Calling commands (including all the validation checks such as
         acl, cluster, read only run, ...)
      2. Set Resp
      3. Set Replication method (AOF/REPLICATION/NONE)
      4. Call Redis back on long-running scripts to allow Redis to reply to clients
         and perform script kill
      
      The commit introduces the new unit and uses it on eval commands to interact with
      Redis.
      
      ### Fourth commit - Moved functionality of invoke Lua code to `script_lua.c`
      This commit moves the logic of invoking the Lua code into `script_lua.c` so
      later it can be used also by Lua engine (`function_lua.c`). The code is located
      on `callFunction` function and assumes the Lua function already located on the
      top of the Lua stack. This commit also change `eval.c` to use the new
      functionality to invoke Lua code.
      
      ### Fith commit - Added Redis Functions unit (`functions.c`) and Lua engine
      (`function_lua.c`)
      Added Redis Functions unit under `functions.c`, included:
      1. FUNCTION command:
           * FUNCTION CREATE
           * FUNCTION CALL
           * FUNCTION DELETE
           * FUNCTION KILL
           * FUNCTION INFO
           * FUNCTION STATS
      2. Register engines
      
      In addition, this commit introduces the first engine that uses the Redis
      Functions capabilities, the Lua engine (`function_lua.c`)
      
      ## API Changes
      ### `lua-time-limit`
      configuration was renamed to `script-time-limit` (keep `lua-time-limit` as alias
      for backward compatibility).
      
      ### Error log changes
      When integrating with Redis from within a Lua script, the `Lua` term was removed
      from all the error messages and instead we write only `script`. For example:
      `Wrong number of args calling Redis command From Lua script` -> `Wrong number
      of args calling Redis command From script`
      
      ### `info memory` changes:
      Before stating all the changes made to memory stats we will try to explain the
      reason behind them and what we want to see on those metrics:
      * memory metrics should show both totals (for all scripting frameworks), as well
        as a breakdown per framework / vm.
      * The totals metrics should have "human" metrics while the breakdown shouldn't.
      * We did try to maintain backward compatibility in some way, that said we did
        make some repurpose to existing metrics where it looks reasonable.
      * We separate between memory used by the script framework (part of redis's
        used_memory), and memory used by the VM (not part of redis's used_memory)
      
      A full breakdown of `info memory` changes:
      * `used_memory_lua` and `used_memory_lua_human` was deprecated,
        `used_memory_vm_eval` has the same meaning as `used_memory_lua`
      * `used_memory_scripts` was renamed to `used_memory_scripts_eval`
      * `used_memory_scripts` and `used_memory_scripts_human` were repurposed and now
        return the total memory used by functions and eval (not including vm memory,
        only code cache, and structs).
      * `used_memory_vm_function` was added and represents the total memory used by
        functions vm's
      * `used_memory_functions` was added and represents the total memory by functions
        (not including vm memory, only code cache, and structs)
      * `used_memory_vm_total` and `used_memory_vm_total_human` was added and
        represents the total memory used by vm's (functions and eval combined)
      
      ### `functions.caches`
      `functions.caches` field was added to `memory stats`, representing the memory
      used by engines that are not functions (this memory includes data structures
      like dictionaries, arrays, ...)
      
      ## New API
      ### FUNCTION CREATE
      
      Usage: FUNCTION CREATE `ENGINE` `NAME` `[REPLACE]` `[DESC <DESCRIPTION>]` `<CODE>`
      
      * `ENGINE` - The name of the engine to use to create the script.
      * `NAME` - the name of the function that can be used later to call the function
        using `FUNCTION CALL` command.
      * `REPLACE` - if given, replace the given function with the existing function
        (if exists).
      * `DESCRIPTION` - optional argument describing the function and what it does
      * `CODE` - function code.
      
      The command will return `OK` if created successfully or error in the following
      cases:
      * The given engine name does not exist
      * The function name is already taken and `REPLACE` was not used.
      * The given function failed on the compilation.
      
      ### FCALL and FCALL_RO
      
      Usage: FCALL/FCALL_RO `NAME` `NUM_KEYS key1 key2` … ` arg1 arg2`
      
      Call and execute the function specified by `NAME`. The function will receive
      all arguments given after `NUM_KEYS`. The return value from the function will
      be returned to the user as a result.
      
      * `NAME` - Name of the function to run.
      * The rest is as today with EVALSHA command.
      
      The command will return an error in the following cases:
      * `NAME` does not exist
      * The function itself returned an error.
      
      The `FCALL_RO` is equivalent to `EVAL_RO` and allows only read-only commands to
      be invoked from the script.
      
      ### FUNCTION DELETE
      
      Usage: FUNCTION DELETE `NAME`
      
      Delete a function identified by `NAME`. Return `OK` on success or error on one
      of the following:
      * The given function does not exist
      
      ### FUNCTION INFO
      
      Usage: FUNCTION INFO `NAME` [WITHCODE]
      
      Return information about a function by function name:
      * Function name
      * Engine name
      * Description
      * Raw code (only if WITHCODE argument is given)
      
      ### FUNCTION LIST
      
      Usage: FUNCTION LIST
      
      Return general information about all the functions:
      * Function name
      * Engine name
      * Description
      
      ### FUNCTION STATS
      
      Usage: FUNCTION STATS
      
      Return information about the current running function:
      * Function name
      * Command that was used to invoke the function
      * Duration in MS that the function is already running
      
      If no function is currently running, this section is just a RESP nil.
      
      Additionally, return a list of all the available engines.
      
      ### FUNCTION KILL
      
      Usage: `FUNCTION KILL`
      
      Kill the currently executing function. The command will fail if the function
      already initiated a write command.
      
      ## Notes
      Note: Function creation/deletion is replicated to AOF but AOFRW is not
      implemented sense its going to be removed: #9794
      64f61596
    • meir@redislabs.com's avatar
      Redis Functions - Added redis function unit and Lua engine · cbd46317
      meir@redislabs.com authored
      Redis function unit is located inside functions.c
      and contains Redis Function implementation:
      1. FUNCTION commands:
        * FUNCTION CREATE
        * FCALL
        * FCALL_RO
        * FUNCTION DELETE
        * FUNCTION KILL
        * FUNCTION INFO
      2. Register engine
      
      In addition, this commit introduce the first engine
      that uses the Redis Function capabilities, the
      Lua engine.
      cbd46317
    • Binbin's avatar
      Fix CONFIG SET test failures in MacOS/FreeBSD (#9881) · e57a4db5
      Binbin authored
      After the introduction of `Multiparam config set` in #9748,
      there are two tests cases failed.
      
      ```
      [exception]: Executing test client: ERR Config set failed - Failed to set current oom_score_adj. Check server logs..
      ERR Config set failed - Failed to set current oom_score_adj. Check server logs.
      ```
      
      `CONFIG sanity` test failed on the `config set oom-score-adj-values`
      which is a "special" config that does not catch no-op changes.
      And then it will update `oom-score-adj` which not supported in
      MacOs. We solve it by adding `oom-score*` to the `skip_configs` list.
      
      ```
      *** [err]: CONFIG SET rollback on apply error in tests/unit/introspection.tcl
      Expected an error but nothing was caught
      ```
      
      `CONFIG SET rollback on apply error` test failed on the
      `config set port $used_port`. In theory, it should throw the
      error `Unable to listen on this port*`. But it failed on MacOs.
      We solve it by adding `-myaddr 127.0.0.1` to the socket call.
      e57a4db5
    • Binbin's avatar
      Fix a harmless bug when using monitor in redis-cli with wrong reply (#9875) · e3c0ea1c
      Binbin authored
      When we use monitor in redis-cli but encounter an error reply,
      we will get stuck until we press Ctrl-C to quit.
      
      This is a harmless bug. It might be useful if we add parameters
      to monitor in the future, suck as monitoring only selected db.
      
      before:
      ```
      127.0.0.1:6379> monitor wrong
      (error) ERR wrong number of arguments for 'monitor' command or subcommand
      ^C(9.98s)
      127.0.0.1:6379>
      ```
      
      after:
      ```
      127.0.0.1:6379> monitor wrong
      (error) ERR wrong number of arguments for 'monitor' command or subcommand
      127.0.0.1:6379>
      ```
      e3c0ea1c
  11. 01 Dec, 2021 5 commits
    • meir@redislabs.com's avatar
      Redis Functions - Moved invoke Lua code functionality to script_lua.c · f21dc38a
      meir@redislabs.com authored
      The functionality was moved to script_lua.c under
      callFunction function. Its purpose is to call the Lua
      function already located on the top of the Lua stack.
      
      Used the new function on eval.c to invoke Lua code.
      The function will also be used to invoke Lua
      code on the Lua engine.
      f21dc38a
    • meir@redislabs.com's avatar
      Redis Functions - Introduce script unit. · fc731bc6
      meir@redislabs.com authored
      Script unit is a new unit located on script.c.
      Its purpose is to provides an API for functions (and eval)
      to interact with Redis. Interaction includes mostly
      executing commands, but also functionalities like calling
      Redis back on long scripts or check if the script was killed.
      
      The interaction is done using a scriptRunCtx object that
      need to be created by the user and initialized using scriptPrepareForRun.
      
      Detailed list of functionalities expose by the unit:
      1. Calling commands (including all the validation checks such as
         acl, cluster, read only run, ...)
      2. Set Resp
      3. Set Replication method (AOF/REPLICATION/NONE)
      4. Call Redis back to on long running scripts to allow Redis reply
         to clients and perform script kill
      
      The commit introduce the new unit and uses it on eval commands to
      interact with Redis.
      fc731bc6
    • meir@redislabs.com's avatar
      Redis Functions - Move Lua related variable into luaCtx struct · e0cd580a
      meir@redislabs.com authored
      The following variable was renamed:
      1. lua_caller 			-> script_caller
      2. lua_time_limit 		-> script_time_limit
      3. lua_timedout 		-> script_timedout
      4. lua_oom 			-> script_oom
      5. lua_disable_deny_script 	-> script_disable_deny_script
      6. in_eval			-> in_script
      
      The following variables was moved to lctx under eval.c
      1.  lua
      2.  lua_client
      3.  lua_cur_script
      4.  lua_scripts
      5.  lua_scripts_mem
      6.  lua_replicate_commands
      7.  lua_write_dirty
      8.  lua_random_dirty
      9.  lua_multi_emitted
      10. lua_repl
      11. lua_kill
      12. lua_time_start
      13. lua_time_snapshot
      
      This commit is in a low risk of introducing any issues and it
      is just moving varibales around and not changing any logic.
      e0cd580a
    • meir@redislabs.com's avatar
      Redis Functions - Move code to make review process easier. · 22aab1ce
      meir@redislabs.com authored
      This commit is only move code around without changing it.
      The reason behind this is to make review process easier
      by allowing the reviewer to simply ignore all code movements.
      
      changes:
      1. rename scripting.c to eval.c
      2. introduce and new file, script_lua.c, and move parts of Lua
         functionality to this new file. script_lua.c will eventually
         contains the shared code between legacy lua and lua engine.
      
      This commit does not compiled on purpose. Its only purpose is to move
      code and rename files.
      22aab1ce
    • yoav-steinberg's avatar
      Multiparam config set (#9748) · 0e5b813e
      yoav-steinberg authored
      We can now do: `config set maxmemory 10m repl-backlog-size 5m`
      
      ## Basic algorithm to support "transaction like" config sets:
      
      1. Backup all relevant current values (via get).
      2. Run "verify" and "set" on everything, if we fail run "restore".
      3. Run "apply" on everything (optional optimization: skip functions already run). If we fail run "restore".
      4. Return success.
      
      ### restore
      1. Run set on everything in backup. If we fail log it and continue (this puts us in an undefined
         state but we decided it's better than the alternative of panicking). This indicates either a bug
         or some unsupported external state.
      2. Run apply on everything in backup (optimization: skip functions already run). If we fail log
         it (see comment above).
      3. Return error.
      
      ## Implementation/design changes:
      * Apply function are idempotent (have no effect if they are run more than once for the same config).
      * No indication in set functions if we're reading the config or running from the `CONFIG SET` command
         (removed `update` argument).
      * Set function should set some config variable and assume an (optional) apply function will use that
         later to apply. If we know this setting can be safely applied immediately and can always be reverted
         and doesn't depend on any other configuration we can apply immediately from within the set function
         (and not store the setting anywhere). This is the case of this `dir` config, for example, which has no
         apply function. No apply function is need also in the case that setting the variable in the `server` struct
         is all that needs to be done to make the configuration take effect. Note that the original concept of `update_fn`,
         which received the old and new values was removed and replaced by the optional apply function.
      * Apply functions use settings written to the `server` struct and don't receive any inputs.
      * I take care that for the generic (non-special) configs if there's no change I avoid calling the setter (possible
         optimization: avoid calling the apply function as well).
      * Passing the same config parameter more than once to `config set` will fail. You can't do `config set my-setting
         value1 my-setting value2`.
      
      Note that getting `save` in the context of the conf file parsing to work here as before was a pain.
      The conf file supports an aggregate `save` definition, where each `save` line is added to the server's
      save params. This is unlike any other line in the config file where each line overwrites any previous
      configuration. Since we now support passing multiple save params in a single line (see top comments
      about `save` in https://github.com/redis/redis/pull/9644) we should deprecate the aggregate nature of
      this config line and perhaps reduce this ugly code in the future.
      0e5b813e
  12. 30 Nov, 2021 5 commits
    • Itamar Haber's avatar
      Adds auto-seq-only-generation via `XADD ... <ms>-*` (#9217) · 21aa1d4b
      Itamar Haber authored
      Adds the ability to autogenerate the sequence part of the millisecond-only explicit ID specified for `XADD`. This is useful in case added entries have an externally-provided timestamp without sub-millisecond resolution.
      21aa1d4b
    • Wen Hui's avatar
      Sentinel master reboot fix (#9438) · 2afa41f6
      Wen Hui authored
      Add master-reboot-down-after-period as a configurable parameter, to make it possible to trigger a failover from a master that is responding with `-LOADING` for a long time after being restarted.
      2afa41f6
    • 丽媛自己动's avatar
      modify misleading note in comment (#9865) · af072c26
      丽媛自己动 authored
      now rdbSaveInfo used in both way, so i think we should update previous notes, in case of misleading
      af072c26
    • Viktor Söderqvist's avatar
      bdf531e3
    • Meir Shpilraien (Spielrein)'s avatar
      Swap '\r\n' with spaces when returning a big number reply from Lua script. (#9870) · b8e82d20
      Meir Shpilraien (Spielrein) authored
      The issue can only happened with a bad Lua script that claims to return
      a big number while actually return data which is not a big number (contains
      chars that are not digits). Such thing will not cause an issue unless the big
      number value contains `\r\n` and then it messes the resp3 structure. The fix
      changes all the appearances of '\r\n' with spaces.
      
      Such an issue can also happened on simple string or error replies but those
      already handle it the same way this PR does (replace `\r\n` with spaces).
      
      Other replies type are not vulnerable to this issue because they are not
      counting on free text that is terminated with `\r\n` (either it contains the
      bulk length like string reply or they are typed reply that can not inject free
      text like boolean or number).
      
      The issue only exists on unstable branch, big number reply on Lua script
      was not yet added to any official release.
      b8e82d20
  13. 29 Nov, 2021 5 commits
    • Binbin's avatar
      Fix CLIENT KILL kill all clients with id 0 (#9853) · 3119a3ae
      Binbin authored
      
      
      * Fix CLIENT KILL kill all clients with id 0 or with skipme
      CLIENT KILL with ID argument should only kill the client with the provided ID. In old code, 
      CLIENT KILL with id 0 will kill all the connected clients.
      Co-authored-by: default avatarOfir Luzon <ofirluzon@gmail.com>
      3119a3ae
    • yoav-steinberg's avatar
      fix deprecation of _BSD_SOURCE feature test macro (#9861) · 2386e541
      yoav-steinberg authored
      caused a build warning in linenoise since glibc 2.20
      2386e541
    • OfirMos's avatar
      fixed mem leak on rdb load error (#9860) · 9f9c7857
      OfirMos authored
      a rare case of short read that can happen when breaking the master-replica
      connection on diskless load mode,
      9f9c7857
    • Binbin's avatar
      Add REDIS_CFLAGS='-Werror' to CI tests (#9828) · 980bb3ae
      Binbin authored
      Update CI so that warnings cause build failures.
      
      Also fix a warning in `test-sanitizer-address`:
      ```
      In function ‘strncpy’,
         inlined from ‘clusterUpdateMyselfIp’ at cluster.c:545:13:
      
      /usr/include/x86_64-linux-gnu/bits/string_fortified.h:106:10:
      error: ‘__builtin_strncpy’ specified bound 46 equals destination size [-Werror=stringop-truncation]
      
        106 |   return __builtin___strncpy_chk (__dest, __src, __len, __bos (__dest));
            |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      cc1: all warnings being treated as errors
      ```
      980bb3ae
    • leishiao's avatar
      improvement of a blocking xread test (#9859) · d56ded89
      leishiao authored
      
      
      This test relies on that `XREAD BLOCK 20000 STREAMS s1{t} s2{t} s3{t} $ $ $`
      is executed by redis before `XADD s2{t} * new abcd1234`. A ` wait_for_blocked_client`
      is needed between the two to ensure the order, otherwise `XADD s2{t} * new abcd1234`
      might be executed first due to network delay causing a test failure.
      Co-authored-by: default avatarxiaolei <xiaolei@91jkys.com>
      d56ded89