1. 06 Apr, 2023 2 commits
  2. 28 Mar, 2023 3 commits
  3. 27 Mar, 2023 1 commit
    • Rafi Einstein's avatar
      Clang: fix for -flto argument (#11961) · 557ca05d
      Rafi Einstein authored
      Starting with the recent #11926 Makefile specifies `-flto=auto` which is unsupported on clang.
      Additionally, detecting clang correctly requires actually running it, since on MacOS gcc can be an alias for clang.
      557ca05d
  4. 26 Mar, 2023 2 commits
    • Binbin's avatar
      Fix redis-cli cluster test timing issue (#11887) · aa2403ca
      Binbin authored
      This test fails sporadically:
      ```
      *** [err]: Migrate the last slot away from a node using redis-cli in tests/unit/cluster/cli.tcl
      cluster size did not reach a consistent size 4
      ```
      
      I guess the time (5s) of wait_for_cluster_size is not enough,
      usually, the waiting time for our other tests for cluster
      consistency is 50s, so also changing it to 50s.
      aa2403ca
    • Binbin's avatar
      Add COMMAND COUNT test to cover reply-schemas-validator test (#11971) · 2cc99c69
      Binbin authored
      Since we remove the COMMAND COUNT call in sentinel test in #11950,
      reply-schemas-validator started reporting this error:
      ```
      WARNING! The following commands were not hit at all:
        command|count
        ERROR! at least one command was not hit by the tests
      ```
      
      This PR add a COMMAND COUNT test to cover it and also fix some
      typos in req-res-log-validator.py
      2cc99c69
  5. 24 Mar, 2023 2 commits
  6. 23 Mar, 2023 3 commits
  7. 22 Mar, 2023 6 commits
  8. 21 Mar, 2023 10 commits
    • Vitaly Arbuzov's avatar
      Refactor dbIteratorNext · 9269c410
      Vitaly Arbuzov authored
      9269c410
    • Vitaly Arbuzov's avatar
      6a10146f
    • Vitaly Arbuzov's avatar
      Cleanup spacing · 41fab4ec
      Vitaly Arbuzov authored
      41fab4ec
    • Vitaly Arbuzov's avatar
      Remove whitespace formatting from aof.c · 57c98d27
      Vitaly Arbuzov authored
      57c98d27
    • Vitaly Arbuzov's avatar
      297922e6
    • Vitaly Arbuzov's avatar
      72935b9d
    • Vitaly Arbuzov's avatar
      Remove empty line · 6baf20af
      Vitaly Arbuzov authored
      6baf20af
    • Roshan Khatri's avatar
      Module commands to have ACL categories. (#11708) · 6948daca
      Roshan Khatri authored
      
      
      This allows modules to register commands to existing ACL categories and blocks the creation of [sub]commands, datatypes and registering the configs outside of the OnLoad function.
      
      For allowing modules to register commands to existing ACL categories,
      This PR implements a new API int RM_SetCommandACLCategories() which takes a pointer to a RedisModuleCommand and a C string aclflags containing the set of space separated ACL categories.
      Example, 'write slow' marks the command as part of the write and slow ACL categories.
      
      The C string aclflags is tokenized by implementing a helper function categoryFlagsFromString(). Theses tokens are matched and the corresponding ACL categories flags are set by a helper function matchAclCategoriesFlags. The helper function categoryFlagsFromString() returns the corresponding categories_flags or returns -1 if some token not processed correctly.
      
      If the module contains commands which are registered to existing ACL categories, the number of [sub]commands are tracked by num_commands_with_acl_categories in struct RedisModule. Further, the allowed command bit-map of the existing users are recomputed from the command_rules list, by implementing a function called ACLRecomputeCommandBitsFromCommandRulesAllUsers() for the existing users to have access to the module commands on runtime.
      
      ## Breaking change
      This change requires that registering commands and subcommands only occur during a modules "OnLoad" function, in order to allow efficient recompilation of ACL bits. We also chose to block registering configs and types, since we believe it's only valid for those to be created during onLoad. We check for this onload flag in struct RedisModule to check if the call is made from the OnLoad function.
      Co-authored-by: default avatarMadelyn Olson <madelyneolson@gmail.com>
      6948daca
    • Binbin's avatar
      Fix race in temp rdb delete shutdown test (#11840) · 78f15b7e
      Binbin authored
      I saw this error once, in the FreeBSD Daily CI:
      ```
      *** [err]: Temp rdb will be deleted if we use bg_unlink when shutdown in tests/unit/shutdown.tcl
      Expected [file exists /xxx/temp-10336.rdb] (context: type eval line 15 cmd {assert {[file exists $temp_rdb]}} proc ::test)
      ```
      
      The log shows that bgsave was executed, and it was successfully executed in the end:
      ```
      Starting test Temp rdb will be deleted if we use bg_unlink when shutdown in tests/unit/shutdown.tcl
      10251:M 22 Feb 2023 11:37:25.441 * Background saving started by pid 10336
      10336:C 22 Feb 2023 11:37:27.949 * DB saved on disk
      10336:C 22 Feb 2023 11:37:27.949 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
      10251:M 22 Feb 2023 11:37:28.060 * Background saving terminated with success
      ```
      
      There may be two reasons:
      1. The child process has been created, but it has not created
         the temp rdb file yet, so [file exists $temp_rdb] check failed.
      2. The child process bgsave has been executed successfully and the
         temp file has been deleted, so [file exists $temp_rdb] check failed.
      
      From the logs pint, it should be the case 2, case 1 is too extreme,
      set rdb-key-save-delay to a higher value to ensure bgsave does not
      succeed early to avoid this case.
      78f15b7e
    • Binbin's avatar
      Add missing master_reboot flag in sentinel instance info (#11888) · 1cb4b1ad
      Binbin authored
      SRI_MASTER_REBOOT flag was added in #9438
      1cb4b1ad
  9. 20 Mar, 2023 4 commits
    • Oran Agra's avatar
      Avoid assertion when MSETNX is used with the same key twice (CVE-2023-28425) (#11940) · 48e0d478
      Oran Agra authored
      Using the same key twice in MSETNX command would trigger an assertion.
      
      This reverts #11594 (introduced in Redis 7.0.8)
      48e0d478
    • Binbin's avatar
      Fix new subscribe mode test in reply-schemas-validator (#11939) · c9124145
      Binbin authored
      The reason is in reply-schemas-validator, the resp of the
      client we create will be client_default_resp (currently 3):
      ```
      client *createClient(connection *conn) {
          client *c = zmalloc(sizeof(client));
       #ifdef LOG_REQ_RES
          reqresReset(c, 0);
          c->resp = server.client_default_resp;
       #else
          c->resp = 2;
       #endif
      }
      ```
      
      But current_resp3 in redis-cli will be inconsistent with it,
      the test adds a simple hello 3 to avoid this failure, test
      was added in #11873.
      
      Added help descriptions for dont-pre-clean option, it was
      added in #10273
      c9124145
    • polaris-alioth's avatar
      passwords printed in the crash log (#11930) · 56eef6fb
      polaris-alioth authored
      When the server crashes during the AUTH command, or another command with
      an AUTH argument, the password was recorded in the log.
      
      Now, when the `auth` keyword is detected (could be in HELLO or MIGRATE, etc),
      the loop exits before printing any additional arguments.
      56eef6fb
    • Shaya Potter's avatar
      Don't run command filter on blocked command reprocessing (#11895) · 6cf8fc08
      Shaya Potter authored
      
      
      Previously we would run the module command filters even upon blocked
      command reprocessing.  This could modify the command, and it's args.
      This is irrelevant in the context of a command being reprocessed (it already
      went through the filters), as well as breaks the crashed command lookup
      that exists in the case of a reprocessed command.
      
      fixes #11894.
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      6cf8fc08
  10. 19 Mar, 2023 2 commits
    • Viktor Söderqvist's avatar
      redis-cli: Accept commands in subscribed mode (#11873) · bbf364a4
      Viktor Söderqvist authored
      The message "Reading messages... (press Ctrl-C to quit)" is replaced by
      "Reading messages... (press Ctrl-C to quit or any key to type command)".
      
      This allows users to subscribe to more channels, to try out UNSUBSCRIBE and to
      combine pubsub with other features such as push messages from client tracking.
      
      The "Reading messages" info message is displayed in the bottom of the output in a
      distinct style and moves downward as more messages appear. When any key is pressed,
      the info message is replaced by the prompt with for entering commands.
      After entering a command and the reply is displayed, the "Reading messages" info
      messages appears again. This is added to the repl loop in redis-cli and in the
      corresponding place for non-interactive mode.
      
      An indication "(subscribed mode)" is included in the prompt when entering commands
      in subscribed mode.
      
      Also:
      * Fixes a problem that UNSUBSCRIBE hanged when used with RESP3 and push callback,
        without first entering subscribe mode. It hanged because UNSUBSCRIBE gets one or
        more push replies but no in-band reply.
      * Exit subscribed mode after RESET.
      bbf364a4
    • Wang Yuan's avatar
      Remove unnecessary `fsync` when sentinel flushs config file (#11910) · c9466b24
      Wang Yuan authored
      `rewriteConfig` already calls `fsync` to make sure changes are committed to disk.
      so it is no need to call `fsync` again here.
      this was added here when rewriteConfigOverwriteFile used the ftruncate approach and didn't fsync
      c9466b24
  11. 17 Mar, 2023 3 commits
    • Rong Tao's avatar
      Fix compile lto-wrapper warning for aarch64 (#11926) · d6910983
      Rong Tao authored
      
      
      Use -flto=auto to use GNU make's job server, if available, or otherwise fall
      back to autodetection of the number of CPU threads present in your system.
      
        Warnings:
      
        lto-wrapper: warning: using serial compilation of 2 LTRANS jobs
        lto-wrapper: note: see the ‘-flto’ option documentation for more information
        lto-wrapper: warning: using serial compilation of 4 LTRANS jobs
        lto-wrapper: note: see the ‘-flto’ option documentation for more information
        lto-wrapper: warning: using serial compilation of 31 LTRANS jobs
        lto-wrapper: note: see the ‘-flto’ option documentation for more information
      Signed-off-by: default avatarRong Tao <rongtao@cestc.cn>
      d6910983
    • Binbin's avatar
      Minor fix to print, set to str (#11934) · 2dd5c3a1
      Binbin authored
      * Minor fix to print, set to str
      
      `{commands_filename}` the extra {} actually make it
      become a Set, and the output print was like this:
      ```
      Processing json files...
      Linking container command to subcommands...
      Checking all commands...
      Generating {'commands'}.c...
      All done, exiting.
      ```
      
      Introduced in #11920
      
      * more fix
      2dd5c3a1
    • Vitaly Arbuzov's avatar
      e6e1853f
  12. 16 Mar, 2023 2 commits
    • Meir Shpilraien (Spielrein)'s avatar
      Support for RM_Call on blocking commands (#11568) · d0da0a6a
      Meir Shpilraien (Spielrein) authored
      Allow running blocking commands from within a module using `RM_Call`.
      
      Today, when `RM_Call` is used, the fake client that is used to run command
      is marked with `CLIENT_DENY_BLOCKING` flag. This flag tells the command
      that it is not allowed to block the client and in case it needs to block, it must
      fallback to some alternative (either return error or perform some default behavior).
      For example, `BLPOP` fallback to simple `LPOP` if it is not allowed to block.
      
      All the commands must respect the `CLIENT_DENY_BLOCKING` flag (including
      module commands). When the command invocation finished, Redis asserts that
      the client was not blocked.
      
      This PR introduces the ability to call blocking command using `RM_Call` by
      passing a callback that will be called when the client will get unblocked.
      In order to do that, the user must explicitly say that he allow to perform blocking
      command by passing a new format specifier argument, `K`, to the `RM_Call`
      function. This new flag will tell Redis that it is allow to run blocking command
      and block the client. In case the command got blocked, Redis will return a new
      type of call reply (`REDISMODULE_REPLY_PROMISE`). This call reply indicates
      that the command got blocked and the user can set the on_unblocked handler using
      `RM_CallReplyPromiseSetUnblockHandler`.
      
      When clients gets unblocked, it eventually reaches `processUnblockedClients` function.
      This is where we check if the client is a fake module client and if it is, we call the unblock
      callback instead of performing the usual unblock operations.
      
      **Notice**: `RM_CallReplyPromiseSetUnblockHandler` must be called atomically
      along side the command invocation (without releasing the Redis lock in between).
      In addition, unlike other CallReply types, the promise call reply must be released
      by the module when the Redis GIL is acquired.
      
      The module can abort the execution on the blocking command (if it was not yet
      executed) using `RM_CallReplyPromiseAbort`. the API will return `REDISMODULE_OK`
      on success and `REDISMODULE_ERR` if the operation is already executed.
      **Notice** that in case of misbehave module, Abort might finished successfully but the
      operation will not really be aborted. This can only happened if the module do not respect
      the disconnect callback of the blocked client. 
      For pure Redis commands this can not happened.
      
      ### Atomicity Guarantees
      
      The API promise that the unblock handler will run atomically as an execution unit.
      This means that all the operation performed on the unblock handler will be wrapped
      with a multi exec transaction when replicated to the replica and AOF.
      The API **do not** grantee any other atomicity properties such as when the unblock
      handler will be called. This gives us the flexibility to strengthen the grantees (or not)
      in the future if we will decide that we need a better guarantees.
      
      That said, the implementation **does** provide a better guarantees when performing
      pure Redis blocking command like `BLPOP`. In this case the unblock handler will run
      atomically with the operation that got unblocked (for example, in case of `BLPOP`, the
      unblock handler will run atomically with the `LPOP` operation that run when the command
      got unblocked). This is an implementation detail that might be change in the future and the
      module writer should not count on that.
      
      ### Calling blocking commands while running on script mode (`S`)
      
      `RM_Call` script mode (`S`) was introduced on #0372. It is used for usecases where the
      command that was invoked on `RM_Call` comes from a user input and we want to make
      sure the user will not run dangerous commands like `shutdown`. Some command, such
      as `BLPOP`, are marked with `NO_SCRIPT` flag, which means they will not be allowed on
      script mode. Those commands are marked with  `NO_SCRIPT` just because they are
      blocking commands and not because they are dangerous. Now that we can run blocking
      commands on RM_Call, there is no real reason not to allow such commands on script mode.
      
      The underline problem is that the `NO_SCRIPT` flag is abused to also mark some of the
      blocking commands (notice that those commands know not to block the client if it is not
      allowed to do so, and have a fallback logic to such cases. So even if those commands
      were not marked with `NO_SCRIPT` flag, it would not harm Redis, and today we can
      already run those commands within multi exec).
      
      In addition, not all blocking commands are marked with `NO_SCRIPT` flag, for example
      `blmpop` are not marked and can run from within a script.
      
      Those facts shows that there are some ambiguity about the meaning of the `NO_SCRIPT`
      flag, and its not fully clear where it should be use.
      
      The PR suggest that blocking commands should not be marked with `NO_SCRIPT` flag,
      those commands should handle `CLIENT_DENY_BLOCKING` flag and only block when
      it's safe (like they already does today). To achieve that, the PR removes the `NO_SCRIPT`
      flag from the following commands:
      * `blmove`
      * `blpop`
      * `brpop`
      * `brpoplpush`
      * `bzpopmax`
      * `bzpopmin`
      * `wait`
      
      This might be considered a breaking change as now, on scripts, instead of getting
      `command is not allowed from script` error, the user will get some fallback behavior
      base on the command implementation. That said, the change matches the behavior
      of scripts and multi exec with respect to those commands and allow running them on
      `RM_Call` even when script mode is used.
      
      ### Additional RedisModule API and changes
      
      * `RM_BlockClientSetPrivateData` - Set private data on the blocked client without the
        need to unblock the client. This allows up to set the promise CallReply as the private
        data of the blocked client and abort it if the client gets disconnected.
      * `RM_BlockClientGetPrivateData` - Return the current private data set on a blocked client.
        We need it so we will have access to this private data on the disconnect callback.
      * On RM_Call, the returned reply will be added to the auto memory context only if auto
        memory is enabled, this allows us to keep the call reply for longer time then the context
        lifetime and does not force an unneeded borrow relationship between the CallReply and
        the RedisModuleContext.
      d0da0a6a
    • Binbin's avatar
      Fix usleep compilation warning in auth.c (#11925) · 484b73a8
      Binbin authored
      There is a -Wimplicit-function-declaration warning in here:
      ```
      auth.c: In function ‘AuthBlock_ThreadMain’:
      auth.c:116:5: warning: implicit declaration of function ‘usleep’; did you mean ‘sleep’? [-Wimplicit-function-declaration]
        116 |     usleep(500000);
            |     ^~~~~~
            |     sleep
      ```
      484b73a8