1. 28 Aug, 2022 1 commit
  2. 22 Aug, 2022 1 commit
  3. 21 Aug, 2022 2 commits
  4. 18 Aug, 2022 1 commit
    • guybe7's avatar
      Repurpose redisCommandArg's name as the unique ID (#11051) · 223046ec
      guybe7 authored
      This PR makes sure that "name" is unique for all arguments in the same
      level (i.e. all args of a command and all args within a block/oneof).
      This means several argument with identical meaning can be referred to together,
      but also if someone needs to refer to a specific one, they can use its full path.
      
      In addition, the "display_text" field has been added, to be used by redis.io
      in order to render the syntax of the command (for the vast majority it is
      identical to "name" but sometimes we want to use a different string
      that is not "name")
      The "display" field is exposed via COMMAND DOCS and will be present
      for every argument, except "oneof" and "block" (which are container
      arguments)
      
      Other changes:
      1. Make sure we do not have any container arguments ("oneof" or "block")
         that contain less than two sub-args (otherwise it doesn't make sense)
      2. migrate.json: both AUTH and AUTH2 should not be "optional"
      3. arg names cannot contain un...
      223046ec
  5. 01 Aug, 2022 1 commit
    • Rudi Floren's avatar
      Fix wrong commands json docs for CLIENT KILL (#10970) · 4ce3fd51
      Rudi Floren authored
      The docs state that there is a new and an old argument format.
      The current state of the arguments allows mixing the old and new format,
      thus the need for two additional oneof blocks.
      One for differentiating the new from the old format and then one to
      allow setting multiple filters using the new format.
      4ce3fd51
  6. 24 Jul, 2022 1 commit
  7. 20 Jul, 2022 1 commit
  8. 19 Jul, 2022 1 commit
  9. 17 Jul, 2022 1 commit
  10. 30 Jun, 2022 1 commit
    • Binbin's avatar
      Add SENTINEL command flag to CLIENT/COMMANDS subcommands (#10904) · 35e836c2
      Binbin authored
      This was harmless because we marked the parent command
      with SENTINEL flag. So the populateCommandTable was ok.
      And we also don't show the flag (SENTINEL and ONLY-SENTNEL)
      in COMMAND INFO.
      
      In this PR, we also add the same CMD_SENTINEL and CMD_ONLY_SENTINEL
      flags check when populating the sub-commands.
      so that in the future it'll be possible to add some sub-commands to sentinel or sentinel-only but not others.
      35e836c2
  11. 29 Jun, 2022 1 commit
  12. 25 Jun, 2022 1 commit
    • Steve Lorello's avatar
      changing min,max in ZRANGE -> start stop (#10097) · a64b2948
      Steve Lorello authored
      In 6.2.0 with the introduction of the REV subcommand in ZRANGE, there was a semantic shift in the arguments of ZRANGE when the REV sub-command is executed. Without the sub-command `min` and `max` (the old names of the arguments) are appropriate because if you put the min value and the max value in everything works fine.
      
      ```bash
      127.0.0.1:6379> ZADD myset 0 foo
      (integer) 1
      127.0.0.1:6379> ZADD myset 1 bar
      (integer) 1
      127.0.0.1:6379> ZRANGE myset 0 inf BYSCORE
      1) "foo"
      2) "bar"
      ``` 
      
      However - if you add the `REV` subcommand, ordering the arguments `min` `max` breaks the command:
      
      ```bash
      127.0.0.1:6379> ZRANGE myset 0 inf BYSCORE REV
      (empty array)
      ```
      
      why? because `ZRANGE` with the `REV` sub-command is expecting the `max` first and the `min` second (because it's a reverse range like `ZREVRANGEBYSCORE`):
      
      ```bash
      127.0.0.1:6379> ZRANGE myset 0 inf BYSCORE REV
      (empty array)
      ```
      a64b2948
  13. 23 Jun, 2022 1 commit
    • Binbin's avatar
      Add bus_port argument in cluster-meet.json (#10304) · aabce893
      Binbin authored
      In `CLUSTER MEET`, bus-port argument was added in 11436b14.
      For cluster announce ip / port implementation, part of the
      4.0-RC1.
      
      And in #9389, we add a new cluster-port config and make
      cluster bus port configurable, part of the 7.0-RC1.
      aabce893
  14. 14 Jun, 2022 1 commit
    • Oran Agra's avatar
      Allow ECHO in loading and stale modes (#10853) · ffa00770
      Oran Agra authored
      I noticed that scripting.tcl uses INFO from within a script and thought it's an
      overkill and concluded it's nicer to use another CMD_STALE command,
      decided to use ECHO, and then noticed it's not at all allowed in stale mode.
      probably overlooked at #6843
      ffa00770
  15. 07 Jun, 2022 3 commits
    • Bjorn Svensson's avatar
      Documentation fixes of `BITFIELD_RO` and `XINFO STREAM` (#10823) · 1067cfb3
      Bjorn Svensson authored
      Correcting the introduction version of the command `BITFIELD_RO`
      Command added by commit: b3e4abf0 
      
      Add history info of the `FULL` modifier in `XINFO STREAM`
      Modifier was added by commit: 1e2aee39
      
      (Includes output from `./utils/generate-command-code.py`)
      1067cfb3
    • Binbin's avatar
      Handle multiple_token flag in generate-command-help.rb (#10822) · 3d56f607
      Binbin authored
      Currently generate-command.help.rb dose not handle the
      multiple_token flag, handle this flag in this PR.
      The format is the same as redis-cli rendering.
      ```diff
      - bitfield_ro key GET encoding offset [encoding offset ...]
      + bitfield_ro key GET encoding offset [GET encoding offset ...]
      ```
      
      Re run generate-command-code.py which was forget in #10820.
      Also change the flag value from string to bool, like "true" to true
      3d56f607
    • ranshid's avatar
      Fix bitfield_ro documentation (#10820) · b4772f63
      ranshid authored
      Current documentation only states a single GET token is needed which is not true. Marking the command with multiple_token.
      b4772f63
  16. 31 May, 2022 1 commit
  17. 30 May, 2022 1 commit
  18. 27 May, 2022 1 commit
    • Binbin's avatar
      Fix some commands key spec in json files (#10779) · 2a099d49
      Binbin authored
      There are some commands that has the wrong key specs.
      This PR adds a key-spec related check in generate-command-code.py.
      Check if the index is valid, or if there is an unused index.
      
      The check result will look like:
      ```
      [root]# python utils/generate-command-code.py
      Processing json files...
      Linking container command to subcommands...
      Checking all commands...
      command: RESTORE_ASKING may have unused key_spec
      command: RENAME may have unused key_spec
      command: PFDEBUG may have unused key_spec
      command: WATCH key_specs missing flags
      command: LCS arg: key2 key_spec_index error
      command: RENAMENX may have unused key_spec
      Error: There are errors in the commands check, please check the above logs.
      ```
      
      The following commands have been fixed according to the check results:
      - RESTORE ASKING: add missing arguments section (and history section)
      - RENAME: newkey's key_spec_index should be 1
      - PFDEBUG: add missing arguments (and change the arity from -3 to 3)
      - WATCH: add missing key_specs flags: RO, like EXIST (it allow you to know the key exists, or is modified, but doesn't "leak" the data)
      - LCS: key2 key_spec_index error, there is only one key-spec
      - RENAMENX: newkey's key_spec_index should be 1
      2a099d49
  19. 13 May, 2022 1 commit
  20. 13 Apr, 2022 1 commit
    • Wen Hui's avatar
      Fix several document error and function comments (#10580) · ca913a5d
      Wen Hui authored
      
      
      This PR fix the following minor errors before Redis 7 release:
      
      ZRANGEBYLEX command in deprecated in 6.2.0, and could be replaced by ZRANGE with the
      BYLEX argument, but in the document, the words is written incorrect in " by ZRANGE with the BYSCORE argument"
      
      Fix function zpopmaxCommand incorrect comment
      
      The comments of function zmpopCommand and bzmpopCommand are not consistent with document description, fix them
      Co-authored-by: default avatarUbuntu <lucas.guang.yang1@huawei.com>
      ca913a5d
  21. 06 Apr, 2022 1 commit
    • Itamar Haber's avatar
      Fixes commands' syntices (#10534) · 3e09a8c0
      Itamar Haber authored
      Fixes in command argument in json files
      * Fixes BITFIELD's syntax ("sub-commands" can be repeated, and OVERFLOW is only valid for SET and INCR)
      * Improves readability of SET (reordered)
      * Fixes GEOSEARCH and GEOSEARCH_RO syntices (use `oneof` for mutually exclusive group instead of `optional`)
      * Fixes MIGRATE syntax (use `oneof` for mutually exclusive group instead of `optional`)
      * Fixes MODULE LOADEX syntax (the `CONFIG` token should be repeated too when using multiple configs)
      
      other:
      * make generate-command-help.rb accept a path to commands.json, or read it from stdin (e.g. `generate-commands-json.py | generate-command-help.rb -`)
      3e09a8c0
  22. 05 Apr, 2022 2 commits
    • Madelyn Olson's avatar
      Update json command files so they only include syntax related information (#10398) · 4ffcec29
      Madelyn Olson authored
      The command json documents should just include information about the "arguments" and the "outputs".
      I removed all of the 'functional wording' so it's clear.
      4ffcec29
    • Meir Shpilraien (Spielrein)'s avatar
      Functions: Move library meta data to be part of the library payload. (#10500) · ae020e3d
      Meir Shpilraien (Spielrein) authored
      ## Move library meta data to be part of the library payload.
      
      Following the discussion on https://github.com/redis/redis/issues/10429 and the intention to add (in the future) library versioning support, we believe that the entire library metadata (like name and engine) should be part of the library payload and not provided by the `FUNCTION LOAD` command. The reasoning behind this is that the programmer who developed the library should be the one who set those values (name, engine, and in the future also version). **It is not the responsibility of the admin who load the library into the database.**
      
      The PR moves all the library metadata (engine and function name) to be part of the library payload. The metadata needs to be provided on the first line of the payload using the shebang format (`#!<engine> name=<name>`), example:
      
      ```lua
      #!lua name=test
      redis.register_function('foo', function() return 1 end)
      ```
      
      The above script will run on the Lua engine and will create a library called `test`.
      
      ## API Changes (compare to 7.0 rc2)
      
      * `FUNCTION LOAD` command was change and now it simply gets the library payload and extract the engine and name from the payload. In addition, the command will now return the function name which can later be used on `FUNCTION DELETE` and `FUNCTION LIST`.
      * The description field was completely removed from`FUNCTION LOAD`, and `FUNCTION LIST`
      
      
      ## Breaking Changes (compare to 7.0 rc2)
      
      * Library description was removed (we can re-add it in the future either as part of the shebang line or an additional line).
      * Loading an AOF file that was generated by either 7.0 rc1 or 7.0 rc2 will fail because the old command syntax is invalid.
      
      ## Notes
      
      * Loading an RDB file that was generated by rc1 / rc2 **is** supported, Redis will automatically add the shebang to the libraries payloads (we can probably delete that code after 7.0.3 or so since there's no need to keep supporting upgrades from an RC build).
      ae020e3d
  23. 30 Mar, 2022 2 commits
    • Nick Chun's avatar
      Module Configurations (#10285) · bda9d74d
      Nick Chun authored
      
      
      This feature adds the ability to add four different types (Bool, Numeric,
      String, Enum) of configurations to a module to be accessed via the redis
      config file, and the CONFIG command.
      
      **Configuration Names**:
      
      We impose a restriction that a module configuration always starts with the
      module name and contains a '.' followed by the config name. If a module passes
      "config1" as the name to a register function, it will be registered as MODULENAME.config1.
      
      **Configuration Persistence**:
      
      Module Configurations exist only as long as a module is loaded. If a module is
      unloaded, the configurations are removed.
      There is now also a minimal core API for removal of standardConfig objects
      from configs by name.
      
      **Get and Set Callbacks**:
      
      Storage of config values is owned by the module that registers them, and provides
      callbacks for Redis to access and manipulate the values.
      This is exposed through a GET and SET callback.
      
      The get callback returns a typed value of the config to redis. The callback takes
      the name of the configuration, and also a privdata pointer. Note that these only
      take the CONFIGNAME portion of the config, not the entire MODULENAME.CONFIGNAME.
      
      ```
       typedef RedisModuleString * (*RedisModuleConfigGetStringFunc)(const char *name, void *privdata);
       typedef long long (*RedisModuleConfigGetNumericFunc)(const char *name, void *privdata);
       typedef int (*RedisModuleConfigGetBoolFunc)(const char *name, void *privdata);
       typedef int (*RedisModuleConfigGetEnumFunc)(const char *name, void *privdata);
      ```
      
      Configs must also must specify a set callback, i.e. what to do on a CONFIG SET XYZ 123
      or when loading configurations from cli/.conf file matching these typedefs. *name* is
      again just the CONFIGNAME portion, *val* is the parsed value from the core,
      *privdata* is the registration time privdata pointer, and *err* is for providing errors to a client.
      
      ```
      typedef int (*RedisModuleConfigSetStringFunc)(const char *name, RedisModuleString *val, void *privdata, RedisModuleString **err);
      typedef int (*RedisModuleConfigSetNumericFunc)(const char *name, long long val, void *privdata, RedisModuleString **err);
      typedef int (*RedisModuleConfigSetBoolFunc)(const char *name, int val, void *privdata, RedisModuleString **err);
      typedef int (*RedisModuleConfigSetEnumFunc)(const char *name, int val, void *privdata, RedisModuleString **err);
      ```
      
      Modules can also specify an optional apply callback that will be called after
      value(s) have been set via CONFIG SET:
      
      ```
      typedef int (*RedisModuleConfigApplyFunc)(RedisModuleCtx *ctx, void *privdata, RedisModuleString **err);
      ```
      
      **Flags:**
      We expose 7 new flags to the module, which are used as part of the config registration.
      
      ```
      #define REDISMODULE_CONFIG_MODIFIABLE 0 /* This is the default for a module config. */
      #define REDISMODULE_CONFIG_IMMUTABLE (1ULL<<0) /* Can this value only be set at startup? */
      #define REDISMODULE_CONFIG_SENSITIVE (1ULL<<1) /* Does this value contain sensitive information */
      #define REDISMODULE_CONFIG_HIDDEN (1ULL<<4) /* This config is hidden in `config get <pattern>` (used for tests/debugging) */
      #define REDISMODULE_CONFIG_PROTECTED (1ULL<<5) /* Becomes immutable if enable-protected-configs is enabled. */
      #define REDISMODULE_CONFIG_DENY_LOADING (1ULL<<6) /* This config is forbidden during loading. */
      /* Numeric Specific Configs */
      #define REDISMODULE_CONFIG_MEMORY (1ULL<<7) /* Indicates if this value can be set as a memory value */
      ```
      
      **Module Registration APIs**:
      
      ```
      int (*RedisModule_RegisterBoolConfig)(RedisModuleCtx *ctx, char *name, int default_val, unsigned int flags, RedisModuleConfigGetBoolFunc getfn, RedisModuleConfigSetBoolFunc setfn, RedisModuleConfigApplyFunc applyfn, void *privdata);
      int (*RedisModule_RegisterNumericConfig)(RedisModuleCtx *ctx, const char *name, long long default_val, unsigned int flags, long long min, long long max, RedisModuleConfigGetNumericFunc getfn, RedisModuleConfigSetNumericFunc setfn, RedisModuleConfigApplyFunc applyfn, void *privdata);
      int (*RedisModule_RegisterStringConfig)(RedisModuleCtx *ctx, const char *name, const char *default_val, unsigned int flags, RedisModuleConfigGetStringFunc getfn, RedisModuleConfigSetStringFunc setfn, RedisModuleConfigApplyFunc applyfn, void *privdata);
      int (*RedisModule_RegisterEnumConfig)(RedisModuleCtx *ctx, const char *name, int default_val, unsigned int flags, const char **enum_values, const int *int_values, int num_enum_vals, RedisModuleConfigGetEnumFunc getfn, RedisModuleConfigSetEnumFunc setfn, RedisModuleConfigApplyFunc applyfn, void *privdata);
      int (*RedisModule_LoadConfigs)(RedisModuleCtx *ctx);
      ```
      
      The module name will be auto appended along with a "." to the front of the name of the config.
      
      **What RM_Register[...]Config does**:
      
      A RedisModule struct now keeps a list of ModuleConfig objects which look like:
      ```
      typedef struct ModuleConfig {
          sds name; /* Name of config without the module name appended to the front */
          void *privdata; /* Optional data passed into the module config callbacks */
          union get_fn { /* The get callback specificed by the module */
              RedisModuleConfigGetStringFunc get_string;
              RedisModuleConfigGetNumericFunc get_numeric;
              RedisModuleConfigGetBoolFunc get_bool;
              RedisModuleConfigGetEnumFunc get_enum;
          } get_fn;
          union set_fn { /* The set callback specified by the module */
              RedisModuleConfigSetStringFunc set_string;
              RedisModuleConfigSetNumericFunc set_numeric;
              RedisModuleConfigSetBoolFunc set_bool;
              RedisModuleConfigSetEnumFunc set_enum;
          } set_fn;
          RedisModuleConfigApplyFunc apply_fn;
          RedisModule *module;
      } ModuleConfig;
      ```
      It also registers a standardConfig in the configs array, with a pointer to the
      ModuleConfig object associated with it.
      
      **What happens on a CONFIG GET/SET MODULENAME.MODULECONFIG:**
      
      For CONFIG SET, we do the same parsing as is done in config.c and pass that
      as the argument to the module set callback. For CONFIG GET, we call the
      module get callback and return that value to config.c to return to a client.
      
      **CONFIG REWRITE**:
      
      Starting up a server with module configurations in a .conf file but no module load
      directive will fail. The flip side is also true, specifying a module load and a bunch
      of module configurations will load those configurations in using the module defined
      set callbacks on a RM_LoadConfigs call. Configs being rewritten works the same
      way as it does for standard configs, as the module has the ability to specify a
      default value. If a module is unloaded with configurations specified in the .conf file
      those configurations will be commented out from the .conf file on the next config rewrite.
      
      **RM_LoadConfigs:**
      
      `RedisModule_LoadConfigs(RedisModuleCtx *ctx);`
      
      This last API is used to make configs available within the onLoad() after they have
      been registered. The expected usage is that a module will register all of its configs,
      then call LoadConfigs to trigger all of the set callbacks, and then can error out if any
      of them were malformed. LoadConfigs will attempt to set all configs registered to
      either a .conf file argument/loadex argument or their default value if an argument is
      not specified. **LoadConfigs is a required function if configs are registered.
      ** Also note that LoadConfigs **does not** call the apply callbacks, but a module
      can do that directly after the LoadConfigs call.
      
      **New Command: MODULE LOADEX [CONFIG NAME VALUE] [ARGS ...]:**
      
      This command provides the ability to provide startup context information to a module.
      LOADEX stands for "load extended" similar to GETEX. Note that provided config
      names need the full MODULENAME.MODULECONFIG name. Any additional
      arguments a module might want are intended to be specified after ARGS.
      Everything after ARGS is passed to onLoad as RedisModuleString **argv.
      Co-authored-by: default avatarMadelyn Olson <madelyneolson@gmail.com>
      Co-authored-by: default avatarMadelyn Olson <matolson@amazon.com>
      Co-authored-by: default avatarsundb <sundbcn@gmail.com>
      Co-authored-by: default avatarMadelyn Olson <34459052+madolson@users.noreply.github.com>
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      Co-authored-by: default avatarYossi Gottlieb <yossigo@gmail.com>
      bda9d74d
    • Binbin's avatar
      command json files cleanups (#10473) · e2fa6aa1
      Binbin authored
      This PR do some command json files cleanups:
      
      1. Add COMMAND TIPS to some commands
      - command-docs: add `NONDETERMINISTIC_OUTPUT_ORDER`
      - command-info: add `NONDETERMINISTIC_OUTPUT_ORDER`
      - command-list: add `NONDETERMINISTIC_OUTPUT_ORDER`
      - command: change `NONDETERMINISTIC_OUTPUT` to `NONDETERMINISTIC_OUTPUT_ORDER`
      - function-list: add `NONDETERMINISTIC_OUTPUT_ORDER`
      - latency-doctor: add `NONDETERMINISTIC_OUTPUT`, `REQUEST_POLICY:ALL_NODES` and `RESPONSE_POLICY:SPECIAL`
      - latency-graph: add `NONDETERMINISTIC_OUTPUT`, `REQUEST_POLICY:ALL_NODES` and `RESPONSE_POLICY:SPECIAL`
      - memory-doctor: add `REQUEST_POLICY:ALL_SHARDS` and `RESPONSE_POLICY:SPECIAL`
      - memory-malloc-stats: add `REQUEST_POLICY:ALL_SHARDS` and `RESPONSE_POLICY:SPECIAL`
      - memory-purge: add `REQUEST_POLICY:ALL_SHARDS` and `RESPONSE_POLICY:ALL_SUCCEEDED`
      - module-list: add `NONDETERMINISTIC_OUTPUT_ORDER`
      - msetnx: add `REQUEST_POLICY:MULTI_SHARD` and `RESPONSE_POLICY:AGG_MIN`
      - object-refcount: add `NONDETERMINISTIC_OUTPUT`
      3. Only (mostly) indentation and formatting changes:
      - cluster-shards
      - latency-history
      - pubsub-shardchannels
      - pubsub-shardnumsub
      - spublish
      - ssubscribe
      - sunsubscribe
      4. add doc_flags (DEPRECATED) to cluster-slots,  replaced_by `CLUSTER SHARDS` in 7.0
      5. command-getkeysandflags: a better summary (the old one is copy from command-getkeys)
      6. adjustment of command parameter types
      - `port` is integer, not string (`MIGRATE`, `REPLICAOF`, `SLAVEOF`)
      - `replicationid` is string, not integer (`PSYNC`)
      - `pattern` is pattern, not string (`PUBSUB CHANNELS`, `SENTINEL RESET`, `SORT`, `SORT_RO`)
      e2fa6aa1
  24. 16 Mar, 2022 1 commit
  25. 15 Mar, 2022 1 commit
  26. 09 Mar, 2022 1 commit
    • Oran Agra's avatar
      Some adjustments to command hints. (#10375) · 311a757c
      Oran Agra authored
      * stats and latency commands have non-deterministic output.
      * the ones about latency should be sent to ALL_NODES (considering
        reads from replicas)
      * the ones about running scripts and memory usage only to masters.
      * stats aggregation is SPECIAL (like in INFO)
      311a757c
  27. 07 Mar, 2022 1 commit
  28. 28 Feb, 2022 1 commit
  29. 23 Feb, 2022 1 commit
    • Itamar Haber's avatar
      Add stream consumer group lag tracking and reporting (#9127) · c81c7f51
      Itamar Haber authored
      
      
      Adds the ability to track the lag of a consumer group (CG), that is, the number
      of entries yet-to-be-delivered from the stream.
      
      The proposed constant-time solution is in the spirit of "best-effort."
      
      Partially addresses #8737.
      
      ## Description of approach
      
      We add a new "entries_added" property to the stream. This starts at 0 for a new
      stream and is incremented by 1 with every `XADD`.  It is essentially an all-time
      counter of the entries added to the stream.
      
      Given the stream's length and this counter value, we can trivially find the logical
      "entries_added" counter of the first ID if and only if the stream is contiguous.
      A fragmented stream contains one or more tombstones generated by `XDEL`s.
      The new "xdel_max_id" stream property tracks the latest tombstone.
      
      The CG also tracks its last delivered ID's as an "entries_read" counter and
      increments it independently when delivering new messages, unless the this
      read counter is invalid (-1 means invalid offset). When the CG's counter is
      available, the reported lag is the difference between added and read counters.
      
      Lastly, this also adds a "first_id" field to the stream structure in order to make
      looking it up cheaper in most cases.
      
      ## Limitations
      
      There are two cases in which the mechanism isn't able to track the lag.
      In these cases, `XINFO` replies with `null` in the "lag" field.
      
      The first case is when a CG is created with an arbitrary last delivered ID,
      that isn't "0-0", nor the first or the last entries of the stream. In this case,
      it is impossible to obtain a valid read counter (short of an O(N) operation).
      The second case is when there are one or more tombstones fragmenting
      the stream's entries range.
      
      In both cases, given enough time and assuming that the consumers are
      active (reading and lacking) and advancing, the CG should be able to
      catch up with the tip of the stream and report zero lag.
      Once that's achieved, lag tracking would resume as normal (until the
      next tombstone is set).
      
      ## API changes
      
      * `XGROUP CREATE` added with the optional named argument `[ENTRIESREAD entries-read]`
        for explicitly specifying the new CG's counter.
      * `XGROUP SETID` added with an optional positional argument `[ENTRIESREAD entries-read]`
        for specifying the CG's counter.
      * `XINFO` reports the maximal tombstone ID, the recorded first entry ID, and total
        number of entries added to the stream.
      * `XINFO` reports the current lag and logical read counter of CGs.
      * `XSETID` is an internal command that's used in replication/aof. It has been added with
        the optional positional arguments `[ENTRIESADDED entries-added] [MAXDELETEDID max-deleted-entry-id]`
        for propagating the CG's offset and maximal tombstone ID of the stream.
      
      ## The generic unsolved problem
      
      The current stream implementation doesn't provide an efficient way to obtain the
      approximate/exact size of a range of entries. While it could've been nice to have
      that ability (#5813) in general, let alone specifically in the context of CGs, the risk
      and complexities involved in such implementation are in all likelihood prohibitive.
      
      ## A refactoring note
      
      The `streamGetEdgeID` has been refactored to accommodate both the existing seek
      of any entry as well as seeking non-deleted entries (the addition of the `skip_tombstones`
      argument). Furthermore, this refactoring also migrated the seek logic to use the
      `streamIterator` (rather than `raxIterator`) that was, in turn, extended with the
      `skip_tombstones` Boolean struct field to control the emission of these.
      Co-authored-by: default avatarGuy Benoish <guy.benoish@redislabs.com>
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      c81c7f51
  30. 22 Feb, 2022 3 commits
    • Wen Hui's avatar
      Fix PUBSUB SHARDNUMSUB command json file (#10331) · de6be885
      Wen Hui authored
      argument was missing, affecting redis.io docs
      de6be885
    • Madelyn Olson's avatar
      Implemented module getchannels api and renamed channel keyspec (#10299) · 71204f96
      Madelyn Olson authored
      This implements the following main pieces of functionality:
      * Renames key spec "CHANNEL" to be "NOT_KEY", and update the documentation to
        indicate it's for cluster routing and not for any other key related purpose.
      * Add the getchannels-api, so that modules can now define commands that are subject to
        ACL channel permission checks. 
      * Add 4 new flags that describe how a module interacts with a command (SUBSCRIBE, PUBLISH,
        UNSUBSCRIBE, and PATTERN). They are all technically composable, however not sure how a
        command could both subscribe and unsubscribe from a command at once, but didn't see
        a reason to add explicit validation there.
      * Add two new module apis RM_ChannelAtPosWithFlags and RM_IsChannelsPositionRequest to
        duplicate the functionality provided by the keys position APIs.
      * The RM_ACLCheckChannelPermissions (only released in 7.0 RC1) was changed to take flags
        rather than a boolean literal.
      * The RM_ACLCheckKeyPermissions (only released in 7.0 RC1) was changed to take flags
        corresponding to keyspecs instead of custom permission flags. These keyspec flags mimic
        the flags for ACLCheckChannelPermissions.
      71204f96
    • Binbin's avatar
      Remove ALLOW_BUSY from REPLICAOF and add it to REPLCONF. Add DEPRECATED doc... · c4c68f5d
      Binbin authored
      Remove ALLOW_BUSY from REPLICAOF and add it to REPLCONF. Add DEPRECATED doc flag to SLAVEOF, mark it as deprecated. (#10315)
      
      * Remove ALLOW_BUSY from REPLICAOF and add it to REPLCONF
      * mark SLAVEOF as deprecated
      c4c68f5d
  31. 08 Feb, 2022 3 commits
    • Wen Hui's avatar
      Make INFO command variadic (#6891) · 2e1bc942
      Wen Hui authored
      
      
      This is an enhancement for INFO command, previously INFO only support one argument
      for different info section , if user want to get more categories information, either perform
      INFO all / default or calling INFO for multiple times.
      
      **Description of the feature**
      
      The goal of adding this feature is to let the user retrieve multiple categories via the INFO
      command, and still avoid emitting the same section twice.
      
      A use case for this is like Redis Sentinel, which periodically calling INFO command to refresh
      info from monitored Master/Slaves, only Server and Replication part categories are used for
      parsing information. If the INFO command can return just enough categories that client side
      needs, it can save a lot of time for client side parsing it as well as network bandwidth.
      
      **Implementation**
      To share code between redis, sentinel, and other users of INFO (DEBUG and modules),
      we have a new `genInfoSectionDict` function that returns a dict and some boolean flags
      (e.g. `all`) to the caller (built from user input).
      Sentinel is later purging unwanted sections from that, and then it is forwarded to the info `genRedisInfoString`.
      
      **Usage Examples**
      INFO Server Replication   
      INFO CPU Memory
      INFO default commandstats
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      2e1bc942
    • guybe7's avatar
      X[AUTO]CLAIM should skip deleted entries (#10227) · 3c3e6cc1
      guybe7 authored
      Fix #7021 #8924 #10198
      
      # Intro
      Before this commit X[AUTO]CLAIM used to transfer deleted entries from one
      PEL to another, but reply with "nil" for every such entry (instead of the entry id).
      The idea (for XCLAIM) was that the caller could see this "nil", realize the entry
      no longer exists, and XACK it in order to remove it from PEL.
      The main problem with that approach is that it assumes there's a correlation
      between the index of the "id" arguments and the array indices, which there
      isn't (in case some of the input IDs to XCLAIM never existed/read):
      
      ```
      127.0.0.1:6379> XADD x 1 f1 v1
      "1-0"
      127.0.0.1:6379> XADD x 2 f1 v1
      "2-0"
      127.0.0.1:6379> XADD x 3 f1 v1
      "3-0"
      127.0.0.1:6379> XGROUP CREATE x grp 0
      OK
      127.0.0.1:6379> XREADGROUP GROUP grp Alice COUNT 2 STREAMS x >
      1) 1) "x"
         2) 1) 1) "1-0"
               2) 1) "f1"
                  2) "v1"
            2) 1) "2-0"
               2) 1) "f1"
                  2) "v1"
      127.0.0.1:6379> XDEL x 1 2
      (integer) 2
      127.0.0.1:6379> XCLAIM x grp Bob 0 0-99 1-0 1-99 2-0
      1) (nil)
      2) (nil)
      ```
      
      # Changes
      Now,  X[AUTO]CLAIM acts in the following way:
      1. If one tries to claim a deleted entry, we delete it from the PEL we found it in
        (and the group PEL too). So de facto, such entry is not claimed, just cleared
        from PEL (since anyway it doesn't exist in the stream)
      2. since we never claim deleted entries, X[AUTO]CLAIM will never return "nil"
        instead of an entry.
      3. add a new element to XAUTOCLAIM's response (see below)
      
      # Knowing which entries were cleared from the PEL
      The caller may want to log any entries that were found in a PEL but deleted from
      the stream itself (it would suggest that there might be a bug in the application:
      trimming the stream while some entries were still no processed by the consumers)
      
      ## XCLAIM
      the set {XCLAIM input ids} - {XCLAIM returned ids} contains all the entry ids that were
      not claimed which means they were deleted (assuming the input contains only entries
      from some PEL). The user doesn't need to XACK them because XCLAIM had already
      deleted them from the source PEL.
      
      ## XAUTOCLAIM
      XAUTOCLAIM has a new element added to its reply: it's an array of all the deleted
      stream IDs it stumbled upon.
      
      This is somewhat of a breaking change since X[AUTO]CLAIM used to be able to reply
      with "nil" and now it can't... But since it was undocumented (and generally a bad idea
      to rely on it, as explained above) the breakage is not that bad.
      3c3e6cc1
    • Oran Agra's avatar
      Handle key-spec flags with modules (#10237) · 66be30f7
      Oran Agra authored
      - add COMMAND GETKEYSANDFLAGS sub-command
      - add RM_KeyAtPosWithFlags and GetCommandKeysWithFlags
      - RM_KeyAtPos and RM_CreateCommand set flags requiring full access for keys
      - RM_CreateCommand set VARIABLE_FLAGS
      - expose `variable_flags` flag in COMMAND INFO key-specs
      - getKeysFromCommandWithSpecs prefers key-specs over getkeys-api
      - add tests for all of these
      66be30f7