1. 15 Oct, 2024 1 commit
  2. 29 Sep, 2024 1 commit
    • Moti Cohen's avatar
      Add new SFLUSH command to cluster for slot-based FLUSH (#13564) · d092d64d
      Moti Cohen authored
      This PR introduces a new `SFLUSH` command to cluster mode that allows
      partial flushing of nodes based on specified slot ranges. Current
      implementation is designed to flush all slots of a shard, but future
      extensions could allow for more granular flushing.
      
      **Command Usage:**
      `SFLUSH <start-slot> <end-slot> [<start-slot> <end-slot>]* [SYNC|ASYNC]`
      
      This command removes all data from the specified slots, either
      synchronously or asynchronously depending on the optional SYNC/ASYNC
      argument.
      
      **Functionality:**
      Current imp of `SFLUSH` command verifies that the provided slot ranges
      are valid and cover all of the node's slots before proceeding. If slots
      are partially or incorrectly specified, the command will fail and return
      an error, ensuring that all slots of a node must be fully covered for
      the flush to proceed.
      
      The function supports both synchronous (default) and asynchronous
      flushing. In addition, if possible, SFLUSH SYNC will be run as blocking
      ASYNC as an optimization.
      d092d64d
  3. 03 Sep, 2024 1 commit
  4. 04 Aug, 2024 1 commit
  5. 14 Jun, 2024 1 commit
    • Jo's avatar
      Update `FIELDS` argument to block type for HFE commands schema (#13339) · 871c9859
      Jo authored
      I reviewed `XREAD` command syntax:
      ```
      XREAD [COUNT count] [BLOCK milliseconds] STREAMS key [key ...] id [id ...]
      ```
      
      Here’s the structure for `XREAD`:
      ```json
      "arguments": [
                  {
                      "token": "COUNT",
                      "name": "count",
                      "type": "integer",
                      "optional": true
                  },
                  {
                      "token": "BLOCK",
                      "name": "milliseconds",
                      "type": "integer",
                      "optional": true
                  },
                  {
                      "name": "streams",
                      "token": "STREAMS",
                      "type": "block",
                      "arguments": [
                          {
                              "name": "key",
                              "type": "key",
                              "key_spec_index": 0,
                              "multiple": true
                          },
                          {
                              "name": "ID",
                              "type": "string",
                              "multiple": true
                          }
                      ]
                  }
      ]
      ```
      
      Now, consider the `HEXPIRE` syntax:
      ```
      HEXPIRE key seconds [NX | XX | GT | LT] FIELDS numfields field [field ...]
      ```
      
      Since the `FIELDS` token functions similarly to `STREAMS`, and given that `STREAMS` is defined as a block, I believe the `FIELDS` in `hepxire` should also be defined as a block.
      871c9859
  6. 29 May, 2024 1 commit
  7. 27 May, 2024 2 commits
  8. 26 May, 2024 1 commit
  9. 16 May, 2024 1 commit
  10. 13 May, 2024 1 commit
    • Ozan Tezcan's avatar
      Fix hgetf/hsetf reply type by returning string (#13263) · 5066e6e9
      Ozan Tezcan authored
      If encoding is listpack, hgetf and hsetf commands reply field value type
      as integer.
      This PR fixes it by returning string.
      
      Problematic cases:
      ```
      127.0.0.1:6379> hset hash one 1
      (integer) 1
      127.0.0.1:6379> hgetf hash fields 1 one
      1) (integer) 1
      127.0.0.1:6379> hsetf hash GETOLD fvs 1 one 2
      1) (integer) 1
      127.0.0.1:6379> hsetf hash DOF GETNEW fvs 1 one 2
      1) (integer) 2
      ```
      
      Additional fixes:
      - hgetf/hsetf command description text
      
      Fixes #13261, #13262
      5066e6e9
  11. 09 May, 2024 1 commit
  12. 08 May, 2024 1 commit
    • Ozan Tezcan's avatar
      Add listpack support, hgetf and hsetf commands (#13209) · ca4ed48d
      Ozan Tezcan authored
      **Changes:**
      - Adds listpack support to hash field expiration 
      - Implements hgetf/hsetf commands
      
      **Listpack support for hash field expiration**
      
      We keep field name and value pairs in listpack for the hash type. With
      this PR, if one of hash field expiration command is called on the key
      for the first time, it converts listpack layout to triplets to hold
      field name, value and ttl per field. If a field does not have a TTL, we
      store zero as the ttl value. Zero is encoded as two bytes in the
      listpack. So, once we convert listpack to hold triplets, for the fields
      that don't have a TTL, it will be consuming those extra 2 bytes per
      item. Fields are ordered by ttl in the listpack to find the field with
      minimum expiry time efficiently.
      
      **New command implementations as part of this PR:** 
      
      - HGETF command
      
      For each specified field get its value and optionally set the field's
      expiration time in sec/msec /unix-sec/unix-msec:
        ```
        HGETF key 
          [NX | XX | GT | LT]
      [EX seconds | PX milliseconds | EXAT unix-time-seconds | PXAT
      unix-time-milliseconds | PERSIST]
          <FIELDS count field [field ...]>
        ```
      
      - HSETF command
      
      For each specified field value pair: set field to value and optionally
      set the field's expiration time in sec/msec /unix-sec/unix-msec:
        ```
        HSETF key 
          [DC] 
          [DCF | DOF] 
          [NX | XX | GT | LT] 
          [GETNEW | GETOLD] 
      [EX seconds | PX milliseconds | EXAT unix-time-seconds | PXAT
      unix-time-milliseconds | KEEPTTL]
          <FVS count field value [field value …]>
        ```
      
      Todo:
      - Performance improvement.
      - rdb load/save
      - aof
      - defrag
      ca4ed48d
  13. 03 May, 2024 1 commit
  14. 18 Apr, 2024 1 commit
    • Moti Cohen's avatar
      Hash Field Expiration - Basic support · c18ff056
      Moti Cohen authored
      - Add ebuckets & mstr data structures
      - Integrate active & lazy expiration
      - Add most of the commands 
      - Add support for dict (listpack is missing)
      TODOs:  RDB, notification, listpack, HSET, HGETF, defrag, aof
      c18ff056
  15. 01 Mar, 2024 1 commit
    • Chen Tianjie's avatar
      Add overhead of all DBs and rehashing dict count to info. (#12913) · 4cae99e7
      Chen Tianjie authored
      
      
      Sometimes we need to make fast judgement about why Redis is suddenly
      taking more memory. One of the reasons is main DB's dicts doing
      rehashing.
      
      We may use `MEMORY STATS` to monitor the overhead memory of each DB, but
      there still lacks a total sum to show an overall trend. So this PR adds
      the total overhead of all DBs to `INFO MEMORY` section, together with
      the total count of rehashing DB dicts, providing some intuitive metrics
      about main dicts rehashing.
      
      This PR adds the following metrics to INFO MEMORY
      * `mem_overhead_db_hashtable_rehashing` - only size of ht[0] in
      dictionaries we're rehashing (i.e. the memory that's gonna get released
      soon)
      
      and a similar ones to MEMORY STATS:
      * `overhead.db.hashtable.lut` (complements the existing
      `overhead.hashtable.main` and `overhead.hashtable.expires` which also
      counts the `dictEntry` structs too)
      * `overhead.db.hashtable.rehashing` - temporary rehashing overhead.
      * `db.dict.rehashing.count` - number of top level dictionaries being
      rehashed.
      
      ---------
      Co-authored-by: default avatarzhaozhao.zz <zhaozhao.zz@alibaba-inc.com>
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      4cae99e7
  16. 29 Feb, 2024 1 commit
    • Binbin's avatar
      Fix propagation of entries_read by calling streamPropagateGroupID unconditionally (#12898) · f17381a3
      Binbin authored
      In XREADGROUP ACK, because streamPropagateXCLAIM does not propagate
      entries-read, entries-read will be inconsistent between master and
      replicas.
      I.e. if no entries were claimed, it would have propagated correctly, but
      if some
      were claimed, then the entries-read field would be inconsistent on the
      replica.
      
      The fix was suggested by guybe7, call streamPropagateGroupID
      unconditionally,
      so that we will normalize entries_read on the replicas. In the past, we
      would
      only set propagate_last_id when NOACK was specified. And in #9127,
      XCLAIM did
      not propagate entries_read in ACK, which would cause entries_read to be
      inconsistent between master and replicas.
      
      Another approach is add another arg to XCLAIM and let it propagate
      entries_read,
      but we decided not to use it. Because we want minimal damage in case
      there's an
      old target and new source (in the worst case scenario, the new source
      doesn't
      recognize XGROUP SETID ... ENTRIES READ and the lag is lost. If we
      change XCLAIM,
      the damage is much more severe).
      
      In this patch, now if the user uses XREADGROUP .. COUNT 1 there will be
      an additional
      overhead of MULTI, EXEC and XGROUPSETID. We assume the extra command in
      case of
      COUNT 1 (4x factor, changing from one XCLAIM to
      MULTI+XCLAIM+XSETID+EXEC), is probably
      ok since reading just one entry is in any case very inefficient (a
      client round trip
      per record), so we're hoping it's not a common case.
      
      Issue was introduced in #9127.
      f17381a3
  17. 22 Feb, 2024 1 commit
  18. 21 Feb, 2024 1 commit
  19. 20 Feb, 2024 1 commit
    • Binbin's avatar
      xinfo-stream add minimum to seen-time, skip logreqres in fuzzer (#13056) · ca5cac99
      Binbin authored
      
      
      Recently I saw in CI that reply-schemas-validator fails here:
      ```
      Failed validating 'minimum' in schema[1]['properties']['groups']['items']['properties']['consumers']['items']['properties']['active-time']:
          {'description': 'Last time this consumer was active (successful '
                          'reading/claiming).',
           'minimum': 0,
           'type': 'integer'}
      
      On instance['groups'][0]['consumers'][0]['active-time']:
          -1729380548878722639
      ```
      
      The reason is that in fuzzer, we may restore corrupted active-time,
      which will cause the reply schema CI to fail.
      
      The fuzzer can cause corrupt the state in many places, which will
      bugs that mess up the reply, so we decided to skip logreqres.
      
      Also, seen-time is the same type as active-time, adding the minimum.
      
      ---------
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      ca5cac99
  20. 19 Feb, 2024 1 commit
  21. 04 Feb, 2024 1 commit
  22. 30 Jan, 2024 2 commits
  23. 23 Jan, 2024 1 commit
    • Binbin's avatar
      Allow running WAITAOF in scripts, remove NOSCRIPT flag (#12977) · 85c31e0c
      Binbin authored
      In #11568 we removed the NOSCRIPT flag from commands, e.g. removing
      NOSCRIPT flag from WAIT. Aiming to allow them in scripts and let them
      implicitly behave in the non-blocking way.
      
      This PR remove NOSCRIPT flag from WAITAOF just like WAIT (to be
      symmetrical)).
      And this PR also add BLOCKING flag for WAIT and WAITAOF.
      85c31e0c
  24. 08 Jan, 2024 1 commit
  25. 10 Dec, 2023 1 commit
  26. 12 Oct, 2023 1 commit
    • zhaozhao.zz's avatar
      support XREAD[GROUP] with BLOCK option in scripts (#12596) · 77a65e82
      zhaozhao.zz authored
      In #11568 we removed the NOSCRIPT flag from commands and keep the BLOCKING flag.
      Aiming to allow them in scripts and let them implicitly behave in the non-blocking way.
      
      In that sense, the old behavior was to allow LPOP and reject BLPOP, and the new behavior,
      is to allow BLPOP too, and fail it only in case it ends up blocking.
      So likewise, so far we allowed XREAD and rejected XREAD BLOCK, and we will now allow
      that too, and only reject it if it ends up blocking.
      77a65e82
  27. 10 Oct, 2023 1 commit
  28. 10 Sep, 2023 1 commit
  29. 04 Sep, 2023 1 commit
  30. 31 Aug, 2023 1 commit
  31. 30 Aug, 2023 1 commit
  32. 15 Aug, 2023 1 commit
  33. 05 Aug, 2023 1 commit
  34. 25 Jul, 2023 1 commit
    • nihohit's avatar
      Update request/response policies. (#12417) · 9f512017
      nihohit authored
      changing the response and request policy of a few commands,
      see https://redis.io/docs/reference/command-tips
      
      
      
      1. RANDOMKEY used to have no response policy, which means
        that when sent to multiple shards, the responses should be aggregated.
        this normally applies to commands that return arrays, but since RANDOMKEY
        replies with a simple string, it actually requires a SPECIAL response policy
        (for the client to select just one)
      2. SCAN used to have no response policy, but although the key names part of
        the response can be aggregated, the cursor part certainly can't.
      3. MSETNX had a request policy of MULTI_SHARD and response policy of AGG_MIN,
        but in fact the contract with MSETNX is that when one key exists, it returns 0
        and doesn't set any key, routing it to multiple shards would mean that if one failed
        and another succeeded, it's atomicity is broken and it's impossible to return a valid
        response to the caller.
      Co-authored-by: default avatarShachar Langbeheim <shachlan@amazon.com>
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      9f512017
  35. 20 Jun, 2023 1 commit
  36. 19 Jun, 2023 1 commit
    • Binbin's avatar
      Optimize PSUBSCRIBE and PUNSUBSCRIBE from O(N*M) to O(N) (#12298) · b5106249
      Binbin authored
      In the original implementation, the time complexity of the commands
      is actually O(N*M), where N is the number of patterns the client is
      already subscribed and M is the number of patterns to subscribe to.
      The docs are all wrong about this.
      
      Specifically, because the original client->pubsub_patterns is a list,
      so we need to do listSearchKey which is O(N). In this PR, we change it
      to a dict, so the search becomes O(1).
      
      At the same time, both pubsub_channels and pubsubshard_channels are dicts.
      Changing pubsub_patterns to a dictionary improves the readability and
      maintainability of the code.
      b5106249
  37. 13 Jun, 2023 1 commit
  38. 26 May, 2023 1 commit
    • Binbin's avatar
      Add new loglevel nothing to sentinel-config.json (#12235) · e775b34e
      Binbin authored
      It was missing in #12223, and the reply-schemas daily
      was failing:
      ```
      jsonschema.exceptions.ValidationError: 'nothing' is not valid under any of the given schemas
      
      Failed validating 'oneOf' in schema[0]['properties']['loglevel']:
          {'oneOf': [{'const': 'debug'},
                     {'const': 'verbose'},
                     {'const': 'notice'},
                     {'const': 'warning'},
                     {'const': 'unknown'}]}
      
      On instance['loglevel']:
          'nothing'
      ```
      e775b34e