1. 04 Nov, 2024 1 commit
    • David Dougherty's avatar
      Update old links for modules-api-ref.md (#13479) · 9906daf5
      David Dougherty authored
      This PR replaces old .../topics/... links with current links,
      specifically for the modules-api-ref.md file and the new automation that
      Paolo Lazzari is working on. A few of the topics links have redirects,
      but some don't. Best to use updated links.
      9906daf5
  2. 30 Oct, 2024 3 commits
  3. 29 Oct, 2024 1 commit
    • Moti Cohen's avatar
      Add KEYSIZES section to INFO (#13592) · 2ec78d26
      Moti Cohen authored
      This PR adds a new section to the `INFO` command output, called
      `keysizes`. This section provides detailed statistics on the
      distribution of key sizes for each data type (strings, lists, sets,
      hashes and zsets) within the dataset. The distribution is tracked using
      a base-2 logarithmic histogram.
      
      # Motivation
      Currently, Redis lacks a built-in feature to track key sizes and item
      sizes per data type at a granular level. Understanding the distribution
      of key sizes is critical for monitoring memory usage and optimizing
      performance, particularly in large datasets. This enhancement will allow
      users to inspect the size distribution of keys directly from the `INFO`
      command, assisting with performance analysis and capacity planning.
      
      # Changes
      New Section in `INFO` Command: A new section called `keysizes` has been
      added to the `INFO` command output. This section reports a per-database,
      per-type histogram of key sizes. It provides insights into how many keys
      fall into specific size ranges (represented in powers of 2).
      
      **Example output:**
      ```
      127.0.0.1:6379> INFO keysizes
      # Keysizes
      db0_distrib_strings_sizes:1=19,2=655,512=100899,1K=31,2K=29,4K=23,8K=16,16K=3,32K=2
      db0_distrib_lists_items:1=5784492,32=3558,64=1047,128=676,256=533,512=218,4K=1,8K=42
      db0_distrib_sets_items:1=735564=50612,8=21462,64=1365,128=974,2K=292,4K=154,8K=89,
      db0_distrib_hashes_items:2=1,4=544,32=141169,64=207329,128=4349,256=136226,1K=1
      ```
      ## Future Use Cases:
      The key size distribution is collected per slot as well, laying the
      groundwork for future enhancements related to Redis Cluster.
      2ec78d26
  4. 28 Oct, 2024 1 commit
    • Shockingly Good's avatar
      Fix crash in RM_GetCurrentUserName() when the user isn't accessible (#13619) · 611c9502
      Shockingly Good authored
      The crash happens whenever the user isn't accessible, for example, it
      isn't set for the context (when it is temporary) or in some other cases
      like `notifyKeyspaceEvent`. To properly check for the ACL compliance, we
      need to get the user name and the user to invoke other APIs. However, it
      is not possible if it crashes, and it is impossible to work that around
      in the code since we don't know (and **shouldn't know**!) when it is
      available and when it is not.
      611c9502
  5. 22 Oct, 2024 1 commit
  6. 18 Oct, 2024 1 commit
  7. 17 Oct, 2024 2 commits
    • hanhui365's avatar
      Optimize bitcount command by using popcnt (#13359) · 3788a055
      hanhui365 authored
      
      
      Nowadays popcnt instruction is almost supported by X86 machine, which is
      used to calculate "Hamming weight", it can bring much performance boost
      in redis bitcount comand.
      
      ---------
      
      Signed-off-by: hanhui365(hanhui@hygon.cn)
      Co-authored-by: default avatardebing.sun <debing.sun@redis.com>
      Co-authored-by: default avataroranagra <oran@redislabs.com>
      Co-authored-by: default avatarNugine <nugine@foxmail.com>
      3788a055
    • Yuan Wang's avatar
      Clean up .rediscli_history_test temporary file (#13601) · b71a610f
      Yuan Wang authored
      After running test in local, there will be a file named
      `.rediscli_history_test`, and it is not in `.gitignore` file, so this is
      considered to have changed the code base. It is a little annoying, this
      commit just clean up the temporary file.
      
      We should delete `.rediscli_history_test` in the end since the second
      server tests also write somethings into it, to make it corresponding, i
      put `set ::env(REDISCLI_HISTFILE) ".rediscli_history_test"` at the
      beginning.
      
      Maybe we also can add this file into `.gitignore`
      b71a610f
  8. 15 Oct, 2024 3 commits
  9. 12 Oct, 2024 1 commit
  10. 10 Oct, 2024 1 commit
    • guybe7's avatar
      Cleanups related to expiry/eviction (#13591) · a38c29b6
      guybe7 authored
      1. `dbRandomKey`: excessive call to `dbFindExpires` (will always return
      1 if `allvolatile` + anyway called inside `expireIfNeeded`
      2. Add `deleteKeyAndPropagate` that is used by both expiry/eviction
      3. Change the order of calls in `expireIfNeeded` to save redundant calls
      to `keyIsExpired`
      4. `expireIfNeeded`: move `OBJ_STATIC_REFCOUNT` to
      `deleteKeyAndPropagate`
      5. `performEvictions` now uses `deleteEvictedKeyAndPropagate`
      6. active-expire: moved `postExecutionUnitOperations` inside
      `activeExpireCycleTryExpire`
      7. `activeExpireCycleTryExpire`: less indentation + expire a key if `now
      == t`
      8. rename `lazy_expire_disabled` to `allow_access_expired`
      a38c29b6
  11. 08 Oct, 2024 5 commits
  12. 29 Sep, 2024 1 commit
    • Moti Cohen's avatar
      Add new SFLUSH command to cluster for slot-based FLUSH (#13564) · d092d64d
      Moti Cohen authored
      This PR introduces a new `SFLUSH` command to cluster mode that allows
      partial flushing of nodes based on specified slot ranges. Current
      implementation is designed to flush all slots of a shard, but future
      extensions could allow for more granular flushing.
      
      **Command Usage:**
      `SFLUSH <start-slot> <end-slot> [<start-slot> <end-slot>]* [SYNC|ASYNC]`
      
      This command removes all data from the specified slots, either
      synchronously or asynchronously depending on the optional SYNC/ASYNC
      argument.
      
      **Functionality:**
      Current imp of `SFLUSH` command verifies that the provided slot ranges
      are valid and cover all of the node's slots before proceeding. If slots
      are partially or incorrectly specified, the command will fail and return
      an error, ensuring that all slots of a node must be fully covered for
      the flush to proceed.
      
      The function supports both synchronous (default) and asynchronous
      flushing. In addition, if possible, SFLUSH SYNC will be run as blocking
      ASYNC as an optimization.
      d092d64d
  13. 25 Sep, 2024 2 commits
  14. 23 Sep, 2024 2 commits
  15. 19 Sep, 2024 1 commit
    • Moti Cohen's avatar
      Extend modules API to read also expired keys and subkeys (#13526) · 3a3cacfe
      Moti Cohen authored
      The PR extends `RedisModule_OpenKey`'s flags to include
      `REDISMODULE_OPEN_KEY_ACCESS_EXPIRED`, which allows to access expired
      keys.
      
      It also allows to access expired subkeys. Currently relevant only for
      hash fields
      and has its impact on `RM_HashGet` and `RM_Scan`.
      3a3cacfe
  16. 18 Sep, 2024 1 commit
  17. 15 Sep, 2024 3 commits
  18. 13 Sep, 2024 2 commits
  19. 12 Sep, 2024 6 commits
    • Filipe Oliveira (Redis)'s avatar
      Optimize SSCAN command in case of listpack or intset encoding: avoid the usage... · f2f85ba3
      Filipe Oliveira (Redis) authored
      Optimize SSCAN command in case of listpack or intset encoding: avoid the usage of intermediate list. From 2N to N iterations (#13530)
      
      On SSCAN, in case of listpack and intset encoding we actually reply the
      entire set, and always reply with the cursor 0.
      
      For those cases, we don't need to accumulate the replies in a list and
      can completely avoid the overhead of list appending and then iterating
      over the list again -- meaning we do N iterations instead of 2N
      iterations over the SET and save intermediate memory as well.
      
      Preliminary benchmarks, `SSCAN set:100 0`, showcased an improvement of
      60% as visible bellow on a SET with 100 string elements (listpack
      encoded).
      f2f85ba3
    • Moti Cohen's avatar
      Add iterator capability to ebuckets DS (#13519) · c115c523
      Moti Cohen authored
      Add basic iterator API for ebuckets of start, next, nextBucket and stop.
      c115c523
    • Moti Cohen's avatar
      Correct spelling error at t_hash.c comment (#13540) · 65a87cb7
      Moti Cohen authored
      spell check error : ./src/t_hash.c:1141: RESOTRE ==> RESTORE
      65a87cb7
    • Moti Cohen's avatar
      HFE - Fix key ref by the hash on RENAME/MOVE/SWAPDB/RESTORE (#13539) · 9a89e32a
      Moti Cohen authored
      If the hash previously had HFEs (hash-fields with expiration) but later no longer
      does, the key ref in the hash might become outdated after a MOVE, COPY,
      RENAME or RESTORE operation. These commands maintain the key ref only
      if HFEs are present. That is, we can only be sure that key ref is valid as long as the
      hash has HFEs.
      9a89e32a
    • Oran Agra's avatar
      RED-129256, Fix TOUCH command from script in no-touch mode (#13512) · 610eb26c
      Oran Agra authored
      
      
      When a client in no-touch mode issues a TOUCH command on a key, the
      key’s access time should be updated, but in scripts, and module's
      RM_Call, it isn’t updated.
      Command proc should be matched to the executing client, not the current
      client.
      Co-authored-by: default avatarUdi Ron <udi@speedb.io>
      610eb26c
    • Steve's avatar
      Avoid cluster.nodes load corruption due to shard-id generation (#13468) · d265a614
      Steve authored
      
      
      PR #13428 doesn't fully resolve an issue where corruption errors can
      still occur on loading of cluster.nodes file - seen on upgrade where
      there were no shard_ids (from old Redis), 7.2.5 loading generated new
      random ones, and persisted them to the file before gossip/handshake
      could propagate the correct ones (or some other nodes unreachable).
      This results in a primary/replica having differing shard_id in the
      cluster.nodes and then the server cannot startup - reports corruption.
      
      This PR builds on #13428 by simply ignoring the replica's shard_id in
      cluster.nodes (if it exists), and uses the replica's primary's shard_id.
      Additional handling was necessary to cover the case where the replica
      appears before the primary in cluster.nodes, where it will first use a
      generated shard_id for the primary, and then correct after it loads the
      primary cluster.nodes entry.
      
      ---------
      Co-authored-by: default avatardebing.sun <debing.sun@redis.com>
      d265a614
  20. 11 Sep, 2024 1 commit
    • debing.sun's avatar
      Increment kvstore's non_empty_dicts only on first insert (#13528) · 2dd4cca3
      debing.sun authored
      
      
      Found by @oranagra 
      
      Currently, when the size of dict becomes 1, we do not check whether
      `delta` is positive or negative.
      As a result, `non_empty_dicts` is still incremented when the size of
      dict changes from 2 to 1.
      We should only increment `non_empty_dicts` when `delta` is positive, as
      this indicates the first time an element is inserted into the dict.
      
      ---------
      Co-authored-by: default avataroranagra <oran@redislabs.com>
      2dd4cca3
  21. 10 Sep, 2024 1 commit
    • Filipe Oliveira (Redis)'s avatar
      Optimize LREM, LPOS, LINSERT, LINDEX: Avoid N-1 sdslen() calls on listTypeEqual (#13529) · bcae7708
      Filipe Oliveira (Redis) authored
      This is a very easy optimization, that avoids duplicate computation of
      the object length for LREM, LPOS, LINSERT na LINDEX.
      
      We can see that sdslen takes 7.7% of the total CPU cycles of the
      benchmarks.
      
      Function Stack | CPU Time: Total | CPU Time: Self | Module | Function
      (Full) | Source File | Start Address
      -- | -- | -- | -- | -- | -- | --
      listTypeEqual | 15.50% | 2.346s | redis-server | listTypeEqual |
      t_list.c | 0x845dd
      sdslen | 7.70% | 2.300s | redis-server | sdslen | sds.h | 0x845e4
      
      Preliminary data showcases 4% improvement in the achievable ops/sec of
      LPOS in string elements, and 2% in int elements.
      bcae7708