1. 29 Sep, 2024 1 commit
    • Moti Cohen's avatar
      Add new SFLUSH command to cluster for slot-based FLUSH (#13564) · d092d64d
      Moti Cohen authored
      This PR introduces a new `SFLUSH` command to cluster mode that allows
      partial flushing of nodes based on specified slot ranges. Current
      implementation is designed to flush all slots of a shard, but future
      extensions could allow for more granular flushing.
      
      **Command Usage:**
      `SFLUSH <start-slot> <end-slot> [<start-slot> <end-slot>]* [SYNC|ASYNC]`
      
      This command removes all data from the specified slots, either
      synchronously or asynchronously depending on the optional SYNC/ASYNC
      argument.
      
      **Functionality:**
      Current imp of `SFLUSH` command verifies that the provided slot ranges
      are valid and cover all of the node's slots before proceeding. If slots
      are partially or incorrectly specified, the command will fail and return
      an error, ensuring that all slots of a node must be fully covered for
      the flush to proceed.
      
      The function supports both synchronous (default) and asynchronous
      flushing. In addition, if possible, SFLUSH SYNC will be run as blocking
      ASYNC as an optimization.
      d092d64d
  2. 25 Sep, 2024 2 commits
  3. 23 Sep, 2024 2 commits
  4. 19 Sep, 2024 1 commit
    • Moti Cohen's avatar
      Extend modules API to read also expired keys and subkeys (#13526) · 3a3cacfe
      Moti Cohen authored
      The PR extends `RedisModule_OpenKey`'s flags to include
      `REDISMODULE_OPEN_KEY_ACCESS_EXPIRED`, which allows to access expired
      keys.
      
      It also allows to access expired subkeys. Currently relevant only for
      hash fields
      and has its impact on `RM_HashGet` and `RM_Scan`.
      3a3cacfe
  5. 18 Sep, 2024 1 commit
  6. 15 Sep, 2024 3 commits
  7. 13 Sep, 2024 2 commits
  8. 12 Sep, 2024 6 commits
    • Filipe Oliveira (Redis)'s avatar
      Optimize SSCAN command in case of listpack or intset encoding: avoid the usage... · f2f85ba3
      Filipe Oliveira (Redis) authored
      Optimize SSCAN command in case of listpack or intset encoding: avoid the usage of intermediate list. From 2N to N iterations (#13530)
      
      On SSCAN, in case of listpack and intset encoding we actually reply the
      entire set, and always reply with the cursor 0.
      
      For those cases, we don't need to accumulate the replies in a list and
      can completely avoid the overhead of list appending and then iterating
      over the list again -- meaning we do N iterations instead of 2N
      iterations over the SET and save intermediate memory as well.
      
      Preliminary benchmarks, `SSCAN set:100 0`, showcased an improvement of
      60% as visible bellow on a SET with 100 string elements (listpack
      encoded).
      f2f85ba3
    • Moti Cohen's avatar
      Add iterator capability to ebuckets DS (#13519) · c115c523
      Moti Cohen authored
      Add basic iterator API for ebuckets of start, next, nextBucket and stop.
      c115c523
    • Moti Cohen's avatar
      Correct spelling error at t_hash.c comment (#13540) · 65a87cb7
      Moti Cohen authored
      spell check error : ./src/t_hash.c:1141: RESOTRE ==> RESTORE
      65a87cb7
    • Moti Cohen's avatar
      HFE - Fix key ref by the hash on RENAME/MOVE/SWAPDB/RESTORE (#13539) · 9a89e32a
      Moti Cohen authored
      If the hash previously had HFEs (hash-fields with expiration) but later no longer
      does, the key ref in the hash might become outdated after a MOVE, COPY,
      RENAME or RESTORE operation. These commands maintain the key ref only
      if HFEs are present. That is, we can only be sure that key ref is valid as long as the
      hash has HFEs.
      9a89e32a
    • Oran Agra's avatar
      RED-129256, Fix TOUCH command from script in no-touch mode (#13512) · 610eb26c
      Oran Agra authored
      
      
      When a client in no-touch mode issues a TOUCH command on a key, the
      key’s access time should be updated, but in scripts, and module's
      RM_Call, it isn’t updated.
      Command proc should be matched to the executing client, not the current
      client.
      Co-authored-by: default avatarUdi Ron <udi@speedb.io>
      610eb26c
    • Steve's avatar
      Avoid cluster.nodes load corruption due to shard-id generation (#13468) · d265a614
      Steve authored
      
      
      PR #13428 doesn't fully resolve an issue where corruption errors can
      still occur on loading of cluster.nodes file - seen on upgrade where
      there were no shard_ids (from old Redis), 7.2.5 loading generated new
      random ones, and persisted them to the file before gossip/handshake
      could propagate the correct ones (or some other nodes unreachable).
      This results in a primary/replica having differing shard_id in the
      cluster.nodes and then the server cannot startup - reports corruption.
      
      This PR builds on #13428 by simply ignoring the replica's shard_id in
      cluster.nodes (if it exists), and uses the replica's primary's shard_id.
      Additional handling was necessary to cover the case where the replica
      appears before the primary in cluster.nodes, where it will first use a
      generated shard_id for the primary, and then correct after it loads the
      primary cluster.nodes entry.
      
      ---------
      Co-authored-by: default avatardebing.sun <debing.sun@redis.com>
      d265a614
  9. 11 Sep, 2024 1 commit
    • debing.sun's avatar
      Increment kvstore's non_empty_dicts only on first insert (#13528) · 2dd4cca3
      debing.sun authored
      
      
      Found by @oranagra 
      
      Currently, when the size of dict becomes 1, we do not check whether
      `delta` is positive or negative.
      As a result, `non_empty_dicts` is still incremented when the size of
      dict changes from 2 to 1.
      We should only increment `non_empty_dicts` when `delta` is positive, as
      this indicates the first time an element is inserted into the dict.
      
      ---------
      Co-authored-by: default avataroranagra <oran@redislabs.com>
      2dd4cca3
  10. 10 Sep, 2024 1 commit
    • Filipe Oliveira (Redis)'s avatar
      Optimize LREM, LPOS, LINSERT, LINDEX: Avoid N-1 sdslen() calls on listTypeEqual (#13529) · bcae7708
      Filipe Oliveira (Redis) authored
      This is a very easy optimization, that avoids duplicate computation of
      the object length for LREM, LPOS, LINSERT na LINDEX.
      
      We can see that sdslen takes 7.7% of the total CPU cycles of the
      benchmarks.
      
      Function Stack | CPU Time: Total | CPU Time: Self | Module | Function
      (Full) | Source File | Start Address
      -- | -- | -- | -- | -- | -- | --
      listTypeEqual | 15.50% | 2.346s | redis-server | listTypeEqual |
      t_list.c | 0x845dd
      sdslen | 7.70% | 2.300s | redis-server | sdslen | sds.h | 0x845e4
      
      Preliminary data showcases 4% improvement in the achievable ops/sec of
      LPOS in string elements, and 2% in int elements.
      bcae7708
  11. 09 Sep, 2024 1 commit
  12. 08 Sep, 2024 1 commit
    • Ozan Tezcan's avatar
      Fix flaky replication tests (#13518) · ac03e372
      Ozan Tezcan authored
      #13495 introduced a change to reply -LOADING while flushing existing db on a replica. Some of our tests are
       sensitive to this change and do no expect -LOADING reply.
      
      Fixing a couple of tests that fail time to time.
      ac03e372
  13. 06 Sep, 2024 1 commit
  14. 05 Sep, 2024 2 commits
  15. 04 Sep, 2024 4 commits
  16. 03 Sep, 2024 7 commits
    • debing.sun's avatar
      Avoid overhead of comparision function pointer calls in lpFind() (#13503) · de7f2f87
      debing.sun authored
      
      
      In #13279 (found by @filipecosta90), for custom lookups, we introduce a
      comparison function for `lpFind()` to compare entry, but it also
      introduces some overhead.
      
      To avoid the overhead of function pointer calls:
      1. Extract the lpFindCb() method into a lpFindCbInternal() method that
      is easier to inline.
      2. Use unlikely to annotate the comparison method, as can only success
      once.
      
      ---------
      Co-authored-by: default avatarOzan Tezcan <ozantezcan@gmail.com>
      de7f2f87
    • Filipe Oliveira (Redis)'s avatar
      changed addReplyHumanLongDouble to addReplyDouble in georadiusGeneric and geoposCommand (#13494) · fb8755a6
      Filipe Oliveira (Redis) authored
      # Summary 
      
      - Addresses https://github.com/redis/redis/issues/11565
      - Measured improvements of 30% and 37% on the simple use-case (GEOSEARCH
      and GEOPOS) (check
      https://github.com/redis/redis/pull/13494#issuecomment-2313668934), and
      of 66% on a dataset with >60M datapoints and pipeline 10 benchmark.
      fb8755a6
    • Meir Shpilraien (Spielrein)'s avatar
      Added new defrag API to allocate and free raw memory. (#13509) · d3d94ccf
      Meir Shpilraien (Spielrein) authored
      All the defrag allocations API expects to get a value and replace it, leaving the old value untouchable.
      In some cases a value might be shared between multiple keys, in such cases we can not simply replace
      it when the defrag callback is called.
      
      To allow support such use cases, the PR adds two new API's to the defrag API:
      
      1. `RM_DefragAllocRaw` - allocate memory base on a given size.
      2. `RM_DefragFreeRaw` - Free the given pointer.
      
      Those API's avoid using tcache so they operate just like `RM_DefragAlloc` but allows the user to split
      the allocation and the memory free operations into two stages and control when those happen.
      
      In addition the PR adds new API to allow the module to receive notifications when defrag start and end: `RM_RegisterDefragCallbacks`
      Those callbacks are the same as `RM_RegisterDefragFunc` but promised to be called and the start
      and the end of the defrag process.
      d3d94ccf
    • Filipe Oliveira (Redis)'s avatar
      Created specific SMEMBERS command logic which avoids sinterGenericCommand, and... · 00a8e72c
      Filipe Oliveira (Redis) authored
      
      Created specific SMEMBERS command logic which avoids sinterGenericCommand, and minimizes processing and memory overhead (#13499)
      
      This PR introduces a dedicated implementation for the SMEMBERS command
      that avoids using the more generalized sinterGenericCommand function.
      By tailoring the logic specifically for SMEMBERS, we reduce unnecessary
      processing and memory overheads that were previously incurred by
      handling more complex cases like set intersections.
      
      ---------
      Co-authored-by: default avatardebing.sun <debing.sun@redis.com>
      00a8e72c
    • Filipe Oliveira (Redis)'s avatar
      Optimize the HELLO command reply (#13490) · a31b516e
      Filipe Oliveira (Redis) authored
      # Overall improvement
      
      TBD ( current is approximately 6% on the achievable ops/sec), coming
      from:
      
      - In case of no module we can skip 1.3% CPU cycles on dict Iterator
      creation/deletion
      - Use addReplyBulkCBuffer instead of addReplyBulkCString to avoid
      runtime strlen overhead within HELLO reply on string constants.
      
      ## Optimization 1: In case of no module we can skip 1.3% CPU cycles on
      dict Iterator creation/deletion.
      
      ## Optimization 2: Use addReplyBulkCBuffer instead of
      addReplyBulkCString to avoid runtime strlen overhead within HELLO reply
      on string constants.
      a31b516e
    • cyy-tag's avatar
      Fixed variable parameter formatting issues in serverPanic function (#13504) · c77b8f45
      cyy-tag authored
      
      
      Currently aeApiPoll panic does not record error code information. Added
      variable parameter formatting to _serverPanic to fix the issue
      
      ---------
      Co-authored-by: default avataryingyin.chen <15816602944@163.com>
      c77b8f45
    • Ozan Tezcan's avatar
      Reply LOADING on replica while flushing the db (#13495) · a7afd1d2
      Ozan Tezcan authored
      On a full sync, replica starts discarding existing db. If the existing 
      db is huge and flush is happening synchronously, replica may become 
      unresponsive. 
      
      Adding a change to yield back to event loop while flushing db on 
      a replica. Replica will reply -LOADING in this case. Note that while 
      replica is loading the new rdb, it may get an error and start flushing
      the partial db. This step may take a long time as well. Similarly, 
      replica will reply -LOADING in this case. 
      
      To call processEventsWhileBlocked() and reply -LOADING, we need to do:
      - Set connSetReadHandler() null not to process further data from the master
      - Set server.loading flag
      - Call blockingOperationStarts()
      
      rdbload() already does these steps and calls processEventsWhileBlocked()
      while loading the rdb. Added a new call rdbLoadWithEmptyFunc() which 
      accepts callback to flush db before loading rdb or when an error 
      happens while loading. 
      
      For diskless replication, doing something similar and calling emptyData()
      after setting required flags.
      
      Additional changes:
      - Allow `appendonly` config change during loading. 
       Config can be changed while loading data on startup or on replication 
       when slave is loading RDB. We allow config change command to update 
       `server.aof_enabled` and then lazily apply config change after loading
       operation is completed.
       
       - Added a test for `replica-lazy-flush` config
      a7afd1d2
  17. 29 Aug, 2024 1 commit
    • Oran Agra's avatar
      testsuite --dump-logs works on servers started before the test (#13500) · 3fcddfb6
      Oran Agra authored
      so far ./runtest --dump-logs used work for servers started within the
      test proc.
      now it'll also work on servers started outside the test proc scope.
      the downside is that these logs can be huge if they served many tests
      and not just the failing one.
      but for some rare failures, we rather have that than nothing.
      this feature isn't enabled y default, but is used by our GH actions.
      3fcddfb6
  18. 28 Aug, 2024 1 commit
  19. 26 Aug, 2024 1 commit
  20. 21 Aug, 2024 1 commit