1. 02 Oct, 2024 4 commits
  2. 29 Jul, 2024 1 commit
  3. 28 Jul, 2024 2 commits
  4. 25 Jul, 2024 2 commits
  5. 24 Jul, 2024 1 commit
  6. 22 Jul, 2024 2 commits
    • Oran Agra's avatar
      solve race conditions in tests (#13433) · 447ce11a
      Oran Agra authored
      [exception]: Executing test client: ERR FAILOVER target replica is not
      online.. ERR FAILOVER target replica is not online.
          while executing
      "$node_0 failover to $node_1_host $node_1_port"
          ("uplevel" body line 16)
          invoked from within
      "uplevel 1 $code"
          (procedure "test" line 58)
          invoked from within
      "test {failover command to specific replica works} {
      
      [err]: client evicted due to percentage of maxmemory in
      tests/unit/client-eviction.tcl
      Expected 33622 >= 220200 && 33622 < 440401 (context: type eval line 17
      cmd {assert {$tot_mem >= $n && $tot_mem < $maxmemory_clients_actual}}
      proc ::test)
      447ce11a
    • Oran Agra's avatar
      Different fix for the race in #13361 (#13434) · 13d227fa
      Oran Agra authored
      Recently in #13361, i attempted to fix a race between FLUSHALL and
      BGSAVE, where despite calling killRDBChild, the
      backgroundSaveDoneHandler will terminate with success.
      Turns out that even if the child didn't yet exit, there's a chance it'll
      still miss our signal and exit with success.
      in that case, we will still mess up the dirty counter (deducting
      dirty_before_bgsave) which is reset by FLUSHALL, and override the
      synchronous rdb file we saved.
      
      instead, we'll set a flag to treat the next done handler as a failed
      one.
      13d227fa
  7. 17 Jul, 2024 1 commit
    • Oran Agra's avatar
      Fix external test hang in redis-cli test when run in a certain order (#13423) · a3319785
      Oran Agra authored
      When the tests are run against an external server in this order:
      `--single unit/introspection --single unit/moduleapi/blockonbackground
      --single integration/redis-cli`
      the test would hang when the "ASK redirect test" test attempts to create
      a listening socket (it fails, and then redis-cli itself hangs waiting
      for a non-responsive socket created by the introspection test).
      
      the reasons are:
      1. the blockedbackground test includes util.tcl and resets the
      `::last_port_attempted` variable
      2. the test in introspection didn't close the listening server, so it's
      still alive.
      3. find_available_port doesn't properly detect the busy port, and it
      thinks that the port is free even though it's busy.
      
      fixing all 3 of these problems, even though fixing just one would be
      enough to let the test pass.
      a3319785
  8. 16 Jul, 2024 3 commits
    • Oran Agra's avatar
      Test infra adjustments for external CI runs (#13421) · fa46aa4d
      Oran Agra authored
      - when uploading server logs, make sure they don't overwrite each other.
      - sort the test units to get consistent order between them (following
      #13220)
      - backup and restore the entire server configuration, to protect one
      unit from config changes another unit performs
      fa46aa4d
    • debing.sun's avatar
      Trigger Lua GC after script loading (#13407) · 88af96c7
      debing.sun authored
      Nowdays we do not trigger LUA GC after loading lua script. This means
      that when a large number of scripts are loaded, such as when functions
      are propagating from the master to the replica, if the LUA scripts are
      never touched on the replica, the garbage might remain there
      indefinitely.
      
      Before this PR, we would share a gc_count between scripts and functions.
      This means that, under certain circumstances, the GC trigger for scripts
      and functions was not fair.
      For example, loading a large number of scripts followed by a small
      number of functions could result in the functions triggering GC.
      In this PR, we assign a unique `gc_count` to each of them, so the GC
      triggers between them will no longer affect each other.
      
      on the other hand, this PR will to bring regession for script loading
      commands(`FUNCTION LOAD` and `SCRIPT LOAD`), but they are not hot path,
      we can ignore it, and it will be replaced
      https://github.com/redis/redis/pull/13375
      
       in the future.
      
      ---------
      Co-authored-by: default avatarOran Agra <oran@redislabs.com>
      88af96c7
    • debing.sun's avatar
      Prevent deleting RDB read event after restarting RDB saving for other diskless replicas (#13410) · 76415fa2
      debing.sun authored
      When we terminate the diskless RDB saving child process and, at the same
      time, we start a new BGSAVE for new replicas, we should not delete the
      RDB read event. Otherwise, these replicas will never receive a response.
      this is a result of the recent change in
      https://github.com/redis/redis/pull/13361
      
      
      
      ---------
      Co-authored-by: default avataroranagra <oran@redislabs.com>
      76415fa2
  9. 15 Jul, 2024 2 commits
  10. 14 Jul, 2024 1 commit
    • guybe7's avatar
      Crash report: Use more chars for argv (#13413) · b10e19e3
      guybe7 authored
      128 is not enough chars when we're talking about commands like RESTORE.
      Of course, it's impossible to find the perfect number, but 1024 is
      better than 128, and it's not obscenely large.
      b10e19e3
  11. 12 Jul, 2024 1 commit
    • debing.sun's avatar
      Avoid starting defrag after config resetstat for defrag test (#13399) · d39548c8
      debing.sun authored
      
      
      If `config resetstat` is executed and a defrag is started after it, the
      `total_active_defrag_time` will not be 0.
      When we start the defrag again, we will skip the following steps:
      1. waiting for the defrag to start. (s total_active_defrag_time is equal
      0)
      2. waiting for the test to complete. (active_defrag_running is euqal 0)
      which result in the test failed.
      
      ---------
      Co-authored-by: default avataroranagra <oran@redislabs.com>
      d39548c8
  12. 11 Jul, 2024 3 commits
  13. 10 Jul, 2024 1 commit
    • debing.sun's avatar
      Rebuild function engines for function flush command (#13383) · ffff7fea
      debing.sun authored
      
      
      ### Issue
      The current implementation of `FUNCTION FLUSH` command uses
      `lua_unref()` to unreference script closures in Lua vm. However,
      invoking `lua_unref()` during lazy free (`ASYNC` argument) is risky
      since it is not thread-safe.
      
      Another issue is that using `lua_unref()` to unreference references does
      not trigger GC, This can result in the Lua VM leaves a significant
      amount of garbage, which may never be cleaned up if not properly GC.
      
      ### Solution
      The proposed solution is to completely rebuild the engines, resulting in
      a brand new Lua VM.
      
      ---------
      Co-authored-by: default avatarmeir <meir@redis.com>
      ffff7fea
  14. 09 Jul, 2024 1 commit
    • debing.sun's avatar
      Hide user data from log (#13400) · 69b480cb
      debing.sun authored
      
      
      This PR is based on the commits from PR #11747.
      
      In the event of an assertion failure, hide command arguments from the
      operator.
      
      In some cases, private client information can be voluntarily exposed
      when a redis instance crashes due to an assertion failure.
      This commit prevent וnintentional client info exposure.
      Operators can still access the hidden data, but they must actively
      request it.
      Any of the client info commands remains the unchanged.
      
      ### Config
      Add a new config `hide-user-data-from-log` to turn this feature on and
      off, default off.
      
      ---------
      Co-authored-by: default avatarnaglera <anagler123@gmail.com>
      Co-authored-by: default avatarnaglera <58042354+naglera@users.noreply.github.com>
      69b480cb
  15. 08 Jul, 2024 1 commit
  16. 07 Jul, 2024 1 commit
    • Moti Cohen's avatar
      HFE - RDB serialize also hash min expiration (For ROF flow) (#13391) · dcf02298
      Moti Cohen authored
      * Following this feature, Redis (ROF) may implement flow that allows hashes to be
        dumped directly from RDB to FLUSH without parsing. In this scenario, it is still
        essential to determine when to update hashes due to expired fields. By writing
        and reading the next minimum hash-field expiration before serializing objects
        to and from RDB, we can effectively track and expire hash fields without the need
        to parse the hash during loading.
      
          Before:
          #define RDB_TYPE_HASH_METADATA 22
          #define RDB_TYPE_HASH_LISTPACK_EX 23
          
          After:
          /* Hash with HFEs. Doesn't attach min TTL at start */
          #define RDB_TYPE_HASH_METADATA_PRE_GA 22      
          /* Hash LP with HFEs. Doesn't attach min TTL at start */
          #define RDB_TYPE_HASH_LISTPACK_EX_PRE_GA 23   
          /* Hash with HFEs. Attach min TTL at start */
          #define RDB_TYPE_HASH_METADATA 24             
          /* Hash LP with HFEs. Attach min TTL at start */
          #define RDB_TYPE_HASH_LISTPACK_EX 25          
      
      
      * Manually test loading RDB file before the change and verify hash and its HFEs 
        is as expected.
      * Added `subexpires` counter to `redis-check-rdb`
      dcf02298
  17. 04 Jul, 2024 1 commit
  18. 03 Jul, 2024 1 commit
    • Filipe Oliveira (Redis)'s avatar
      Reduce getNodeByQuery overhead (#13221) · 26a2dcb9
      Filipe Oliveira (Redis) authored
      
      
      The following PR does the following changes based upon on CPU profile
      info. The `getNodeByQuery` function represents 8.2% of an overhead of
      12.3% when comparing single shard cluster with standalone.
      Proposed changes:
      - inlinging keyHashSlot to reduce overhead of that function call
      - Reduce duplicate calls to getCommandFlags within getNodeByQuery
      
      The above changes represent an improvement of approximately 5% on the achievable ops/sec.
      Co-authored-by: default avatarfilipecosta90 <filipecosta.90@gmail.com>
      26a2dcb9
  19. 02 Jul, 2024 3 commits
  20. 01 Jul, 2024 2 commits
    • Oran Agra's avatar
      Solve a race between BGSAVE and FLUSHALL messing up the dirty counter (#13361) · 799c5e5f
      Oran Agra authored
      If we run FLUSHALL when the 'save' config is set, and there's a fork
      child ding BGSAVE, there's a chance the child is already finished, and
      the parent process is unaware of it. in that case the child will not get
      the kill signal and will finish successfully, but the parent process
      thinks it killed it and will reset the dirty counter to 0, then the
      backgroundSaveDoneHandlerDisk method can set the dirty counter to a
      negative value.
      799c5e5f
    • Oran Agra's avatar
      Fix possible crash due to OOM panic on invalid command (#13380) · 69b7137d
      Oran Agra authored
      getKeysUsingKeySpece had the range check AFTER the allocation, of the
      keys buffer, which could lead to an OOM panic when invalid arguments are
      provided, leading to an overflow.
      The allocated memory is only used after the range check, so there's no
      risk of buffer overrun.
      The OOM panic can happen on 32bit builds, or 64 builds running on
      systems with less than 4GB of RAM, and is reachable via the COMMAND
      GETKEYSANDFLAGS, and ACL key name validation.
      69b7137d
  21. 27 Jun, 2024 3 commits
    • YaacovHazan's avatar
      Redis 7.4 RC2 (#13371) · 0637b4ea
      YaacovHazan authored
      Upgrade urgency LOW: This is the second Release Candidate for Redis 7.4.
      
      Performance and resource utilization improvements
      =================================================
      * #13296 Optimize CPU cache efficiency
      
      Changes to new 7.4 new features (compared to 7.4 RC1)
      =====================================================
      * #13343 Hash - expiration of individual fields: when key does not exist
      - reply with an array (nonexisting code for each field)
      * #13329 Hash - expiration of individual fields: new keyspace event:
      `hexpired`
      
      Modules API - Potentially breaking changes to new 7.4 features (compared
      to 7.4 RC1)
      
      ====================================================================================
      * #13326 Hash - expiration of individual fields: avoid lazy expire when
      called from a Modules API function
      0637b4ea
    • YaacovHazan's avatar
      Redis 7.4 RC2 · 4000bb2e
      YaacovHazan authored
      4000bb2e
    • YaacovHazan's avatar
      4e9acd4b
  22. 26 Jun, 2024 2 commits
    • Moti Cohen's avatar
      HFE - count in command must match actual number of fields (#13369) · a9267137
      Moti Cohen authored
      There was wrong preliminary assumption that we can optionally provide
      vector of arguments more than count.
      This is error-prone approach that leaded to actual error in that case.
      This PR enforce that vector of argument match count.
      
      Also fixed flaky HRANDFIELD test.
      a9267137
    • debing.sun's avatar
      Don't keep global replication buffer reference for replicas marked CLIENT_CLOSE_ASAP (#13363) · 52e12d8b
      debing.sun authored
      
      
      In certain situations, we might generate a large number of propagates
      (e.g., multi/exec, Lua script, or a single command generating tons of
      propagations) within an event loop.
      During the process of propagating to a replica, if the replica is
      disconnected(marked as CLIENT_CLOSE_ASAP) due to exceeding the output
      buffer limit, we should remove its reference to the global replication
      buffer to avoid the global replication buffer being unable to be
      properly trimmed due to being referenced.
      
      ---------
      Co-authored-by: default avataroranagra <oran@redislabs.com>
      52e12d8b
  23. 25 Jun, 2024 1 commit
    • Moti Cohen's avatar
      Fix H(P)EXPIREAT command to propagate HDEL as well (#13364) · 5eac99c3
      Moti Cohen authored
      H(P)EXPIREAT command might delete fields in case the absolute time is in the 
      past. Those HDELs need to be propagated as well.
       
      In general, as we need to propagate H(P)EXPIRE(AT) command to the replica, each 
      field that is mentioned in the command should be categorized into one of the four
      options:
      1. Managed to update field’s expiration time - propagate it to replica as part 
         of the HPEXPIREAT command.
      2. Deleted the field because the time is in the past - propagate also HDEL command
         to delete the field and remove the field from the propagated HPEXPIREAT.
      3. Condition not met for the field - Remove the field from the propagated
         HPEXPIREAT command.
      4. Field does not exists - Remove the field from the propagated HPEXPIREAT command.
      
      If none of the provided fields match option number 1, then avoid also propagating 
      the HPEXPIREAT command to the replica.
      
      This approach is aligned with the EXPIRE command:
      If a given key has already expired, then DEL will be propagated instead of
      EXPIRE command. If condition not met, then command will be rejected. Otherwise, 
      EXPIRE command will be propagated for given key.
      5eac99c3